Added GCS kms key variable.

Author: ilakhtenkov

CHANGELOG.md

@@ -7,7 +7,8 @@ The project is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 
 ### Added 
 
-- Option `create_service_account` to be able to use custom Compute Engine service account. And manage some permissions for it outside of the module. 
+- Option `create_service_account` to be able to use custom Compute Engine service account. And manage some permissions for it outside of the module.
+- Options `pubsub_kms_key_name` and `gcs_kms_key_name` to be able to add encryption of created resources with customer managed keys.
 
 <a name="v1.0.0"></a>
 ## [v1.0.0] - 2022-12-20

README.md

@@ -36,6 +36,7 @@ These deployment templates are provided as is, without warranty. See [Copyright
 | <a name="input_dataflow_template_version"></a> [dataflow_template_version](#input_dataflow_template_version) | (Optional) Dataflow template release version (default 'latest'). Override this for version pinning e.g. '2021-08-02-00_RC00'. Must specify version only since template GCS path will be deduced automatically: 'gs://dataflow-templates/`version`/Cloud_PubSub_to_Splunk' | `string` |
 | <a name="input_dataflow_worker_service_account"></a> [dataflow_worker_service_account](#input_dataflow_worker_service_account) | (Optional) Name of worker service account to be created and used to execute job operations. Must be 6-30 characters long, and match the regular expression [a-z]([-a-z0-9]*[a-z0-9]). If parameter is empty, worker service account defaults to project's Compute Engine default service account. | `string` |
 | <a name="input_deploy_replay_job"></a> [deploy_replay_job](#input_deploy_replay_job) | (Optional) Defines if replay pipeline should be deployed or not (default: `false`) | `bool` |
+| <a name="input_gcs_kms_key_name"></a> [gcs_kms_key_name](#input_gcs_kms_key_name) | (Optional) The `id` of a Cloud KMS key that will be used to encrypt objects inserted into temporary bucket. User is responsible for permissions to this key for Cloud Storage Service Account. | `string` |
 | <a name="input_primary_subnet_cidr"></a> [primary_subnet_cidr](#input_primary_subnet_cidr) | The CIDR Range of the primary subnet | `string` |
 | <a name="input_pubsub_kms_key_name"></a> [pubsub_kms_key_name](#input_pubsub_kms_key_name) | (Optional) The resource name of the Cloud KMS CryptoKey to be used to protect access to messages published on created topics. Your project's PubSub service account (`service-{{PROJECT_NUMBER}}@gcp-sa-pubsub.iam.gserviceaccount.com`) must have `roles/cloudkms.cryptoKeyEncrypterDecrypter` to use this feature. The expected format is `projects/*/locations/*/keyRings/*/cryptoKeys/*`. | `string` |
 | <a name="input_scoping_project"></a> [scoping_project](#input_scoping_project) | Cloud Monitoring scoping project ID to create dashboard under.<br>This assumes a pre-existing scoping project whose metrics scope contains the `project` where dataflow job is to be deployed.<br>See [Cloud Monitoring settings](https://cloud.google.com/monitoring/settings) for more details on scoping project.<br>If parameter is empty, scoping project defaults to value of `project` parameter above. | `string` |

pipeline.tf

@@ -35,6 +35,12 @@ resource "google_storage_bucket" "dataflow_job_temp_bucket" {
   name          = local.dataflow_temporary_gcs_bucket_name
   location      = var.region
   storage_class = "REGIONAL"
+  dynamic "encryption" {
+    for_each = (var.gcs_kms_key_name == "") ? [] : [1]
+    content {
+      default_kms_key_name = var.gcs_kms_key_name
+    }
+  }
 }
 
 resource "google_storage_bucket_object" "dataflow_job_temp_object" {

sample.tfvars

@@ -47,3 +47,7 @@ scoping_project = "[MY_PROJECT]"
 
 # Replay job settings
 deploy_replay_job = false
+
+# Security parameters
+pubsub_kms_key_name = "projects/[MY_PROJECT]/locations/[MY_REGION]/keyRings/[MY_KEYRING_NAME]/cryptoKeys/[MY_KEY_NAME]"
+gcs_kms_key_name    = "projects/[MY_PROJECT]/locations/[MY_REGION]/keyRings/[MY_KEYRING_NAME]/cryptoKeys/[MY_KEY_NAME]"

variables.tf

@@ -196,11 +196,21 @@ variable "create_service_account" {
 }
 
 variable "pubsub_kms_key_name" {
-  type = string
+  type        = string
   description = "(Optional) The resource name of the Cloud KMS CryptoKey to be used to protect access to messages published on created topics. Your project's PubSub service account (`service-{{PROJECT_NUMBER}}@gcp-sa-pubsub.iam.gserviceaccount.com`) must have `roles/cloudkms.cryptoKeyEncrypterDecrypter` to use this feature. The expected format is `projects/*/locations/*/keyRings/*/cryptoKeys/*`."
-  default = ""
+  default     = ""
   validation {
     condition     = can(regex("^projects\\/[^\\n\\r\\/]+\\/locations\\/[^\\n\\r\\/]+\\/keyRings\\/[^\\n\\r\\/]+\\/cryptoKeys\\/[^\\n\\r\\/]+$", var.pubsub_kms_key_name)) || var.pubsub_kms_key_name == ""
     error_message = "Pub/Sub KMS key name must match: '^projects\\/[^\\n\\r\\/]+\\/locations\\/[^\\n\\r\\/]+\\/keyRings\\/[^\\n\\r\\/]+\\/cryptoKeys\\/[^\\n\\r\\/]+$' pattern."
   }
 }
+
+variable "gcs_kms_key_name" {
+  type        = string
+  description = "(Optional) The `id` of a Cloud KMS key that will be used to encrypt objects inserted into temporary bucket. User is responsible for permissions to this key for Cloud Storage Service Account."
+  default     = ""
+  validation {
+    condition     = can(regex("^projects\\/[^\\n\\r\\/]+\\/locations\\/[^\\n\\r\\/]+\\/keyRings\\/[^\\n\\r\\/]+\\/cryptoKeys\\/[^\\n\\r\\/]+$", var.gcs_kms_key_name)) || var.gcs_kms_key_name == ""
+    error_message = "Cloud Storage KMS key name must match: '^projects\\/[^\\n\\r\\/]+\\/locations\\/[^\\n\\r\\/]+\\/keyRings\\/[^\\n\\r\\/]+\\/cryptoKeys\\/[^\\n\\r\\/]+$' pattern."
+  }
+}