Skip to content

Commit bb5fcdb

Browse files
marvinbussmarvinbuss
authored
Merge pull request #115 from PerfectThymeTech/marvinbuss/enhancements
Enhance Management Zone - Secure-by-default
2 parents cf68500 + f4e0918 commit bb5fcdb

38 files changed

+1098
-239
lines changed

‎.github/workflows/terraform.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ jobs:
1818
name: "Terraform Deploy"
1919
with:
2020
environment: "dev"
21-
terraform_version: "1.9.8"
21+
terraform_version: "1.10.2"
2222
node_version: 20
2323
working_directory: "./tests/e2e"
2424
tenant_id: "37963dd4-f4e6-40f8-a7d6-24b97919e452"
@@ -33,7 +33,7 @@ jobs:
3333
if: github.event_name == 'push' || github.event_name == 'release'
3434
with:
3535
environment: "dev"
36-
terraform_version: "1.9.8"
36+
terraform_version: "1.10.2"
3737
node_version: 20
3838
working_directory: "./tests/e2e"
3939
tenant_id: "37963dd4-f4e6-40f8-a7d6-24b97919e452"

‎README.md

Lines changed: 15 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -125,61 +125,23 @@ The following requirements are needed by this module:
125125

126126
- <a name="requirement_databricks"></a> [databricks](#requirement\_databricks) (~> 1.58)
127127

128-
- <a name="requirement_external"></a> [external](#requirement\_external) (~> 2.3)
129-
130128
- <a name="requirement_time"></a> [time](#requirement\_time) (~> 0.9)
131129

132130
## Modules
133131

134132
The following Modules are called:
135133

136-
### <a name="module_container_registry"></a> [container\_registry](#module\_container\_registry)
137-
138-
Source: github.com/PerfectThymeTech/terraform-azurerm-modules//modules/containerregistry
139-
140-
Version: main
141-
142-
### <a name="module_databricks_access_connector"></a> [databricks\_access\_connector](#module\_databricks\_access\_connector)
143-
144-
Source: github.com/PerfectThymeTech/terraform-azurerm-modules//modules/databricksaccessconnector
145-
146-
Version: main
147-
148-
### <a name="module_databricks_workspace"></a> [databricks\_workspace](#module\_databricks\_workspace)
149-
150-
Source: github.com/PerfectThymeTech/terraform-azurerm-modules//modules/databricksworkspace
151-
152-
Version: main
153-
154-
### <a name="module_key_vault_purview"></a> [key\_vault\_purview](#module\_key\_vault\_purview)
134+
### <a name="module_datamanagement"></a> [datamanagement](#module\_datamanagement)
155135

156-
Source: github.com/PerfectThymeTech/terraform-azurerm-modules//modules/keyvault
136+
Source: ./modules/datamanagement
157137

158-
Version: main
138+
Version:
159139

160-
### <a name="module_key_vault_scim"></a> [key\_vault\_scim](#module\_key\_vault\_scim)
140+
### <a name="module_platform"></a> [platform](#module\_platform)
161141

162-
Source: github.com/PerfectThymeTech/terraform-azurerm-modules//modules/keyvault
142+
Source: ./modules/platform
163143

164-
Version: main
165-
166-
### <a name="module_purview_account"></a> [purview\_account](#module\_purview\_account)
167-
168-
Source: github.com/PerfectThymeTech/terraform-azurerm-modules//modules/purview
169-
170-
Version: main
171-
172-
### <a name="module_synapse_private_link_hub"></a> [synapse\_private\_link\_hub](#module\_synapse\_private\_link\_hub)
173-
174-
Source: github.com/PerfectThymeTech/terraform-azurerm-modules//modules/synapseprivetlinkhub
175-
176-
Version: main
177-
178-
### <a name="module_user_assigned_identity"></a> [user\_assigned\_identity](#module\_user\_assigned\_identity)
179-
180-
Source: github.com/PerfectThymeTech/terraform-azurerm-modules//modules/userassignedidentity
181-
182-
Version: main
144+
Version:
183145

184146
<!-- markdownlint-disable MD013 -->
185147
<!-- markdownlint-disable MD034 -->
@@ -306,6 +268,14 @@ Type: `string`
306268

307269
Default: `""`
308270

271+
### <a name="input_private_dns_zone_id_dfs"></a> [private\_dns\_zone\_id\_dfs](#input\_private\_dns\_zone\_id\_dfs)
272+
273+
Description: Specifies the resource ID of the private DNS zone for Azure Storage dfs endpoints. Not required if DNS A-records get created via Azure Policy.
274+
275+
Type: `string`
276+
277+
Default: `""`
278+
309279
### <a name="input_private_dns_zone_id_purview_platform"></a> [private\_dns\_zone\_id\_purview\_platform](#input\_private\_dns\_zone\_id\_purview\_platform)
310280

311281
Description: Specifies the resource ID of the private DNS zone for Azure Key Vault. Not required if DNS A-records get created via Azure Policy.
@@ -394,35 +364,7 @@ Default: `true`
394364

395365
## Outputs
396366

397-
The following outputs are exported:
398-
399-
### <a name="output_container_registry_id"></a> [container\_registry\_id](#output\_container\_registry\_id)
400-
401-
Description: Specifies the id of the container registry.
402-
403-
### <a name="output_databricks_access_connector_id"></a> [databricks\_access\_connector\_id](#output\_databricks\_access\_connector\_id)
404-
405-
Description: Specifies the ids of the databricks access connectors.
406-
407-
### <a name="output_databricks_workspace_ids"></a> [databricks\_workspace\_ids](#output\_databricks\_workspace\_ids)
408-
409-
Description: Specifies the ids of the databricks workspaces.
410-
411-
### <a name="output_key_vault_purview_id"></a> [key\_vault\_purview\_id](#output\_key\_vault\_purview\_id)
412-
413-
Description: Specifies the id of the Azure key vault provisioned for Microsoft Purview.
414-
415-
### <a name="output_key_vault_scim_id"></a> [key\_vault\_scim\_id](#output\_key\_vault\_scim\_id)
416-
417-
Description: Specifies the id of the Azure key vault provisioned for SCIM.
418-
419-
### <a name="output_purview_id"></a> [purview\_id](#output\_purview\_id)
420-
421-
Description: Specifies the id of the Microsoft Purview account.
422-
423-
### <a name="output_synapse_private_link_hub_id"></a> [synapse\_private\_link\_hub\_id](#output\_synapse\_private\_link\_hub\_id)
424-
425-
Description: Specifies the id of the Azure synapse private link hub.
367+
No outputs.
426368

427369
<!-- markdownlint-enable -->
428370
## License

‎locals.tf

Lines changed: 5 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,18 @@
11
locals {
2+
# General locals
23
prefix = "${lower(var.prefix)}-${var.environment}"
34

4-
# General locals
5+
# Diagnostics locals
56
diagnostics_configurations = var.log_analytics_workspace_id == null ? [] : [
67
{
78
log_analytics_workspace_id = var.log_analytics_workspace_id
89
storage_account_id = ""
910
}
1011
]
12+
13+
# Customer managed key locals
1114
customer_managed_key = null
1215

1316
# Network locals
14-
virtual_network = {
15-
resource_group_name = split("/", var.vnet_id)[4]
16-
name = split("/", var.vnet_id)[8]
17-
}
18-
network_security_group = {
19-
resource_group_name = try(split("/", var.nsg_id)[4], "")
20-
name = try(split("/", var.nsg_id)[8], "")
21-
}
22-
route_table = {
23-
resource_group_name = try(split("/", var.route_table_id)[4], "")
24-
name = try(split("/", var.route_table_id)[8], "")
25-
}
26-
subnet_cidr_ranges = {
27-
private_endpoint_subnet = var.subnet_cidr_ranges.private_endpoint_subnet != "" ? var.subnet_cidr_ranges.private_endpoint_subnet : tostring(cidrsubnet(data.azurerm_virtual_network.virtual_network.address_space[0], 27 - tonumber(reverse(split("/", data.azurerm_virtual_network.virtual_network.address_space[0]))[0]), 0))
28-
}
29-
connectivity_delay_in_seconds = 30
30-
databricks_private_subnet_name = "DatabricksPrivateSubnet"
31-
databricks_public_subnet_name = "DatabricksPublicSubnet"
32-
33-
# Databricks locals
34-
databricks_account_scim_secret_name = "scim-token"
17+
connectivity_delay_in_seconds = 30
3518
}

‎main.tf

Lines changed: 61 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,70 @@
1-
data "azurerm_client_config" "current" {}
1+
module "platform" {
2+
source = "./modules/platform"
23

3-
resource "azurerm_resource_group" "governance_rg" {
4-
name = "${local.prefix}-governance-rg"
5-
location = var.location
6-
tags = var.tags
7-
}
4+
providers = {
5+
azurerm = azurerm
6+
azapi = azapi
7+
}
88

9-
resource "azurerm_resource_group" "connectivity_adb_rg" {
10-
for_each = toset(var.locations_databricks)
9+
# General variables
10+
location = var.location
11+
locations_databricks = var.locations_databricks
12+
environment = var.environment
13+
prefix = var.prefix
14+
tags = var.tags
1115

12-
name = "${local.prefix}-connectivity-adb-${each.value}-rg"
13-
location = var.location
14-
tags = var.tags
16+
# Service variables
17+
vnet_id = var.vnet_id
18+
nsg_id = var.nsg_id
19+
route_table_id = var.route_table_id
20+
subnet_cidr_range_private_endpoints = var.subnet_cidr_ranges.private_endpoint_subnet
1521
}
1622

17-
resource "azurerm_resource_group" "container_rg" {
18-
name = "${local.prefix}-container-rg"
19-
location = var.location
20-
tags = var.tags
21-
}
23+
module "datamanagement" {
24+
source = "./modules/datamanagement"
2225

23-
resource "azurerm_resource_group" "connectivity_synapse_rg" {
24-
name = "${local.prefix}-connectivity-syn-rg"
25-
location = var.location
26-
tags = var.tags
27-
}
26+
providers = {
27+
azurerm = azurerm
28+
azapi = azapi
29+
time = time
30+
}
2831

29-
resource "azurerm_resource_group" "automation_rg" {
30-
name = "${local.prefix}-automation-rg"
31-
location = var.location
32-
tags = var.tags
33-
}
32+
# General variables
33+
company_name = var.company_name
34+
location = var.location
35+
location_purview = var.location_purview
36+
locations_databricks = var.locations_databricks
37+
environment = var.environment
38+
prefix = var.prefix
39+
tags = var.tags
40+
41+
# Service variables
42+
purview_enabled = var.purview_enabled
43+
purview_account_root_collection_admins = var.purview_account_root_collection_admins
44+
databricks_account_id = var.databricks_account_id
45+
46+
# HA/DR variables
47+
zone_redundancy_enabled = var.zone_redundancy_enabled
48+
49+
# Logging and monitoring variables
50+
diagnostics_configurations = local.diagnostics_configurations
51+
52+
# Network variables
53+
subnet_id_private_endpoints = module.platform.subnet_id_private_endpoints
54+
subnet_ids_databricks = module.platform.subnet_ids_databricks
55+
connectivity_delay_in_seconds = local.connectivity_delay_in_seconds
56+
57+
# DNS variables
58+
private_dns_zone_id_purview_platform = var.private_dns_zone_id_purview_platform
59+
private_dns_zone_id_blob = var.private_dns_zone_id_blob
60+
private_dns_zone_id_dfs = var.private_dns_zone_id_dfs
61+
private_dns_zone_id_queue = var.private_dns_zone_id_queue
62+
private_dns_zone_id_databricks = var.private_dns_zone_id_databricks
63+
private_dns_zone_id_container_registry = var.private_dns_zone_id_container_registry
64+
private_dns_zone_id_vault = var.private_dns_zone_id_vault
65+
private_dns_zone_id_synapse_portal = var.private_dns_zone_id_synapse_portal
66+
databricks_private_dns_zone_ids = module.platform.private_dns_zone_ids
3467

35-
resource "azurerm_resource_group" "scim_rg" {
36-
name = "${local.prefix}-scim-rg"
37-
location = var.location
38-
tags = var.tags
68+
# Customer-managed key variables
69+
customer_managed_key = var.customer_managed_key
3970
}

‎containerregistry.tf renamed to ‎modules/datamanagement/containerregistry.tf

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -17,9 +17,9 @@ module "container_registry" {
1717
container_registry_retention_policy_in_days = 7
1818
container_registry_trust_policy_enabled = false
1919
container_registry_zone_redundancy_enabled = var.zone_redundancy_enabled
20-
diagnostics_configurations = local.diagnostics_configurations
21-
subnet_id = azapi_resource.private_endpoint_subnet.id
22-
connectivity_delay_in_seconds = local.connectivity_delay_in_seconds
20+
diagnostics_configurations = var.diagnostics_configurations
21+
subnet_id = var.subnet_id_private_endpoints
22+
connectivity_delay_in_seconds = var.connectivity_delay_in_seconds
2323
private_dns_zone_id_container_registry = var.private_dns_zone_id_container_registry
24-
customer_managed_key = null
24+
customer_managed_key = var.customer_managed_key
2525
}

‎modules/datamanagement/data.tf

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
data "azurerm_client_config" "current" {}
2+
3+
data "azurerm_key_vault_secret" "key_vault_secret_databricks_account_scim_token" {
4+
key_vault_id = module.key_vault_scim.key_vault_id
5+
name = local.databricks_account_scim_secret_name
6+
7+
depends_on = [
8+
null_resource.databricks_account_scim_token,
9+
]
10+
}

‎databricksaccessconnector.tf renamed to ‎modules/datamanagement/databricksaccessconnector.tf

File renamed without changes.

‎databricksappscim.tf renamed to ‎modules/datamanagement/databricksappscim.tf

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
resource "time_rotating" "databricks_account_scim_token_regenerate" {
2-
rotation_months = 1
2+
rotation_months = 12
33
}
44

55
resource "null_resource" "databricks_account_scim_token" {
@@ -8,7 +8,7 @@ resource "null_resource" "databricks_account_scim_token" {
88
}
99

1010
provisioner "local-exec" {
11-
working_dir = "${path.module}/scripts/"
11+
working_dir = "${path.module}/../../scripts/"
1212
command = "pwsh ./Get-AccountScimToken.ps1 -DatabricksAccountId ${var.databricks_account_id} -KeyVaultName ${module.key_vault_scim.key_vault_name} -KeyVaultSecretName ${local.databricks_account_scim_secret_name}"
1313
environment = {}
1414
}

‎modules/datamanagement/databricksprivateendpoints.tf

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
resource "azurerm_private_endpoint" "private_endpoint_databricks_workspace_databricks_ui_api" {
2+
for_each = toset(var.locations_databricks)
3+
4+
name = "${module.databricks_workspace[each.key].databricks_workspace_name}-ddctd-uiapi-pe"
5+
location = each.key
6+
resource_group_name = azurerm_resource_group.connectivity_adb_rg[each.key].name
7+
tags = var.tags
8+
9+
custom_network_interface_name = "${module.databricks_workspace[each.key].databricks_workspace_name}-ddctd-uiapi-nic"
10+
private_service_connection {
11+
name = "${module.databricks_workspace[each.key].databricks_workspace_name}-uiapi-svc"
12+
is_manual_connection = false
13+
private_connection_resource_id = module.databricks_workspace[each.key].databricks_workspace_id
14+
subresource_names = ["databricks_ui_api"]
15+
}
16+
private_dns_zone_group {
17+
name = "${module.databricks_workspace[each.key].databricks_workspace_name}-arecord"
18+
private_dns_zone_ids = [
19+
var.databricks_private_dns_zone_ids["databricks"].id
20+
]
21+
}
22+
subnet_id = var.subnet_ids_databricks[each.key].subnet_databricks_private_endpoint_id
23+
}
24+
25+
resource "azurerm_private_endpoint" "private_endpoint_databricks_workspace_dbfs_blob" {
26+
for_each = toset(var.locations_databricks)
27+
28+
name = "${module.databricks_workspace[each.key].databricks_workspace_name}-ddctd-blob-pe"
29+
location = each.key
30+
resource_group_name = azurerm_resource_group.connectivity_adb_rg[each.key].name
31+
tags = var.tags
32+
33+
custom_network_interface_name = "${module.databricks_workspace[each.key].databricks_workspace_name}-ddctd-blob-nic"
34+
private_service_connection {
35+
name = "${module.databricks_workspace[each.key].databricks_workspace_name}-blob-svc"
36+
is_manual_connection = false
37+
private_connection_resource_id = "${module.databricks_workspace[each.key].databricks_workspace_managed_resource_group_id}/providers/Microsoft.Storage/storageAccounts/${module.databricks_workspace[each.key].databricks_workspace_managed_storage_account_name}"
38+
subresource_names = ["blob"]
39+
}
40+
private_dns_zone_group {
41+
name = "${module.databricks_workspace[each.key].databricks_workspace_name}-arecord"
42+
private_dns_zone_ids = [
43+
var.databricks_private_dns_zone_ids["blob"].id
44+
]
45+
}
46+
subnet_id = var.subnet_ids_databricks[each.key].subnet_databricks_private_endpoint_id
47+
}
48+
49+
resource "azurerm_private_endpoint" "private_endpoint_databricks_workspace_dbfs_dfs" {
50+
for_each = toset(var.locations_databricks)
51+
52+
name = "${module.databricks_workspace[each.key].databricks_workspace_name}-ddctd-dfs-pe"
53+
location = each.key
54+
resource_group_name = azurerm_resource_group.connectivity_adb_rg[each.key].name
55+
tags = var.tags
56+
57+
custom_network_interface_name = "${module.databricks_workspace[each.key].databricks_workspace_name}-ddctd-dfs-nic"
58+
private_service_connection {
59+
name = "${module.databricks_workspace[each.key].databricks_workspace_name}-dfs-svc"
60+
is_manual_connection = false
61+
private_connection_resource_id = "${module.databricks_workspace[each.key].databricks_workspace_managed_resource_group_id}/providers/Microsoft.Storage/storageAccounts/${module.databricks_workspace[each.key].databricks_workspace_managed_storage_account_name}"
62+
subresource_names = ["dfs"]
63+
}
64+
private_dns_zone_group {
65+
name = "${module.databricks_workspace[each.key].databricks_workspace_name}-arecord"
66+
private_dns_zone_ids = [
67+
var.databricks_private_dns_zone_ids["dfs"].id
68+
]
69+
}
70+
subnet_id = var.subnet_ids_databricks[each.key].subnet_databricks_private_endpoint_id
71+
}

0 commit comments

Comments
 (0)