add KMS keys interpolation to project factory (#3145)

This commit is contained in:
Ludovico Magnocavallo
2025-06-10 10:24:25 +02:00
committed by GitHub
parent 12b206a72f
commit 3c38bc643e
8 changed files with 47 additions and 20 deletions

View File

@@ -354,18 +354,19 @@ The approach is not shown here but reasonably easy to implement. The main projec
|---|---|:---:|:---:|:---:|:---:|
| [automation](variables-fast.tf#L17) | Automation resources created by the bootstrap stage. | <code title="object&#40;&#123;&#10; outputs_bucket &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>0-bootstrap</code> |
| [billing_account](variables-fast.tf#L26) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | <code title="object&#40;&#123;&#10; id &#61; string&#10; is_org_level &#61; optional&#40;bool, true&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | | <code>0-bootstrap</code> |
| [prefix](variables-fast.tf#L93) | Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants. | <code>string</code> | ✓ | | <code>0-bootstrap</code> |
| [factories_config](variables.tf#L17) | Configuration for YAML-based factories. | <code title="object&#40;&#123;&#10; folders_data_path &#61; optional&#40;string, &#34;data&#47;hierarchy&#34;&#41;&#10; projects_data_path &#61; optional&#40;string, &#34;data&#47;projects&#34;&#41;&#10; budgets &#61; optional&#40;object&#40;&#123;&#10; billing_account &#61; string&#10; budgets_data_path &#61; optional&#40;string, &#34;data&#47;budgets&#34;&#41;&#10; notification_channels &#61; optional&#40;map&#40;any&#41;, &#123;&#125;&#41;&#10; &#125;&#41;&#41;&#10; context &#61; optional&#40;object&#40;&#123;&#10; folder_ids &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; iam_principals &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; tag_values &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; vpc_host_projects &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | |
| [prefix](variables-fast.tf#L101) | Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants. | <code>string</code> | ✓ | | <code>0-bootstrap</code> |
| [factories_config](variables.tf#L17) | Configuration for YAML-based factories. | <code title="object&#40;&#123;&#10; folders_data_path &#61; optional&#40;string, &#34;data&#47;hierarchy&#34;&#41;&#10; projects_data_path &#61; optional&#40;string, &#34;data&#47;projects&#34;&#41;&#10; budgets &#61; optional&#40;object&#40;&#123;&#10; billing_account &#61; string&#10; budgets_data_path &#61; optional&#40;string, &#34;data&#47;budgets&#34;&#41;&#10; notification_channels &#61; optional&#40;map&#40;any&#41;, &#123;&#125;&#41;&#10; &#125;&#41;&#41;&#10; context &#61; optional&#40;object&#40;&#123;&#10; folder_ids &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; kms_keys &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; iam_principals &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; tag_values &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; vpc_host_projects &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | |
| [folder_ids](variables-fast.tf#L39) | Folders created in the resource management stage. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>1-resman</code> |
| [groups](variables-fast.tf#L47) | Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>0-bootstrap</code> |
| [host_project_ids](variables-fast.tf#L56) | Host project for the shared VPC. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>2-networking</code> |
| [locations](variables-fast.tf#L64) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object&#40;&#123;&#10; gcs &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | <code>0-bootstrap</code> |
| [org_policy_tags](variables-fast.tf#L82) | Optional organization policy tag values. | <code title="object&#40;&#123;&#10; key_name &#61; optional&#40;string, &#34;org-policies&#34;&#41;&#10; values &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | <code>0-bootstrap</code> |
| [kms_keys](variables-fast.tf#L64) | KMS key ids. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>2-security</code> |
| [locations](variables-fast.tf#L72) | Optional locations for GCS, BigQuery, and logging buckets created here. | <code title="object&#40;&#123;&#10; gcs &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | <code>0-bootstrap</code> |
| [org_policy_tags](variables-fast.tf#L90) | Optional organization policy tag values. | <code title="object&#40;&#123;&#10; key_name &#61; optional&#40;string, &#34;org-policies&#34;&#41;&#10; values &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> | <code>0-bootstrap</code> |
| [outputs_location](variables.tf#L39) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | <code>string</code> | | <code>null</code> | |
| [perimeters](variables-fast.tf#L74) | Optional VPC-SC perimeter ids. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>1-vpcsc</code> |
| [service_accounts](variables-fast.tf#L103) | Automation service accounts in name => email format. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>1-resman</code> |
| [perimeters](variables-fast.tf#L82) | Optional VPC-SC perimeter ids. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>1-vpcsc</code> |
| [service_accounts](variables-fast.tf#L111) | Automation service accounts in name => email format. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>1-resman</code> |
| [stage_name](variables.tf#L45) | FAST stage name. Used to separate output files across different factories. | <code>string</code> | | <code>&#34;2-project-factory&#34;</code> | |
| [tag_values](variables-fast.tf#L111) | FAST-managed resource manager tag values. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>1-resman</code> |
| [tag_values](variables-fast.tf#L119) | FAST-managed resource manager tag values. | <code>map&#40;string&#41;</code> | | <code>&#123;&#125;</code> | <code>1-resman</code> |
## Outputs

View File

@@ -45,6 +45,10 @@ module "projects" {
var.groups,
var.factories_config.context.iam_principals
)
kms_keys = merge(
var.kms_keys,
var.factories_config.context.kms_keys
)
perimeters = var.perimeters
tag_values = merge(
{

View File

@@ -61,6 +61,14 @@ variable "host_project_ids" {
default = {}
}
variable "kms_keys" {
# tfdoc:variable:source 2-security
description = "KMS key ids."
type = map(string)
nullable = false
default = {}
}
variable "locations" {
# tfdoc:variable:source 0-bootstrap
description = "Optional locations for GCS, BigQuery, and logging buckets created here."

View File

@@ -25,8 +25,8 @@ variable "factories_config" {
notification_channels = optional(map(any), {})
}))
context = optional(object({
# TODO: add KMS keys
folder_ids = optional(map(string), {})
kms_keys = optional(map(string), {})
iam_principals = optional(map(string), {})
tag_values = optional(map(string), {})
vpc_host_projects = optional(map(string), {})

View File

@@ -279,6 +279,9 @@ module "project-factory" {
default = "folders/5678901234"
teams = "folders/5678901234"
}
kms_keys = {
compute-prod-ew1 = "projects/kms-central-prj/locations/europe-west1/keyRings/my-keyring/cryptoKeys/ew1-compute"
}
iam_principals = {
gcp-devops = "group:gcp-devops@example.org"
}
@@ -351,9 +354,12 @@ labels:
team: team-a
parent: team-a/app-0
service_encryption_key_ids:
storage.googleapis.com:
- projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce
storage.googleapis.com:
- projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce
compute.googleapis.com:
- compute-prod-ew1
services:
- compute.googleapis.com
- container.googleapis.com
- storage.googleapis.com
iam_by_principals:
@@ -517,7 +523,7 @@ service_accounts:
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [factories_config](variables.tf#L131) | Path to folder with YAML resource description data files. | <code title="object&#40;&#123;&#10; budgets &#61; optional&#40;object&#40;&#123;&#10; billing_account &#61; string&#10; budgets_data_path &#61; string&#10; notification_channels &#61; optional&#40;map&#40;any&#41;, &#123;&#125;&#41;&#10; &#125;&#41;&#41;&#10; context &#61; optional&#40;object&#40;&#123;&#10; folder_ids &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; iam_principals &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; perimeters &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; tag_values &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; vpc_host_projects &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; notification_channels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10; folders_data_path &#61; optional&#40;string&#41;&#10; projects_data_path &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | |
| [factories_config](variables.tf#L131) | Path to folder with YAML resource description data files. | <code title="object&#40;&#123;&#10; budgets &#61; optional&#40;object&#40;&#123;&#10; billing_account &#61; string&#10; budgets_data_path &#61; string&#10; notification_channels &#61; optional&#40;map&#40;any&#41;, &#123;&#125;&#41;&#10; &#125;&#41;&#41;&#10; context &#61; optional&#40;object&#40;&#123;&#10; folder_ids &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; iam_principals &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; kms_keys &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; perimeters &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; tag_values &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; vpc_host_projects &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; notification_channels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10; folders_data_path &#61; optional&#40;string&#41;&#10; projects_data_path &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | |
| [data_defaults](variables.tf#L17) | Optional default values used when corresponding project data from files are missing. | <code title="object&#40;&#123;&#10; billing_account &#61; optional&#40;string&#41;&#10; contacts &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; deletion_policy &#61; optional&#40;string&#41;&#10; factories_config &#61; optional&#40;object&#40;&#123;&#10; custom_roles &#61; optional&#40;string&#41;&#10; observability &#61; optional&#40;string&#41;&#10; org_policies &#61; optional&#40;string&#41;&#10; quotas &#61; optional&#40;string&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10; labels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; metric_scopes &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; parent &#61; optional&#40;string&#41;&#10; prefix &#61; optional&#40;string&#41;&#10; project_reuse &#61; optional&#40;object&#40;&#123;&#10; use_data_source &#61; optional&#40;bool, true&#41;&#10; project_attributes &#61; optional&#40;object&#40;&#123;&#10; name &#61; string&#10; number &#61; number&#10; services_enabled &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;&#10; service_encryption_key_ids &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; services &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; shared_vpc_service_config &#61; optional&#40;object&#40;&#123;&#10; host_project &#61; string&#10; network_users &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; service_agent_iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_agent_subnet_iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_iam_grants &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; network_subnet_users &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; &#125;&#41;&#41;&#10; storage_location &#61; optional&#40;string&#41;&#10; tag_bindings &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; service_accounts &#61; optional&#40;map&#40;object&#40;&#123;&#10; display_name &#61; optional&#40;string, &#34;Terraform-managed.&#34;&#41;&#10; iam_self_roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10; vpc_sc &#61; optional&#40;object&#40;&#123;&#10; perimeter_name &#61; string&#10; perimeter_bridges &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; is_dry_run &#61; optional&#40;bool, false&#41;&#10; &#125;&#41;&#41;&#10; logging_data_access &#61; optional&#40;map&#40;object&#40;&#123;&#10; ADMIN_READ &#61; optional&#40;object&#40;&#123; exempted_members &#61; optional&#40;list&#40;string&#41;&#41; &#125;&#41;&#41;,&#10; DATA_READ &#61; optional&#40;object&#40;&#123; exempted_members &#61; optional&#40;list&#40;string&#41;&#41; &#125;&#41;&#41;,&#10; DATA_WRITE &#61; optional&#40;object&#40;&#123; exempted_members &#61; optional&#40;list&#40;string&#41;&#41; &#125;&#41;&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [data_merges](variables.tf#L73) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | <code title="object&#40;&#123;&#10; contacts &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; labels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; metric_scopes &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; service_encryption_key_ids &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; services &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; tag_bindings &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; service_accounts &#61; optional&#40;map&#40;object&#40;&#123;&#10; display_name &#61; optional&#40;string, &#34;Terraform-managed.&#34;&#41;&#10; iam_self_roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [data_overrides](variables.tf#L92) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | <code title="object&#40;&#123;&#10; billing_account &#61; optional&#40;string&#41;&#10; contacts &#61; optional&#40;map&#40;list&#40;string&#41;&#41;&#41;&#10; deletion_policy &#61; optional&#40;string&#41;&#10; factories_config &#61; optional&#40;object&#40;&#123;&#10; custom_roles &#61; optional&#40;string&#41;&#10; observability &#61; optional&#40;string&#41;&#10; org_policies &#61; optional&#40;string&#41;&#10; quotas &#61; optional&#40;string&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10; parent &#61; optional&#40;string&#41;&#10; prefix &#61; optional&#40;string&#41;&#10; service_encryption_key_ids &#61; optional&#40;map&#40;list&#40;string&#41;&#41;&#41;&#10; storage_location &#61; optional&#40;string&#41;&#10; tag_bindings &#61; optional&#40;map&#40;string&#41;&#41;&#10; services &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_accounts &#61; optional&#40;map&#40;object&#40;&#123;&#10; display_name &#61; optional&#40;string, &#34;Terraform-managed.&#34;&#41;&#10; iam_self_roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#41;&#10; vpc_sc &#61; optional&#40;object&#40;&#123;&#10; perimeter_name &#61; string&#10; perimeter_bridges &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; is_dry_run &#61; optional&#40;bool, false&#41;&#10; &#125;&#41;&#41;&#10; logging_data_access &#61; optional&#40;map&#40;object&#40;&#123;&#10; ADMIN_READ &#61; optional&#40;object&#40;&#123; exempted_members &#61; optional&#40;list&#40;string&#41;&#41; &#125;&#41;&#41;,&#10; DATA_READ &#61; optional&#40;object&#40;&#123; exempted_members &#61; optional&#40;list&#40;string&#41;&#41; &#125;&#41;&#41;,&#10; DATA_WRITE &#61; optional&#40;object&#40;&#123; exempted_members &#61; optional&#40;list&#40;string&#41;&#41; &#125;&#41;&#41;&#10; &#125;&#41;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |

View File

@@ -90,10 +90,14 @@ module "projects" {
))
notification_channels = try(each.value.notification_channels, null)
org_policies = each.value.org_policies
service_encryption_key_ids = merge(
each.value.service_encryption_key_ids,
var.data_merges.service_encryption_key_ids
)
service_encryption_key_ids = {
for k, v in merge(
each.value.service_encryption_key_ids,
var.data_merges.service_encryption_key_ids
) : k => [
for key in v : lookup(var.factories_config.context.kms_keys, key, key)
]
}
services = distinct(concat(
each.value.services,
var.data_merges.services

View File

@@ -138,9 +138,9 @@ variable "factories_config" {
notification_channels = optional(map(any), {})
}))
context = optional(object({
# TODO: add KMS keys
folder_ids = optional(map(string), {})
iam_principals = optional(map(string), {})
kms_keys = optional(map(string), {})
perimeters = optional(map(string), {})
tag_values = optional(map(string), {})
vpc_host_projects = optional(map(string), {})

View File

@@ -201,6 +201,10 @@ values:
- ALL
parent: projects/test-pf-dev-ta-app0-be
timeouts: null
? module.project-factory.module.projects["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["key-0.compute-system"]
: condition: []
crypto_key_id: projects/kms-central-prj/locations/europe-west1/keyRings/my-keyring/cryptoKeys/ew1-compute
role: roles/cloudkms.cryptoKeyEncrypterDecrypter
? module.project-factory.module.projects["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["key-0.gs-project-accounts"]
: condition: []
crypto_key_id: projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce
@@ -545,12 +549,12 @@ counts:
google_essential_contacts_contact: 4
google_folder: 5
google_folder_iam_binding: 1
google_kms_crypto_key_iam_member: 1
google_kms_crypto_key_iam_member: 2
google_monitoring_notification_channel: 1
google_project: 4
google_project_iam_binding: 6
google_project_iam_member: 20
google_project_service: 12
google_project_iam_member: 21
google_project_service: 13
google_project_service_identity: 4
google_service_account: 6
google_service_account_iam_binding: 1
@@ -562,6 +566,6 @@ counts:
google_tags_tag_value: 2
google_tags_tag_value_iam_binding: 1
modules: 21
resources: 80
resources: 83
outputs: {}