Add support for dynamic tags (#3897)

* Allow creation of dynamic tags * Extend project factory and related modules to support dynamic values * Extend folder and organization modules * project and organization readme * Simplify dynamic tag support and remove unnecessary restrictions • Schemas & Validations: Removed the restriction that forbade combining IAM fields with allowed_values_regex on tags. Updated validations in project and organization modules, and simplified all relevant JSON schemas. • Module Tag Bindings: Simplified the tag_value assignment in folder , project , gcs , bigquery-dataset , and kms modules by removing the defensive can(regex(...)) check and calling templatestring directly. • Outputs: Removed the tags_dynamic output from project and organization modules, as the same information is now available in tag_keys . • Project Factory: Updated tag_vars_projects in projects.tf to use the native namespaced_name attribute and filtered manually for dynamic tags. * fix(organization, project): fix linting and tests for dynamic tag support - Align allowed_values_regex and description extraction in _tags_merged locals to use lookup() for consistency with other fields. - Fix spacing in project context variable (alphabetical ordering). - Update organization tags test to include the new cost_center tag key with allowed_values_regex. - Update project tags test to include the new cost_center tag key and reflect the resolved allowed_values_regex on environment. * refactor(gcs): refine tag bindings and fix context test - Add _tag_bindings local to pre-resolve context references, enabling templatestring to receive a direct map reference (required by Terraform). - Use var.context.tag_vars instead of the non-existent local.ctx.tag_vars. - Fix HCL syntax in context.tfvars (escaped inner quotes). - Update context test inventory to reflect 3 tag bindings including a dynamic value resolved via templatestring. * refactor: align modules with tag binding context pattern - Add _tag_bindings local + templatestring dance to cloud-run-v2, compute-vm, folder, kms modules (bigquery-dataset already had it) - Exclude tag_vars from local.ctx in cloud-run-v2, compute-vm, folder, kms, project modules (bigquery-dataset already had it) - Add tag_vars to context variable in cloud-run-v2, compute-vm modules (others already had it) - Update all context tests with dynamic tag binding values using var.context.tag_vars * docs: add module-level tftest.yaml test instructions to GEMINI.md * docs: regenerate READMEs after tag-regex alignment - Regenerate variable tables in 7 module READMEs to reflect line number shifts from prior tag-regex changes - Add tag_vars exclusion to gcs ctx local - Fix whitespace alignment in iam-service-account and project-factory tag_vars blocks - Update tftest resource counts for organization and project - Remove tags_dynamic from organization/project output tables * fix(project-factory): update test inventory for tag_bindings module split - Move tag binding address from folder-2 to folder-2-iam in test inventory (tag_bindings moved from creation to IAM modules) - Update module instance count from 34 to 35 - Regenerate README tables after terraform fmt line shifts - Apply terraform fmt to variables.tf * refactor(project-factory): remove unnecessary depends_on from folder-iam modules Folder IAM modules depend on their own folder creation modules, not on module.projects. The explicit depends_on was leftover from an earlier design. * FAST stages * Address review comments. - FAST Stages: - Added tag_keys to output-files.tf in 0-org-setup to pass org tags via tfvars. - Sorted tag_keys and tag_values in output-files.tf. - Updated project-factory, networking, and security stages to use tag_keys. - Filtered tag_keys for dynamic tags only. - Modules: - Excluded tag_vars from local.ctx in iam-service-account and organization. - Simplified tag_value in iam-service-account. - Tests: - Updated test inventories for 0-org-setup and project-factory. * Fix tf format * Fix tfdoc * docs: add ADR for templatestring vars convention and update status of base path ADR * More tfdoc * Update schemas * Use endswith in context loop * Address review * Update FAST readmes * Update last modules * Terraform fmt * Revert alloydb * Fix whitespace --------- Co-authored-by: Ludovico Magnocavallo <ludo@qix.it>
2026-04-24 22:45:45 +02:00
parent fb33752d8d
commit 2eaa0d5e27
118 changed files with 1018 additions and 378 deletions
--- a/modules/project-factory/README.md
+++ b/modules/project-factory/README.md
@@ -881,11 +881,11 @@ compute.disableSerialPortAccess:

 | name | description | type | required | default |
 |---|---|:---:|:---:|:---:|
-| [factories_config](variables.tf#L166) | Path to folder with YAML resource description data files. | <code>object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  |
+| [factories_config](variables.tf#L170) | Path to folder with YAML resource description data files. | <code>object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ |  |
 | [context](variables.tf#L17) | Context-specific interpolations. | <code>object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [data_defaults](variables.tf#L43) | Optional default values used when corresponding project or folder data from files are missing. | <code>object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [data_merges](variables.tf#L108) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | <code>object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [data_overrides](variables.tf#L127) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | <code>object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [data_defaults](variables.tf#L47) | Optional default values used when corresponding project or folder data from files are missing. | <code>object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [data_merges](variables.tf#L112) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | <code>object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [data_overrides](variables.tf#L131) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | <code>object&#40;&#123;&#8230;&#125;&#41;</code> |  | <code>&#123;&#125;</code> |
 | [folders](variables-folders.tf#L17) | Folders data merged with factory data. | <code>map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
 | [notification_channels](variables-billing.tf#L17) | Notification channels used by budget alerts. | <code>map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
 | [projects](variables-projects.tf#L17) | Projects data merged with factory data. | <code>map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
--- a/modules/project-factory/folders.tf
+++ b/modules/project-factory/folders.tf
@@ -69,7 +69,6 @@ module "folder-1" {
  }
  org_policies            = lookup(each.value, "org_policies", {})
  pam_entitlements        = lookup(each.value, "pam_entitlements", {})
-  tag_bindings            = lookup(each.value, "tag_bindings", {})
  assured_workload_config = lookup(each.value, "assured_workload_config", null)
  logging_settings = anytrue([
    try(each.value.logging.storage_location, null) != null,
@@ -107,7 +106,12 @@ module "folder-1-iam" {
  iam_by_principals_conditional = lookup(each.value, "iam_by_principals_conditional", {})
  logging_data_access           = lookup(each.value, "data_access_logs", {})
  logging_sinks                 = try(each.value.logging.sinks, {})
+  tag_bindings                  = lookup(each.value, "tag_bindings", {})
  context = merge(local.ctx, {
+    tag_vars = {
+      projects     = merge(try(local.ctx.tag_vars.projects, {}), local.tag_vars_projects)
+      organization = try(local.ctx.tag_vars.organization, {})
+    }
    iam_principals  = local.ctx_iam_principals
    kms_keys        = merge(local.ctx.kms_keys, local.kms_keys)
    project_ids     = local.ctx_project_ids
@@ -137,7 +141,6 @@ module "folder-2" {
  }
  org_policies            = lookup(each.value, "org_policies", {})
  pam_entitlements        = lookup(each.value, "pam_entitlements", {})
-  tag_bindings            = lookup(each.value, "tag_bindings", {})
  assured_workload_config = lookup(each.value, "assured_workload_config", null)
  logging_settings = anytrue([
    try(each.value.logging.storage_location, null) != null,
@@ -180,10 +183,15 @@ module "folder-2-iam" {
  iam_by_principals_conditional = lookup(each.value, "iam_by_principals_conditional", {})
  logging_data_access           = lookup(each.value, "data_access_logs", {})
  logging_sinks                 = try(each.value.logging.sinks, {})
+  tag_bindings                  = lookup(each.value, "tag_bindings", {})
  context = merge(local.ctx, {
    folder_ids = merge(local.ctx.folder_ids, {
      for k, v in module.folder-1 : k => v.id
    })
+    tag_vars = {
+      projects     = merge(try(local.ctx.tag_vars.projects, {}), local.tag_vars_projects)
+      organization = try(local.ctx.tag_vars.organization, {})
+    }
    iam_principals  = local.ctx_iam_principals
    kms_keys        = merge(local.ctx.kms_keys, local.kms_keys)
    project_ids     = local.ctx_project_ids
@@ -213,7 +221,6 @@ module "folder-3" {
  }
  org_policies            = lookup(each.value, "org_policies", {})
  pam_entitlements        = lookup(each.value, "pam_entitlements", {})
-  tag_bindings            = lookup(each.value, "tag_bindings", {})
  assured_workload_config = lookup(each.value, "assured_workload_config", null)
  logging_settings = anytrue([
    try(each.value.logging.storage_location, null) != null,
@@ -256,10 +263,15 @@ module "folder-3-iam" {
  iam_by_principals_conditional = lookup(each.value, "iam_by_principals_conditional", {})
  logging_data_access           = lookup(each.value, "data_access_logs", {})
  logging_sinks                 = try(each.value.logging.sinks, {})
+  tag_bindings                  = lookup(each.value, "tag_bindings", {})
  context = merge(local.ctx, {
    folder_ids = merge(local.ctx.folder_ids, {
      for k, v in module.folder-2 : k => v.id
    })
+    tag_vars = {
+      projects     = merge(try(local.ctx.tag_vars.projects, {}), local.tag_vars_projects)
+      organization = try(local.ctx.tag_vars.organization, {})
+    }
    iam_principals  = local.ctx_iam_principals
    kms_keys        = merge(local.ctx.kms_keys, local.kms_keys)
    project_ids     = local.ctx_project_ids
@@ -289,7 +301,6 @@ module "folder-4" {
  }
  org_policies            = lookup(each.value, "org_policies", {})
  pam_entitlements        = lookup(each.value, "pam_entitlements", {})
-  tag_bindings            = lookup(each.value, "tag_bindings", {})
  assured_workload_config = lookup(each.value, "assured_workload_config", null)
  logging_settings = anytrue([
    try(each.value.logging.storage_location, null) != null,
@@ -332,10 +343,15 @@ module "folder-4-iam" {
  iam_by_principals_conditional = lookup(each.value, "iam_by_principals_conditional", {})
  logging_data_access           = lookup(each.value, "data_access_logs", {})
  logging_sinks                 = try(each.value.logging.sinks, {})
+  tag_bindings                  = lookup(each.value, "tag_bindings", {})
  context = merge(local.ctx, {
    folder_ids = merge(local.ctx.folder_ids, {
      for k, v in module.folder-3 : k => v.id
    })
+    tag_vars = {
+      projects     = merge(try(local.ctx.tag_vars.projects, {}), local.tag_vars_projects)
+      organization = try(local.ctx.tag_vars.organization, {})
+    }
    iam_principals  = local.ctx_iam_principals
    kms_keys        = merge(local.ctx.kms_keys, local.kms_keys)
    project_ids     = local.ctx_project_ids
--- a/modules/project-factory/projects-bigquery.tf
+++ b/modules/project-factory/projects-bigquery.tf
@@ -51,6 +51,10 @@ module "bigquery-datasets" {
  project_id = module.projects-iam[each.value.project_key].project_id
  id         = each.value.id
  context = merge(local.ctx, {
+    tag_vars = {
+      projects     = merge(try(local.ctx.tag_vars.projects, {}), local.tag_vars_projects)
+      organization = try(local.ctx.tag_vars.organization, {})
+    }
    iam_principals = merge(
      local.ctx.iam_principals,
      local.projects_sas_iam_emails,
--- a/modules/project-factory/projects-buckets.tf
+++ b/modules/project-factory/projects-buckets.tf
@@ -73,6 +73,10 @@ module "buckets" {
  encryption_key = each.value.encryption_key
  force_destroy  = each.value.force_destroy
  context = merge(local.ctx, {
+    tag_vars = {
+      projects     = merge(try(local.ctx.tag_vars.projects, {}), local.tag_vars_projects)
+      organization = try(local.ctx.tag_vars.organization, {})
+    }
    iam_principals = merge(
      local.ctx.iam_principals,
      local.projects_sas_iam_emails,
--- a/modules/project-factory/projects-defaults.tf
+++ b/modules/project-factory/projects-defaults.tf
@@ -149,6 +149,7 @@ locals {
      )
      tags = {
        for tag_name, tag_data in try(v.tags, {}) : tag_name => {
+          allowed_values_regex = try(tag_data.allowed_values_regex, null)
          description = try(
            tag_data.description,
            "Managed by the Terraform project-factory module."
--- a/modules/project-factory/projects-kms.tf
+++ b/modules/project-factory/projects-kms.tf
@@ -68,6 +68,10 @@ module "kms" {
  tag_bindings          = each.value.tag_bindings
  keys                  = each.value.keys
  context = merge(local.ctx, {
+    tag_vars = {
+      projects     = merge(try(local.ctx.tag_vars.projects, {}), local.tag_vars_projects)
+      organization = try(local.ctx.tag_vars.organization, {})
+    }
    iam_principals = merge(
      local.ctx.iam_principals,
      local.projects_sas_iam_emails,
--- a/modules/project-factory/projects-service-accounts.tf
+++ b/modules/project-factory/projects-service-accounts.tf
@@ -85,6 +85,10 @@ module "service-accounts" {
  description  = each.value.description
  display_name = each.value.display_name
  context = merge(local.ctx, {
+    tag_vars = {
+      projects     = merge(try(local.ctx.tag_vars.projects, {}), local.tag_vars_projects)
+      organization = try(local.ctx.tag_vars.organization, {})
+    }
    project_ids = local.ctx_project_ids
    tag_values  = local.ctx_tag_values
  })
--- a/modules/project-factory/projects.tf
+++ b/modules/project-factory/projects.tf
@@ -63,6 +63,13 @@ locals {
      }
    ]...) : k => v
  })
+  tag_vars_projects = {
+    for k, v in local.projects_input : v.name => {
+      for kk, vv in module.projects[k].tag_keys :
+      kk => vv.namespaced_name
+      if vv.allowed_values_regex != null
+    }
+  }
  per_project_service_agents = {
    for k, v in module.projects : k => {
      for kk, vv in v.service_agents :
@@ -169,6 +176,10 @@ module "projects-iam" {
    }
  }
  context = merge(local.ctx, {
+    tag_vars = {
+      projects     = merge(try(local.ctx.tag_vars.projects, {}), local.tag_vars_projects)
+      organization = try(local.ctx.tag_vars.organization, {})
+    }
    folder_ids = local.ctx.folder_ids
    kms_keys   = merge(local.ctx.kms_keys, local.kms_keys)
    iam_principals = merge(
--- a/modules/project-factory/schemas/project.schema.json
+++ b/modules/project-factory/schemas/project.schema.json
@@ -834,6 +834,9 @@
        "type": "object",
        "additionalProperties": false,
        "properties": {
+          "allowed_values_regex": {
+            "type": "string"
+          },
          "description": {
            "type": "string"
          },
@@ -873,7 +876,23 @@
              }
            }
          }
-        }
+        },
+        "allOf": [
+          {
+            "if": {
+              "required": [
+                "allowed_values_regex"
+              ]
+            },
+            "then": {
+              "not": {
+                "required": [
+                  "values"
+                ]
+              }
+            }
+          }
+        ]
      }
    },
    "tag_bindings": {
--- a/modules/project-factory/variables.tf
+++ b/modules/project-factory/variables.tf
@@ -33,8 +33,12 @@ variable "context" {
    storage_buckets       = optional(map(string), {})
    tag_keys              = optional(map(string), {})
    tag_values            = optional(map(string), {})
-    vpc_host_projects     = optional(map(string), {})
-    vpc_sc_perimeters     = optional(map(string), {})
+    tag_vars = optional(object({
+      projects     = optional(map(map(string)), {})
+      organization = optional(map(string), {})
+    }), {})
+    vpc_host_projects = optional(map(string), {})
+    vpc_sc_perimeters = optional(map(string), {})
  })
  default  = {}
  nullable = false