Files
hunfabric/modules/bigquery-connection
Julio Castillo 2eaa0d5e27 Add support for dynamic tags (#3897)
* Allow creation of dynamic tags

* Extend project factory and related modules to support dynamic values

* Extend folder and organization modules

* project and organization readme

* Simplify dynamic tag support and remove unnecessary restrictions

  • Schemas & Validations: Removed the restriction that forbade combining IAM fields with  allowed_values_regex  on tags. Updated validations in  project  and  organization  modules, and
  simplified all relevant JSON schemas.
  • Module Tag Bindings: Simplified the  tag_value  assignment in  folder ,  project ,  gcs ,  bigquery-dataset , and  kms  modules by removing the defensive  can(regex(...))  check and
  calling  templatestring  directly.
  • Outputs: Removed the  tags_dynamic  output from  project  and  organization  modules, as the same information is now available in  tag_keys .
  • Project Factory: Updated  tag_vars_projects  in  projects.tf  to use the native  namespaced_name  attribute and filtered manually for dynamic tags.

* fix(organization, project): fix linting and tests for dynamic tag support

- Align allowed_values_regex and description extraction in _tags_merged
  locals to use lookup() for consistency with other fields.
- Fix spacing in project context variable (alphabetical ordering).
- Update organization tags test to include the new cost_center tag key
  with allowed_values_regex.
- Update project tags test to include the new cost_center tag key and
  reflect the resolved allowed_values_regex on environment.

* refactor(gcs): refine tag bindings and fix context test

- Add _tag_bindings local to pre-resolve context references, enabling
  templatestring to receive a direct map reference (required by Terraform).
- Use var.context.tag_vars instead of the non-existent local.ctx.tag_vars.
- Fix HCL syntax in context.tfvars (escaped inner quotes).
- Update context test inventory to reflect 3 tag bindings including a
  dynamic value resolved via templatestring.

* refactor: align modules with tag binding context pattern

- Add _tag_bindings local + templatestring dance to cloud-run-v2,
  compute-vm, folder, kms modules (bigquery-dataset already had it)
- Exclude tag_vars from local.ctx in cloud-run-v2, compute-vm, folder,
  kms, project modules (bigquery-dataset already had it)
- Add tag_vars to context variable in cloud-run-v2, compute-vm modules
  (others already had it)
- Update all context tests with dynamic tag binding values using
  var.context.tag_vars

* docs: add module-level tftest.yaml test instructions to GEMINI.md

* docs: regenerate READMEs after tag-regex alignment

- Regenerate variable tables in 7 module READMEs to reflect
  line number shifts from prior tag-regex changes
- Add tag_vars exclusion to gcs ctx local
- Fix whitespace alignment in iam-service-account and
  project-factory tag_vars blocks
- Update tftest resource counts for organization and project
- Remove tags_dynamic from organization/project output tables

* fix(project-factory): update test inventory for tag_bindings module split

- Move tag binding address from folder-2 to folder-2-iam in test
  inventory (tag_bindings moved from creation to IAM modules)
- Update module instance count from 34 to 35
- Regenerate README tables after terraform fmt line shifts
- Apply terraform fmt to variables.tf

* refactor(project-factory): remove unnecessary depends_on from folder-iam modules

Folder IAM modules depend on their own folder creation modules, not
on module.projects. The explicit depends_on was leftover from an
earlier design.

* FAST stages

* Address review comments.

- FAST Stages:
  - Added tag_keys to output-files.tf in 0-org-setup to pass org tags via tfvars.
  - Sorted tag_keys and tag_values in output-files.tf.
  - Updated project-factory, networking, and security stages to use tag_keys.
  - Filtered tag_keys for dynamic tags only.
- Modules:
  - Excluded tag_vars from local.ctx in iam-service-account and organization.
  - Simplified tag_value in iam-service-account.
- Tests:
  - Updated test inventories for 0-org-setup and project-factory.

* Fix tf format

* Fix tfdoc

* docs: add ADR for templatestring vars convention and update status of base path ADR

* More tfdoc

* Update schemas

* Use endswith in context loop

* Address review

* Update FAST readmes

* Update last modules

* Terraform fmt

* Revert alloydb

* Fix whitespace

---------

Co-authored-by: Ludovico Magnocavallo <ludo@qix.it>
2026-04-24 20:45:45 +00:00
..
2025-10-16 15:15:36 +00:00
2026-04-24 20:45:45 +00:00
2025-10-16 15:15:36 +00:00

BigQuery Connection

This module allows creating a BigQuery connection.

Cloud SQL Connection

module "bigquery-connection" {
  source        = "./fabric/modules/bigquery-connection"
  project_id    = var.project_id
  location      = "EU"
  connection_id = "my-connection"
  friendly_name = "My Cloud SQL Connection"
  description   = "A connection to a Cloud SQL instance."

  connection_config = {
    cloud_sql = {
      instance_id = "my-instance-id"
      database    = "my-database"
      type        = "POSTGRES"
      credential = {
        username = "my-username"
        password = "my-password"
      }
    }
  }
  iam = {
    "roles/bigquery.connectionUser" = ["user:my-user@example.com"]
  }
}
# tftest modules=1 resources=2 inventory=cloudsql.yaml

Cloud SQL Connection with Cloud KMS

module "bigquery-connection" {
  source         = "./fabric/modules/bigquery-connection"
  project_id     = var.project_id
  location       = "EU"
  connection_id  = "my-connection"
  friendly_name  = "My BigQuery Connection"
  description    = "A connection to a Cloud SQL instance."
  encryption_key = "my-key"

  connection_config = {
    cloud_sql = {
      instance_id = "my-instance-id"
      database    = "my-database"
      type        = "POSTGRES"
      credential = {
        username = "my-username"
        password = "my-password"
      }
    }
  }
}
# tftest modules=1 resources=1 inventory=cloudsql_kms.yaml

Spanner Connection

module "bigquery-connection" {
  source        = "./fabric/modules/bigquery-connection"
  project_id    = var.project_id
  location      = "EU"
  connection_id = "my-connection"
  friendly_name = "My BigQuery Connection"
  description   = "A connection to a Spanner instance."

  connection_config = {
    cloud_spanner = {
      database        = "projects/my-project/instances/my-instance/databases/my-database"
      use_parallelism = true
      use_data_boost  = true
      max_parallelism = 2
    }
  }
  iam = {
    "roles/bigquery.connectionUser" = ["user:my-user@example.com"]
  }
}
# tftest modules=1 resources=2 inventory=spanner.yaml

Spanner Connection with Context interpolations

module "bigquery-connection" {
  source        = "./fabric/modules/bigquery-connection"
  project_id    = var.project_id
  location      = "EU"
  connection_id = "my-connection"
  friendly_name = "My BigQuery Connection"
  description   = "A connection to a Spanner instance."

  connection_config = {
    cloud_spanner = {
      database        = "projects/my-project/instances/my-instance/databases/my-database"
      use_parallelism = true
      use_data_boost  = true
      max_parallelism = 2
    }
  }
  context = {
    iam_principals = {
      myuser = "user:my-user@example.com"
    }
  }
  iam = {
    "roles/bigquery.connectionUser" = ["$iam_principals:myuser"]
  }
}
# tftest modules=1 resources=2 inventory=spanner_context.yaml

Variables

name description type required default
connection_id The ID of the connection. string
location The geographic location where the connection should reside. string
project_id The ID of the project in which the resource belongs. string
connection_config Connection properties. object({…}) {}
context Context-specific interpolations. object({…}) {}
description A description of the connection. string null
encryption_key The name of the KMS key used for encryption. string null
friendly_name A descriptive name for the connection. string null
iam IAM bindings for the connection in {ROLE => [MEMBERS]} format. map(list(string)) {}
iam_bindings Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. map(object({…})) {}
iam_bindings_additive Individual additive IAM bindings. Keys are arbitrary. map(object({…})) {}
iam_by_principals Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the iam variable. map(list(string)) {}

Outputs

name description sensitive
connection_config The connection configuration.
connection_id The ID of the BigQuery connection.
description The description of the connection.
location The location of the connection.