2eaa0d5e27
* Allow creation of dynamic tags * Extend project factory and related modules to support dynamic values * Extend folder and organization modules * project and organization readme * Simplify dynamic tag support and remove unnecessary restrictions • Schemas & Validations: Removed the restriction that forbade combining IAM fields with allowed_values_regex on tags. Updated validations in project and organization modules, and simplified all relevant JSON schemas. • Module Tag Bindings: Simplified the tag_value assignment in folder , project , gcs , bigquery-dataset , and kms modules by removing the defensive can(regex(...)) check and calling templatestring directly. • Outputs: Removed the tags_dynamic output from project and organization modules, as the same information is now available in tag_keys . • Project Factory: Updated tag_vars_projects in projects.tf to use the native namespaced_name attribute and filtered manually for dynamic tags. * fix(organization, project): fix linting and tests for dynamic tag support - Align allowed_values_regex and description extraction in _tags_merged locals to use lookup() for consistency with other fields. - Fix spacing in project context variable (alphabetical ordering). - Update organization tags test to include the new cost_center tag key with allowed_values_regex. - Update project tags test to include the new cost_center tag key and reflect the resolved allowed_values_regex on environment. * refactor(gcs): refine tag bindings and fix context test - Add _tag_bindings local to pre-resolve context references, enabling templatestring to receive a direct map reference (required by Terraform). - Use var.context.tag_vars instead of the non-existent local.ctx.tag_vars. - Fix HCL syntax in context.tfvars (escaped inner quotes). - Update context test inventory to reflect 3 tag bindings including a dynamic value resolved via templatestring. * refactor: align modules with tag binding context pattern - Add _tag_bindings local + templatestring dance to cloud-run-v2, compute-vm, folder, kms modules (bigquery-dataset already had it) - Exclude tag_vars from local.ctx in cloud-run-v2, compute-vm, folder, kms, project modules (bigquery-dataset already had it) - Add tag_vars to context variable in cloud-run-v2, compute-vm modules (others already had it) - Update all context tests with dynamic tag binding values using var.context.tag_vars * docs: add module-level tftest.yaml test instructions to GEMINI.md * docs: regenerate READMEs after tag-regex alignment - Regenerate variable tables in 7 module READMEs to reflect line number shifts from prior tag-regex changes - Add tag_vars exclusion to gcs ctx local - Fix whitespace alignment in iam-service-account and project-factory tag_vars blocks - Update tftest resource counts for organization and project - Remove tags_dynamic from organization/project output tables * fix(project-factory): update test inventory for tag_bindings module split - Move tag binding address from folder-2 to folder-2-iam in test inventory (tag_bindings moved from creation to IAM modules) - Update module instance count from 34 to 35 - Regenerate README tables after terraform fmt line shifts - Apply terraform fmt to variables.tf * refactor(project-factory): remove unnecessary depends_on from folder-iam modules Folder IAM modules depend on their own folder creation modules, not on module.projects. The explicit depends_on was leftover from an earlier design. * FAST stages * Address review comments. - FAST Stages: - Added tag_keys to output-files.tf in 0-org-setup to pass org tags via tfvars. - Sorted tag_keys and tag_values in output-files.tf. - Updated project-factory, networking, and security stages to use tag_keys. - Filtered tag_keys for dynamic tags only. - Modules: - Excluded tag_vars from local.ctx in iam-service-account and organization. - Simplified tag_value in iam-service-account. - Tests: - Updated test inventories for 0-org-setup and project-factory. * Fix tf format * Fix tfdoc * docs: add ADR for templatestring vars convention and update status of base path ADR * More tfdoc * Update schemas * Use endswith in context loop * Address review * Update FAST readmes * Update last modules * Terraform fmt * Revert alloydb * Fix whitespace --------- Co-authored-by: Ludovico Magnocavallo <ludo@qix.it>
95 lines
3.4 KiB
Markdown
95 lines
3.4 KiB
Markdown
# Adopt a base path for datasets
|
|
|
|
**authors:** [Julio](https://github.com/juliocc) [Ludo](https://github.com/ludoo)
|
|
**date:** Feb 10, 2026
|
|
|
|
## Status
|
|
|
|
Implemented
|
|
|
|
## Context
|
|
|
|
This ADR provides a potential solution to two concurrent issues.
|
|
|
|
FAST stages still use the old per-factory path approach, which makes it harder to switch datasets.
|
|
|
|
```hcl
|
|
variable "factories_config" {
|
|
description = "Configuration for the resource factories or external data."
|
|
type = object({
|
|
billing_accounts = optional(string, "datasets/classic/billing-accounts")
|
|
cicd_workflows = optional(string)
|
|
defaults = optional(string, "datasets/classic/defaults.yaml")
|
|
folders = optional(string, "datasets/classic/folders")
|
|
observability = optional(string, "datasets/classic/observability")
|
|
organization = optional(string, "datasets/classic/organization")
|
|
project_templates = optional(string, "datasets/classic/templates")
|
|
projects = optional(string, "datasets/classic/projects")
|
|
})
|
|
nullable = false
|
|
default = {}
|
|
}
|
|
```
|
|
|
|
Project-level factories in the project factory module use relative paths based on the root module in scope, forcing users to embed paths in the YAML files.
|
|
|
|
```yaml
|
|
factories_config:
|
|
observability: datasets/classic/observability/iac-0
|
|
```
|
|
|
|
## Proposed Approach
|
|
|
|
The proposed approach changes the `factories_config` variable in FAST stages so that a new `dataset` attribute is added, and existing lower-level paths are moved to a `paths` attribute.
|
|
|
|
```hcl
|
|
variable "factories_config" {
|
|
description = "Configuration for the resource factories or external data."
|
|
type = object({
|
|
dataset = optional(string, "datasets/classic")
|
|
paths = optional(object({
|
|
billing_accounts = optional(string, "billing-accounts")
|
|
cicd_workflows = optional(string)
|
|
defaults = optional(string, "defaults.yaml")
|
|
folders = optional(string, "folders")
|
|
observability = optional(string, "observability")
|
|
organization = optional(string, "organization")
|
|
project_templates = optional(string, "templates")
|
|
projects = optional(string, "projects")
|
|
}), {})
|
|
})
|
|
nullable = false
|
|
default = {}
|
|
}
|
|
```
|
|
|
|
This allows one-line configuration of the dataset, while still providing a way to cancel out individual factories by omitting the path, or pointing to a non-existing folder. The base path will not be prepended for paths starting with `/` or `.`, to allow for different absolute or relative paths, and to also allow our testing framweork to inject fixtures.
|
|
|
|
On the project factory side, the `factories_config` variable will also change by adopting a "base path" and grouping existing attributes under a `paths` variable.
|
|
|
|
```hcl
|
|
variable "factories_config" {
|
|
description = "Path to folder with YAML resource description data files."
|
|
type = object({
|
|
base_path = "data"
|
|
paths = optional(object({
|
|
folders = optional(string)
|
|
project_templates = optional(string)
|
|
projects = optional(string)
|
|
budgets = optional(object({
|
|
billing_account_id = string
|
|
data = string
|
|
}))
|
|
}), {})
|
|
})
|
|
nullable = false
|
|
}
|
|
```
|
|
|
|
The lower level modules will not change interface, but the base path will be prepended by the project factory to in-project `factories_config` paths, to allow decoupling from the dataset and creating portable files.
|
|
|
|
```yaml
|
|
factories_config:
|
|
observability: observability/iac-0
|
|
```
|