diff --git a/fast/stages-multitenant/0-bootstrap-tenant/README.md b/fast/stages-multitenant/0-bootstrap-tenant/README.md index bbeaf9f69..515148c44 100644 --- a/fast/stages-multitenant/0-bootstrap-tenant/README.md +++ b/fast/stages-multitenant/0-bootstrap-tenant/README.md @@ -147,6 +147,20 @@ Configure the tenant variable in a tfvars file for this stage. A few minor point Once the configuration is done just go through the usual `init/apply` cycle. On successful apply, a tfvars file specific for this tenant and a set of provider files will be created. +#### Using delayed billing association for projects + +This configuration is possible but unsupported and only exists for development purposes, use at your own risk: + +- temporarily switch `billing_account.id` to `null` in `globals.auto.tfvars.json` +- for each project resources in the project modules used in this stage (`automation-project`, `log-export-project`) + - apply using `-target`, for example + `terraform apply -target 'module.automation-project.google_project.project[0]'` + - untaint the project resource after applying, for example + `terraform untaint 'module.automation-project.google_project.project[0]'` +- go through the process to associate the billing account with the two projects +- switch `billing_account.id` back to the real billing account id +- resume applying normally + ### TODO - [ ] tenant-level Workload Identity Federation pool and providers configuration @@ -177,25 +191,25 @@ Once the configuration is done just go through the usual `init/apply` cycle. On | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| | [automation](variables.tf#L20) | Automation resources created by the organization-level bootstrap stage. | object({…}) | ✓ | | 0-bootstrap | -| [billing_account](variables.tf#L38) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | object({…}) | ✓ | | 0-bootstrap | -| [organization](variables.tf#L194) | Organization details. | object({…}) | ✓ | | 0-bootstrap | -| [prefix](variables.tf#L210) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 0-bootstrap | -| [tag_keys](variables.tf#L233) | Organization tag keys. | object({…}) | ✓ | | 1-resman | -| [tag_names](variables.tf#L244) | Customized names for resource management tags. | object({…}) | ✓ | | 1-resman | -| [tag_values](variables.tf#L255) | Organization resource management tag values. | map(string) | ✓ | | 1-resman | -| [tenant_config](variables.tf#L262) | Tenant configuration. Short name must be 4 characters or less. | object({…}) | ✓ | | | -| [cicd_repositories](variables.tf#L51) | CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | object({…}) | | null | | -| [custom_roles](variables.tf#L97) | Custom roles defined at the organization level, in key => id format. | object({…}) | | null | 0-bootstrap | -| [fast_features](variables.tf#L107) | Selective control for top-level FAST features. | object({…}) | | {} | 0-bootstrap | -| [federated_identity_providers](variables.tf#L121) | Workload Identity Federation pools. The `cicd_repositories` variable references keys here. | map(object({…})) | | {} | | -| [group_iam](variables.tf#L135) | Tenant-level custom group IAM settings in group => [roles] format. | map(list(string)) | | {} | | -| [iam](variables.tf#L141) | Tenant-level custom IAM settings in role => [principal] format. | map(list(string)) | | {} | | -| [iam_additive](variables.tf#L147) | Tenant-level custom IAM settings in role => [principal] format for non-authoritative bindings. | map(list(string)) | | {} | | -| [locations](variables.tf#L153) | Optional locations for GCS, BigQuery, and logging buckets created here. These are the defaults set at the organization level, and can be overridden via the tenant config variable. | object({…}) | | {…} | 0-bootstrap | -| [log_sinks](variables.tf#L173) | Tenant-level log sinks, in name => {type, filter} format. | map(object({…})) | | {…} | | -| [outputs_location](variables.tf#L204) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | string | | null | | -| [project_parent_ids](variables.tf#L220) | Optional parents for projects created here in folders/nnnnnnn format. Null values will use the tenant folder as parent. | object({…}) | | {…} | | -| [test_principal](variables.tf#L302) | Used when testing to bypass the data source returning the current identity. | string | | null | | +| [billing_account](variables.tf#L38) | Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`. | object({…}) | ✓ | | | +| [organization](variables.tf#L191) | Organization details. | object({…}) | ✓ | | 0-bootstrap | +| [prefix](variables.tf#L207) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 0-bootstrap | +| [tag_keys](variables.tf#L230) | Organization tag keys. | object({…}) | ✓ | | 1-resman | +| [tag_names](variables.tf#L241) | Customized names for resource management tags. | object({…}) | ✓ | | 1-resman | +| [tag_values](variables.tf#L252) | Organization resource management tag values. | map(string) | ✓ | | 1-resman | +| [tenant_config](variables.tf#L259) | Tenant configuration. Short name must be 4 characters or less. | object({…}) | ✓ | | | +| [cicd_repositories](variables.tf#L48) | CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | object({…}) | | null | | +| [custom_roles](variables.tf#L94) | Custom roles defined at the organization level, in key => id format. | object({…}) | | null | 0-bootstrap | +| [fast_features](variables.tf#L104) | Selective control for top-level FAST features. | object({…}) | | {} | 0-bootstrap | +| [federated_identity_providers](variables.tf#L118) | Workload Identity Federation pools. The `cicd_repositories` variable references keys here. | map(object({…})) | | {} | | +| [group_iam](variables.tf#L132) | Tenant-level custom group IAM settings in group => [roles] format. | map(list(string)) | | {} | | +| [iam](variables.tf#L138) | Tenant-level custom IAM settings in role => [principal] format. | map(list(string)) | | {} | | +| [iam_additive](variables.tf#L144) | Tenant-level custom IAM settings in role => [principal] format for non-authoritative bindings. | map(list(string)) | | {} | | +| [locations](variables.tf#L150) | Optional locations for GCS, BigQuery, and logging buckets created here. These are the defaults set at the organization level, and can be overridden via the tenant config variable. | object({…}) | | {…} | 0-bootstrap | +| [log_sinks](variables.tf#L170) | Tenant-level log sinks, in name => {type, filter} format. | map(object({…})) | | {…} | | +| [outputs_location](variables.tf#L201) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | string | | null | | +| [project_parent_ids](variables.tf#L217) | Optional parents for projects created here in folders/nnnnnnn format. Null values will use the tenant folder as parent. | object({…}) | | {…} | | +| [test_principal](variables.tf#L299) | Used when testing to bypass the data source returning the current identity. | string | | null | | ## Outputs diff --git a/fast/stages-multitenant/0-bootstrap-tenant/automation-sas.tf b/fast/stages-multitenant/0-bootstrap-tenant/automation-sas.tf index cae14093e..453b6b8d1 100644 --- a/fast/stages-multitenant/0-bootstrap-tenant/automation-sas.tf +++ b/fast/stages-multitenant/0-bootstrap-tenant/automation-sas.tf @@ -93,12 +93,12 @@ module "automation-tf-resman-sa-stage2-3" { name = "${each.key}-0" display_name = "Terraform ${each.value.description} service account." prefix = local.prefix - iam_billing_roles = !var.billing_account.is_org_level ? { + iam_billing_roles = local.billing_mode == "resource" ? { (var.billing_account.id) = [ "roles/billing.user", "roles/billing.costsManager" ] } : {} - iam_organization_roles = var.billing_account.is_org_level ? { + iam_organization_roles = local.billing_mode == "org" ? { (var.organization.id) = [ "roles/billing.user", "roles/billing.costsManager" ] diff --git a/fast/stages-multitenant/0-bootstrap-tenant/automation.tf b/fast/stages-multitenant/0-bootstrap-tenant/automation.tf index 9684e7ca3..145210dd2 100644 --- a/fast/stages-multitenant/0-bootstrap-tenant/automation.tf +++ b/fast/stages-multitenant/0-bootstrap-tenant/automation.tf @@ -125,12 +125,12 @@ module "automation-tf-resman-sa" { try(module.automation-tf-cicd-sa-resman["0"].iam_email, null) ]) } - iam_billing_roles = !var.billing_account.is_org_level ? { + iam_billing_roles = local.billing_mode == "resource" ? { (var.billing_account.id) = [ "roles/billing.admin", "roles/billing.costsManager" ] } : {} - iam_organization_roles = var.billing_account.is_org_level ? { + iam_organization_roles = local.billing_mode == "org" ? { (var.organization.id) = [ "roles/billing.admin", "roles/billing.costsManager" ] diff --git a/fast/stages-multitenant/0-bootstrap-tenant/billing.tf b/fast/stages-multitenant/0-bootstrap-tenant/billing.tf index 77c26b919..b62b79cbc 100644 --- a/fast/stages-multitenant/0-bootstrap-tenant/billing.tf +++ b/fast/stages-multitenant/0-bootstrap-tenant/billing.tf @@ -16,23 +16,39 @@ # tfdoc:file:description Billing roles for standalone billing accounts. +locals { + billing_mode = ( + var.billing_account.no_iam + ? null + : var.billing_account.is_org_level ? "org" : "resource" + ) +} + # service account billing roles are in the SA module in automation.tf resource "google_billing_account_iam_member" "billing_ext_admin" { - for_each = toset(var.billing_account.is_org_level ? [] : [ - "group:${local.groups.gcp-admins}", - module.automation-tf-resman-sa.iam_email - ]) + for_each = toset( + local.billing_mode == "resource" + ? [ + "group:${local.groups.gcp-admins}", + module.automation-tf-resman-sa.iam_email + ] + : [] + ) billing_account_id = var.billing_account.id role = "roles/billing.admin" member = each.key } resource "google_billing_account_iam_member" "billing_ext_cost_manager" { - for_each = toset(var.billing_account.is_org_level ? [] : [ - "group:${local.groups.gcp-admins}", - module.automation-tf-resman-sa.iam_email - ]) + for_each = toset( + local.billing_mode == "resource" + ? [ + "group:${local.groups.gcp-admins}", + module.automation-tf-resman-sa.iam_email + ] + : [] + ) billing_account_id = var.billing_account.id role = "roles/billing.costsManager" member = each.key diff --git a/fast/stages-multitenant/0-bootstrap-tenant/organization.tf b/fast/stages-multitenant/0-bootstrap-tenant/organization.tf index 46f8c0d4f..f08f0fbae 100644 --- a/fast/stages-multitenant/0-bootstrap-tenant/organization.tf +++ b/fast/stages-multitenant/0-bootstrap-tenant/organization.tf @@ -32,7 +32,7 @@ module "organization" { "group:${local.groups.gcp-admins}" ] }, - var.billing_account.is_org_level ? { + local.billing_mode == "org" ? { "roles/billing.admin" = [ "group:${local.groups.gcp-admins}", module.automation-tf-resman-sa.iam_email diff --git a/fast/stages-multitenant/0-bootstrap-tenant/variables.tf b/fast/stages-multitenant/0-bootstrap-tenant/variables.tf index d218986fd..aa90f400f 100644 --- a/fast/stages-multitenant/0-bootstrap-tenant/variables.tf +++ b/fast/stages-multitenant/0-bootstrap-tenant/variables.tf @@ -36,16 +36,13 @@ variable "automation" { } variable "billing_account" { - # tfdoc:variable:source 0-bootstrap - description = "Billing account id. If billing account is not part of the same org set `is_org_level` to false." + description = "Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`." type = object({ id = string is_org_level = optional(bool, true) + no_iam = optional(bool, false) }) - validation { - condition = var.billing_account.is_org_level != null - error_message = "Invalid `null` value for `billing_account.is_org_level`." - } + nullable = false } variable "cicd_repositories" { diff --git a/fast/stages-multitenant/1-resman-tenant/README.md b/fast/stages-multitenant/1-resman-tenant/README.md index ae42fc305..8ff59c254 100644 --- a/fast/stages-multitenant/1-resman-tenant/README.md +++ b/fast/stages-multitenant/1-resman-tenant/README.md @@ -149,22 +149,22 @@ Once the configuration is done just go through the usual `init/apply` cycle. On | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| | [automation](variables.tf#L20) | Automation resources created by the bootstrap stage. | object({…}) | ✓ | | 0-bootstrap | -| [billing_account](variables.tf#L51) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | object({…}) | ✓ | | 0-bootstrap | -| [organization](variables.tf#L206) | Organization details. | object({…}) | ✓ | | 0-bootstrap | -| [prefix](variables.tf#L228) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 0-bootstrap | -| [root_node](variables.tf#L239) | Root folder node for the tenant, in folders/nnnnnn format. | string | ✓ | | | -| [short_name](variables.tf#L244) | Short name used to identify the tenant. | string | ✓ | | | -| [tags](variables.tf#L249) | Resource management tags. | object({…}) | ✓ | | | -| [cicd_repositories](variables.tf#L64) | CI/CD repository configuration. Identity providers reference keys in the `automation.federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | object({…}) | | null | | -| [custom_roles](variables.tf#L146) | Custom roles defined at the org level, in key => id format. | object({…}) | | null | 0-bootstrap | -| [data_dir](variables.tf#L155) | Relative path for the folder storing configuration data. | string | | "data" | | -| [fast_features](variables.tf#L161) | Selective control for top-level FAST features. | object({…}) | | {} | 0-0-bootstrap | -| [groups](variables.tf#L175) | Group names to grant organization-level permissions. | object({…}) | | {} | 0-bootstrap | -| [locations](variables.tf#L188) | Optional locations for GCS, BigQuery, and logging buckets created here. | object({…}) | | {…} | 0-bootstrap | -| [organization_policy_data_path](variables.tf#L216) | Path for the data folder used by the organization policies factory. | string | | null | | -| [outputs_location](variables.tf#L222) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | string | | null | | -| [team_folders](variables.tf#L267) | Team folders to be created. Format is described in a code comment. | map(object({…})) | | null | | -| [test_skip_data_sources](variables.tf#L277) | Used when testing to bypass data sources. | bool | | false | | +| [billing_account](variables.tf#L51) | Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`. | object({…}) | ✓ | | 0-bootstrap | +| [organization](variables.tf#L204) | Organization details. | object({…}) | ✓ | | 0-bootstrap | +| [prefix](variables.tf#L226) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 0-bootstrap | +| [root_node](variables.tf#L237) | Root folder node for the tenant, in folders/nnnnnn format. | string | ✓ | | | +| [short_name](variables.tf#L242) | Short name used to identify the tenant. | string | ✓ | | | +| [tags](variables.tf#L247) | Resource management tags. | object({…}) | ✓ | | | +| [cicd_repositories](variables.tf#L62) | CI/CD repository configuration. Identity providers reference keys in the `automation.federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | object({…}) | | null | | +| [custom_roles](variables.tf#L144) | Custom roles defined at the org level, in key => id format. | object({…}) | | null | 0-bootstrap | +| [data_dir](variables.tf#L153) | Relative path for the folder storing configuration data. | string | | "data" | | +| [fast_features](variables.tf#L159) | Selective control for top-level FAST features. | object({…}) | | {} | 0-0-bootstrap | +| [groups](variables.tf#L173) | Group names to grant organization-level permissions. | object({…}) | | {} | 0-bootstrap | +| [locations](variables.tf#L186) | Optional locations for GCS, BigQuery, and logging buckets created here. | object({…}) | | {…} | 0-bootstrap | +| [organization_policy_data_path](variables.tf#L214) | Path for the data folder used by the organization policies factory. | string | | null | | +| [outputs_location](variables.tf#L220) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | string | | null | | +| [team_folders](variables.tf#L265) | Team folders to be created. Format is described in a code comment. | map(object({…})) | | null | | +| [test_skip_data_sources](variables.tf#L275) | Used when testing to bypass data sources. | bool | | false | | ## Outputs diff --git a/fast/stages-multitenant/1-resman-tenant/variables.tf b/fast/stages-multitenant/1-resman-tenant/variables.tf index 0229dd78f..1698b7e15 100644 --- a/fast/stages-multitenant/1-resman-tenant/variables.tf +++ b/fast/stages-multitenant/1-resman-tenant/variables.tf @@ -50,15 +50,13 @@ variable "automation" { variable "billing_account" { # tfdoc:variable:source 0-bootstrap - description = "Billing account id. If billing account is not part of the same org set `is_org_level` to false." + description = "Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`." type = object({ id = string is_org_level = optional(bool, true) + no_iam = optional(bool, false) }) - validation { - condition = var.billing_account.is_org_level != null - error_message = "Invalid `null` value for `billing_account.is_org_level`." - } + nullable = false } variable "cicd_repositories" { diff --git a/fast/stages/0-bootstrap/README.md b/fast/stages/0-bootstrap/README.md index 20ed998f2..af02d1b9a 100644 --- a/fast/stages/0-bootstrap/README.md +++ b/fast/stages/0-bootstrap/README.md @@ -481,16 +481,7 @@ The remaining configuration is manual, as it regards the repositories themselves | name | description | modules | resources | |---|---|---|---| | [automation.tf](./automation.tf) | Automation project and resources. | gcs · iam-service-account · project | | -| [billing.tf](./billing.tf) | Billing export project and dataset. | bigquery-dataset · project | - ) -} - -# billing account in same org (IAM is in the organization.tf file) - -module · ? local.billing_ext_admins : [] - ) - billing_account_id = var.billing_account.id - role = · google_billing_account_iam_member | +| [billing.tf](./billing.tf) | Billing export project and dataset. | bigquery-dataset · project | google_billing_account_iam_member | | [cicd.tf](./cicd.tf) | Workload Identity Federation configurations for CI/CD. | iam-service-account · source-repository | | | [identity-providers.tf](./identity-providers.tf) | Workload Identity Federation provider definitions. | | google_iam_workload_identity_pool · google_iam_workload_identity_pool_provider | | [log-export.tf](./log-export.tf) | Audit log project and sink. | bigquery-dataset · gcs · logging-bucket · project · pubsub | | diff --git a/fast/stages/1-resman/README.md b/fast/stages/1-resman/README.md index 781dc14d3..efa338959 100644 --- a/fast/stages/1-resman/README.md +++ b/fast/stages/1-resman/README.md @@ -174,18 +174,7 @@ Due to its simplicity, this stage lends itself easily to customizations: adding | name | description | modules | resources | |---|---|---|---| -| [billing.tf](./billing.tf) | Billing resources for external billing use cases. | | - ) -} - -# billing account in same org (resources is in the organization.tf file) - -# standalone billing account - -resource · ? local.billing_ext_users : [] - ) - billing_account_id = var.billing_account.id - role = · google_billing_account_iam_member | +| [billing.tf](./billing.tf) | Billing resources for external billing use cases. | | google_billing_account_iam_member | | [branch-data-platform.tf](./branch-data-platform.tf) | Data Platform stages resources. | folder · gcs · iam-service-account | google_organization_iam_member | | [branch-gke.tf](./branch-gke.tf) | GKE multitenant stage resources. | folder · gcs · iam-service-account | | | [branch-networking.tf](./branch-networking.tf) | Networking stage resources. | folder · gcs · iam-service-account | | diff --git a/tools/tfdoc.py b/tools/tfdoc.py index d06dedb9e..a69d94abf 100755 --- a/tools/tfdoc.py +++ b/tools/tfdoc.py @@ -60,7 +60,7 @@ FILE_DESC_DEFAULTS = { } FILE_RE_MODULES = re.compile( r'(?sm)module\s*"[^"]+"\s*\{[^\}]*?source\s*=\s*"([^"]+)"') -FILE_RE_RESOURCES = re.compile(r'(?sm)resource\s*"([^"]+)"') +FILE_RE_RESOURCES = re.compile(r'(?sm)resource\s+"([^"]+)"') HEREDOC_RE = re.compile(r'(?sm)^<<\-?END(\s*.*?)\s*END$') MARK_BEGIN = '' MARK_END = ''