diff --git a/fast/README.md b/fast/README.md index 6652465ee..35255fba1 100644 --- a/fast/README.md +++ b/fast/README.md @@ -32,7 +32,7 @@ FAST also aims to minimize the number of permissions granted to principals accor A resource factory consumes a simple representation of a resource (e.g., in YAML) and deploys it (e.g., using Terraform). Used correctly, factories can help decrease the management overhead of large-scale infrastructure deployments. See "[Resource Factories: A descriptive approach to Terraform](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c)" for more details and the rationale behind factories. -FAST uses YAML-based factories to deploy subnets and firewall rules and, as its name suggests, in the [project factory](./stages/3-project-factory/) stage. +FAST uses YAML-based factories to deploy subnets and firewall rules and, as its name suggests, in the [project factory](./stages/2-project-factory/) stage. ### CI/CD diff --git a/fast/stage-links.sh b/fast/stage-links.sh index b690066d3..1363f5f43 100755 --- a/fast/stage-links.sh +++ b/fast/stage-links.sh @@ -45,6 +45,7 @@ fi GLOBALS="tfvars/0-globals.auto.tfvars.json" PROVIDER_CMD=$CMD STAGE_NAME=$(basename "$(pwd)") +EXTRA_FILES="" case $STAGE_NAME in @@ -74,6 +75,21 @@ case $STAGE_NAME in tenants/$TENANT/tfvars/1-resman.auto.tfvars.json" fi ;; +"2-project-factory"*) + if [[ -z "$TENANT" ]]; then + echo "# if this is a tenant stage, set a \$TENANT variable with the tenant shortname and run the command again" + PROVIDER="providers/2-project-factory-providers.tf" + TFVARS="tfvars/0-bootstrap.auto.tfvars.json + tfvars/1-resman.auto.tfvars.json" + EXTRA_FILES="tfvars/2-networking.auto.tfvars.json" + else + unset GLOBALS + PROVIDER="tenants/$TENANT/providers/2-project-factory-providers.tf" + TFVARS="tenants/$TENANT/tfvars/0-bootstrap-tenant.auto.tfvars.json + tenants/$TENANT/tfvars/1-resman.auto.tfvars.json" + EXTRA_FILES="tenants/$TENANT/tfvars/2-networking.auto.tfvars.json" + fi + ;; "2-security"*) if [[ -z "$TENANT" ]]; then echo "# if this is a tenant stage, set a \$TENANT variable with the tenant shortname and run the command again" @@ -138,6 +154,13 @@ for f in $TFVARS; do echo "$CMD/$f ./" done +if [[ ! -z ${EXTRA_FILES+x} ]]; then + echo "# optional files" + for f in $EXTRA_FILES; do + echo "$CMD/$f ./" + done +fi + if [[ ! -z ${MESSAGE+x} ]]; then echo -e "\n# ---> $MESSAGE <---" fi diff --git a/fast/stages/1-resman/README.md b/fast/stages/1-resman/README.md index 3e9ad6dc3..533b66b4f 100644 --- a/fast/stages/1-resman/README.md +++ b/fast/stages/1-resman/README.md @@ -60,13 +60,13 @@ For a discussion on naming, please refer to the [Bootstrap stage documentation]( Top-level folders for teams or departments can be easily created via the `top_level_folders` variable or the associated factory, which expose the full power of the underlying [folder module](../../../modules/folder/). -The suggestion is to use this feature sparingly so at to keep the top level of the hierarchy simple, and minimize changes to this stage due to its security implications. One approach is to create a grouping folder (e.g. `Departments` or `Teams`) here, and delegate management of lower level folders to the [project factory](../3-project-factory/) stage. +The suggestion is to use this feature sparingly so as to keep the top level of the hierarchy simple, and minimize changes to this stage due to its security implications. One approach is to create a grouping folder (e.g. `Departments` or `Teams`) here, and delegate management of lower level folders to the [project factory](../2-project-factory/) stage. Top-level folders also support defining associated resources for automation, and auto-created provider files to bootstrap Infrastructure and Code. An example is provided below. ### Multitenancy -Multitenancy is supported via a [separate stage](../1-tenant-factory/), which is entirely optional and can be applied after resource management has been deployed. For simpler use cases that do not require complex organization-level multitenancy, [top-level folders](#top-level-folders) can be used in combination with the [project factory stage](../3-project-factory/) support for folder and project management. +Multitenancy is supported via a [separate stage](../1-tenant-factory/), which is entirely optional and can be applied after resource management has been deployed. For simpler use cases that do not require complex organization-level multitenancy, [top-level folders](#top-level-folders) can be used in combination with the [project factory stage](../2-project-factory/) support for folder and project management. ### Workload Identity Federation and CI/CD @@ -148,7 +148,7 @@ The `fast_features` variable consists of 5 toggles: - **`data_platform`** controls the creation of required resources (folders, service accounts, buckets, IAM bindings) to deploy the [3-data-platform](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/fast/stages/3-data-platform) stage - **`gcve`** controls the creation of required resources (folders, service accounts, buckets, IAM bindings) to deploy the [3-gcve](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/fast/stages/3-gcve) stage - **`gke`** controls the creation of required resources (folders, service accounts, buckets, IAM bindings) to deploy the [3-gke-multitenant](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/fast/stages/3-gke-multitenant) stage -- **`project_factory`** controls the creation of required resources (folders, service accounts, buckets, IAM bindings) to deploy the [3-project-factory](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/fast/stages/3-project-factory) stage +- **`project_factory`** controls the creation of required resources (folders, service accounts, buckets, IAM bindings) to deploy the [2-project-factory](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/fast/stages/2-project-factory) stage - **`sandbox`** controls the creation of a "Sandbox" top level folder with relaxed policies, intended for sandbox environments where users can experiment - **`teams`** controls the creation of the top level "Teams" folder used by the [teams feature in resman](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/tree/master/fast/stages/1-resman#team-folders). @@ -272,30 +272,30 @@ A full reference of IAM roles managed by this stage [is available here](./IAM.md | [prefix](variables-fast.tf#L126) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 0-bootstrap | | [cicd_repositories](variables.tf#L20) | CI/CD repository configuration. Identity providers reference keys in the `automation.federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | object({…}) | | null | | | [custom_roles](variables-fast.tf#L53) | Custom roles defined at the org level, in key => id format. | object({…}) | | null | 0-bootstrap | -| [factories_config](variables.tf#L122) | Configuration for the resource factories or external data. | object({…}) | | {} | | -| [fast_features](variables.tf#L133) | Selective control for top-level FAST features. | object({…}) | | {} | | -| [folder_iam](variables.tf#L146) | Authoritative IAM for top-level folders. | object({…}) | | {} | | +| [factories_config](variables.tf#L122) | Configuration for the resource factories or external data. | object({…}) | | {} | | +| [fast_features](variables.tf#L133) | Selective control for top-level FAST features. | object({…}) | | {} | | +| [folder_iam](variables.tf#L145) | Authoritative IAM for top-level folders. | object({…}) | | {} | | | [groups](variables-fast.tf#L67) | Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated. | object({…}) | | {} | 0-bootstrap | | [locations](variables-fast.tf#L82) | Optional locations for GCS, BigQuery, and logging buckets created here. | object({…}) | | {} | 0-bootstrap | -| [outputs_location](variables.tf#L160) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | string | | null | | +| [outputs_location](variables.tf#L159) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | string | | null | | | [root_node](variables-fast.tf#L132) | Root node for the hierarchy, if running in tenant mode. | string | | null | 0-bootstrap | -| [tag_names](variables.tf#L166) | Customized names for resource management tags. | object({…}) | | {} | | -| [tags](variables.tf#L180) | Custom secure tags by key name. The `iam` attribute behaves like the similarly named one at module level. | map(object({…})) | | {} | | -| [top_level_folders](variables.tf#L201) | Additional top-level folders. Keys are used for service account and bucket names, values implement the folders module interface with the addition of the 'automation' attribute. | map(object({…})) | | {} | | +| [tag_names](variables.tf#L165) | Customized names for resource management tags. | object({…}) | | {} | | +| [tags](variables.tf#L179) | Custom secure tags by key name. The `iam` attribute behaves like the similarly named one at module level. | map(object({…})) | | {} | | +| [top_level_folders](variables.tf#L200) | Additional top-level folders. Keys are used for service account and bucket names, values implement the folders module interface with the addition of the 'automation' attribute. | map(object({…})) | | {} | | ## Outputs | name | description | sensitive | consumers | |---|---|:---:|---| -| [cicd_repositories](outputs.tf#L402) | WIF configuration for CI/CD repositories. | | | -| [dataplatform](outputs.tf#L416) | Data for the Data Platform stage. | | | -| [folder_ids](outputs.tf#L432) | Folder ids. | | | -| [gcve](outputs.tf#L437) | Data for the GCVE stage. | | 03-gcve | -| [gke_multitenant](outputs.tf#L458) | Data for the GKE multitenant stage. | | 03-gke-multitenant | -| [networking](outputs.tf#L479) | Data for the networking stage. | | | -| [project_factories](outputs.tf#L488) | Data for the project factories stage. | | | -| [providers](outputs.tf#L507) | Terraform provider files for this stage and dependent stages. | ✓ | 02-networking · 02-security · 03-dataplatform · 03-network-security | -| [sandbox](outputs.tf#L514) | Data for the sandbox stage. | | xx-sandbox | -| [security](outputs.tf#L528) | Data for the networking stage. | | 02-security | -| [tfvars](outputs.tf#L539) | Terraform variable files for the following stages. | ✓ | | +| [cicd_repositories](outputs.tf#L374) | WIF configuration for CI/CD repositories. | | | +| [dataplatform](outputs.tf#L388) | Data for the Data Platform stage. | | | +| [folder_ids](outputs.tf#L404) | Folder ids. | | | +| [gcve](outputs.tf#L409) | Data for the GCVE stage. | | 03-gcve | +| [gke_multitenant](outputs.tf#L430) | Data for the GKE multitenant stage. | | 03-gke-multitenant | +| [networking](outputs.tf#L451) | Data for the networking stage. | | | +| [project_factories](outputs.tf#L460) | Data for the project factories stage. | | | +| [providers](outputs.tf#L479) | Terraform provider files for this stage and dependent stages. | ✓ | 02-networking · 02-security · 03-dataplatform · 03-network-security | +| [sandbox](outputs.tf#L486) | Data for the sandbox stage. | | xx-sandbox | +| [security](outputs.tf#L500) | Data for the networking stage. | | 02-security | +| [tfvars](outputs.tf#L511) | Terraform variable files for the following stages. | ✓ | | diff --git a/fast/stages/1-resman/billing.tf b/fast/stages/1-resman/billing.tf index afa6958dc..c289ddfec 100644 --- a/fast/stages/1-resman/billing.tf +++ b/fast/stages/1-resman/billing.tf @@ -18,21 +18,19 @@ locals { # used here for convenience, in organization.tf members are explicit - billing_ext_users = concat( - [ - module.branch-network-sa.iam_email, - module.branch-security-sa.iam_email, - ], - local.branch_optional_sa_lists.dp-dev, - local.branch_optional_sa_lists.dp-prod, - local.branch_optional_sa_lists.gke-dev, - local.branch_optional_sa_lists.gke-prod, - local.branch_optional_sa_lists.gcve-dev, - local.branch_optional_sa_lists.gcve-prod, - local.branch_optional_sa_lists.pf, - local.branch_optional_sa_lists.pf-dev, - local.branch_optional_sa_lists.pf-prod, - ) + billing_ext_users = compact([ + try(module.branch-network-sa.iam_email, null), + try(module.branch-pf-dev-sa.iam_email, null), + try(module.branch-pf-prod-sa.iam_email, null), + try(module.branch-pf-sa.iam_email, null), + try(module.branch-security-sa.iam_email, null), + try(module.branch-dp-dev-sa[0].iam_email, null), + try(module.branch-dp-prod-sa[0].iam_email, null), + try(module.branch-gcve-dev-sa[0].iam_email, null), + try(module.branch-gcve-prod-sa[0].iam_email, null), + try(module.branch-gke-dev-sa[0].iam_email, null), + try(module.branch-gke-prod-sa[0].iam_email, null) + ]) billing_mode = ( var.billing_account.no_iam ? null diff --git a/fast/stages/1-resman/branch-networking.tf b/fast/stages/1-resman/branch-networking.tf index 263d1c4ae..ca2791d70 100644 --- a/fast/stages/1-resman/branch-networking.tf +++ b/fast/stages/1-resman/branch-networking.tf @@ -67,22 +67,24 @@ module "branch-network-prod-folder" { name = "Production" iam = { # read-write (apply) automation service accounts - (local.custom_roles.service_project_network_admin) = concat( - local.branch_optional_sa_lists.dp-prod, - local.branch_optional_sa_lists.gke-prod, - local.branch_optional_sa_lists.gcve-prod, - local.branch_optional_sa_lists.pf, - local.branch_optional_sa_lists.pf-prod, - ) + (local.custom_roles.service_project_network_admin) = compact([ + try(module.branch-dp-prod-sa[0].iam_email, null), + try(module.branch-gcve-prod-sa[0].iam_email, null), + try(module.branch-gke-prod-sa[0].iam_email, null), + try(module.branch-pf-sa.iam_email, null), + try(module.branch-pf-prod-sa.iam_email, null) + ]) # read-only (plan) automation service accounts - "roles/compute.networkViewer" = concat( - local.branch_optional_r_sa_lists.dp-prod, - local.branch_optional_r_sa_lists.gke-prod, - local.branch_optional_r_sa_lists.gcve-prod, - local.branch_optional_r_sa_lists.pf, - local.branch_optional_r_sa_lists.pf-prod, - ) - (local.custom_roles.gcve_network_admin) = local.branch_optional_sa_lists.gcve-prod + "roles/compute.networkViewer" = compact([ + try(module.branch-dp-prod-r-sa[0].iam_email, null), + try(module.branch-gcve-prod-r-sa[0].iam_email, null), + try(module.branch-gke-prod-r-sa[0].iam_email, null), + try(module.branch-pf-r-sa.iam_email, null), + try(module.branch-pf-prod-r-sa.iam_email, null) + ]) + (local.custom_roles.gcve_network_admin) = compact([ + try(module.branch-gcve-prod-sa[0].iam_email, null) + ]) } tag_bindings = { environment = try( @@ -98,22 +100,24 @@ module "branch-network-dev-folder" { name = "Development" iam = { # read-write (apply) automation service accounts - (local.custom_roles.service_project_network_admin) = concat( - local.branch_optional_sa_lists.dp-dev, - local.branch_optional_sa_lists.gke-dev, - local.branch_optional_sa_lists.gcve-dev, - local.branch_optional_sa_lists.pf, - local.branch_optional_sa_lists.pf-dev, - ) + (local.custom_roles.service_project_network_admin) = compact([ + try(module.branch-dp-dev-sa[0].iam_email, null), + try(module.branch-gcve-dev-sa[0].iam_email, null), + try(module.branch-gke-dev-sa[0].iam_email, null), + try(module.branch-pf-sa.iam_email, null), + try(module.branch-pf-dev-sa.iam_email, null) + ]) # read-only (plan) automation service accounts - "roles/compute.networkViewer" = concat( - local.branch_optional_r_sa_lists.dp-dev, - local.branch_optional_r_sa_lists.gke-dev, - local.branch_optional_r_sa_lists.gcve-dev, - local.branch_optional_r_sa_lists.pf, - local.branch_optional_r_sa_lists.pf-dev, - ) - (local.custom_roles.gcve_network_admin) = local.branch_optional_sa_lists.gcve-dev + "roles/compute.networkViewer" = compact([ + try(module.branch-dp-dev-r-sa[0].iam_email, null), + try(module.branch-gcve-dev-r-sa[0].iam_email, null), + try(module.branch-gke-dev-r-sa[0].iam_email, null), + try(module.branch-pf-r-sa.iam_email, null), + try(module.branch-pf-dev-r-sa.iam_email, null) + ]) + (local.custom_roles.gcve_network_admin) = compact([ + try(module.branch-gcve-dev-sa[0].iam_email, null) + ]) } tag_bindings = { environment = try( diff --git a/fast/stages/1-resman/branch-project-factory.tf b/fast/stages/1-resman/branch-project-factory.tf index 519fa587b..0e6c6134c 100644 --- a/fast/stages/1-resman/branch-project-factory.tf +++ b/fast/stages/1-resman/branch-project-factory.tf @@ -18,9 +18,13 @@ # automation service accounts +moved { + from = module.branch-pf-sa[0] + to = module.branch-pf-sa +} + module "branch-pf-sa" { source = "../../../modules/iam-service-account" - count = var.fast_features.project_factory ? 1 : 0 project_id = var.automation.project_id name = "resman-pf-0" display_name = "Terraform project factory main service account." @@ -38,9 +42,13 @@ module "branch-pf-sa" { } } +moved { + from = module.branch-pf-dev-sa[0] + to = module.branch-pf-dev-sa +} + module "branch-pf-dev-sa" { source = "../../../modules/iam-service-account" - count = var.fast_features.project_factory ? 1 : 0 project_id = var.automation.project_id name = "dev-resman-pf-0" display_name = "Terraform project factory development service account." @@ -58,9 +66,13 @@ module "branch-pf-dev-sa" { } } +moved { + from = module.branch-pf-prod-sa[0] + to = module.branch-pf-prod-sa +} + module "branch-pf-prod-sa" { source = "../../../modules/iam-service-account" - count = var.fast_features.project_factory ? 1 : 0 project_id = var.automation.project_id name = "prod-resman-pf-0" display_name = "Terraform project factory production service account." @@ -80,9 +92,13 @@ module "branch-pf-prod-sa" { # automation read-only service accounts +moved { + from = module.branch-pf-r-sa[0] + to = module.branch-pf-r-sa +} + module "branch-pf-r-sa" { source = "../../../modules/iam-service-account" - count = var.fast_features.project_factory ? 1 : 0 project_id = var.automation.project_id name = "resman-pf-0r" display_name = "Terraform project factory main service account (read-only)." @@ -100,9 +116,13 @@ module "branch-pf-r-sa" { } } +moved { + from = module.branch-pf-dev-r-sa[0] + to = module.branch-pf-dev-r-sa +} + module "branch-pf-dev-r-sa" { source = "../../../modules/iam-service-account" - count = var.fast_features.project_factory ? 1 : 0 project_id = var.automation.project_id name = "dev-resman-pf-0r" display_name = "Terraform project factory development service account (read-only)." @@ -120,9 +140,13 @@ module "branch-pf-dev-r-sa" { } } +moved { + from = module.branch-pf-prod-r-sa[0] + to = module.branch-pf-prod-r-sa +} + module "branch-pf-prod-r-sa" { source = "../../../modules/iam-service-account" - count = var.fast_features.project_factory ? 1 : 0 project_id = var.automation.project_id name = "prod-resman-pf-0r" display_name = "Terraform project factory production service account (read-only)." @@ -142,9 +166,13 @@ module "branch-pf-prod-r-sa" { # automation buckets +moved { + from = module.branch-pf-gcs[0] + to = module.branch-pf-gcs +} + module "branch-pf-gcs" { source = "../../../modules/gcs" - count = var.fast_features.project_factory ? 1 : 0 project_id = var.automation.project_id name = "resman-pf-0" prefix = var.prefix @@ -152,14 +180,18 @@ module "branch-pf-gcs" { storage_class = local.gcs_storage_class versioning = true iam = { - "roles/storage.objectAdmin" = [module.branch-pf-sa[0].iam_email] - "roles/storage.objectViewer" = [module.branch-pf-r-sa[0].iam_email] + "roles/storage.objectAdmin" = [module.branch-pf-sa.iam_email] + "roles/storage.objectViewer" = [module.branch-pf-r-sa.iam_email] } } +moved { + from = module.branch-pf-dev-gcs[0] + to = module.branch-pf-dev-gcs +} + module "branch-pf-dev-gcs" { source = "../../../modules/gcs" - count = var.fast_features.project_factory ? 1 : 0 project_id = var.automation.project_id name = "dev-resman-pf-0" prefix = var.prefix @@ -167,14 +199,18 @@ module "branch-pf-dev-gcs" { storage_class = local.gcs_storage_class versioning = true iam = { - "roles/storage.objectAdmin" = [module.branch-pf-dev-sa[0].iam_email] - "roles/storage.objectViewer" = [module.branch-pf-dev-r-sa[0].iam_email] + "roles/storage.objectAdmin" = [module.branch-pf-dev-sa.iam_email] + "roles/storage.objectViewer" = [module.branch-pf-dev-r-sa.iam_email] } } +moved { + from = module.branch-pf-prod-gcs[0] + to = module.branch-pf-prod-gcs +} + module "branch-pf-prod-gcs" { source = "../../../modules/gcs" - count = var.fast_features.project_factory ? 1 : 0 project_id = var.automation.project_id name = "prod-resman-pf-0" prefix = var.prefix @@ -182,7 +218,7 @@ module "branch-pf-prod-gcs" { storage_class = local.gcs_storage_class versioning = true iam = { - "roles/storage.objectAdmin" = [module.branch-pf-prod-sa[0].iam_email] - "roles/storage.objectViewer" = [module.branch-pf-prod-r-sa[0].iam_email] + "roles/storage.objectAdmin" = [module.branch-pf-prod-sa.iam_email] + "roles/storage.objectViewer" = [module.branch-pf-prod-r-sa.iam_email] } } diff --git a/fast/stages/1-resman/data/top-level-folders/teams.yaml b/fast/stages/1-resman/data/top-level-folders/teams.yaml new file mode 100644 index 000000000..3695ce69d --- /dev/null +++ b/fast/stages/1-resman/data/top-level-folders/teams.yaml @@ -0,0 +1,32 @@ +# Copyright 2024 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# yaml-language-server: $schema=../../schemas/top-level-folder.schema.json + +name: Teams +automation: + enable: false +iam: + "roles/owner": + - project-factory + "roles/resourcemanager.folderAdmin": + - project-factory + "roles/resourcemanager.projectCreator": + - project-factory + "roles/resourcemanager.tagUser": + - project-factory + "service_project_network_admin": + - project-factory +tag_bindings: + context: context/project-factory diff --git a/fast/stages/1-resman/iam.tf b/fast/stages/1-resman/iam.tf index a5ff698c5..b396541ad 100644 --- a/fast/stages/1-resman/iam.tf +++ b/fast/stages/1-resman/iam.tf @@ -80,37 +80,37 @@ locals { } }, # optional billing roles for project factory - local.billing_mode != "org" || !var.fast_features.project_factory ? {} : { + local.billing_mode != "org" ? {} : { sa_pf_billing = { - member = module.branch-pf-sa[0].iam_email + member = module.branch-pf-sa.iam_email role = "roles/billing.user" } sa_pf_costs_manager = { - member = module.branch-pf-sa[0].iam_email + member = module.branch-pf-sa.iam_email role = "roles/billing.costsManager" } sa_pf_dev_billing = { - member = module.branch-pf-dev-sa[0].iam_email + member = module.branch-pf-dev-sa.iam_email role = "roles/billing.user" } sa_pf_dev_costs_manager = { - member = module.branch-pf-dev-sa[0].iam_email + member = module.branch-pf-dev-sa.iam_email role = "roles/billing.costsManager" } sa_pf_prod_billing = { - member = module.branch-pf-prod-sa[0].iam_email + member = module.branch-pf-prod-sa.iam_email role = "roles/billing.user" } sa_pf_prod_costs_manager = { - member = module.branch-pf-prod-sa[0].iam_email + member = module.branch-pf-prod-sa.iam_email role = "roles/billing.costsManager" } }, # scoped org policy admin grants for project factory # TODO: change to use context and environment tags, and tag bindings in stage 2s - !var.fast_features.project_factory || var.root_node != null ? {} : { + var.root_node != null ? {} : { sa_pf_conditional_org_policy = { - member = module.branch-pf-sa[0].iam_email + member = module.branch-pf-sa.iam_email role = "roles/orgpolicy.policyAdmin" condition = { title = "org_policy_tag_pf_scoped" @@ -121,23 +121,27 @@ locals { } } sa_pf_dev_conditional_org_policy = { - member = module.branch-pf-dev-sa[0].iam_email + member = module.branch-pf-dev-sa.iam_email role = "roles/orgpolicy.policyAdmin" condition = { title = "org_policy_tag_pf_scoped_dev" description = "Org policy tag scoped grant for project factory dev." expression = <<-END + resource.matchTag('${local.tag_root}/${var.tag_names.context}', 'project-factory') + && resource.matchTag('${local.tag_root}/${var.tag_names.environment}', 'development') END } } sa_pf_prod_conditional_org_policy = { - member = module.branch-pf-prod-sa[0].iam_email + member = module.branch-pf-prod-sa.iam_email role = "roles/orgpolicy.policyAdmin" condition = { title = "org_policy_tag_pf_scoped_prod" description = "Org policy tag scoped grant for project factory prod." expression = <<-END + resource.matchTag('${local.tag_root}/${var.tag_names.context}', 'project-factory') + && resource.matchTag('${local.tag_root}/${var.tag_names.environment}', 'production') END } diff --git a/fast/stages/1-resman/main.tf b/fast/stages/1-resman/main.tf index d94321acc..de21be519 100644 --- a/fast/stages/1-resman/main.tf +++ b/fast/stages/1-resman/main.tf @@ -16,33 +16,36 @@ locals { # leaving this here to document how to get self identity in a stage - # automation_resman_sa = try( # data.google_client_openid_userinfo.provider_identity[0].email, null # ) - - # service accounts that receive additional grants on networking/security - branch_optional_sa_lists = { - dp-dev = compact([try(module.branch-dp-dev-sa[0].iam_email, "")]) - dp-prod = compact([try(module.branch-dp-prod-sa[0].iam_email, "")]) - gcve-dev = compact([try(module.branch-gcve-dev-sa[0].iam_email, "")]) - gcve-prod = compact([try(module.branch-gcve-prod-sa[0].iam_email, "")]) - gke-dev = compact([try(module.branch-gke-dev-sa[0].iam_email, "")]) - gke-prod = compact([try(module.branch-gke-prod-sa[0].iam_email, "")]) - pf = compact([try(module.branch-pf-sa[0].iam_email, "")]) - pf-dev = compact([try(module.branch-pf-dev-sa[0].iam_email, "")]) - pf-prod = compact([try(module.branch-pf-prod-sa[0].iam_email, "")]) - } - branch_optional_r_sa_lists = { - dp-dev = compact([try(module.branch-dp-dev-r-sa[0].iam_email, "")]) - dp-prod = compact([try(module.branch-dp-prod-r-sa[0].iam_email, "")]) - gcve-dev = compact([try(module.branch-gcve-dev-r-sa[0].iam_email, "")]) - gcve-prod = compact([try(module.branch-gcve-prod-r-sa[0].iam_email, "")]) - gke-dev = compact([try(module.branch-gke-dev-r-sa[0].iam_email, "")]) - gke-prod = compact([try(module.branch-gke-prod-r-sa[0].iam_email, "")]) - pf = compact([try(module.branch-pf-r-sa[0].iam_email, "")]) - pf-dev = compact([try(module.branch-pf-dev-r-sa[0].iam_email, "")]) - pf-prod = compact([try(module.branch-pf-prod-r-sa[0].iam_email, "")]) + # stage service accounts, used in top folders and outputs + branch_service_accounts = { + data-platform-dev = try(module.branch-dp-dev-sa[0].email, null) + data-platform-dev-r = try(module.branch-dp-dev-r-sa[0].email, null) + data-platform-prod = try(module.branch-dp-prod-sa[0].email, null) + data-platform-prod-r = try(module.branch-dp-prod-r-sa[0].email, null) + gcve-dev = try(module.branch-gcve-dev-sa[0].email, null) + gcve-dev-r = try(module.branch-gcve-dev-r-sa[0].email, null) + gcve-prod = try(module.branch-gcve-prod-sa[0].email, null) + gcve-prod-r = try(module.branch-gcve-prod-r-sa[0].email, null) + gke-dev = try(module.branch-gke-dev-sa[0].email, null) + gke-dev-r = try(module.branch-gke-dev-r-sa[0].email, null) + gke-prod = try(module.branch-gke-prod-sa[0].email, null) + gke-prod-r = try(module.branch-gke-prod-r-sa[0].email, null) + nsec = module.branch-nsec-sa.email + nsec-r = module.branch-nsec-r-sa.email + networking = module.branch-network-sa.email + networking-r = module.branch-network-r-sa.email + project-factory = module.branch-pf-sa.email + project-factory-r = module.branch-pf-r-sa.email + project-factory-dev = module.branch-pf-dev-sa.email + project-factory-dev-r = module.branch-pf-dev-r-sa.email + project-factory-prod = module.branch-pf-prod-sa.email + project-factory-prod-r = module.branch-pf-prod-r-sa.email + sandbox = try(module.branch-sandbox-sa[0].email, null) + security = module.branch-security-sa.email + security-r = module.branch-security-r-sa.email } # normalize CI/CD repositories cicd_repositories = { diff --git a/fast/stages/1-resman/organization.tf b/fast/stages/1-resman/organization.tf index f5ec5c765..d6d04d370 100644 --- a/fast/stages/1-resman/organization.tf +++ b/fast/stages/1-resman/organization.tf @@ -84,11 +84,23 @@ module "organization" { iam = try(local.tags.environment.iam, {}) values = { development = { - iam = try(local.tags.environment.values.development.iam, {}) + iam = try(local.tags.environment.values.development.iam, {}) + iam_bindings = { + pf = { + members = [module.branch-pf-sa.iam_email] + role = "roles/resourcemanager.tagUser" + } + } description = try(local.tags.environment.values.development.description, null) } production = { - iam = try(local.tags.environment.values.production.iam, {}) + iam = try(local.tags.environment.values.production.iam, {}) + iam_bindings = { + pf = { + members = [module.branch-pf-sa.iam_email] + role = "roles/resourcemanager.tagUser" + } + } description = try(local.tags.environment.values.production.description, null) } } diff --git a/fast/stages/1-resman/outputs.tf b/fast/stages/1-resman/outputs.tf index 4dac75cef..a552c05ba 100644 --- a/fast/stages/1-resman/outputs.tf +++ b/fast/stages/1-resman/outputs.tf @@ -197,6 +197,42 @@ locals { name = "networking" sa = module.branch-network-r-sa.email }) + "2-project-factory" = templatefile(local._tpl_providers, { + backend_extra = null + bucket = module.branch-pf-gcs.name + name = "project-factory" + sa = module.branch-pf-sa.email + }) + "2-project-factory-r" = templatefile(local._tpl_providers, { + backend_extra = null + bucket = module.branch-pf-gcs.name + name = "project-factory" + sa = module.branch-pf-r-sa.email + }) + "2-project-factory-dev" = templatefile(local._tpl_providers, { + backend_extra = null + bucket = module.branch-pf-dev-gcs.name + name = "project-factory-dev" + sa = module.branch-pf-dev-sa.email + }) + "2-project-factory-dev-r" = templatefile(local._tpl_providers, { + backend_extra = null + bucket = module.branch-pf-dev-gcs.name + name = "project-factory-dev" + sa = module.branch-pf-dev-r-sa.email + }) + "2-project-factory-prod" = templatefile(local._tpl_providers, { + backend_extra = null + bucket = module.branch-pf-prod-gcs.name + name = "project-factory-prod" + sa = module.branch-pf-prod-sa.email + }) + "2-project-factory-prod-r" = templatefile(local._tpl_providers, { + backend_extra = null + bucket = module.branch-pf-prod-gcs.name + name = "project-factory-prod" + sa = module.branch-pf-prod-r-sa.email + }) "2-security" = templatefile(local._tpl_providers, { backend_extra = null bucket = module.branch-security-gcs.name @@ -309,44 +345,6 @@ locals { sa = module.branch-gcve-prod-r-sa[0].email }) }, - !var.fast_features.project_factory ? {} : { - "3-project-factory" = templatefile(local._tpl_providers, { - backend_extra = null - bucket = module.branch-pf-gcs[0].name - name = "project-factory" - sa = module.branch-pf-sa[0].email - }) - "3-project-factory-r" = templatefile(local._tpl_providers, { - backend_extra = null - bucket = module.branch-pf-gcs[0].name - name = "project-factory" - sa = module.branch-pf-r-sa[0].email - }) - "3-project-factory-dev" = templatefile(local._tpl_providers, { - backend_extra = null - bucket = module.branch-pf-dev-gcs[0].name - name = "project-factory-dev" - sa = module.branch-pf-dev-sa[0].email - }) - "3-project-factory-dev-r" = templatefile(local._tpl_providers, { - backend_extra = null - bucket = module.branch-pf-dev-gcs[0].name - name = "project-factory-dev" - sa = module.branch-pf-dev-r-sa[0].email - }) - "3-project-factory-prod" = templatefile(local._tpl_providers, { - backend_extra = null - bucket = module.branch-pf-prod-gcs[0].name - name = "project-factory-prod" - sa = module.branch-pf-prod-sa[0].email - }) - "3-project-factory-prod-r" = templatefile(local._tpl_providers, { - backend_extra = null - bucket = module.branch-pf-prod-gcs[0].name - name = "project-factory-prod" - sa = module.branch-pf-prod-r-sa[0].email - }) - }, !var.fast_features.sandbox ? {} : { "9-sandbox" = templatefile(local._tpl_providers, { backend_extra = null @@ -357,33 +355,7 @@ locals { }, ) service_accounts = merge( - { - data-platform-dev = try(module.branch-dp-dev-sa[0].email, null) - data-platform-dev-r = try(module.branch-dp-dev-r-sa[0].email, null) - data-platform-prod = try(module.branch-dp-prod-sa[0].email, null) - data-platform-prod-r = try(module.branch-dp-prod-r-sa[0].email, null) - gcve-dev = try(module.branch-gcve-dev-sa[0].email, null) - gcve-dev-r = try(module.branch-gcve-dev-r-sa[0].email, null) - gcve-prod = try(module.branch-gcve-prod-sa[0].email, null) - gcve-prod-r = try(module.branch-gcve-prod-r-sa[0].email, null) - gke-dev = try(module.branch-gke-dev-sa[0].email, null) - gke-dev-r = try(module.branch-gke-dev-r-sa[0].email, null) - gke-prod = try(module.branch-gke-prod-sa[0].email, null) - gke-prod-r = try(module.branch-gke-prod-r-sa[0].email, null) - nsec = module.branch-nsec-sa.email - nsec-r = module.branch-nsec-r-sa.email - networking = module.branch-network-sa.email - networking-r = module.branch-network-r-sa.email - project-factory = try(module.branch-pf-sa[0].email, null) - project-factory-r = try(module.branch-pf-r-sa[0].email, null) - project-factory-dev = try(module.branch-pf-dev-sa[0].email, null) - project-factory-dev-r = try(module.branch-pf-dev-r-sa[0].email, null) - project-factory-prod = try(module.branch-pf-prod-sa[0].email, null) - project-factory-prod-r = try(module.branch-pf-prod-r-sa[0].email, null) - sandbox = try(module.branch-sandbox-sa[0].email, null) - security = module.branch-security-sa.email - security-r = module.branch-security-r-sa.email - }, + local.branch_service_accounts, { for k, v in module.top-level-sa : k => try(v.email) } @@ -487,18 +459,18 @@ output "networking" { output "project_factories" { description = "Data for the project factories stage." - value = !var.fast_features.project_factory ? {} : { + value = { dev = { - bucket = module.branch-pf-dev-gcs[0].name - sa = module.branch-pf-dev-sa[0].email + bucket = module.branch-pf-dev-gcs.name + sa = module.branch-pf-dev-sa.email } main = { - bucket = module.branch-pf-gcs[0].name - sa = module.branch-pf-sa[0].email + bucket = module.branch-pf-gcs.name + sa = module.branch-pf-sa.email } prod = { - bucket = module.branch-pf-prod-gcs[0].name - sa = module.branch-pf-prod-sa[0].email + bucket = module.branch-pf-prod-gcs.name + sa = module.branch-pf-prod-sa.email } } } diff --git a/fast/stages/1-resman/schemas/top-level-folder.schema.json b/fast/stages/1-resman/schemas/top-level-folder.schema.json new file mode 100644 index 000000000..60e22c8ae --- /dev/null +++ b/fast/stages/1-resman/schemas/top-level-folder.schema.json @@ -0,0 +1,233 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "title": "Folder", + "type": "object", + "additionalProperties": false, + "properties": { + "automation": { + "type": "object", + "additionalProperties": false, + "properties": { + "enable": { + "type": "boolean" + }, + "sa_impersonation_principals": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "iam": { + "$ref": "#/$defs/iam" + }, + "iam_bindings": { + "$ref": "#/$defs/iam_bindings" + }, + "iam_bindings_additive": { + "$ref": "#/$defs/iam_bindings_additive" + }, + "iam_by_principals": { + "$ref": "#/$defs/iam_by_principals" + }, + "name": { + "type": "string" + }, + "org_policies": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z]+\\.": { + "inherit_from_parent": { + "type": "boolean" + }, + "reset": { + "type": "boolean" + }, + "rules": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "properties": { + "allow": { + "type": "object", + "additionalProperties": false, + "properties": { + "all": { + "type": "boolean" + }, + "values": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "deny": { + "type": "object", + "additionalProperties": false, + "properties": { + "all": { + "type": "boolean" + }, + "values": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "enforce": { + "type": "boolean" + }, + "condition": { + "type": "object", + "additionalProperties": false, + "properties": { + "description": { + "type": "string" + }, + "expression": { + "type": "string" + }, + "location": { + "type": "string" + }, + "title": { + "type": "string" + } + } + } + } + } + } + } + } + }, + "parent": { + "type": "string" + }, + "tag_bindings": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9_-]+$": { + "type": "string" + } + } + } + }, + "$defs": { + "iam": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^(?:roles/|[a-z_]+)": { + "type": "array", + "items": { + "type": "string", + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|project-factory|project-factory-dev|project-factory-prod|networking|security|vpcsc)" + } + } + } + }, + "iam_bindings": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9_-]+$": { + "type": "object", + "additionalProperties": false, + "properties": { + "members": { + "type": "array", + "items": { + "type": "string", + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|project-factory|project-factory-dev|project-factory-prod|networking|security|vpcsc)" + } + }, + "role": { + "type": "string", + "pattern": "^(?:roles/|[a-z_]+)" + }, + "condition": { + "type": "object", + "additionalProperties": false, + "required": [ + "expression", + "title" + ], + "properties": { + "expression": { + "type": "string" + }, + "title": { + "type": "string" + }, + "description": { + "type": "string" + } + } + } + } + } + } + }, + "iam_bindings_additive": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9_-]+$": { + "type": "object", + "additionalProperties": false, + "properties": { + "member": { + "type": "string", + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|project-factory|project-factory-dev|project-factory-prod|networking|security|vpcsc)" + }, + "role": { + "type": "string", + "pattern": "^(?:roles/|[a-z_]+)" + }, + "condition": { + "type": "object", + "additionalProperties": false, + "required": [ + "expression", + "title" + ], + "properties": { + "expression": { + "type": "string" + }, + "title": { + "type": "string" + }, + "description": { + "type": "string" + } + } + } + } + } + } + }, + "iam_by_principals": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:)": { + "type": "array", + "items": { + "type": "string", + "pattern": "^(?:roles/|[a-z_]+)" + } + } + } + } + } +} \ No newline at end of file diff --git a/fast/stages/1-resman/top-level-folders.tf b/fast/stages/1-resman/top-level-folders.tf index 2de5eed9e..d7feeb8c1 100644 --- a/fast/stages/1-resman/top-level-folders.tf +++ b/fast/stages/1-resman/top-level-folders.tf @@ -56,25 +56,52 @@ locals { }, var.top_level_folders ) + top_level_sa = { + for k, v in local.branch_service_accounts : + k => "serviceAccount:${v}" if v != null + } + top_level_tags = { + for k, v in try(local.tag_values, {}) : k => v.id + } } module "top-level-folder" { - source = "../../../modules/folder" - for_each = local.top_level_folders - parent = "organizations/${var.organization.id}" - name = each.value.name - contacts = each.value.contacts - firewall_policy = each.value.firewall_policy - logging_data_access = each.value.logging_data_access - logging_exclusions = each.value.logging_exclusions - logging_settings = each.value.logging_settings - logging_sinks = each.value.logging_sinks - iam = each.value.iam - iam_bindings = each.value.iam_bindings - iam_bindings_additive = each.value.iam_bindings_additive - iam_by_principals = each.value.iam_by_principals - org_policies = each.value.org_policies - tag_bindings = each.value.tag_bindings + source = "../../../modules/folder" + for_each = local.top_level_folders + parent = "organizations/${var.organization.id}" + name = each.value.name + contacts = each.value.contacts + firewall_policy = each.value.firewall_policy + logging_data_access = each.value.logging_data_access + logging_exclusions = each.value.logging_exclusions + logging_settings = each.value.logging_settings + logging_sinks = each.value.logging_sinks + iam = { + for role, members in each.value.iam : + lookup(var.custom_roles, role, role) => [ + for member in members : lookup(local.top_level_sa, member, member) + ] + } + iam_bindings = { + for k, v in each.value.iam_bindings : k => merge(v, { + member = lookup(local.top_level_sa, v.member, v.member) + role = lookup(var.custom_roles, v.role, v.role) + }) + } + iam_bindings_additive = { + for k, v in each.value.iam_bindings_additive : k => merge(v, { + member = lookup(local.top_level_sa, v.member, v.member) + role = lookup(var.custom_roles, v.role, v.role) + }) + } + # we don't replace here to avoid dynamic values in keys + iam_by_principals = each.value.iam_by_principals + org_policies = each.value.org_policies + tag_bindings = { + for k, v in each.value.tag_bindings : k => lookup( + local.top_level_tags, v, v + ) + } } module "top-level-sa" { diff --git a/fast/stages/1-resman/variables.tf b/fast/stages/1-resman/variables.tf index f9f39ea14..c111404f8 100644 --- a/fast/stages/1-resman/variables.tf +++ b/fast/stages/1-resman/variables.tf @@ -124,7 +124,7 @@ variable "factories_config" { type = object({ checklist_data = optional(string) org_policies = optional(string, "data/org-policies") - top_level_folders = optional(string) + top_level_folders = optional(string, "data/top-level-folders") }) nullable = false default = {} @@ -133,11 +133,10 @@ variable "factories_config" { variable "fast_features" { description = "Selective control for top-level FAST features." type = object({ - data_platform = optional(bool, false) - gke = optional(bool, false) - gcve = optional(bool, false) - project_factory = optional(bool, false) - sandbox = optional(bool, false) + data_platform = optional(bool, false) + gke = optional(bool, false) + gcve = optional(bool, false) + sandbox = optional(bool, false) }) default = {} nullable = false diff --git a/fast/stages/2-project-factory/README.md b/fast/stages/2-project-factory/README.md new file mode 100644 index 000000000..f1ab1d6e8 --- /dev/null +++ b/fast/stages/2-project-factory/README.md @@ -0,0 +1,357 @@ +# Project factory + + +- [Design overview and choices](#design-overview-and-choices) +- [How to run this stage](#how-to-run-this-stage) + - [Resource Management stage configuration](#resource-management-stage-configuration) + - [Factory configuration](#factory-configuration) + - [Stage provider and Terraform variables](#stage-provider-and-terraform-variables) +- [Managing folders and projects](#managing-folders-and-projects) + - [Folder and hierarchy management](#folder-and-hierarchy-management) + - [Folder parent-child relationship and variable substitutions](#folder-parent-child-relationship-and-variable-substitutions) + - [Project Creation](#project-creation) + - [Automation Resources for Projects](#automation-resources-for-projects) +- [Alternative patterns](#alternative-patterns) + - [Per-environment Factories](#per-environment-factories) +- [Files](#files) +- [Variables](#variables) +- [Outputs](#outputs) + + +The Project Factory stage allows simplified management of folder hierarchies and projects via YAML-based configuration files. Multiple project factories can coexist in the same landing zone, and different patterns can be implemented by pointing them at different configuration files. + +The pattern implemented here by default allows management of a teams (or business units, applications, etc.) hierarchy. Different patterns are possible, and this document also tries to provide some guidance on how to implement them. + +

+ Project factory teams pattern +

+ +## Design overview and choices + +The project factory is "primed" by the resource management stage via + +- a set of service accounts with different scopes +- one or more user-defined top-level folders where those service accounts operate + +This stage does not directly depend on other stage 2 like networking and security, but it can optionally leverage resources created there like Shared VPC host projects, which are used to define service projects. + +The project factory stage is a thin wrapper of the underlying [project-factory module](../../../modules/project-factory/), which in turn exposes the full interface of the [project](../../../modules/project/) and [folder](../../../modules/folder/) modules. + +## How to run this stage + +This stage is meant to be executed after the [bootstrap](../0-bootstrap/) and [resource management](../1-resman/) "foundational stages". As mentioned above it runs in parallel with other stage 2 and can leverage resources they create but does not depend on them. + +### Resource Management stage configuration + +The resource management stage already contains a sample "Teams" folder defined via YAML, which can be used as-is or modified to provide a top-level folder for the project factory. More folders can of course be added, and Terraform variables used instead of or in addition to YAML files in the resource management stage. + +This is the teams YAML in resource management, leveraging attribute substitutions from provided context for the project factory service account and tag value. + +```yaml +name: Teams +automation: + enable: false +iam: + "roles/owner": + - project-factory + "roles/resourcemanager.folderAdmin": + - project-factory + "roles/resourcemanager.projectCreator": + - project-factory + "roles/resourcemanager.tagUser": + - project-factory + "service_project_network_admin": + - project-factory +tag_bindings: + context: context/project-factory +``` + +This is the alternative version that can be used instead of the YAML file above. + +```tfvars +top_level_folders = { + # more top-level folders might be present here + teams = { + name = "Teams" + iam = { + "roles/owner" = ["project-factory"] + "roles/resourcemanager.folderAdmin" = ["project-factory"] + "roles/resourcemanager.projectCreator" = ["project-factory"] + "roles/resourcemanager.tagUser" = ["project-factory"] + "service_project_network_admin" = ["project-factory"] + } + tag_bindings = { + context = "context/project-factory" + } + } +} +# tftest skip +``` + +You can of course extend these snippets to grant additional roles to groups or other service accounts via the `iam`, `iam_by_principals`, and `iam_bindings` folder-level variables. + +The project factory tag binding on the folder allows management of organization policies in the Teams hierarchy. If this functionality is not needed, the tag binding can be safely omitted. + +### Factory configuration + +The `data` folder in this stage contains factory files that can be used as examples to implement the team-based design shown above. Before running `terraform apply` check the YAML files, as project names and other attributes will need basic editing to match your desired setup. + +### Stage provider and Terraform variables + +As all other FAST stages, the [mechanism](../0-bootstrap/README.md#output-files-and-cross-stage-variables) used to pass variable values and pre-built provider files from one stage to the next is also leveraged here. + +The commands to link or copy the provider and terraform variable files can be easily derived from the `stage-links.sh` script in the FAST root folder, passing it a single argument with the local output files folder (if configured) or the GCS output bucket in the automation project (derived from stage 0 outputs). The following examples demonstrate both cases, and the resulting commands that then need to be copy/pasted and run. + +```bash +../../../stage-links.sh ~/fast-config + +# copy and paste the following commands for '2-project-factory' + +ln -s ~/fast-config/providers/2-project-factory-providers.tf ./ +ln -s ~/fast-config/tfvars/0-globals.auto.tfvars.json ./ +ln -s ~/fast-config/tfvars/0-bootstrap.auto.tfvars.json ./ +ln -s ~/fast-config/tfvars/1-resman.auto.tfvars.json ./ +# optional but recommended +ln -s ~/fast-config/tfvars/2-networking.auto.tfvars.json ./ +ln -s ~/fast-config/tfvars/2-security.auto.tfvars.json ./ +``` + +```bash +../../../stage-links.sh gs://xxx-prod-iac-core-outputs-0 + +# copy and paste the following commands for '2-project-factory' + +gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/2-project-factory-providers.tf ./ +gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json ./ +gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./ +gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json ./ +# optional but recommended +gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/2-networking.auto.tfvars.json ./ +gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/2-security.auto.tfvars.json ./ +``` + +If you're not using FAST, refer to the [Variables](#variables) table at the bottom of this document for a full list of variables, their origin (e.g., a stage or specific to this one), and descriptions explaining their meaning. + +Besides the values above, the project factory is driven by YAML data files, with one file per project. Please refer to the underlying [project factory module](../../../modules/project-factory/) documentation for details on the format. + +Once the configuration is complete, run the project factory with: + +```bash +terraform init +terraform apply +``` + +## Managing folders and projects + +The YAML data files are self-explanatory and the included [schema files](./schemas/) provide a reliable framework to allow editing the sample data, or starting from scratch to implement a different pattern. This section lists some general considerations on how folder and project files work to help getting up to speed with operations. + +### Folder and hierarchy management + +The project factory manages its folder hierarchy via a filesystem tree, rooted in the path defined via the `factories_config.hierarchy_data` variable. + +Filesystem folders which contain a `_config.yaml` file are mapped to folders in the resource management hierarchy. Their YAML configuration files allow definining folder attributes like descriptive name, IAM bindings, organization policies, tag bindings. + +This is the simple filesystem hierarchy provided here as an example. + +```bash +hierarchy +├── team-a +│   ├── _config.yaml +│   ├── dev +│   │   └── _config.yaml +│   └── prod +│   └── _config.yaml +└── team-b + ├── _config.yaml + ├── dev + │   └── _config.yaml + └── prod + └── _config.yaml +``` + +The approach is intentionally explicit and repetitive in order to simplify operations: copy/pasting an existing set of folders (or an ad hoc template) and changing a few YAML variables allows to quickly define new sub-hierarchy branches. Mass editing via search and replace functionality allows sweeping changes across the whole hierarchy. + +Where inheritance is leveraged in the overall design config files can be deceptively simple: the following is the config file for the dev Team A folder in the provided example. + +```yaml +name: Development +tag_bindings: + environment: environment/development +iam_by_principals: + "group:team-a-admins@example.com": + - roles/editor +``` + +All of the [folder module](../../../modules/folder/) attributes can of course be leveraged in the configuration files. Refer to the [folder schema](./schemas/folder.schema.json) for the complete set of available attributes. + +### Folder parent-child relationship and variable substitutions + +In the example YAML configuration above there's no explicitly specified folder parent: it is derived from the filesystem hierarchy, and set to the "Team A" folder. + +But what about the "Team A" folder itself? From the point of view of the project factory it's a top-level folder attached to the root of its hierarchy (the "Teams" folder), so how does it know where to create it in the GCP hierarchy? + +There are three different ways to pass this information to the project factory: + +- in the YAML file itself, by explicitly setting the folder's `parent` attribute to the explicit numeric id of the "Teams" folder +- in the YAML file itself, by explicitly setting the folder's `parent` attribute to the short name of the "Teams" folder in the resource management stage's outputs +- in the stage Terraform variables, by setting the `default` folder for the project factory to the numeric id of the "Teams" folder + +This flexibility is what allows the project factory to manage folders under multiple roots, and to also be used for folders created outside of FAST. Imagine a scenario where there's no single "Teams" folder, but multiple ones for different subsidiaries, or for internal and external teams, etc. + +The snippets below show how to set the `parent` attribute explicitly or via substitution in the YAML file. + +```yaml +name: Team A +# use the explicit id of the Teams folder +parent: folders/1234567890 +``` + +```yaml +name: Team A +# use variable substitution from stage 1 tfvars (preferred approach) +parent: teams +``` + +The third way explained above does not explicitly define a root folder in the YAML files, but sets a default folder in the Terraform variables for the stage via the `factories_config.substitutions.folder_ids`, by adding a `default` key pointing to the folder id of the root ("Teams") folder. + +```tfvars +factories_config = { + substitutions = { + folder_ids = { + # id of the top-level Teams folder + # derived from the 1-resman.auto.tfvars.json file + default = "folders/12345678" + } + } +} +# tftest skip +``` + +### Project Creation + +Project YAML files can be created in two different filesystem paths: + +- in the filesystem folder defined via the `factories_config.project_data` variable, and then explicitly setting their `parent` attribute in YAML files, or +- in the filesystem hierarchy discussed above, so that their `parent` attribute is automatically derived from the containing folder + +The two approaches can be mixed and matched, but the first approach is safer as is avoids potentially dangerous situations when folders are deleted with project configuration files still inside. + +When specifying projects outside of the folder hierarchy, setting the parent folder works in pretty much the same way as discussed above, with substitutions available for any folder defined in the filesystem hierarchy. This allows writing portable files, by referring to short names instead of resource ids. + +```yaml +# use the explicit id of the parent folder +parent: folders/1234509876 +``` + +```yaml +# use variable substitution from managed folders (preferred approach) +parent: team-a/dev +``` + +All of the [project module](../../../modules/project/) attributes (and some service account attributes) can of course be leveraged in the configuration files. Refer to the [project schema](./schemas/folder.schema.json) for the complete set of available attributes. + +### Automation Resources for Projects + +When created projects are meant to be managed via IaC downstream, an initial set of automation resources can be created in a "controlling project". The preferred pattern is to first create one or more controlling projects for the project factory, and then leverage them for service account and GCS bucket creation. + +```yaml +# controlling project shown in the diagram above +parent: teams +name: xxx-prod-iac-teams-0 +services: + - compute.googleapis.com + - storage.googleapis.com + # ... + # enable all services used by service accounts in this project +``` + +Once a controlling project is in place, it can be used in any other project declaration to host service accounts and buckets for automation. The service accounts can be used in IAM bindings in the same file by referring to their name via substitutions, as shown here. + +```yaml +# team or application-level project with automation resources +parent: team-a/dev +# project prefix is forced via override in `main.tf` +name: dev-ta-app-0 +iam: + roles/owner: + # refer to the rw service account defined below + - rw + roles/viewer: + # refer to the ro service account defined below + - ro +automation: + # no context is possible here + # use the complete project id + project: xxx-prod-iac-teams-0 + service_accounts: + # resulting sa name: xxx-dev-ta-app-0-rw + rw: + description: Read/write automation sa for team a app 0. + # resulting sa name: xxx-dev-ta-app-0-ro + ro: + description: Read-only automation sa for team a app 0. + buckets: + # resulting bucket name: xxx-dev-ta-app-0-state + state: + description: Terraform state bucket for team a app 0. + iam: + # service accounts can use short name substitutions from context + roles/storage.objectCreator: + - rw + roles/storage.objectViewer: + - rw + - ro + - group:devops@example.org +``` + +## Alternative patterns + +Some alternative patterns are captured here, the list will grow as we generalize approaches seen in the field. + +### Per-environment Factories + +A variation of this pattern uses separate project factories for each environment, as in the following diagram. + +

+ Project factory team-level per environment. +

+ +This approach leverages the per-environment project factory service accounts and tags created by the resource management stage, so that + +- the Teams folder hierarchy and IaC project are managed by a cross-environment factory using the "main" project factory service account +- IAM permissions are set on the environment folders to grant control to the prod and dev project factory service accounts +- one additional factory per environment manages project creation leveraging the folders created above + +The approach is not shown here but reasonably easy to implement. The main project factory output file can also be used to set up folder id susbtitution in the per-environment factories. + + + +## Files + +| name | description | modules | +|---|---|---| +| [main.tf](./main.tf) | Project factory. | project-factory | +| [outputs.tf](./outputs.tf) | Module outputs. | | +| [variables-fast.tf](./variables-fast.tf) | None | | +| [variables.tf](./variables.tf) | Module variables. | | + +## Variables + +| name | description | type | required | default | producer | +|---|---|:---:|:---:|:---:|:---:| +| [billing_account](variables-fast.tf#L17) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | object({…}) | ✓ | | 0-bootstrap | +| [prefix](variables-fast.tf#L55) | Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants. | string | ✓ | | 0-bootstrap | +| [factories_config](variables.tf#L17) | Configuration for YAML-based factories. | object({…}) | | {} | | +| [folder_ids](variables-fast.tf#L30) | Folders created in the resource management stage. | map(string) | | {} | 1-resman | +| [groups](variables-fast.tf#L38) | Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated. | map(string) | | {} | 0-bootstrap | +| [host_project_ids](variables-fast.tf#L47) | Host project for the shared VPC. | map(string) | | {} | 2-networking | +| [service_accounts](variables-fast.tf#L65) | Automation service accounts in name => email format. | map(string) | | {} | 1-resman | +| [tag_values](variables-fast.tf#L73) | FAST-managed resource manager tag values. | map(string) | | {} | 1-resman | + +## Outputs + +| name | description | sensitive | consumers | +|---|---|:---:|---| +| [projects](outputs.tf#L17) | Created projects. | | | +| [service_accounts](outputs.tf#L27) | Created service accounts. | | | + diff --git a/fast/stages/2-project-factory/data/hierarchy/team-a/_config.yaml b/fast/stages/2-project-factory/data/hierarchy/team-a/_config.yaml new file mode 100644 index 000000000..410d9e86f --- /dev/null +++ b/fast/stages/2-project-factory/data/hierarchy/team-a/_config.yaml @@ -0,0 +1,21 @@ +# Copyright 2024 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# yaml-language-server: $schema=../../../schemas/folder.schema.json + +name: Team A +parent: teams +# iam_by_principals: +# "group:team-a-admins@example.com": +# - roles/viewer diff --git a/fast/stages/2-project-factory/data/hierarchy/team-a/dev/_config.yaml b/fast/stages/2-project-factory/data/hierarchy/team-a/dev/_config.yaml new file mode 100644 index 000000000..da77cb7f1 --- /dev/null +++ b/fast/stages/2-project-factory/data/hierarchy/team-a/dev/_config.yaml @@ -0,0 +1,22 @@ +# Copyright 2024 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# yaml-language-server: $schema=../../../../schemas/folder.schema.json + +name: Development +tag_bindings: + environment: environment/development +# iam_by_principals: +# "group:team-a-admins@example.com": +# - roles/editor diff --git a/tests/fast/stages/s3_project_factory/simple.yaml b/fast/stages/2-project-factory/data/hierarchy/team-a/prod/_config.yaml similarity index 80% rename from tests/fast/stages/s3_project_factory/simple.yaml rename to fast/stages/2-project-factory/data/hierarchy/team-a/prod/_config.yaml index af751a985..a7079ab36 100644 --- a/tests/fast/stages/s3_project_factory/simple.yaml +++ b/fast/stages/2-project-factory/data/hierarchy/team-a/prod/_config.yaml @@ -12,9 +12,8 @@ # See the License for the specific language governing permissions and # limitations under the License. -counts: - google_project: 1 - google_project_service: 3 - google_storage_project_service_account: 1 - modules: 2 - resources: 6 +# yaml-language-server: $schema=../../../../schemas/folder.schema.json + +name: Production +tag_bindings: + environment: environment/production \ No newline at end of file diff --git a/fast/stages/2-project-factory/data/hierarchy/team-b/_config.yaml b/fast/stages/2-project-factory/data/hierarchy/team-b/_config.yaml new file mode 100644 index 000000000..80d5faa67 --- /dev/null +++ b/fast/stages/2-project-factory/data/hierarchy/team-b/_config.yaml @@ -0,0 +1,21 @@ +# Copyright 2024 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# yaml-language-server: $schema=../../../schemas/folder.schema.json + +name: Team B +parent: teams +# iam_by_principals: +# "group:team-b-admins@example.com": +# - roles/viewer diff --git a/fast/stages/2-project-factory/data/hierarchy/team-b/dev/_config.yaml b/fast/stages/2-project-factory/data/hierarchy/team-b/dev/_config.yaml new file mode 100644 index 000000000..e50bb7308 --- /dev/null +++ b/fast/stages/2-project-factory/data/hierarchy/team-b/dev/_config.yaml @@ -0,0 +1,22 @@ +# Copyright 2024 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# yaml-language-server: $schema=../../../../schemas/folder.schema.json + +name: Development +tag_bindings: + environment: environment/development +# iam_by_principals: +# "group:team-b-admins@example.com": +# - roles/editor diff --git a/fast/stages/3-project-factory/dev/data/projects/test-project.yaml b/fast/stages/2-project-factory/data/hierarchy/team-b/prod/_config.yaml similarity index 76% rename from fast/stages/3-project-factory/dev/data/projects/test-project.yaml rename to fast/stages/2-project-factory/data/hierarchy/team-b/prod/_config.yaml index dfe34e6cc..a7079ab36 100644 --- a/fast/stages/3-project-factory/dev/data/projects/test-project.yaml +++ b/fast/stages/2-project-factory/data/hierarchy/team-b/prod/_config.yaml @@ -1,4 +1,4 @@ -# Copyright 2023 Google LLC +# Copyright 2024 Google LLC # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -12,9 +12,8 @@ # See the License for the specific language governing permissions and # limitations under the License. -labels: - team: team-0 -parent: folders/1234567890 -services: -- compute.googleapis.com -- storage.googleapis.com +# yaml-language-server: $schema=../../../../schemas/folder.schema.json + +name: Production +tag_bindings: + environment: environment/production \ No newline at end of file diff --git a/tests/fast/stages/s3_project_factory/data/projects/project.yaml b/fast/stages/2-project-factory/data/projects/dev-ta-0.yaml similarity index 73% rename from tests/fast/stages/s3_project_factory/data/projects/project.yaml rename to fast/stages/2-project-factory/data/projects/dev-ta-0.yaml index 922b4044f..c285e790c 100644 --- a/tests/fast/stages/s3_project_factory/data/projects/project.yaml +++ b/fast/stages/2-project-factory/data/projects/dev-ta-0.yaml @@ -1,4 +1,4 @@ -# Copyright 2023 Google LLC +# Copyright 2024 Google LLC # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -12,8 +12,10 @@ # See the License for the specific language governing permissions and # limitations under the License. -parent: folders/012345678901 -services: - - storage.googleapis.com - - stackdriver.googleapis.com - - compute.googleapis.com +# yaml-language-server: $schema=../../schemas/project.schema.json + +parent: team-a/dev +shared_vpc_service_config: + host_project: dev-spoke-0 + network_users: + - gcp-devops \ No newline at end of file diff --git a/fast/stages/2-project-factory/data/projects/dev-tb-0.yaml b/fast/stages/2-project-factory/data/projects/dev-tb-0.yaml new file mode 100644 index 000000000..1dd414fac --- /dev/null +++ b/fast/stages/2-project-factory/data/projects/dev-tb-0.yaml @@ -0,0 +1,21 @@ +# Copyright 2024 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# yaml-language-server: $schema=../../schemas/project.schema.json + +parent: team-b/dev +shared_vpc_service_config: + host_project: dev-spoke-0 + network_users: + - gcp-devops \ No newline at end of file diff --git a/fast/stages/2-project-factory/data/projects/prod-ta-0.yaml b/fast/stages/2-project-factory/data/projects/prod-ta-0.yaml new file mode 100644 index 000000000..1bc5c895e --- /dev/null +++ b/fast/stages/2-project-factory/data/projects/prod-ta-0.yaml @@ -0,0 +1,21 @@ +# Copyright 2024 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# yaml-language-server: $schema=../../schemas/project.schema.json + +parent: team-a/prod +shared_vpc_service_config: + host_project: prod-spoke-0 + network_users: + - gcp-devops \ No newline at end of file diff --git a/fast/stages/2-project-factory/data/projects/prod-tb-0.yaml b/fast/stages/2-project-factory/data/projects/prod-tb-0.yaml new file mode 100644 index 000000000..ee1e12cc1 --- /dev/null +++ b/fast/stages/2-project-factory/data/projects/prod-tb-0.yaml @@ -0,0 +1,21 @@ +# Copyright 2024 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# yaml-language-server: $schema=../../schemas/project.schema.json + +parent: team-b/prod +shared_vpc_service_config: + host_project: prod-spoke-0 + network_users: + - gcp-devops \ No newline at end of file diff --git a/fast/stages/2-project-factory/diagram-env.png b/fast/stages/2-project-factory/diagram-env.png new file mode 100644 index 000000000..f7761028f Binary files /dev/null and b/fast/stages/2-project-factory/diagram-env.png differ diff --git a/fast/stages/2-project-factory/diagram.png b/fast/stages/2-project-factory/diagram.png new file mode 100644 index 000000000..b442808b8 Binary files /dev/null and b/fast/stages/2-project-factory/diagram.png differ diff --git a/fast/stages/3-project-factory/dev/main.tf b/fast/stages/2-project-factory/main.tf similarity index 55% rename from fast/stages/3-project-factory/dev/main.tf rename to fast/stages/2-project-factory/main.tf index c0b0094ad..0ec6053d3 100644 --- a/fast/stages/3-project-factory/dev/main.tf +++ b/fast/stages/2-project-factory/main.tf @@ -17,23 +17,41 @@ # tfdoc:file:description Project factory. module "projects" { - source = "../../../../modules/project-factory" + source = "../../../modules/project-factory" data_defaults = { - billing_account = var.billing_account.id # more defaults are available, check the project factory variables + billing_account = var.billing_account.id } data_merges = { - labels = { - environment = "dev" - } services = [ "stackdriver.googleapis.com" ] } data_overrides = { - prefix = "${var.prefix}-dev" + prefix = var.prefix } - factories_config = var.factories_config + factories_config = merge(var.factories_config, { + context = { + folder_ids = merge( + { for k, v in var.folder_ids : k => v if v != null }, + var.factories_config.context.folder_ids + ) + iam_principals = merge( + { + for k, v in var.service_accounts : + k => "serviceAccount:${v}" if v != null + }, + var.groups, + var.factories_config.context.iam_principals + ) + tag_values = merge( + var.tag_values, + var.factories_config.context.tag_values + ) + vpc_host_projects = merge( + var.host_project_ids, + var.factories_config.context.vpc_host_projects + ) + } + }) } - - diff --git a/fast/stages/3-project-factory/dev/outputs.tf b/fast/stages/2-project-factory/outputs.tf similarity index 100% rename from fast/stages/3-project-factory/dev/outputs.tf rename to fast/stages/2-project-factory/outputs.tf diff --git a/fast/stages/2-project-factory/schemas/budget.schema.json b/fast/stages/2-project-factory/schemas/budget.schema.json new file mode 120000 index 000000000..cc5d28d4d --- /dev/null +++ b/fast/stages/2-project-factory/schemas/budget.schema.json @@ -0,0 +1 @@ +../../../../modules/billing-account/schemas/budget.schema.json \ No newline at end of file diff --git a/fast/stages/2-project-factory/schemas/folder.schema.json b/fast/stages/2-project-factory/schemas/folder.schema.json new file mode 120000 index 000000000..d58a2759b --- /dev/null +++ b/fast/stages/2-project-factory/schemas/folder.schema.json @@ -0,0 +1 @@ +../../../../modules/project-factory/schemas/folder.schema.json \ No newline at end of file diff --git a/fast/stages/2-project-factory/schemas/project.schema.json b/fast/stages/2-project-factory/schemas/project.schema.json new file mode 120000 index 000000000..11f161f17 --- /dev/null +++ b/fast/stages/2-project-factory/schemas/project.schema.json @@ -0,0 +1 @@ +../../../../modules/project-factory/schemas/project.schema.json \ No newline at end of file diff --git a/fast/stages/3-project-factory/dev/variables-fast.tf b/fast/stages/2-project-factory/variables-fast.tf similarity index 54% rename from fast/stages/3-project-factory/dev/variables-fast.tf rename to fast/stages/2-project-factory/variables-fast.tf index 0144aae29..9b4c11807 100644 --- a/fast/stages/3-project-factory/dev/variables-fast.tf +++ b/fast/stages/2-project-factory/variables-fast.tf @@ -27,6 +27,31 @@ variable "billing_account" { } } +variable "folder_ids" { + # tfdoc:variable:source 1-resman + description = "Folders created in the resource management stage." + type = map(string) + nullable = false + default = {} +} + +variable "groups" { + # tfdoc:variable:source 0-bootstrap + # https://cloud.google.com/docs/enterprise/setup-checklist + description = "Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated." + type = map(string) + nullable = false + default = {} +} + +variable "host_project_ids" { + # tfdoc:variable:source 2-networking + description = "Host project for the shared VPC." + type = map(string) + nullable = false + default = {} +} + variable "prefix" { # tfdoc:variable:source 0-bootstrap description = "Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants." @@ -36,3 +61,19 @@ variable "prefix" { error_message = "Use a maximum of 9 chars for organizations, and 11 chars for tenants." } } + +variable "service_accounts" { + # tfdoc:variable:source 1-resman + description = "Automation service accounts in name => email format." + type = map(string) + nullable = false + default = {} +} + +variable "tag_values" { + # tfdoc:variable:source 1-resman + description = "FAST-managed resource manager tag values." + type = map(string) + nullable = false + default = {} +} diff --git a/fast/stages/3-project-factory/dev/variables.tf b/fast/stages/2-project-factory/variables.tf similarity index 59% rename from fast/stages/3-project-factory/dev/variables.tf rename to fast/stages/2-project-factory/variables.tf index f5eb18eb1..a0bc5d7fd 100644 --- a/fast/stages/3-project-factory/dev/variables.tf +++ b/fast/stages/2-project-factory/variables.tf @@ -15,18 +15,23 @@ */ variable "factories_config" { - description = "Path to folder with YAML resource description data files." + description = "Configuration for YAML-based factories." type = object({ - hierarchy = optional(object({ - folders_data_path = string - parent_ids = optional(map(string), {}) - })) - projects_data_path = optional(string) + folders_data_path = optional(string, "data/hierarchy") + projects_data_path = optional(string, "data/projects") budgets = optional(object({ billing_account = string - budgets_data_path = string + budgets_data_path = optional(string, "data/budgets") notification_channels = optional(map(any), {}) })) + context = optional(object({ + # TODO: add KMS keys + folder_ids = optional(map(string), {}) + iam_principals = optional(map(string), {}) + tag_values = optional(map(string), {}) + vpc_host_projects = optional(map(string), {}) + }), {}) }) nullable = false + default = {} } diff --git a/fast/stages/3-gke-multitenant/README.md b/fast/stages/3-gke-multitenant/README.md index 9f9d9498e..f5d73a490 100644 --- a/fast/stages/3-gke-multitenant/README.md +++ b/fast/stages/3-gke-multitenant/README.md @@ -2,7 +2,7 @@ This directory contains a stage that can be used to centralize management of GKE multinenant clusters. -The Terraform code follows the same general approach used for the [project factory](../3-project-factory/) and [data platform](../3-data-platform/) stages, where a "fat module" contains the stage code and is used by thin code wrappers that localize it for each environment or specialized configuration: +The Terraform code follows the same general approach used for the [project factory](../2-project-factory/) and [data platform](../3-data-platform/) stages, where a "fat module" contains the stage code and is used by thin code wrappers that localize it for each environment or specialized configuration: The [`dev` folder](./dev/) contains an example setup for a generic development environment, and can be used as-is or cloned to implement other environments, or more specialized setups diff --git a/fast/stages/3-project-factory/README.md b/fast/stages/3-project-factory/README.md deleted file mode 100644 index 3a1394116..000000000 --- a/fast/stages/3-project-factory/README.md +++ /dev/null @@ -1,6 +0,0 @@ -# Project factory - -The Project Factory (PF) builds on top of your foundations to create and set up projects (and related resources) to be used for your workloads. -It is organized in folders representing environments (e.g. "dev", "prod"), each implemented by a stand-alone terraform [resource factory](https://medium.com/google-cloud/resource-factories-a-descriptive-approach-to-terraform-581b3ebb59c). - -This directory contains a single project factory ([`dev/`](./dev/)) as an example - to implement multiple environments (e.g. "prod" and "dev") you'll need to copy the `dev` folder into one folder per environment, then customize each one following the instructions found in [`dev/README.md`](./dev/README.md). diff --git a/fast/stages/3-project-factory/dev/README.md b/fast/stages/3-project-factory/dev/README.md deleted file mode 100644 index 933e8ec18..000000000 --- a/fast/stages/3-project-factory/dev/README.md +++ /dev/null @@ -1,92 +0,0 @@ -# Project factory - -The Project Factory (or PF) builds on top of your foundations to create and set up projects (and related resources) to be used for your workloads. -It is organized in folders representing environments (e.g., "dev", "prod"), each implemented by a stand-alone terraform [process factory](../../../../blueprints/factories/README.md). - -## Design overview and choices - -

- Project factory diagram -

- -A single factory creates projects in a well-defined context, according to your resource management structure. For example, in the diagram above, each Team is structured to have specific folders projects for a given environment, such as Production and Development, per the resource management structure configured in stage `01-resman`. - -Projects for each environment across different teams are created by dedicated service accounts, as exemplified in the diagram above. While there's no intrinsic limitation regarding where the project factory can create a projects, the IAM bindings for the service account effectively enforce boundaries (e.g., the production service account shouldn't be able to create or have any access to the development projects, and vice versa). - -The project factory stage lightly wraps the underlying [project-factory module](../../../../modules/project-factory/), including Shared VPC service project attachment, VPC SC perimeter membership, etc. - -## How to run this stage - -This stage is meant to be executed after "foundational stages" (i.e., stages [`00-bootstrap`](../../0-bootstrap), [`01-resman`](../../1-resman), 02-networking (either [Peering/VPN](../../2-networking-a-simple), [NVA (w/ optional BGP support)](../../2-networking-b-nva) and [`02-security`](../../2-security)) have been run. - -It's of course possible to run this stage in isolation, by making sure the architectural prerequisites are satisfied (e.g., networking), and that the Service Account running the stage is granted the appropriate roles. - -### Provider and Terraform variables - -As all other FAST stages, the [mechanism used to pass variable values and pre-built provider files from one stage to the next](../../0-bootstrap/README.md#output-files-and-cross-stage-variables) is also leveraged here. - -The commands to link or copy the provider and terraform variable files can be easily derived from the `stage-links.sh` script in the FAST root folder, passing it a single argument with the local output files folder (if configured) or the GCS output bucket in the automation project (derived from stage 0 outputs). The following examples demonstrate both cases, and the resulting commands that then need to be copy/pasted and run. - -```bash -../../../stage-links.sh ~/fast-config - -# copy and paste the following commands for '3-project-factory' - -ln -s ~/fast-config/providers/3-project-factory-providers.tf ./ -ln -s ~/fast-config/tfvars/0-globals.auto.tfvars.json ./ -ln -s ~/fast-config/tfvars/0-bootstrap.auto.tfvars.json ./ -ln -s ~/fast-config/tfvars/1-resman.auto.tfvars.json ./ -ln -s ~/fast-config/tfvars/2-networking.auto.tfvars.json ./ -ln -s ~/fast-config/tfvars/2-security.auto.tfvars.json ./ -``` - -```bash -../../../stage-links.sh gs://xxx-prod-iac-core-outputs-0 - -# copy and paste the following commands for '3-project-factory' - -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/3-project-factory-providers.tf ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/2-networking.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/2-security.auto.tfvars.json ./ -``` - -If you're not using FAST, refer to the [Variables](#variables) table at the bottom of this document for a full list of variables, their origin (e.g., a stage or specific to this one), and descriptions explaining their meaning. - -Besides the values above, the project factory is driven by YAML data files, with one file per project. Please refer to the underlying [project factory module](../../../../modules/project-factory/) documentation for details on the format. - -Once the configuration is complete, run the project factory with: - -```bash -terraform init -terraform apply -``` - - - -## Files - -| name | description | modules | -|---|---|---| -| [main.tf](./main.tf) | Project factory. | project-factory | -| [outputs.tf](./outputs.tf) | Module outputs. | | -| [variables-fast.tf](./variables-fast.tf) | None | | -| [variables.tf](./variables.tf) | Module variables. | | - -## Variables - -| name | description | type | required | default | producer | -|---|---|:---:|:---:|:---:|:---:| -| [billing_account](variables-fast.tf#L17) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | object({…}) | ✓ | | 0-bootstrap | -| [factories_config](variables.tf#L17) | Path to folder with YAML resource description data files. | object({…}) | ✓ | | | -| [prefix](variables-fast.tf#L30) | Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants. | string | ✓ | | 0-bootstrap | - -## Outputs - -| name | description | sensitive | consumers | -|---|---|:---:|---| -| [projects](outputs.tf#L17) | Created projects. | | | -| [service_accounts](outputs.tf#L27) | Created service accounts. | | | - diff --git a/fast/stages/3-project-factory/dev/diagram.png b/fast/stages/3-project-factory/dev/diagram.png deleted file mode 100644 index b942ea47d..000000000 Binary files a/fast/stages/3-project-factory/dev/diagram.png and /dev/null differ diff --git a/fast/stages/3-project-factory/dev/diagram.svg b/fast/stages/3-project-factory/dev/diagram.svg deleted file mode 100644 index d7821c607..000000000 --- a/fast/stages/3-project-factory/dev/diagram.svg +++ /dev/null @@ -1,1530 +0,0 @@ - - - - - - image/svg+xml - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/fast/stages/README.md b/fast/stages/README.md index 8f5800ed8..b05c0bfc8 100644 --- a/fast/stages/README.md +++ b/fast/stages/README.md @@ -42,12 +42,12 @@ Implemented as an [add-on stage 1](./1-tenant-factory/), with optional FAST comp - Networking ([Peering/VPN](2-networking-a-simple/README.md)/[NVA (w/ optional BGP support)](2-networking-b-nva/README.md)/[Separate environments](2-networking-c-separate-envs/README.md)) Manages centralized network resources in a separate stage, and is typically owned by the networking team. This stage implements a hub-and-spoke design, and includes connectivity via VPN to on-premises, and YAML-based factories for firewall rules (hierarchical and VPC-level) and subnets. It's currently available in four flavors: [spokes connected via VPC peering/VPN](2-networking-a-simple/README.md), [spokes connected via appliances (w/ optional BGP support)](2-networking-b-nva/README.md) and [separated network environments](2-networking-c-separate-envs/README.md).\ Exports: host project ids and numbers, vpc self links +- [Project Factory](./2-project-factory/) + YAML-based factory to create and configure application or team-level projects. Configuration includes VPC-level settings for Shared VPC, service-level configuration for CMEK encryption via centralized keys, and service account creation for workloads and applications. This stage can be cloned if an org-wide or dedicated per-environment factories are needed. ## Environment-level resources (3) - [Networking Security](./3-network-security/) Manages NGFW Enterprise deployment for the production and development environments. -- [Project Factory](./3-project-factory/) - YAML-based factory to create and configure application or team-level projects. Configuration includes VPC-level settings for Shared VPC, service-level configuration for CMEK encryption via centralized keys, and service account creation for workloads and applications. This stage can be cloned if an org-wide or dedicated per-environment factories are needed. - [Data Platform](3-data-platform/dev/) - [GKE Multitenant](3-gke-multitenant/dev/) - [Google Cloud VMware Engine](3-gcve/) diff --git a/fast/stages/diagrams.excalidraw.gz b/fast/stages/diagrams.excalidraw.gz index 676741f0a..5cf62be67 100644 Binary files a/fast/stages/diagrams.excalidraw.gz and b/fast/stages/diagrams.excalidraw.gz differ diff --git a/modules/project-factory/README.md b/modules/project-factory/README.md index 3beef0803..aaa82f78d 100644 --- a/modules/project-factory/README.md +++ b/modules/project-factory/README.md @@ -27,6 +27,7 @@ The code is meant to be executed by a high level service accounts with powerful - [Service accounts](#service-accounts) - [Automation project and resources](#automation-project-and-resources) - [Billing budgets](#billing-budgets) +- [Substitutions in YAML configurations attributes](#substitutions-in-yaml-configurations-attributes) - [Example](#example) - [Files](#files) - [Variables](#variables) @@ -38,12 +39,9 @@ The code is meant to be executed by a high level service accounts with powerful The hierarchy supports up to three levels of folders, which are defined via filesystem directories each including a `_config.yaml` files detailing their attributes. -The hierarchy factory is configured via the `factories_config.hierarchy` variable via one mandatory and one optional argument: +The hierarchy factory is configured via the `factories_config.folders_data_path` variable, which sets the the path containing the YAML definitions for folders. -- `factories_config.hierarchy.folders_data_path` is required to enable the hierarchy factory, and must be set to the path containing the YAML definitions -- `factories_config.hierarchy.parent_ids` is an optional map where keys are arbitrary and values are set to resource node ids - -Top-level folders in the filesystem hierarchy have no explicit parent, so their parent ids need to be provided in the YAML by either referencing the full id (e.g. `folders/12345678`) or by referencing a key in the `parent_ids` attribute described above. As a shortcut, a `default` key can be defined whose value is used for any top-level folder which does not directly provide a parent id. +Parent ids for top-level folders can either be set explicitly (e.g. `folders/12345678`) or via substitutions, by referring to keys in the `context.folder_ids` variable. The special `default` key in the substitutions folder variable is used if present and no folder id/key has been specified in the YAML. Filesystem directories can also contain project definitions in the same YAML format described below. This approach must be used with caution and is best adopted for stable scenarios, as problems in the filesystem hierarchy definitions might result in the project files not being read and the resources being deleted by Terraform. @@ -67,7 +65,7 @@ Some examples on where to use each of the three sets are [provided below](#examp Service accounts can be managed as part of each project's YAML configuration. This allows creation of default service accounts used for GCE instances, in firewall rules, or for application-level credentials without resorting to a separate Terraform configuration. -Each service account is represented by one key and a set of optional key/value pairs in the `service_accounts` top-level YAML map, which expose most of the variables available in the `iam-service-account` module: +Each service account is represented by one key and a set of optional key/value pairs in the `service_accounts` top-level YAML map, which exposes most of the variables available in the `iam-service-account` module: ```yaml service_accounts: @@ -154,6 +152,58 @@ billing_budgets: A simple billing budget example is show in the [example](#example) below. +## Substitutions in YAML configurations attributes + +Substitutions allow referring via short mnemonic names to resources which are either created at runtime, or externally manages. + +This feature has two main benefits: + +- being able to refer to resource ids which cannot be known before creation, for example project automation service accounts in IAM bindings +- making YAML configuration files more easily readable and portable, by using mnemonic keys which are not specific to an organization or project + +One example of both types of substitutions is in this project snippet. The automation service account is used in IAM bindings via its `rw` key, while the parent folder is set by referring to its path in the hierarchy factory. + +```yaml +parent: teams/team-a +iam: + "roles/owner": + - rw +automation: + project: ta-app0-0 + service_accounts: + rw: + description: Read/write automation sa for team a app 0. + buckets: + state: + description: Terraform state bucket for team a app 0. + iam: + roles/storage.objectCreator: + - rw +``` + +Substitutions come from two separate context sources: an internal set for resources managed by the project factory (folders, service accounts, etc.), and an external user-defined set passed in via the `factories_config.context` variable. + +Internal substitutions are: + +- hierarchy folders, used to set project parents via the filesystem path of folders (e.g. `teams/team-a`) +- automation service accounts, used in project IAM bindings via their keys; this does not work in folder IAM bindings + +External substitution are: + +- the map of folder ids in `factories_config.context.folder_ids`, used to set top-level folder parents; the `default` key if present is used when no explicit parent has been set in the YAML file +- the map of IAM principals in `factories_config.context.iam_principals`, used in IAM bindings for folders and projects; the exception is the `iam_by_principals` attribute which uses no interpolation to prevent dynamic cycles +- the map of tag value ids in `factories_config.context.tag_values` used in tag bindings for folders and projects +- the map of Shared VPC host project ids in `factories_config.context.vpc_host_projects` used in service project configurations for projects + +External substitution maps are optional, and there's no harm in not defining them if not used. + +Some caveats on substitutions: + +- project-own service accounts are not part of substitutions to prevent cycles, you can use the `iam_project_roles` and `iam_self_roles` attributes for additive IAM on projects +- project shared vpc configurations and project-own service accounts only support external substitutions to prevent cycles +- projects for automation service accounts and buckets do not support substitutions to prevent cycles +- no substitutions are implemented (yet) for budgets + ## Example The module invocation using all optional features: @@ -177,7 +227,7 @@ module "project-factory" { # always use this contaxt and prefix, regardless of what is in the yaml file data_overrides = { contacts = { - "admin@example.com" = ["ALL"] + "admin@example.org" = ["ALL"] } prefix = "test-pf" } @@ -191,47 +241,69 @@ module "project-factory" { project_id = "foo-billing-audit" type = "email" labels = { - email_address = "gcp-billing-admins@example.com" + email_address = "gcp-billing-admins@example.org" } } } } - hierarchy = { - folders_data_path = "data/hierarchy" - parent_ids = { - default = "folders/12345678" + folders_data_path = "data/hierarchy" + projects_data_path = "data/projects" + context = { + folder_ids = { + default = "folders/5678901234" + teams = "folders/5678901234" + } + iam_principals = { + gcp-devops = "group:gcp-devops@example.org" + } + tag_values = { + "org-policies/drs-allow-all" = "tagValues/123456" + } + vpc_host_projects = { + dev-spoke-0 = "test-pf-dev-net-spoke-0" } } - projects_data_path = "data/projects" } } -# tftest modules=16 resources=70 files=prj-app-1,prj-app-2,prj-app-3,budget-test-100,h-0-0,h-1-0,h-0-1,h-1-1,h-1-1-p0 inventory=example.yaml +# tftest modules=15 resources=56 files=0,1,2,3,4,5,6,7,8 inventory=example.yaml ``` A simple hierarchy of folders: ```yaml -name: Foo (level 1) +name: Team A +# implicit parent definition via 'default' key iam: roles/viewer: - - group:a@example.com -# tftest-file id=h-0-0 path=data/hierarchy/foo/_config.yaml schema=folder.schema.json + - group:team-a-admins@example.org + - gcp-devops +# tftest-file id=0 path=data/hierarchy/team-a/_config.yaml schema=folder.schema.json ``` ```yaml -name: Bar (level 1) -parent: folders/4567890 -# tftest-file id=h-1-0 path=data/hierarchy/bar/_config.yaml schema=folder.schema.json +name: Team B +# explicit parent definition via key +parent: teams +# tftest-file id=1 path=data/hierarchy/team-b/_config.yaml schema=folder.schema.json ``` ```yaml -name: Foo Baz (level 2) -# tftest-file id=h-0-1 path=data/hierarchy/foo/baz/_config.yaml schema=folder.schema.json +name: Team C +# explicit parent definition via folder id +parent: folders/5678901234 +# tftest-file id=2 path=data/hierarchy/team-c/_config.yaml schema=folder.schema.json ``` ```yaml -name: Bar Baz (level 2) -# tftest-file id=h-1-1 path=data/hierarchy/bar/baz/_config.yaml schema=folder.schema.json +name: App 0 +# tftest-file id=3 path=data/hierarchy/team-a/app-0/_config.yaml schema=folder.schema.json +``` + +```yaml +name: App 0 +tag_bindings: + drs-allow-all: org-policies/drs-allow-all +# tftest-file id=4 path=data/hierarchy/team-b/app-0/_config.yaml schema=folder.schema.json ``` One project defined within the folder hierarchy: @@ -241,83 +313,58 @@ billing_account: 012345-67890A-BCDEF0 services: - container.googleapis.com - storage.googleapis.com -# tftest-file id=h-1-1-p0 path=data/hierarchy/bar/baz/bar-baz-iac-0.yaml schema=project.schema.json +# tftest-file id=5 path=data/hierarchy/teams-iac-0.yaml schema=project.schema.json ``` More traditional project definitions via the project factory data: ```yaml -# project app-1 billing_account: 012345-67890A-BCDEF0 labels: - app: app-1 - team: foo -parent: folders/12345678 + app: app-0 + team: team-a +parent: team-a/app-0 service_encryption_key_ids: storage.googleapis.com: - - projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce + - projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce services: - container.googleapis.com - storage.googleapis.com service_accounts: - app-1-be: + app-0-be: + display_name: "Backend instances." + iam_project_roles: + dev-spoke-0: + - roles/compute.networkUser iam_self_roles: - - roles/logging.logWriter - - roles/monitoring.metricWriter + - roles/logging.logWriter + - roles/monitoring.metricWriter + app-0-fe: + display_name: "Frontend instances." iam_project_roles: - my-host-project: - - roles/compute.networkUser - app-1-fe: - display_name: "Test app 1 frontend." - iam_project_roles: - my-host-project: + dev-spoke-0: - roles/compute.networkUser + iam_self_roles: + - roles/logging.logWriter + - roles/monitoring.metricWriter +shared_vpc_service_config: + host_project: dev-spoke-0 + network_users: + - gcp-devops + service_agent_iam: + "roles/container.hostServiceAgentUser": + - container-engine + "roles/compute.networkUser": + - container-engine billing_budgets: - test-100 -# tftest-file id=prj-app-1 path=data/projects/prj-app-1.yaml schema=project.schema.json +# tftest-file id=6 path=data/projects/dev-ta-app0-be.yaml schema=project.schema.json ``` -```yaml -# project app-2 -labels: - app: app-2 - team: foo -parent: folders/12345678 -org_policies: - "compute.restrictSharedVpcSubnetworks": - rules: - - allow: - values: - - projects/foo-host/regions/europe-west1/subnetworks/prod-default-ew1 -service_accounts: - app-2-be: {} -services: -- compute.googleapis.com -- container.googleapis.com -- run.googleapis.com -- storage.googleapis.com -shared_vpc_service_config: - host_project: foo-host - service_agent_iam: - "roles/vpcaccess.user": - - cloudrun - "roles/container.hostServiceAgentUser": - - container-engine - service_agent_subnet_iam: - europe-west1/prod-default-ew1: - - cloudservices - - container-engine - network_subnet_users: - europe-west1/prod-default-ew1: - - group:team-1@example.com - -# tftest-file id=prj-app-2 path=data/projects/prj-app-2.yaml schema=project.schema.json -``` - -This project uses a reference to a hierarchy folder, and defines a controlling project via the `automation` attributes: +This project defines a controlling project via the `automation` attributes: ```yaml -parent: bar/baz +parent: team-b/app-0 services: - run.googleapis.com - storage.googleapis.com @@ -329,28 +376,28 @@ iam: shared_vpc_host_config: enabled: true automation: - project: bar-baz-iac-0 + project: test-pf-teams-iac-0 service_accounts: rw: - description: Read/write automation sa for app example 0. + description: Team B app 0 read/write automation sa. ro: - description: Read-only automation sa for app example 0. + description: Team B app 0 read-only automation sa. buckets: state: - description: Terraform state bucket for app example 0. + description: Team B app 0 Terraform state bucket. iam: roles/storage.objectCreator: - rw roles/storage.objectViewer: + - gcp-devops + - group:team-b-admins@example.org - rw - ro - - group:devops@example.org - -# tftest-file id=prj-app-3 path=data/projects/prj-app-3.yaml schema=project.schema.json +# tftest-file id=7 path=data/projects/dev-tb-app0-0.yaml schema=project.schema.json ``` -And a billing budget: +A billing budget: ```yaml # billing budget test-100 @@ -370,7 +417,7 @@ update_rules: disable_default_iam_recipients: true monitoring_notification_channels: - billing-default -# tftest-file id=budget-test-100 path=data/budgets/test-100.yaml schema=budget.schema.json +# tftest-file id=8 path=data/budgets/test-100.yaml schema=budget.schema.json ``` @@ -392,7 +439,7 @@ update_rules: | name | description | type | required | default | |---|---|:---:|:---:|:---:| -| [factories_config](variables.tf#L96) | Path to folder with YAML resource description data files. | object({…}) | ✓ | | +| [factories_config](variables.tf#L96) | Path to folder with YAML resource description data files. | object({…}) | ✓ | | | [data_defaults](variables.tf#L17) | Optional default values used when corresponding project data from files are missing. | object({…}) | | {} | | [data_merges](variables.tf#L52) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | object({…}) | | {} | | [data_overrides](variables.tf#L71) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | object({…}) | | {} | diff --git a/modules/project-factory/automation.tf b/modules/project-factory/automation.tf index 104739f72..b5ff81451 100644 --- a/modules/project-factory/automation.tf +++ b/modules/project-factory/automation.tf @@ -48,11 +48,11 @@ module "automation-buckets" { prefix = each.value.prefix name = "${each.value.project}-${each.value.name}" encryption_key = lookup(each.value, "encryption_key", null) - # try interpolating service accounts by key in principals iam = { for k, v in lookup(each.value, "iam", {}) : k => [ for vv in v : try( module.automation-service-accounts["${each.value.project}/${vv}"].iam_email, + var.factories_config.context.iam_principals[vv], vv ) ] @@ -62,6 +62,7 @@ module "automation-buckets" { members = [ for vv in v.members : try( module.automation-service-accounts["${each.value.project}/${vv}"].iam_email, + var.factories_config.context.iam_principals[vv], vv ) ] @@ -71,6 +72,7 @@ module "automation-buckets" { for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { member = try( module.automation-service-accounts["${each.value.project}/${v.member}"].iam_email, + var.factories_config.context.iam_principals[v.member], v.member ) }) @@ -96,9 +98,29 @@ module "automation-service-accounts" { "display_name", "Service account ${each.value.name} for ${each.value.project}." ) - iam = lookup(each.value, "iam", {}) - iam_bindings = lookup(each.value, "iam_bindings", {}) - iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) + iam = { + for k, v in lookup(each.value, "iam", {}) : k => [ + for vv in v : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + } + iam_bindings = { + for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { + members = [ + for vv in v.members : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + }) + } + iam_bindings_additive = { + for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { + member = lookup( + var.factories_config.context.iam_principals, v.member, v.member + ) + }) + } iam_billing_roles = lookup(each.value, "iam_billing_roles", {}) iam_folder_roles = lookup(each.value, "iam_folder_roles", {}) iam_organization_roles = lookup(each.value, "iam_organization_roles", {}) diff --git a/modules/project-factory/factory-folders.tf b/modules/project-factory/factory-folders.tf index cd9537f4f..bc9fb812a 100644 --- a/modules/project-factory/factory-folders.tf +++ b/modules/project-factory/factory-folders.tf @@ -18,11 +18,11 @@ locals { _folders_path = try( - pathexpand(var.factories_config.hierarchy.folders_data_path), null + pathexpand(var.factories_config.folders_data_path), null ) _folders = { for f in local._hierarchy_files : dirname(f) => yamldecode(file( - "${coalesce(var.factories_config.hierarchy.folders_data_path, "-")}/${f}" + "${coalesce(var.factories_config.folders_data_path, "-")}/${f}" )) } _hierarchy_files = try( @@ -37,19 +37,8 @@ locals { }) } hierarchy = merge( - try(var.factories_config.hierarchy.parent_ids, {}), { for k, v in module.hierarchy-folder-lvl-1 : k => v.id }, { for k, v in module.hierarchy-folder-lvl-2 : k => v.id }, { for k, v in module.hierarchy-folder-lvl-3 : k => v.id }, ) } - -check "hierarchy-data" { - assert { - condition = ( - var.factories_config.hierarchy == null || - try(var.factories_config.hierarchy.parent_ids.default, null) != null - ) - error_message = "No default set for hierarchy parent ids." - } -} diff --git a/modules/project-factory/factory-projects.tf b/modules/project-factory/factory-projects.tf index 6c66f9066..17e1add5c 100644 --- a/modules/project-factory/factory-projects.tf +++ b/modules/project-factory/factory-projects.tf @@ -21,7 +21,7 @@ locals { { for f in try(fileset(local._folders_path, "**/*.yaml"), []) : basename(trimsuffix(f, ".yaml")) => merge( - { parent = dirname(f) }, + { parent = dirname(f) == "." ? "default" : dirname(f) }, yamldecode(file("${local._folders_path}/${f}")) ) if !endswith(f, "/_config.yaml") diff --git a/modules/project-factory/folders.tf b/modules/project-factory/folders.tf index a846ce273..77199b2d5 100644 --- a/modules/project-factory/folders.tf +++ b/modules/project-factory/folders.tf @@ -16,52 +16,135 @@ # tfdoc:file:description Folder hierarchy factory resources. +locals { + folder_parent_default = try( + var.factories_config.context.folder_ids.default, null + ) +} + module "hierarchy-folder-lvl-1" { source = "../folder" for_each = { for k, v in local.folders : k => v if v.level == 1 } parent = try( # allow the YAML data to set the parent for this level lookup( - var.factories_config.hierarchy.parent_ids, + var.factories_config.context.folder_ids, each.value.parent, - # use the value as is if it's not in the parents map each.value.parent ), # use the default value in the initial parents map - var.factories_config.hierarchy.parent_ids.default + local.folder_parent_default # fail if we don't have an explicit parent ) - name = each.value.name - iam = lookup(each.value, "iam", {}) - iam_bindings = lookup(each.value, "iam_bindings", {}) - iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) - iam_by_principals = lookup(each.value, "iam_by_principals", {}) - org_policies = lookup(each.value, "org_policies", {}) - tag_bindings = lookup(each.value, "tag_bindings", {}) + name = each.value.name + iam = { + for k, v in lookup(each.value, "iam", {}) : k => [ + # don't interpolate automation service account to prevent cycles + for vv in v : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + } + iam_bindings = { + for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { + members = [ + # don't interpolate automation service account to prevent cycles + for vv in v.members : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + }) + } + iam_bindings_additive = { + for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { + # don't interpolate automation service account to prevent cycles + member = lookup( + var.factories_config.context.iam_principals, v.member, v.member + ) + }) + } + iam_by_principals = lookup(each.value, "iam_by_principals", {}) + org_policies = lookup(each.value, "org_policies", {}) + tag_bindings = { + for k, v in lookup(each.value, "tag_bindings", {}) : + k => lookup(var.factories_config.context.tag_values, v, v) + } } module "hierarchy-folder-lvl-2" { - source = "../folder" - for_each = { for k, v in local.folders : k => v if v.level == 2 } - parent = module.hierarchy-folder-lvl-1[each.value.parent_key].id - name = each.value.name - iam = lookup(each.value, "iam", {}) - iam_bindings = lookup(each.value, "iam_bindings", {}) - iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) - iam_by_principals = lookup(each.value, "iam_by_principals", {}) - org_policies = lookup(each.value, "org_policies", {}) - tag_bindings = lookup(each.value, "tag_bindings", {}) + source = "../folder" + for_each = { for k, v in local.folders : k => v if v.level == 2 } + parent = module.hierarchy-folder-lvl-1[each.value.parent_key].id + name = each.value.name + iam = { + for k, v in lookup(each.value, "iam", {}) : k => [ + # don't interpolate automation service account to prevent cycles + for vv in v : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + } + iam_bindings = { + for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { + members = [ + # don't interpolate automation service account to prevent cycles + for vv in v.members : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + }) + } + iam_bindings_additive = { + for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { + # don't interpolate automation service account to prevent cycles + member = lookup( + var.factories_config.context.iam_principals, v.member, v.member + ) + }) + } + iam_by_principals = lookup(each.value, "iam_by_principals", {}) + org_policies = lookup(each.value, "org_policies", {}) + tag_bindings = { + for k, v in lookup(each.value, "tag_bindings", {}) : + k => lookup(var.factories_config.context.tag_values, v, v) + } } module "hierarchy-folder-lvl-3" { - source = "../folder" - for_each = { for k, v in local.folders : k => v if v.level == 3 } - parent = module.hierarchy-folder-lvl-2[each.value.parent_key].id - name = each.value.name - iam = lookup(each.value, "iam", {}) - iam_bindings = lookup(each.value, "iam_bindings", {}) - iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) - iam_by_principals = lookup(each.value, "iam_by_principals", {}) - org_policies = lookup(each.value, "org_policies", {}) - tag_bindings = lookup(each.value, "tag_bindings", {}) + source = "../folder" + for_each = { for k, v in local.folders : k => v if v.level == 3 } + parent = module.hierarchy-folder-lvl-2[each.value.parent_key].id + name = each.value.name + iam = { + for k, v in lookup(each.value, "iam", {}) : k => [ + # don't interpolate automation service account to prevent cycles + for vv in v : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + } + iam_bindings = { + for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { + members = [ + # don't interpolate automation service account to prevent cycles + for vv in v.members : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + }) + } + iam_bindings_additive = { + for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { + # don't interpolate automation service account to prevent cycles + member = lookup( + var.factories_config.context.iam_principals, v.member, v.member + ) + }) + } + iam_by_principals = lookup(each.value, "iam_by_principals", {}) + org_policies = lookup(each.value, "org_policies", {}) + tag_bindings = { + for k, v in lookup(each.value, "tag_bindings", {}) : + k => lookup(var.factories_config.context.tag_values, v, v) + } } diff --git a/modules/project-factory/main.tf b/modules/project-factory/main.tf index d123aa548..c806a7593 100644 --- a/modules/project-factory/main.tf +++ b/modules/project-factory/main.tf @@ -16,13 +16,29 @@ # tfdoc:file:description Projects and billing budgets factory resources. +locals { + context = { + folder_ids = merge( + var.factories_config.context.folder_ids, + local.hierarchy + ) + iam_principals = merge( + var.factories_config.context.iam_principals, + { + for k, v in module.automation-service-accounts : + k => v.iam_email + } + ) + } +} + module "projects" { source = "../project" for_each = local.projects billing_account = each.value.billing_account name = each.key - parent = try( - lookup(local.hierarchy, each.value.parent, each.value.parent), null + parent = lookup( + local.context.folder_ids, each.value.parent, each.value.parent ) prefix = each.value.prefix auto_create_network = try(each.value.auto_create_network, false) @@ -33,11 +49,14 @@ module "projects" { ) default_service_account = try(each.value.default_service_account, "keep") descriptive_name = try(each.value.descriptive_name, null) - # IAM interpolates automation service accounts iam = { for k, v in lookup(each.value, "iam", {}) : k => [ for vv in v : try( - module.automation-service-accounts["${each.key}/${vv}"].iam_email, + # automation service account + local.context.iam_principals["${each.key}/${vv}"], + # other context + local.context.iam_principals[vv], + # passthrough vv ) ] @@ -46,7 +65,11 @@ module "projects" { for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { members = [ for vv in v.members : try( - module.automation-service-accounts["${each.key}/${vv}"].iam_email, + # automation service account + local.context.iam_principals["${each.key}/${vv}"], + # other context + local.context.iam_principals[vv], + # passthrough vv ) ] @@ -55,12 +78,16 @@ module "projects" { iam_bindings_additive = { for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { member = try( - module.automation-service-accounts["${each.key}/${v.member}"].iam_email, + # automation service account + local.context.iam_principals["${each.key}/${v.member}"], + # other context + local.context.iam_principals[v.member], + # passthrough v.member ) }) } - # IAM principals would trigger dynamic key errors so we don't interpolate + # IAM by principals would trigger dynamic key errors so we don't interpolate iam_by_principals = try(each.value.iam_by_principals, {}) labels = merge( each.value.labels, var.data_merges.labels @@ -81,12 +108,27 @@ module "projects" { each.value.services, var.data_merges.services )) - shared_vpc_host_config = each.value.shared_vpc_host_config - shared_vpc_service_config = each.value.shared_vpc_service_config - tag_bindings = merge( - each.value.tag_bindings, - var.data_merges.tag_bindings + shared_vpc_host_config = each.value.shared_vpc_host_config + shared_vpc_service_config = ( + try(each.value.shared_vpc_service_config.host_project, null) == null + ? null + : merge(each.value.shared_vpc_service_config, { + host_project = lookup( + var.factories_config.context.vpc_host_projects, + each.value.shared_vpc_service_config.host_project, + each.value.shared_vpc_service_config.host_project + ) + network_users = [ + for v in try(each.value.shared_vpc_service_config.network_users, []) : + lookup(local.context.iam_principals, v, v) + ] + # TODO: network subnet users + }) ) + tag_bindings = { + for k, v in merge(each.value.tag_bindings, var.data_merges.tag_bindings) : + k => lookup(var.factories_config.context.tag_values, v, v) + } vpc_sc = each.value.vpc_sc } @@ -99,7 +141,10 @@ module "service-accounts" { name = each.value.name display_name = each.value.display_name iam_project_roles = merge( - each.value.iam_project_roles, + { + for k, v in each.value.iam_project_roles : + lookup(var.factories_config.context.vpc_host_projects, k, k) => v + }, each.value.iam_self_roles == null ? {} : { (module.projects[each.value.project].project_id) = each.value.iam_self_roles } diff --git a/modules/project-factory/schemas/folder.schema.json b/modules/project-factory/schemas/folder.schema.json index 85e80c2bd..99bbf727a 100644 --- a/modules/project-factory/schemas/folder.schema.json +++ b/modules/project-factory/schemas/folder.schema.json @@ -115,7 +115,7 @@ "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|ro|rw)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" } } } @@ -132,7 +132,7 @@ "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|ro|rw)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" } }, "role": { @@ -172,7 +172,7 @@ "properties": { "member": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|ro|rw)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" }, "role": { "type": "string", @@ -205,7 +205,7 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|ro|rw)": { + "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])": { "type": "array", "items": { "type": "string", diff --git a/modules/project-factory/schemas/project.schema.json b/modules/project-factory/schemas/project.schema.json index c2d63cbf7..82a2aa16e 100644 --- a/modules/project-factory/schemas/project.schema.json +++ b/modules/project-factory/schemas/project.schema.json @@ -389,7 +389,7 @@ "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|ro|rw)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" } } } @@ -406,7 +406,7 @@ "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|ro|rw)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" } }, "role": { @@ -446,7 +446,7 @@ "properties": { "member": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|ro|rw)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" }, "role": { "type": "string", @@ -479,7 +479,7 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|ro|rw)": { + "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])": { "type": "array", "items": { "type": "string", @@ -561,4 +561,4 @@ } } } -} +} \ No newline at end of file diff --git a/modules/project-factory/variables.tf b/modules/project-factory/variables.tf index 538eb4995..6fc369a58 100644 --- a/modules/project-factory/variables.tf +++ b/modules/project-factory/variables.tf @@ -96,10 +96,7 @@ variable "data_overrides" { variable "factories_config" { description = "Path to folder with YAML resource description data files." type = object({ - hierarchy = optional(object({ - folders_data_path = string - parent_ids = optional(map(string), {}) - })) + folders_data_path = optional(string) projects_data_path = optional(string) budgets = optional(object({ billing_account = string @@ -107,6 +104,13 @@ variable "factories_config" { # TODO: allow defining notification channels via YAML files notification_channels = optional(map(any), {}) })) + context = optional(object({ + # TODO: add KMS keys + folder_ids = optional(map(string), {}) + iam_principals = optional(map(string), {}) + tag_values = optional(map(string), {}) + vpc_host_projects = optional(map(string), {}) + }), {}) }) nullable = false } diff --git a/tests/fast/stages/s1_resman/checklist.yaml b/tests/fast/stages/s1_resman/checklist.yaml index fc32e6aef..2a1c0e3f5 100644 --- a/tests/fast/stages/s1_resman/checklist.yaml +++ b/tests/fast/stages/s1_resman/checklist.yaml @@ -415,18 +415,19 @@ values: timeouts: null counts: - google_folder: 56 - google_folder_iam_binding: 71 - google_organization_iam_member: 7 - google_project_iam_member: 6 - google_service_account: 6 - google_service_account_iam_binding: 6 - google_storage_bucket: 3 - google_storage_bucket_iam_binding: 6 - google_storage_bucket_iam_member: 6 - google_storage_bucket_object: 7 - google_tags_tag_binding: 4 + google_folder: 57 + google_folder_iam_binding: 76 + google_organization_iam_member: 16 + google_project_iam_member: 12 + google_service_account: 12 + google_service_account_iam_binding: 12 + google_storage_bucket: 6 + google_storage_bucket_iam_binding: 12 + google_storage_bucket_iam_member: 12 + google_storage_bucket_object: 13 + google_tags_tag_binding: 5 google_tags_tag_key: 2 google_tags_tag_value: 9 - modules: 66 - resources: 189 + google_tags_tag_value_iam_binding: 2 + modules: 76 + resources: 246 diff --git a/tests/fast/stages/s1_resman/simple.yaml b/tests/fast/stages/s1_resman/simple.yaml index a757afcf6..25823e40d 100644 --- a/tests/fast/stages/s1_resman/simple.yaml +++ b/tests/fast/stages/s1_resman/simple.yaml @@ -12,19 +12,48 @@ # See the License for the specific language governing permissions and # limitations under the License. +values: + module.top-level-folder["teams"].google_folder.folder[0]: + display_name: Teams + parent: organizations/123456789012 + timeouts: null + ? module.top-level-folder["teams"].google_folder_iam_binding.authoritative["organizations/123456789012/roles/xpnServiceAdmin"] + : condition: [] + members: + - serviceAccount:fast2-resman-pf-0@fast-prod-automation.iam.gserviceaccount.com + role: organizations/123456789012/roles/xpnServiceAdmin + module.top-level-folder["teams"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:fast2-resman-pf-0@fast-prod-automation.iam.gserviceaccount.com + role: roles/owner + module.top-level-folder["teams"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderAdmin"]: + condition: [] + members: + - serviceAccount:fast2-resman-pf-0@fast-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderAdmin + module.top-level-folder["teams"].google_folder_iam_binding.authoritative["roles/resourcemanager.projectCreator"]: + condition: [] + members: + - serviceAccount:fast2-resman-pf-0@fast-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.projectCreator + module.top-level-folder["teams"].google_tags_tag_binding.binding["context"]: + timeouts: null + counts: - google_folder: 4 - google_folder_iam_binding: 25 - google_organization_iam_member: 7 - google_project_iam_member: 6 - google_service_account: 6 - google_service_account_iam_binding: 6 - google_storage_bucket: 3 - google_storage_bucket_iam_binding: 6 - google_storage_bucket_iam_member: 6 - google_storage_bucket_object: 7 - google_tags_tag_binding: 4 + google_folder: 5 + google_folder_iam_binding: 30 + google_organization_iam_member: 16 + google_project_iam_member: 12 + google_service_account: 12 + google_service_account_iam_binding: 12 + google_storage_bucket: 6 + google_storage_bucket_iam_binding: 12 + google_storage_bucket_iam_member: 12 + google_storage_bucket_object: 13 + google_tags_tag_binding: 5 google_tags_tag_key: 2 google_tags_tag_value: 9 - modules: 14 - resources: 91 + google_tags_tag_value_iam_binding: 2 + modules: 24 + resources: 148 diff --git a/tests/fast/stages/s2_project_factory/simple.tfvars b/tests/fast/stages/s2_project_factory/simple.tfvars new file mode 100644 index 000000000..29b537332 --- /dev/null +++ b/tests/fast/stages/s2_project_factory/simple.tfvars @@ -0,0 +1,13 @@ +prefix = "test" +billing_account = { + id = "000000-111111-222222" +} +folder_ids = { + teams = "folders/1234567890" +} +groups = { + gcp-devops = "group:gcp-devops@example.org" +} +tag_values = { + "environment/development" = "tagValues/1234567890" +} diff --git a/tests/fast/stages/s2_project_factory/simple.yaml b/tests/fast/stages/s2_project_factory/simple.yaml new file mode 100644 index 000000000..ac5f5dccc --- /dev/null +++ b/tests/fast/stages/s2_project_factory/simple.yaml @@ -0,0 +1,153 @@ +# Copyright 2024 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +tests/fast/stages/s2_project_factory/tftest.yaml values: + module.projects.module.hierarchy-folder-lvl-1["team-a"].google_folder.folder[0]: + display_name: Team A + parent: folders/1234567890 + timeouts: null + module.projects.module.hierarchy-folder-lvl-1["team-b"].google_folder.folder[0]: + display_name: Team B + parent: folders/1234567890 + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-a/dev"].google_folder.folder[0]: + display_name: Development + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-a/dev"].google_tags_tag_binding.binding["environment"]: + tag_value: tagValues/1234567890 + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-a/prod"].google_folder.folder[0]: + display_name: Production + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-a/prod"].google_tags_tag_binding.binding["environment"]: + tag_value: environment/production + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-b/dev"].google_folder.folder[0]: + display_name: Development + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-b/dev"].google_tags_tag_binding.binding["environment"]: + tag_value: tagValues/1234567890 + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-b/prod"].google_folder.folder[0]: + display_name: Production + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-b/prod"].google_tags_tag_binding.binding["environment"]: + tag_value: environment/production + timeouts: null + module.projects.module.projects["dev-ta-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: + deletion_policy: null + host_project: dev-net-spoke-0 + service_project: test-dev-ta-0 + timeouts: null + module.projects.module.projects["dev-ta-0"].google_project.project[0]: + auto_create_network: false + billing_account: 000000-111111-222222 + deletion_policy: DELETE + labels: null + name: test-dev-ta-0 + project_id: test-dev-ta-0 + timeouts: null + module.projects.module.projects["dev-ta-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"]: + condition: [] + member: group:gcp-devops@example.org + project: dev-net-spoke-0 + role: roles/compute.networkUser + module.projects.module.projects["dev-ta-0"].google_project_service.project_services["stackdriver.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-dev-ta-0 + service: stackdriver.googleapis.com + timeouts: null + module.projects.module.projects["dev-tb-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: + deletion_policy: null + host_project: dev-net-spoke-0 + service_project: test-dev-tb-0 + timeouts: null + module.projects.module.projects["dev-tb-0"].google_project.project[0]: + auto_create_network: false + billing_account: 000000-111111-222222 + deletion_policy: DELETE + labels: null + name: test-dev-tb-0 + project_id: test-dev-tb-0 + timeouts: null + module.projects.module.projects["dev-tb-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"]: + condition: [] + member: group:gcp-devops@example.org + project: dev-net-spoke-0 + role: roles/compute.networkUser + module.projects.module.projects["dev-tb-0"].google_project_service.project_services["stackdriver.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-dev-tb-0 + service: stackdriver.googleapis.com + timeouts: null + module.projects.module.projects["prod-ta-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: + deletion_policy: null + host_project: prod-net-spoke-0 + service_project: test-prod-ta-0 + timeouts: null + module.projects.module.projects["prod-ta-0"].google_project.project[0]: + auto_create_network: false + billing_account: 000000-111111-222222 + deletion_policy: DELETE + labels: null + name: test-prod-ta-0 + project_id: test-prod-ta-0 + timeouts: null + module.projects.module.projects["prod-ta-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"]: + condition: [] + member: group:gcp-devops@example.org + project: prod-net-spoke-0 + role: roles/compute.networkUser + module.projects.module.projects["prod-ta-0"].google_project_service.project_services["stackdriver.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-prod-ta-0 + service: stackdriver.googleapis.com + timeouts: null + module.projects.module.projects["prod-tb-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: + deletion_policy: null + host_project: prod-net-spoke-0 + service_project: test-prod-tb-0 + timeouts: null + module.projects.module.projects["prod-tb-0"].google_project.project[0]: + auto_create_network: false + billing_account: 000000-111111-222222 + deletion_policy: DELETE + labels: null + name: test-prod-tb-0 + project_id: test-prod-tb-0 + timeouts: null + module.projects.module.projects["prod-tb-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"]: + condition: [] + member: group:gcp-devops@example.org + project: prod-net-spoke-0 + role: roles/compute.networkUser + module.projects.module.projects["prod-tb-0"].google_project_service.project_services["stackdriver.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-prod-tb-0 + service: stackdriver.googleapis.com + timeouts: null + +counts: + google_compute_shared_vpc_service_project: 4 + google_folder: 6 + google_project: 4 + google_project_iam_member: 4 + google_project_service: 4 + google_tags_tag_binding: 4 + modules: 11 + resources: 26 diff --git a/tests/fast/stages/s3_project_factory/tftest.yaml b/tests/fast/stages/s2_project_factory/tftest.yaml similarity index 93% rename from tests/fast/stages/s3_project_factory/tftest.yaml rename to tests/fast/stages/s2_project_factory/tftest.yaml index 0ee0f70a0..470b47fbc 100644 --- a/tests/fast/stages/s3_project_factory/tftest.yaml +++ b/tests/fast/stages/s2_project_factory/tftest.yaml @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -module: fast/stages/3-project-factory/dev +module: fast/stages/2-project-factory tests: simple: diff --git a/tests/fast/stages/s3_project_factory/simple.tfvars b/tests/fast/stages/s3_project_factory/simple.tfvars deleted file mode 100644 index e2a4a2c09..000000000 --- a/tests/fast/stages/s3_project_factory/simple.tfvars +++ /dev/null @@ -1,10 +0,0 @@ -factories_config = { - projects_data_path = "../../../../tests/fast/stages/s3_project_factory/data/projects/" -} -prefix = "test" -billing_account = { - id = "000000-111111-222222" -} -vpc_self_links = { - dev-spoke-0 = "link" -} diff --git a/tests/modules/project_factory/examples/example.yaml b/tests/modules/project_factory/examples/example.yaml index 266cda152..788f85f0e 100644 --- a/tests/modules/project_factory/examples/example.yaml +++ b/tests/modules/project_factory/examples/example.yaml @@ -13,7 +13,7 @@ # limitations under the License. values: - module.project-factory.module.automation-buckets["prj-app-3/state"].google_storage_bucket.bucket: + module.project-factory.module.automation-buckets["dev-tb-app0-0/state"].google_storage_bucket.bucket: autoclass: - enabled: false cors: [] @@ -26,8 +26,8 @@ values: lifecycle_rule: [] location: EU logging: [] - name: test-pf-prj-app-3-state - project: bar-baz-iac-0 + name: test-pf-dev-tb-app0-0-state + project: test-pf-teams-iac-0 requester_pays: null retention_policy: [] storage_class: MULTI_REGIONAL @@ -35,35 +35,36 @@ values: uniform_bucket_level_access: true versioning: - enabled: false - ? module.project-factory.module.automation-buckets["prj-app-3/state"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectCreator"] - : bucket: test-pf-prj-app-3-state + ? module.project-factory.module.automation-buckets["dev-tb-app0-0/state"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectCreator"] + : bucket: test-pf-dev-tb-app0-0-state condition: [] members: - - serviceAccount:test-pf-prj-app-3-rw@bar-baz-iac-0.iam.gserviceaccount.com + - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com role: roles/storage.objectCreator - ? module.project-factory.module.automation-buckets["prj-app-3/state"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"] - : bucket: test-pf-prj-app-3-state + ? module.project-factory.module.automation-buckets["dev-tb-app0-0/state"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"] + : bucket: test-pf-dev-tb-app0-0-state condition: [] members: - - group:devops@example.org - - serviceAccount:test-pf-prj-app-3-ro@bar-baz-iac-0.iam.gserviceaccount.com - - serviceAccount:test-pf-prj-app-3-rw@bar-baz-iac-0.iam.gserviceaccount.com + - group:gcp-devops@example.org + - group:team-b-admins@example.org + - serviceAccount:test-pf-dev-tb-app0-0-ro@test-pf-teams-iac-0.iam.gserviceaccount.com + - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com role: roles/storage.objectViewer - module.project-factory.module.automation-service-accounts["prj-app-3/ro"].google_service_account.service_account[0]: - account_id: test-pf-prj-app-3-ro + module.project-factory.module.automation-service-accounts["dev-tb-app0-0/ro"].google_service_account.service_account[0]: + account_id: test-pf-dev-tb-app0-0-ro create_ignore_already_exists: null - description: Read-only automation sa for app example 0. + description: Team B app 0 read-only automation sa. disabled: false - display_name: Service account ro for prj-app-3. - project: bar-baz-iac-0 + display_name: Service account ro for dev-tb-app0-0. + project: test-pf-teams-iac-0 timeouts: null - module.project-factory.module.automation-service-accounts["prj-app-3/rw"].google_service_account.service_account[0]: - account_id: test-pf-prj-app-3-rw + module.project-factory.module.automation-service-accounts["dev-tb-app0-0/rw"].google_service_account.service_account[0]: + account_id: test-pf-dev-tb-app0-0-rw create_ignore_already_exists: null - description: Read/write automation sa for app example 0. + description: Team B app 0 read/write automation sa. disabled: false - display_name: Service account rw for prj-app-3. - project: bar-baz-iac-0 + display_name: Service account rw for dev-tb-app0-0. + project: test-pf-teams-iac-0 timeouts: null module.project-factory.module.billing-account[0].google_billing_budget.default["test-100"]: all_updates_rule: @@ -97,407 +98,298 @@ values: enabled: true force_delete: false labels: - email_address: gcp-billing-admins@example.com + email_address: gcp-billing-admins@example.org project: foo-billing-audit sensitive_labels: [] timeouts: null type: email user_labels: null - module.project-factory.module.hierarchy-folder-lvl-1["bar"].google_folder.folder[0]: - display_name: Bar (level 1) - parent: folders/4567890 + module.project-factory.module.hierarchy-folder-lvl-1["team-a"].google_folder.folder[0]: + display_name: Team A + parent: folders/5678901234 timeouts: null - module.project-factory.module.hierarchy-folder-lvl-1["foo"].google_folder.folder[0]: - display_name: Foo (level 1) - parent: folders/12345678 - timeouts: null - module.project-factory.module.hierarchy-folder-lvl-1["foo"].google_folder_iam_binding.authoritative["roles/viewer"]: + module.project-factory.module.hierarchy-folder-lvl-1["team-a"].google_folder_iam_binding.authoritative["roles/viewer"]: condition: [] members: - - group:a@example.com + - group:gcp-devops@example.org + - group:team-a-admins@example.org role: roles/viewer - module.project-factory.module.hierarchy-folder-lvl-2["bar/baz"].google_folder.folder[0]: - display_name: Bar Baz (level 2) + module.project-factory.module.hierarchy-folder-lvl-1["team-b"].google_folder.folder[0]: + display_name: Team B + parent: folders/5678901234 timeouts: null - module.project-factory.module.hierarchy-folder-lvl-2["foo/baz"].google_folder.folder[0]: - display_name: Foo Baz (level 2) + module.project-factory.module.hierarchy-folder-lvl-1["team-c"].google_folder.folder[0]: + display_name: Team C + parent: folders/5678901234 timeouts: null - module.project-factory.module.projects["bar-baz-iac-0"].data.google_storage_project_service_account.gcs_sa[0]: - project: test-pf-bar-baz-iac-0 + module.project-factory.module.hierarchy-folder-lvl-2["team-a/app-0"].google_folder.folder[0]: + display_name: App 0 + timeouts: null + module.project-factory.module.hierarchy-folder-lvl-2["team-b/app-0"].google_folder.folder[0]: + display_name: App 0 + timeouts: null + module.project-factory.module.hierarchy-folder-lvl-2["team-b/app-0"].google_tags_tag_binding.binding["drs-allow-all"]: + tag_value: tagValues/123456 + timeouts: null + module.project-factory.module.projects["dev-ta-app0-be"].data.google_storage_project_service_account.gcs_sa[0]: + project: test-pf-dev-ta-app0-be user_project: null - module.project-factory.module.projects["bar-baz-iac-0"].google_essential_contacts_contact.contact["admin@example.com"]: - email: admin@example.com + module.project-factory.module.projects["dev-ta-app0-be"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: + deletion_policy: null + host_project: test-pf-dev-net-spoke-0 + service_project: test-pf-dev-ta-app0-be + timeouts: null + module.project-factory.module.projects["dev-ta-app0-be"].google_essential_contacts_contact.contact["admin@example.org"]: + email: admin@example.org language_tag: en notification_category_subscriptions: - ALL - parent: projects/test-pf-bar-baz-iac-0 + parent: projects/test-pf-dev-ta-app0-be timeouts: null - module.project-factory.module.projects["bar-baz-iac-0"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: 'DELETE' - effective_labels: - environment: test - labels: - environment: test - name: test-pf-bar-baz-iac-0 - project_id: test-pf-bar-baz-iac-0 - terraform_labels: - environment: test - timeouts: null - module.project-factory.module.projects["bar-baz-iac-0"].google_project_iam_member.service_agents["container-engine-robot"]: - condition: [] - project: test-pf-bar-baz-iac-0 - role: roles/container.serviceAgent - module.project-factory.module.projects["bar-baz-iac-0"].google_project_iam_member.service_agents["gkenode"]: - condition: [] - project: test-pf-bar-baz-iac-0 - role: roles/container.nodeServiceAgent - ? module.project-factory.module.projects["bar-baz-iac-0"].google_project_service.project_services["container.googleapis.com"] - : disable_dependent_services: false - disable_on_destroy: false - project: test-pf-bar-baz-iac-0 - service: container.googleapis.com - timeouts: null - ? module.project-factory.module.projects["bar-baz-iac-0"].google_project_service.project_services["stackdriver.googleapis.com"] - : disable_dependent_services: false - disable_on_destroy: false - project: test-pf-bar-baz-iac-0 - service: stackdriver.googleapis.com - timeouts: null - module.project-factory.module.projects["bar-baz-iac-0"].google_project_service.project_services["storage.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-bar-baz-iac-0 - service: storage.googleapis.com - timeouts: null - ? module.project-factory.module.projects["bar-baz-iac-0"].google_project_service_identity.default["container.googleapis.com"] - : project: test-pf-bar-baz-iac-0 - service: container.googleapis.com - timeouts: null - module.project-factory.module.projects["prj-app-1"].data.google_storage_project_service_account.gcs_sa[0]: - project: test-pf-prj-app-1 - user_project: null - module.project-factory.module.projects["prj-app-1"].google_essential_contacts_contact.contact["admin@example.com"]: - email: admin@example.com - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-prj-app-1 - timeouts: null - ? module.project-factory.module.projects["prj-app-1"].google_kms_crypto_key_iam_member.service_agent_cmek["projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce.gs-project-accounts"] + ? module.project-factory.module.projects["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce.gs-project-accounts"] : condition: [] crypto_key_id: projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce role: roles/cloudkms.cryptoKeyEncrypterDecrypter - module.project-factory.module.projects["prj-app-1"].google_project.project[0]: + module.project-factory.module.projects["dev-ta-app0-be"].google_project.project[0]: auto_create_network: false billing_account: 012345-67890A-BCDEF0 - deletion_policy: 'DELETE' + deletion_policy: DELETE effective_labels: - app: app-1 + app: app-0 environment: test - team: foo - folder_id: '12345678' + team: team-a labels: - app: app-1 + app: app-0 environment: test - team: foo - name: test-pf-prj-app-1 - org_id: null - project_id: test-pf-prj-app-1 + team: team-a + name: test-pf-dev-ta-app0-be + project_id: test-pf-dev-ta-app0-be terraform_labels: - app: app-1 + app: app-0 environment: test - team: foo + team: team-a timeouts: null - module.project-factory.module.projects["prj-app-1"].google_project_iam_member.service_agents["container-engine-robot"]: - condition: [] - project: test-pf-prj-app-1 + ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_iam_member.service_agents["container-engine-robot"] + : condition: [] + project: test-pf-dev-ta-app0-be role: roles/container.serviceAgent - module.project-factory.module.projects["prj-app-1"].google_project_iam_member.service_agents["gkenode"]: + module.project-factory.module.projects["dev-ta-app0-be"].google_project_iam_member.service_agents["gkenode"]: condition: [] - project: test-pf-prj-app-1 + project: test-pf-dev-ta-app0-be role: roles/container.nodeServiceAgent - module.project-factory.module.projects["prj-app-1"].google_project_service.project_services["container.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-prj-app-1 - service: container.googleapis.com - timeouts: null - module.project-factory.module.projects["prj-app-1"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-prj-app-1 - service: stackdriver.googleapis.com - timeouts: null - module.project-factory.module.projects["prj-app-1"].google_project_service.project_services["storage.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-prj-app-1 - service: storage.googleapis.com - timeouts: null - module.project-factory.module.projects["prj-app-1"].google_project_service_identity.default["container.googleapis.com"]: - project: test-pf-prj-app-1 - service: container.googleapis.com - timeouts: null - module.project-factory.module.projects["prj-app-2"].data.google_storage_project_service_account.gcs_sa[0]: - project: test-pf-prj-app-2 - user_project: null - module.project-factory.module.projects["prj-app-2"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: foo-host - service_project: test-pf-prj-app-2 - timeouts: null - ? module.project-factory.module.projects["prj-app-2"].google_compute_subnetwork_iam_member.shared_vpc_host_robots["europe-west1:prod-default-ew1:cloudservices"] + ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] : condition: [] - project: foo-host - region: europe-west1 + member: group:gcp-devops@example.org + project: test-pf-dev-net-spoke-0 role: roles/compute.networkUser - subnetwork: prod-default-ew1 - ? module.project-factory.module.projects["prj-app-2"].google_compute_subnetwork_iam_member.shared_vpc_host_robots["europe-west1:prod-default-ew1:container-engine"] + ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_robots["roles/compute.networkUser:container-engine"] : condition: [] - project: foo-host - region: europe-west1 + project: test-pf-dev-net-spoke-0 role: roles/compute.networkUser - subnetwork: prod-default-ew1 - ? module.project-factory.module.projects["prj-app-2"].google_compute_subnetwork_iam_member.shared_vpc_host_subnets_iam["europe-west1:prod-default-ew1:group:team-1@example.com"] + ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_robots["roles/container.hostServiceAgentUser:container-engine"] : condition: [] - member: group:team-1@example.com - project: foo-host - region: europe-west1 - role: roles/compute.networkUser - subnetwork: prod-default-ew1 - module.project-factory.module.projects["prj-app-2"].google_essential_contacts_contact.contact["admin@example.com"]: - email: admin@example.com - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-prj-app-2 - timeouts: null - ? module.project-factory.module.projects["prj-app-2"].google_org_policy_policy.default["compute.restrictSharedVpcSubnetworks"] - : dry_run_spec: [] - name: projects/test-pf-prj-app-2/policies/compute.restrictSharedVpcSubnetworks - parent: projects/test-pf-prj-app-2 - spec: - - inherit_from_parent: null - reset: null - rules: - - allow_all: null - condition: [] - deny_all: null - enforce: null - values: - - allowed_values: - - projects/foo-host/regions/europe-west1/subnetworks/prod-default-ew1 - denied_values: null - timeouts: null - module.project-factory.module.projects["prj-app-2"].google_project.project[0]: - auto_create_network: false - billing_account: 123456-123456-123456 - deletion_policy: 'DELETE' - effective_labels: - app: app-2 - environment: test - team: foo - folder_id: '12345678' - labels: - app: app-2 - environment: test - team: foo - name: test-pf-prj-app-2 - org_id: null - project_id: test-pf-prj-app-2 - terraform_labels: - app: app-2 - environment: test - team: foo - timeouts: null - module.project-factory.module.projects["prj-app-2"].google_project_iam_member.service_agents["compute-system"]: - condition: [] - project: test-pf-prj-app-2 - role: roles/compute.serviceAgent - module.project-factory.module.projects["prj-app-2"].google_project_iam_member.service_agents["container-engine-robot"]: - condition: [] - project: test-pf-prj-app-2 - role: roles/container.serviceAgent - module.project-factory.module.projects["prj-app-2"].google_project_iam_member.service_agents["gkenode"]: - condition: [] - project: test-pf-prj-app-2 - role: roles/container.nodeServiceAgent - module.project-factory.module.projects["prj-app-2"].google_project_iam_member.service_agents["serverless-robot-prod"]: - condition: [] - project: test-pf-prj-app-2 - role: roles/run.serviceAgent - ? module.project-factory.module.projects["prj-app-2"].google_project_iam_member.shared_vpc_host_robots["roles/container.hostServiceAgentUser:container-engine"] - : condition: [] - project: foo-host + project: test-pf-dev-net-spoke-0 role: roles/container.hostServiceAgentUser - ? module.project-factory.module.projects["prj-app-2"].google_project_iam_member.shared_vpc_host_robots["roles/vpcaccess.user:cloudrun"] - : condition: [] - project: foo-host - role: roles/vpcaccess.user - module.project-factory.module.projects["prj-app-2"].google_project_service.project_services["compute.googleapis.com"]: - disable_dependent_services: false + ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_service.project_services["container.googleapis.com"] + : disable_dependent_services: false disable_on_destroy: false - project: test-pf-prj-app-2 - service: compute.googleapis.com - timeouts: null - module.project-factory.module.projects["prj-app-2"].google_project_service.project_services["container.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-prj-app-2 + project: test-pf-dev-ta-app0-be service: container.googleapis.com timeouts: null - module.project-factory.module.projects["prj-app-2"].google_project_service.project_services["run.googleapis.com"]: - disable_dependent_services: false + ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_service.project_services["stackdriver.googleapis.com"] + : disable_dependent_services: false disable_on_destroy: false - project: test-pf-prj-app-2 - service: run.googleapis.com - timeouts: null - module.project-factory.module.projects["prj-app-2"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-prj-app-2 + project: test-pf-dev-ta-app0-be service: stackdriver.googleapis.com timeouts: null - module.project-factory.module.projects["prj-app-2"].google_project_service.project_services["storage.googleapis.com"]: + module.project-factory.module.projects["dev-ta-app0-be"].google_project_service.project_services["storage.googleapis.com"]: disable_dependent_services: false disable_on_destroy: false - project: test-pf-prj-app-2 + project: test-pf-dev-ta-app0-be service: storage.googleapis.com timeouts: null - module.project-factory.module.projects["prj-app-2"].google_project_service_identity.default["container.googleapis.com"]: - project: test-pf-prj-app-2 + ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_service_identity.default["container.googleapis.com"] + : project: test-pf-dev-ta-app0-be service: container.googleapis.com timeouts: null - module.project-factory.module.projects["prj-app-2"].google_project_service_identity.default["run.googleapis.com"]: - project: test-pf-prj-app-2 - service: run.googleapis.com - timeouts: null - module.project-factory.module.projects["prj-app-3"].data.google_storage_project_service_account.gcs_sa[0]: - project: test-pf-prj-app-3 + module.project-factory.module.projects["dev-tb-app0-0"].data.google_storage_project_service_account.gcs_sa[0]: + project: test-pf-dev-tb-app0-0 user_project: null - module.project-factory.module.projects["prj-app-3"].google_compute_shared_vpc_host_project.shared_vpc_host[0]: - project: test-pf-prj-app-3 + module.project-factory.module.projects["dev-tb-app0-0"].google_compute_shared_vpc_host_project.shared_vpc_host[0]: + project: test-pf-dev-tb-app0-0 timeouts: null - module.project-factory.module.projects["prj-app-3"].google_essential_contacts_contact.contact["admin@example.com"]: - email: admin@example.com + module.project-factory.module.projects["dev-tb-app0-0"].google_essential_contacts_contact.contact["admin@example.org"]: + email: admin@example.org language_tag: en notification_category_subscriptions: - ALL - parent: projects/test-pf-prj-app-3 + parent: projects/test-pf-dev-tb-app0-0 timeouts: null - module.project-factory.module.projects["prj-app-3"].google_project.project[0]: + module.project-factory.module.projects["dev-tb-app0-0"].google_project.project[0]: auto_create_network: false billing_account: 123456-123456-123456 - deletion_policy: 'DELETE' + deletion_policy: DELETE effective_labels: environment: test labels: environment: test - name: test-pf-prj-app-3 - project_id: test-pf-prj-app-3 + name: test-pf-dev-tb-app0-0 + project_id: test-pf-dev-tb-app0-0 terraform_labels: environment: test timeouts: null - module.project-factory.module.projects["prj-app-3"].google_project_iam_binding.authoritative["roles/owner"]: + module.project-factory.module.projects["dev-tb-app0-0"].google_project_iam_binding.authoritative["roles/owner"]: condition: [] members: - - serviceAccount:test-pf-prj-app-3-rw@bar-baz-iac-0.iam.gserviceaccount.com - project: test-pf-prj-app-3 + - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com + project: test-pf-dev-tb-app0-0 role: roles/owner - module.project-factory.module.projects["prj-app-3"].google_project_iam_binding.authoritative["roles/viewer"]: + module.project-factory.module.projects["dev-tb-app0-0"].google_project_iam_binding.authoritative["roles/viewer"]: condition: [] members: - - serviceAccount:test-pf-prj-app-3-ro@bar-baz-iac-0.iam.gserviceaccount.com - project: test-pf-prj-app-3 + - serviceAccount:test-pf-dev-tb-app0-0-ro@test-pf-teams-iac-0.iam.gserviceaccount.com + project: test-pf-dev-tb-app0-0 role: roles/viewer - module.project-factory.module.projects["prj-app-3"].google_project_iam_member.service_agents["serverless-robot-prod"]: + module.project-factory.module.projects["dev-tb-app0-0"].google_project_iam_member.service_agents["serverless-robot-prod"]: condition: [] - project: test-pf-prj-app-3 + project: test-pf-dev-tb-app0-0 role: roles/run.serviceAgent - module.project-factory.module.projects["prj-app-3"].google_project_service.project_services["run.googleapis.com"]: + module.project-factory.module.projects["dev-tb-app0-0"].google_project_service.project_services["run.googleapis.com"]: disable_dependent_services: false disable_on_destroy: false - project: test-pf-prj-app-3 + project: test-pf-dev-tb-app0-0 service: run.googleapis.com timeouts: null - module.project-factory.module.projects["prj-app-3"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false + ? module.project-factory.module.projects["dev-tb-app0-0"].google_project_service.project_services["stackdriver.googleapis.com"] + : disable_dependent_services: false disable_on_destroy: false - project: test-pf-prj-app-3 + project: test-pf-dev-tb-app0-0 service: stackdriver.googleapis.com timeouts: null - module.project-factory.module.projects["prj-app-3"].google_project_service.project_services["storage.googleapis.com"]: + module.project-factory.module.projects["dev-tb-app0-0"].google_project_service.project_services["storage.googleapis.com"]: disable_dependent_services: false disable_on_destroy: false - project: test-pf-prj-app-3 + project: test-pf-dev-tb-app0-0 service: storage.googleapis.com timeouts: null - module.project-factory.module.projects["prj-app-3"].google_project_service_identity.default["run.googleapis.com"]: - project: test-pf-prj-app-3 + module.project-factory.module.projects["dev-tb-app0-0"].google_project_service_identity.default["run.googleapis.com"]: + project: test-pf-dev-tb-app0-0 service: run.googleapis.com timeouts: null - ? module.project-factory.module.service-accounts["prj-app-1/app-1-be"].google_project_iam_member.project-roles["my-host-project-roles/compute.networkUser"] + module.project-factory.module.projects["teams-iac-0"].data.google_storage_project_service_account.gcs_sa[0]: + project: test-pf-teams-iac-0 + user_project: null + module.project-factory.module.projects["teams-iac-0"].google_essential_contacts_contact.contact["admin@example.org"]: + email: admin@example.org + language_tag: en + notification_category_subscriptions: + - ALL + parent: projects/test-pf-teams-iac-0 + timeouts: null + module.project-factory.module.projects["teams-iac-0"].google_project.project[0]: + auto_create_network: false + billing_account: 012345-67890A-BCDEF0 + deletion_policy: DELETE + effective_labels: + environment: test + folder_id: '5678901234' + labels: + environment: test + name: test-pf-teams-iac-0 + org_id: null + project_id: test-pf-teams-iac-0 + terraform_labels: + environment: test + timeouts: null + module.project-factory.module.projects["teams-iac-0"].google_project_iam_member.service_agents["container-engine-robot"]: + condition: [] + project: test-pf-teams-iac-0 + role: roles/container.serviceAgent + module.project-factory.module.projects["teams-iac-0"].google_project_iam_member.service_agents["gkenode"]: + condition: [] + project: test-pf-teams-iac-0 + role: roles/container.nodeServiceAgent + module.project-factory.module.projects["teams-iac-0"].google_project_service.project_services["container.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-pf-teams-iac-0 + service: container.googleapis.com + timeouts: null + ? module.project-factory.module.projects["teams-iac-0"].google_project_service.project_services["stackdriver.googleapis.com"] + : disable_dependent_services: false + disable_on_destroy: false + project: test-pf-teams-iac-0 + service: stackdriver.googleapis.com + timeouts: null + module.project-factory.module.projects["teams-iac-0"].google_project_service.project_services["storage.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-pf-teams-iac-0 + service: storage.googleapis.com + timeouts: null + module.project-factory.module.projects["teams-iac-0"].google_project_service_identity.default["container.googleapis.com"]: + project: test-pf-teams-iac-0 + service: container.googleapis.com + timeouts: null + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-net-spoke-0-roles/compute.networkUser"] : condition: [] - project: my-host-project + project: test-pf-dev-net-spoke-0 role: roles/compute.networkUser - ? module.project-factory.module.service-accounts["prj-app-1/app-1-be"].google_project_iam_member.project-roles["test-pf-prj-app-1-roles/logging.logWriter"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/logging.logWriter"] : condition: [] - project: test-pf-prj-app-1 + project: test-pf-dev-ta-app0-be role: roles/logging.logWriter - ? module.project-factory.module.service-accounts["prj-app-1/app-1-be"].google_project_iam_member.project-roles["test-pf-prj-app-1-roles/monitoring.metricWriter"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/monitoring.metricWriter"] : condition: [] - project: test-pf-prj-app-1 + project: test-pf-dev-ta-app0-be role: roles/monitoring.metricWriter - module.project-factory.module.service-accounts["prj-app-1/app-1-be"].google_service_account.service_account[0]: - account_id: app-1-be + module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_service_account.service_account[0]: + account_id: app-0-be create_ignore_already_exists: null description: null disabled: false - display_name: Terraform-managed. - project: test-pf-prj-app-1 + display_name: Backend instances. + project: test-pf-dev-ta-app0-be timeouts: null - ? module.project-factory.module.service-accounts["prj-app-1/app-1-fe"].google_project_iam_member.project-roles["my-host-project-roles/compute.networkUser"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["test-pf-dev-net-spoke-0-roles/compute.networkUser"] : condition: [] - project: my-host-project + project: test-pf-dev-net-spoke-0 role: roles/compute.networkUser - module.project-factory.module.service-accounts["prj-app-1/app-1-fe"].google_service_account.service_account[0]: - account_id: app-1-fe + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/logging.logWriter"] + : condition: [] + project: test-pf-dev-ta-app0-be + role: roles/logging.logWriter + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/monitoring.metricWriter"] + : condition: [] + project: test-pf-dev-ta-app0-be + role: roles/monitoring.metricWriter + module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_service_account.service_account[0]: + account_id: app-0-fe create_ignore_already_exists: null description: null disabled: false - display_name: Test app 1 frontend. - project: test-pf-prj-app-1 - timeouts: null - module.project-factory.module.service-accounts["prj-app-2/app-2-be"].google_service_account.service_account[0]: - account_id: app-2-be - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - project: test-pf-prj-app-2 + display_name: Frontend instances. + project: test-pf-dev-ta-app0-be timeouts: null counts: google_billing_budget: 1 google_compute_shared_vpc_host_project: 1 google_compute_shared_vpc_service_project: 1 - google_compute_subnetwork_iam_member: 3 - google_essential_contacts_contact: 4 - google_folder: 4 + google_essential_contacts_contact: 3 + google_folder: 5 google_folder_iam_binding: 1 google_kms_crypto_key_iam_member: 1 google_monitoring_notification_channel: 1 - google_org_policy_policy: 1 - google_project: 4 + google_project: 3 google_project_iam_binding: 2 - google_project_iam_member: 15 - google_project_service: 14 - google_project_service_identity: 5 - google_service_account: 5 + google_project_iam_member: 14 + google_project_service: 9 + google_project_service_identity: 3 + google_service_account: 4 google_storage_bucket: 1 google_storage_bucket_iam_binding: 2 - google_storage_project_service_account: 4 - modules: 16 - resources: 70 + google_storage_project_service_account: 3 + google_tags_tag_binding: 1 + modules: 15 + resources: 56