diff --git a/fast/README.md b/fast/README.md index 8d1f7b2ae..2881bb19f 100644 --- a/fast/README.md +++ b/fast/README.md @@ -36,14 +36,10 @@ FAST uses YAML-based factories to deploy subnets and firewall rules and, as its ### CI/CD -One of our objectives with FAST is to provide a lightweight reference design for the IaC repositories, and a built-in implementation for running our code in automated pipelines. Our CI/CD approach leverages [Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation), and provides sample workflow configurations for several major providers. Refer to the [CI/CD section in the bootstrap stage](./stages/0-bootstrap/README.md#cicd) for more details. We also provide separate [optional small stages](./extras/) to help you configure your CI/CD provider. +One of our objectives with FAST is to provide a lightweight reference design for the IaC repositories, and a built-in implementation for running our code in automated pipelines. Our CI/CD approach leverages [Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation), and provides sample workflow configurations for several major providers. Refer to the [CI/CD section in the bootstrap stage](./stages/0-bootstrap/README.md#cicd-configuration) for more details. We also provide separate [optional small stages](./extras/) to help you configure your CI/CD provider. -### Multitenant organizations - -FAST has built-in support for multitenancy implemented in [an add-on stage](./addons/1-resman-tenants/). Tenants can optionally be created with FAST compatibility, allowing them independent use of stages 1+ in their own context. - ## Implementation There are many decisions and tasks required to convert an empty GCP organization to one that can host production environments safely. Arguably, FAST could expose those decisions as configuration options to allow for different outcomes. However, supporting all the possible combinations is almost impossible and leads to code which is hard to maintain efficiently. diff --git a/fast/addons/1-resman-tenants/.fast-stage.env b/fast/addons/1-resman-tenants/.fast-stage.env deleted file mode 100644 index 2feb424dd..000000000 --- a/fast/addons/1-resman-tenants/.fast-stage.env +++ /dev/null @@ -1,5 +0,0 @@ -FAST_STAGE_DESCRIPTION="tenant factory" -FAST_STAGE_LEVEL=1 -FAST_STAGE_NAME=tenant-factory -FAST_STAGE_DEPS="0-globals 0-bootstrap" -# FAST_STAGE_OPTIONAL="" diff --git a/fast/addons/1-resman-tenants/README.md b/fast/addons/1-resman-tenants/README.md deleted file mode 100644 index 4a16731da..000000000 --- a/fast/addons/1-resman-tenants/README.md +++ /dev/null @@ -1,350 +0,0 @@ -# Tenant Factory Resource Manager Add-on - -This add-on implements multitenancy on top of the resource management stage, where a limited number of tenants need a high degree of autonomy over their slice of the shared organization, while still being subject to a measure of central control. - -Typical use cases include large organizations managing a single Cloud subscription for multiple semi-independent entities (governments, state-wide associations), multinational groups with different local subsidiaries, or even business units who own their cloud presence while still consuming centralized resources or services. - - -- [Design overview and choices](#design-overview-and-choices) - - [Regular tenants](#regular-tenants) - - [FAST-compatible tenants](#fast-compatible-tenants) -- [How to run this stage](#how-to-run-this-stage) - - [Provider and Terraform variables](#provider-and-terraform-variables) - - [Impersonating the automation service account](#impersonating-the-automation-service-account) - - [Variable configuration](#variable-configuration) - - [Running the stage](#running-the-stage) - - [Organization policy errors](#organization-policy-errors) -- [Tenant configuration](#tenant-configuration) - - [Configurations for both simple and FAST tenants](#configurations-for-both-simple-and-fast-tenants) - - [Configurations for FAST tenants](#configurations-for-fast-tenants) - - [Deploying FAST stages](#deploying-fast-stages) -- [Files](#files) -- [Variables](#variables) -- [Outputs](#outputs) - - -## Design overview and choices - -Our tenant design creates two folders per tenant: - -- a higher level folder under central control, where services specific for the tenant but not controlled by them can be created (log sinks, shared networking connections) -- a lower level folder under tenant control, where their projects and services can be created - -Each tenant can optionally: - -- use a separate billing account -- use a separate Cloud Identity / Workspace -- be configured for full FAST compatibility, to allow independent deployment of a FAST Landing Zone in their environment - -This stage is configured as a factory and allows managing multiple tenants together. When a tenant is configured in FAST compatible mode, this stage effectively acts as its bootstrap stage. - -The following is a high level diagram of this stage design. - -![Stage diagram](diagram.png) - - - -### Regular tenants - -Where FAST compatibility is not needed this stage creates minimal tenant environments, configuring the minimum amount of resources to allow them to operate independently: - -- a centrally-managed folder with - - one log sink to export audit-related tenant events - - DRS organization policy configuration to allow the tenants's own Cloud Identity in IAM policies (if one is used) -- a minimal set of automation resources (service account, bucket) in the organization-level IaC project -- a tenant-managed folder with IAM roles assigned to the tenant administrators principal and the automation service account -- an optional VPC-SC policy scoped to the tenant folder and managed by the tenant - -This allows quick bootstrapping of a large number of tenants which are either self-managed or which use customized IaC code. - -Tenants of this type can be "upgraded" at any time to FAST compatibility by simply extending their configuration. - -### FAST-compatible tenants - -Tenants can also be configured for FAST compatibility. This approach effectively emulates the org-level bootstrap stage, allowing tenants to independently bring up a complete Landing Zone in their environment using FAST. - -The main differences compared to organization-level FAST are: - -- no bootstrap service account is created for tenants, as this stage is their effective bootstrap -- tenant-mamaged log sinks are configured in stage 1, since their bootstrap stage (this one) is under central control -- secure tags are created in the tenant automation project since tenants cannot operate at the organization level -- tenants cannot self-manage organization policies on their folder (this might change in a future release) - -While this stage's approach to organization policies is to keep them under centralized management, it's still possible to allow tenants limited or full control over organization policies by either - -- assigning them permissions on secure tags used in policy conditions, or -- assignign them organization policy admin permissions on the organization, with a condition based on the secure tag value bound to their folder - -Once a FAST-enabled tenant is created, the admin principal for the tenant has access to a dedicated resource management service account and set of input files (provider, tfvars) and can then proceed to setup FAST using the regular stage 1. - -## How to run this stage - -This stage is designed as an add-on to the [resource management](../../stages/1-resman/README.md) stage, and reuses its IaC resources and IAM configurations. - -Once the bootstrap and resource management stages are applied, configure the bootstrap stage `fast_addon` variable to enable this stage, as explained in the [add-ons documentation](../README.md). - -### Provider and Terraform variables - -As all other FAST stages, the [mechanism used to pass variable values and pre-built provider files from one stage to the next](../../stages/0-bootstrap/README.md#output-files-and-cross-stage-variables) is also leveraged here. - -The commands to link or copy the provider and terraform variable files can be easily derived from the `fast-links.sh` script in the FAST stages folder, passing it a single argument with the local output files folder (if configured) or the GCS output bucket in the automation project (derived from stage 0 outputs). The following examples demonstrate both cases, and the resulting commands that then need to be copy/pasted and run. - -```bash -../fast-links.sh ~/fast-config - -# File linking commands for tenant factory stage - -# provider file -ln -s ~/fast-config/fast-test-00/providers/1-resman-providers.tf ./ - -# input files from other stages -ln -s ~/fast-config/fast-test-00/tfvars/0-globals.auto.tfvars.json ./ -ln -s ~/fast-config/fast-test-00/tfvars/0-bootstrap.auto.tfvars.json ./ - -# conventional place for stage tfvars (manually created) -ln -s ~/fast-config/fast-test-00/1-tenant-factory.auto.tfvars ./ -``` - -```bash -../fast-links.sh gs://xxx-prod-iac-core-outputs-0 - -# File linking commands for tenant factory stage - -# provider file -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/1-resman-providers.tf ./ - -# input files from other stages -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./ - -# conventional place for stage tfvars (manually created) -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/1-tenant-factory.auto.tfvars ./ -``` - -### Impersonating the automation service account - -The preconfigured provider file uses impersonation to run with this stage's automation service account's credentials. The `gcp-devops` and `organization-admins` groups have the necessary IAM bindings in place to do that, so make sure the current user is a member of one of those groups. - -### Variable configuration - -Variables in this stage -- like most other FAST stages -- are broadly divided into three separate sets: - -- variables which refer to global values for the whole organization (org id, billing account id, prefix, etc.), which are pre-populated via the `0-globals.auto.tfvars.json` file linked or copied above -- variables which refer to resources managed by previous stages, which are prepopulated here via the `0-bootstrap.auto.tfvars.json` file linked or copied above -- and finally variables that optionally control this stage's behaviour and customizations, and should be defined in a custom `1-tenant-factory.auto.tfvars` file - -The latter set is explained in the [Tenant configuration](#tenant-configuration) section below, and the full list can be found in the [Variables](#variables) table at the bottom of this document. - -Note that the `outputs_location` variable is disabled by default, if you want output files to be generated by this stage you need to explicitly set it in your `tfvars` file like this: - -```tfvars -outputs_location = "~/fast-config" -``` - -For additional details on output files and how they are used, refer to the [bootstrap stage documentation](../../stages/0-bootstrap/README.md#output-files-and-cross-stage-variables). - -### Running the stage - -Once provider and variable values are in place and the correct user is configured, the stage can be run: - -```bash -terraform init -terraform apply -``` - -#### Organization policy errors - -If you get an organization policy error assigning IAM roles or setting essential contacts on tenant-level resources, make sure the tenant configuration contains the right customer id and domain in the `cloud_identity` attributes, and the administrative principals and essential contacts for the tenant belong to the right Cloud Identity. - -If both are correct, wait a couple of minutes for the organization policies to be enforced and retry. Remember to also check the organization-level IaC project org policies, which can be customized via the bootstrap stage variables. - -## Tenant configuration - -This stage has only three variables that can be customized: - -- `root_node` specifies the top-level folder under which all tenant folders are created; if it's not specified (the default) tenants are created directly under the organization -- `tag_names.tenant` defines the name of the tag key used to hold one tag value per tenant, and defaults to `"tenant"` -- `tenant_configs` is a map containing the configuration for each tenant, and is explained below - -### Configurations for both simple and FAST tenants - -A small number of attributes can be configured for each tenant in `tenant_configs` regardless of its type (simple or FAST-enabled). - -The key in the tenant map is used as the tenant shortname, and should be selected with care as it becomes part of resource names. If the tenant plans on using FAST stages, the total combined length of string `{fast-prefix}-{tenant-shortname}` should not exceed 11 characters combined, unless a custom prefix is also defined for the tenant. - -`admin_principal` is a IAM-format principal (e.g. `"group:tenant-admins@example.org"`) which is assigned administrative permissions on the tenant environment, and impersonation permissions on the automation service account. - -`descriptive_name` is the name used for the tenant folder, and in some resource descriptions. - -`billing_account` is optional and defaults to the organization billing account if not specified. If a custom billing account is used by the tenant, set its id in `billing_account.id`. When a custom billing account is used, this stage can optionally manage billing account permissions for tenant principals and service accounts by setting `billing_account.no_iam` to `false`. By default IAM is not managed for external billing accounts. - -`cloud_identity` is optional and defaults to the organization Cloud Identity instance if not specified. If the tenant manages users and group via a separate Cloud Identity, set its configuration in this attribute. - -`locations` is optional and allows overriding the organization-level locations. It is only really meaningful for FAST-enabled tenants, where this field is used for the locations of automation and log-related resources (GCS, log buckets, etc.). - -`vpc_sc_policy_create` is optional and when `true` creates a VPC-SC policy for the tenant scoped to its folder, assigning administrative permissions on it to the tenant's admin principal and service account. - -This is an example of two simple non-FAST enabled tenants: - -```hcl -root_node = "folders/1234567890" -tenant_configs = { - s0 = { - admin_principal = "group:gcp-admins@s0.example.org" - billing_account = { - id = "0123456-0123456-0123456" - no_iam = false - } - descriptive_name = "Simple 0" - cloud_identity = { - customer_id = "CCC000CCC" - domain = "s0.example.org" - id = 1234567890 - } - vpc_sc_policy_create = true - } - s1 = { - admin_principal = "group:s1-admins@example.org" - descriptive_name = "Simple 1" - } -} -``` - -### Configurations for FAST tenants - -FAST compatibility is enabled for a tenant by defining the `fast_config` attribute in their configuration, in addition to the attributes outlined above. - -The `fast_config` attributes control the FAST bootstrap emulation for a tenant, and behave in a similar way to the corresponding variables that control the [bootstrap stage](../../stages/0-bootstrap/README.md#variables). They are all optional, and their behaviour is explained in the bootstrap stage documentation. - -This is an example of two FAST-enabled tenants: - -```hcl -tenant_configs = { - f0 = { - admin_principal = "group:gcp-admins@f0.example.org" - billing_account = { - # implicit use of org-level BA with IAM roles - no_iam = false - } - descriptive_name = "Fast 0" - cloud_identity = { - customer_id = "CdCdCdCd" - domain = "f0.example.org" - id = 1234567890 - } - fast_config = { - groups = { - gcp-network-admins = "gcp-network-admins" - } - cicd_config = { - identity_provider = "github" - name = "ExampleF0/resman" - type = "github" - branch = "main" - } - workload_identity_providers = { - github = { - attribute_condition = "attribute.repository_owner==\"foobar\"" - issuer = "github" - } - } - } - vpc_sc_policy_create = true - } - f1 = { - admin_principal = "group:f1-admins@example.org" - # implicit use of org-level BA without IAM roles - descriptive_name = "Fast 1" - # implicit use of org-level Cloud Identity - groups = { - gcp-billing-admins ="f1-gcp-billing-admins" - gcp-devops ="f1-gcp-devops" - gcp-network-admins ="f1-gcp-vpc-network-admins" - gcp-organization-admins ="f1-gcp-organization-admins" - gcp-security-admins ="f1-gcp-security-admins" - gcp-support ="f1-gcp-devops" - } - } -} -``` - -#### Deploying FAST stages - -Mirroring the regular FAST behavior, the provider and variable files for a bootstrapped tenant will be generated on a tenant-specific storage bucket named `{prefix}-{tenant-shortname}-prod-iac-core-outputs-0` in (also tenant-specific) project `{prefix}-{tenant-shortname}-prod-iac-core-0`. - -Since the tenant is already bootstrapped, a FAST deployment for tenants start from stage `1-resman`, which can be configured as usual, leveraging `stage-links.sh`, which should point to either the tenant-specific `var.outputs_location`, or to the tenant-specific GCS bucket. - -For example: - -```bash -/path/to/stage-links.sh ~/fast-config/tenants/tenant-a - -# copy and paste the following commands for 'tenant-a/1-resman' - -ln -s ~/fast-config/tenants/tenant-a/providers/1-tenant-factory-providers.tf ./ -ln -s ~/fast-config/tenants/tenant-a/tfvars/0-globals.auto.tfvars.json ./ -ln -s ~/fast-config/tenants/tenant-a/tfvars/0-bootstrap.auto.tfvars.json ./ -``` - -```bash -/path/to/stage-links.sh gs://{prefix}-{tenant-shortname}-prod-iac-core-0 - -# copy and paste the following commands for 'tenant-a/1-resman' - -gcloud storage cp gs://{prefix}-{tenant-shortname}-prod-iac-core-0/providers/1-tenant-factory-providers.tf ./ -gcloud storage cp gs://{prefix}-{tenant-shortname}-prod-iac-core-0/tfvars/0-globals.auto.tfvars.json ./ -gcloud storage cp gs://{prefix}-{tenant-shortname}-prod-iac-core-0/tfvars/0-bootstrap.auto.tfvars.json ./ -``` - - - -## Files - -| name | description | modules | resources | -|---|---|---|---| -| [identity-providers-defs.tf](./identity-providers-defs.tf) | Identity provider definitions. | | | -| [main.tf](./main.tf) | Module-level locals and resources. | organization | | -| [outputs-files.tf](./outputs-files.tf) | Output files persistence to local filesystem. | | local_file | -| [outputs-gcs.tf](./outputs-gcs.tf) | Output files persistence to automation GCS bucket. | | google_storage_bucket_object | -| [outputs.tf](./outputs.tf) | Module outputs. | | | -| [tenant-billing-iam.tf](./tenant-billing-iam.tf) | Per-tenant billing IAM. | billing-account · organization | | -| [tenant-core.tf](./tenant-core.tf) | Per-tenant centrally managed resources. | folder · logging-bucket | | -| [tenant-fast-automation.tf](./tenant-fast-automation.tf) | Per-tenant FAST bootstrap emulation (automation). | gcs · iam-service-account · project | | -| [tenant-fast-cicd.tf](./tenant-fast-cicd.tf) | Per-tenant CI/CD resources. | iam-service-account | | -| [tenant-fast-identity-providers.tf](./tenant-fast-identity-providers.tf) | Per-tenant Workload Identity Federation providers. | | google_iam_workload_identity_pool · google_iam_workload_identity_pool_provider | -| [tenant-fast-logging.tf](./tenant-fast-logging.tf) | Per-tenant FAST bootstrap emulation (logging). | project | | -| [tenant-fast-vpcsc.tf](./tenant-fast-vpcsc.tf) | Per-tenant VPC-SC resources. | vpc-sc | | -| [tenant.tf](./tenant.tf) | Per-tenant resources. | folder · gcs · iam-service-account | | -| [variables-fast.tf](./variables-fast.tf) | FAST stage interface. | | | -| [variables.tf](./variables.tf) | Module variables. | | | - -## Variables - -| name | description | type | required | default | producer | -|---|---|:---:|:---:|:---:|:---:| -| [automation](variables-fast.tf#L19) | Automation resources created by the bootstrap stage. | object({…}) | ✓ | | 0-bootstrap | -| [billing_account](variables-fast.tf#L42) | Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`. | object({…}) | ✓ | | 0-bootstrap | -| [environments](variables-fast.tf#L75) | Environment names. | map(object({…})) | ✓ | | 0-globals | -| [logging](variables-fast.tf#L121) | Logging resources created by the bootstrap stage. | object({…}) | ✓ | | 0-bootstrap | -| [org_policy_tags](variables-fast.tf#L140) | Organization policy tags. | object({…}) | ✓ | | 0-bootstrap | -| [organization](variables-fast.tf#L130) | Organization details. | object({…}) | ✓ | | 0-bootstrap | -| [prefix](variables-fast.tf#L157) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | 0-bootstrap | -| [custom_roles](variables-fast.tf#L53) | Custom roles defined at the org level, in key => id format. | object({…}) | | null | 0-bootstrap | -| [groups](variables-fast.tf#L93) | Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated. | object({…}) | | {} | 0-bootstrap | -| [locations](variables-fast.tf#L108) | Optional locations for GCS, BigQuery, and logging buckets created here. | object({…}) | | {} | 0-bootstrap | -| [names](variables.tf#L18) | Configuration for names used for resources and output files. | object({…}) | | {} | | -| [outputs_location](variables.tf#L28) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | string | | null | | -| [root_node](variables.tf#L34) | Root folder under which tenants are created, in folders/nnnn format. Defaults to the organization if null. | string | | null | | -| [tag_names](variables.tf#L47) | Customized names for resource management tags. | object({…}) | | {} | | -| [tenant_configs](variables.tf#L60) | Tenant configurations. Keys are the short names used for naming resources and should not be changed once defined. | map(object({…})) | | {} | | - -## Outputs - -| name | description | sensitive | consumers | -|---|---|:---:|---| -| [tenants](outputs.tf#L139) | Tenant base configuration. | | | - diff --git a/fast/addons/1-resman-tenants/diagram-flow.png b/fast/addons/1-resman-tenants/diagram-flow.png deleted file mode 100644 index 7e91e8b26..000000000 Binary files a/fast/addons/1-resman-tenants/diagram-flow.png and /dev/null differ diff --git a/fast/addons/1-resman-tenants/diagram.png b/fast/addons/1-resman-tenants/diagram.png deleted file mode 100644 index e1c85eb47..000000000 Binary files a/fast/addons/1-resman-tenants/diagram.png and /dev/null differ diff --git a/fast/addons/1-resman-tenants/identity-providers-defs.tf b/fast/addons/1-resman-tenants/identity-providers-defs.tf deleted file mode 100644 index 1635068d8..000000000 --- a/fast/addons/1-resman-tenants/identity-providers-defs.tf +++ /dev/null @@ -1,91 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Identity provider definitions. - -locals { - # tflint-ignore: terraform_unused_declarations - workforce_identity_providers_defs = { - azuread = { - attribute_mapping = { - "google.subject" = "assertion.subject" - "google.display_name" = "assertion.attributes.userprincipalname[0]" - "google.groups" = "assertion.attributes.groups" - "attribute.first_name" = "assertion.attributes.givenname[0]" - "attribute.last_name" = "assertion.attributes.surname[0]" - "attribute.user_email" = "assertion.attributes.mail[0]" - } - } - } - workload_identity_providers_defs = { - # https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect - github = { - attribute_mapping = { - "google.subject" = "assertion.sub" - "attribute.sub" = "assertion.sub" - "attribute.actor" = "assertion.actor" - "attribute.repository" = "assertion.repository" - "attribute.repository_owner" = "assertion.repository_owner" - "attribute.ref" = "assertion.ref" - "attribute.fast_sub" = "\"repo:\" + assertion.repository + \":ref:\" + assertion.ref" - } - issuer_uri = "https://token.actions.githubusercontent.com" - principal_branch = "principalSet://iam.googleapis.com/%s/attribute.fast_sub/repo:%s:ref:refs/heads/%s" - principal_repo = "principalSet://iam.googleapis.com/%s/attribute.repository/%s" - } - # https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html#token-payload - gitlab = { - attribute_mapping = { - "google.subject" = "assertion.sub" - "attribute.sub" = "assertion.sub" - "attribute.environment" = "assertion.environment" - "attribute.environment_protected" = "assertion.environment_protected" - "attribute.namespace_id" = "assertion.namespace_id" - "attribute.namespace_path" = "assertion.namespace_path" - "attribute.pipeline_id" = "assertion.pipeline_id" - "attribute.pipeline_source" = "assertion.pipeline_source" - "attribute.project_id" = "assertion.project_id" - "attribute.project_path" = "assertion.project_path" - "attribute.repository" = "assertion.project_path" - "attribute.ref" = "assertion.ref" - "attribute.ref_protected" = "assertion.ref_protected" - "attribute.ref_type" = "assertion.ref_type" - } - issuer_uri = "https://gitlab.com" - principal_branch = "principalSet://iam.googleapis.com/%s/attribute.sub/project_path:%s:ref_type:branch:ref:%s" - principal_repo = "principalSet://iam.googleapis.com/%s/attribute.repository/%s" - } - # https://developer.hashicorp.com/terraform/cloud-docs/workspaces/dynamic-provider-credentials/workload-identity-tokens#token-structure - terraform = { - attribute_mapping = { - "google.subject" = "assertion.terraform_workspace_id" - "attribute.aud" = "assertion.aud" - "attribute.terraform_run_phase" = "assertion.terraform_run_phase" - "attribute.terraform_project_id" = "assertion.terraform_project_id" - "attribute.terraform_project_name" = "assertion.terraform_project_name" - "attribute.terraform_workspace_id" = "assertion.terraform_workspace_id" - "attribute.terraform_workspace_name" = "assertion.terraform_workspace_name" - "attribute.terraform_organization_id" = "assertion.terraform_organization_id" - "attribute.terraform_organization_name" = "assertion.terraform_organization_name" - "attribute.terraform_run_id" = "assertion.terraform_run_id" - "attribute.terraform_full_workspace" = "assertion.terraform_full_workspace" - } - issuer_uri = "https://app.terraform.io" - principal_branch = "principalSet://iam.googleapis.com/%s/attribute.terraform_workspace_id/%s" - principal_repo = "principalSet://iam.googleapis.com/%s/attribute.terraform_project_id/%s" - } - } -} diff --git a/fast/addons/1-resman-tenants/main.tf b/fast/addons/1-resman-tenants/main.tf deleted file mode 100644 index 89ecb32fe..000000000 --- a/fast/addons/1-resman-tenants/main.tf +++ /dev/null @@ -1,50 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - default_environment = [ - for k, v in var.environments : v if v.is_default == true - ][0] - tenants = { - for k, v in var.tenant_configs : k => merge(v, { - billing_account = merge(v.billing_account, { - id = coalesce(v.billing_account.id, var.billing_account.id) - # only set is_org_level when using the org billing account - is_org_level = ( - v.billing_account.id == null || - v.billing_account.id == var.billing_account.id - ) ? var.billing_account.is_org_level : false - }) - locations = coalesce(v.locations, var.locations) - organization = coalesce(v.cloud_identity, var.organization) - }) - } -} - -module "organization" { - source = "../../../modules/organization" - organization_id = "organizations/${var.organization.id}" - tags = { - (var.tag_names.tenant) = { - description = "Resource management tenant." - values = { - for k, v in local.tenants : k => { - description = v.descriptive_name - } - } - } - } -} diff --git a/fast/addons/1-resman-tenants/outputs-files.tf b/fast/addons/1-resman-tenants/outputs-files.tf deleted file mode 100644 index 047831acd..000000000 --- a/fast/addons/1-resman-tenants/outputs-files.tf +++ /dev/null @@ -1,78 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Output files persistence to local filesystem. - -resource "local_file" "providers-simple" { - for_each = var.outputs_location == null ? {} : { - for k, v in local.tenants : k => local.tenant_data[k] - } - file_permission = "0644" - filename = "${try(pathexpand(var.outputs_location), "")}/providers/${var.names.output_files_prefix}-${each.key}.tf" - content = templatefile(local._tpl_providers, { - backend_extra = null - bucket = each.value.gcs_bucket - name = each.key - sa = each.value.service_account - }) -} - -resource "local_file" "tfvars-simple" { - for_each = var.outputs_location == null ? {} : { - for k, v in local.tenants : k => local.tenant_data[k] - } - file_permission = "0644" - filename = "${try(pathexpand(var.outputs_location), "")}/tfvars/${var.names.output_files_prefix}-${each.key}.auto.tfvars.json" - content = jsonencode(each.value) -} - -resource "local_file" "providers" { - for_each = var.outputs_location == null ? {} : local.tenant_providers - file_permission = "0644" - filename = "${try(pathexpand(var.outputs_location), "")}/${var.names.output_files_prefix}/${each.key}/providers/1-resman-providers.tf" - content = try(each.value, null) -} - -resource "local_file" "providers-r" { - for_each = var.outputs_location == null ? {} : local.tenant_providers_r - file_permission = "0644" - filename = "${try(pathexpand(var.outputs_location), "")}/${var.names.output_files_prefix}/${each.key}/providers/1-resman-r-providers.tf" - content = try(each.value, null) -} - -resource "local_file" "tfvars" { - # work around Terraform's botched ternary type check on maps - for_each = { - for k, v in local.tenant_tfvars : k => v if var.outputs_location != null - } - file_permission = "0644" - filename = "${try(pathexpand(var.outputs_location), "")}/${var.names.output_files_prefix}/${each.key}/tfvars/0-bootstrap.auto.tfvars.json" - content = jsonencode(each.value) -} - -resource "local_file" "tfvars_globals" { - for_each = var.outputs_location == null ? {} : local.tenant_globals - file_permission = "0644" - filename = "${try(pathexpand(var.outputs_location), "")}/${var.names.output_files_prefix}/${each.key}/tfvars/0-globals.auto.tfvars.json" - content = jsonencode(each.value) -} - -resource "local_file" "workflows" { - for_each = var.outputs_location == null ? {} : local.tenant_cicd_workflows - file_permission = "0644" - filename = "${try(pathexpand(var.outputs_location), "")}/${var.names.output_files_prefix}/${each.key}/workflows/1-resman-workflow.yaml" - content = try(each.value, null) -} diff --git a/fast/addons/1-resman-tenants/outputs-gcs.tf b/fast/addons/1-resman-tenants/outputs-gcs.tf deleted file mode 100644 index 23dc34292..000000000 --- a/fast/addons/1-resman-tenants/outputs-gcs.tf +++ /dev/null @@ -1,75 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Output files persistence to automation GCS bucket. - -resource "google_storage_bucket_object" "providers-simple" { - for_each = { - for k, v in local.tenants : k => local.tenant_data[k] - } - bucket = var.automation.outputs_bucket - name = "providers/${var.names.output_files_prefix}-${each.key}.tf" - content = templatefile(local._tpl_providers, { - backend_extra = null - bucket = each.value.gcs_bucket - name = each.key - sa = each.value.service_account - }) -} - -resource "google_storage_bucket_object" "tfvars-simple" { - for_each = { - for k, v in local.tenants : k => local.tenant_data[k] - } - bucket = var.automation.outputs_bucket - name = "tfvars/${var.names.output_files_prefix}-${each.key}.auto.tfvars.json" - content = jsonencode(each.value) -} - -resource "google_storage_bucket_object" "providers" { - for_each = local.tenant_providers - bucket = module.tenant-automation-tf-output-gcs[each.key].name - name = "providers/1-resman-providers.tf" - content = each.value -} - -resource "google_storage_bucket_object" "providers_r" { - for_each = local.tenant_providers_r - bucket = module.tenant-automation-tf-output-gcs[each.key].name - name = "providers/1-resman-r-providers.tf" - content = each.value -} - -resource "google_storage_bucket_object" "tfvars" { - for_each = local.tenant_tfvars - bucket = module.tenant-automation-tf-output-gcs[each.key].name - name = "tfvars/0-bootstrap.auto.tfvars.json" - content = jsonencode(each.value) -} - -resource "google_storage_bucket_object" "tfvars_globals" { - for_each = local.tenant_globals - bucket = module.tenant-automation-tf-output-gcs[each.key].name - name = "tfvars/0-globals.auto.tfvars.json" - content = jsonencode(each.value) -} - -resource "google_storage_bucket_object" "workflows" { - for_each = local.tenant_cicd_workflows - bucket = module.tenant-automation-tf-output-gcs[each.key].name - name = "workflows/1-resman-workflow.yaml" - content = each.value -} diff --git a/fast/addons/1-resman-tenants/outputs.tf b/fast/addons/1-resman-tenants/outputs.tf deleted file mode 100644 index bb0c75d7b..000000000 --- a/fast/addons/1-resman-tenants/outputs.tf +++ /dev/null @@ -1,142 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - _tpl_providers = "${path.module}/templates/providers.tf.tpl" - tenant_cicd_workflows = { - for k, v in local.cicd_repositories : - k => templatefile("${path.module}/templates/workflow-${v.type}.yaml", { - audiences = try( - local.identity_providers[v.tenant][v.identity_provider].audiences, null - ) - identity_provider = try( - local.identity_providers[v.tenant][v.identity_provider].name, null - ) - outputs_bucket = try( - module.tenant-automation-tf-output-gcs[k].name, null - ) - service_accounts = { - apply = try(module.tenant-automation-tf-resman-sa[k].email, null) - plan = try(module.tenant-automation-tf-resman-r-sa[k].email, null) - } - stage_name = "1-resman" - tf_providers_files = { - apply = "1-resman-providers.tf" - plan = "1-resman-r-providers.tf" - } - tf_var_files = [ - "0-bootstrap.auto.tfvars.json", - "0-globals.auto.tfvars.json" - ] - }) - } - tenant_data = { - for k, v in local.tenants : k => { - folder_id = module.tenant-folder[k].id - gcs_bucket = module.tenant-gcs[k].id - service_account = module.tenant-sa[k].email - vpcsc_policy_id = try(module.tenant-vpcsc-policy[k].id, null) - } - } - tenant_providers = { - for k, v in local.fast_tenants : k => templatefile(local._tpl_providers, { - backend_extra = null - bucket = module.tenant-automation-tf-resman-gcs[k].name - name = k - sa = module.tenant-automation-tf-resman-sa[k].email - }) - } - tenant_providers_r = { - for k, v in local.fast_tenants : k => templatefile(local._tpl_providers, { - backend_extra = null - bucket = module.tenant-automation-tf-resman-gcs[k].name - name = k - sa = module.tenant-automation-tf-resman-r-sa[k].email - }) - } - tenant_globals = { - for k, v in local.fast_tenants : k => { - billing_account = v.billing_account - groups = v.principals - environments = { - for k, v in var.environments : k => { - is_default = v.is_default - key = k - name = v.name - short_name = v.short_name != null ? v.short_name : k - tag_name = v.tag_name != null ? v.tag_name : lower(v.name) - } - } - locations = v.locations - organization = v.organization - prefix = v.prefix - } - } - tenant_tfvars = { - for k, v in local.fast_tenants : k => { - access_policy = try(module.tenant-vpcsc-policy[k].id, null) - automation = { - federated_identity_pool = null - federated_identity_providers = local.identity_providers[k] - outputs_bucket = module.tenant-automation-tf-output-gcs[k].name - project_id = module.tenant-automation-project[k].project_id - project_number = module.tenant-automation-project[k].number - service_accounts = { - resman = module.tenant-automation-tf-resman-sa[k].email - resman-r = module.tenant-automation-tf-resman-r-sa[k].email - } - tenant_service_accounts = { - network = module.tenant-automation-tf-network-sa[k].email - security = module.tenant-automation-tf-security-sa[k].email - security-r = module.tenant-automation-tf-security-r-sa[k].email - } - } - custom_roles = var.custom_roles - logging = { - log_sinks = { - audit-logs = { - filter = <<-FILTER - log_id("cloudaudit.googleapis.com/activity") OR - log_id("cloudaudit.googleapis.com/system_event") OR - log_id("cloudaudit.googleapis.com/policy") OR - log_id("cloudaudit.googleapis.com/access_transparency") - FILTER - type = "logging" - } - vpc-sc = { - filter = <<-FILTER - protoPayload.metadata.@type="type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata" - FILTER - type = "logging" - } - } - project_id = module.tenant-log-export-project[k].project_id - project_number = module.tenant-log-export-project[k].number - writer_identities = {} - } - org_policy_tags = var.org_policy_tags - root_node = module.tenant-folder[k].id - security = { - access_policy_id = try(module.tenant-vpcsc-policy[k].id, null) - } - } - } -} - -output "tenants" { - description = "Tenant base configuration." - value = local.tenant_data -} diff --git a/fast/addons/1-resman-tenants/templates/workflow-github.yaml b/fast/addons/1-resman-tenants/templates/workflow-github.yaml deleted file mode 100644 index 8e7a94dc9..000000000 --- a/fast/addons/1-resman-tenants/templates/workflow-github.yaml +++ /dev/null @@ -1,229 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -name: "FAST ${stage_name} stage" - -on: - pull_request: - branches: - - main - types: - - closed - - opened - - synchronize - -env: - FAST_SERVICE_ACCOUNT: ${service_accounts.apply} - FAST_SERVICE_ACCOUNT_PLAN: ${service_accounts.plan} - FAST_WIF_PROVIDER: ${identity_provider} - SSH_AUTH_SOCK: /tmp/ssh_agent.sock - TF_PROVIDERS_FILE: ${tf_providers_files.apply} - TF_PROVIDERS_FILE_PLAN: ${tf_providers_files.plan} - TF_VERSION: 1.11.4 - -jobs: - fast-pr: - # Skip PRs which are closed without being merged. - if: >- - github.event.action == 'closed' && - github.event.pull_request.merged == true || - github.event.action == 'opened' || - github.event.action == 'synchronize' - permissions: - contents: read - id-token: write - issues: write - pull-requests: write - runs-on: ubuntu-latest - steps: - - id: checkout - name: Checkout repository - uses: actions/checkout@v4 - - # set up SSH key authentication to the modules repository - - - id: ssh-config - name: Configure SSH authentication - run: | - ssh-agent -a "$SSH_AUTH_SOCK" > /dev/null - ssh-add - <<< "$${{ secrets.CICD_MODULES_KEY }}" - - # set up step variables for plan / apply - - - id: vars-plan - if: github.event.pull_request.merged != true && success() - name: Set up plan variables - run: | - echo "plan_opts=-lock=false" >> "$GITHUB_ENV" - echo "provider_file=$${{env.TF_PROVIDERS_FILE_PLAN}}" >> "$GITHUB_ENV" - echo "service_account=$${{env.FAST_SERVICE_ACCOUNT_PLAN}}" >> "$GITHUB_ENV" - - - id: vars-apply - if: github.event.pull_request.merged == true && success() - name: Set up apply variables - run: | - echo "provider_file=$${{env.TF_PROVIDERS_FILE}}" >> "$GITHUB_ENV" - echo "service_account=$${{env.FAST_SERVICE_ACCOUNT}}" >> "$GITHUB_ENV" - - # set up authentication via Workload identity Federation and gcloud - - - id: gcp-auth - name: Authenticate to Google Cloud - uses: google-github-actions/auth@v2 - with: - workload_identity_provider: $${{env.FAST_WIF_PROVIDER}} - service_account: $${{env.service_account}} - access_token_lifetime: 900s - - - id: gcp-sdk - name: Set up Cloud SDK - uses: google-github-actions/setup-gcloud@v2 - with: - install_components: alpha - - # copy provider file - - - id: tf-config-provider - name: Copy Terraform provider file - run: | - gcloud storage cp -r \ - "gs://${outputs_bucket}/providers/$${{env.provider_file}}" ./ - %{~ for f in tf_var_files ~} - gcloud storage cp -r \ - "gs://${outputs_bucket}/tfvars/${f}" ./ - %{~ endfor ~} - - - id: tf-setup - name: Set up Terraform - uses: hashicorp/setup-terraform@v3 - with: - terraform_version: $${{env.TF_VERSION}} - - # run Terraform init/validate/plan - - - id: tf-init - name: Terraform init - continue-on-error: true - run: | - terraform init -no-color - - - id: tf-validate - continue-on-error: true - name: Terraform validate - run: terraform validate -no-color - - - id: tf-plan - name: Terraform plan - continue-on-error: true - run: | - terraform plan -input=false -out ../plan.out -no-color $${{env.plan_opts}} - - - id: tf-apply - if: github.event.pull_request.merged == true && success() - name: Terraform apply - continue-on-error: true - run: | - terraform apply -input=false -auto-approve -no-color ../plan.out - - # PR comment with Terraform result from previous steps - # length is checked and trimmed for length so as to stay within the limit - - - id: pr-comment - name: Post comment to Pull Request - continue-on-error: true - uses: actions/github-script@v7 - if: github.event_name == 'pull_request' - env: - PLAN: $${{steps.tf-plan.outputs.stdout}}\n$${{steps.tf-plan.outputs.stderr}} - with: - script: | - const output = `### Terraform Initialization \`$${{steps.tf-init.outcome}}\` - - ### Terraform Validation \`$${{steps.tf-validate.outcome}}\` - -
Validation Output - - \`\`\`\n - $${{steps.tf-validate.outputs.stdout}} - \`\`\` - -
- - ### Terraform Plan \`$${{steps.tf-plan.outcome}}\` - -
Show Plan - - \`\`\`\n - $${process.env.PLAN.split('\n').filter(l => l.match(/^([A-Z\s].*|)$$/)).join('\n')} - \`\`\` - -
- - ### Terraform Apply \`$${{steps.tf-apply.outcome}}\` - - *Pusher: @$${{github.actor}}, Action: \`$${{github.event_name}}\`, Working Directory: \`$${{env.tf_actions_working_dir}}\`, Workflow: \`$${{github.workflow}}\`*`; - - github.rest.issues.createComment({ - issue_number: context.issue.number, - owner: context.repo.owner, - repo: context.repo.repo, - body: output - }) - - - id: pr-short-comment - name: Post comment to Pull Request (abbreviated) - uses: actions/github-script@v7 - if: github.event_name == 'pull_request' && steps.pr-comment.outcome != 'success' - with: - script: | - const output = `### Terraform Initialization \`$${{steps.tf-init.outcome}}\` - - ### Terraform Validation \`$${{steps.tf-validate.outcome}}\` - - ### Terraform Plan \`$${{steps.tf-plan.outcome}}\` - - Plan output is in the action log. - - ### Terraform Apply \`$${{steps.tf-apply.outcome}}\` - - *Pusher: @$${{github.actor}}, Action: \`$${{github.event_name}}\`, Working Directory: \`$${{env.tf_actions_working_dir}}\`, Workflow: \`$${{github.workflow}}\`*`; - - github.rest.issues.createComment({ - issue_number: context.issue.number, - owner: context.repo.owner, - repo: context.repo.repo, - body: output - }) - - # exit on error from previous steps - - - id: check-init - name: Check init failure - if: steps.tf-init.outcome != 'success' - run: exit 1 - - - id: check-validate - name: Check validate failure - if: steps.tf-validate.outcome != 'success' - run: exit 1 - - - id: check-plan - name: Check plan failure - if: steps.tf-plan.outcome != 'success' - run: exit 1 - - - id: check-apply - name: Check apply failure - if: github.event.pull_request.merged == true && steps.tf-apply.outcome != 'success' - run: exit 1 diff --git a/fast/addons/1-resman-tenants/tenant-billing-iam.tf b/fast/addons/1-resman-tenants/tenant-billing-iam.tf deleted file mode 100644 index eae0471ae..000000000 --- a/fast/addons/1-resman-tenants/tenant-billing-iam.tf +++ /dev/null @@ -1,89 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Per-tenant billing IAM. - -locals { - # additive only since these operate at org level or BA level - _billing_bindings = flatten([ - for k, v in local.tenants : [ - v.fast_config == null - # non-fast tenant - ? [ - { - role = "roles/billing.user" - member = v.admin_principal - tenant = k - } - ] - # fast tenant - : [ - { - role = "roles/billing.admin" - member = local.fast_tenants[k].principals.gcp-billing-admins - tenant = k - }, - { - role = "roles/billing.admin" - member = local.fast_tenants[k].principals.gcp-organization-admins - tenant = k - }, - { - role = "roles/billing.admin" - member = module.tenant-automation-tf-resman-sa[k].iam_email - tenant = k - }, - { - role = "roles/billing.viewer" - member = module.tenant-automation-tf-resman-r-sa[k].iam_email - tenant = k - }, - ] - ] if v.billing_account.no_iam == false - ]) - # group bindings applied per billing account - _billing_ba_bindings = { - for v in local._billing_bindings : - local.tenants[v.tenant].billing_account.id => v... - if local.tenants[v.tenant].billing_account.is_org_level != true - } - # convert billing account grouped lists to maps - billing_ba_bindings = { - for k, v in local._billing_ba_bindings : k => { - for vv in v : - "${vv.tenant}-${vv.role}-${vv.member}" => vv - } - } - # convert org bindings to a map - billing_org_bindings = { - for v in local._billing_bindings : - "${v.tenant}-${v.role}-${v.member}" => v - if local.tenants[v.tenant].billing_account.is_org_level == true - } -} - -module "billing-account" { - source = "../../../modules/billing-account" - for_each = local.billing_ba_bindings - id = each.key - iam_bindings_additive = each.value -} - -module "organization-billing" { - source = "../../../modules/organization" - organization_id = "organizations/${var.organization.id}" - iam_bindings_additive = local.billing_org_bindings -} diff --git a/fast/addons/1-resman-tenants/tenant-core.tf b/fast/addons/1-resman-tenants/tenant-core.tf deleted file mode 100644 index 9f918d9bc..000000000 --- a/fast/addons/1-resman-tenants/tenant-core.tf +++ /dev/null @@ -1,77 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Per-tenant centrally managed resources. - -locals { - root_node = coalesce(var.root_node, "organizations/${var.organization.id}") -} - -module "tenant-core-logbucket" { - source = "../../../modules/logging-bucket" - for_each = local.tenants - parent = var.logging.project_id - name = "${var.names.resource_short_name}-${each.key}-audit" - location = var.locations.logging - log_analytics = { enable = true } -} - -module "tenant-core-folder" { - source = "../../../modules/folder" - for_each = local.tenants - parent = local.root_node - name = "${each.value.descriptive_name} Core" - logging_sinks = { - "${var.names.resource_short_name}-${each.key}-audit" = { - destination = module.tenant-core-logbucket[each.key].id - filter = <<-FILTER - log_id("cloudaudit.googleapis.com/activity") OR - log_id("cloudaudit.googleapis.com/system_event") OR - log_id("cloudaudit.googleapis.com/policy") OR - log_id("cloudaudit.googleapis.com/access_transparency") - FILTER - type = "logging" - } - } - org_policies = each.value.cloud_identity == null ? {} : { - "essentialcontacts.allowedContactDomains" = { - rules = [{ - allow = { - values = formatlist("@%s", compact([ - var.organization.domain, - try(each.value.cloud_identity.domain, null) - ])) - } - }] - } - "iam.allowedPolicyMemberDomains" = { - rules = [{ - allow = { - values = compact([ - var.organization.customer_id, - try(each.value.cloud_identity.customer_id, null) - ]) - } - }] - } - } - tag_bindings = { - tenant = try( - module.organization.tag_values["${var.tag_names.tenant}/${each.key}"].id, - null - ) - } -} diff --git a/fast/addons/1-resman-tenants/tenant-fast-automation.tf b/fast/addons/1-resman-tenants/tenant-fast-automation.tf deleted file mode 100644 index 6135f2d9e..000000000 --- a/fast/addons/1-resman-tenants/tenant-fast-automation.tf +++ /dev/null @@ -1,293 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Per-tenant FAST bootstrap emulation (automation). - -locals { - _fast_tenants = { - for k, v in local.tenants : k => merge(v, { - groups = coalesce(v.fast_config.groups, var.groups) - prefix = coalesce(v.fast_config.prefix, "${var.prefix}-${k}") - wif_provider = try(v.fast_config.cicd_config.identity_provider, "-") - }) if v.fast_config != null - } - fast_tenants = { - for k, v in local._fast_tenants : k => merge(v, { - stage_0_prefix = "${v.prefix}-${local.default_environment.short_name}" - principals = { - for gk, gv in v.groups : gk => ( - can(regex("^[a-zA-Z]+:", gv)) - ? gv - : "group:${gv}@${v.organization.domain}" - ) - } - }) - } -} - -module "tenant-automation-project" { - source = "../../../modules/project" - for_each = local.fast_tenants - billing_account = each.value.billing_account.id - name = "iac-core-0" - parent = module.tenant-folder[each.key].id - prefix = each.value.stage_0_prefix - # this is needed when destroying, resources cannot depend on the - # project-iam module to avoid circular dependencies - iam_bindings_additive = { - owner_org_resman = { - role = "roles/owner" - member = "serviceAccount:${var.automation.service_accounts.resman}" - } - } - services = [ - "accesscontextmanager.googleapis.com", - "bigquery.googleapis.com", - "bigqueryreservation.googleapis.com", - "bigquerystorage.googleapis.com", - "billingbudgets.googleapis.com", - "cloudasset.googleapis.com", - "cloudbilling.googleapis.com", - "cloudbuild.googleapis.com", - "cloudkms.googleapis.com", - "cloudquotas.googleapis.com", - "cloudresourcemanager.googleapis.com", - "compute.googleapis.com", - "container.googleapis.com", - "essentialcontacts.googleapis.com", - "iam.googleapis.com", - "iamcredentials.googleapis.com", - "logging.googleapis.com", - "monitoring.googleapis.com", - "orgpolicy.googleapis.com", - "pubsub.googleapis.com", - "servicenetworking.googleapis.com", - "serviceusage.googleapis.com", - "storage-component.googleapis.com", - "storage.googleapis.com", - "sts.googleapis.com", - ] - logging_data_access = { - "iam.googleapis.com" = { - ADMIN_READ = {} - } - } -} - - -module "tenant-automation-project-iam" { - source = "../../../modules/project" - for_each = local.fast_tenants - name = module.tenant-automation-project[each.key].project_id - project_reuse = { - use_data_source = false - attributes = { - name = module.tenant-automation-project[each.key].name - number = module.tenant-automation-project[each.key].number - } - } - # human (groups) IAM bindings - iam_by_principals = { - (each.value.principals.gcp-devops) = [ - "roles/iam.serviceAccountAdmin", - "roles/iam.serviceAccountTokenCreator", - ] - (each.value.principals.gcp-organization-admins) = [ - "roles/iam.serviceAccountTokenCreator", - "roles/iam.workloadIdentityPoolAdmin" - ] - } - # machine (service accounts) IAM bindings - iam = { - "roles/browser" = [ - module.tenant-automation-tf-resman-r-sa[each.key].iam_email - ] - "roles/cloudbuild.builds.editor" = [ - module.tenant-automation-tf-resman-sa[each.key].iam_email - ] - "roles/cloudbuild.builds.viewer" = [ - module.tenant-automation-tf-resman-r-sa[each.key].iam_email - ] - "roles/iam.serviceAccountAdmin" = [ - module.tenant-automation-tf-resman-sa[each.key].iam_email - ] - "roles/iam.serviceAccountViewer" = [ - module.tenant-automation-tf-resman-r-sa[each.key].iam_email - ] - "roles/iam.workloadIdentityPoolAdmin" = [ - module.tenant-automation-tf-resman-sa[each.key].iam_email - ] - "roles/iam.workloadIdentityPoolViewer" = [ - module.tenant-automation-tf-resman-r-sa[each.key].iam_email - ] - "roles/source.admin" = [ - module.tenant-automation-tf-resman-sa[each.key].iam_email - ] - "roles/source.reader" = [ - module.tenant-automation-tf-resman-r-sa[each.key].iam_email - ] - "roles/storage.admin" = [ - module.tenant-automation-tf-resman-sa[each.key].iam_email - ] - (var.custom_roles["storage_viewer"]) = [ - module.tenant-automation-tf-resman-r-sa[each.key].iam_email - ] - "roles/viewer" = [ - "serviceAccount:${var.automation.service_accounts.resman-r}", - module.tenant-automation-tf-resman-r-sa[each.key].iam_email - ] - } - iam_bindings = { - delegated_grants_resman = { - members = [module.tenant-automation-tf-resman-sa[each.key].iam_email] - role = "roles/resourcemanager.projectIamAdmin" - condition = { - title = "resman_delegated_grant" - description = "Resource manager service account delegated grant." - expression = format( - "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly(['%s'])", - "roles/serviceusage.serviceUsageConsumer" - ) - } - } - } - iam_bindings_additive = { - serviceusage_resman = { - member = module.tenant-automation-tf-resman-sa[each.key].iam_email - role = "roles/serviceusage.serviceUsageConsumer" - } - serviceusage_resman_r = { - member = module.tenant-automation-tf-resman-r-sa[each.key].iam_email - role = "roles/serviceusage.serviceUsageViewer" - } - } - depends_on = [module.tenant-automation-project] -} - -# output files bucket - -module "tenant-automation-tf-output-gcs" { - source = "../../../modules/gcs" - for_each = local.fast_tenants - project_id = module.tenant-automation-project[each.key].project_id - name = "iac-core-outputs-0" - prefix = each.value.stage_0_prefix - location = each.value.locations.gcs - versioning = true -} - -# resource hierarchy stage's bucket and service account - -module "tenant-automation-tf-resman-gcs" { - source = "../../../modules/gcs" - for_each = local.fast_tenants - project_id = module.tenant-automation-project[each.key].project_id - name = "iac-core-resman-0" - prefix = each.value.stage_0_prefix - location = each.value.locations.gcs - versioning = true - iam = { - "roles/storage.objectAdmin" = [ - module.tenant-automation-tf-resman-sa[each.key].iam_email - ] - "roles/storage.objectViewer" = [ - module.tenant-automation-tf-resman-r-sa[each.key].iam_email - ] - } -} - -module "tenant-automation-tf-resman-sa" { - source = "../../../modules/iam-service-account" - for_each = local.fast_tenants - project_id = module.tenant-automation-project[each.key].project_id - name = "resman-0" - display_name = "Terraform stage 1 resman service account." - prefix = each.value.stage_0_prefix - # allow SA used by CI/CD workflow to impersonate this SA - # we use additive IAM to allow tenant CI/CD SAs to impersonate it - iam_bindings_additive = ( - lookup(local.cicd_repositories, each.key, null) == null ? {} : { - cicd_token_creator = { - member = module.tenant-automation-tf-cicd-sa[each.key].iam_email - role = "roles/iam.serviceAccountTokenCreator" - } - } - ) - iam_storage_roles = { - (module.tenant-automation-tf-output-gcs[each.key].name) = [ - "roles/storage.admin" - ] - } -} - -module "tenant-automation-tf-resman-r-sa" { - source = "../../../modules/iam-service-account" - for_each = local.fast_tenants - project_id = module.tenant-automation-project[each.key].project_id - name = "resman-0r" - display_name = "Terraform stage 1 resman service account (read-only)." - prefix = each.value.stage_0_prefix - # allow SA used by CI/CD workflow to impersonate this SA - # we use additive IAM to allow tenant CI/CD SAs to impersonate it - iam_bindings_additive = ( - lookup(local.cicd_repositories, each.key, null) == null ? {} : { - cicd_token_creator = { - member = module.automation-tf-cicd-r-sa[each.key].iam_email - role = "roles/iam.serviceAccountTokenCreator" - } - } - ) - iam_storage_roles = { - (module.tenant-automation-tf-output-gcs[each.key].name) = [ - var.custom_roles["storage_viewer"] - ] - } -} - -# tenant-level stage 2 service accounts are created here so that we can -# grant permissions on the org or VPC SC policy - -module "tenant-automation-tf-network-sa" { - source = "../../../modules/iam-service-account" - for_each = local.fast_tenants - project_id = module.tenant-automation-project[each.key].project_id - name = "resman-net-0" - display_name = "Terraform resman networking service account." - prefix = each.value.stage_0_prefix - iam_organization_roles = { - (var.organization.id) = [ - var.custom_roles.tenant_network_admin - ] - } -} - -module "tenant-automation-tf-security-sa" { - source = "../../../modules/iam-service-account" - for_each = local.fast_tenants - project_id = module.tenant-automation-project[each.key].project_id - name = "resman-sec-0" - display_name = "Terraform resman security service account." - prefix = each.value.stage_0_prefix -} - -module "tenant-automation-tf-security-r-sa" { - source = "../../../modules/iam-service-account" - for_each = local.fast_tenants - project_id = module.tenant-automation-project[each.key].project_id - name = "resman-sec-0r" - display_name = "Terraform resman security service account (read-only)." - prefix = each.value.stage_0_prefix -} diff --git a/fast/addons/1-resman-tenants/tenant-fast-cicd.tf b/fast/addons/1-resman-tenants/tenant-fast-cicd.tf deleted file mode 100644 index 8a1d68905..000000000 --- a/fast/addons/1-resman-tenants/tenant-fast-cicd.tf +++ /dev/null @@ -1,156 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Per-tenant CI/CD resources. - -locals { - # alias resources for readability - _wif_providers = { - for k, v in google_iam_workload_identity_pool_provider.default : k => v - } - # aggregate provider data from configurations and resources - _cicd_providers = [ - for k, v in local.workload_identity_providers : { - audiences = concat( - local._wif_providers[k].oidc[0].allowed_audiences, - ["https://iam.googleapis.com/${local._wif_providers[k].name}"] - ) - issuer = v.issuer - issuer_uri = try(local._wif_providers[k].oidc[0].issuer_uri, null) - name = local._wif_providers[k].name - principal_branch = v.principal_branch - principal_repo = v.principal_repo - provider = v.provider - tenant = v.tenant - } - ] - # group provider data by tenant - _cicd_tenant_providers = { - for v in local._cicd_providers : v.tenant => v... - } - # reconstitue per-tenant provider lists as maps - cicd_tenant_providers = { - for k, v in local._cicd_tenant_providers : k => { - for pv in v : pv.provider => pv - } - } - # filter tenant provider definitions to only keep valid ones - cicd_repositories = { - for k, v in local.fast_tenants : - k => merge(v.fast_config.cicd_config, { - tenant = k - }) - # only keep CI/CD configurations that - if( - # are not null - try(v.fast_config.cicd_config, null) != null - && - # are of a valid type (a template file exists for the type) - fileexists( - "${path.module}/templates/workflow-${try(v.fast_config.cicd_config.type, "")}.yaml" - ) - && - # either - ( - # use an org-level WIF provider, or - try(var.automation.federated_identity_providers[v.wif_provider], null) != null - || - # use a tenant-level WIF provider - try(v.fast_config.workload_identity_providers[v.wif_provider], null) != null - ) - ) - } - # merge org-level and tenant-level providers for each tenant - identity_providers = { - for k, v in local.fast_tenants : k => merge( - try(var.automation.federated_identity_providers, {}), - try(local.cicd_tenant_providers[k], {}) - ) - } -} - -# read-write (apply) SA used by CI/CD workflows to impersonate automation SA - -module "tenant-automation-tf-cicd-sa" { - source = "../../../modules/iam-service-account" - for_each = local.cicd_repositories - project_id = var.automation.project_id - name = "${each.key}-1" - display_name = "Terraform CI/CD ${each.key} service account." - prefix = var.prefix - iam = { - "roles/iam.workloadIdentityUser" = [ - each.value.branch == null - ? format( - local.identity_providers[each.value.tenant][each.value.identity_provider].principal_repo, - var.automation.federated_identity_pool, - each.value.name - ) - : length(regexall("%s", local.workload_identity_providers_defs[each.value.type].principal_branch)) == 2 - ? format( - local.identity_providers[each.value.tenant][each.value.identity_provider].principal_branch, - var.automation.federated_identity_pool, - each.value.branch - ) - : format( - local.identity_providers[each.value.tenant][each.value.identity_provider].principal_branch, - var.automation.federated_identity_pool, - each.value.name, - each.value.branch - ) - ] - } - iam_project_roles = { - (module.tenant-automation-project[each.key].project_id) = [ - "roles/logging.logWriter" - ] - } - iam_storage_roles = { - (module.tenant-automation-tf-output-gcs[each.key].name) = [ - "roles/storage.objectViewer" - ] - } -} - -# read-only (plan) SA used by CI/CD workflows to impersonate automation SA - -module "automation-tf-cicd-r-sa" { - source = "../../../modules/iam-service-account" - for_each = local.cicd_repositories - project_id = var.automation.project_id - name = "${each.key}-1r" - display_name = "Terraform CI/CD ${each.key} service account (read-only)." - prefix = var.prefix - iam = { - "roles/iam.workloadIdentityUser" = [ - format( - local.identity_providers[each.value.tenant][each.value.identity_provider].principal_repo, - var.automation.federated_identity_pool, - each.value.name - ) - ] - } - iam_project_roles = { - (module.tenant-automation-project[each.key].project_id) = [ - "roles/logging.logWriter" - ] - } - iam_storage_roles = { - (module.tenant-automation-tf-output-gcs[each.key].name) = [ - "roles/storage.objectViewer" - ] - } -} diff --git a/fast/addons/1-resman-tenants/tenant-fast-identity-providers.tf b/fast/addons/1-resman-tenants/tenant-fast-identity-providers.tf deleted file mode 100644 index a2b1128d1..000000000 --- a/fast/addons/1-resman-tenants/tenant-fast-identity-providers.tf +++ /dev/null @@ -1,76 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Per-tenant Workload Identity Federation providers. - -locals { - # flatten tenant provider configurations into a single list and derive key - _workload_identity_providers = flatten([ - for k, v in local.fast_tenants : [ - for pk, pv in v.fast_config.workload_identity_providers : merge( - pv, - lookup(local.workload_identity_providers_defs, pv.issuer, {}), - { - key = "${k}-${pk}" - prefix = v.prefix - provider = pk - tenant = k - } - ) - ] - ]) - # identify FAST tenants with WIF configurations - workload_identity_pools = { - for k, v in local.fast_tenants : k => v.prefix - if length(v.fast_config.workload_identity_providers) > 0 - } - # reconstitute all tenant provider configurations as a map - workload_identity_providers = { - for v in local._workload_identity_providers : v.key => v - } -} - -resource "google_iam_workload_identity_pool" "default" { - provider = google-beta - for_each = local.workload_identity_pools - project = module.tenant-automation-project[each.key].project_id - workload_identity_pool_id = "${each.value}-bootstrap" -} - -resource "google_iam_workload_identity_pool_provider" "default" { - provider = google-beta - for_each = local.workload_identity_providers - project = module.tenant-automation-project[each.value.tenant].project_id - workload_identity_pool_id = ( - google_iam_workload_identity_pool.default[each.value.tenant].workload_identity_pool_id - ) - workload_identity_pool_provider_id = "${each.value.prefix}-bootstrap-${each.value.provider}" - attribute_condition = each.value.attribute_condition - attribute_mapping = each.value.attribute_mapping - oidc { - # Setting an empty list configures allowed_audiences to the url of the provider - allowed_audiences = each.value.custom_settings.audiences - # If users don't provide an issuer_uri, we set the public one for the platform chosen. - issuer_uri = ( - each.value.custom_settings.issuer_uri != null - ? each.value.custom_settings.issuer_uri - : try(each.value.issuer_uri, null) - ) - # OIDC JWKs in JSON String format. If no value is provided, they key is - # fetched from the `.well-known` path for the issuer_uri - jwks_json = each.value.custom_settings.jwks_json - } -} diff --git a/fast/addons/1-resman-tenants/tenant-fast-logging.tf b/fast/addons/1-resman-tenants/tenant-fast-logging.tf deleted file mode 100644 index 5ee8c462e..000000000 --- a/fast/addons/1-resman-tenants/tenant-fast-logging.tf +++ /dev/null @@ -1,42 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Per-tenant FAST bootstrap emulation (logging). - -module "tenant-log-export-project" { - source = "../../../modules/project" - for_each = local.fast_tenants - billing_account = each.value.billing_account.id - name = "audit-logs-0" - parent = module.tenant-folder[each.key].id - prefix = each.value.stage_0_prefix - iam = { - "roles/owner" = [ - "serviceAccount:${var.automation.service_accounts.resman}" - ] - "roles/viewer" = [ - "serviceAccount:${var.automation.service_accounts.resman-r}" - ] - } - services = [ - # "cloudresourcemanager.googleapis.com", - # "iam.googleapis.com", - # "serviceusage.googleapis.com", - "bigquery.googleapis.com", - "storage.googleapis.com", - "stackdriver.googleapis.com" - ] -} diff --git a/fast/addons/1-resman-tenants/tenant-fast-vpcsc.tf b/fast/addons/1-resman-tenants/tenant-fast-vpcsc.tf deleted file mode 100644 index 12b235bec..000000000 --- a/fast/addons/1-resman-tenants/tenant-fast-vpcsc.tf +++ /dev/null @@ -1,57 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Per-tenant VPC-SC resources. - -module "tenant-vpcsc-policy" { - source = "../../../modules/vpc-sc" - for_each = { - for k, v in local.tenants : k => v if v.vpc_sc_policy_create == true - } - access_policy = null - access_policy_create = { - parent = "organizations/${var.organization.id}" - title = "tenant-${each.key}" - scopes = [module.tenant-core-folder[each.key].id] - } - iam_bindings_additive = merge( - { - # uncomment this if tenant admins are allowed by org-level DRS policy - # tenant_admins = { - # role = "roles/accesscontextmanager.policyAdmin" - # member = each.value.admin_principal - # } - tenant_sa = { - role = "roles/accesscontextmanager.policyAdmin" - member = module.tenant-sa[each.key].iam_email - } - }, - each.value.fast_config == null ? {} : { - tenant_sa_resman = { - role = "roles/accesscontextmanager.policyAdmin" - member = module.tenant-automation-tf-resman-sa[each.key].iam_email - } - tenant_sa_security = { - role = "roles/accesscontextmanager.policyAdmin" - member = module.tenant-automation-tf-security-sa[each.key].iam_email - } - tenant_sa_security_r = { - role = "roles/accesscontextmanager.policyReader" - member = module.tenant-automation-tf-security-r-sa[each.key].iam_email - } - } - ) -} diff --git a/fast/addons/1-resman-tenants/tenant.tf b/fast/addons/1-resman-tenants/tenant.tf deleted file mode 100644 index f53fb0338..000000000 --- a/fast/addons/1-resman-tenants/tenant.tf +++ /dev/null @@ -1,124 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Per-tenant resources. - -module "tenant-folder" { - source = "../../../modules/folder" - for_each = local.tenants - parent = module.tenant-core-folder[each.key].id - name = each.value.descriptive_name - contacts = ( - each.value.fast_config != null - ? {} - : { - (split(":", each.value.admin_principal)[1]) = ["ALL"] - } - ) -} - -module "tenant-folder-iam" { - source = "../../../modules/folder" - for_each = local.tenants - id = module.tenant-folder[each.key].id - folder_create = false - iam = { - "roles/logging.admin" = compact([ - each.value.admin_principal, - module.tenant-sa[each.key].iam_email, - try(module.tenant-automation-tf-resman-sa[each.key].iam_email, null) - ]) - "roles/owner" = [ - each.value.admin_principal, - module.tenant-sa[each.key].iam_email - ] - "roles/resourcemanager.folderAdmin" = compact([ - each.value.admin_principal, - module.tenant-sa[each.key].iam_email, - try(module.tenant-automation-tf-resman-sa[each.key].iam_email, null) - ]) - "roles/resourcemanager.projectCreator" = compact([ - each.value.admin_principal, - module.tenant-sa[each.key].iam_email, - try(module.tenant-automation-tf-resman-sa[each.key].iam_email, null) - ]) - "roles/serviceusage.serviceUsageViewer" = compact([ - try(module.tenant-automation-tf-resman-r-sa[each.key].iam_email, null) - ]) - "roles/resourcemanager.tagAdmin" = compact([ - try(module.tenant-automation-tf-resman-sa[each.key].iam_email, null) - ]) - "roles/resourcemanager.tagUser" = compact([ - try(module.tenant-automation-tf-resman-sa[each.key].iam_email, null) - ]) - "roles/viewer" = compact([ - try(module.tenant-automation-tf-resman-r-sa[each.key].iam_email, null) - ]) - } - iam_bindings = each.value.fast_config == null ? {} : { - tenant_iam_admin_conditional = { - members = [module.tenant-automation-tf-resman-sa[each.key].iam_email] - role = "roles/resourcemanager.folderIamAdmin" - condition = { - expression = format( - "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])", - join(",", formatlist("'%s'", [ - "roles/accesscontextmanager.policyAdmin", - "roles/cloudasset.viewer", - "roles/compute.orgFirewallPolicyAdmin", - "roles/compute.xpnAdmin", - var.custom_roles["tenant_network_admin"] - ])) - ) - title = "tenant_automation_sa_delegated_grants" - description = "Automation service account delegated grants." - } - } - } - depends_on = [module.tenant-automation-project] -} - -# automation service account - -module "tenant-sa" { - source = "../../../modules/iam-service-account" - for_each = local.tenants - project_id = var.automation.project_id - name = "${var.names.resource_short_name}-${each.key}-0" - display_name = "Terraform tenant ${each.key} service account." - prefix = var.prefix - iam = { - "roles/iam.serviceAccountTokenCreator" = [each.value.admin_principal] - } - iam_project_roles = { - (var.automation.project_id) = ["roles/serviceusage.serviceUsageConsumer"] - } -} - -# automation bucket - -module "tenant-gcs" { - source = "../../../modules/gcs" - for_each = local.tenants - project_id = var.automation.project_id - name = "${var.names.resource_short_name}-${each.key}-0" - prefix = var.prefix - location = each.value.locations.gcs - versioning = true - iam = { - "roles/storage.objectAdmin" = [module.tenant-sa[each.key].iam_email] - } -} diff --git a/fast/addons/1-resman-tenants/variables-fast.tf b/fast/addons/1-resman-tenants/variables-fast.tf deleted file mode 100644 index 81c68ef2b..000000000 --- a/fast/addons/1-resman-tenants/variables-fast.tf +++ /dev/null @@ -1,161 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description FAST stage interface. - -variable "automation" { - # tfdoc:variable:source 0-bootstrap - description = "Automation resources created by the bootstrap stage." - type = object({ - outputs_bucket = string - project_id = string - project_number = string - federated_identity_pool = string - federated_identity_providers = map(object({ - audiences = list(string) - issuer = string - issuer_uri = string - name = string - principal_branch = string - principal_repo = string - })) - service_accounts = object({ - resman = string - resman-r = string - }) - }) -} - -variable "billing_account" { - # tfdoc:variable:source 0-bootstrap - description = "Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`." - type = object({ - id = string - is_org_level = optional(bool, true) - no_iam = optional(bool, false) - }) - nullable = false -} - -variable "custom_roles" { - # tfdoc:variable:source 0-bootstrap - description = "Custom roles defined at the org level, in key => id format." - type = object({ - billing_viewer = string - dns_zone_binder = string - kms_key_encryption_admin = string - kms_key_viewer = string - organization_admin_viewer = string - project_iam_viewer = string - service_project_network_admin = string - storage_viewer = string - gcve_network_admin = optional(string) - gcve_network_viewer = optional(string) - network_firewall_policies_admin = optional(string) - ngfw_enterprise_admin = optional(string) - ngfw_enterprise_viewer = optional(string) - tenant_network_admin = string - }) - default = null -} - -variable "environments" { - # tfdoc:variable:source 0-globals - description = "Environment names." - type = map(object({ - name = string - short_name = string - tag_name = string - is_default = optional(bool, false) - })) - nullable = false - validation { - condition = anytrue([ - for k, v in var.environments : v.is_default == true - ]) - error_message = "At least one environment should be marked as default." - } -} - -variable "groups" { - # tfdoc:variable:source 0-bootstrap - # https://cloud.google.com/docs/enterprise/setup-checklist - description = "Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated." - type = object({ - gcp-billing-admins = optional(string, "gcp-billing-admins") - gcp-devops = optional(string, "gcp-devops") - gcp-network-admins = optional(string, "gcp-vpc-network-admins") - gcp-organization-admins = optional(string, "gcp-organization-admins") - gcp-security-admins = optional(string, "gcp-security-admins") - }) - nullable = false - default = {} -} - -variable "locations" { - # tfdoc:variable:source 0-bootstrap - description = "Optional locations for GCS, BigQuery, and logging buckets created here." - type = object({ - bq = optional(string, "EU") - gcs = optional(string, "EU") - logging = optional(string, "global") - pubsub = optional(list(string), []) - }) - nullable = false - default = {} -} - -variable "logging" { - # tfdoc:variable:source 0-bootstrap - description = "Logging resources created by the bootstrap stage." - type = object({ - project_id = string - }) - nullable = false -} - -variable "organization" { - # tfdoc:variable:source 0-bootstrap - description = "Organization details." - type = object({ - domain = string - id = number - customer_id = string - }) -} - -variable "org_policy_tags" { - # tfdoc:variable:source 0-bootstrap - description = "Organization policy tags." - type = object({ - key_id = string - key_name = string - values = map(string) - }) -} - -check "prefix_validator" { - assert { - condition = (try(length(var.prefix), 0) < 10) || (try(length(var.prefix), 0) < 12 && var.root_node != null) - error_message = "var.prefix must be 9 characters or shorter for organizations, and 11 chars or shorter for tenants." - } -} - -variable "prefix" { - # tfdoc:variable:source 0-bootstrap - description = "Prefix used for resources that need unique names. Use 9 characters or less." - type = string -} diff --git a/fast/addons/1-resman-tenants/variables.tf b/fast/addons/1-resman-tenants/variables.tf deleted file mode 100644 index d9af1de91..000000000 --- a/fast/addons/1-resman-tenants/variables.tf +++ /dev/null @@ -1,125 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# TODO: backport names variable from resman stage -variable "names" { - description = "Configuration for names used for resources and output files." - type = object({ - output_files_prefix = optional(string, "2-resman-tenants") - resource_short_name = optional(string, "tn") - }) - nullable = false - default = {} -} - -variable "outputs_location" { - description = "Path where providers and tfvars files for the following stages are written. Leave empty to disable." - type = string - default = null -} - -variable "root_node" { - description = "Root folder under which tenants are created, in folders/nnnn format. Defaults to the organization if null." - type = string - default = null - validation { - condition = ( - var.root_node == null || - startswith(coalesce(var.root_node, "-"), "folders/") - ) - error_message = "Root node must be a folder in folders/nnnn format." - } -} - -variable "tag_names" { - description = "Customized names for resource management tags." - type = object({ - tenant = optional(string, "tenant") - }) - default = {} - nullable = false - validation { - condition = alltrue([for k, v in var.tag_names : v != null]) - error_message = "Tag names cannot be null." - } -} - -variable "tenant_configs" { - description = "Tenant configurations. Keys are the short names used for naming resources and should not be changed once defined." - type = map(object({ - admin_principal = string - descriptive_name = string - billing_account = optional(object({ - id = optional(string) - # is_org_level is only meaningful when using the org BA - # and set implicitly in tenant locals - no_iam = optional(bool, true) - }), {}) - cloud_identity = optional(object({ - customer_id = string - domain = string - id = string - })) - locations = optional(object({ - bq = optional(string, "EU") - gcs = optional(string, "EU") - logging = optional(string, "global") - pubsub = optional(list(string), []) - })) - fast_config = optional(object({ - cicd_config = optional(object({ - name = string - type = string - branch = optional(string) - identity_provider = optional(string) - })) - groups = optional(object({ - gcp-billing-admins = optional(string, "gcp-billing-admins") - gcp-devops = optional(string, "gcp-devops") - gcp-network-admins = optional(string, "gcp-vpc-network-admins") - gcp-organization-admins = optional(string, "gcp-organization-admins") - gcp-security-admins = optional(string, "gcp-security-admins") - gcp-support = optional(string, "gcp-devops") - })) - prefix = optional(string) - workload_identity_providers = optional(map(object({ - attribute_condition = optional(string) - issuer = string - custom_settings = optional(object({ - issuer_uri = optional(string) - audiences = optional(list(string), []) - jwks_json = optional(string) - }), {}) - })), {}) - })) - vpc_sc_policy_create = optional(bool, false) - })) - nullable = false - default = {} - validation { - condition = alltrue([ - for k, v in var.tenant_configs : - length(coalesce(try(v.fast_config.prefix, null), "-")) < 11 - ]) - error_message = "Tenant prefix too long, use a maximum of 10 characters." - } - validation { - condition = alltrue([ - for k, v in var.tenant_configs : length(k) <= 3 - ]) - error_message = "Tenant short name too long, use a maximum of 3 characters." - } -} diff --git a/fast/stages/0-bootstrap-experimental/README.md b/fast/stages/0-bootstrap-experimental/README.md deleted file mode 100644 index c7bce05a9..000000000 --- a/fast/stages/0-bootstrap-experimental/README.md +++ /dev/null @@ -1,648 +0,0 @@ -# FAST Light Bootstrap (Experimental) - - -- [TODO](#todo) -- [Quickstart](#quickstart) - - [Prerequisites](#prerequisites) - - [Select/configure a factory dataset](#selectconfigure-a-factory-dataset) - - [Configure defaults](#configure-defaults) - - [Initial user permissions](#initial-user-permissions) - - [First apply cycle](#first-apply-cycle) - - [Importing org policies](#importing-org-policies) - - [Local output files storage](#local-output-files-storage) - - [Init and apply the stage](#init-and-apply-the-stage) - - [Provider setup and final apply cycle](#provider-setup-and-final-apply-cycle) -- [Default factory datasets](#default-factory-datasets) - - ["Classic FAST" dataset](#classic-fast-dataset) - - ["Minimal" dataset](#minimal-dataset) - - ["Tenants" dataset](#tenants-dataset) -- [Detailed configuration](#detailed-configuration) - - [Factory data](#factory-data) - - [Defaults configuration](#defaults-configuration) - - [Billing account IAM](#billing-account-iam) - - [Context-based replacement in the billing account factory](#context-based-replacement-in-the-billing-account-factory) - - [Organization configuration](#organization-configuration) - - [Context-based replacement in organization factories](#context-based-replacement-in-organization-factories) - - [Resource management hierarchy](#resource-management-hierarchy) - - [Context-based replacement in the folders factory](#context-based-replacement-in-the-folders-factory) - - [Project factory](#project-factory) - - [CI/CD configuration](#cicd-configuration) -- [Leveraging classic FAST Stages](#leveraging-classic-fast-stages) - - [VPC Service Controls](#vpc-service-controls) - - [Security](#security) -- [Files](#files) -- [Variables](#variables) -- [Outputs](#outputs) - - -## TODO - -- [x] add support for log sinks to billing schema/code -- [ ] minimal dataset -- [ ] tenants dataset -- [x] clean up classic dataset -- [x] finish and review documentation - -This stage implements a flexible approach to organization bootstrapping and resource management, that offers full customization via YAML factories. - -It heavily relies on a new [project factory module](../../../modules/project-factory-experimental/) for folder and project configurations, and leverages a new approach to [context-based interpolation](../../../modules/project-factory-experimental/README.md#context-based-interpolation) that allows writing legible, portable YAML definitions. - -The default set of YAML configuration files in the `data` folder mirrors the traditional FAST layout, and implements full compatibility with existing FAST stages like VPC-SC, security, networking, etc. - -The default configuration can be used as a starting point to implement radically different Landing Zone designs, or trimmed down to its bare minimum where the requirements are simply to have a secure organization-level configuration (possibly with VPC-SC), and a working project factory. - -## Quickstart - -The high-level flow for running this stage is: - -- ensure all **pre-requisites** are in place, and identify at least one GCP organization admin principal (ideally a group) -- select the **factory data set** for the factories among those available - populate the **defaults file** with attributes matching your configuration (organization id, billing account, etc.) -(`data`, `data-minimal`, etc.) or edit/create your own -- assign a set of **initial IAM roles** to the admin principal -- run a **first init/apply cycle** using user credentials -- copy the generated provider file, **migrate state**, then run a second init/apply cycle using service account impersonated credentials - -### Prerequisites - -This stage only requires minimal prerequisites: - -- one organization -- credentials with admin access to the organization and one billing account - -The organization ideally needs to be empty. If pre-existing resources are present some care needs to be put into preserving their existing IAM and org policies, ideally my moving legacy projects to a dedicated folder where the current org-level configuration (IAM and org policies) can be replicated. - -Billing admin permissions are ideally available on either an org-contained billing account or an external one. If those are unavailable, the YAML configuration files need to be updated to remove billing IAM bindings, and those need to be assigned via an external flow. Refer to the [billing section](#billing-account-iam) for more details or non-standard configurations. - -The admin principal is typically a group that includes the user running the first apply, but any kind of principal is supported. More principals (network admins, security admins, etc.) are present in some of the [default factories datasets](#default-factory-datasets), and others can be added if needed by editing the YAML configuration files. - -### Select/configure a factory dataset - -The `factories_config` variable points to several paths containing the YAML configuration files used by this stage. The default variable configuration points to the legacy FAST compatible fileset in the `data` folder. - -If you are fine with this configuration nothing needs to be changed at this stage. To select a different setup create a `tfvars` file and set paths to the desired data folder, like shown in the example below. The different configurations produced by each fileset are described [later in this document](#default-factory-datasets). - -```bash -# create a file named 0-bootstrap.auto.tfvars containing the following -# and replace paths by pointing them to the desired data folder -factories_config = { - billing_accounts = "data/billing-accounts" - cicd = "data/cicd.yaml" - defaults = "data/defaults.yaml" - folders = "data/folders" - organization = "data/organization" - projects = "data/projects" -} -``` - -### Configure defaults - -Configurations defaults are stored in the `defaults.yaml` file in the dataset selected above. Before starting, edit the following attributes in the file to match your configuration. - -The standard datasets use the `gcp-organization-admins` alias to assign administrator roles. The alias is expanded via the `context.iam_principals` attribute in the default file, which should be set to a valid group. Also make sure that the user running the initial apply is a member. - -```yaml -global: - # gcloud beta billing accounts list - billing_account: 123456-123456-123456 - locations: - bigquery: europe-west1 - logging: europe-west1 - organization: - # gcloud organizations list - domain: example.org - id: 1234567890 - customer_id: ABC0123CDE -projects: - defaults: - # define a unique prefix with a maximum of 9 characters - prefix: foo-1 - storage_location: europe-west1 -context: - iam_principals: - # make sure the user running apply is a member of this group - gcp-organization-admins: group:fabric-fast-owners@example.com -``` - -A more detailed example containing a few other attributes that can be set in the file is in a [later section](#defaults-configuration) in this document. - -### Initial user permissions - -Like in classic FAST, the user running the first apply cycle needs specific permissions on the organization and billing account. Copy the following snippet, edit it to match your organization/billing account ids, then run each command. - -To quickly self-grant the above roles, run the following code snippet as the initial Organization Admin. The best approach is to use the same group used for organization admins above. - -```bash -export FAST_PRINCIPAL="group:fabric-fast-owners@example.com" - -# find your organization and export its id in the FAST_ORG variable -gcloud organizations list -export FAST_ORG_ID=123456 - -# set needed roles (billing role only needed for organization-owned account) -export FAST_ROLES="\ - roles/billing.admin \ - roles/logging.admin \ - roles/iam.organizationRoleAdmin \ - roles/orgpolicy.policyAdmin \ - roles/resourcemanager.folderAdmin \ - roles/resourcemanager.organizationAdmin \ - roles/resourcemanager.projectCreator \ - roles/resourcemanager.tagAdmin \ - roles/owner" - -for role in $FAST_ROLES; do - gcloud organizations add-iam-policy-binding $FAST_ORG_ID \ - --member $FAST_PRINCIPAL --role $role --condition None -done -``` - -If you are using an externally managed billing account, make sure user has Billing Admin role assigned on the account. - -### First apply cycle - -#### Importing org policies - -If your dataset includes org policies which are already set in the organization, you need to either comment them out from the relevant YAML files or tell this stage to import them. To figure out which policies are set, run `gcloud org-policies list --organization [your org id]`, then set the `org_policies_imports` variable in your tfvars file. The following is an example. - -```bash -gcloud org-policies list --organization 1234567890 -CONSTRAINT LIST_POLICY BOOLEAN_POLICY -iam.allowedPolicyMemberDomains SET - -compute.disableSerialPortAccess - SET -``` - -```tfvars -# create or edit the 0-bootstrap.auto.tfvars.file -org_policies_imports = [ - 'iam.allowedPolicyMemberDomains', - 'compute.disableSerialPortAccess' -] -``` - -Once org policies have been imported, the variable definition can be removed from the tfvars file. - -#### Local output files storage - -Like any other FAST stage, this stage creates output files that contain information about the resources it manages, or provide initial provider and backend configuration for the following stages. - -These files are only persisted by default on a special outputs bucket, but can additionally be also persisted to a local path. This is very useful during the initial deployment, as it allows rapid apply iteration cycles between stages, and provides an easy way to check or derive resource ids. - -To enable local output files storage, set the `outputs_location` variable in your tfvars file to a filesystem path dedicated to this organization's output files. The following snippet provides an example. - -```tfvars -# create or edit the 0-bootstrap.auto.tfvars.file -outputs_location = "~/fast-configs/test-0" -``` - -#### Init and apply the stage - -Once everything has been configured go through the standard Terraform init/apply cycle. - -```bash -terraform init -terraform apply -``` - -### Provider setup and final apply cycle - -When the first apply cycle has completed successfully, you are ready to switch Terraform to use the new GCS backend and service account credentials. - -The first step is to link the generated provider file, either copying it from the GCS bucket or linking it from the local path if it has been configured in the previous step. - -The instructions also assume that you have moved the `0-bootstrap.auto.tfvars` file (if you have one) to the GCS bucket or the local config files. This is good practice in order to have the tfvars file persisted, either via GCS or by committing it to a repository with the source code in a dedicated config folder. The file needs to be copied or moved by hand. Alternatively, the last copy/link command can be ignored. - -If local output files are available adjust the path, run the script, then copy/paste the resulting commands. - -```bash -# if local outputs file are available -../fast-links.sh ~/fast-configs/test-0 -# File linking commands for FAST Bootstrap. stage - -# provider file -ln -s /home/user/fast-configs/test-0/providers/0-bootstrap-providers.tf ./ - -# conventional location for this stage terraform.tfvars (manually managed) -ln -s /home/user/fast-configs/test-0/0-bootstrap.auto.tfvars ./ -``` - -If you did not configure local output files use the GCS bucket to fetch output files. The bucket name can be derived from the `tfvars.bootstrap.automation.outputs_bucket` Terraform output. Adjust the path, run the script, then copy/paste the resulting commands. - -```bash -../fast-links.sh gs://test0-prod-iac-core-0-iac-outputs -# File linking commands for FAST Bootstrap. stage - -# provider file -gcloud storage cp gs://test0-prod-iac-core-0-iac-outputs/providers/0-bootstrap-providers.tf ./ - -# conventional location for this stage terraform.tfvars (manually managed) -gcloud storage cp gs://test0-prod-iac-core-0-iac-outputs/0-bootstrap.auto.tfvars ./ -``` - -Once the provider file has been setup, migrate local state to the GCS backend and re-run apply. - -```bash -terraform init -migrate-state -terraform apply -``` - -## Default factory datasets - -A few example datasets are included with the stage, each implementing a different widely used organizational design. The datasets can be used as-is, potentially with slight changes to better suit specific use cases, or they can serve as a starting point to implement radically different approaches. - -### "Classic FAST" dataset - -This dataset implements a Classic FAST design that replicates legacy bootstrap and resource management stages. The resulting layout is easy to customize, and supports VPC SC, networking, security and potentially any FAST stage 3 directly as explained in a [later section](#leveraging-classic-fast-stages). - -The organizational layout mirrors the consolidated FAST one, where shared infrastructure (stage 2 and 3) is partitioned via folders at the top, and further subdivided in environment-level folders for data or fleet management (Stage 3). An example "Teams" folder allows hooking up an application-level project factory as a separate stage, which is then used to define per-team subdivisions and create projects. - -

- Classic FAST organization-level diagram. -

- -### "Minimal" dataset - -This dataset is meant as a minimalistic starting point for organizations where a security baseline and a project factory are all that's needed, at least initially. The design can then organically grow to support more functionality, converging to the Classic or other types of layouts. - -### "Tenants" dataset - -TBD - -## Detailed configuration - -The following sections explain how to configure and run this stage, and should be read in sequence when using it for the first time. - -### Factory data - -The resources created by this stage are controlled by several factories, which point to YAML configuration files and folders. Data locations for each factory are controlled via the `var.factories_config` variable, and each factory path can be overridden individually. - -The default paths point to the dataset in the `data` folder which deploys a FAST-compliant configuration. These are the available factories in this stage, with file-level factories based on a single YAML file, and folder-level factories based on sets of YAML files contained withing a filesystem folder: - -- **defaults** (`data/defaults.yaml`) \ - file-level factory to define stage defaults (organization id, locations, prefix, etc.) and static context mappings -- **billing_accounts** (`data/billing-accounts`) \ - folder-level factory where each YAML file defines billing-account level IAM for one billing account; only used for externally managed accounts -- **organization** (`data/organization/.config.yaml`) \ - file-level factory to define organization IAM and log sinks - - **custom roles** (`data/organization/custom-roles`) \ - folder-level factory to define organization-level custom roles - - **org policies** (`data/organization/org-policies`) \ - folder-level factory to define organization-level org policies - - **tags** (`data/organization/tags`) \ - folder-level factory to define organization-level resource management tags -- **folders** (`data/folders`) \ - folder-level factory to define the resource management hierarchy and individual folder attributes (IAM, org policies, tag bindings, etc.); also supports defining folder-level IaC resources -- **projects** (`data/projects`) \ - folder-level factory to define projects and their attributes (projejct factory) -- **cicd** (`data/cicd.yaml`) \ - file-level factory to define CI/CD configurations for this and subsequent stages - -### Defaults configuration - -The prerequisite configuration for this stage is done via a `defaults.yaml` file, which implements part or all of the [relevant JSON schema](./schemas/defaults.schema.json). The location of the file defaults to `data/defaults.yaml` but can be easily changed via the `factories_config.defaults` variable. - -This is a commented example of a defaults file, showing a minimal working configuration. Refer to the YAML schema for all available options. - -```yaml -# global defaults used by bootstrap and persisted in the globals output file -global: - # billing account also set as default in the internal project factory - billing_account: 123456-123456-123456 - # default locations for this stage resources - locations: - bigquery: europe-west1 - logging: europe-west1 - # organization attributes (id is required) - organization: - domain: example.org - id: 1234567890 - customer_id: ABC0123CDE -# project defaults and overrides used by the internal project factory -projects: - defaults: - # setting a prefix either here or in overrides is required - prefix: foo-1 - # default location for storage buckets - storage_location: europe-west1 - overrides: {} -# FAST output files generated by this stage -output_files: - # optional path for locally persisted output files - local_path: ~/fast-config/foo-1 - # required storage bucket for output files (supports context interpolation) - storage_bucket: $storage_buckets:iac-0/iac-outputs - # FAST stage provider files (supports context interpolation) - providers: - 0-bootstrap: - bucket: $storage_buckets:iac-0/iac-bootstrap-state - service_account: $iam_principals:service_accounts/iac-0/iac-bootstrap-rw - # [...] -# static values added to context interpolation tables and used in factories -context: - iam_principals: - gcp-organization-admins: group:fabric-fast-owners@example.com -``` - -### Billing account IAM - -FAST traditionally supports three different billing configurations: - -- billing account in the same organization, where billing IAM is set via organization-level bindings -- external billing account, where billing IAM is set via account-level bindings -- no billing IAM, where FAST assumes bindings are managed by some externally defined process - -This stage allows the same flexibility, and even makes it possible to mix and match approaches by making billing IAM explicit: - -- if billing-account level IAM bindings are needed, they can be set via the billing account factory -- if organization-level IAM bindings are needed, they can be set via the organization factory -- if no billing IAM can be managed here, it's enough to disable the billing account factory by pointing it to an empty or non-existent filesystem folder - -The default dataset assumes an externally managed billing account is used, and configures its IAM accordingly via the billing account factory. The example below shows some of the IAM bindings configured at the billing account level, and how context-based interpolation is used there. - -
-Context-based replacement examples for the billing acccounts factory - -#### Context-based replacement in the billing account factory - -Principal expansion leverages the `$iam_principals:` context, which is populated from the static mappings defined in defaults, and the service accounts generated via the internal project factory [described in a later section](#project-factory). Log sink definitions also support `$project_ids:` and `$storage_buckets` expansions. - -```yaml -# example billing account factory file -# file: billing-accounts/default.yaml -id: $defaults:billing_account -iam_bindings_additive: - billing_admin_org_admins: - role: roles/billing.admin - # statically defined principal (via defaults.yaml) - member: $iam_principals:gcp-organization-admins - billing_admin_bootstrap_sa: - role: roles/billing.admin - # internally managed principal (project factory service account) - member: $iam_principals:service_accounts/iac-0/iac-bootstrap-rw -logging_sinks: - test: - description: Test sink - destination: $project_ids:log-0 - type: project -``` - -
- -### Organization configuration - -The default dataset implements a classic FAST design, re-creating the required custom roles, IAM bindings, org policies, tags, and log sinks via the factories described in a previous section. - -Compared to classic FAST this approach makes org-level configuration explicit, allowing easy customization of IAM and all other attributes. Before running this stage, check that the data files match your expected design. - -Context-based interpolation is heavily used in the organization configuration files to refer to external or project-level resources, so as to make the factory files portable. Some examples are provided below to better illustrate usage and facilitate editing organization-level data. - -
-Context-based replacement examples for organization factories - -#### Context-based replacement in organization factories - -Principal expansion leverages the `$iam_principals:` context, which is populated from the static mappings defined in defaults, and the service accounts generated via the internal project factory [described in a later section](#project-factory). - -```yaml -# example principal-level context interpolation -# file: data/organization/.config.yaml -iam_by_principals: - # statically defined principal (via defaults.yaml) - $iam_principals:gcp-organization-admins: - - roles/cloudasset.owner - - roles/cloudsupport.admin - - roles/compute.osAdminLogin - # [...] - # internally managed principal (project factory service account) - $iam_principals:service_accounts/iac-0/iac-bootstrap-rw: - - roles/accesscontextmanager.policyAdmin - - roles/cloudasset.viewer - - roles/essentialcontacts.admin - # [...] -``` - -Log sinks can refer to project-level destination via different contexts. - -```yaml -# example log sinks showing different destination contexts -# file: data/organization/.config.yaml -logging: - storage_location: $locations:default - sinks: - # log bucket destination - audit-logs: - destination: $log_buckets:log-0/audit-logs - filter: | - log_id("cloudaudit.googleapis.com/activity") OR - log_id("cloudaudit.googleapis.com/system_event") OR - log_id("cloudaudit.googleapis.com/policy") OR - log_id("cloudaudit.googleapis.com/access_transparency") - # storage bucket destination - iam: - destination: $storage_buckets:log-0/iam-sink - filter: | - protoPayload.serviceName="iamcredentials.googleapis.com" OR - protoPayload.serviceName="iam.googleapis.com" OR - protoPayload.serviceName="sts.googleapis.com" - # project destination - vpc-sc: - destination: $projject_ids:log-0 - filter: | - protoPayload.metadata.@type="type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata" -``` - -Context-based expansion is not limited to the organization's `.config.yaml` file, but is also available in the other factories, like in this example for the organization-level tag factory. - -```yaml -# example usage of context interpolation in tag values IAM -# file: data/organization/tags/environment.yaml -description: "Organization-level environments." -values: - development: - description: "Development." - iam: - "roles/resourcemanager.tagUser": - - $iam_principals:service_accounts/iac-0/iac-networking-rw - - $iam_principals:service_accounts/iac-0/iac-security-rw - - $iam_principals:service_accounts/iac-0/iac-pf-rw - "roles/resourcemanager.tagViewer": - - $iam_principals:service_accounts/iac-0/iac-networking-ro - - $iam_principals:service_accounts/iac-0/iac-security-ro - - $iam_principals:service_accounts/iac-0/iac-pf-ro - # [...] -``` - -An exception to the namespaced-based context replacements is in IAM conditions, where Terraform limitations force use of native string templating, as in the example below. - -```yaml -iam_bindings: - pf_org_policy_admin: - role: roles/orgpolicy.policyAdmin - members: - - $iam_principals:service_accounts/iac-0/iac-pf-rw - condition: - # $organization is set as a string template variable by the module - expression: resource.matchTag('${organization}/context', 'project-factory') - title: Project factory org policy admin -``` - -
- -### Resource management hierarchy - -The folder hierarchy is managed via a filesystem tree of YAML configuration files, and leverages the [project factory module](../../../modules/project-factory-experimental/README.md#folder-hierarchy) implementation, which supports up to 3 levels of folders (4 or more can be easily implemented in the module if needed). The module documentation provides additional information on this factory usage and formats. - -The default dataset implements a classic FAST layout, with top-level folders for stage 2 and stage 3, and can be easily tweaked by adding or removing any needed folder. - -```bash -data/folders -├── networking -│   ├── .config.yaml -│   ├── dev -│   │   └── .config.yaml -│   └── prod -│   └── .config.yaml -├── security -│   └── .config.yaml -└── teams - └── .config.yaml -``` - -Different layouts are very easy to implement by simply modeling the desired hierarchy in the filesystem, and configuring each folder via `.config.yaml` files. - -The project factory also supports embedding folder-aware project definitions in folders, but that approach is best used with caution to prevent potential race conditions when moving or deleting folders and projects. - -As with the factories described above, context replacements can be used in folder configurations. Some examples are provided below. - -
-Context-based replacement examples for the folder factory - -#### Context-based replacement in the folders factory - -As with other examples before, the main use case is to infer IAM principals from either the static or internally defined context. One additional context which is often useful here is tag values, which allows defining a scope for organization-level conditional IAM bindings or org policies. - -```yaml -# file: data/folders/teams/.config.yaml -name: Teams -iam_by_principals: - $iam_principals:service_accounts/iac-0/iac-pf-rw: - - roles/owner - - roles/resourcemanager.folderAdmin - # [...] -tag_bindings: - context: $tag_values:context/project-factory -``` - -
- -### Project factory - -The project factory is managed via a set of YAML configuration files, which like folders leverages the [project factory module](../../../modules/project-factory-experimental/README.md#folder-hierarchy) implementation. The module documentation provides additional information on this factory usage and formats. - -The default dataset implements a classic FAST layout, with two top-level projects for log exports and IaC resources. Those projects can easily be changed, for example rooting them in a folder by specifying the folder id or context name in their `parent` attribute. - -The provided project configurations also create several key resources for the stage like log buckets, storage buckets, and service accounts. Context-based expansions for projects are very similar to the ones defined for folders, you can refer to the above section for details. - -### CI/CD configuration - -CI/CD support is implemented in a similar way to classic/legacy FAST, except for being driven by a factory tha points to a single file. - -This allows defining a single Workload Identity provider that will be used to exchange external tokens for the pipelines, and one or more workflows that can interpolate internal (from the project factory) or external (user defined) attributes. - -This is the default file which implements a workflow for this stage. To enable it, pass the file path to the `factories_config.cicd` variable. - -```yaml -workload_identity_federation: - pool_name: iac-0 - project: $project_ids:iac-0 - providers: - github: - # the condition is optional but recommented, use your GitHub org name - attribute_condition: attribute.repository_owner=="my_org" - issuer: github - # custom_settings: - # issuer_uri: - # audiences: [] - # jwks_json_path: -workflows: - bootstrap: - template: github - workload_identity_provider: - id: $wif_providers:github - audiences: [] - repository: - name: bootstrap - branch: main - output_files: - storage_bucket: $storage_buckets:iac-0/iac-outputs - providers: - apply: $output_files:providers/0-bootstrap - plan: $output_files:providers/0-bootstrap-ro - files: - - tfvars/0-boostrap.auto.tfvars.json - service_accounts: - apply: $iam_principals:service_accounts/iac-0/iac-bootstrap-cicd-rw - plan: $iam_principals:service_accounts/iac-0/iac-bootstrap-cicd-ro -``` - -## Leveraging classic FAST Stages - -Classic Fast stage 2 and 3 can be directly used after applying this if the [Classic FAST layout](#classic-fast-dataset) is used, or similar identities and permissions are implemented in a different design. - -Specific changes or considerations needed for each stage are described below. - -### VPC Service Controls - -To use the predefined logging ingress policy in the VPC SC stage, define it like in the following example. - -```yaml -from: - access_levels: - - "*" - identities: - - $identity_sets:logging_identities -to: - operations: - - service_name: "*" - resources: - - $project_numbers:log-0 -``` - -### Security - -Define values for the `var.environments` variable in a tfvars file. - - - -## Files - -| name | description | modules | resources | -|---|---|---|---| -| [billing.tf](./billing.tf) | None | billing-account | | -| [cicd.tf](./cicd.tf) | None | | google_iam_workload_identity_pool · google_iam_workload_identity_pool_provider · google_storage_bucket_object · local_file | -| [factory.tf](./factory.tf) | None | project-factory-experimental | | -| [imports.tf](./imports.tf) | None | | | -| [main.tf](./main.tf) | Module-level locals and resources. | | terraform_data | -| [organization.tf](./organization.tf) | None | organization | | -| [output-files.tf](./output-files.tf) | None | | google_storage_bucket_object · local_file | -| [outputs.tf](./outputs.tf) | Module outputs. | | | -| [variables.tf](./variables.tf) | Module variables. | | | -| [wif-definitions.tf](./wif-definitions.tf) | Workload Identity provider definitions. | | | - -## Variables - -| name | description | type | required | default | -|---|---|:---:|:---:|:---:| -| [bootstrap_user](variables.tf#L17) | Email of the nominal user running this stage for the first time. | string | | null | -| [context](variables.tf#L23) | Context-specific interpolations. | object({…}) | | {} | -| [factories_config](variables.tf#L43) | Configuration for the resource factories or external data. | object({…}) | | {} | -| [org_policies_imports](variables.tf#L57) | List of org policies to import. These need to also be defined in data files. | list(string) | | [] | - -## Outputs - -| name | description | sensitive | -|---|---|:---:| -| [iam_principals](outputs.tf#L17) | IAM principals. | | -| [locations](outputs.tf#L22) | Default locations. | | -| [projects](outputs.tf#L27) | Attributes for managed projects. | | -| [tfvars](outputs.tf#L32) | Stage tfvars. | | - diff --git a/fast/stages/0-bootstrap-experimental/billing.tf b/fast/stages/0-bootstrap-experimental/billing.tf deleted file mode 100644 index 1e098327f..000000000 --- a/fast/stages/0-bootstrap-experimental/billing.tf +++ /dev/null @@ -1,73 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - _billing_accounts_path = try( - pathexpand(var.factories_config.billing_accounts), null - ) - _billing_accounts_raw = { - for f in try(fileset(local._billing_accounts_path, "*.yaml"), []) : - trimsuffix(f, ".yaml") => merge( - { id = null }, - yamldecode(file("${local._billing_accounts_path}/${f}")) - ) - } - billing_accounts = { - for k, v in local._billing_accounts_raw : k => merge(v, { - id = ( - local.defaults.billing_account != null && v.id == "$defaults:billing_account" - ? local.defaults.billing_account - : v.id - ) - logging_sinks = lookup(v, "logging_sinks", {}) - }) if v.id != null - } -} - -module "billing-accounts" { - source = "../../../modules/billing-account" - for_each = local.billing_accounts - id = each.value.id - context = merge(local.ctx, { - custom_roles = merge( - local.ctx.custom_roles, module.organization[0].custom_role_id - ) - iam_principals = merge( - local.ctx.iam_principals, - module.factory.iam_principals - ) - project_ids = merge( - local.ctx.project_ids, module.factory.project_ids - ) - storage_buckets = module.factory.storage_buckets - tag_keys = merge( - local.ctx.tag_keys, - local.org_tag_keys - ) - tag_values = merge( - local.ctx.tag_values, - local.org_tag_values - ) - }) - iam = lookup(each.value, "iam", {}) - iam_by_principals = lookup(each.value, "iam_by_principals", {}) - iam_bindings = lookup(each.value, "iam_bindings", {}) - iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) - logging_sinks = { - for k, v in each.value.logging_sinks : k => v - if lookup(v, "destination", null) != null && lookup(v, "type", null) != null - } -} diff --git a/fast/stages/0-bootstrap-experimental/cicd.tf b/fast/stages/0-bootstrap-experimental/cicd.tf deleted file mode 100644 index 6e329b7b7..000000000 --- a/fast/stages/0-bootstrap-experimental/cicd.tf +++ /dev/null @@ -1,138 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - _cicd = try(yamldecode(file(local.paths.cicd)), {}) - _cicd_identity_providers = { - for k, v in google_iam_workload_identity_pool_provider.default : - "$wif_providers:${k}" => v.id - } - _cicd_output_files = { - for k, v in google_storage_bucket_object.providers : - "$output_files:providers/${k}" => v.name - } - cicd_project_ids = { - for k, v in merge( - var.context.project_ids, module.factory.project_ids - ) : "$project_ids:${k}" => v - } - cicd_workflows = { - for k, v in lookup(local._cicd, "workflows", {}) : k => { - outputs_bucket = lookup( - local.of_buckets, - v.output_files.storage_bucket, - v.output_files.storage_bucket - ) - workflow = templatefile("assets/workflow-${v.template}.yaml", { - identity_provider = lookup( - local._cicd_identity_providers, - v.workload_identity_provider.id, - v.workload_identity_provider.id - ) - service_accounts = { - apply = lookup( - local.of_service_accounts, - v.service_accounts.apply, - v.service_accounts.apply - ) - plan = lookup( - local.of_service_accounts, - v.service_accounts.plan, - v.service_accounts.plan - ) - } - outputs_bucket = lookup( - local.of_buckets, - v.output_files.storage_bucket, - v.output_files.storage_bucket - ) - stage_name = k - tf_providers_files = { - apply = lookup( - local._cicd_output_files, - v.output_files.providers.apply, - v.output_files.providers.apply - ) - plan = lookup( - local._cicd_output_files, - v.output_files.providers.plan, - v.output_files.providers.plan - ) - } - tf_var_files = try(v.output_files.files, []) - }) - } - } - wif_project = try(local._cicd.workload_identity_federation.project, null) - wif_providers = { - for k, v in try(local._cicd.workload_identity_federation.providers, {}) : - k => merge(v, lookup(local.wif_defs, v.issuer, {})) - } -} - -resource "google_iam_workload_identity_pool" "default" { - count = local.wif_project == null ? 0 : 1 - project = lookup( - local.cicd_project_ids, local.wif_project, local.wif_project - ) - workload_identity_pool_id = try( - local._cicd.workload_identity_federation.pool_name, "iac-0" - ) -} - -resource "google_iam_workload_identity_pool_provider" "default" { - for_each = local.wif_providers - project = ( - google_iam_workload_identity_pool.default[0].project - ) - workload_identity_pool_id = ( - google_iam_workload_identity_pool.default[0].workload_identity_pool_id - ) - workload_identity_pool_provider_id = each.key - attribute_condition = lookup( - each.value, "attribute_condition", null - ) - attribute_mapping = lookup( - each.value, "attribute_mapping", {} - ) - oidc { - # Setting an empty list configures allowed_audiences to the url of the provider - allowed_audiences = try(each.value.custom_settings.audiences, []) - # If users don't provide an issuer_uri, we set the public one for the platform chosen. - issuer_uri = ( - try(each.value.custom_settings.issuer_uri, null) != null - ? each.value.custom_settings.issuer_uri - : try(each.value.issuer_uri, null) - ) - # OIDC JWKs in JSON String format. If no value is provided, they key is - # fetched from the `.well-known` path for the issuer_uri - jwks_json = try(each.value.custom_settings.jwks_json, null) - } -} - -resource "local_file" "workflows" { - for_each = local.of_path == null ? {} : local.cicd_workflows - file_permission = "0644" - filename = "${local.of_path}/workflows/${each.key}.yaml" - content = each.value.workflow -} - -resource "google_storage_bucket_object" "workflows" { - for_each = local.cicd_workflows - bucket = each.value.outputs_bucket - name = "workflows/${each.key}.yaml" - content = each.value.workflow -} diff --git a/fast/stages/0-bootstrap-experimental/main.tf b/fast/stages/0-bootstrap-experimental/main.tf deleted file mode 100644 index 45d7a5c9e..000000000 --- a/fast/stages/0-bootstrap-experimental/main.tf +++ /dev/null @@ -1,96 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - paths = { - for k, v in var.factories_config : k => try(pathexpand(v), null) - } - # fail if we have no valid defaults - _defaults = yamldecode(file(local.paths.defaults)) - ctx = merge(var.context, { - iam_principals = local.iam_principals - locations = { - for k, v in local.defaults.locations : - k => v if k != "pubsub" - } - }) - defaults = { - billing_account = try(local._defaults.global.billing_account, null) - locations = merge(try(local._defaults.global.locations, {}), { - bigquery = "eu" - logging = "global" - pubsub = [] - storage = "eu" - }) - organization = ( - try(local._defaults.global.organization.id, null) == null - ? null - : local._defaults.global.organization - ) - prefix = try( - local.project_defaults.defaults.prefix, - local.project_defaults.overrides.prefix, - null - ) - } - iam_principals = merge( - local.org_iam_principals, - var.context.iam_principals, - try(local._defaults.context.iam_principals, {}) - ) - output_files = { - local_path = try(local._defaults.output_files.local_path, null) - storage_bucket = try(local._defaults.output_files.storage_bucket, null) - providers = try(local._defaults.output_files.providers, {}) - } - project_defaults = { - defaults = try(local._defaults.projects.defaults, {}) - overrides = try(local._defaults.projects.overrides, {}) - } -} - -# TODO: streamine location replacements - -resource "terraform_data" "precondition" { - lifecycle { - precondition { - condition = try(local.defaults.billing_account, null) != null - error_message = "No billing account set in global defaults." - } - precondition { - condition = ( - local.organization_id != null || - try(local.project_defaults.defaults.parent, null) != null || - try(local.project_defaults.overrides.parent, null) != null - ) - error_message = "Project parent must be set in project defaults or overrides if no organization id is set." - } - precondition { - condition = ( - try(local.project_defaults.defaults.prefix, null) != null || - try(local.project_defaults.overrides.prefix, null) != null - ) - error_message = "Prefix must be set in project defaults or overrides." - } - precondition { - condition = ( - try(local.project_defaults.defaults.storage_location, null) != null || - try(local.project_defaults.overrides.storage_location, null) != null - ) - error_message = "Storage location must be set in project defaults or overrides." - } - } -} diff --git a/fast/stages/0-bootstrap-experimental/organization.tf b/fast/stages/0-bootstrap-experimental/organization.tf deleted file mode 100644 index 9df1aa54d..000000000 --- a/fast/stages/0-bootstrap-experimental/organization.tf +++ /dev/null @@ -1,125 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - # prepare organization data - organization = merge( - # initialize required attributes - { domain = null, id = null }, - # merge defaults - lookup(local.defaults, "organization", {}), - # merge attributes defined in yaml - try(yamldecode(file("${local.paths.organization}/.config.yaml")), {}) - ) - # interpolate organization id if required - organization_id = ( - local.organization.id == "$defaults:organization/id" - ? try(local.defaults.organization.id, local.organization.id) - : local.organization.id - ) - # build map of predefined groups if organization domain is set - org_iam_principals = local.organization.domain == null ? {} : { - domain = "domain:${local.organization.domain}" - gcp-billing-admins = "group:gcp-billing-admins@${local.organization.domain}" - gcp-devops = "group:gcp-devops@${local.organization.domain}" - gcp-network-admins = "group:gcp-network-admins@${local.organization.domain}" - gcp-organization-admins = "group:gcp-organization-admins@${local.organization.domain}" - gcp-secops-admins = "group:gcp-secops-admins@${local.organization.domain}" - gcp-security-admins = "group:gcp-security-admins@${local.organization.domain}" - gcp-support = "group:gcp-support@${local.organization.domain}" - } - org_tag_keys = { - for k, v in module.organization[0].tag_keys : k => v.id - } - org_tag_values = { - for k, v in module.organization[0].tag_values : k => v.id - } -} - -module "organization" { - source = "../../../modules/organization" - count = local.organization_id != null ? 1 : 0 - organization_id = "organizations/${local.organization_id}" - logging_settings = lookup(local.organization, "logging", null) - context = { - locations = { - default = local.defaults.locations.logging - } - } - factories_config = { - org_policy_custom_constraints = "${local.paths.organization}/custom-constraints" - custom_roles = "${local.paths.organization}/custom-roles" - tags = "${local.paths.organization}/tags" - } - tags_config = { - ignore_iam = true - } -} - -module "organization-iam" { - source = "../../../modules/organization" - count = local.organization.id != null ? 1 : 0 - organization_id = module.organization[0].id - context = merge(local.ctx, { - custom_roles = merge( - local.ctx.custom_roles, module.organization[0].custom_role_id - ) - iam_principals = merge( - local.ctx.iam_principals, - module.factory.iam_principals - ) - log_buckets = module.factory.log_buckets - org_policies = { - organization = local.defaults.organization - tags = merge( - local.ctx.tag_values, - local.org_tag_values - ) - } - project_ids = merge( - local.ctx.project_ids, module.factory.project_ids - ) - storage_buckets = module.factory.storage_buckets - tag_keys = merge( - local.ctx.tag_keys, - local.org_tag_keys - ) - tag_values = merge( - local.ctx.tag_values, - local.org_tag_values - ) - }) - factories_config = { - org_policies = "${local.paths.organization}/org-policies" - tags = "${local.paths.organization}/tags" - } - iam = lookup( - local.organization, "iam", {} - ) - iam_by_principals = lookup( - local.organization, "iam_by_principals", {} - ) - iam_bindings = lookup( - local.organization, "iam_bindings", {} - ) - iam_bindings_additive = lookup( - local.organization, "iam_bindings_additive", {} - ) - logging_sinks = try(local.organization.logging.sinks, {}) - tags_config = { - force_context_ids = true - } -} diff --git a/fast/stages/0-bootstrap-experimental/outputs.tf b/fast/stages/0-bootstrap-experimental/outputs.tf deleted file mode 100644 index 27e0e1cc9..000000000 --- a/fast/stages/0-bootstrap-experimental/outputs.tf +++ /dev/null @@ -1,35 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -output "iam_principals" { - description = "IAM principals." - value = local.iam_principals -} - -output "locations" { - description = "Default locations." - value = local.defaults.locations -} - -output "projects" { - description = "Attributes for managed projects." - value = module.factory.projects -} - -output "tfvars" { - description = "Stage tfvars." - value = local.of_tfvars -} diff --git a/fast/stages/0-bootstrap-experimental/schemas/custom-role.schema.json b/fast/stages/0-bootstrap-experimental/schemas/custom-role.schema.json deleted file mode 100644 index d7526482c..000000000 --- a/fast/stages/0-bootstrap-experimental/schemas/custom-role.schema.json +++ /dev/null @@ -1,18 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-07/schema#", - "title": "Custom Role", - "type": "object", - "additionalProperties": false, - "properties": { - "name": { - "type": "string" - }, - "includedPermissions": { - "type": "array", - "items": { - "type": "string", - "pattern": "^[a-zA-Z-]+\\.[a-zA-Z-]+\\.[a-zA-Z-]+$" - } - } - } -} \ No newline at end of file diff --git a/fast/stages/0-bootstrap-experimental/schemas/org-policies.schema.json b/fast/stages/0-bootstrap-experimental/schemas/org-policies.schema.json deleted file mode 100644 index 6c29331ec..000000000 --- a/fast/stages/0-bootstrap-experimental/schemas/org-policies.schema.json +++ /dev/null @@ -1,76 +0,0 @@ -{ - "$schema": "http://json-schema.org/draft-07/schema#", - "title": "Organization Policies", - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-z-]+[a-zA-Z0-9\\.]+$": { - "type": "object", - "additionalProperties": false, - "properties": { - "inherit_from_parent": { - "type": "boolean" - }, - "reset": { - "type": "boolean" - }, - "rules": { - "type": "array", - "items": { - "type": "object", - "additionalProperties": false, - "properties": { - "allow": { - "$ref": "#/$defs/allow-deny" - }, - "deny": { - "$ref": "#/$defs/allow-deny" - }, - "enforce": { - "type": "boolean" - }, - "condition": { - "type": "object", - "additionalProperties": false, - "properties": { - "description": { - "type": "string" - }, - "expression": { - "type": "string" - }, - "location": { - "type": "string" - }, - "title": { - "type": "string" - } - } - }, - "parameters": { - "type": "string" - } - } - } - } - } - } - }, - "$defs": { - "allow-deny": { - "type": "object", - "additionalProperties": false, - "properties": { - "all": { - "type": "boolean" - }, - "values": { - "type": "array", - "items": { - "type": "string" - } - } - } - } - } -} diff --git a/fast/stages/0-bootstrap-experimental/variables.tf b/fast/stages/0-bootstrap-experimental/variables.tf deleted file mode 100644 index 1b4fabf92..000000000 --- a/fast/stages/0-bootstrap-experimental/variables.tf +++ /dev/null @@ -1,62 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -variable "bootstrap_user" { - description = "Email of the nominal user running this stage for the first time." - type = string - default = null -} - -variable "context" { - description = "Context-specific interpolations." - type = object({ - custom_roles = optional(map(string), {}) - folder_ids = optional(map(string), {}) - iam_principals = optional(map(string), {}) - locations = optional(map(string), {}) - kms_keys = optional(map(string), {}) - notification_channels = optional(map(string), {}) - project_ids = optional(map(string), {}) - service_account_ids = optional(map(string), {}) - tag_keys = optional(map(string), {}) - tag_values = optional(map(string), {}) - vpc_host_projects = optional(map(string), {}) - vpc_sc_perimeters = optional(map(string), {}) - }) - default = {} - nullable = false -} - -variable "factories_config" { - description = "Configuration for the resource factories or external data." - type = object({ - billing_accounts = optional(string, "data/billing-accounts") - cicd = optional(string) - defaults = optional(string, "data/defaults.yaml") - folders = optional(string, "data/folders") - organization = optional(string, "data/organization") - projects = optional(string, "data/projects") - }) - nullable = false - default = {} -} - -variable "org_policies_imports" { - description = "List of org policies to import. These need to also be defined in data files." - type = list(string) - nullable = false - default = [] -} diff --git a/fast/stages/0-bootstrap-experimental/.fast-stage.env b/fast/stages/0-bootstrap-legacy/.fast-stage.env similarity index 69% rename from fast/stages/0-bootstrap-experimental/.fast-stage.env rename to fast/stages/0-bootstrap-legacy/.fast-stage.env index c739edfc8..a842174c3 100644 --- a/fast/stages/0-bootstrap-experimental/.fast-stage.env +++ b/fast/stages/0-bootstrap-legacy/.fast-stage.env @@ -1,4 +1,4 @@ -FAST_STAGE_DESCRIPTION="FAST Bootstrap." +FAST_STAGE_DESCRIPTION="organization bootstrap" FAST_STAGE_LEVEL=0 FAST_STAGE_NAME=bootstrap # FAST_STAGE_DEPS="0-globals 0-bootstrap" diff --git a/fast/stages/0-bootstrap/IAM.md b/fast/stages/0-bootstrap-legacy/IAM.md similarity index 100% rename from fast/stages/0-bootstrap/IAM.md rename to fast/stages/0-bootstrap-legacy/IAM.md diff --git a/fast/stages/0-bootstrap-legacy/README.md b/fast/stages/0-bootstrap-legacy/README.md new file mode 100644 index 000000000..48c253e08 --- /dev/null +++ b/fast/stages/0-bootstrap-legacy/README.md @@ -0,0 +1,726 @@ +# Organization bootstrap (legacy) + +The primary purpose of this stage is to enable critical organization-level functionalities that depend on broad administrative permissions, and prepare the prerequisites needed to enable automation in this and future stages. + +It is intentionally simple, to minimize usage of administrative-level permissions and enable simple auditing and troubleshooting, and only deals with three sets of resources: + +- project, service accounts, and GCS buckets for automation +- projects, BQ datasets, and sinks for audit log and billing exports +- IAM bindings on the organization + +Use the following diagram as a simple high level reference for the following sections, which describe the stage and its possible customizations in detail. + +

+ Organization-level diagram +

+ + +- [Design overview and choices](#design-overview-and-choices) + - [User groups](#user-groups) + - [Organization-level IAM](#organization-level-iam) + - [Organization policies](#organization-policies) + - [Security Command Center Enterprise](#security-command-center-enterprise) + - [Tags and Organization Policy conditions](#tags-and-organization-policy-conditions) + - [Automation project and resources](#automation-project-and-resources) + - [Billing account](#billing-account) + - [Organization-level logging](#organization-level-logging) + - [Naming](#naming) + - [Workforce Identity Federation](#workforce-identity-federation) + - [Workload Identity Federation and CI/CD](#workload-identity-federation-and-cicd) +- [How to run this stage](#how-to-run-this-stage) + - [Prerequisites](#prerequisites) + - [Standalone billing account](#standalone-billing-account) + - [Preventing creation of billing-related IAM bindings](#preventing-creation-of-billing-related-iam-bindings) + - [Groups](#groups) + - [Configure variables](#configure-variables) + - [Output files and cross-stage variables](#output-files-and-cross-stage-variables) + - [Running the stage](#running-the-stage) +- [Customizations](#customizations) + - [Group names](#group-names) + - [IAM](#iam) + - [Log sinks and log destinations](#log-sinks-and-log-destinations) + - [Names and naming convention](#names-and-naming-convention) + - [Workload Identity Federation](#workload-identity-federation) + - [Project folders](#project-folders) + - [CI/CD repositories](#cicd-repositories) + - [Add-ons](#add-ons) +- [Files](#files) +- [Variables](#variables) +- [Outputs](#outputs) + + +## Design overview and choices + +As mentioned above, this stage only does the bare minimum required to bootstrap automation, and ensure that base audit and billing exports are in place from the start to provide some measure of accountability, even before the security configurations are applied in a later stage. + +It also sets up organization-level IAM bindings so the Organization Administrator role is only used here, trading off some design freedom for ease of auditing and troubleshooting, and reducing the risk of costly security mistakes down the line. The only exception to this rule is for the [Resource Management stage](../1-resman-legacy) service account, described below. + +### User groups + +User groups are important, not only here but throughout the whole automation process. They provide a stable frame of reference that allows decoupling the final set of permissions for each group, from the stage where entities and resources are created and their IAM bindings defined. For example, the final set of roles for the networking group is contributed by this stage at the organization level (XPN Admin, Cloud Asset Viewer, etc.), and by the Resource Management stage at the folder level. + +We have standardized the initial set of groups on those outlined in the [GCP Enterprise Setup Checklist](https://cloud.google.com/docs/enterprise/setup-checklist) to simplify adoption. They provide a comprehensive and flexible starting point that can suit most users. Adding new groups, or deviating from the initial setup is possible and reasonably simple, and it's briefly outlined in the customization section below. + +### Organization-level IAM + +The service account used in the [Resource Management stage](../1-resman-legacy) needs to be able to grant specific permissions at the organizational level, to enable specific functionality for subsequent stages that deal with network or security resources, or billing-related activities. + +In order to be able to assign those roles without having the full authority of the Organization Admin role, this stage defines a custom role that only allows setting IAM policies on the organization, and grants it via a [delegated role grant](https://cloud.google.com/iam/docs/setting-limits-on-granting-roles) that only allows it to be used to grant a limited subset of roles. + +In this way, the Resource Management service account can effectively act as an Organization Admin, but only to grant the specific roles it needs to control. + +One consequence of the above setup is the need to configure IAM bindings that can be assigned via the condition as non-authoritative, since those same roles are effectively under the control of two stages: this one and Resource Management. Using authoritative bindings for these roles (instead of non-authoritative ones) would generate potential conflicts, where each stage could try to overwrite and negate the bindings applied by the other at each `apply` cycle. + +A full reference of IAM roles managed by this stage [is available here](./IAM.md). + +### Organization policies + +It's often desirable to have organization policies deployed before any other resource in the org, so as to ensure compliance with specific requirements (e.g. location restrictions), or control the configuration of specific resources (e.g. default network at project creation or service account grants). + +To cover this use case, organization policies have been moved from the resource management to the bootstrap stage in FAST versions after 26.0.0. They are managed via the usual factory approach, and a [sample set of data files](./data/org-policies/) is included with this stage. They are not applied during the initial run when the `bootstrap_user` variable is set, to work around incompatibilities with user credentials. + +FAST uses unmanaged organization policies by default. For those who prefer managed policies, a separate sample set is available. To use these managed policies, configure `factories_config` as shown below. + +```tfvars +factories_config = { + org_policies = "data/org-policies-managed" +} +``` + +#### Security Command Center Enterprise + +The DRS policy mentioned above might make it complex to [enable Security Command Center Enterprise](https://cloud.google.com/security-command-center/docs/activate-enterprise-tier#verify_organization_policies). If this is the case, you can temporarily disable it via the Cloud Console, enable SCC Enterprise, then re-enable the policy. + +#### Tags and Organization Policy conditions + +Organization policy exceptions are managed via a dedicated resource management tag hierarchy, rooted in the `org-policies` tag key. A default condition is already present for the the `iam.allowedPolicyMemberDomains` constraint, that relaxes the policy on resources that have the `org-policies/allowed-policy-member-domains-all` tag value bound or inherited, and similarly for `essentialcontacts.allowedContactDomains` via the `allowed-essential-contacts-domains-all` tag value. + +Further tag values can be defined via the `org_policies_config.tag_values` variable, and IAM access can be granted on them via the same variable. Once a tag value has been created, its id can be used in constraint rule conditions. Note that only one tag value from a given tag key can be bound to a node (organization, folder, or project) in the resource hierarchy. Since these tag values are all rooted in the `org-policies` key, this limits the ability to apply fine-grained policy constraints. It may be more desirable to model policy overrides using coarser groups of tag values to create a policy "profile". For example, instead of separating `compute.skipDefaultNetworkCreation` and `compute.vmExternalIpAccess`, enforce both constraints by default and relax them both using the same tag value such as `sandbox`. See [tags overview](https://cloud.google.com/resource-manager/docs/tags/tags-overview) for more information. + +Management of the rest of the tag hierarchy is delegated to the resource management stage, as that is often intimately tied to the folder hierarchy design. + +The organization policy tag key and values managed by this stage have been added to the `0-bootstrap.auto.tfvars` stage, so that IAM can be delegated to the resource management or successive stages via their ids. + +The following example shows an example on how to define an additional tag value, and use it in a boolean constraint rule. + +This snippet defines a new tag value under the `org-policies` tag key via the `org_policies_config` variable, and assigns the permission to bind it to a group. + +```hcl +# stage 0 custom tfvars +org_policies_config = { + tag_values = { + compute-require-oslogin-false = { + description = "Bind this tag to set oslogin to false." + iam = { + "roles/resourcemanager.tagUser" = [ + "group:foo@example.com" + ] + } + } + } +} +# tftest skip +``` + +The above tag can be used to define a constraint condition via the `data/org-policies/compute.yaml` or similar factory file. The name of the tag can be referenced from the factory files using `tags.org_policies_config`, as shown below. + +```yaml +compute.requireOsLogin: + rules: + - enforce: true + - enforce: false + condition: + expression: resource.matchTag('${tags.org_policies_tag_name}', 'compute-require-oslogin-false') +``` + +### Automation project and resources + +One other design choice worth mentioning here is using a single automation project for all foundational stages. We trade off some complexity on the API side (single source for usage quota, multiple service activation) for increased flexibility and simpler operations, while still effectively providing the same degree of separation via resource-level IAM. + +### Billing account + +We support three use cases in regards to billing: + +- the billing account is part of this same organization, IAM bindings will be set at the organization level +- the billing account is not considered part of an organization (even though it might be), billing IAM bindings are set on the billing account itself +- billing IAM is managed separately, and no bindings should (or can) be set via Terraform, this requires a few extra steps and is definitely not recommended and mainly used for development purposes + +For same-organization billing, we configure a custom organization role that can set IAM bindings, via a delegated role grant to limit its scope to the relevant roles. + +For details on configuring the different billing account modes, refer to the [How to run this stage](#how-to-run-this-stage) section below. + +Because of limitations of API availability, manual steps have to be followed to enable billing export within billing project to BigQuery dataset `billing_export` which will be created as part of the bootstrap stage. The process to share billing data [is outlined here](https://cloud.google.com/billing/docs/how-to/export-data-bigquery-setup#enable-bq-export). + +### Organization-level logging + +We create organization-level log sinks early in the bootstrap process to ensure a proper audit trail is in place from the very beginning. By default, we provide log filters to capture [Cloud Audit Logs](https://cloud.google.com/logging/docs/audit), [VPC Service Controls violations](https://cloud.google.com/vpc-service-controls/docs/troubleshooting#vpc-sc-errors) and [Workspace Logs](https://cloud.google.com/logging/docs/audit/configure-gsuite-audit-logs) into logging buckets in the top-level audit logging project. + +An organization-level sink captures IAM data access logs, including authentication and impersonation events for service accounts. To manage logging costs, the default configuration enables IAM data access logging only within the automation project (where sensitive service accounts reside). For enhanced security across the entire organization, consider enabling these logs at the organization level. + +The [Customizations](#log-sinks-and-log-destinations) section explains how to change the logs captured and their destination. + +### Naming + +We are intentionally not supporting random prefix/suffixes for names, as that is an antipattern typically only used in development. It does not map to our customer's actual production usage, where they always adopt a fixed naming convention. + +What is implemented here is a fairly common convention, composed of tokens ordered by relative importance: + +- an organization-level static prefix less or equal to 9 characters (e.g. `myco` or `myco-gcp`) +- an optional tenant-level prefix, if using tenant factory +- an environment identifier (e.g. `prod`) +- a team/owner identifier (e.g. `sec` for Security) +- a context identifier (e.g. `core` or `kms`) +- an arbitrary identifier used to distinguish similar resources (e.g. `0`, `1`) + +> [!WARNING] +> When using tenant factory, a tenant prefix will be automatically generated as `{prefix}-{tenant-shortname}`. The maximum length of such prefix must be 11 characters or less, which means that the longer org-level prefix you use, the less chars you'll have available for the `tenant-shortname`. + +Tokens are joined by a `-` character, making it easy to separate the individual tokens visually, and to programmatically split them in billing exports to derive initial high-level groupings for cost attribution. + +The convention is used in its full form only for specific resources with globally unique names (projects, GCS buckets). Other resources adopt a shorter version for legibility, as the full context can always be derived from their project. + +The [Customizations](#names-and-naming-convention) section on names below explains how to configure tokens, or implement a different naming convention. + +### Workforce Identity Federation + +This stage supports configuration of [Workforce Identity Federation](https://cloud.google.com/iam/docs/workforce-identity-federation) which lets an external identity provider (IdP) to authenticate and authorize a group of users (usually employees) using IAM, so that the users can access Google Cloud services. + +The following example shows an example on how to define a Workforce Identity pool for the organization. + +```hcl +# stage 0 wif tfvars +workforce_identity_providers = { + test = { + issuer = "azuread" + display_name = "wif-provider" + description = "Workforce Identity pool" + saml = { + idp_metadata_xml = "..." + } + } +} +# tftest skip +``` + +### Workload Identity Federation and CI/CD + +This stage also implements initial support for two interrelated features + +- configuration of [Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation) pools and providers +- configuration of CI/CD repositories to allow impersonation via Workload identity Federation, and stage running via provided workflow templates + +Workload Identity Federation support allows configuring external providers independently from CI/CD, and offers predefined attributes for a few well known ones (more can be easily added by editing the `identity-providers-wlif.tf` file). Once providers have been configured their names are passed to the following stages via interface outputs, and can be leveraged to set up access or impersonation in IAM bindings. + +CI/CD support is fully implemented for GitHub, Gitlab, and Cloud Source Repositories / Cloud Build. For GitHub, we also offer a [separate supporting setup](../../extras/0-cicd-github/) to quickly create / configure repositories. The same applies for Gitlab with the [following extra stage](../../extras/0-cicd-gitlab/). + + + +For details on how to configure both features, refer to the Customizations sections below on [Workload Identity Federation](#workload-identity-federation) and [CI/CD repositories](#cicd-repositories). + +These features are optional and only enabled if the relevant variables have been populated. + +## How to run this stage + +This stage has straightforward initial requirements, as it is designed to work on newly created GCP organizations. Four steps are needed to bring up this stage: + +- an Organization Admin self-assigns the required roles listed below +- the same administrator runs the first `init/apply` sequence passing a special variable to `apply` +- the providers configuration file is derived from the Terraform output or linked from the generated file +- a second `init` is run to migrate state, and from then on, the stage is run via impersonation + +### Prerequisites + +The roles that the Organization Admin used in the first `apply` needs to self-grant are: + +- Billing Account Administrator (`roles/billing.admin`) + either on the organization or the billing account (see the following section for details) +- Logging Admin (`roles/logging.admin`) +- Organization Role Administrator (`roles/iam.organizationRoleAdmin`) +- Organization Administrator (`roles/resourcemanager.organizationAdmin`) +- Project Creator (`roles/resourcemanager.projectCreator`) +- Tag Admin (`roles/resourcemanager.tagAdmin`) +- Owner (`roles/owner`) + +To quickly self-grant the above roles, run the following code snippet as the initial Organization Admin: + +```bash +# set variable for current logged in user +export FAST_BU=$(gcloud config list --format 'value(core.account)') + +# find and set your org id +gcloud organizations list +export FAST_ORG_ID=123456 + +# set needed roles +export FAST_ROLES="roles/billing.admin roles/logging.admin \ + roles/iam.organizationRoleAdmin roles/resourcemanager.projectCreator \ + roles/resourcemanager.organizationAdmin roles/resourcemanager.tagAdmin \ + roles/owner" + +for role in $FAST_ROLES; do + gcloud organizations add-iam-policy-binding $FAST_ORG_ID \ + --member user:$FAST_BU --role $role --condition None +done +``` + +Then make sure the same user is also part of the `gcp-organization-admins` group so that impersonating the automation service account later on will be possible. + +#### Standalone billing account + +If you are using a standalone billing account, the identity applying this stage for the first time needs to be a billing account administrator: + +```bash +export FAST_BILLING_ACCOUNT_ID=ABCD-01234-ABCD +gcloud beta billing accounts add-iam-policy-binding $FAST_BILLING_ACCOUNT_ID \ + --member user:$FAST_BU --role roles/billing.admin +``` + +#### Preventing creation of billing-related IAM bindings + +This configuration is possible but unsupported and only present for development purposes, use at your own risk: + +- configure `billing_account.id` as `null` and `billing_account.no_iam` to `true` in your `tfvars` file +- apply with `terraform apply -target 'module.automation-project.google_project.project[0]'` in addition to the initial user variable +- once Terraform raises an error run `terraform untaint 'module.automation-project.google_project.project[0]'` +- repeat the two steps above for `'module.log-export-project.google_project.project[0]'` +- go through the process to associate the billing account with the two projects +- configure `billing_account.id` with the real billing account id +- resume applying normally + +#### Groups + +Before the first run, the following IAM groups must exist to allow IAM bindings to be created (actual names are flexible, see the [Customization](#customizations) section): + +- `gcp-billing-admins` +- `gcp-devops` +- `gcp-vpc-network-admins` +- `gcp-organization-admins` +- `gcp-security-admins` + +You can refer to [this animated image](./groups.gif) for a step by step on group creation via the [Google Cloud Enterprise Checklist](https://cloud.google.com/docs/enterprise/setup-checklist). + +Please note that not all groups defined by the Checklist are actually used by FAST, as our approach to IAM is slightly different. As an example, we do not centralize monitoring functions as in our experience those are typically domain-specific (e.g. networking or application-level), so we don't leverage the corresponding groups. You are free of course to create those groups via the Checklist, and assign them roles via the IAM variables exposed by this stage. + +One more difference compared to the Checklist is the use in FAST of an additional group to centralize support functions like viewing tickets and accessing logging and monitoring data. To remain consistent with the [Google Cloud Enterprise Checklist](https://cloud.google.com/docs/enterprise/setup-checklist) we map these permissions to the `gcp-devops` group by default. However, we recommend creating a dedicated `gcp-support` group and updating the `groups` variable with the right value. + +#### Configure variables + +Then make sure you have configured the correct values for the following variables by providing a `terraform.tfvars` file: + +- `billing_account` + an object containing `id` as the id of your billing account, derived from the Cloud Console UI or by running `gcloud beta billing accounts list`, and the `is_org_level` flag that controls whether organization or account-level bindings are used, and a billing export project and dataset are created +- `groups` + the name mappings for your groups, if you're following the default convention you can leave this to the provided default +- `organization.id`, `organization.domain`, `organization.customer_id` + the id, domain and customer id of your organization, derived from the Cloud Console UI or by running `gcloud organizations list` +- `prefix` + the fixed org-level prefix used in your naming, maximum 9 characters long. Note that if you are using multitenant stages, then you will later need to configure a `tenant prefix`. + This `tenant prefix` can have a maximum length of 2 characters, + plus any unused characters from the from the `prefix`. + For example, if you specify a `prefix` that is 7 characters long, + then your `tenant prefix` can have a maximum of 4 characters. + +You can also adapt the example that follows to your needs: + +```tfvars +# use `gcloud beta billing accounts list` +# if you have too many accounts, check the Cloud Console :) +billing_account = { + id = "012345-67890A-BCDEF0" +} + +# use `gcloud organizations list` +organization = { + domain = "example.org" + id = 1234567890 + customer_id = "C000001" +} + +# local path to store tfvars/provider outputs generated by this stage +outputs_location = "~/fast-config" + +# locations for GCS, BigQuery, and logging buckets created here +locations = { + bq = "EU" + gcs = "EU" + logging = "global" + pubsub = [] +} + +# use something unique and no longer than 9 characters +prefix = "abcd" +``` + +### Output files and cross-stage variables + +Each foundational FAST stage generates provider configurations and variable files can be consumed by the following stages, and saves them in a dedicated GCS bucket in the automation project. These files are a handy way to simplify stage configuration, and are also used by our CI/CD workflows to configure the repository files in the pipelines that validate and apply the code. + +Alongside the GCS stored files, you can also configure a second copy to be saves on the local filesystem, as a convenience when developing or bringing up the infrastructure before a proper CI/CD setup is in place. + +This second set of files is disabled by default, you can enable it by setting the `outputs_location` variable to a valid path on a local filesystem, e.g. + +```tfvars +outputs_location = "~/fast-config" +``` + +Once the variable is set, `apply` will generate and manage providers and variables files, including the initial one used for this stage after the first run. You can then link these files in the relevant stages, instead of manually transferring outputs from one stage, to Terraform variables in another. + +Below is the outline of the output files generated by all stages, which is identical for both the GCS and local filesystem copies: + +```bash +[path specified in outputs_location] +├── providers +│   ├── 0-bootstrap-providers.tf +│   ├── 1-resman-providers.tf +│   ├── 2-networking-providers.tf +│   ├── 2-security-providers.tf +│   ├── 2-project-factory-dev-providers.tf +│   ├── 2-project-factory-prod-providers.tf +│   └── 9-sandbox-providers.tf +└── tfvars +│ ├── 0-bootstrap.auto.tfvars.json +│ ├── 1-resman.auto.tfvars.json +│ ├── 2-networking.auto.tfvars.json +│ └── 2-security.auto.tfvars.json +└── workflows + └── [optional depending on the configured CI/CD repositories] +``` + +### Running the stage + +Before running `init` and `apply`, check your environment so no extra variables that might influence authentication are present (e.g. `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT`). In general you should use user application credentials, and FAST will then take care to provision automation identities and configure impersonation for you. + +When running the first `apply` as a user, you need to pass a special runtime variable so that the user roles are preserved when setting IAM bindings. + +```bash +terraform init +terraform apply \ + -var bootstrap_user=$(gcloud config list --format 'value(core.account)') +``` + +> If you see an error related to project name already exists, please make sure the project name is unique or the project was not deleted recently + +Once the initial `apply` completes successfully, configure a remote backend using the new GCS bucket, and impersonation on the automation service account for this stage. To do this you can use the generated `providers.tf` file from either + +- the local filesystem if you have configured output files as described above +- the GCS bucket where output files are always stored +- Terraform outputs (not recommended as it's more complex) + +The following two snippets show how to leverage the `fast-links.sh` script in the FAST stages folder to fetch the commands required for output files linking or copying, using either the local output folder configured via Terraform variables, or the GCS bucket which can be derived from the `automation` output. + +```bash +../fast-links.sh ~/fast-config + +# File linking commands for organization bootstrap stage + +# provider file +ln -s ~/fast-config/fast-test-00/providers/0-bootstrap-providers.tf ./ + +# conventional place for stage tfvars (manually created) +ln -s ~/fast-config/fast-test-00/0-bootstrap.auto.tfvars ./ +``` + +```bash +../fast-links.sh gs://xxx-prod-iac-core-outputs-0 + +# File linking commands for organization bootstrap stage + +# provider file +gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/0-bootstrap-providers.tf ./ + +# conventional place for stage tfvars (manually created) +gcloud storage cp gs://xxx-prod-iac-core-outputs-0/0-bootstrap.auto.tfvars ./ +``` + +- important for CI/CD +The `0-bootstrap.auto.tfvars` file is a crucial component of the CI/CD pipeline and must be manually created. This file is essentially the `terraform.tfvars` file renamed to avoid being ignored in version control systems like GitHub or GitLab, where `terraform.tfvars` is often included in `.gitignore`. By renaming it and committing `0-bootstrap.auto.tfvars` to your source control, you ensure that the necessary configurations are available in the pipeline. + +Copy/paste the command returned by the script to link or copy the provider file, then migrate state with `terraform init` and run `terraform apply`. If your organization was created with "Secure by Default Org Policy", that is with some of the org policies enabled, add `-var 'org_policies_config={"import_defaults": true}'` to `terraform apply`: + +```bash +terraform init -migrate-state +terraform apply +``` + +or + +```bash +terraform init -migrate-state +terraform apply -var 'org_policies_config={"import_defaults": true}' +``` + +if there default policies are enabled. + +Make sure the user you're logged in with is a member of the `gcp-organization-admins` group or impersonation will not be possible. + +## Customizations + +Most variables (e.g. `billing_account` and `organization`) are only used to input actual values and should be self-explanatory. The only meaningful customizations that apply here are groups, and IAM roles. + +### Group names + +As we mentioned above, groups reflect the convention used in the [GCP Enterprise Setup Checklist](https://cloud.google.com/docs/enterprise/setup-checklist), with an added level of indirection: the `groups` variable maps logical names to actual names, so that you don't need to delve into the code if your group names do not comply with the checklist convention. + +For example, if your network admins team is called `net-rockstars@example.com`, simply set that name in the variable, minus the domain which is interpolated internally with the organization domain: + +```hcl +variable "groups" { + description = "Group names to grant organization-level permissions." + type = map(string) + default = { + gcp-network-admins = "net-rockstars" + # [...] + } +} +# tftest skip +``` + +If your groups layout differs substantially from the checklist, define all relevant groups in the `groups` variable, then rearrange IAM roles in the code to match your setup. + +### IAM + +One other area where we directly support customizations is IAM. The code here, as in all stages, follows a simple pattern derived from best practices: + +- operational roles for humans are assigned to groups +- any other principal is a service account + +In code, the distinction above reflects on how IAM bindings are specified in the underlying module variables: + +- group roles "for humans" always use `iam_by_principals` variables +- service account roles always use `iam` variables + +This makes it easy to tweak user roles by adding mappings to the `iam_by_principals` variables of the relevant resources, without having to understand and deal with the details of service account roles. + +One more critical difference in IAM bindings is between authoritative and additive: + +- authoritative bindings have complete control on principals for a given role; this is the recommended best practice when a single automation actor controls the role, as it removes drift each time Terraform runs +- additive bindings have control only on given role/principal pairs, and need to be used whenever multiple automation actors need to control the role, as is the case for the network user role in Shared VPC setups, and many other situations + +This stage groups all IAM definitions in the [organization-iam.tf](./organization-iam.tf) file, to allow easy parsing of roles assigned to each group and machine identity. + +When customizations are needed, three stage-level variables allow injecting additional bindings to match the desired setup: + +- `iam_by_principals` allows adding authoritative bindings for groups +- `iam` allows adding authoritative bindings for any type of supported principal, and is merged with the internal `iam` local and then with group bindings at the module level +- `iam_bindings_additive` allows adding individual role/member pairs, and also supports IAM conditions + +Refer to the [project module](../../../modules/project/) for examples on how to use the IAM variables, and they are an interface shared across all our modules. + +### Log sinks and log destinations + +You can customize organization-level logs through the `log_sinks` variable in two ways: + +- creating additional log sinks to capture more logs +- changing the destination of captured logs + +By default, all logs are exported to a log bucket, but FAST can create sinks to BigQuery, GCS, or PubSub. + +If you need to capture additional logs, please refer to GCP's documentation on [scenarios for exporting logging data](https://cloud.google.com/architecture/exporting-stackdriver-logging-for-security-and-access-analytics), where you can find ready-made filter expressions for different use cases. + +When using Pubsub or BigQuery destinations, make sure the read-only stage service account (`prefix-prod-bootstrap-0r@prefix-prod-iac-core-0.iam.gserviceaccount.com`) has the necessary permissions to view destination resources. You can add them manually via the authoritative `iam` or the additive `iam_bindings_additive` variables. Refer to issue [#2540](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/issues/2540) for a discussion on this topic, and simple commands to verify proper permissions have been added. + +### Names and naming convention + +Configuring the individual tokens for the naming convention described above, has varying degrees of complexity: + +- the static prefix can be set via the `prefix` variable once +- the environment identifier is set to `prod` as resources here influence production and are considered as such, and can be changed in `main.tf` locals + +All other tokens are set directly in resource names, as providing abstractions to manage them would have added too much complexity to the code, making it less readable and more fragile. + +If a different convention is needed, identify names via search/grep (e.g. with `^\s+name\s+=\s+"`) and change them in an editor: it should take a couple of minutes at most, as there's just a handful of modules and resources to change. + +Names used in internal references (e.g. `module.foo-prod.id`) are only used by Terraform and do not influence resource naming, so they are best left untouched to avoid having to debug complex errors. + +### Workload Identity Federation + +At any time during this stage's lifecycle you can configure a Workload Identity Federation pool, and one or more providers. These are part of this stage's interface, included in the automatically generated `.tfvars` files and accepted by the Resource Managent stage that follows. + +The variable maps each provider's `issuer` attribute with the definitions in the `identity-providers-wlif.tf` file. We currently support GitHub and Gitlab directly, and extending to definitions to support more providers is trivial (send us a PR if you do!). + +Provider key names are used by the `cicd_repositories` variable to configure authentication for CI/CD repositories, and generally from your Terraform code whenever you need to configure IAM access or impersonation for federated identities. + +This is a sample configuration of a GitHub and a Gitlab provider. Every parameter is optional. + +The `custom_settings` attributes are used to configure the provider to work with privately managed installations of Github and Gitlab: + +- `issuer_uri` (defaults to the public platforms one if not set) +- `audience` (defaults to the public URL of the provider if not set, as recommended in the [WIF FAQ section](https://cloud.google.com/iam/docs/best-practices-for-using-workload-identity-federation#provider-audience)) +- `jwks_json` for public key upload + +```tfvars +workload_identity_providers = { + # Use the public GitHub and specify an attribute condition + github-public-sample = { + attribute_condition = "attribute.repository_owner==\"my-github-org\"" + issuer = "github" + } + # Use a private instance of Gitlab and specify a custom issuer_uri + gitlab-private-sample = { + issuer = "gitlab" + custom_settings = { + issuer_uri = "https://gitlab.fast.example.com" + } + } + # Use a private instance of Gitlab. + # Specify a custom audience and a custom issuer_uri + gitlab-private-aud-sample = { + attribute_condition = "attribute.namespace_path==\"my-gitlab-org\"" + issuer = "gitlab" + custom_settings = { + audiences = ["https://gitlab.fast.example.com"] + issuer_uri = "https://gitlab.fast.example.com" + } + } +} +``` + +### Project folders + +By default this stage creates all its projects directly under the orgaization node. If desired, projects can be moved under a folder using the `project_parent_ids` variable. + +```tfvars +project_parent_ids = { + automation = "folders/1234567890" + billing = "folders/9876543210" + logging = "folders/1234567890" +} +``` + +### CI/CD repositories + +FAST is designed to directly support running in automated workflows from separate repositories for each stage. The `cicd_repositories` variable allows you to configure impersonation from external repositories leveraging Workload identity Federation, and pre-configures a FAST workflow file that can be used to validate and apply the code in each repository. + +The repository design we support is fairly simple, with a repository for modules that enables centralization and versioning, and one repository for each stage optionally configured from the previous stage. + +This is an example of configuring the bootstrap and resource management repositories in this stage. CI/CD configuration is optional, so the entire variable or any of its attributes can be set to null if not needed. + +```tfvars +cicd_config = { + bootstrap = { + identity_provider = "github-sample" + repository = { + branch = null + name = "my-gh-org/fast-bootstrap" + type = "github" + } + } + resman = { + identity_provider = "github-sample" + repository = { + branch = "main" + name = "my-gh-org/fast-resman" + type = "github" + } + } +} +``` + +The `type` attribute can be set to one of the supported repository types: `github` or `gitlab`. + +Once the stage is applied the generated output files will contain pre-configured workflow files for each repository, that will use Workload Identity Federation via a dedicated service account for each repository to impersonate the automation service account for the stage. + +You can use Terraform to automate creation of the repositories using the extra stage defined in [fast/extras/0-cicd-github](../../extras/0-cicd-github/) (only for Github for now). + +The remaining configuration is manual, as it regards the repositories themselves: + +- create a repository for modules + - clone and populate it with the Fabric modules + - configure authentication to the modules repository + - for GitHub + - create a key pair + - create a [deploy key](https://docs.github.com/en/developers/overview/managing-deploy-keys#deploy-keys) in the modules repository with the public key + - create a `CICD_MODULES_KEY` secret with the private key in each of the repositories that need to access modules (for Gitlab, please Base64 encode the private key for masking) + - for Gitlab + - TODO + - for Source Repositories + - assign the reader role to the CI/CD service accounts +- create one repository for each stage + - do an initial apply cycle for the stage so that state exists + - clone and populate them with the stage source + - edit the modules source to match your modules repository + - a simple way is using the "Replace in files" function of your editor + - search for `source\s*= "../../../modules/([^"]+)"` + - replace with: + - modules stored on GitHub: `source = "git@github.com:my-org/fast-modules.git//$1?ref=v1.0"` + - modules stored on Gitlab: `source = "git::ssh://git@gitlab.com/my-org/fast-modules.git//$1?ref=v1.0"` + - modules stored on Source Repositories: `"source = git::https://source.developers.google.com/p/my-project/r/my-repository//$1?ref=v1.0"`. You may need to run `git config --global credential.'https://source.developers.google.com'.helper gcloud.sh` first as documented [here](https://cloud.google.com/source-repositories/docs/adding-repositories-as-remotes#add_the_repository_as_a_remote) + - copy the generated workflow file for the stage from the GCS output files bucket or from the local clone if enabled + - for GitHub, place it in a `.github/workflows` folder in the repository root + - for Gitlab, rename it to `.gitlab-ci.yml` and place it in the repository root + - for Source Repositories, place it in `.cloudbuild/workflow.yaml` + - To prevent the creation of local files in the CI/CD pipeline, comment out the `outputs_location` line in the `terraform.tfvars` file by adding a `#` at the beginning, like so: `# outputs_location = "~/fast-config"`. This configuration is only necessary for the initial local deployments and should not be used in the CI/CD environment. + +### Add-ons + +FAST defines a simple mechanism to extend stage functionality via the use of [add-ons](../../addons/). Configuration for stage 1 add-ons happens here via the `fast_addon` variable. Refer to the add-ons documentation for more details on their use. + + + +## Files + +| name | description | modules | resources | +|---|---|---|---| +| [automation.tf](./automation.tf) | Automation project and resources. | gcs · iam-service-account · project | | +| [billing.tf](./billing.tf) | Billing export project and dataset. | bigquery-dataset · billing-account · logging-bucket · project | | +| [cicd.tf](./cicd.tf) | CI/CD locals and resources. | iam-service-account | | +| [identity-providers-wfif-defs.tf](./identity-providers-wfif-defs.tf) | Workforce Identity provider definitions. | | | +| [identity-providers-wfif.tf](./identity-providers-wfif.tf) | Workforce Identity Federation provider definitions. | | google_iam_workforce_pool · google_iam_workforce_pool_provider | +| [identity-providers-wlif-defs.tf](./identity-providers-wlif-defs.tf) | Workload Identity provider definitions. | | | +| [identity-providers-wlif.tf](./identity-providers-wlif.tf) | Workload Identity Federation provider definitions. | | google_iam_workload_identity_pool · google_iam_workload_identity_pool_provider | +| [log-export.tf](./log-export.tf) | Audit log project and sink. | bigquery-dataset · gcs · logging-bucket · project · pubsub | | +| [main.tf](./main.tf) | Module-level locals and resources. | | | +| [organization-iam.tf](./organization-iam.tf) | Organization-level IAM bindings locals. | | | +| [organization.tf](./organization.tf) | Organization-level IAM. | organization | | +| [outputs-files.tf](./outputs-files.tf) | Output files persistence to local filesystem. | | local_file | +| [outputs-gcs.tf](./outputs-gcs.tf) | Output files persistence to automation GCS bucket. | | google_storage_bucket_object | +| [outputs-providers.tf](./outputs-providers.tf) | Locals for provider output files. | | | +| [outputs.tf](./outputs.tf) | Module outputs. | | | +| [variables-addons.tf](./variables-addons.tf) | None | | | +| [variables.tf](./variables.tf) | Module variables. | | | + +## Variables + +| name | description | type | required | default | producer | +|---|---|:---:|:---:|:---:|:---:| +| [billing_account](variables.tf#L17) | Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`. | object({…}) | ✓ | | | +| [organization](variables.tf#L282) | Organization details. | object({…}) | ✓ | | | +| [prefix](variables.tf#L297) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | | +| [bootstrap_user](variables.tf#L39) | Email of the nominal user running this stage for the first time. | string | | null | | +| [cicd_config](variables.tf#L45) | CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | object({…}) | | {} | | +| [custom_roles](variables.tf#L86) | Map of role names => list of permissions to additionally create at the organization level. | map(list(string)) | | {} | | +| [environments](variables.tf#L93) | Environment names. When not defined, short name is set to the key and tag name to lower(name). | map(object({…})) | | {…} | | +| [essential_contacts](variables.tf#L133) | Email used for essential contacts, unset if null. | string | | null | | +| [factories_config](variables.tf#L139) | Configuration for the resource factories or external data. | object({…}) | | {} | | +| [fast_addon](variables-addons.tf#L17) | FAST addons configurations for stages 1. Keys are used as short names for the add-on resources. | map(object({…})) | | {} | | +| [groups](variables.tf#L151) | Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated. | object({…}) | | {} | | +| [iam](variables.tf#L168) | Organization-level custom IAM settings in role => [principal] format. | map(list(string)) | | {} | | +| [iam_bindings_additive](variables.tf#L175) | Organization-level custom additive IAM bindings. Keys are arbitrary. | map(object({…})) | | {} | | +| [iam_by_principals](variables.tf#L190) | Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the `iam` variable. | map(list(string)) | | {} | | +| [locations](variables.tf#L197) | Optional locations for GCS, BigQuery, and logging buckets created here. | object({…}) | | {} | | +| [log_sinks](variables.tf#L211) | Org-level log sinks, in name => {type, filter} format. | map(object({…})) | | {…} | | +| [org_policies_config](variables.tf#L267) | Organization policies customization. | object({…}) | | {} | | +| [outputs_location](variables.tf#L291) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | string | | null | | +| [project_parent_ids](variables.tf#L306) | Optional parents for projects created here in folders/nnnnnnn format. Null values will use the organization as parent. | object({…}) | | {} | | +| [resource_names](variables.tf#L317) | Resource names overrides for specific resources. Prefix is always set via code, except where noted in the variable type. | object({…}) | | {} | | +| [universe](variables.tf#L349) | Target GCP universe. | object({…}) | | null | | +| [workforce_identity_providers](variables.tf#L359) | Workforce Identity Federation pools. | map(object({…})) | | {} | | +| [workload_identity_providers](variables.tf#L375) | Workload Identity Federation pools. The `cicd_repositories` variable references keys here. | map(object({…})) | | {} | | + +## Outputs + +| name | description | sensitive | consumers | +|---|---|:---:|---| +| [automation](outputs.tf#L113) | Automation resources. | | | +| [billing_dataset](outputs.tf#L118) | BigQuery dataset prepared for billing export. | | | +| [cicd_repositories](outputs.tf#L123) | CI/CD repository configurations. | | | +| [custom_roles](outputs.tf#L135) | Organization-level custom roles. | | | +| [outputs_bucket](outputs.tf#L140) | GCS bucket where generated output files are stored. | | | +| [project_ids](outputs.tf#L145) | Projects created by this stage. | | | +| [providers](outputs.tf#L155) | Terraform provider files for this stage and dependent stages. | ✓ | stage-01 | +| [service_accounts](outputs.tf#L162) | Automation service accounts created by this stage. | | | +| [tfvars](outputs.tf#L171) | Terraform variable files for the following stages. | ✓ | | +| [tfvars_globals](outputs.tf#L177) | Terraform Globals variable files for the following stages. | ✓ | | +| [workforce_identity_pool](outputs.tf#L183) | Workforce Identity Federation pool. | | | +| [workload_identity_pool](outputs.tf#L192) | Workload Identity Federation pool and providers. | | | + diff --git a/fast/stages/0-bootstrap/automation.tf b/fast/stages/0-bootstrap-legacy/automation.tf similarity index 100% rename from fast/stages/0-bootstrap/automation.tf rename to fast/stages/0-bootstrap-legacy/automation.tf diff --git a/fast/stages/0-bootstrap-legacy/billing.tf b/fast/stages/0-bootstrap-legacy/billing.tf new file mode 100644 index 000000000..941e5c741 --- /dev/null +++ b/fast/stages/0-bootstrap-legacy/billing.tf @@ -0,0 +1,127 @@ +/** + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Billing export project and dataset. + +locals { + billing_mode = ( + var.billing_account.no_iam + ? null + : var.billing_account.is_org_level ? "org" : "resource" + ) + + _billing_iam_bindings = { + "roles/billing.admin" = [ + local.principals.gcp-billing-admins, + local.principals.gcp-organization-admins, + module.automation-tf-bootstrap-sa.iam_email, + module.automation-tf-resman-sa.iam_email + ], + "roles/billing.viewer" = [ + module.automation-tf-bootstrap-r-sa.iam_email, + module.automation-tf-resman-r-sa.iam_email + ], + "roles/logging.configWriter" = local.billing_mode == "org" || !var.billing_account.force_create.log_bucket ? [] : [ + module.automation-tf-bootstrap-sa.iam_email + ] + } + + _billing_iam_bindings_add = flatten([for role, bindings in local._billing_iam_bindings : [ + for member in bindings : { + member = member, + role = role + } + ]]) + + billing_iam_bindings_additive = { + for b in local._billing_iam_bindings_add : "${b.role}-${b.member}" => { + member = b.member + role = b.role + } + } +} + +# billing account in same org (IAM is in the organization.tf file) + +module "billing-export-project" { + source = "../../../modules/project" + count = ( + local.billing_mode == "org" || var.billing_account.force_create.project == true ? 1 : 0 + ) + billing_account = var.billing_account.id + name = var.resource_names["project-billing"] + parent = coalesce( + var.project_parent_ids.billing, "organizations/${var.organization.id}" + ) + prefix = var.prefix + universe = var.universe + contacts = ( + var.bootstrap_user != null || var.essential_contacts == null + ? {} + : { (var.essential_contacts) = ["ALL"] } + ) + iam = { + "roles/owner" = [module.automation-tf-bootstrap-sa.iam_email] + "roles/viewer" = [module.automation-tf-bootstrap-r-sa.iam_email] + } + services = [ + # "cloudresourcemanager.googleapis.com", + # "iam.googleapis.com", + # "serviceusage.googleapis.com", + "bigquery.googleapis.com", + "bigquerydatatransfer.googleapis.com", + "storage.googleapis.com" + ] +} + +module "billing-export-dataset" { + source = "../../../modules/bigquery-dataset" + count = ( + local.billing_mode == "org" || var.billing_account.force_create.dataset == true ? 1 : 0 + ) + project_id = module.billing-export-project[0].project_id + id = var.resource_names["bq-billing"] + friendly_name = "Billing export." + location = local.locations.bq +} + +# standalone billing account + +module "billing-account-logbucket" { + source = "../../../modules/logging-bucket" + count = local.billing_mode == "resource" && var.billing_account.force_create.log_bucket ? 1 : 0 + parent_type = "project" + parent = module.log-export-project.project_id + name = "billing-account" + location = local.locations.logging + log_analytics = { enable = true } + # org-level logging settings ready before we create any logging buckets + depends_on = [module.organization-logging] +} + +module "billing-account" { + source = "../../../modules/billing-account" + count = local.billing_mode == "resource" ? 1 : 0 + id = var.billing_account.id + iam_bindings_additive = local.billing_iam_bindings_additive + logging_sinks = !var.billing_account.force_create.log_bucket ? {} : { + billing_bucket_log_sink = { + destination = module.billing-account-logbucket[0].id + type = "logging" + description = "billing-account sink (Terraform-managed)." + } + } +} diff --git a/fast/stages/0-bootstrap-legacy/cicd.tf b/fast/stages/0-bootstrap-legacy/cicd.tf new file mode 100644 index 000000000..1c2771c45 --- /dev/null +++ b/fast/stages/0-bootstrap-legacy/cicd.tf @@ -0,0 +1,127 @@ +/** + * Copyright 2024 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description CI/CD locals and resources. + +locals { + _cicd_configs = merge( + # stages + { + for k, v in var.cicd_config : k => merge(v, { + level = k == "bootstrap" ? 0 : 1 + stage = k + }) if v != null + }, + # addons + { + for k, v in var.fast_addon : k => merge(v.cicd_config, { + level = 1 + stage = substr(v.parent_stage, 2, -1) + }) if v.cicd_config != null + } + ) + cicd_providers = { + for k, v in google_iam_workload_identity_pool_provider.default : + k => { + audiences = concat( + v.oidc[0].allowed_audiences, + ["https://iam.googleapis.com/${v.name}"] + ) + issuer = local.workload_identity_providers[k].issuer + issuer_uri = try(v.oidc[0].issuer_uri, null) + name = v.name + principal_branch = local.workload_identity_providers[k].principal_branch + principal_repo = local.workload_identity_providers[k].principal_repo + } + } + cicd_repositories = { + for k, v in local._cicd_configs : k => v if( + contains(keys(local.workload_identity_providers), v.identity_provider) && + fileexists("${path.module}/templates/workflow-${v.repository.type}.yaml") + ) + } + cicd_workflow_providers = merge( + { + for k, v in local.cicd_repositories : + k => "${v.level}-${k}-providers.tf" + }, + { + for k, v in local.cicd_repositories : + "${k}-r" => "${v.level}-${k}-r-providers.tf" + } + ) +} + +# SAs used by CI/CD workflows to impersonate automation SAs + +module "automation-tf-cicd-sa" { + source = "../../../modules/iam-service-account" + for_each = local.cicd_repositories + project_id = module.automation-project.project_id + name = templatestring( + var.resource_names["sa-cicd_template"], { key = each.key } + ) + display_name = "Terraform CI/CD ${each.key} service account." + prefix = var.prefix + iam = { + "roles/iam.workloadIdentityUser" = [ + each.value.repository.branch == null + ? format( + local.workload_identity_providers_defs[each.value.repository.type].principal_repo, + google_iam_workload_identity_pool.default[0].name, + each.value.repository.name + ) + : format( + local.workload_identity_providers_defs[each.value.repository.type].principal_branch, + google_iam_workload_identity_pool.default[0].name, + each.value.repository.name, + each.value.repository.branch + ) + ] + } + iam_project_roles = { + (module.automation-project.project_id) = ["roles/logging.logWriter"] + } + iam_storage_roles = { + (module.automation-tf-output-gcs.name) = ["roles/storage.objectViewer"] + } +} + +module "automation-tf-cicd-r-sa" { + source = "../../../modules/iam-service-account" + for_each = local.cicd_repositories + project_id = module.automation-project.project_id + name = templatestring( + var.resource_names["sa-cicd_template_ro"], { key = each.key } + ) + display_name = "Terraform CI/CD ${each.key} service account (read-only)." + prefix = var.prefix + iam = { + "roles/iam.workloadIdentityUser" = [ + format( + local.workload_identity_providers_defs[each.value.repository.type].principal_repo, + google_iam_workload_identity_pool.default[0].name, + each.value.repository.name + ) + ] + } + iam_project_roles = { + (module.automation-project.project_id) = ["roles/logging.logWriter"] + } + iam_storage_roles = { + (module.automation-tf-output-gcs.name) = ["roles/storage.objectViewer"] + } +} diff --git a/fast/stages/0-bootstrap/data/custom-constraints/accesscontextmanager.yaml b/fast/stages/0-bootstrap-legacy/data/custom-constraints/accesscontextmanager.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-constraints/accesscontextmanager.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-constraints/accesscontextmanager.yaml diff --git a/fast/stages/0-bootstrap/data/custom-constraints/gke.yaml b/fast/stages/0-bootstrap-legacy/data/custom-constraints/gke.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-constraints/gke.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-constraints/gke.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/billing_viewer.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/billing_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/billing_viewer.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/billing_viewer.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/dns_zone_binder.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/dns_zone_binder.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/dns_zone_binder.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/dns_zone_binder.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/gcve_network_admin.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/gcve_network_admin.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/gcve_network_admin.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/gcve_network_admin.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/gcve_network_viewer.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/gcve_network_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/gcve_network_viewer.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/gcve_network_viewer.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/kms_key_encryption_admin.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/kms_key_encryption_admin.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/kms_key_encryption_admin.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/kms_key_encryption_admin.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/kms_key_viewer.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/kms_key_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/kms_key_viewer.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/kms_key_viewer.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/network_firewall_policies_admin.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/network_firewall_policies_admin.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/network_firewall_policies_admin.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/network_firewall_policies_admin.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/ngfw_enterprise_admin.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/ngfw_enterprise_admin.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/ngfw_enterprise_admin.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/ngfw_enterprise_admin.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/ngfw_enterprise_viewer.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/ngfw_enterprise_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/ngfw_enterprise_viewer.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/ngfw_enterprise_viewer.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/organization_admin_viewer.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/organization_admin_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/organization_admin_viewer.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/organization_admin_viewer.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/organization_iam_admin.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/organization_iam_admin.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/organization_iam_admin.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/organization_iam_admin.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/project_iam_viewer.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/project_iam_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/project_iam_viewer.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/project_iam_viewer.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/service_project_network_admin.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/service_project_network_admin.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/service_project_network_admin.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/service_project_network_admin.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/storage_viewer.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/storage_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/storage_viewer.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/storage_viewer.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/tag_viewer.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/tag_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/tag_viewer.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/tag_viewer.yaml diff --git a/fast/stages/0-bootstrap/data/custom-roles/tenant_network_admin.yaml b/fast/stages/0-bootstrap-legacy/data/custom-roles/tenant_network_admin.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/custom-roles/tenant_network_admin.yaml rename to fast/stages/0-bootstrap-legacy/data/custom-roles/tenant_network_admin.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-iac/compute.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-iac/compute.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-iac/compute.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-iac/compute.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-iac/iam.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-iac/iam.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-iac/iam.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-iac/iam.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-managed/accesscontextmanager.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-managed/accesscontextmanager.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-managed/accesscontextmanager.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-managed/accesscontextmanager.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-managed/cloudbuild.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-managed/cloudbuild.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-managed/cloudbuild.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-managed/cloudbuild.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-managed/compute.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-managed/compute.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-managed/compute.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-managed/compute.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-managed/essentialcontacts.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-managed/essentialcontacts.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-managed/essentialcontacts.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-managed/essentialcontacts.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-managed/gcp.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-managed/gcp.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-managed/gcp.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-managed/gcp.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-managed/gke.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-managed/gke.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-managed/gke.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-managed/gke.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-managed/iam.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-managed/iam.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-managed/iam.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-managed/iam.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-managed/serverless.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-managed/serverless.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-managed/serverless.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-managed/serverless.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-managed/sql.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-managed/sql.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-managed/sql.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-managed/sql.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies-managed/storage.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies-managed/storage.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies-managed/storage.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies-managed/storage.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies/accesscontextmanager.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies/accesscontextmanager.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies/accesscontextmanager.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies/accesscontextmanager.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies/cloudbuild.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies/cloudbuild.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies/cloudbuild.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies/cloudbuild.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies/compute.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies/compute.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies/compute.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies/compute.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies/essentialcontacts.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies/essentialcontacts.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies/essentialcontacts.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies/essentialcontacts.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies/gcp.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies/gcp.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies/gcp.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies/gcp.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies/gke.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies/gke.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies/gke.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies/gke.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies/iam.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies/iam.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies/iam.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies/iam.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies/serverless.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies/serverless.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies/serverless.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies/serverless.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies/sql.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies/sql.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies/sql.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies/sql.yaml diff --git a/fast/stages/0-bootstrap/data/org-policies/storage.yaml b/fast/stages/0-bootstrap-legacy/data/org-policies/storage.yaml similarity index 100% rename from fast/stages/0-bootstrap/data/org-policies/storage.yaml rename to fast/stages/0-bootstrap-legacy/data/org-policies/storage.yaml diff --git a/fast/stages/0-bootstrap/diagram.png b/fast/stages/0-bootstrap-legacy/diagram.png similarity index 100% rename from fast/stages/0-bootstrap/diagram.png rename to fast/stages/0-bootstrap-legacy/diagram.png diff --git a/fast/stages/0-bootstrap-experimental/fast_version.txt b/fast/stages/0-bootstrap-legacy/fast_version.txt similarity index 100% rename from fast/stages/0-bootstrap-experimental/fast_version.txt rename to fast/stages/0-bootstrap-legacy/fast_version.txt diff --git a/fast/stages/0-bootstrap/groups.gif b/fast/stages/0-bootstrap-legacy/groups.gif similarity index 100% rename from fast/stages/0-bootstrap/groups.gif rename to fast/stages/0-bootstrap-legacy/groups.gif diff --git a/fast/stages/0-bootstrap/identity-providers-wfif-defs.tf b/fast/stages/0-bootstrap-legacy/identity-providers-wfif-defs.tf similarity index 100% rename from fast/stages/0-bootstrap/identity-providers-wfif-defs.tf rename to fast/stages/0-bootstrap-legacy/identity-providers-wfif-defs.tf diff --git a/fast/stages/0-bootstrap/identity-providers-wfif.tf b/fast/stages/0-bootstrap-legacy/identity-providers-wfif.tf similarity index 100% rename from fast/stages/0-bootstrap/identity-providers-wfif.tf rename to fast/stages/0-bootstrap-legacy/identity-providers-wfif.tf diff --git a/fast/stages/0-bootstrap/identity-providers-wlif-defs.tf b/fast/stages/0-bootstrap-legacy/identity-providers-wlif-defs.tf similarity index 100% rename from fast/stages/0-bootstrap/identity-providers-wlif-defs.tf rename to fast/stages/0-bootstrap-legacy/identity-providers-wlif-defs.tf diff --git a/fast/stages/0-bootstrap/identity-providers-wlif.tf b/fast/stages/0-bootstrap-legacy/identity-providers-wlif.tf similarity index 100% rename from fast/stages/0-bootstrap/identity-providers-wlif.tf rename to fast/stages/0-bootstrap-legacy/identity-providers-wlif.tf diff --git a/fast/stages/0-bootstrap/log-export.tf b/fast/stages/0-bootstrap-legacy/log-export.tf similarity index 100% rename from fast/stages/0-bootstrap/log-export.tf rename to fast/stages/0-bootstrap-legacy/log-export.tf diff --git a/fast/stages/2-project-factory-experimental/outputs.tf b/fast/stages/0-bootstrap-legacy/main.tf similarity index 60% rename from fast/stages/2-project-factory-experimental/outputs.tf rename to fast/stages/0-bootstrap-legacy/main.tf index 4d7c73f06..c7ce07cf7 100644 --- a/fast/stages/2-project-factory-experimental/outputs.tf +++ b/fast/stages/0-bootstrap-legacy/main.tf @@ -1,5 +1,5 @@ /** - * Copyright 2022 Google LLC + * Copyright 2024 Google LLC * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -14,14 +14,18 @@ * limitations under the License. */ -output "projects" { - description = "Attributes for managed projects." - value = module.factory.projects -} - -resource "google_storage_bucket_object" "version" { - count = fileexists("fast_version.txt") ? 1 : 0 - bucket = var.automation.outputs_bucket - name = "versions/2-${var.stage_name}-version.txt" - source = "fast_version.txt" +locals { + principals = { + for k, v in var.groups : k => ( + can(regex("^[a-zA-Z]+:", v)) + ? v + : "group:${v}@${var.organization.domain}" + ) + } + locations = { + bq = var.locations.bq + gcs = var.locations.gcs + logging = var.locations.logging + pubsub = var.locations.pubsub + } } diff --git a/fast/stages/0-bootstrap/organization-iam.tf b/fast/stages/0-bootstrap-legacy/organization-iam.tf similarity index 100% rename from fast/stages/0-bootstrap/organization-iam.tf rename to fast/stages/0-bootstrap-legacy/organization-iam.tf diff --git a/fast/stages/0-bootstrap-legacy/organization.tf b/fast/stages/0-bootstrap-legacy/organization.tf new file mode 100644 index 000000000..a19ba76d9 --- /dev/null +++ b/fast/stages/0-bootstrap-legacy/organization.tf @@ -0,0 +1,237 @@ +/** + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Organization-level IAM. + +locals { + # reassemble logical bindings into the formats expected by the module + _iam_bindings = merge( + local.iam_domain_bindings, + local.iam_sa_bindings, + local.iam_user_bootstrap_bindings, + { + for k, v in local.iam_principal_bindings : k => { + authoritative = [] + additive = v.additive + } + } + ) + _iam_bindings_auth = flatten([ + for member, data in local._iam_bindings : [ + for role in data.authoritative : { + member = member + role = role + } + ] + ]) + _iam_bindings_add = flatten([ + for member, data in local._iam_bindings : [ + for role in data.additive : { + member = member + role = role + } + ] + ]) + org_policies_tag_name = "${var.organization.id}/${var.org_policies_config.tag_name}" + iam_principals = { + for k, v in local.iam_principal_bindings : k => v.authoritative + } + iam = merge( + { + for r in local.iam_delete_roles : r => [] + }, + { + for b in local._iam_bindings_auth : b.role => b.member... + } + ) + iam_bindings_additive = { + for b in local._iam_bindings_add : "${b.role}-${b.member}" => { + member = b.member + role = b.role + } + } +} + +# TODO: add a check block to ensure our custom roles exist in the factory files + +# import org policy constraints enabled by default in new orgs since February 2024 +import { + for_each = ( + !var.org_policies_config.import_defaults || var.bootstrap_user != null + ? toset([]) + : toset([ + # source: https://cloud.google.com/resource-manager/docs/secure-by-default-organizations#organization_policies_enforced_on_organization_resources + # listed in the order as on page + "iam.disableServiceAccountKeyCreation", + "iam.disableServiceAccountKeyUpload", + "iam.automaticIamGrantsForDefaultServiceAccounts", + "iam.allowedPolicyMemberDomains", + "essentialcontacts.allowedContactDomains", + "storage.uniformBucketLevelAccess", + "compute.setNewProjectDefaultToZonalDNSOnly", # Verified as of 2024-09-13 + "compute.restrictProtocolForwardingCreationForTypes", # Verified as of 2025-02-13 + ]) + ) + id = "organizations/${var.organization.id}/policies/${each.key}" + to = module.organization.google_org_policy_policy.default[each.key] +} + +module "organization-logging" { + # Preconfigure organization-wide logging settings to ensure project + # log buckets (_Default, _Required) are created in the location + # specified by `var.locations.logging`. This separate + # organization-block prevents circular dependencies with later + # project creation. + source = "../../../modules/organization" + organization_id = "organizations/${var.organization.id}" + logging_settings = { + storage_location = var.locations.logging + } +} + +module "organization" { + source = "../../../modules/organization" + organization_id = module.organization-logging.id + # human (groups) IAM bindings + iam_by_principals = { + for key in distinct(concat( + keys(local.iam_principals), + keys(var.iam_by_principals), + )) : + key => distinct(concat( + lookup(local.iam_principals, key, []), + lookup(var.iam_by_principals, key, []), + )) + } + # machine (service accounts) IAM bindings + iam = merge( + { + for k, v in local.iam : k => distinct(concat(v, lookup(var.iam, k, []))) + }, + { + for k, v in var.iam : k => v if lookup(local.iam, k, null) == null + } + ) + # additive bindings, used for roles co-managed by different stages + iam_bindings_additive = merge( + local.iam_bindings_additive, + var.iam_bindings_additive + ) + # delegated role grant for resource manager service account + iam_bindings = merge( + { + organization_iam_admin_conditional = { + members = [module.automation-tf-resman-sa.iam_email] + role = module.organization.custom_role_id["organization_iam_admin"] + condition = { + expression = ( + format( + <<-EOT + api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s]) + || api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s]) + || api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s]) + EOT + , join(",", formatlist("'%s'", [ + "roles/accesscontextmanager.policyEditor", + "roles/accesscontextmanager.policyReader", + "roles/cloudasset.viewer", + "roles/compute.orgFirewallPolicyAdmin", + "roles/compute.orgFirewallPolicyUser", + "roles/compute.xpnAdmin", + "roles/orgpolicy.policyAdmin", + "roles/orgpolicy.policyViewer", + "roles/resourcemanager.organizationViewer", + ])) + , join(",", formatlist("'%s'", [ + "roles/iam.workforcePoolAdmin", + "roles/iam.workforcePoolViewer" + ])) + , join(",", formatlist("'%s'", [ + module.organization.custom_role_id["billing_viewer"], + module.organization.custom_role_id["network_firewall_policies_admin"], + module.organization.custom_role_id["ngfw_enterprise_admin"], + module.organization.custom_role_id["ngfw_enterprise_viewer"], + module.organization.custom_role_id["service_project_network_admin"], + module.organization.custom_role_id["tenant_network_admin"] + ])) + ) + ) + title = "automation_sa_delegated_grants" + description = "Automation service account delegated grants." + } + } + }, + local.billing_mode != "org" ? {} : { + organization_billing_conditional = { + members = [module.automation-tf-resman-sa.iam_email] + role = module.organization.custom_role_id["organization_iam_admin"] + condition = { + expression = format( + "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])", + join(",", formatlist("'%s'", [ + "roles/billing.admin", + "roles/billing.costsManager", + "roles/billing.user", + ])) + ) + title = "automation_sa_delegated_grants" + description = "Automation service account delegated grants." + } + } + } + ) + custom_roles = var.custom_roles + context = { + condition_vars = { + organization = var.organization + tags = { + org_policies_tag_name = local.org_policies_tag_name + } + } + } + factories_config = { + custom_roles = var.factories_config.custom_roles + org_policy_custom_constraints = ( + var.bootstrap_user != null ? null : var.factories_config.custom_constraints + ) + org_policies = ( + var.bootstrap_user != null ? null : var.factories_config.org_policies + ) + } + logging_sinks = { + for name, attrs in var.log_sinks : name => { + bq_partitioned_table = attrs.type == "bigquery" + destination = local.log_sink_destinations[name].id + filter = attrs.filter + type = attrs.type + disabled = attrs.disabled + exclusions = attrs.exclusions + } + } + tags = { + (var.org_policies_config.tag_name) = { + description = "Organization policy conditions." + iam = {} + values = merge( + { + allowed-essential-contacts-domains-all = {} + allowed-policy-member-domains-all = {} + }, + var.org_policies_config.tag_values + ) + } + } +} diff --git a/fast/stages/0-bootstrap/outputs-files.tf b/fast/stages/0-bootstrap-legacy/outputs-files.tf similarity index 100% rename from fast/stages/0-bootstrap/outputs-files.tf rename to fast/stages/0-bootstrap-legacy/outputs-files.tf diff --git a/fast/stages/0-bootstrap/outputs-gcs.tf b/fast/stages/0-bootstrap-legacy/outputs-gcs.tf similarity index 100% rename from fast/stages/0-bootstrap/outputs-gcs.tf rename to fast/stages/0-bootstrap-legacy/outputs-gcs.tf diff --git a/fast/stages/0-bootstrap/outputs-providers.tf b/fast/stages/0-bootstrap-legacy/outputs-providers.tf similarity index 100% rename from fast/stages/0-bootstrap/outputs-providers.tf rename to fast/stages/0-bootstrap-legacy/outputs-providers.tf diff --git a/fast/stages/0-bootstrap-legacy/outputs.tf b/fast/stages/0-bootstrap-legacy/outputs.tf new file mode 100644 index 000000000..8ac8dc5d0 --- /dev/null +++ b/fast/stages/0-bootstrap-legacy/outputs.tf @@ -0,0 +1,200 @@ +/** + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +locals { + _tpl_providers = "${path.module}/templates/providers.tf.tpl" + # render CI/CD workflow templates + cicd_workflows = { + for k, v in local.cicd_repositories : "${v.level}-${k}" => templatefile( + "${path.module}/templates/workflow-${v.repository.type}.yaml", { + # If users give a list of custom audiences we set by default the first element. + # If no audiences are given, we set https://iam.googleapis.com/{PROVIDER_NAME} + audiences = try( + local.cicd_providers[v.identity_provider].audiences, [] + ) + identity_provider = try( + local.cicd_providers[v.identity_provider].name, "" + ) + outputs_bucket = module.automation-tf-output-gcs.name + service_accounts = { + apply = try(module.automation-tf-cicd-sa[k].email, "") + plan = try(module.automation-tf-cicd-r-sa[k].email, "") + } + stage_name = k + tf_providers_files = { + apply = local.cicd_workflow_providers[k] + plan = local.cicd_workflow_providers["${k}-r"] + } + tf_var_files = k == "bootstrap" ? [] : [ + "0-bootstrap.auto.tfvars.json", + "0-globals.auto.tfvars.json" + ] + } + ) + } + tfvars = { + automation = { + federated_identity_pool = try( + google_iam_workload_identity_pool.default[0].name, null + ) + federated_identity_providers = local.cicd_providers + outputs_bucket = module.automation-tf-output-gcs.name + project_id = module.automation-project.project_id + project_number = module.automation-project.number + service_accounts = { + bootstrap = module.automation-tf-bootstrap-sa.email + bootstrap-r = module.automation-tf-bootstrap-r-sa.email + resman = module.automation-tf-resman-sa.email + resman-r = module.automation-tf-resman-r-sa.email + vpcsc = module.automation-tf-vpcsc-sa.email + vpcsc-r = module.automation-tf-vpcsc-r-sa.email + } + } + billing = { + dataset = try(module.billing-export-dataset[0].id, null) + project_id = try(module.billing-export-project[0].project_id, null) + project_number = try(module.billing-export-project[0].number, null) + } + custom_roles = module.organization.custom_role_id + logging = { + project_id = module.log-export-project.project_id + project_number = module.log-export-project.number + writer_identities = module.organization.sink_writer_identities + destinations = { + bigquery = try(module.log-export-dataset[0].id, null) + logging = { for k, v in module.log-export-logbucket : k => v.id } + pubsub = { for k, v in module.log-export-pubsub : k => v.id } + storage = try(module.log-export-gcs[0].id, null) + } + } + org_policy_tags = { + key_id = ( + module.organization.tag_keys[var.org_policies_config.tag_name].id + ) + key_name = var.org_policies_config.tag_name + values = { + for k, v in module.organization.tag_values : + split("/", k)[1] => v.id + } + } + universe = var.universe + } + tfvars_globals = { + billing_account = var.billing_account + groups = local.principals + environments = { + for k, v in var.environments : k => { + is_default = v.is_default + key = k + name = v.name + short_name = v.short_name != null ? v.short_name : k + tag_name = v.tag_name != null ? v.tag_name : lower(v.name) + } + } + locations = local.locations + organization = var.organization + prefix = var.prefix + } +} + +output "automation" { + description = "Automation resources." + value = local.tfvars.automation +} + +output "billing_dataset" { + description = "BigQuery dataset prepared for billing export." + value = try(module.billing-export-dataset[0].id, null) +} + +output "cicd_repositories" { + description = "CI/CD repository configurations." + value = { + for k, v in local.cicd_repositories : k => { + branch = v.repository.branch + name = v.repository.name + provider = try(local.cicd_providers[v.identity_provider].name, null) + service_account = try(module.automation-tf-cicd-sa[k].email, null) + } + } +} + +output "custom_roles" { + description = "Organization-level custom roles." + value = module.organization.custom_role_id +} + +output "outputs_bucket" { + description = "GCS bucket where generated output files are stored." + value = module.automation-tf-output-gcs.name +} + +output "project_ids" { + description = "Projects created by this stage." + value = { + automation = module.automation-project.project_id + billing-export = try(module.billing-export-project[0].project_id, null) + log-export = module.log-export-project.project_id + } +} + +# ready to use provider configurations for subsequent stages when not using files +output "providers" { + # tfdoc:output:consumers stage-01 + description = "Terraform provider files for this stage and dependent stages." + sensitive = true + value = local.providers +} + +output "service_accounts" { + description = "Automation service accounts created by this stage." + value = { + bootstrap = module.automation-tf-bootstrap-sa.email + resman = module.automation-tf-resman-sa.email + } +} + +# ready to use variable values for subsequent stages +output "tfvars" { + description = "Terraform variable files for the following stages." + sensitive = true + value = local.tfvars +} + +output "tfvars_globals" { + description = "Terraform Globals variable files for the following stages." + sensitive = true + value = local.tfvars_globals +} + +output "workforce_identity_pool" { + description = "Workforce Identity Federation pool." + value = { + pool = try( + google_iam_workforce_pool.default[0].name, null + ) + } +} + +output "workload_identity_pool" { + description = "Workload Identity Federation pool and providers." + value = { + pool = try( + google_iam_workload_identity_pool.default[0].name, null + ) + providers = local.cicd_providers + } +} diff --git a/fast/stages/0-bootstrap-legacy/schemas/custom-role.schema.json b/fast/stages/0-bootstrap-legacy/schemas/custom-role.schema.json new file mode 120000 index 000000000..a1d6e5658 --- /dev/null +++ b/fast/stages/0-bootstrap-legacy/schemas/custom-role.schema.json @@ -0,0 +1 @@ +../../../../modules/organization/schemas/custom-role.schema.json \ No newline at end of file diff --git a/fast/stages/0-bootstrap-experimental/schemas/custom-role.schema.md b/fast/stages/0-bootstrap-legacy/schemas/custom-role.schema.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/custom-role.schema.md rename to fast/stages/0-bootstrap-legacy/schemas/custom-role.schema.md diff --git a/fast/stages/1-resman/schemas/org-policies.schema.json b/fast/stages/0-bootstrap-legacy/schemas/org-policies.schema.json similarity index 100% rename from fast/stages/1-resman/schemas/org-policies.schema.json rename to fast/stages/0-bootstrap-legacy/schemas/org-policies.schema.json diff --git a/fast/stages/0-bootstrap-experimental/schemas/org-policies.schema.md b/fast/stages/0-bootstrap-legacy/schemas/org-policies.schema.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/org-policies.schema.md rename to fast/stages/0-bootstrap-legacy/schemas/org-policies.schema.md diff --git a/fast/stages/0-bootstrap/schemas/org-policy-custom-constraint.schema.json b/fast/stages/0-bootstrap-legacy/schemas/org-policy-custom-constraint.schema.json similarity index 100% rename from fast/stages/0-bootstrap/schemas/org-policy-custom-constraint.schema.json rename to fast/stages/0-bootstrap-legacy/schemas/org-policy-custom-constraint.schema.json diff --git a/fast/stages/0-bootstrap/schemas/org-policy-custom-constraint.schema.md b/fast/stages/0-bootstrap-legacy/schemas/org-policy-custom-constraint.schema.md similarity index 100% rename from fast/stages/0-bootstrap/schemas/org-policy-custom-constraint.schema.md rename to fast/stages/0-bootstrap-legacy/schemas/org-policy-custom-constraint.schema.md diff --git a/fast/stages/0-bootstrap/templates/providers.tf.tpl b/fast/stages/0-bootstrap-legacy/templates/providers.tf.tpl similarity index 100% rename from fast/stages/0-bootstrap/templates/providers.tf.tpl rename to fast/stages/0-bootstrap-legacy/templates/providers.tf.tpl diff --git a/fast/addons/1-resman-tenants/templates/providers_terraform.tf.tpl b/fast/stages/0-bootstrap-legacy/templates/providers_terraform.tf.tpl similarity index 100% rename from fast/addons/1-resman-tenants/templates/providers_terraform.tf.tpl rename to fast/stages/0-bootstrap-legacy/templates/providers_terraform.tf.tpl diff --git a/fast/stages/0-bootstrap-experimental/assets/workflow-github.yaml b/fast/stages/0-bootstrap-legacy/templates/workflow-github.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/assets/workflow-github.yaml rename to fast/stages/0-bootstrap-legacy/templates/workflow-github.yaml diff --git a/fast/addons/1-resman-tenants/templates/workflow-gitlab.yaml b/fast/stages/0-bootstrap-legacy/templates/workflow-gitlab.yaml similarity index 100% rename from fast/addons/1-resman-tenants/templates/workflow-gitlab.yaml rename to fast/stages/0-bootstrap-legacy/templates/workflow-gitlab.yaml diff --git a/fast/stages/0-bootstrap/terraform.tfvars.sample b/fast/stages/0-bootstrap-legacy/terraform.tfvars.sample similarity index 100% rename from fast/stages/0-bootstrap/terraform.tfvars.sample rename to fast/stages/0-bootstrap-legacy/terraform.tfvars.sample diff --git a/fast/stages/0-bootstrap/variables-addons.tf b/fast/stages/0-bootstrap-legacy/variables-addons.tf similarity index 100% rename from fast/stages/0-bootstrap/variables-addons.tf rename to fast/stages/0-bootstrap-legacy/variables-addons.tf diff --git a/fast/stages/0-bootstrap-legacy/variables.tf b/fast/stages/0-bootstrap-legacy/variables.tf new file mode 100644 index 000000000..fec2f8a8f --- /dev/null +++ b/fast/stages/0-bootstrap-legacy/variables.tf @@ -0,0 +1,393 @@ +/** + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +variable "billing_account" { + description = "Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`." + type = object({ + id = string + force_create = optional(object({ + dataset = optional(bool, false) + project = optional(bool, false) + log_bucket = optional(bool, false) + }), {}) + is_org_level = optional(bool, true) + no_iam = optional(bool, false) + }) + nullable = false + validation { + condition = ( + var.billing_account.force_create.dataset != true || + var.billing_account.force_create.project == true + ) + error_message = "Forced dataset creation also needs project creation." + } +} + +variable "bootstrap_user" { + description = "Email of the nominal user running this stage for the first time." + type = string + default = null +} + +variable "cicd_config" { + description = "CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed." + type = object({ + bootstrap = optional(object({ + identity_provider = string + repository = object({ + name = string + branch = optional(string) + type = optional(string, "github") + }) + })) + resman = optional(object({ + identity_provider = string + repository = object({ + name = string + branch = optional(string) + type = optional(string, "github") + }) + })) + vpcsc = optional(object({ + identity_provider = string + repository = object({ + name = string + branch = optional(string) + type = optional(string, "github") + }) + })) + }) + nullable = false + default = {} + validation { + condition = alltrue([ + for k, v in coalesce(var.cicd_config, {}) : + v == null || ( + contains(["github", "gitlab", "terraform"], coalesce(try(v.repository.type, null), "null")) + ) + ]) + error_message = "Invalid repository type, supported types: 'github', 'gitlab', or 'terraform'." + } +} + +variable "custom_roles" { + description = "Map of role names => list of permissions to additionally create at the organization level." + type = map(list(string)) + nullable = false + default = {} +} + +variable "environments" { + description = "Environment names. When not defined, short name is set to the key and tag name to lower(name)." + type = map(object({ + name = string + is_default = optional(bool, false) + short_name = optional(string) + tag_name = optional(string) + })) + nullable = false + default = { + dev = { + name = "Development" + } + prod = { + name = "Production" + is_default = true + } + } + validation { + condition = anytrue([ + for k, v in var.environments : v.is_default == true + ]) + error_message = "At least one environment should be marked as default." + } + validation { + condition = alltrue([ + for k, v in var.environments : join(" ", regexall( + "[a-zA-Z][a-zA-Z0-9\\s-]+[a-zA-Z0-9]", v.name + )) == v.name + ]) + error_message = "Environment names can only contain letters numbers dashes or spaces." + } + validation { + condition = alltrue([ + for k, v in var.environments : (length(coalesce(v.short_name, k)) <= 4) + ]) + error_message = "If environment key is longer than 4 characters, provide short_name that is at most 4 characters long." + } +} + +variable "essential_contacts" { + description = "Email used for essential contacts, unset if null." + type = string + default = null +} + +variable "factories_config" { + description = "Configuration for the resource factories or external data." + type = object({ + custom_constraints = optional(string, "data/custom-constraints") + custom_roles = optional(string, "data/custom-roles") + org_policies = optional(string, "data/org-policies") + org_policies_iac = optional(string, "data/org-policies-iac") + }) + nullable = false + default = {} +} + +variable "groups" { + # https://cloud.google.com/docs/enterprise/setup-checklist + description = "Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated." + type = object({ + gcp-billing-admins = optional(string, "gcp-billing-admins") + gcp-devops = optional(string, "gcp-devops") + gcp-network-admins = optional(string, "gcp-vpc-network-admins") + gcp-organization-admins = optional(string, "gcp-organization-admins") + gcp-secops-admins = optional(string, "gcp-security-admins") + gcp-security-admins = optional(string, "gcp-security-admins") + # aliased to gcp-devops as the checklist does not create it + gcp-support = optional(string, "gcp-devops") + }) + nullable = false + default = {} +} + +variable "iam" { + description = "Organization-level custom IAM settings in role => [principal] format." + type = map(list(string)) + nullable = false + default = {} +} + +variable "iam_bindings_additive" { + description = "Organization-level custom additive IAM bindings. Keys are arbitrary." + type = map(object({ + member = string + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })) + nullable = false + default = {} +} + +variable "iam_by_principals" { + description = "Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the `iam` variable." + type = map(list(string)) + default = {} + nullable = false +} + +variable "locations" { + description = "Optional locations for GCS, BigQuery, and logging buckets created here." + type = object({ + bq = optional(string, "EU") + gcs = optional(string, "EU") + logging = optional(string, "global") + pubsub = optional(list(string), []) + }) + nullable = false + default = {} +} + +# See https://cloud.google.com/architecture/exporting-stackdriver-logging-for-security-and-access-analytics +# for additional logging filter examples +variable "log_sinks" { + description = "Org-level log sinks, in name => {type, filter} format." + type = map(object({ + filter = string + type = string + disabled = optional(bool, false) + exclusions = optional(map(string), {}) + })) + nullable = false + default = { + audit-logs = { + # activity logs include Google Workspace / Cloud Identity logs + # exclude them via additional filter stanza if needed + filter = <<-FILTER + log_id("cloudaudit.googleapis.com/activity") OR + log_id("cloudaudit.googleapis.com/system_event") OR + log_id("cloudaudit.googleapis.com/policy") OR + log_id("cloudaudit.googleapis.com/access_transparency") + FILTER + type = "logging" + # exclusions = { + # gke-audit = "protoPayload.serviceName=\"k8s.io\"" + # } + } + iam = { + filter = <<-FILTER + protoPayload.serviceName="iamcredentials.googleapis.com" OR + protoPayload.serviceName="iam.googleapis.com" OR + protoPayload.serviceName="sts.googleapis.com" + FILTER + type = "logging" + } + vpc-sc = { + filter = <<-FILTER + protoPayload.metadata.@type="type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata" + FILTER + type = "logging" + } + workspace-audit-logs = { + filter = <<-FILTER + protoPayload.serviceName="admin.googleapis.com" OR + protoPayload.serviceName="cloudidentity.googleapis.com" OR + protoPayload.serviceName="login.googleapis.com" + FILTER + type = "logging" + } + } + validation { + condition = alltrue([ + for k, v in var.log_sinks : + contains(["bigquery", "logging", "project", "pubsub", "storage"], v.type) + ]) + error_message = "Type must be one of 'bigquery', 'logging', 'project', 'pubsub', 'storage'." + } +} + +variable "org_policies_config" { + description = "Organization policies customization." + type = object({ + iac_policy_member_domains = optional(list(string)) + import_defaults = optional(bool, false) + tag_name = optional(string, "org-policies") + tag_values = optional(map(object({ + description = optional(string, "Managed by the Terraform organization module.") + iam = optional(map(list(string)), {}) + id = optional(string) + })), {}) + }) + default = {} +} + +variable "organization" { + description = "Organization details." + type = object({ + id = number + domain = optional(string) + customer_id = optional(string) + }) +} + +variable "outputs_location" { + description = "Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable." + type = string + default = null +} + +variable "prefix" { + description = "Prefix used for resources that need unique names. Use 9 characters or less." + type = string + validation { + condition = try(length(var.prefix), 0) < 10 + error_message = "Use a maximum of 9 characters for prefix." + } +} + +variable "project_parent_ids" { + description = "Optional parents for projects created here in folders/nnnnnnn format. Null values will use the organization as parent." + type = object({ + automation = optional(string) + billing = optional(string) + logging = optional(string) + }) + default = {} + nullable = false +} + +variable "resource_names" { + description = "Resource names overrides for specific resources. Prefix is always set via code, except where noted in the variable type." + type = object({ + bq-billing = optional(string, "billing_export") + bq-logs = optional(string, "logs") + gcs-bootstrap = optional(string, "prod-iac-core-bootstrap-0") + gcs-logs = optional(string, "prod-logs") + gcs-outputs = optional(string, "prod-iac-core-outputs-0") + gcs-resman = optional(string, "prod-iac-core-resman-0") + gcs-vpcsc = optional(string, "prod-iac-core-vpcsc-0") + project-automation = optional(string, "prod-iac-core-0") + project-billing = optional(string, "prod-billing-exp-0") + project-logs = optional(string, "prod-audit-logs-0") + pubsub-logs_template = optional(string, "$${key}") + sa-bootstrap = optional(string, "prod-bootstrap-0") + sa-bootstrap_ro = optional(string, "prod-bootstrap-0r") + sa-cicd_template = optional(string, "prod-$${key}-1") + sa-cicd_template_ro = optional(string, "prod-$${key}-1r") + sa-resman = optional(string, "prod-resman-0") + sa-resman_ro = optional(string, "prod-resman-0r") + sa-vpcsc = optional(string, "prod-vpcsc-0") + sa-vpcsc_ro = optional(string, "prod-vpcsc-0r") + # the identity provider resources also interpolate prefix + wf-bootstrap = optional(string, "$${prefix}-bootstrap") + wf-provider_template = optional(string, "$${prefix}-bootstrap-$${key}") + wif-bootstrap = optional(string, "$${prefix}-bootstrap") + wif-provider_template = optional(string, "$${prefix}-bootstrap-$${key}") + }) + nullable = false + default = {} +} + +variable "universe" { + description = "Target GCP universe." + type = object({ + domain = string + prefix = string + unavailable_services = optional(list(string), []) + }) + default = null +} + +variable "workforce_identity_providers" { + description = "Workforce Identity Federation pools." + type = map(object({ + attribute_condition = optional(string) + issuer = string + display_name = string + description = string + disabled = optional(bool, false) + saml = optional(object({ + idp_metadata_xml = string + }), null) + })) + default = {} + nullable = false +} + +variable "workload_identity_providers" { + description = "Workload Identity Federation pools. The `cicd_repositories` variable references keys here." + type = map(object({ + attribute_condition = optional(string) + issuer = string + custom_settings = optional(object({ + issuer_uri = optional(string) + audiences = optional(list(string), []) + jwks_json = optional(string) + }), {}) + })) + default = {} + nullable = false + # TODO: fix validation + # validation { + # condition = var.federated_identity_providers.custom_settings == null + # error_message = "Custom settings cannot be null." + # } +} diff --git a/fast/stages/0-bootstrap/.fast-stage.env b/fast/stages/0-bootstrap/.fast-stage.env index a842174c3..c739edfc8 100644 --- a/fast/stages/0-bootstrap/.fast-stage.env +++ b/fast/stages/0-bootstrap/.fast-stage.env @@ -1,4 +1,4 @@ -FAST_STAGE_DESCRIPTION="organization bootstrap" +FAST_STAGE_DESCRIPTION="FAST Bootstrap." FAST_STAGE_LEVEL=0 FAST_STAGE_NAME=bootstrap # FAST_STAGE_DEPS="0-globals 0-bootstrap" diff --git a/fast/stages/0-bootstrap/README.md b/fast/stages/0-bootstrap/README.md index 8e11db17f..9a8ba9046 100644 --- a/fast/stages/0-bootstrap/README.md +++ b/fast/stages/0-bootstrap/README.md @@ -1,726 +1,639 @@ -# Organization bootstrap - -The primary purpose of this stage is to enable critical organization-level functionalities that depend on broad administrative permissions, and prepare the prerequisites needed to enable automation in this and future stages. - -It is intentionally simple, to minimize usage of administrative-level permissions and enable simple auditing and troubleshooting, and only deals with three sets of resources: - -- project, service accounts, and GCS buckets for automation -- projects, BQ datasets, and sinks for audit log and billing exports -- IAM bindings on the organization - -Use the following diagram as a simple high level reference for the following sections, which describe the stage and its possible customizations in detail. - -

- Organization-level diagram -

+# FAST Organization Setup -- [Design overview and choices](#design-overview-and-choices) - - [User groups](#user-groups) - - [Organization-level IAM](#organization-level-iam) - - [Organization policies](#organization-policies) - - [Security Command Center Enterprise](#security-command-center-enterprise) - - [Tags and Organization Policy conditions](#tags-and-organization-policy-conditions) - - [Automation project and resources](#automation-project-and-resources) - - [Billing account](#billing-account) - - [Organization-level logging](#organization-level-logging) - - [Naming](#naming) - - [Workforce Identity Federation](#workforce-identity-federation) - - [Workload Identity Federation and CI/CD](#workload-identity-federation-and-cicd) -- [How to run this stage](#how-to-run-this-stage) +- [Quickstart](#quickstart) - [Prerequisites](#prerequisites) - - [Standalone billing account](#standalone-billing-account) - - [Preventing creation of billing-related IAM bindings](#preventing-creation-of-billing-related-iam-bindings) - - [Groups](#groups) - - [Configure variables](#configure-variables) - - [Output files and cross-stage variables](#output-files-and-cross-stage-variables) - - [Running the stage](#running-the-stage) -- [Customizations](#customizations) - - [Group names](#group-names) - - [IAM](#iam) - - [Log sinks and log destinations](#log-sinks-and-log-destinations) - - [Names and naming convention](#names-and-naming-convention) - - [Workload Identity Federation](#workload-identity-federation) - - [Project folders](#project-folders) - - [CI/CD repositories](#cicd-repositories) - - [Add-ons](#add-ons) + - [Select/configure a factory dataset](#selectconfigure-a-factory-dataset) + - [Configure defaults](#configure-defaults) + - [Initial user permissions](#initial-user-permissions) + - [First apply cycle](#first-apply-cycle) + - [Importing org policies](#importing-org-policies) + - [Local output files storage](#local-output-files-storage) + - [Init and apply the stage](#init-and-apply-the-stage) + - [Provider setup and final apply cycle](#provider-setup-and-final-apply-cycle) +- [Default factory datasets](#default-factory-datasets) + - ["Classic FAST" dataset](#classic-fast-dataset) + - ["Minimal" dataset](#minimal-dataset) + - ["Tenants" dataset](#tenants-dataset) +- [Detailed configuration](#detailed-configuration) + - [Factory data](#factory-data) + - [Defaults configuration](#defaults-configuration) + - [Billing account IAM](#billing-account-iam) + - [Context-based replacement in the billing account factory](#context-based-replacement-in-the-billing-account-factory) + - [Organization configuration](#organization-configuration) + - [Context-based replacement in organization factories](#context-based-replacement-in-organization-factories) + - [Resource management hierarchy](#resource-management-hierarchy) + - [Context-based replacement in the folders factory](#context-based-replacement-in-the-folders-factory) + - [Project factory](#project-factory) + - [CI/CD configuration](#cicd-configuration) +- [Leveraging classic FAST Stages](#leveraging-classic-fast-stages) + - [VPC Service Controls](#vpc-service-controls) + - [Security](#security) - [Files](#files) - [Variables](#variables) - [Outputs](#outputs) -## Design overview and choices +This stage implements a flexible approach to organization bootstrapping and resource management, that offers full customization via YAML factories. -As mentioned above, this stage only does the bare minimum required to bootstrap automation, and ensure that base audit and billing exports are in place from the start to provide some measure of accountability, even before the security configurations are applied in a later stage. +It heavily relies on a new [project factory module](../../../modules/project-factory/) for folder and project configurations, and leverages a new approach to [context-based interpolation](../../../modules/project-factory/README.md#context-based-interpolation) that allows writing legible, portable YAML definitions. -It also sets up organization-level IAM bindings so the Organization Administrator role is only used here, trading off some design freedom for ease of auditing and troubleshooting, and reducing the risk of costly security mistakes down the line. The only exception to this rule is for the [Resource Management stage](../1-resman) service account, described below. +The default set of YAML configuration files in the `data` folder mirrors the traditional FAST layout, and implements full compatibility with existing FAST stages like VPC-SC, security, networking, etc. -### User groups +The default configuration can be used as a starting point to implement radically different Landing Zone designs, or trimmed down to its bare minimum where the requirements are simply to have a secure organization-level configuration (possibly with VPC-SC), and a working project factory. -User groups are important, not only here but throughout the whole automation process. They provide a stable frame of reference that allows decoupling the final set of permissions for each group, from the stage where entities and resources are created and their IAM bindings defined. For example, the final set of roles for the networking group is contributed by this stage at the organization level (XPN Admin, Cloud Asset Viewer, etc.), and by the Resource Management stage at the folder level. +## Quickstart -We have standardized the initial set of groups on those outlined in the [GCP Enterprise Setup Checklist](https://cloud.google.com/docs/enterprise/setup-checklist) to simplify adoption. They provide a comprehensive and flexible starting point that can suit most users. Adding new groups, or deviating from the initial setup is possible and reasonably simple, and it's briefly outlined in the customization section below. +The high-level flow for running this stage is: -### Organization-level IAM - -The service account used in the [Resource Management stage](../1-resman) needs to be able to grant specific permissions at the organizational level, to enable specific functionality for subsequent stages that deal with network or security resources, or billing-related activities. - -In order to be able to assign those roles without having the full authority of the Organization Admin role, this stage defines a custom role that only allows setting IAM policies on the organization, and grants it via a [delegated role grant](https://cloud.google.com/iam/docs/setting-limits-on-granting-roles) that only allows it to be used to grant a limited subset of roles. - -In this way, the Resource Management service account can effectively act as an Organization Admin, but only to grant the specific roles it needs to control. - -One consequence of the above setup is the need to configure IAM bindings that can be assigned via the condition as non-authoritative, since those same roles are effectively under the control of two stages: this one and Resource Management. Using authoritative bindings for these roles (instead of non-authoritative ones) would generate potential conflicts, where each stage could try to overwrite and negate the bindings applied by the other at each `apply` cycle. - -A full reference of IAM roles managed by this stage [is available here](./IAM.md). - -### Organization policies - -It's often desirable to have organization policies deployed before any other resource in the org, so as to ensure compliance with specific requirements (e.g. location restrictions), or control the configuration of specific resources (e.g. default network at project creation or service account grants). - -To cover this use case, organization policies have been moved from the resource management to the bootstrap stage in FAST versions after 26.0.0. They are managed via the usual factory approach, and a [sample set of data files](./data/org-policies/) is included with this stage. They are not applied during the initial run when the `bootstrap_user` variable is set, to work around incompatibilities with user credentials. - -FAST uses unmanaged organization policies by default. For those who prefer managed policies, a separate sample set is available. To use these managed policies, configure `factories_config` as shown below. - -```tfvars -factories_config = { - org_policies = "data/org-policies-managed" -} -``` - -#### Security Command Center Enterprise - -The DRS policy mentioned above might make it complex to [enable Security Command Center Enterprise](https://cloud.google.com/security-command-center/docs/activate-enterprise-tier#verify_organization_policies). If this is the case, you can temporarily disable it via the Cloud Console, enable SCC Enterprise, then re-enable the policy. - -#### Tags and Organization Policy conditions - -Organization policy exceptions are managed via a dedicated resource management tag hierarchy, rooted in the `org-policies` tag key. A default condition is already present for the the `iam.allowedPolicyMemberDomains` constraint, that relaxes the policy on resources that have the `org-policies/allowed-policy-member-domains-all` tag value bound or inherited, and similarly for `essentialcontacts.allowedContactDomains` via the `allowed-essential-contacts-domains-all` tag value. - -Further tag values can be defined via the `org_policies_config.tag_values` variable, and IAM access can be granted on them via the same variable. Once a tag value has been created, its id can be used in constraint rule conditions. Note that only one tag value from a given tag key can be bound to a node (organization, folder, or project) in the resource hierarchy. Since these tag values are all rooted in the `org-policies` key, this limits the ability to apply fine-grained policy constraints. It may be more desirable to model policy overrides using coarser groups of tag values to create a policy "profile". For example, instead of separating `compute.skipDefaultNetworkCreation` and `compute.vmExternalIpAccess`, enforce both constraints by default and relax them both using the same tag value such as `sandbox`. See [tags overview](https://cloud.google.com/resource-manager/docs/tags/tags-overview) for more information. - -Management of the rest of the tag hierarchy is delegated to the resource management stage, as that is often intimately tied to the folder hierarchy design. - -The organization policy tag key and values managed by this stage have been added to the `0-bootstrap.auto.tfvars` stage, so that IAM can be delegated to the resource management or successive stages via their ids. - -The following example shows an example on how to define an additional tag value, and use it in a boolean constraint rule. - -This snippet defines a new tag value under the `org-policies` tag key via the `org_policies_config` variable, and assigns the permission to bind it to a group. - -```hcl -# stage 0 custom tfvars -org_policies_config = { - tag_values = { - compute-require-oslogin-false = { - description = "Bind this tag to set oslogin to false." - iam = { - "roles/resourcemanager.tagUser" = [ - "group:foo@example.com" - ] - } - } - } -} -# tftest skip -``` - -The above tag can be used to define a constraint condition via the `data/org-policies/compute.yaml` or similar factory file. The name of the tag can be referenced from the factory files using `tags.org_policies_config`, as shown below. - -```yaml -compute.requireOsLogin: - rules: - - enforce: true - - enforce: false - condition: - expression: resource.matchTag('${tags.org_policies_tag_name}', 'compute-require-oslogin-false') -``` - -### Automation project and resources - -One other design choice worth mentioning here is using a single automation project for all foundational stages. We trade off some complexity on the API side (single source for usage quota, multiple service activation) for increased flexibility and simpler operations, while still effectively providing the same degree of separation via resource-level IAM. - -### Billing account - -We support three use cases in regards to billing: - -- the billing account is part of this same organization, IAM bindings will be set at the organization level -- the billing account is not considered part of an organization (even though it might be), billing IAM bindings are set on the billing account itself -- billing IAM is managed separately, and no bindings should (or can) be set via Terraform, this requires a few extra steps and is definitely not recommended and mainly used for development purposes - -For same-organization billing, we configure a custom organization role that can set IAM bindings, via a delegated role grant to limit its scope to the relevant roles. - -For details on configuring the different billing account modes, refer to the [How to run this stage](#how-to-run-this-stage) section below. - -Because of limitations of API availability, manual steps have to be followed to enable billing export within billing project to BigQuery dataset `billing_export` which will be created as part of the bootstrap stage. The process to share billing data [is outlined here](https://cloud.google.com/billing/docs/how-to/export-data-bigquery-setup#enable-bq-export). - -### Organization-level logging - -We create organization-level log sinks early in the bootstrap process to ensure a proper audit trail is in place from the very beginning. By default, we provide log filters to capture [Cloud Audit Logs](https://cloud.google.com/logging/docs/audit), [VPC Service Controls violations](https://cloud.google.com/vpc-service-controls/docs/troubleshooting#vpc-sc-errors) and [Workspace Logs](https://cloud.google.com/logging/docs/audit/configure-gsuite-audit-logs) into logging buckets in the top-level audit logging project. - -An organization-level sink captures IAM data access logs, including authentication and impersonation events for service accounts. To manage logging costs, the default configuration enables IAM data access logging only within the automation project (where sensitive service accounts reside). For enhanced security across the entire organization, consider enabling these logs at the organization level. - -The [Customizations](#log-sinks-and-log-destinations) section explains how to change the logs captured and their destination. - -### Naming - -We are intentionally not supporting random prefix/suffixes for names, as that is an antipattern typically only used in development. It does not map to our customer's actual production usage, where they always adopt a fixed naming convention. - -What is implemented here is a fairly common convention, composed of tokens ordered by relative importance: - -- an organization-level static prefix less or equal to 9 characters (e.g. `myco` or `myco-gcp`) -- an optional tenant-level prefix, if using tenant factory -- an environment identifier (e.g. `prod`) -- a team/owner identifier (e.g. `sec` for Security) -- a context identifier (e.g. `core` or `kms`) -- an arbitrary identifier used to distinguish similar resources (e.g. `0`, `1`) - -> [!WARNING] -> When using tenant factory, a tenant prefix will be automatically generated as `{prefix}-{tenant-shortname}`. The maximum length of such prefix must be 11 characters or less, which means that the longer org-level prefix you use, the less chars you'll have available for the `tenant-shortname`. - -Tokens are joined by a `-` character, making it easy to separate the individual tokens visually, and to programmatically split them in billing exports to derive initial high-level groupings for cost attribution. - -The convention is used in its full form only for specific resources with globally unique names (projects, GCS buckets). Other resources adopt a shorter version for legibility, as the full context can always be derived from their project. - -The [Customizations](#names-and-naming-convention) section on names below explains how to configure tokens, or implement a different naming convention. - -### Workforce Identity Federation - -This stage supports configuration of [Workforce Identity Federation](https://cloud.google.com/iam/docs/workforce-identity-federation) which lets an external identity provider (IdP) to authenticate and authorize a group of users (usually employees) using IAM, so that the users can access Google Cloud services. - -The following example shows an example on how to define a Workforce Identity pool for the organization. - -```hcl -# stage 0 wif tfvars -workforce_identity_providers = { - test = { - issuer = "azuread" - display_name = "wif-provider" - description = "Workforce Identity pool" - saml = { - idp_metadata_xml = "..." - } - } -} -# tftest skip -``` - -### Workload Identity Federation and CI/CD - -This stage also implements initial support for two interrelated features - -- configuration of [Workload Identity Federation](https://cloud.google.com/iam/docs/workload-identity-federation) pools and providers -- configuration of CI/CD repositories to allow impersonation via Workload identity Federation, and stage running via provided workflow templates - -Workload Identity Federation support allows configuring external providers independently from CI/CD, and offers predefined attributes for a few well known ones (more can be easily added by editing the `identity-providers-wlif.tf` file). Once providers have been configured their names are passed to the following stages via interface outputs, and can be leveraged to set up access or impersonation in IAM bindings. - -CI/CD support is fully implemented for GitHub, Gitlab, and Cloud Source Repositories / Cloud Build. For GitHub, we also offer a [separate supporting setup](../../extras/0-cicd-github/) to quickly create / configure repositories. The same applies for Gitlab with the [following extra stage](../../extras/0-cicd-gitlab/). - - - -For details on how to configure both features, refer to the Customizations sections below on [Workload Identity Federation](#workload-identity-federation) and [CI/CD repositories](#cicd-repositories). - -These features are optional and only enabled if the relevant variables have been populated. - -## How to run this stage - -This stage has straightforward initial requirements, as it is designed to work on newly created GCP organizations. Four steps are needed to bring up this stage: - -- an Organization Admin self-assigns the required roles listed below -- the same administrator runs the first `init/apply` sequence passing a special variable to `apply` -- the providers configuration file is derived from the Terraform output or linked from the generated file -- a second `init` is run to migrate state, and from then on, the stage is run via impersonation +- ensure all **pre-requisites** are in place, and identify at least one GCP organization admin principal (ideally a group) +- select the **factory data set** for the factories among those available - populate the **defaults file** with attributes matching your configuration (organization id, billing account, etc.) +(`data`, `data-minimal`, etc.) or edit/create your own +- assign a set of **initial IAM roles** to the admin principal +- run a **first init/apply cycle** using user credentials +- copy the generated provider file, **migrate state**, then run a second init/apply cycle using service account impersonated credentials ### Prerequisites -The roles that the Organization Admin used in the first `apply` needs to self-grant are: +This stage only requires minimal prerequisites: -- Billing Account Administrator (`roles/billing.admin`) - either on the organization or the billing account (see the following section for details) -- Logging Admin (`roles/logging.admin`) -- Organization Role Administrator (`roles/iam.organizationRoleAdmin`) -- Organization Administrator (`roles/resourcemanager.organizationAdmin`) -- Project Creator (`roles/resourcemanager.projectCreator`) -- Tag Admin (`roles/resourcemanager.tagAdmin`) -- Owner (`roles/owner`) +- one organization +- credentials with admin access to the organization and one billing account -To quickly self-grant the above roles, run the following code snippet as the initial Organization Admin: +The organization ideally needs to be empty. If pre-existing resources are present some care needs to be put into preserving their existing IAM and org policies, ideally my moving legacy projects to a dedicated folder where the current org-level configuration (IAM and org policies) can be replicated. + +Billing admin permissions are ideally available on either an org-contained billing account or an external one. If those are unavailable, the YAML configuration files need to be updated to remove billing IAM bindings, and those need to be assigned via an external flow. Refer to the [billing section](#billing-account-iam) for more details or non-standard configurations. + +The admin principal is typically a group that includes the user running the first apply, but any kind of principal is supported. More principals (network admins, security admins, etc.) are present in some of the [default factories datasets](#default-factory-datasets), and others can be added if needed by editing the YAML configuration files. + +### Select/configure a factory dataset + +The `factories_config` variable points to several paths containing the YAML configuration files used by this stage. The default variable configuration points to the legacy FAST compatible fileset in the `data` folder. + +If you are fine with this configuration nothing needs to be changed at this stage. To select a different setup create a `tfvars` file and set paths to the desired data folder, like shown in the example below. The different configurations produced by each fileset are described [later in this document](#default-factory-datasets). ```bash -# set variable for current logged in user -export FAST_BU=$(gcloud config list --format 'value(core.account)') +# create a file named 0-bootstrap.auto.tfvars containing the following +# and replace paths by pointing them to the desired data folder +factories_config = { + billing_accounts = "data/billing-accounts" + cicd = "data/cicd.yaml" + defaults = "data/defaults.yaml" + folders = "data/folders" + organization = "data/organization" + projects = "data/projects" +} +``` -# find and set your org id +### Configure defaults + +Configurations defaults are stored in the `defaults.yaml` file in the dataset selected above. Before starting, edit the following attributes in the file to match your configuration. + +The standard datasets use the `gcp-organization-admins` alias to assign administrator roles. The alias is expanded via the `context.iam_principals` attribute in the default file, which should be set to a valid group. Also make sure that the user running the initial apply is a member. + +```yaml +global: + # gcloud beta billing accounts list + billing_account: 123456-123456-123456 + locations: + bigquery: europe-west1 + logging: europe-west1 + organization: + # gcloud organizations list + domain: example.org + id: 1234567890 + customer_id: ABC0123CDE +projects: + defaults: + # define a unique prefix with a maximum of 9 characters + prefix: foo-1 + storage_location: europe-west1 +context: + iam_principals: + # make sure the user running apply is a member of this group + gcp-organization-admins: group:fabric-fast-owners@example.com +``` + +A more detailed example containing a few other attributes that can be set in the file is in a [later section](#defaults-configuration) in this document. + +### Initial user permissions + +Like in classic FAST, the user running the first apply cycle needs specific permissions on the organization and billing account. Copy the following snippet, edit it to match your organization/billing account ids, then run each command. + +To quickly self-grant the above roles, run the following code snippet as the initial Organization Admin. The best approach is to use the same group used for organization admins above. + +```bash +export FAST_PRINCIPAL="group:fabric-fast-owners@example.com" + +# find your organization and export its id in the FAST_ORG variable gcloud organizations list export FAST_ORG_ID=123456 -# set needed roles -export FAST_ROLES="roles/billing.admin roles/logging.admin \ - roles/iam.organizationRoleAdmin roles/resourcemanager.projectCreator \ - roles/resourcemanager.organizationAdmin roles/resourcemanager.tagAdmin \ +# set needed roles (billing role only needed for organization-owned account) +export FAST_ROLES="\ + roles/billing.admin \ + roles/logging.admin \ + roles/iam.organizationRoleAdmin \ + roles/orgpolicy.policyAdmin \ + roles/resourcemanager.folderAdmin \ + roles/resourcemanager.organizationAdmin \ + roles/resourcemanager.projectCreator \ + roles/resourcemanager.tagAdmin \ roles/owner" for role in $FAST_ROLES; do gcloud organizations add-iam-policy-binding $FAST_ORG_ID \ - --member user:$FAST_BU --role $role --condition None + --member $FAST_PRINCIPAL --role $role --condition None done ``` -Then make sure the same user is also part of the `gcp-organization-admins` group so that impersonating the automation service account later on will be possible. +If you are using an externally managed billing account, make sure user has Billing Admin role assigned on the account. -#### Standalone billing account +### First apply cycle -If you are using a standalone billing account, the identity applying this stage for the first time needs to be a billing account administrator: +#### Importing org policies + +If your dataset includes org policies which are already set in the organization, you need to either comment them out from the relevant YAML files or tell this stage to import them. To figure out which policies are set, run `gcloud org-policies list --organization [your org id]`, then set the `org_policies_imports` variable in your tfvars file. The following is an example. ```bash -export FAST_BILLING_ACCOUNT_ID=ABCD-01234-ABCD -gcloud beta billing accounts add-iam-policy-binding $FAST_BILLING_ACCOUNT_ID \ - --member user:$FAST_BU --role roles/billing.admin +gcloud org-policies list --organization 1234567890 +CONSTRAINT LIST_POLICY BOOLEAN_POLICY +iam.allowedPolicyMemberDomains SET - +compute.disableSerialPortAccess - SET ``` -#### Preventing creation of billing-related IAM bindings - -This configuration is possible but unsupported and only present for development purposes, use at your own risk: - -- configure `billing_account.id` as `null` and `billing_account.no_iam` to `true` in your `tfvars` file -- apply with `terraform apply -target 'module.automation-project.google_project.project[0]'` in addition to the initial user variable -- once Terraform raises an error run `terraform untaint 'module.automation-project.google_project.project[0]'` -- repeat the two steps above for `'module.log-export-project.google_project.project[0]'` -- go through the process to associate the billing account with the two projects -- configure `billing_account.id` with the real billing account id -- resume applying normally - -#### Groups - -Before the first run, the following IAM groups must exist to allow IAM bindings to be created (actual names are flexible, see the [Customization](#customizations) section): - -- `gcp-billing-admins` -- `gcp-devops` -- `gcp-vpc-network-admins` -- `gcp-organization-admins` -- `gcp-security-admins` - -You can refer to [this animated image](./groups.gif) for a step by step on group creation via the [Google Cloud Enterprise Checklist](https://cloud.google.com/docs/enterprise/setup-checklist). - -Please note that not all groups defined by the Checklist are actually used by FAST, as our approach to IAM is slightly different. As an example, we do not centralize monitoring functions as in our experience those are typically domain-specific (e.g. networking or application-level), so we don't leverage the corresponding groups. You are free of course to create those groups via the Checklist, and assign them roles via the IAM variables exposed by this stage. - -One more difference compared to the Checklist is the use in FAST of an additional group to centralize support functions like viewing tickets and accessing logging and monitoring data. To remain consistent with the [Google Cloud Enterprise Checklist](https://cloud.google.com/docs/enterprise/setup-checklist) we map these permissions to the `gcp-devops` group by default. However, we recommend creating a dedicated `gcp-support` group and updating the `groups` variable with the right value. - -#### Configure variables - -Then make sure you have configured the correct values for the following variables by providing a `terraform.tfvars` file: - -- `billing_account` - an object containing `id` as the id of your billing account, derived from the Cloud Console UI or by running `gcloud beta billing accounts list`, and the `is_org_level` flag that controls whether organization or account-level bindings are used, and a billing export project and dataset are created -- `groups` - the name mappings for your groups, if you're following the default convention you can leave this to the provided default -- `organization.id`, `organization.domain`, `organization.customer_id` - the id, domain and customer id of your organization, derived from the Cloud Console UI or by running `gcloud organizations list` -- `prefix` - the fixed org-level prefix used in your naming, maximum 9 characters long. Note that if you are using multitenant stages, then you will later need to configure a `tenant prefix`. - This `tenant prefix` can have a maximum length of 2 characters, - plus any unused characters from the from the `prefix`. - For example, if you specify a `prefix` that is 7 characters long, - then your `tenant prefix` can have a maximum of 4 characters. - -You can also adapt the example that follows to your needs: - ```tfvars -# use `gcloud beta billing accounts list` -# if you have too many accounts, check the Cloud Console :) -billing_account = { - id = "012345-67890A-BCDEF0" -} - -# use `gcloud organizations list` -organization = { - domain = "example.org" - id = 1234567890 - customer_id = "C000001" -} - -# local path to store tfvars/provider outputs generated by this stage -outputs_location = "~/fast-config" - -# locations for GCS, BigQuery, and logging buckets created here -locations = { - bq = "EU" - gcs = "EU" - logging = "global" - pubsub = [] -} - -# use something unique and no longer than 9 characters -prefix = "abcd" +# create or edit the 0-bootstrap.auto.tfvars.file +org_policies_imports = [ + 'iam.allowedPolicyMemberDomains', + 'compute.disableSerialPortAccess' +] ``` -### Output files and cross-stage variables +Once org policies have been imported, the variable definition can be removed from the tfvars file. -Each foundational FAST stage generates provider configurations and variable files can be consumed by the following stages, and saves them in a dedicated GCS bucket in the automation project. These files are a handy way to simplify stage configuration, and are also used by our CI/CD workflows to configure the repository files in the pipelines that validate and apply the code. +#### Local output files storage -Alongside the GCS stored files, you can also configure a second copy to be saves on the local filesystem, as a convenience when developing or bringing up the infrastructure before a proper CI/CD setup is in place. +Like any other FAST stage, this stage creates output files that contain information about the resources it manages, or provide initial provider and backend configuration for the following stages. -This second set of files is disabled by default, you can enable it by setting the `outputs_location` variable to a valid path on a local filesystem, e.g. +These files are only persisted by default on a special outputs bucket, but can additionally be also persisted to a local path. This is very useful during the initial deployment, as it allows rapid apply iteration cycles between stages, and provides an easy way to check or derive resource ids. + +To enable local output files storage, set the `outputs_location` variable in your tfvars file to a filesystem path dedicated to this organization's output files. The following snippet provides an example. ```tfvars -outputs_location = "~/fast-config" +# create or edit the 0-bootstrap.auto.tfvars.file +outputs_location = "~/fast-configs/test-0" ``` -Once the variable is set, `apply` will generate and manage providers and variables files, including the initial one used for this stage after the first run. You can then link these files in the relevant stages, instead of manually transferring outputs from one stage, to Terraform variables in another. +#### Init and apply the stage -Below is the outline of the output files generated by all stages, which is identical for both the GCS and local filesystem copies: - -```bash -[path specified in outputs_location] -├── providers -│   ├── 0-bootstrap-providers.tf -│   ├── 1-resman-providers.tf -│   ├── 2-networking-providers.tf -│   ├── 2-security-providers.tf -│   ├── 2-project-factory-dev-providers.tf -│   ├── 2-project-factory-prod-providers.tf -│   └── 9-sandbox-providers.tf -└── tfvars -│ ├── 0-bootstrap.auto.tfvars.json -│ ├── 1-resman.auto.tfvars.json -│ ├── 2-networking.auto.tfvars.json -│ └── 2-security.auto.tfvars.json -└── workflows - └── [optional depending on the configured CI/CD repositories] -``` - -### Running the stage - -Before running `init` and `apply`, check your environment so no extra variables that might influence authentication are present (e.g. `GOOGLE_IMPERSONATE_SERVICE_ACCOUNT`). In general you should use user application credentials, and FAST will then take care to provision automation identities and configure impersonation for you. - -When running the first `apply` as a user, you need to pass a special runtime variable so that the user roles are preserved when setting IAM bindings. +Once everything has been configured go through the standard Terraform init/apply cycle. ```bash terraform init -terraform apply \ - -var bootstrap_user=$(gcloud config list --format 'value(core.account)') +terraform apply ``` -> If you see an error related to project name already exists, please make sure the project name is unique or the project was not deleted recently +### Provider setup and final apply cycle -Once the initial `apply` completes successfully, configure a remote backend using the new GCS bucket, and impersonation on the automation service account for this stage. To do this you can use the generated `providers.tf` file from either +When the first apply cycle has completed successfully, you are ready to switch Terraform to use the new GCS backend and service account credentials. -- the local filesystem if you have configured output files as described above -- the GCS bucket where output files are always stored -- Terraform outputs (not recommended as it's more complex) +The first step is to link the generated provider file, either copying it from the GCS bucket or linking it from the local path if it has been configured in the previous step. -The following two snippets show how to leverage the `fast-links.sh` script in the FAST stages folder to fetch the commands required for output files linking or copying, using either the local output folder configured via Terraform variables, or the GCS bucket which can be derived from the `automation` output. +The instructions also assume that you have moved the `0-bootstrap.auto.tfvars` file (if you have one) to the GCS bucket or the local config files. This is good practice in order to have the tfvars file persisted, either via GCS or by committing it to a repository with the source code in a dedicated config folder. The file needs to be copied or moved by hand. Alternatively, the last copy/link command can be ignored. + +If local output files are available adjust the path, run the script, then copy/paste the resulting commands. ```bash -../fast-links.sh ~/fast-config - -# File linking commands for organization bootstrap stage +# if local outputs file are available +../fast-links.sh ~/fast-configs/test-0 +# File linking commands for FAST Bootstrap. stage # provider file -ln -s ~/fast-config/fast-test-00/providers/0-bootstrap-providers.tf ./ +ln -s /home/user/fast-configs/test-0/providers/0-bootstrap-providers.tf ./ -# conventional place for stage tfvars (manually created) -ln -s ~/fast-config/fast-test-00/0-bootstrap.auto.tfvars ./ +# conventional location for this stage terraform.tfvars (manually managed) +ln -s /home/user/fast-configs/test-0/0-bootstrap.auto.tfvars ./ ``` +If you did not configure local output files use the GCS bucket to fetch output files. The bucket name can be derived from the `tfvars.bootstrap.automation.outputs_bucket` Terraform output. Adjust the path, run the script, then copy/paste the resulting commands. + ```bash -../fast-links.sh gs://xxx-prod-iac-core-outputs-0 - -# File linking commands for organization bootstrap stage +../fast-links.sh gs://test0-prod-iac-core-0-iac-outputs +# File linking commands for FAST Bootstrap. stage # provider file -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/0-bootstrap-providers.tf ./ +gcloud storage cp gs://test0-prod-iac-core-0-iac-outputs/providers/0-bootstrap-providers.tf ./ -# conventional place for stage tfvars (manually created) -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/0-bootstrap.auto.tfvars ./ +# conventional location for this stage terraform.tfvars (manually managed) +gcloud storage cp gs://test0-prod-iac-core-0-iac-outputs/0-bootstrap.auto.tfvars ./ ``` -- important for CI/CD -The `0-bootstrap.auto.tfvars` file is a crucial component of the CI/CD pipeline and must be manually created. This file is essentially the `terraform.tfvars` file renamed to avoid being ignored in version control systems like GitHub or GitLab, where `terraform.tfvars` is often included in `.gitignore`. By renaming it and committing `0-bootstrap.auto.tfvars` to your source control, you ensure that the necessary configurations are available in the pipeline. - -Copy/paste the command returned by the script to link or copy the provider file, then migrate state with `terraform init` and run `terraform apply`. If your organization was created with "Secure by Default Org Policy", that is with some of the org policies enabled, add `-var 'org_policies_config={"import_defaults": true}'` to `terraform apply`: +Once the provider file has been setup, migrate local state to the GCS backend and re-run apply. ```bash terraform init -migrate-state terraform apply ``` -or +## Default factory datasets + +A few example datasets are included with the stage, each implementing a different widely used organizational design. The datasets can be used as-is, potentially with slight changes to better suit specific use cases, or they can serve as a starting point to implement radically different approaches. + +### "Classic FAST" dataset + +This dataset implements a Classic FAST design that replicates legacy bootstrap and resource management stages. The resulting layout is easy to customize, and supports VPC SC, networking, security and potentially any FAST stage 3 directly as explained in a [later section](#leveraging-classic-fast-stages). + +The organizational layout mirrors the consolidated FAST one, where shared infrastructure (stage 2 and 3) is partitioned via folders at the top, and further subdivided in environment-level folders for data or fleet management (Stage 3). An example "Teams" folder allows hooking up an application-level project factory as a separate stage, which is then used to define per-team subdivisions and create projects. + +

+ Classic FAST organization-level diagram. +

+ +### "Minimal" dataset + +This dataset is meant as a minimalistic starting point for organizations where a security baseline and a project factory are all that's needed, at least initially. The design can then organically grow to support more functionality, converging to the Classic or other types of layouts. + +### "Tenants" dataset + +TBD + +## Detailed configuration + +The following sections explain how to configure and run this stage, and should be read in sequence when using it for the first time. + +### Factory data + +The resources created by this stage are controlled by several factories, which point to YAML configuration files and folders. Data locations for each factory are controlled via the `var.factories_config` variable, and each factory path can be overridden individually. + +The default paths point to the dataset in the `data` folder which deploys a FAST-compliant configuration. These are the available factories in this stage, with file-level factories based on a single YAML file, and folder-level factories based on sets of YAML files contained withing a filesystem folder: + +- **defaults** (`data/defaults.yaml`) \ + file-level factory to define stage defaults (organization id, locations, prefix, etc.) and static context mappings +- **billing_accounts** (`data/billing-accounts`) \ + folder-level factory where each YAML file defines billing-account level IAM for one billing account; only used for externally managed accounts +- **organization** (`data/organization/.config.yaml`) \ + file-level factory to define organization IAM and log sinks + - **custom roles** (`data/organization/custom-roles`) \ + folder-level factory to define organization-level custom roles + - **org policies** (`data/organization/org-policies`) \ + folder-level factory to define organization-level org policies + - **tags** (`data/organization/tags`) \ + folder-level factory to define organization-level resource management tags +- **folders** (`data/folders`) \ + folder-level factory to define the resource management hierarchy and individual folder attributes (IAM, org policies, tag bindings, etc.); also supports defining folder-level IaC resources +- **projects** (`data/projects`) \ + folder-level factory to define projects and their attributes (projejct factory) +- **cicd** (`data/cicd.yaml`) \ + file-level factory to define CI/CD configurations for this and subsequent stages + +### Defaults configuration + +The prerequisite configuration for this stage is done via a `defaults.yaml` file, which implements part or all of the [relevant JSON schema](./schemas/defaults.schema.json). The location of the file defaults to `data/defaults.yaml` but can be easily changed via the `factories_config.defaults` variable. + +This is a commented example of a defaults file, showing a minimal working configuration. Refer to the YAML schema for all available options. + +```yaml +# global defaults used by bootstrap and persisted in the globals output file +global: + # billing account also set as default in the internal project factory + billing_account: 123456-123456-123456 + # default locations for this stage resources + locations: + bigquery: europe-west1 + logging: europe-west1 + # organization attributes (id is required) + organization: + domain: example.org + id: 1234567890 + customer_id: ABC0123CDE +# project defaults and overrides used by the internal project factory +projects: + defaults: + # setting a prefix either here or in overrides is required + prefix: foo-1 + # default location for storage buckets + storage_location: europe-west1 + overrides: {} +# FAST output files generated by this stage +output_files: + # optional path for locally persisted output files + local_path: ~/fast-config/foo-1 + # required storage bucket for output files (supports context interpolation) + storage_bucket: $storage_buckets:iac-0/iac-outputs + # FAST stage provider files (supports context interpolation) + providers: + 0-bootstrap: + bucket: $storage_buckets:iac-0/iac-bootstrap-state + service_account: $iam_principals:service_accounts/iac-0/iac-bootstrap-rw + # [...] +# static values added to context interpolation tables and used in factories +context: + iam_principals: + gcp-organization-admins: group:fabric-fast-owners@example.com +``` + +### Billing account IAM + +FAST traditionally supports three different billing configurations: + +- billing account in the same organization, where billing IAM is set via organization-level bindings +- external billing account, where billing IAM is set via account-level bindings +- no billing IAM, where FAST assumes bindings are managed by some externally defined process + +This stage allows the same flexibility, and even makes it possible to mix and match approaches by making billing IAM explicit: + +- if billing-account level IAM bindings are needed, they can be set via the billing account factory +- if organization-level IAM bindings are needed, they can be set via the organization factory +- if no billing IAM can be managed here, it's enough to disable the billing account factory by pointing it to an empty or non-existent filesystem folder + +The default dataset assumes an externally managed billing account is used, and configures its IAM accordingly via the billing account factory. The example below shows some of the IAM bindings configured at the billing account level, and how context-based interpolation is used there. + +
+Context-based replacement examples for the billing acccounts factory + +#### Context-based replacement in the billing account factory + +Principal expansion leverages the `$iam_principals:` context, which is populated from the static mappings defined in defaults, and the service accounts generated via the internal project factory [described in a later section](#project-factory). Log sink definitions also support `$project_ids:` and `$storage_buckets` expansions. + +```yaml +# example billing account factory file +# file: billing-accounts/default.yaml +id: $defaults:billing_account +iam_bindings_additive: + billing_admin_org_admins: + role: roles/billing.admin + # statically defined principal (via defaults.yaml) + member: $iam_principals:gcp-organization-admins + billing_admin_bootstrap_sa: + role: roles/billing.admin + # internally managed principal (project factory service account) + member: $iam_principals:service_accounts/iac-0/iac-bootstrap-rw +logging_sinks: + test: + description: Test sink + destination: $project_ids:log-0 + type: project +``` + +
+ +### Organization configuration + +The default dataset implements a classic FAST design, re-creating the required custom roles, IAM bindings, org policies, tags, and log sinks via the factories described in a previous section. + +Compared to classic FAST this approach makes org-level configuration explicit, allowing easy customization of IAM and all other attributes. Before running this stage, check that the data files match your expected design. + +Context-based interpolation is heavily used in the organization configuration files to refer to external or project-level resources, so as to make the factory files portable. Some examples are provided below to better illustrate usage and facilitate editing organization-level data. + +
+Context-based replacement examples for organization factories + +#### Context-based replacement in organization factories + +Principal expansion leverages the `$iam_principals:` context, which is populated from the static mappings defined in defaults, and the service accounts generated via the internal project factory [described in a later section](#project-factory). + +```yaml +# example principal-level context interpolation +# file: data/organization/.config.yaml +iam_by_principals: + # statically defined principal (via defaults.yaml) + $iam_principals:gcp-organization-admins: + - roles/cloudasset.owner + - roles/cloudsupport.admin + - roles/compute.osAdminLogin + # [...] + # internally managed principal (project factory service account) + $iam_principals:service_accounts/iac-0/iac-bootstrap-rw: + - roles/accesscontextmanager.policyAdmin + - roles/cloudasset.viewer + - roles/essentialcontacts.admin + # [...] +``` + +Log sinks can refer to project-level destination via different contexts. + +```yaml +# example log sinks showing different destination contexts +# file: data/organization/.config.yaml +logging: + storage_location: $locations:default + sinks: + # log bucket destination + audit-logs: + destination: $log_buckets:log-0/audit-logs + filter: | + log_id("cloudaudit.googleapis.com/activity") OR + log_id("cloudaudit.googleapis.com/system_event") OR + log_id("cloudaudit.googleapis.com/policy") OR + log_id("cloudaudit.googleapis.com/access_transparency") + # storage bucket destination + iam: + destination: $storage_buckets:log-0/iam-sink + filter: | + protoPayload.serviceName="iamcredentials.googleapis.com" OR + protoPayload.serviceName="iam.googleapis.com" OR + protoPayload.serviceName="sts.googleapis.com" + # project destination + vpc-sc: + destination: $projject_ids:log-0 + filter: | + protoPayload.metadata.@type="type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata" +``` + +Context-based expansion is not limited to the organization's `.config.yaml` file, but is also available in the other factories, like in this example for the organization-level tag factory. + +```yaml +# example usage of context interpolation in tag values IAM +# file: data/organization/tags/environment.yaml +description: "Organization-level environments." +values: + development: + description: "Development." + iam: + "roles/resourcemanager.tagUser": + - $iam_principals:service_accounts/iac-0/iac-networking-rw + - $iam_principals:service_accounts/iac-0/iac-security-rw + - $iam_principals:service_accounts/iac-0/iac-pf-rw + "roles/resourcemanager.tagViewer": + - $iam_principals:service_accounts/iac-0/iac-networking-ro + - $iam_principals:service_accounts/iac-0/iac-security-ro + - $iam_principals:service_accounts/iac-0/iac-pf-ro + # [...] +``` + +An exception to the namespaced-based context replacements is in IAM conditions, where Terraform limitations force use of native string templating, as in the example below. + +```yaml +iam_bindings: + pf_org_policy_admin: + role: roles/orgpolicy.policyAdmin + members: + - $iam_principals:service_accounts/iac-0/iac-pf-rw + condition: + # $organization is set as a string template variable by the module + expression: resource.matchTag('${organization}/context', 'project-factory') + title: Project factory org policy admin +``` + +
+ +### Resource management hierarchy + +The folder hierarchy is managed via a filesystem tree of YAML configuration files, and leverages the [project factory module](../../../modules/project-factory/README.md#folder-hierarchy) implementation, which supports up to 3 levels of folders (4 or more can be easily implemented in the module if needed). The module documentation provides additional information on this factory usage and formats. + +The default dataset implements a classic FAST layout, with top-level folders for stage 2 and stage 3, and can be easily tweaked by adding or removing any needed folder. ```bash -terraform init -migrate-state -terraform apply -var 'org_policies_config={"import_defaults": true}' +data/folders +├── networking +│   ├── .config.yaml +│   ├── dev +│   │   └── .config.yaml +│   └── prod +│   └── .config.yaml +├── security +│   └── .config.yaml +└── teams + └── .config.yaml ``` -if there default policies are enabled. +Different layouts are very easy to implement by simply modeling the desired hierarchy in the filesystem, and configuring each folder via `.config.yaml` files. -Make sure the user you're logged in with is a member of the `gcp-organization-admins` group or impersonation will not be possible. +The project factory also supports embedding folder-aware project definitions in folders, but that approach is best used with caution to prevent potential race conditions when moving or deleting folders and projects. -## Customizations +As with the factories described above, context replacements can be used in folder configurations. Some examples are provided below. -Most variables (e.g. `billing_account` and `organization`) are only used to input actual values and should be self-explanatory. The only meaningful customizations that apply here are groups, and IAM roles. +
+Context-based replacement examples for the folder factory -### Group names +#### Context-based replacement in the folders factory -As we mentioned above, groups reflect the convention used in the [GCP Enterprise Setup Checklist](https://cloud.google.com/docs/enterprise/setup-checklist), with an added level of indirection: the `groups` variable maps logical names to actual names, so that you don't need to delve into the code if your group names do not comply with the checklist convention. +As with other examples before, the main use case is to infer IAM principals from either the static or internally defined context. One additional context which is often useful here is tag values, which allows defining a scope for organization-level conditional IAM bindings or org policies. -For example, if your network admins team is called `net-rockstars@example.com`, simply set that name in the variable, minus the domain which is interpolated internally with the organization domain: - -```hcl -variable "groups" { - description = "Group names to grant organization-level permissions." - type = map(string) - default = { - gcp-network-admins = "net-rockstars" +```yaml +# file: data/folders/teams/.config.yaml +name: Teams +iam_by_principals: + $iam_principals:service_accounts/iac-0/iac-pf-rw: + - roles/owner + - roles/resourcemanager.folderAdmin # [...] - } -} -# tftest skip +tag_bindings: + context: $tag_values:context/project-factory ``` -If your groups layout differs substantially from the checklist, define all relevant groups in the `groups` variable, then rearrange IAM roles in the code to match your setup. +
-### IAM +### Project factory -One other area where we directly support customizations is IAM. The code here, as in all stages, follows a simple pattern derived from best practices: +The project factory is managed via a set of YAML configuration files, which like folders leverages the [project factory module](../../../modules/project-factory/README.md#folder-hierarchy) implementation. The module documentation provides additional information on this factory usage and formats. -- operational roles for humans are assigned to groups -- any other principal is a service account +The default dataset implements a classic FAST layout, with two top-level projects for log exports and IaC resources. Those projects can easily be changed, for example rooting them in a folder by specifying the folder id or context name in their `parent` attribute. -In code, the distinction above reflects on how IAM bindings are specified in the underlying module variables: +The provided project configurations also create several key resources for the stage like log buckets, storage buckets, and service accounts. Context-based expansions for projects are very similar to the ones defined for folders, you can refer to the above section for details. -- group roles "for humans" always use `iam_by_principals` variables -- service account roles always use `iam` variables +### CI/CD configuration -This makes it easy to tweak user roles by adding mappings to the `iam_by_principals` variables of the relevant resources, without having to understand and deal with the details of service account roles. +CI/CD support is implemented in a similar way to classic/legacy FAST, except for being driven by a factory tha points to a single file. -One more critical difference in IAM bindings is between authoritative and additive: +This allows defining a single Workload Identity provider that will be used to exchange external tokens for the pipelines, and one or more workflows that can interpolate internal (from the project factory) or external (user defined) attributes. -- authoritative bindings have complete control on principals for a given role; this is the recommended best practice when a single automation actor controls the role, as it removes drift each time Terraform runs -- additive bindings have control only on given role/principal pairs, and need to be used whenever multiple automation actors need to control the role, as is the case for the network user role in Shared VPC setups, and many other situations +This is the default file which implements a workflow for this stage. To enable it, pass the file path to the `factories_config.cicd` variable. -This stage groups all IAM definitions in the [organization-iam.tf](./organization-iam.tf) file, to allow easy parsing of roles assigned to each group and machine identity. - -When customizations are needed, three stage-level variables allow injecting additional bindings to match the desired setup: - -- `iam_by_principals` allows adding authoritative bindings for groups -- `iam` allows adding authoritative bindings for any type of supported principal, and is merged with the internal `iam` local and then with group bindings at the module level -- `iam_bindings_additive` allows adding individual role/member pairs, and also supports IAM conditions - -Refer to the [project module](../../../modules/project/) for examples on how to use the IAM variables, and they are an interface shared across all our modules. - -### Log sinks and log destinations - -You can customize organization-level logs through the `log_sinks` variable in two ways: - -- creating additional log sinks to capture more logs -- changing the destination of captured logs - -By default, all logs are exported to a log bucket, but FAST can create sinks to BigQuery, GCS, or PubSub. - -If you need to capture additional logs, please refer to GCP's documentation on [scenarios for exporting logging data](https://cloud.google.com/architecture/exporting-stackdriver-logging-for-security-and-access-analytics), where you can find ready-made filter expressions for different use cases. - -When using Pubsub or BigQuery destinations, make sure the read-only stage service account (`prefix-prod-bootstrap-0r@prefix-prod-iac-core-0.iam.gserviceaccount.com`) has the necessary permissions to view destination resources. You can add them manually via the authoritative `iam` or the additive `iam_bindings_additive` variables. Refer to issue [#2540](https://github.com/GoogleCloudPlatform/cloud-foundation-fabric/issues/2540) for a discussion on this topic, and simple commands to verify proper permissions have been added. - -### Names and naming convention - -Configuring the individual tokens for the naming convention described above, has varying degrees of complexity: - -- the static prefix can be set via the `prefix` variable once -- the environment identifier is set to `prod` as resources here influence production and are considered as such, and can be changed in `main.tf` locals - -All other tokens are set directly in resource names, as providing abstractions to manage them would have added too much complexity to the code, making it less readable and more fragile. - -If a different convention is needed, identify names via search/grep (e.g. with `^\s+name\s+=\s+"`) and change them in an editor: it should take a couple of minutes at most, as there's just a handful of modules and resources to change. - -Names used in internal references (e.g. `module.foo-prod.id`) are only used by Terraform and do not influence resource naming, so they are best left untouched to avoid having to debug complex errors. - -### Workload Identity Federation - -At any time during this stage's lifecycle you can configure a Workload Identity Federation pool, and one or more providers. These are part of this stage's interface, included in the automatically generated `.tfvars` files and accepted by the Resource Managent stage that follows. - -The variable maps each provider's `issuer` attribute with the definitions in the `identity-providers-wlif.tf` file. We currently support GitHub and Gitlab directly, and extending to definitions to support more providers is trivial (send us a PR if you do!). - -Provider key names are used by the `cicd_repositories` variable to configure authentication for CI/CD repositories, and generally from your Terraform code whenever you need to configure IAM access or impersonation for federated identities. - -This is a sample configuration of a GitHub and a Gitlab provider. Every parameter is optional. - -The `custom_settings` attributes are used to configure the provider to work with privately managed installations of Github and Gitlab: - -- `issuer_uri` (defaults to the public platforms one if not set) -- `audience` (defaults to the public URL of the provider if not set, as recommended in the [WIF FAQ section](https://cloud.google.com/iam/docs/best-practices-for-using-workload-identity-federation#provider-audience)) -- `jwks_json` for public key upload - -```tfvars -workload_identity_providers = { - # Use the public GitHub and specify an attribute condition - github-public-sample = { - attribute_condition = "attribute.repository_owner==\"my-github-org\"" - issuer = "github" - } - # Use a private instance of Gitlab and specify a custom issuer_uri - gitlab-private-sample = { - issuer = "gitlab" - custom_settings = { - issuer_uri = "https://gitlab.fast.example.com" - } - } - # Use a private instance of Gitlab. - # Specify a custom audience and a custom issuer_uri - gitlab-private-aud-sample = { - attribute_condition = "attribute.namespace_path==\"my-gitlab-org\"" - issuer = "gitlab" - custom_settings = { - audiences = ["https://gitlab.fast.example.com"] - issuer_uri = "https://gitlab.fast.example.com" - } - } -} +```yaml +workload_identity_federation: + pool_name: iac-0 + project: $project_ids:iac-0 + providers: + github: + # the condition is optional but recommented, use your GitHub org name + attribute_condition: attribute.repository_owner=="my_org" + issuer: github + # custom_settings: + # issuer_uri: + # audiences: [] + # jwks_json_path: +workflows: + bootstrap: + template: github + workload_identity_provider: + id: $wif_providers:github + audiences: [] + repository: + name: bootstrap + branch: main + output_files: + storage_bucket: $storage_buckets:iac-0/iac-outputs + providers: + apply: $output_files:providers/0-bootstrap + plan: $output_files:providers/0-bootstrap-ro + files: + - tfvars/0-boostrap.auto.tfvars.json + service_accounts: + apply: $iam_principals:service_accounts/iac-0/iac-bootstrap-cicd-rw + plan: $iam_principals:service_accounts/iac-0/iac-bootstrap-cicd-ro ``` -### Project folders +## Leveraging classic FAST Stages -By default this stage creates all its projects directly under the orgaization node. If desired, projects can be moved under a folder using the `project_parent_ids` variable. +Classic Fast stage 2 and 3 can be directly used after applying this if the [Classic FAST layout](#classic-fast-dataset) is used, or similar identities and permissions are implemented in a different design. -```tfvars -project_parent_ids = { - automation = "folders/1234567890" - billing = "folders/9876543210" - logging = "folders/1234567890" -} +Specific changes or considerations needed for each stage are described below. + +### VPC Service Controls + +To use the predefined logging ingress policy in the VPC SC stage, define it like in the following example. + +```yaml +from: + access_levels: + - "*" + identities: + - $identity_sets:logging_identities +to: + operations: + - service_name: "*" + resources: + - $project_numbers:log-0 ``` -### CI/CD repositories +### Security -FAST is designed to directly support running in automated workflows from separate repositories for each stage. The `cicd_repositories` variable allows you to configure impersonation from external repositories leveraging Workload identity Federation, and pre-configures a FAST workflow file that can be used to validate and apply the code in each repository. +Define values for the `var.environments` variable in a tfvars file. -The repository design we support is fairly simple, with a repository for modules that enables centralization and versioning, and one repository for each stage optionally configured from the previous stage. - -This is an example of configuring the bootstrap and resource management repositories in this stage. CI/CD configuration is optional, so the entire variable or any of its attributes can be set to null if not needed. - -```tfvars -cicd_config = { - bootstrap = { - identity_provider = "github-sample" - repository = { - branch = null - name = "my-gh-org/fast-bootstrap" - type = "github" - } - } - resman = { - identity_provider = "github-sample" - repository = { - branch = "main" - name = "my-gh-org/fast-resman" - type = "github" - } - } -} -``` - -The `type` attribute can be set to one of the supported repository types: `github` or `gitlab`. - -Once the stage is applied the generated output files will contain pre-configured workflow files for each repository, that will use Workload Identity Federation via a dedicated service account for each repository to impersonate the automation service account for the stage. - -You can use Terraform to automate creation of the repositories using the extra stage defined in [fast/extras/0-cicd-github](../../extras/0-cicd-github/) (only for Github for now). - -The remaining configuration is manual, as it regards the repositories themselves: - -- create a repository for modules - - clone and populate it with the Fabric modules - - configure authentication to the modules repository - - for GitHub - - create a key pair - - create a [deploy key](https://docs.github.com/en/developers/overview/managing-deploy-keys#deploy-keys) in the modules repository with the public key - - create a `CICD_MODULES_KEY` secret with the private key in each of the repositories that need to access modules (for Gitlab, please Base64 encode the private key for masking) - - for Gitlab - - TODO - - for Source Repositories - - assign the reader role to the CI/CD service accounts -- create one repository for each stage - - do an initial apply cycle for the stage so that state exists - - clone and populate them with the stage source - - edit the modules source to match your modules repository - - a simple way is using the "Replace in files" function of your editor - - search for `source\s*= "../../../modules/([^"]+)"` - - replace with: - - modules stored on GitHub: `source = "git@github.com:my-org/fast-modules.git//$1?ref=v1.0"` - - modules stored on Gitlab: `source = "git::ssh://git@gitlab.com/my-org/fast-modules.git//$1?ref=v1.0"` - - modules stored on Source Repositories: `"source = git::https://source.developers.google.com/p/my-project/r/my-repository//$1?ref=v1.0"`. You may need to run `git config --global credential.'https://source.developers.google.com'.helper gcloud.sh` first as documented [here](https://cloud.google.com/source-repositories/docs/adding-repositories-as-remotes#add_the_repository_as_a_remote) - - copy the generated workflow file for the stage from the GCS output files bucket or from the local clone if enabled - - for GitHub, place it in a `.github/workflows` folder in the repository root - - for Gitlab, rename it to `.gitlab-ci.yml` and place it in the repository root - - for Source Repositories, place it in `.cloudbuild/workflow.yaml` - - To prevent the creation of local files in the CI/CD pipeline, comment out the `outputs_location` line in the `terraform.tfvars` file by adding a `#` at the beginning, like so: `# outputs_location = "~/fast-config"`. This configuration is only necessary for the initial local deployments and should not be used in the CI/CD environment. - -### Add-ons - -FAST defines a simple mechanism to extend stage functionality via the use of [add-ons](../../addons/). Configuration for stage 1 add-ons happens here via the `fast_addon` variable. Refer to the add-ons documentation for more details on their use. - - + ## Files | name | description | modules | resources | |---|---|---|---| -| [automation.tf](./automation.tf) | Automation project and resources. | gcs · iam-service-account · project | | -| [billing.tf](./billing.tf) | Billing export project and dataset. | bigquery-dataset · billing-account · logging-bucket · project | | -| [cicd.tf](./cicd.tf) | CI/CD locals and resources. | iam-service-account | | -| [identity-providers-wfif-defs.tf](./identity-providers-wfif-defs.tf) | Workforce Identity provider definitions. | | | -| [identity-providers-wfif.tf](./identity-providers-wfif.tf) | Workforce Identity Federation provider definitions. | | google_iam_workforce_pool · google_iam_workforce_pool_provider | -| [identity-providers-wlif-defs.tf](./identity-providers-wlif-defs.tf) | Workload Identity provider definitions. | | | -| [identity-providers-wlif.tf](./identity-providers-wlif.tf) | Workload Identity Federation provider definitions. | | google_iam_workload_identity_pool · google_iam_workload_identity_pool_provider | -| [log-export.tf](./log-export.tf) | Audit log project and sink. | bigquery-dataset · gcs · logging-bucket · project · pubsub | | -| [main.tf](./main.tf) | Module-level locals and resources. | | | -| [organization-iam.tf](./organization-iam.tf) | Organization-level IAM bindings locals. | | | -| [organization.tf](./organization.tf) | Organization-level IAM. | organization | | -| [outputs-files.tf](./outputs-files.tf) | Output files persistence to local filesystem. | | local_file | -| [outputs-gcs.tf](./outputs-gcs.tf) | Output files persistence to automation GCS bucket. | | google_storage_bucket_object | -| [outputs-providers.tf](./outputs-providers.tf) | Locals for provider output files. | | | +| [billing.tf](./billing.tf) | None | billing-account | | +| [cicd.tf](./cicd.tf) | None | | google_iam_workload_identity_pool · google_iam_workload_identity_pool_provider · google_storage_bucket_object · local_file | +| [factory.tf](./factory.tf) | None | project-factory | | +| [imports.tf](./imports.tf) | None | | | +| [main.tf](./main.tf) | Module-level locals and resources. | | terraform_data | +| [organization.tf](./organization.tf) | None | organization | | +| [output-files.tf](./output-files.tf) | None | | google_storage_bucket_object · local_file | | [outputs.tf](./outputs.tf) | Module outputs. | | | -| [variables-addons.tf](./variables-addons.tf) | None | | | | [variables.tf](./variables.tf) | Module variables. | | | +| [wif-definitions.tf](./wif-definitions.tf) | Workload Identity provider definitions. | | | ## Variables -| name | description | type | required | default | producer | -|---|---|:---:|:---:|:---:|:---:| -| [billing_account](variables.tf#L17) | Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`. | object({…}) | ✓ | | | -| [organization](variables.tf#L282) | Organization details. | object({…}) | ✓ | | | -| [prefix](variables.tf#L297) | Prefix used for resources that need unique names. Use 9 characters or less. | string | ✓ | | | -| [bootstrap_user](variables.tf#L39) | Email of the nominal user running this stage for the first time. | string | | null | | -| [cicd_config](variables.tf#L45) | CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed. | object({…}) | | {} | | -| [custom_roles](variables.tf#L86) | Map of role names => list of permissions to additionally create at the organization level. | map(list(string)) | | {} | | -| [environments](variables.tf#L93) | Environment names. When not defined, short name is set to the key and tag name to lower(name). | map(object({…})) | | {…} | | -| [essential_contacts](variables.tf#L133) | Email used for essential contacts, unset if null. | string | | null | | -| [factories_config](variables.tf#L139) | Configuration for the resource factories or external data. | object({…}) | | {} | | -| [fast_addon](variables-addons.tf#L17) | FAST addons configurations for stages 1. Keys are used as short names for the add-on resources. | map(object({…})) | | {} | | -| [groups](variables.tf#L151) | Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated. | object({…}) | | {} | | -| [iam](variables.tf#L168) | Organization-level custom IAM settings in role => [principal] format. | map(list(string)) | | {} | | -| [iam_bindings_additive](variables.tf#L175) | Organization-level custom additive IAM bindings. Keys are arbitrary. | map(object({…})) | | {} | | -| [iam_by_principals](variables.tf#L190) | Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the `iam` variable. | map(list(string)) | | {} | | -| [locations](variables.tf#L197) | Optional locations for GCS, BigQuery, and logging buckets created here. | object({…}) | | {} | | -| [log_sinks](variables.tf#L211) | Org-level log sinks, in name => {type, filter} format. | map(object({…})) | | {…} | | -| [org_policies_config](variables.tf#L267) | Organization policies customization. | object({…}) | | {} | | -| [outputs_location](variables.tf#L291) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | string | | null | | -| [project_parent_ids](variables.tf#L306) | Optional parents for projects created here in folders/nnnnnnn format. Null values will use the organization as parent. | object({…}) | | {} | | -| [resource_names](variables.tf#L317) | Resource names overrides for specific resources. Prefix is always set via code, except where noted in the variable type. | object({…}) | | {} | | -| [universe](variables.tf#L349) | Target GCP universe. | object({…}) | | null | | -| [workforce_identity_providers](variables.tf#L359) | Workforce Identity Federation pools. | map(object({…})) | | {} | | -| [workload_identity_providers](variables.tf#L375) | Workload Identity Federation pools. The `cicd_repositories` variable references keys here. | map(object({…})) | | {} | | +| name | description | type | required | default | +|---|---|:---:|:---:|:---:| +| [bootstrap_user](variables.tf#L17) | Email of the nominal user running this stage for the first time. | string | | null | +| [context](variables.tf#L23) | Context-specific interpolations. | object({…}) | | {} | +| [factories_config](variables.tf#L43) | Configuration for the resource factories or external data. | object({…}) | | {} | +| [org_policies_imports](variables.tf#L57) | List of org policies to import. These need to also be defined in data files. | list(string) | | [] | ## Outputs -| name | description | sensitive | consumers | -|---|---|:---:|---| -| [automation](outputs.tf#L113) | Automation resources. | | | -| [billing_dataset](outputs.tf#L118) | BigQuery dataset prepared for billing export. | | | -| [cicd_repositories](outputs.tf#L123) | CI/CD repository configurations. | | | -| [custom_roles](outputs.tf#L135) | Organization-level custom roles. | | | -| [outputs_bucket](outputs.tf#L140) | GCS bucket where generated output files are stored. | | | -| [project_ids](outputs.tf#L145) | Projects created by this stage. | | | -| [providers](outputs.tf#L155) | Terraform provider files for this stage and dependent stages. | ✓ | stage-01 | -| [service_accounts](outputs.tf#L162) | Automation service accounts created by this stage. | | | -| [tfvars](outputs.tf#L171) | Terraform variable files for the following stages. | ✓ | | -| [tfvars_globals](outputs.tf#L177) | Terraform Globals variable files for the following stages. | ✓ | | -| [workforce_identity_pool](outputs.tf#L183) | Workforce Identity Federation pool. | | | -| [workload_identity_pool](outputs.tf#L192) | Workload Identity Federation pool and providers. | | | +| name | description | sensitive | +|---|---|:---:| +| [iam_principals](outputs.tf#L17) | IAM principals. | | +| [locations](outputs.tf#L22) | Default locations. | | +| [projects](outputs.tf#L27) | Attributes for managed projects. | | +| [tfvars](outputs.tf#L32) | Stage tfvars. | | diff --git a/fast/stages/0-bootstrap-experimental/WORKLOG.md b/fast/stages/0-bootstrap/WORKLOG.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/WORKLOG.md rename to fast/stages/0-bootstrap/WORKLOG.md diff --git a/fast/stages/0-bootstrap-experimental/assets/providers.tf.tpl b/fast/stages/0-bootstrap/assets/providers.tf.tpl similarity index 100% rename from fast/stages/0-bootstrap-experimental/assets/providers.tf.tpl rename to fast/stages/0-bootstrap/assets/providers.tf.tpl diff --git a/fast/stages/0-bootstrap/templates/workflow-github.yaml b/fast/stages/0-bootstrap/assets/workflow-github.yaml similarity index 100% rename from fast/stages/0-bootstrap/templates/workflow-github.yaml rename to fast/stages/0-bootstrap/assets/workflow-github.yaml diff --git a/fast/stages/0-bootstrap-experimental/assets/workflow-gitlab.yaml b/fast/stages/0-bootstrap/assets/workflow-gitlab.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/assets/workflow-gitlab.yaml rename to fast/stages/0-bootstrap/assets/workflow-gitlab.yaml diff --git a/fast/stages/0-bootstrap/billing.tf b/fast/stages/0-bootstrap/billing.tf index 941e5c741..1e098327f 100644 --- a/fast/stages/0-bootstrap/billing.tf +++ b/fast/stages/0-bootstrap/billing.tf @@ -14,114 +14,60 @@ * limitations under the License. */ -# tfdoc:file:description Billing export project and dataset. - locals { - billing_mode = ( - var.billing_account.no_iam - ? null - : var.billing_account.is_org_level ? "org" : "resource" + _billing_accounts_path = try( + pathexpand(var.factories_config.billing_accounts), null ) - - _billing_iam_bindings = { - "roles/billing.admin" = [ - local.principals.gcp-billing-admins, - local.principals.gcp-organization-admins, - module.automation-tf-bootstrap-sa.iam_email, - module.automation-tf-resman-sa.iam_email - ], - "roles/billing.viewer" = [ - module.automation-tf-bootstrap-r-sa.iam_email, - module.automation-tf-resman-r-sa.iam_email - ], - "roles/logging.configWriter" = local.billing_mode == "org" || !var.billing_account.force_create.log_bucket ? [] : [ - module.automation-tf-bootstrap-sa.iam_email - ] + _billing_accounts_raw = { + for f in try(fileset(local._billing_accounts_path, "*.yaml"), []) : + trimsuffix(f, ".yaml") => merge( + { id = null }, + yamldecode(file("${local._billing_accounts_path}/${f}")) + ) } - - _billing_iam_bindings_add = flatten([for role, bindings in local._billing_iam_bindings : [ - for member in bindings : { - member = member, - role = role - } - ]]) - - billing_iam_bindings_additive = { - for b in local._billing_iam_bindings_add : "${b.role}-${b.member}" => { - member = b.member - role = b.role - } + billing_accounts = { + for k, v in local._billing_accounts_raw : k => merge(v, { + id = ( + local.defaults.billing_account != null && v.id == "$defaults:billing_account" + ? local.defaults.billing_account + : v.id + ) + logging_sinks = lookup(v, "logging_sinks", {}) + }) if v.id != null } } -# billing account in same org (IAM is in the organization.tf file) - -module "billing-export-project" { - source = "../../../modules/project" - count = ( - local.billing_mode == "org" || var.billing_account.force_create.project == true ? 1 : 0 - ) - billing_account = var.billing_account.id - name = var.resource_names["project-billing"] - parent = coalesce( - var.project_parent_ids.billing, "organizations/${var.organization.id}" - ) - prefix = var.prefix - universe = var.universe - contacts = ( - var.bootstrap_user != null || var.essential_contacts == null - ? {} - : { (var.essential_contacts) = ["ALL"] } - ) - iam = { - "roles/owner" = [module.automation-tf-bootstrap-sa.iam_email] - "roles/viewer" = [module.automation-tf-bootstrap-r-sa.iam_email] - } - services = [ - # "cloudresourcemanager.googleapis.com", - # "iam.googleapis.com", - # "serviceusage.googleapis.com", - "bigquery.googleapis.com", - "bigquerydatatransfer.googleapis.com", - "storage.googleapis.com" - ] -} - -module "billing-export-dataset" { - source = "../../../modules/bigquery-dataset" - count = ( - local.billing_mode == "org" || var.billing_account.force_create.dataset == true ? 1 : 0 - ) - project_id = module.billing-export-project[0].project_id - id = var.resource_names["bq-billing"] - friendly_name = "Billing export." - location = local.locations.bq -} - -# standalone billing account - -module "billing-account-logbucket" { - source = "../../../modules/logging-bucket" - count = local.billing_mode == "resource" && var.billing_account.force_create.log_bucket ? 1 : 0 - parent_type = "project" - parent = module.log-export-project.project_id - name = "billing-account" - location = local.locations.logging - log_analytics = { enable = true } - # org-level logging settings ready before we create any logging buckets - depends_on = [module.organization-logging] -} - -module "billing-account" { - source = "../../../modules/billing-account" - count = local.billing_mode == "resource" ? 1 : 0 - id = var.billing_account.id - iam_bindings_additive = local.billing_iam_bindings_additive - logging_sinks = !var.billing_account.force_create.log_bucket ? {} : { - billing_bucket_log_sink = { - destination = module.billing-account-logbucket[0].id - type = "logging" - description = "billing-account sink (Terraform-managed)." - } +module "billing-accounts" { + source = "../../../modules/billing-account" + for_each = local.billing_accounts + id = each.value.id + context = merge(local.ctx, { + custom_roles = merge( + local.ctx.custom_roles, module.organization[0].custom_role_id + ) + iam_principals = merge( + local.ctx.iam_principals, + module.factory.iam_principals + ) + project_ids = merge( + local.ctx.project_ids, module.factory.project_ids + ) + storage_buckets = module.factory.storage_buckets + tag_keys = merge( + local.ctx.tag_keys, + local.org_tag_keys + ) + tag_values = merge( + local.ctx.tag_values, + local.org_tag_values + ) + }) + iam = lookup(each.value, "iam", {}) + iam_by_principals = lookup(each.value, "iam_by_principals", {}) + iam_bindings = lookup(each.value, "iam_bindings", {}) + iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) + logging_sinks = { + for k, v in each.value.logging_sinks : k => v + if lookup(v, "destination", null) != null && lookup(v, "type", null) != null } } diff --git a/fast/stages/0-bootstrap/cicd.tf b/fast/stages/0-bootstrap/cicd.tf index 1c2771c45..6e329b7b7 100644 --- a/fast/stages/0-bootstrap/cicd.tf +++ b/fast/stages/0-bootstrap/cicd.tf @@ -1,5 +1,5 @@ /** - * Copyright 2024 Google LLC + * Copyright 2025 Google LLC * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -14,114 +14,125 @@ * limitations under the License. */ -# tfdoc:file:description CI/CD locals and resources. - locals { - _cicd_configs = merge( - # stages - { - for k, v in var.cicd_config : k => merge(v, { - level = k == "bootstrap" ? 0 : 1 - stage = k - }) if v != null - }, - # addons - { - for k, v in var.fast_addon : k => merge(v.cicd_config, { - level = 1 - stage = substr(v.parent_stage, 2, -1) - }) if v.cicd_config != null - } - ) - cicd_providers = { + _cicd = try(yamldecode(file(local.paths.cicd)), {}) + _cicd_identity_providers = { for k, v in google_iam_workload_identity_pool_provider.default : - k => { - audiences = concat( - v.oidc[0].allowed_audiences, - ["https://iam.googleapis.com/${v.name}"] + "$wif_providers:${k}" => v.id + } + _cicd_output_files = { + for k, v in google_storage_bucket_object.providers : + "$output_files:providers/${k}" => v.name + } + cicd_project_ids = { + for k, v in merge( + var.context.project_ids, module.factory.project_ids + ) : "$project_ids:${k}" => v + } + cicd_workflows = { + for k, v in lookup(local._cicd, "workflows", {}) : k => { + outputs_bucket = lookup( + local.of_buckets, + v.output_files.storage_bucket, + v.output_files.storage_bucket ) - issuer = local.workload_identity_providers[k].issuer - issuer_uri = try(v.oidc[0].issuer_uri, null) - name = v.name - principal_branch = local.workload_identity_providers[k].principal_branch - principal_repo = local.workload_identity_providers[k].principal_repo + workflow = templatefile("assets/workflow-${v.template}.yaml", { + identity_provider = lookup( + local._cicd_identity_providers, + v.workload_identity_provider.id, + v.workload_identity_provider.id + ) + service_accounts = { + apply = lookup( + local.of_service_accounts, + v.service_accounts.apply, + v.service_accounts.apply + ) + plan = lookup( + local.of_service_accounts, + v.service_accounts.plan, + v.service_accounts.plan + ) + } + outputs_bucket = lookup( + local.of_buckets, + v.output_files.storage_bucket, + v.output_files.storage_bucket + ) + stage_name = k + tf_providers_files = { + apply = lookup( + local._cicd_output_files, + v.output_files.providers.apply, + v.output_files.providers.apply + ) + plan = lookup( + local._cicd_output_files, + v.output_files.providers.plan, + v.output_files.providers.plan + ) + } + tf_var_files = try(v.output_files.files, []) + }) } } - cicd_repositories = { - for k, v in local._cicd_configs : k => v if( - contains(keys(local.workload_identity_providers), v.identity_provider) && - fileexists("${path.module}/templates/workflow-${v.repository.type}.yaml") + wif_project = try(local._cicd.workload_identity_federation.project, null) + wif_providers = { + for k, v in try(local._cicd.workload_identity_federation.providers, {}) : + k => merge(v, lookup(local.wif_defs, v.issuer, {})) + } +} + +resource "google_iam_workload_identity_pool" "default" { + count = local.wif_project == null ? 0 : 1 + project = lookup( + local.cicd_project_ids, local.wif_project, local.wif_project + ) + workload_identity_pool_id = try( + local._cicd.workload_identity_federation.pool_name, "iac-0" + ) +} + +resource "google_iam_workload_identity_pool_provider" "default" { + for_each = local.wif_providers + project = ( + google_iam_workload_identity_pool.default[0].project + ) + workload_identity_pool_id = ( + google_iam_workload_identity_pool.default[0].workload_identity_pool_id + ) + workload_identity_pool_provider_id = each.key + attribute_condition = lookup( + each.value, "attribute_condition", null + ) + attribute_mapping = lookup( + each.value, "attribute_mapping", {} + ) + oidc { + # Setting an empty list configures allowed_audiences to the url of the provider + allowed_audiences = try(each.value.custom_settings.audiences, []) + # If users don't provide an issuer_uri, we set the public one for the platform chosen. + issuer_uri = ( + try(each.value.custom_settings.issuer_uri, null) != null + ? each.value.custom_settings.issuer_uri + : try(each.value.issuer_uri, null) ) - } - cicd_workflow_providers = merge( - { - for k, v in local.cicd_repositories : - k => "${v.level}-${k}-providers.tf" - }, - { - for k, v in local.cicd_repositories : - "${k}-r" => "${v.level}-${k}-r-providers.tf" - } - ) -} - -# SAs used by CI/CD workflows to impersonate automation SAs - -module "automation-tf-cicd-sa" { - source = "../../../modules/iam-service-account" - for_each = local.cicd_repositories - project_id = module.automation-project.project_id - name = templatestring( - var.resource_names["sa-cicd_template"], { key = each.key } - ) - display_name = "Terraform CI/CD ${each.key} service account." - prefix = var.prefix - iam = { - "roles/iam.workloadIdentityUser" = [ - each.value.repository.branch == null - ? format( - local.workload_identity_providers_defs[each.value.repository.type].principal_repo, - google_iam_workload_identity_pool.default[0].name, - each.value.repository.name - ) - : format( - local.workload_identity_providers_defs[each.value.repository.type].principal_branch, - google_iam_workload_identity_pool.default[0].name, - each.value.repository.name, - each.value.repository.branch - ) - ] - } - iam_project_roles = { - (module.automation-project.project_id) = ["roles/logging.logWriter"] - } - iam_storage_roles = { - (module.automation-tf-output-gcs.name) = ["roles/storage.objectViewer"] + # OIDC JWKs in JSON String format. If no value is provided, they key is + # fetched from the `.well-known` path for the issuer_uri + jwks_json = try(each.value.custom_settings.jwks_json, null) } } -module "automation-tf-cicd-r-sa" { - source = "../../../modules/iam-service-account" - for_each = local.cicd_repositories - project_id = module.automation-project.project_id - name = templatestring( - var.resource_names["sa-cicd_template_ro"], { key = each.key } - ) - display_name = "Terraform CI/CD ${each.key} service account (read-only)." - prefix = var.prefix - iam = { - "roles/iam.workloadIdentityUser" = [ - format( - local.workload_identity_providers_defs[each.value.repository.type].principal_repo, - google_iam_workload_identity_pool.default[0].name, - each.value.repository.name - ) - ] - } - iam_project_roles = { - (module.automation-project.project_id) = ["roles/logging.logWriter"] - } - iam_storage_roles = { - (module.automation-tf-output-gcs.name) = ["roles/storage.objectViewer"] - } +resource "local_file" "workflows" { + for_each = local.of_path == null ? {} : local.cicd_workflows + file_permission = "0644" + filename = "${local.of_path}/workflows/${each.key}.yaml" + content = each.value.workflow +} + +resource "google_storage_bucket_object" "workflows" { + for_each = local.cicd_workflows + bucket = each.value.outputs_bucket + name = "workflows/${each.key}.yaml" + content = each.value.workflow } diff --git a/fast/stages/0-bootstrap-experimental/data/billing-accounts/default.yaml b/fast/stages/0-bootstrap/data/billing-accounts/default.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/billing-accounts/default.yaml rename to fast/stages/0-bootstrap/data/billing-accounts/default.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/cicd.yaml b/fast/stages/0-bootstrap/data/cicd.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/cicd.yaml rename to fast/stages/0-bootstrap/data/cicd.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/defaults.yaml b/fast/stages/0-bootstrap/data/defaults.yaml similarity index 90% rename from fast/stages/0-bootstrap-experimental/data/defaults.yaml rename to fast/stages/0-bootstrap/data/defaults.yaml index f3deb50ca..8499fc807 100644 --- a/fast/stages/0-bootstrap-experimental/data/defaults.yaml +++ b/fast/stages/0-bootstrap/data/defaults.yaml @@ -35,7 +35,7 @@ context: # you can populate context variables here for use in YAML replacements iam_principals: # this is the default group used in boostrap, initial user must be a member - gcp-organization-admins: group:fabric-fast-domainless-owners@twosync.google.com + gcp-organization-admins: group:gcp-organization-admins@example.com output_files: # local path is optional but recommended when starting local_path: ~/fast-config/fast-test-00 @@ -63,3 +63,8 @@ output_files: bucket: $storage_buckets:iac-0/iac-stage-state prefix: 2-project-factory service_account: $iam_principals:service_accounts/iac-0/iac-pf-rw + 3-data-platform-dev: + bucket: $storage_buckets:iac-0/iac-stage-state + prefix: 3-data-platform-dev + service_account: $iam_principals:service_accounts/iac-0/iac-dp-dev-rw + diff --git a/tests/modules/project_factory_experimental/data/data_overrides_defaults/projects/service2.yaml b/fast/stages/0-bootstrap/data/folders/data-platform/.config.yaml similarity index 86% rename from tests/modules/project_factory_experimental/data/data_overrides_defaults/projects/service2.yaml rename to fast/stages/0-bootstrap/data/folders/data-platform/.config.yaml index fb254a9d2..c4e00c202 100644 --- a/tests/modules/project_factory_experimental/data/data_overrides_defaults/projects/service2.yaml +++ b/fast/stages/0-bootstrap/data/folders/data-platform/.config.yaml @@ -12,6 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -billing_account: 012345-67890A-BCDEF0 +# yaml-language-server: $schema=../../../schemas/folder.schema.json -# take defaults + overrides only +name: Data Platform diff --git a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-a/dev/_config.yaml b/fast/stages/0-bootstrap/data/folders/data-platform/dev/.config.yaml similarity index 60% rename from tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-a/dev/_config.yaml rename to fast/stages/0-bootstrap/data/folders/data-platform/dev/.config.yaml index da77cb7f1..7e855e148 100644 --- a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-a/dev/_config.yaml +++ b/fast/stages/0-bootstrap/data/folders/data-platform/dev/.config.yaml @@ -1,4 +1,4 @@ -# Copyright 2024 Google LLC +# Copyright 2025 Google LLC # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -15,8 +15,15 @@ # yaml-language-server: $schema=../../../../schemas/folder.schema.json name: Development +iam_by_principals: + $iam_principals:service_accounts/iac-0/iac-dp-dev-rw: + - roles/logging.admin + - roles/owner + - roles/resourcemanager.folderAdmin + - roles/resourcemanager.projectCreator + - roles/compute.xpnAdmin + $iam_principals:service_accounts/iac-0/iac-dp-dev-ro: + - roles/viewer + - roles/resourcemanager.folderViewer tag_bindings: - environment: environment/development -# iam_by_principals: -# "group:team-a-admins@example.com": -# - roles/editor + environment: $tag_values:environment/development diff --git a/fast/stages/0-bootstrap-experimental/data/folders/networking/prod/.config.yaml b/fast/stages/0-bootstrap/data/folders/data-platform/prod/.config.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/folders/networking/prod/.config.yaml rename to fast/stages/0-bootstrap/data/folders/data-platform/prod/.config.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/folders/networking/.config.yaml b/fast/stages/0-bootstrap/data/folders/networking/.config.yaml similarity index 66% rename from fast/stages/0-bootstrap-experimental/data/folders/networking/.config.yaml rename to fast/stages/0-bootstrap/data/folders/networking/.config.yaml index 03d6e2556..c40718b38 100644 --- a/fast/stages/0-bootstrap-experimental/data/folders/networking/.config.yaml +++ b/fast/stages/0-bootstrap/data/folders/networking/.config.yaml @@ -30,4 +30,21 @@ iam_by_principals: $iam_principals:service_accounts/iac-0/iac-pf-rw: - $custom_roles:service_project_network_admin $iam_principals:service_accounts/iac-0/iac-pf-ro: - - roles/compute.viewer \ No newline at end of file + - roles/compute.viewer +iam_bindings: + dp_dev_rw: + members: + - $iam_principals:service_accounts/iac-0/iac-dp-dev-rw + role: $custom_roles:service_project_network_admin + condition: + expression: | + resource.matchTag('${organization.id}/environment', 'development') + title: Data platform dev service project admin. + dp_dev_ro: + role: roles/compute.networkViewer + members: + - $iam_principals:service_accounts/iac-0/iac-dp-dev-ro + condition: + title: Data platform dev network viewer. + expression: | + resource.matchTag('${organization.id}/environment', 'development') diff --git a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-b/dev/_config.yaml b/fast/stages/0-bootstrap/data/folders/networking/dev/.config.yaml similarity index 53% rename from tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-b/dev/_config.yaml rename to fast/stages/0-bootstrap/data/folders/networking/dev/.config.yaml index e50bb7308..90f1578a4 100644 --- a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-b/dev/_config.yaml +++ b/fast/stages/0-bootstrap/data/folders/networking/dev/.config.yaml @@ -1,4 +1,4 @@ -# Copyright 2024 Google LLC +# Copyright 2025 Google LLC # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -15,8 +15,19 @@ # yaml-language-server: $schema=../../../../schemas/folder.schema.json name: Development +iam: + $custom_roles:project_iam_viewer: + - $iam_principals:service_accounts/iac-0/iac-dp-dev-ro +iam_bindings: + dp_dev: + role: roles/resourcemanager.projectIamAdmin + members: + - $iam_principals:service_accounts/iac-0/iac-dp-dev-rw + condition: + title: Data platform dev delegated IAM grant. + expression: | + api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([ + '${custom_roles.service_project_network_admin}' + ]) tag_bindings: - environment: environment/development -# iam_by_principals: -# "group:team-b-admins@example.com": -# - roles/editor + environment: $tag_values:environment/development diff --git a/fast/stages/0-bootstrap-experimental/data/folders/networking/dev/.config.yaml b/fast/stages/0-bootstrap/data/folders/networking/prod/.config.yaml similarity index 90% rename from fast/stages/0-bootstrap-experimental/data/folders/networking/dev/.config.yaml rename to fast/stages/0-bootstrap/data/folders/networking/prod/.config.yaml index 9c09302ac..763d6916c 100644 --- a/fast/stages/0-bootstrap-experimental/data/folders/networking/dev/.config.yaml +++ b/fast/stages/0-bootstrap/data/folders/networking/prod/.config.yaml @@ -14,6 +14,6 @@ # yaml-language-server: $schema=../../../../schemas/folder.schema.json -name: Development +name: Production tag_bindings: - environment: $tag_values:environment/development \ No newline at end of file + environment: $tag_values:environment/production \ No newline at end of file diff --git a/fast/stages/0-bootstrap-experimental/data/folders/security/.config.yaml b/fast/stages/0-bootstrap/data/folders/security/.config.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/folders/security/.config.yaml rename to fast/stages/0-bootstrap/data/folders/security/.config.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/folders/teams/.config.yaml b/fast/stages/0-bootstrap/data/folders/teams/.config.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/folders/teams/.config.yaml rename to fast/stages/0-bootstrap/data/folders/teams/.config.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/.config.yaml b/fast/stages/0-bootstrap/data/organization/.config.yaml similarity index 96% rename from fast/stages/0-bootstrap-experimental/data/organization/.config.yaml rename to fast/stages/0-bootstrap/data/organization/.config.yaml index bca3d45ac..f323d6397 100644 --- a/fast/stages/0-bootstrap-experimental/data/organization/.config.yaml +++ b/fast/stages/0-bootstrap/data/organization/.config.yaml @@ -28,14 +28,14 @@ iam_bindings: members: - $iam_principals:service_accounts/iac-0/iac-pf-rw condition: - expression: resource.matchTag('${organization}/context', 'project-factory') + expression: resource.matchTag('${organization.id}/context', 'project-factory') title: Project factory org policy admin pf_org_policy_viewer: role: roles/orgpolicy.policyViewer members: - $iam_principals:service_accounts/iac-0/iac-pf-ro condition: - expression: resource.matchTag('${organization}/context', 'project-factory') + expression: resource.matchTag('${organization.id}/context', 'project-factory') title: Project factory org policy viewer # authoritative IAM bindings by principal iam_by_principals: diff --git a/fast/stages/0-bootstrap-experimental/data/organization/custom-constraints/accesscontextmanager.yaml b/fast/stages/0-bootstrap/data/organization/custom-constraints/accesscontextmanager.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/custom-constraints/accesscontextmanager.yaml rename to fast/stages/0-bootstrap/data/organization/custom-constraints/accesscontextmanager.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/custom-constraints/gke.yaml b/fast/stages/0-bootstrap/data/organization/custom-constraints/gke.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/custom-constraints/gke.yaml rename to fast/stages/0-bootstrap/data/organization/custom-constraints/gke.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/custom-roles/network_firewall_policies_admin.yaml b/fast/stages/0-bootstrap/data/organization/custom-roles/network_firewall_policies_admin.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/custom-roles/network_firewall_policies_admin.yaml rename to fast/stages/0-bootstrap/data/organization/custom-roles/network_firewall_policies_admin.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/custom-roles/organization_admin_viewer.yaml b/fast/stages/0-bootstrap/data/organization/custom-roles/organization_admin_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/custom-roles/organization_admin_viewer.yaml rename to fast/stages/0-bootstrap/data/organization/custom-roles/organization_admin_viewer.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/custom-roles/organization_iam_admin.yaml b/fast/stages/0-bootstrap/data/organization/custom-roles/organization_iam_admin.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/custom-roles/organization_iam_admin.yaml rename to fast/stages/0-bootstrap/data/organization/custom-roles/organization_iam_admin.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/custom-roles/project_iam_viewer.yaml b/fast/stages/0-bootstrap/data/organization/custom-roles/project_iam_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/custom-roles/project_iam_viewer.yaml rename to fast/stages/0-bootstrap/data/organization/custom-roles/project_iam_viewer.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/custom-roles/service_project_network_admin.yaml b/fast/stages/0-bootstrap/data/organization/custom-roles/service_project_network_admin.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/custom-roles/service_project_network_admin.yaml rename to fast/stages/0-bootstrap/data/organization/custom-roles/service_project_network_admin.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/custom-roles/storage_viewer.yaml b/fast/stages/0-bootstrap/data/organization/custom-roles/storage_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/custom-roles/storage_viewer.yaml rename to fast/stages/0-bootstrap/data/organization/custom-roles/storage_viewer.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/custom-roles/tag_viewer.yaml b/fast/stages/0-bootstrap/data/organization/custom-roles/tag_viewer.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/custom-roles/tag_viewer.yaml rename to fast/stages/0-bootstrap/data/organization/custom-roles/tag_viewer.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/org-policies/accesscontextmanager.yaml b/fast/stages/0-bootstrap/data/organization/org-policies/accesscontextmanager.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/org-policies/accesscontextmanager.yaml rename to fast/stages/0-bootstrap/data/organization/org-policies/accesscontextmanager.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/org-policies/cloudbuild.yaml b/fast/stages/0-bootstrap/data/organization/org-policies/cloudbuild.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/org-policies/cloudbuild.yaml rename to fast/stages/0-bootstrap/data/organization/org-policies/cloudbuild.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/org-policies/compute.yaml b/fast/stages/0-bootstrap/data/organization/org-policies/compute.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/org-policies/compute.yaml rename to fast/stages/0-bootstrap/data/organization/org-policies/compute.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/org-policies/essentialcontacts.yaml b/fast/stages/0-bootstrap/data/organization/org-policies/essentialcontacts.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/org-policies/essentialcontacts.yaml rename to fast/stages/0-bootstrap/data/organization/org-policies/essentialcontacts.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/org-policies/gcp.yaml b/fast/stages/0-bootstrap/data/organization/org-policies/gcp.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/org-policies/gcp.yaml rename to fast/stages/0-bootstrap/data/organization/org-policies/gcp.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/org-policies/gke.yaml b/fast/stages/0-bootstrap/data/organization/org-policies/gke.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/org-policies/gke.yaml rename to fast/stages/0-bootstrap/data/organization/org-policies/gke.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/org-policies/iam.yaml b/fast/stages/0-bootstrap/data/organization/org-policies/iam.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/org-policies/iam.yaml rename to fast/stages/0-bootstrap/data/organization/org-policies/iam.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/org-policies/serverless.yaml b/fast/stages/0-bootstrap/data/organization/org-policies/serverless.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/org-policies/serverless.yaml rename to fast/stages/0-bootstrap/data/organization/org-policies/serverless.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/org-policies/sql.yaml b/fast/stages/0-bootstrap/data/organization/org-policies/sql.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/org-policies/sql.yaml rename to fast/stages/0-bootstrap/data/organization/org-policies/sql.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/org-policies/storage.yaml b/fast/stages/0-bootstrap/data/organization/org-policies/storage.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/org-policies/storage.yaml rename to fast/stages/0-bootstrap/data/organization/org-policies/storage.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/tags/context.yaml b/fast/stages/0-bootstrap/data/organization/tags/context.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/tags/context.yaml rename to fast/stages/0-bootstrap/data/organization/tags/context.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/tags/environment.yaml b/fast/stages/0-bootstrap/data/organization/tags/environment.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/tags/environment.yaml rename to fast/stages/0-bootstrap/data/organization/tags/environment.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/organization/tags/org-policies.yaml b/fast/stages/0-bootstrap/data/organization/tags/org-policies.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/organization/tags/org-policies.yaml rename to fast/stages/0-bootstrap/data/organization/tags/org-policies.yaml diff --git a/fast/stages/0-bootstrap-experimental/data/projects/core/iac-0.yaml b/fast/stages/0-bootstrap/data/projects/core/iac-0.yaml similarity index 91% rename from fast/stages/0-bootstrap-experimental/data/projects/core/iac-0.yaml rename to fast/stages/0-bootstrap/data/projects/core/iac-0.yaml index 561552b2b..118196d71 100644 --- a/fast/stages/0-bootstrap-experimental/data/projects/core/iac-0.yaml +++ b/fast/stages/0-bootstrap/data/projects/core/iac-0.yaml @@ -106,18 +106,26 @@ buckets: - $iam_principals:service_accounts/iac-0/iac-pf-rw $custom_roles:storage_viewer: - $iam_principals:service_accounts/iac-0/iac-pf-ro + 3-data-platform-dev: + iam: + roles/storage.admin: + - $iam_principals:service_accounts/iac-0/iac-dp-dev-rw + $custom_roles:storage_viewer: + - $iam_principals:service_accounts/iac-0/iac-dp-dev-ro # Terraform state bucket for FAST outputs iac-outputs: description: Terraform state for the org-level automation. iam: roles/storage.admin: - $iam_principals:service_accounts/iac-0/iac-bootstrap-rw + - $iam_principals:service_accounts/iac-0/iac-dp-dev-rw - $iam_principals:service_accounts/iac-0/iac-networking-rw - $iam_principals:service_accounts/iac-0/iac-security-rw - $iam_principals:service_accounts/iac-0/iac-pf-rw - $iam_principals:service_accounts/iac-0/iac-vpcsc-rw $custom_roles:storage_viewer: - $iam_principals:service_accounts/iac-0/iac-bootstrap-ro + - $iam_principals:service_accounts/iac-0/iac-dp-dev-ro - $iam_principals:service_accounts/iac-0/iac-networking-ro - $iam_principals:service_accounts/iac-0/iac-security-ro - $iam_principals:service_accounts/iac-0/iac-pf-ro @@ -159,3 +167,8 @@ service_accounts: display_name: IaC service account for project factory (read-only). iac-pf-rw: display_name: IaC service account for project factory (read-write). + # IaC service accounts for data platform (dev) stage + iac-dp-dev-ro: + display_name: IaC service account for data platform dev (read-only). + iac-dp-dev-rw: + display_name: IaC service account for data platform dev (read-write). diff --git a/fast/stages/0-bootstrap-experimental/data/projects/core/log-0.yaml b/fast/stages/0-bootstrap/data/projects/core/log-0.yaml similarity index 100% rename from fast/stages/0-bootstrap-experimental/data/projects/core/log-0.yaml rename to fast/stages/0-bootstrap/data/projects/core/log-0.yaml diff --git a/fast/stages/0-bootstrap-experimental/diagram-classic-fast.png b/fast/stages/0-bootstrap/diagram-classic-fast.png similarity index 100% rename from fast/stages/0-bootstrap-experimental/diagram-classic-fast.png rename to fast/stages/0-bootstrap/diagram-classic-fast.png diff --git a/fast/stages/0-bootstrap-experimental/factory.tf b/fast/stages/0-bootstrap/factory.tf similarity index 95% rename from fast/stages/0-bootstrap-experimental/factory.tf rename to fast/stages/0-bootstrap/factory.tf index 048a665f5..fb80d5bc0 100644 --- a/fast/stages/0-bootstrap-experimental/factory.tf +++ b/fast/stages/0-bootstrap/factory.tf @@ -26,7 +26,7 @@ locals { } module "factory" { - source = "../../../modules/project-factory-experimental" + source = "../../../modules/project-factory" data_defaults = merge( local.project_defaults.defaults, local.factory_billing, @@ -34,6 +34,7 @@ module "factory" { ) data_overrides = local.project_defaults.overrides context = merge(local.ctx, { + condition_vars = local.ctx_condition_vars custom_roles = merge( local.ctx.custom_roles, module.organization[0].custom_role_id diff --git a/fast/stages/0-bootstrap-experimental/imports.tf b/fast/stages/0-bootstrap/imports.tf similarity index 100% rename from fast/stages/0-bootstrap-experimental/imports.tf rename to fast/stages/0-bootstrap/imports.tf diff --git a/fast/stages/0-bootstrap/main.tf b/fast/stages/0-bootstrap/main.tf index c7ce07cf7..45d7a5c9e 100644 --- a/fast/stages/0-bootstrap/main.tf +++ b/fast/stages/0-bootstrap/main.tf @@ -1,5 +1,5 @@ /** - * Copyright 2024 Google LLC + * Copyright 2025 Google LLC * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -15,17 +15,82 @@ */ locals { - principals = { - for k, v in var.groups : k => ( - can(regex("^[a-zA-Z]+:", v)) - ? v - : "group:${v}@${var.organization.domain}" + paths = { + for k, v in var.factories_config : k => try(pathexpand(v), null) + } + # fail if we have no valid defaults + _defaults = yamldecode(file(local.paths.defaults)) + ctx = merge(var.context, { + iam_principals = local.iam_principals + locations = { + for k, v in local.defaults.locations : + k => v if k != "pubsub" + } + }) + defaults = { + billing_account = try(local._defaults.global.billing_account, null) + locations = merge(try(local._defaults.global.locations, {}), { + bigquery = "eu" + logging = "global" + pubsub = [] + storage = "eu" + }) + organization = ( + try(local._defaults.global.organization.id, null) == null + ? null + : local._defaults.global.organization + ) + prefix = try( + local.project_defaults.defaults.prefix, + local.project_defaults.overrides.prefix, + null ) } - locations = { - bq = var.locations.bq - gcs = var.locations.gcs - logging = var.locations.logging - pubsub = var.locations.pubsub + iam_principals = merge( + local.org_iam_principals, + var.context.iam_principals, + try(local._defaults.context.iam_principals, {}) + ) + output_files = { + local_path = try(local._defaults.output_files.local_path, null) + storage_bucket = try(local._defaults.output_files.storage_bucket, null) + providers = try(local._defaults.output_files.providers, {}) + } + project_defaults = { + defaults = try(local._defaults.projects.defaults, {}) + overrides = try(local._defaults.projects.overrides, {}) + } +} + +# TODO: streamine location replacements + +resource "terraform_data" "precondition" { + lifecycle { + precondition { + condition = try(local.defaults.billing_account, null) != null + error_message = "No billing account set in global defaults." + } + precondition { + condition = ( + local.organization_id != null || + try(local.project_defaults.defaults.parent, null) != null || + try(local.project_defaults.overrides.parent, null) != null + ) + error_message = "Project parent must be set in project defaults or overrides if no organization id is set." + } + precondition { + condition = ( + try(local.project_defaults.defaults.prefix, null) != null || + try(local.project_defaults.overrides.prefix, null) != null + ) + error_message = "Prefix must be set in project defaults or overrides." + } + precondition { + condition = ( + try(local.project_defaults.defaults.storage_location, null) != null || + try(local.project_defaults.overrides.storage_location, null) != null + ) + error_message = "Storage location must be set in project defaults or overrides." + } } } diff --git a/fast/stages/0-bootstrap/organization.tf b/fast/stages/0-bootstrap/organization.tf index 947ddd9a2..139cafda1 100644 --- a/fast/stages/0-bootstrap/organization.tf +++ b/fast/stages/0-bootstrap/organization.tf @@ -14,224 +14,131 @@ * limitations under the License. */ -# tfdoc:file:description Organization-level IAM. - locals { - # reassemble logical bindings into the formats expected by the module - _iam_bindings = merge( - local.iam_domain_bindings, - local.iam_sa_bindings, - local.iam_user_bootstrap_bindings, - { - for k, v in local.iam_principal_bindings : k => { - authoritative = [] - additive = v.additive - } + ctx_condition_vars = { + custom_roles = merge( + local.ctx.custom_roles, + module.organization[0].custom_role_id + ) + organization = { + id = local.organization_id + customer_id = local.organization.customer_id + domain = local.organization.domain } - ) - _iam_bindings_auth = flatten([ - for member, data in local._iam_bindings : [ - for role in data.authoritative : { - member = member - role = role - } - ] - ]) - _iam_bindings_add = flatten([ - for member, data in local._iam_bindings : [ - for role in data.additive : { - member = member - role = role - } - ] - ]) - org_policies_tag_name = "${var.organization.id}/${var.org_policies_config.tag_name}" - iam_principals = { - for k, v in local.iam_principal_bindings : k => v.authoritative + tag_keys = merge( + local.ctx.tag_keys, + local.org_tag_keys + ) + tag_values = merge( + local.ctx.tag_values, + local.org_tag_values + ) } - iam = merge( - { - for r in local.iam_delete_roles : r => [] - }, - { - for b in local._iam_bindings_auth : b.role => b.member... - } + # prepare organization data + organization = merge( + # initialize required attributes + { customer_id = null, domain = null, id = null }, + # merge defaults + lookup(local.defaults, "organization", {}), + # merge attributes defined in yaml + try(yamldecode(file("${local.paths.organization}/.config.yaml")), {}) ) - iam_bindings_additive = { - for b in local._iam_bindings_add : "${b.role}-${b.member}" => { - member = b.member - role = b.role - } + # interpolate organization id if required + organization_id = ( + local.organization.id == "$defaults:organization/id" + ? try(local.defaults.organization.id, local.organization.id) + : local.organization.id + ) + # build map of predefined groups if organization domain is set + org_iam_principals = local.organization.domain == null ? {} : { + domain = "domain:${local.organization.domain}" + gcp-billing-admins = "group:gcp-billing-admins@${local.organization.domain}" + gcp-devops = "group:gcp-devops@${local.organization.domain}" + gcp-network-admins = "group:gcp-network-admins@${local.organization.domain}" + gcp-organization-admins = "group:gcp-organization-admins@${local.organization.domain}" + gcp-secops-admins = "group:gcp-secops-admins@${local.organization.domain}" + gcp-security-admins = "group:gcp-security-admins@${local.organization.domain}" + gcp-support = "group:gcp-support@${local.organization.domain}" } -} - -# TODO: add a check block to ensure our custom roles exist in the factory files - -# import org policy constraints enabled by default in new orgs since February 2024 -import { - for_each = ( - !var.org_policies_config.import_defaults || var.bootstrap_user != null - ? toset([]) - : toset([ - # source: https://cloud.google.com/resource-manager/docs/secure-by-default-organizations#organization_policies_enforced_on_organization_resources - # listed in the order as on page - "iam.disableServiceAccountKeyCreation", - "iam.disableServiceAccountKeyUpload", - "iam.automaticIamGrantsForDefaultServiceAccounts", - "iam.allowedPolicyMemberDomains", - "essentialcontacts.allowedContactDomains", - "storage.uniformBucketLevelAccess", - "compute.setNewProjectDefaultToZonalDNSOnly", # Verified as of 2024-09-13 - "compute.restrictProtocolForwardingCreationForTypes", # Verified as of 2025-02-13 - ]) - ) - id = "organizations/${var.organization.id}/policies/${each.key}" - to = module.organization.google_org_policy_policy.default[each.key] -} - -module "organization-logging" { - # Preconfigure organization-wide logging settings to ensure project - # log buckets (_Default, _Required) are created in the location - # specified by `var.locations.logging`. This separate - # organization-block prevents circular dependencies with later - # project creation. - source = "../../../modules/organization" - organization_id = "organizations/${var.organization.id}" - logging_settings = { - storage_location = var.locations.logging + org_tag_keys = { + for k, v in module.organization[0].tag_keys : k => v.id + } + org_tag_values = { + for k, v in module.organization[0].tag_values : k => v.id } } module "organization" { - source = "../../../modules/organization" - organization_id = module.organization-logging.id - # human (groups) IAM bindings - iam_by_principals = { - for key in distinct(concat( - keys(local.iam_principals), - keys(var.iam_by_principals), - )) : - key => distinct(concat( - lookup(local.iam_principals, key, []), - lookup(var.iam_by_principals, key, []), - )) - } - # machine (service accounts) IAM bindings - iam = merge( - { - for k, v in local.iam : k => distinct(concat(v, lookup(var.iam, k, []))) - }, - { - for k, v in var.iam : k => v if lookup(local.iam, k, null) == null - } - ) - # additive bindings, used for roles co-managed by different stages - iam_bindings_additive = merge( - local.iam_bindings_additive, - var.iam_bindings_additive - ) - # delegated role grant for resource manager service account - iam_bindings = merge( - { - organization_iam_admin_conditional = { - members = [module.automation-tf-resman-sa.iam_email] - role = module.organization.custom_role_id["organization_iam_admin"] - condition = { - expression = ( - format( - <<-EOT - api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s]) - || api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s]) - || api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s]) - EOT - , join(",", formatlist("'%s'", [ - "roles/accesscontextmanager.policyEditor", - "roles/accesscontextmanager.policyReader", - "roles/cloudasset.viewer", - "roles/compute.orgFirewallPolicyAdmin", - "roles/compute.orgFirewallPolicyUser", - "roles/compute.xpnAdmin", - "roles/orgpolicy.policyAdmin", - "roles/orgpolicy.policyViewer", - "roles/resourcemanager.organizationViewer", - ])) - , join(",", formatlist("'%s'", [ - "roles/iam.workforcePoolAdmin", - "roles/iam.workforcePoolViewer" - ])) - , join(",", formatlist("'%s'", [ - module.organization.custom_role_id["billing_viewer"], - module.organization.custom_role_id["network_firewall_policies_admin"], - module.organization.custom_role_id["ngfw_enterprise_admin"], - module.organization.custom_role_id["ngfw_enterprise_viewer"], - module.organization.custom_role_id["service_project_network_admin"], - module.organization.custom_role_id["tenant_network_admin"] - ])) - ) - ) - title = "automation_sa_delegated_grants" - description = "Automation service account delegated grants." - } - } - }, - local.billing_mode != "org" ? {} : { - organization_billing_conditional = { - members = [module.automation-tf-resman-sa.iam_email] - role = module.organization.custom_role_id["organization_iam_admin"] - condition = { - expression = format( - "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([%s])", - join(",", formatlist("'%s'", [ - "roles/billing.admin", - "roles/billing.costsManager", - "roles/billing.user", - ])) - ) - title = "automation_sa_delegated_grants" - description = "Automation service account delegated grants." - } - } - } - ) - custom_roles = var.custom_roles + source = "../../../modules/organization" + count = local.organization_id != null ? 1 : 0 + organization_id = "organizations/${local.organization_id}" + logging_settings = lookup(local.organization, "logging", null) context = { - org_policies = { - organization = var.organization - tags = { - org_policies_tag_name = local.org_policies_tag_name + condition_vars = { + organization = { + id = local.organization_id } } + locations = { + default = local.defaults.locations.logging + } } factories_config = { - custom_roles = var.factories_config.custom_roles - org_policy_custom_constraints = ( - var.bootstrap_user != null ? null : var.factories_config.custom_constraints - ) - org_policies = ( - var.bootstrap_user != null ? null : var.factories_config.org_policies - ) + org_policy_custom_constraints = "${local.paths.organization}/custom-constraints" + custom_roles = "${local.paths.organization}/custom-roles" + tags = "${local.paths.organization}/tags" } - logging_sinks = { - for name, attrs in var.log_sinks : name => { - bq_partitioned_table = attrs.type == "bigquery" - destination = local.log_sink_destinations[name].id - filter = attrs.filter - type = attrs.type - disabled = attrs.disabled - exclusions = attrs.exclusions - } - } - tags = { - (var.org_policies_config.tag_name) = { - description = "Organization policy conditions." - iam = {} - values = merge( - { - allowed-essential-contacts-domains-all = {} - allowed-policy-member-domains-all = {} - }, - var.org_policies_config.tag_values - ) - } + tags_config = { + ignore_iam = true + } +} + +module "organization-iam" { + source = "../../../modules/organization" + count = local.organization.id != null ? 1 : 0 + organization_id = module.organization[0].id + context = merge(local.ctx, { + condition_vars = local.ctx_condition_vars + custom_roles = merge( + local.ctx.custom_roles, + module.organization[0].custom_role_id + ) + iam_principals = merge( + local.ctx.iam_principals, + module.factory.iam_principals + ) + log_buckets = module.factory.log_buckets + project_ids = merge( + local.ctx.project_ids, module.factory.project_ids + ) + storage_buckets = module.factory.storage_buckets + tag_keys = merge( + local.ctx.tag_keys, + local.org_tag_keys + ) + tag_values = merge( + local.ctx.tag_values, + local.org_tag_values + ) + }) + factories_config = { + org_policies = "${local.paths.organization}/org-policies" + tags = "${local.paths.organization}/tags" + } + iam = lookup( + local.organization, "iam", {} + ) + iam_by_principals = lookup( + local.organization, "iam_by_principals", {} + ) + iam_bindings = lookup( + local.organization, "iam_bindings", {} + ) + iam_bindings_additive = lookup( + local.organization, "iam_bindings_additive", {} + ) + logging_sinks = try(local.organization.logging.sinks, {}) + tags_config = { + force_context_ids = true } } diff --git a/fast/stages/0-bootstrap-experimental/output-files.tf b/fast/stages/0-bootstrap/output-files.tf similarity index 100% rename from fast/stages/0-bootstrap-experimental/output-files.tf rename to fast/stages/0-bootstrap/output-files.tf diff --git a/fast/stages/0-bootstrap/outputs.tf b/fast/stages/0-bootstrap/outputs.tf index 8ac8dc5d0..27e0e1cc9 100644 --- a/fast/stages/0-bootstrap/outputs.tf +++ b/fast/stages/0-bootstrap/outputs.tf @@ -14,187 +14,22 @@ * limitations under the License. */ -locals { - _tpl_providers = "${path.module}/templates/providers.tf.tpl" - # render CI/CD workflow templates - cicd_workflows = { - for k, v in local.cicd_repositories : "${v.level}-${k}" => templatefile( - "${path.module}/templates/workflow-${v.repository.type}.yaml", { - # If users give a list of custom audiences we set by default the first element. - # If no audiences are given, we set https://iam.googleapis.com/{PROVIDER_NAME} - audiences = try( - local.cicd_providers[v.identity_provider].audiences, [] - ) - identity_provider = try( - local.cicd_providers[v.identity_provider].name, "" - ) - outputs_bucket = module.automation-tf-output-gcs.name - service_accounts = { - apply = try(module.automation-tf-cicd-sa[k].email, "") - plan = try(module.automation-tf-cicd-r-sa[k].email, "") - } - stage_name = k - tf_providers_files = { - apply = local.cicd_workflow_providers[k] - plan = local.cicd_workflow_providers["${k}-r"] - } - tf_var_files = k == "bootstrap" ? [] : [ - "0-bootstrap.auto.tfvars.json", - "0-globals.auto.tfvars.json" - ] - } - ) - } - tfvars = { - automation = { - federated_identity_pool = try( - google_iam_workload_identity_pool.default[0].name, null - ) - federated_identity_providers = local.cicd_providers - outputs_bucket = module.automation-tf-output-gcs.name - project_id = module.automation-project.project_id - project_number = module.automation-project.number - service_accounts = { - bootstrap = module.automation-tf-bootstrap-sa.email - bootstrap-r = module.automation-tf-bootstrap-r-sa.email - resman = module.automation-tf-resman-sa.email - resman-r = module.automation-tf-resman-r-sa.email - vpcsc = module.automation-tf-vpcsc-sa.email - vpcsc-r = module.automation-tf-vpcsc-r-sa.email - } - } - billing = { - dataset = try(module.billing-export-dataset[0].id, null) - project_id = try(module.billing-export-project[0].project_id, null) - project_number = try(module.billing-export-project[0].number, null) - } - custom_roles = module.organization.custom_role_id - logging = { - project_id = module.log-export-project.project_id - project_number = module.log-export-project.number - writer_identities = module.organization.sink_writer_identities - destinations = { - bigquery = try(module.log-export-dataset[0].id, null) - logging = { for k, v in module.log-export-logbucket : k => v.id } - pubsub = { for k, v in module.log-export-pubsub : k => v.id } - storage = try(module.log-export-gcs[0].id, null) - } - } - org_policy_tags = { - key_id = ( - module.organization.tag_keys[var.org_policies_config.tag_name].id - ) - key_name = var.org_policies_config.tag_name - values = { - for k, v in module.organization.tag_values : - split("/", k)[1] => v.id - } - } - universe = var.universe - } - tfvars_globals = { - billing_account = var.billing_account - groups = local.principals - environments = { - for k, v in var.environments : k => { - is_default = v.is_default - key = k - name = v.name - short_name = v.short_name != null ? v.short_name : k - tag_name = v.tag_name != null ? v.tag_name : lower(v.name) - } - } - locations = local.locations - organization = var.organization - prefix = var.prefix - } +output "iam_principals" { + description = "IAM principals." + value = local.iam_principals } -output "automation" { - description = "Automation resources." - value = local.tfvars.automation +output "locations" { + description = "Default locations." + value = local.defaults.locations } -output "billing_dataset" { - description = "BigQuery dataset prepared for billing export." - value = try(module.billing-export-dataset[0].id, null) +output "projects" { + description = "Attributes for managed projects." + value = module.factory.projects } -output "cicd_repositories" { - description = "CI/CD repository configurations." - value = { - for k, v in local.cicd_repositories : k => { - branch = v.repository.branch - name = v.repository.name - provider = try(local.cicd_providers[v.identity_provider].name, null) - service_account = try(module.automation-tf-cicd-sa[k].email, null) - } - } -} - -output "custom_roles" { - description = "Organization-level custom roles." - value = module.organization.custom_role_id -} - -output "outputs_bucket" { - description = "GCS bucket where generated output files are stored." - value = module.automation-tf-output-gcs.name -} - -output "project_ids" { - description = "Projects created by this stage." - value = { - automation = module.automation-project.project_id - billing-export = try(module.billing-export-project[0].project_id, null) - log-export = module.log-export-project.project_id - } -} - -# ready to use provider configurations for subsequent stages when not using files -output "providers" { - # tfdoc:output:consumers stage-01 - description = "Terraform provider files for this stage and dependent stages." - sensitive = true - value = local.providers -} - -output "service_accounts" { - description = "Automation service accounts created by this stage." - value = { - bootstrap = module.automation-tf-bootstrap-sa.email - resman = module.automation-tf-resman-sa.email - } -} - -# ready to use variable values for subsequent stages output "tfvars" { - description = "Terraform variable files for the following stages." - sensitive = true - value = local.tfvars -} - -output "tfvars_globals" { - description = "Terraform Globals variable files for the following stages." - sensitive = true - value = local.tfvars_globals -} - -output "workforce_identity_pool" { - description = "Workforce Identity Federation pool." - value = { - pool = try( - google_iam_workforce_pool.default[0].name, null - ) - } -} - -output "workload_identity_pool" { - description = "Workload Identity Federation pool and providers." - value = { - pool = try( - google_iam_workload_identity_pool.default[0].name, null - ) - providers = local.cicd_providers - } + description = "Stage tfvars." + value = local.of_tfvars } diff --git a/fast/stages/0-bootstrap-experimental/schemas/billing-account.schema.json b/fast/stages/0-bootstrap/schemas/billing-account.schema.json similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/billing-account.schema.json rename to fast/stages/0-bootstrap/schemas/billing-account.schema.json diff --git a/fast/stages/0-bootstrap-experimental/schemas/billing-account.schema.md b/fast/stages/0-bootstrap/schemas/billing-account.schema.md similarity index 70% rename from fast/stages/0-bootstrap-experimental/schemas/billing-account.schema.md rename to fast/stages/0-bootstrap/schemas/billing-account.schema.md index 6a2334fe6..2f0e6be80 100644 --- a/fast/stages/0-bootstrap-experimental/schemas/billing-account.schema.md +++ b/fast/stages/0-bootstrap/schemas/billing-account.schema.md @@ -44,6 +44,24 @@ - **`^(?:\$[a-z_-]+:|domain:|group:|serviceAccount:|user:|principal:|principalSet:)`**: *array* - items: *string*
*pattern: ^roles/* +- **logging_sinks**: *object* +
*additional properties: false* + - **`^[a-z][a-z0-9_-]+`**: *object* +
*additional properties: false* + - ⁺**destination**: *string* + - ⁺**type**: *string* +
*enum: ['bigquery', 'logging', 'project', 'pubsub', 'storage']* + - **bq_partitioned_table**: *boolean* + - **description**: *string* + - **disabled**: *boolean* + - **exclusions**: *object* +
*additional properties: false* + - **`^[a-z][a-z0-9_-]+`**: *object* +
*additional properties: false* + - ⁺**filter**: *string* + - **description**: *string* + - **disabled**: *boolean* + - **filter**: *string* ## Definitions diff --git a/fast/stages/0-bootstrap-experimental/schemas/budget.schema.json b/fast/stages/0-bootstrap/schemas/budget.schema.json similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/budget.schema.json rename to fast/stages/0-bootstrap/schemas/budget.schema.json diff --git a/fast/stages/0-bootstrap-experimental/schemas/budget.schema.md b/fast/stages/0-bootstrap/schemas/budget.schema.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/budget.schema.md rename to fast/stages/0-bootstrap/schemas/budget.schema.md diff --git a/fast/stages/0-bootstrap-experimental/schemas/cicd.schema.json b/fast/stages/0-bootstrap/schemas/cicd.schema.json similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/cicd.schema.json rename to fast/stages/0-bootstrap/schemas/cicd.schema.json diff --git a/fast/stages/0-bootstrap-experimental/schemas/cicd.schema.md b/fast/stages/0-bootstrap/schemas/cicd.schema.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/cicd.schema.md rename to fast/stages/0-bootstrap/schemas/cicd.schema.md diff --git a/fast/stages/0-bootstrap-experimental/schemas/custom-constraint.schema.json b/fast/stages/0-bootstrap/schemas/custom-constraint.schema.json similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/custom-constraint.schema.json rename to fast/stages/0-bootstrap/schemas/custom-constraint.schema.json diff --git a/fast/stages/0-bootstrap-experimental/schemas/custom-constraint.schema.md b/fast/stages/0-bootstrap/schemas/custom-constraint.schema.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/custom-constraint.schema.md rename to fast/stages/0-bootstrap/schemas/custom-constraint.schema.md diff --git a/fast/stages/0-bootstrap/schemas/custom-role.schema.json b/fast/stages/0-bootstrap/schemas/custom-role.schema.json deleted file mode 120000 index a1d6e5658..000000000 --- a/fast/stages/0-bootstrap/schemas/custom-role.schema.json +++ /dev/null @@ -1 +0,0 @@ -../../../../modules/organization/schemas/custom-role.schema.json \ No newline at end of file diff --git a/fast/stages/0-bootstrap/schemas/custom-role.schema.json b/fast/stages/0-bootstrap/schemas/custom-role.schema.json new file mode 100644 index 000000000..d7526482c --- /dev/null +++ b/fast/stages/0-bootstrap/schemas/custom-role.schema.json @@ -0,0 +1,18 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "title": "Custom Role", + "type": "object", + "additionalProperties": false, + "properties": { + "name": { + "type": "string" + }, + "includedPermissions": { + "type": "array", + "items": { + "type": "string", + "pattern": "^[a-zA-Z-]+\\.[a-zA-Z-]+\\.[a-zA-Z-]+$" + } + } + } +} \ No newline at end of file diff --git a/fast/stages/0-bootstrap-experimental/schemas/defaults.schema.json b/fast/stages/0-bootstrap/schemas/defaults.schema.json similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/defaults.schema.json rename to fast/stages/0-bootstrap/schemas/defaults.schema.json diff --git a/fast/stages/0-bootstrap-experimental/schemas/defaults.schema.md b/fast/stages/0-bootstrap/schemas/defaults.schema.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/defaults.schema.md rename to fast/stages/0-bootstrap/schemas/defaults.schema.md diff --git a/fast/stages/0-bootstrap-experimental/schemas/folder.schema.json b/fast/stages/0-bootstrap/schemas/folder.schema.json similarity index 98% rename from fast/stages/0-bootstrap-experimental/schemas/folder.schema.json rename to fast/stages/0-bootstrap/schemas/folder.schema.json index 5fbe2769c..0ca0be056 100644 --- a/fast/stages/0-bootstrap-experimental/schemas/folder.schema.json +++ b/fast/stages/0-bootstrap/schemas/folder.schema.json @@ -270,7 +270,7 @@ }, "role": { "type": "string", - "pattern": "^roles/" + "pattern": "^(?:roles/|\\$custom_roles:)" }, "condition": { "type": "object", @@ -309,7 +309,7 @@ }, "role": { "type": "string", - "pattern": "^roles/" + "pattern": "^(?:roles/|\\$custom_roles:)" }, "condition": { "type": "object", @@ -420,4 +420,4 @@ } } } -} \ No newline at end of file +} diff --git a/modules/project-factory-experimental/schemas/folder.schema.md b/fast/stages/0-bootstrap/schemas/folder.schema.md similarity index 98% rename from modules/project-factory-experimental/schemas/folder.schema.md rename to fast/stages/0-bootstrap/schemas/folder.schema.md index 4c5fac144..cb340e2d2 100644 --- a/modules/project-factory-experimental/schemas/folder.schema.md +++ b/fast/stages/0-bootstrap/schemas/folder.schema.md @@ -99,7 +99,7 @@ - items: *string*
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:)* - **role**: *string* -
*pattern: ^roles/* +
*pattern: ^(?:roles/|\$custom_roles:)* - **condition**: *object*
*additional properties: false* - ⁺**expression**: *string* @@ -112,7 +112,7 @@ - **member**: *string*
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:)* - **role**: *string* -
*pattern: ^roles/* +
*pattern: ^(?:roles/|\$custom_roles:)* - **condition**: *object*
*additional properties: false* - ⁺**expression**: *string* diff --git a/fast/stages/0-bootstrap/schemas/org-policies.schema.json b/fast/stages/0-bootstrap/schemas/org-policies.schema.json deleted file mode 120000 index c5ebcfaf7..000000000 --- a/fast/stages/0-bootstrap/schemas/org-policies.schema.json +++ /dev/null @@ -1 +0,0 @@ -../../../../modules/organization/schemas/org-policies.schema.json \ No newline at end of file diff --git a/fast/stages/0-bootstrap/schemas/org-policies.schema.json b/fast/stages/0-bootstrap/schemas/org-policies.schema.json new file mode 100644 index 000000000..6c29331ec --- /dev/null +++ b/fast/stages/0-bootstrap/schemas/org-policies.schema.json @@ -0,0 +1,76 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "title": "Organization Policies", + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z-]+[a-zA-Z0-9\\.]+$": { + "type": "object", + "additionalProperties": false, + "properties": { + "inherit_from_parent": { + "type": "boolean" + }, + "reset": { + "type": "boolean" + }, + "rules": { + "type": "array", + "items": { + "type": "object", + "additionalProperties": false, + "properties": { + "allow": { + "$ref": "#/$defs/allow-deny" + }, + "deny": { + "$ref": "#/$defs/allow-deny" + }, + "enforce": { + "type": "boolean" + }, + "condition": { + "type": "object", + "additionalProperties": false, + "properties": { + "description": { + "type": "string" + }, + "expression": { + "type": "string" + }, + "location": { + "type": "string" + }, + "title": { + "type": "string" + } + } + }, + "parameters": { + "type": "string" + } + } + } + } + } + } + }, + "$defs": { + "allow-deny": { + "type": "object", + "additionalProperties": false, + "properties": { + "all": { + "type": "boolean" + }, + "values": { + "type": "array", + "items": { + "type": "string" + } + } + } + } + } +} diff --git a/fast/stages/0-bootstrap-experimental/schemas/organization.schema.json b/fast/stages/0-bootstrap/schemas/organization.schema.json similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/organization.schema.json rename to fast/stages/0-bootstrap/schemas/organization.schema.json diff --git a/fast/stages/0-bootstrap-experimental/schemas/organization.schema.md b/fast/stages/0-bootstrap/schemas/organization.schema.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/organization.schema.md rename to fast/stages/0-bootstrap/schemas/organization.schema.md diff --git a/fast/stages/0-bootstrap-experimental/schemas/project.schema.json b/fast/stages/0-bootstrap/schemas/project.schema.json similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/project.schema.json rename to fast/stages/0-bootstrap/schemas/project.schema.json diff --git a/fast/stages/0-bootstrap-experimental/schemas/project.schema.md b/fast/stages/0-bootstrap/schemas/project.schema.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/project.schema.md rename to fast/stages/0-bootstrap/schemas/project.schema.md diff --git a/fast/stages/0-bootstrap-experimental/schemas/tags.schema.json b/fast/stages/0-bootstrap/schemas/tags.schema.json similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/tags.schema.json rename to fast/stages/0-bootstrap/schemas/tags.schema.json diff --git a/fast/stages/0-bootstrap-experimental/schemas/tags.schema.md b/fast/stages/0-bootstrap/schemas/tags.schema.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/tags.schema.md rename to fast/stages/0-bootstrap/schemas/tags.schema.md diff --git a/fast/stages/0-bootstrap/variables.tf b/fast/stages/0-bootstrap/variables.tf index fec2f8a8f..1b4fabf92 100644 --- a/fast/stages/0-bootstrap/variables.tf +++ b/fast/stages/0-bootstrap/variables.tf @@ -14,380 +14,49 @@ * limitations under the License. */ -variable "billing_account" { - description = "Billing account id. If billing account is not part of the same org set `is_org_level` to `false`. To disable handling of billing IAM roles set `no_iam` to `true`." - type = object({ - id = string - force_create = optional(object({ - dataset = optional(bool, false) - project = optional(bool, false) - log_bucket = optional(bool, false) - }), {}) - is_org_level = optional(bool, true) - no_iam = optional(bool, false) - }) - nullable = false - validation { - condition = ( - var.billing_account.force_create.dataset != true || - var.billing_account.force_create.project == true - ) - error_message = "Forced dataset creation also needs project creation." - } -} - variable "bootstrap_user" { description = "Email of the nominal user running this stage for the first time." type = string default = null } -variable "cicd_config" { - description = "CI/CD repository configuration. Identity providers reference keys in the `federated_identity_providers` variable. Set to null to disable, or set individual repositories to null if not needed." +variable "context" { + description = "Context-specific interpolations." type = object({ - bootstrap = optional(object({ - identity_provider = string - repository = object({ - name = string - branch = optional(string) - type = optional(string, "github") - }) - })) - resman = optional(object({ - identity_provider = string - repository = object({ - name = string - branch = optional(string) - type = optional(string, "github") - }) - })) - vpcsc = optional(object({ - identity_provider = string - repository = object({ - name = string - branch = optional(string) - type = optional(string, "github") - }) - })) + custom_roles = optional(map(string), {}) + folder_ids = optional(map(string), {}) + iam_principals = optional(map(string), {}) + locations = optional(map(string), {}) + kms_keys = optional(map(string), {}) + notification_channels = optional(map(string), {}) + project_ids = optional(map(string), {}) + service_account_ids = optional(map(string), {}) + tag_keys = optional(map(string), {}) + tag_values = optional(map(string), {}) + vpc_host_projects = optional(map(string), {}) + vpc_sc_perimeters = optional(map(string), {}) }) - nullable = false default = {} - validation { - condition = alltrue([ - for k, v in coalesce(var.cicd_config, {}) : - v == null || ( - contains(["github", "gitlab", "terraform"], coalesce(try(v.repository.type, null), "null")) - ) - ]) - error_message = "Invalid repository type, supported types: 'github', 'gitlab', or 'terraform'." - } -} - -variable "custom_roles" { - description = "Map of role names => list of permissions to additionally create at the organization level." - type = map(list(string)) - nullable = false - default = {} -} - -variable "environments" { - description = "Environment names. When not defined, short name is set to the key and tag name to lower(name)." - type = map(object({ - name = string - is_default = optional(bool, false) - short_name = optional(string) - tag_name = optional(string) - })) nullable = false - default = { - dev = { - name = "Development" - } - prod = { - name = "Production" - is_default = true - } - } - validation { - condition = anytrue([ - for k, v in var.environments : v.is_default == true - ]) - error_message = "At least one environment should be marked as default." - } - validation { - condition = alltrue([ - for k, v in var.environments : join(" ", regexall( - "[a-zA-Z][a-zA-Z0-9\\s-]+[a-zA-Z0-9]", v.name - )) == v.name - ]) - error_message = "Environment names can only contain letters numbers dashes or spaces." - } - validation { - condition = alltrue([ - for k, v in var.environments : (length(coalesce(v.short_name, k)) <= 4) - ]) - error_message = "If environment key is longer than 4 characters, provide short_name that is at most 4 characters long." - } -} - -variable "essential_contacts" { - description = "Email used for essential contacts, unset if null." - type = string - default = null } variable "factories_config" { description = "Configuration for the resource factories or external data." type = object({ - custom_constraints = optional(string, "data/custom-constraints") - custom_roles = optional(string, "data/custom-roles") - org_policies = optional(string, "data/org-policies") - org_policies_iac = optional(string, "data/org-policies-iac") + billing_accounts = optional(string, "data/billing-accounts") + cicd = optional(string) + defaults = optional(string, "data/defaults.yaml") + folders = optional(string, "data/folders") + organization = optional(string, "data/organization") + projects = optional(string, "data/projects") }) nullable = false default = {} } -variable "groups" { - # https://cloud.google.com/docs/enterprise/setup-checklist - description = "Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated." - type = object({ - gcp-billing-admins = optional(string, "gcp-billing-admins") - gcp-devops = optional(string, "gcp-devops") - gcp-network-admins = optional(string, "gcp-vpc-network-admins") - gcp-organization-admins = optional(string, "gcp-organization-admins") - gcp-secops-admins = optional(string, "gcp-security-admins") - gcp-security-admins = optional(string, "gcp-security-admins") - # aliased to gcp-devops as the checklist does not create it - gcp-support = optional(string, "gcp-devops") - }) - nullable = false - default = {} -} - -variable "iam" { - description = "Organization-level custom IAM settings in role => [principal] format." - type = map(list(string)) +variable "org_policies_imports" { + description = "List of org policies to import. These need to also be defined in data files." + type = list(string) nullable = false - default = {} -} - -variable "iam_bindings_additive" { - description = "Organization-level custom additive IAM bindings. Keys are arbitrary." - type = map(object({ - member = string - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })) - nullable = false - default = {} -} - -variable "iam_by_principals" { - description = "Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the `iam` variable." - type = map(list(string)) - default = {} - nullable = false -} - -variable "locations" { - description = "Optional locations for GCS, BigQuery, and logging buckets created here." - type = object({ - bq = optional(string, "EU") - gcs = optional(string, "EU") - logging = optional(string, "global") - pubsub = optional(list(string), []) - }) - nullable = false - default = {} -} - -# See https://cloud.google.com/architecture/exporting-stackdriver-logging-for-security-and-access-analytics -# for additional logging filter examples -variable "log_sinks" { - description = "Org-level log sinks, in name => {type, filter} format." - type = map(object({ - filter = string - type = string - disabled = optional(bool, false) - exclusions = optional(map(string), {}) - })) - nullable = false - default = { - audit-logs = { - # activity logs include Google Workspace / Cloud Identity logs - # exclude them via additional filter stanza if needed - filter = <<-FILTER - log_id("cloudaudit.googleapis.com/activity") OR - log_id("cloudaudit.googleapis.com/system_event") OR - log_id("cloudaudit.googleapis.com/policy") OR - log_id("cloudaudit.googleapis.com/access_transparency") - FILTER - type = "logging" - # exclusions = { - # gke-audit = "protoPayload.serviceName=\"k8s.io\"" - # } - } - iam = { - filter = <<-FILTER - protoPayload.serviceName="iamcredentials.googleapis.com" OR - protoPayload.serviceName="iam.googleapis.com" OR - protoPayload.serviceName="sts.googleapis.com" - FILTER - type = "logging" - } - vpc-sc = { - filter = <<-FILTER - protoPayload.metadata.@type="type.googleapis.com/google.cloud.audit.VpcServiceControlAuditMetadata" - FILTER - type = "logging" - } - workspace-audit-logs = { - filter = <<-FILTER - protoPayload.serviceName="admin.googleapis.com" OR - protoPayload.serviceName="cloudidentity.googleapis.com" OR - protoPayload.serviceName="login.googleapis.com" - FILTER - type = "logging" - } - } - validation { - condition = alltrue([ - for k, v in var.log_sinks : - contains(["bigquery", "logging", "project", "pubsub", "storage"], v.type) - ]) - error_message = "Type must be one of 'bigquery', 'logging', 'project', 'pubsub', 'storage'." - } -} - -variable "org_policies_config" { - description = "Organization policies customization." - type = object({ - iac_policy_member_domains = optional(list(string)) - import_defaults = optional(bool, false) - tag_name = optional(string, "org-policies") - tag_values = optional(map(object({ - description = optional(string, "Managed by the Terraform organization module.") - iam = optional(map(list(string)), {}) - id = optional(string) - })), {}) - }) - default = {} -} - -variable "organization" { - description = "Organization details." - type = object({ - id = number - domain = optional(string) - customer_id = optional(string) - }) -} - -variable "outputs_location" { - description = "Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable." - type = string - default = null -} - -variable "prefix" { - description = "Prefix used for resources that need unique names. Use 9 characters or less." - type = string - validation { - condition = try(length(var.prefix), 0) < 10 - error_message = "Use a maximum of 9 characters for prefix." - } -} - -variable "project_parent_ids" { - description = "Optional parents for projects created here in folders/nnnnnnn format. Null values will use the organization as parent." - type = object({ - automation = optional(string) - billing = optional(string) - logging = optional(string) - }) - default = {} - nullable = false -} - -variable "resource_names" { - description = "Resource names overrides for specific resources. Prefix is always set via code, except where noted in the variable type." - type = object({ - bq-billing = optional(string, "billing_export") - bq-logs = optional(string, "logs") - gcs-bootstrap = optional(string, "prod-iac-core-bootstrap-0") - gcs-logs = optional(string, "prod-logs") - gcs-outputs = optional(string, "prod-iac-core-outputs-0") - gcs-resman = optional(string, "prod-iac-core-resman-0") - gcs-vpcsc = optional(string, "prod-iac-core-vpcsc-0") - project-automation = optional(string, "prod-iac-core-0") - project-billing = optional(string, "prod-billing-exp-0") - project-logs = optional(string, "prod-audit-logs-0") - pubsub-logs_template = optional(string, "$${key}") - sa-bootstrap = optional(string, "prod-bootstrap-0") - sa-bootstrap_ro = optional(string, "prod-bootstrap-0r") - sa-cicd_template = optional(string, "prod-$${key}-1") - sa-cicd_template_ro = optional(string, "prod-$${key}-1r") - sa-resman = optional(string, "prod-resman-0") - sa-resman_ro = optional(string, "prod-resman-0r") - sa-vpcsc = optional(string, "prod-vpcsc-0") - sa-vpcsc_ro = optional(string, "prod-vpcsc-0r") - # the identity provider resources also interpolate prefix - wf-bootstrap = optional(string, "$${prefix}-bootstrap") - wf-provider_template = optional(string, "$${prefix}-bootstrap-$${key}") - wif-bootstrap = optional(string, "$${prefix}-bootstrap") - wif-provider_template = optional(string, "$${prefix}-bootstrap-$${key}") - }) - nullable = false - default = {} -} - -variable "universe" { - description = "Target GCP universe." - type = object({ - domain = string - prefix = string - unavailable_services = optional(list(string), []) - }) - default = null -} - -variable "workforce_identity_providers" { - description = "Workforce Identity Federation pools." - type = map(object({ - attribute_condition = optional(string) - issuer = string - display_name = string - description = string - disabled = optional(bool, false) - saml = optional(object({ - idp_metadata_xml = string - }), null) - })) - default = {} - nullable = false -} - -variable "workload_identity_providers" { - description = "Workload Identity Federation pools. The `cicd_repositories` variable references keys here." - type = map(object({ - attribute_condition = optional(string) - issuer = string - custom_settings = optional(object({ - issuer_uri = optional(string) - audiences = optional(list(string), []) - jwks_json = optional(string) - }), {}) - })) - default = {} - nullable = false - # TODO: fix validation - # validation { - # condition = var.federated_identity_providers.custom_settings == null - # error_message = "Custom settings cannot be null." - # } + default = [] } diff --git a/fast/stages/0-bootstrap-experimental/wif-definitions.tf b/fast/stages/0-bootstrap/wif-definitions.tf similarity index 100% rename from fast/stages/0-bootstrap-experimental/wif-definitions.tf rename to fast/stages/0-bootstrap/wif-definitions.tf diff --git a/fast/stages/1-resman/.fast-stage.env b/fast/stages/1-resman-legacy/.fast-stage.env similarity index 100% rename from fast/stages/1-resman/.fast-stage.env rename to fast/stages/1-resman-legacy/.fast-stage.env diff --git a/fast/stages/1-resman/IAM.md b/fast/stages/1-resman-legacy/IAM.md similarity index 100% rename from fast/stages/1-resman/IAM.md rename to fast/stages/1-resman-legacy/IAM.md diff --git a/fast/stages/1-resman/README.md b/fast/stages/1-resman-legacy/README.md similarity index 98% rename from fast/stages/1-resman/README.md rename to fast/stages/1-resman-legacy/README.md index 0a1499dec..e915991a5 100644 --- a/fast/stages/1-resman/README.md +++ b/fast/stages/1-resman-legacy/README.md @@ -22,7 +22,6 @@ The following diagram is a high level reference of the resources created and man - [Other design considerations](#other-design-considerations) - [Secure tags](#secure-tags) - [Organization policy tag values from the bootstrap stage](#organization-policy-tag-values-from-the-bootstrap-stage) - - [Multitenancy](#multitenancy) - [Workload Identity Federation and CI/CD](#workload-identity-federation-and-cicd) - [How to run this stage](#how-to-run-this-stage) - [Provider and Terraform variables](#provider-and-terraform-variables) @@ -201,12 +200,6 @@ tags = { } ``` -### Multitenancy - -Multitenancy is supported via an [add-on](../../addons/1-resman-tenants/) which is entirely optional, and is be applied after resource management has been deployed. The add-on needs to be enabled before use via the `fast_addon` variable in the bootstrap stage. - -For simpler use cases that do not require complex organization-level multitenancy, [top-level folders](#top-level-folders) can be used in combination with the [project factory stage](../2-project-factory/) support for folder and project management. - ### Workload Identity Federation and CI/CD This stage also implements optional support for CI/CD, much in the same way as the bootstrap stage. The only difference is on Workload Identity Federation, which is only configured in bootstrap and made available here via stage interface variables (the automatically generated `.tfvars` files). diff --git a/fast/stages/1-resman/billing.tf b/fast/stages/1-resman-legacy/billing.tf similarity index 100% rename from fast/stages/1-resman/billing.tf rename to fast/stages/1-resman-legacy/billing.tf diff --git a/fast/stages/1-resman/data/org-policies/sandbox/compute.yaml b/fast/stages/1-resman-legacy/data/org-policies/sandbox/compute.yaml similarity index 100% rename from fast/stages/1-resman/data/org-policies/sandbox/compute.yaml rename to fast/stages/1-resman-legacy/data/org-policies/sandbox/compute.yaml diff --git a/fast/stages/1-resman/data/org-policies/sandbox/sql.yaml b/fast/stages/1-resman-legacy/data/org-policies/sandbox/sql.yaml similarity index 100% rename from fast/stages/1-resman/data/org-policies/sandbox/sql.yaml rename to fast/stages/1-resman-legacy/data/org-policies/sandbox/sql.yaml diff --git a/fast/stages/1-resman/data/stage-2/networking.yaml b/fast/stages/1-resman-legacy/data/stage-2/networking.yaml similarity index 100% rename from fast/stages/1-resman/data/stage-2/networking.yaml rename to fast/stages/1-resman-legacy/data/stage-2/networking.yaml diff --git a/fast/stages/1-resman/data/stage-2/project-factory.yaml b/fast/stages/1-resman-legacy/data/stage-2/project-factory.yaml similarity index 100% rename from fast/stages/1-resman/data/stage-2/project-factory.yaml rename to fast/stages/1-resman-legacy/data/stage-2/project-factory.yaml diff --git a/fast/stages/1-resman/data/stage-2/secops.yaml b/fast/stages/1-resman-legacy/data/stage-2/secops.yaml similarity index 100% rename from fast/stages/1-resman/data/stage-2/secops.yaml rename to fast/stages/1-resman-legacy/data/stage-2/secops.yaml diff --git a/fast/stages/1-resman/data/stage-2/security.yaml b/fast/stages/1-resman-legacy/data/stage-2/security.yaml similarity index 100% rename from fast/stages/1-resman/data/stage-2/security.yaml rename to fast/stages/1-resman-legacy/data/stage-2/security.yaml diff --git a/fast/stages/1-resman/data/stage-3/data-platform-dev.yaml b/fast/stages/1-resman-legacy/data/stage-3/data-platform-dev.yaml similarity index 100% rename from fast/stages/1-resman/data/stage-3/data-platform-dev.yaml rename to fast/stages/1-resman-legacy/data/stage-3/data-platform-dev.yaml diff --git a/fast/stages/1-resman/data/stage-3/gcve-dev.yaml b/fast/stages/1-resman-legacy/data/stage-3/gcve-dev.yaml similarity index 100% rename from fast/stages/1-resman/data/stage-3/gcve-dev.yaml rename to fast/stages/1-resman-legacy/data/stage-3/gcve-dev.yaml diff --git a/fast/stages/1-resman/data/stage-3/gke-dev.yaml b/fast/stages/1-resman-legacy/data/stage-3/gke-dev.yaml similarity index 100% rename from fast/stages/1-resman/data/stage-3/gke-dev.yaml rename to fast/stages/1-resman-legacy/data/stage-3/gke-dev.yaml diff --git a/fast/stages/1-resman/data/stage-3/secops-dev.yaml b/fast/stages/1-resman-legacy/data/stage-3/secops-dev.yaml similarity index 100% rename from fast/stages/1-resman/data/stage-3/secops-dev.yaml rename to fast/stages/1-resman-legacy/data/stage-3/secops-dev.yaml diff --git a/fast/stages/1-resman/data/top-level-folders/data-platform.yaml b/fast/stages/1-resman-legacy/data/top-level-folders/data-platform.yaml similarity index 100% rename from fast/stages/1-resman/data/top-level-folders/data-platform.yaml rename to fast/stages/1-resman-legacy/data/top-level-folders/data-platform.yaml diff --git a/fast/stages/1-resman/data/top-level-folders/gcve.yaml b/fast/stages/1-resman-legacy/data/top-level-folders/gcve.yaml similarity index 100% rename from fast/stages/1-resman/data/top-level-folders/gcve.yaml rename to fast/stages/1-resman-legacy/data/top-level-folders/gcve.yaml diff --git a/fast/stages/1-resman/data/top-level-folders/gke.yaml b/fast/stages/1-resman-legacy/data/top-level-folders/gke.yaml similarity index 100% rename from fast/stages/1-resman/data/top-level-folders/gke.yaml rename to fast/stages/1-resman-legacy/data/top-level-folders/gke.yaml diff --git a/fast/stages/1-resman/data/top-level-folders/sandbox.yaml b/fast/stages/1-resman-legacy/data/top-level-folders/sandbox.yaml similarity index 100% rename from fast/stages/1-resman/data/top-level-folders/sandbox.yaml rename to fast/stages/1-resman-legacy/data/top-level-folders/sandbox.yaml diff --git a/fast/stages/1-resman/data/top-level-folders/teams.yaml b/fast/stages/1-resman-legacy/data/top-level-folders/teams.yaml similarity index 100% rename from fast/stages/1-resman/data/top-level-folders/teams.yaml rename to fast/stages/1-resman-legacy/data/top-level-folders/teams.yaml diff --git a/fast/stages/1-resman/diagram.png b/fast/stages/1-resman-legacy/diagram.png similarity index 100% rename from fast/stages/1-resman/diagram.png rename to fast/stages/1-resman-legacy/diagram.png diff --git a/fast/stages/1-resman/fast_version.txt b/fast/stages/1-resman-legacy/fast_version.txt similarity index 100% rename from fast/stages/1-resman/fast_version.txt rename to fast/stages/1-resman-legacy/fast_version.txt diff --git a/fast/stages/1-resman/main.tf b/fast/stages/1-resman-legacy/main.tf similarity index 100% rename from fast/stages/1-resman/main.tf rename to fast/stages/1-resman-legacy/main.tf diff --git a/fast/stages/1-resman/moved/v33.0.0-v34.0.0.tf b/fast/stages/1-resman-legacy/moved/v33.0.0-v34.0.0.tf similarity index 100% rename from fast/stages/1-resman/moved/v33.0.0-v34.0.0.tf rename to fast/stages/1-resman-legacy/moved/v33.0.0-v34.0.0.tf diff --git a/fast/stages/1-resman/moved/v35.1.0-v36.0.0.tf b/fast/stages/1-resman-legacy/moved/v35.1.0-v36.0.0.tf similarity index 100% rename from fast/stages/1-resman/moved/v35.1.0-v36.0.0.tf rename to fast/stages/1-resman-legacy/moved/v35.1.0-v36.0.0.tf diff --git a/fast/stages/1-resman/moved/v36.0.1-v37.0.0.tf b/fast/stages/1-resman-legacy/moved/v36.0.1-v37.0.0.tf similarity index 100% rename from fast/stages/1-resman/moved/v36.0.1-v37.0.0.tf rename to fast/stages/1-resman-legacy/moved/v36.0.1-v37.0.0.tf diff --git a/fast/stages/1-resman/moved/v37.4.0-v38.0.0.tf b/fast/stages/1-resman-legacy/moved/v37.4.0-v38.0.0.tf similarity index 100% rename from fast/stages/1-resman/moved/v37.4.0-v38.0.0.tf rename to fast/stages/1-resman-legacy/moved/v37.4.0-v38.0.0.tf diff --git a/fast/stages/1-resman/organization-tags.tf b/fast/stages/1-resman-legacy/organization-tags.tf similarity index 100% rename from fast/stages/1-resman/organization-tags.tf rename to fast/stages/1-resman-legacy/organization-tags.tf diff --git a/fast/stages/1-resman/organization.tf b/fast/stages/1-resman-legacy/organization.tf similarity index 96% rename from fast/stages/1-resman/organization.tf rename to fast/stages/1-resman-legacy/organization.tf index 432b12628..067036900 100644 --- a/fast/stages/1-resman/organization.tf +++ b/fast/stages/1-resman-legacy/organization.tf @@ -17,6 +17,10 @@ # tfdoc:file:description Organization-level IAM and org policies. locals { + condition_vars = { + organization = var.organization + tag_names = var.tag_names + } # combine org-level IAM additive from billing and stage 2s iam_bindings_additive = merge( merge([ diff --git a/fast/stages/1-resman/outputs-cicd.tf b/fast/stages/1-resman-legacy/outputs-cicd.tf similarity index 100% rename from fast/stages/1-resman/outputs-cicd.tf rename to fast/stages/1-resman-legacy/outputs-cicd.tf diff --git a/fast/stages/1-resman/outputs-files.tf b/fast/stages/1-resman-legacy/outputs-files.tf similarity index 100% rename from fast/stages/1-resman/outputs-files.tf rename to fast/stages/1-resman-legacy/outputs-files.tf diff --git a/fast/stages/1-resman/outputs-providers.tf b/fast/stages/1-resman-legacy/outputs-providers.tf similarity index 100% rename from fast/stages/1-resman/outputs-providers.tf rename to fast/stages/1-resman-legacy/outputs-providers.tf diff --git a/fast/stages/1-resman/outputs.tf b/fast/stages/1-resman-legacy/outputs.tf similarity index 100% rename from fast/stages/1-resman/outputs.tf rename to fast/stages/1-resman-legacy/outputs.tf diff --git a/fast/stages/1-resman/schemas/fast-stage2.schema.json b/fast/stages/1-resman-legacy/schemas/fast-stage2.schema.json similarity index 100% rename from fast/stages/1-resman/schemas/fast-stage2.schema.json rename to fast/stages/1-resman-legacy/schemas/fast-stage2.schema.json diff --git a/fast/stages/1-resman/schemas/fast-stage2.schema.md b/fast/stages/1-resman-legacy/schemas/fast-stage2.schema.md similarity index 100% rename from fast/stages/1-resman/schemas/fast-stage2.schema.md rename to fast/stages/1-resman-legacy/schemas/fast-stage2.schema.md diff --git a/fast/stages/1-resman/schemas/fast-stage3.schema.json b/fast/stages/1-resman-legacy/schemas/fast-stage3.schema.json similarity index 100% rename from fast/stages/1-resman/schemas/fast-stage3.schema.json rename to fast/stages/1-resman-legacy/schemas/fast-stage3.schema.json diff --git a/fast/stages/1-resman/schemas/fast-stage3.schema.md b/fast/stages/1-resman-legacy/schemas/fast-stage3.schema.md similarity index 100% rename from fast/stages/1-resman/schemas/fast-stage3.schema.md rename to fast/stages/1-resman-legacy/schemas/fast-stage3.schema.md diff --git a/fast/stages/1-resman-legacy/schemas/org-policies.schema.json b/fast/stages/1-resman-legacy/schemas/org-policies.schema.json new file mode 120000 index 000000000..c5ebcfaf7 --- /dev/null +++ b/fast/stages/1-resman-legacy/schemas/org-policies.schema.json @@ -0,0 +1 @@ +../../../../modules/organization/schemas/org-policies.schema.json \ No newline at end of file diff --git a/fast/stages/1-resman/schemas/org-policies.schema.md b/fast/stages/1-resman-legacy/schemas/org-policies.schema.md similarity index 100% rename from fast/stages/1-resman/schemas/org-policies.schema.md rename to fast/stages/1-resman-legacy/schemas/org-policies.schema.md diff --git a/fast/stages/1-resman/schemas/tags.schema.json b/fast/stages/1-resman-legacy/schemas/tags.schema.json similarity index 100% rename from fast/stages/1-resman/schemas/tags.schema.json rename to fast/stages/1-resman-legacy/schemas/tags.schema.json diff --git a/fast/stages/1-resman/schemas/tags.schema.md b/fast/stages/1-resman-legacy/schemas/tags.schema.md similarity index 100% rename from fast/stages/1-resman/schemas/tags.schema.md rename to fast/stages/1-resman-legacy/schemas/tags.schema.md diff --git a/fast/stages/1-resman/schemas/top-level-folder.schema.json b/fast/stages/1-resman-legacy/schemas/top-level-folder.schema.json similarity index 100% rename from fast/stages/1-resman/schemas/top-level-folder.schema.json rename to fast/stages/1-resman-legacy/schemas/top-level-folder.schema.json diff --git a/fast/stages/1-resman/schemas/top-level-folder.schema.md b/fast/stages/1-resman-legacy/schemas/top-level-folder.schema.md similarity index 100% rename from fast/stages/1-resman/schemas/top-level-folder.schema.md rename to fast/stages/1-resman-legacy/schemas/top-level-folder.schema.md diff --git a/fast/stages/1-resman/stage-2.tf b/fast/stages/1-resman-legacy/stage-2.tf similarity index 99% rename from fast/stages/1-resman/stage-2.tf rename to fast/stages/1-resman-legacy/stage-2.tf index 139bd9357..afc78b18b 100644 --- a/fast/stages/1-resman/stage-2.tf +++ b/fast/stages/1-resman-legacy/stage-2.tf @@ -174,6 +174,9 @@ module "stage2-folder" { ) ) name = each.value.folder_config.name + context = { + condition_vars = local.condition_vars + } iam = { for k, v in each.value.folder_config.iam : lookup(var.custom_roles, k, k) => [ diff --git a/fast/stages/1-resman/stage-3.tf b/fast/stages/1-resman-legacy/stage-3.tf similarity index 99% rename from fast/stages/1-resman/stage-3.tf rename to fast/stages/1-resman-legacy/stage-3.tf index 97e4f7c39..430b3d993 100644 --- a/fast/stages/1-resman/stage-3.tf +++ b/fast/stages/1-resman-legacy/stage-3.tf @@ -157,6 +157,9 @@ module "stage3-folder" { ) ) name = each.value.folder_config.name + context = { + condition_vars = local.condition_vars + } iam = { # merge inputs/factory bindings with static role bindings in loocal._stage_3_iam for role in concat(keys(each.value.folder_config.iam), keys(local._stage_3_iam[each.key])) : @@ -167,7 +170,6 @@ module "stage3-folder" { ) : lookup(local.principals_iam, m, m) ] } - iam_bindings = { for k, v in each.value.folder_config.iam_bindings : k => merge(v, { members = [ diff --git a/fast/stages/1-resman/stage-cicd.tf b/fast/stages/1-resman-legacy/stage-cicd.tf similarity index 100% rename from fast/stages/1-resman/stage-cicd.tf rename to fast/stages/1-resman-legacy/stage-cicd.tf diff --git a/fast/addons/1-resman-tenants/templates/providers.tf.tpl b/fast/stages/1-resman-legacy/templates/providers.tf.tpl similarity index 100% rename from fast/addons/1-resman-tenants/templates/providers.tf.tpl rename to fast/stages/1-resman-legacy/templates/providers.tf.tpl diff --git a/fast/stages/0-bootstrap/templates/providers_terraform.tf.tpl b/fast/stages/1-resman-legacy/templates/providers_terraform.tf.tpl similarity index 100% rename from fast/stages/0-bootstrap/templates/providers_terraform.tf.tpl rename to fast/stages/1-resman-legacy/templates/providers_terraform.tf.tpl diff --git a/fast/stages/1-resman/templates/workflow-github.yaml b/fast/stages/1-resman-legacy/templates/workflow-github.yaml similarity index 100% rename from fast/stages/1-resman/templates/workflow-github.yaml rename to fast/stages/1-resman-legacy/templates/workflow-github.yaml diff --git a/fast/stages/0-bootstrap/templates/workflow-gitlab.yaml b/fast/stages/1-resman-legacy/templates/workflow-gitlab.yaml similarity index 100% rename from fast/stages/0-bootstrap/templates/workflow-gitlab.yaml rename to fast/stages/1-resman-legacy/templates/workflow-gitlab.yaml diff --git a/fast/stages/1-resman/tenant-logging.tf b/fast/stages/1-resman-legacy/tenant-logging.tf similarity index 100% rename from fast/stages/1-resman/tenant-logging.tf rename to fast/stages/1-resman-legacy/tenant-logging.tf diff --git a/fast/stages/1-resman/tenant-root.tf b/fast/stages/1-resman-legacy/tenant-root.tf similarity index 100% rename from fast/stages/1-resman/tenant-root.tf rename to fast/stages/1-resman-legacy/tenant-root.tf diff --git a/fast/stages/1-resman/top-level-folders.tf b/fast/stages/1-resman-legacy/top-level-folders.tf similarity index 99% rename from fast/stages/1-resman/top-level-folders.tf rename to fast/stages/1-resman-legacy/top-level-folders.tf index 0e7b753f3..c927b494f 100644 --- a/fast/stages/1-resman/top-level-folders.tf +++ b/fast/stages/1-resman-legacy/top-level-folders.tf @@ -83,6 +83,9 @@ module "top-level-folder" { logging_exclusions = each.value.logging_exclusions logging_settings = each.value.logging_settings logging_sinks = each.value.logging_sinks + context = { + condition_vars = local.condition_vars + } iam = { for role, members in each.value.iam : lookup(var.custom_roles, role, role) => [ diff --git a/fast/stages/1-resman/variables-addons.tf b/fast/stages/1-resman-legacy/variables-addons.tf similarity index 100% rename from fast/stages/1-resman/variables-addons.tf rename to fast/stages/1-resman-legacy/variables-addons.tf diff --git a/fast/stages/1-resman/variables-fast.tf b/fast/stages/1-resman-legacy/variables-fast.tf similarity index 100% rename from fast/stages/1-resman/variables-fast.tf rename to fast/stages/1-resman-legacy/variables-fast.tf diff --git a/fast/stages/1-resman/variables-stages.tf b/fast/stages/1-resman-legacy/variables-stages.tf similarity index 100% rename from fast/stages/1-resman/variables-stages.tf rename to fast/stages/1-resman-legacy/variables-stages.tf diff --git a/fast/stages/1-resman/variables-toplevel-folders.tf b/fast/stages/1-resman-legacy/variables-toplevel-folders.tf similarity index 100% rename from fast/stages/1-resman/variables-toplevel-folders.tf rename to fast/stages/1-resman-legacy/variables-toplevel-folders.tf diff --git a/fast/stages/1-resman/variables.tf b/fast/stages/1-resman-legacy/variables.tf similarity index 100% rename from fast/stages/1-resman/variables.tf rename to fast/stages/1-resman-legacy/variables.tf diff --git a/fast/stages/1-resman/templates/providers.tf.tpl b/fast/stages/1-resman/templates/providers.tf.tpl deleted file mode 100644 index d1c224c5c..000000000 --- a/fast/stages/1-resman/templates/providers.tf.tpl +++ /dev/null @@ -1,33 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -terraform { - backend "gcs" { - bucket = "${bucket}" - impersonate_service_account = "${sa}" - %{~ if backend_extra != null ~} - ${indent(4, backend_extra)} - %{~ endif ~} - } -} -provider "google" { - impersonate_service_account = "${sa}" -} -provider "google-beta" { - impersonate_service_account = "${sa}" -} - -# end provider.tf for ${name} diff --git a/fast/stages/1-resman/templates/providers_terraform.tf.tpl b/fast/stages/1-resman/templates/providers_terraform.tf.tpl deleted file mode 100644 index b581e50ed..000000000 --- a/fast/stages/1-resman/templates/providers_terraform.tf.tpl +++ /dev/null @@ -1,44 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -terraform { - cloud { - organization = "${organization}" - %{~ if hostname != null ~} - hostname = "${hostname}" - %{~ endif ~} - workspaces { - %{~ if workspaces.name != null ~} - name = "${workspaces.name}" - %{~ endif ~} - %{~ if workspaces.tags != null ~} - tags = [ %{ for tags in workspaces.tags ~} "${tags}", %{ endfor ~} ] - %{~ endif ~} - %{~ if workspaces.project != null ~} - project = "${workspaces.project}" - %{~ endif ~} - } - } -} - -provider "google" { - impersonate_service_account = "${sa}" -} -provider "google-beta" { - impersonate_service_account = "${sa}" -} - -# end provider.tf for ${name} \ No newline at end of file diff --git a/fast/stages/1-resman/templates/workflow-gitlab.yaml b/fast/stages/1-resman/templates/workflow-gitlab.yaml deleted file mode 100644 index 150340835..000000000 --- a/fast/stages/1-resman/templates/workflow-gitlab.yaml +++ /dev/null @@ -1,106 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -variables: - GOOGLE_CREDENTIALS: cicd-sa-credentials.json - FAST_OUTPUTS_BUCKET: ${outputs_bucket} - FAST_WIF_PROVIDER: ${identity_provider} - SSH_AUTH_SOCK: /tmp/ssh_agent.sock - %{~ if tf_var_files != [] ~} - TF_VAR_FILES: ${join("\n ", tf_var_files)} - %{~ endif ~} - -workflow: - rules: - # merge / apply - - if: $CI_PIPELINE_SOURCE == 'push' && $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH - variables: - COMMAND: apply - FAST_SERVICE_ACCOUNT: ${service_accounts.apply} - TF_PROVIDERS_FILE: ${tf_providers_files.apply} - # pr / plan - - if: $CI_PIPELINE_SOURCE == 'merge_request_event' - variables: - COMMAND: plan - FAST_SERVICE_ACCOUNT: ${service_accounts.plan} - TF_PROVIDERS_FILE: ${tf_providers_files.plan} - -stages: - - gcp-setup - - tf-plan-apply - -# TODO: document project-level deploy key used to fetch modules - -gcp-setup: - stage: gcp-setup - image: - name: google/cloud-sdk:slim - artifacts: - paths: - - cicd-sa-credentials.json - - providers.tf - %{~ for f in tf_var_files ~} - - ${f} - %{~ endfor ~} - id_tokens: - GITLAB_TOKEN: - aud: - %{~ for aud in audiences ~} - - ${aud} - %{~ endfor ~} - before_script: - - echo "$GITLAB_TOKEN" > token.txt - script: - - | - gcloud iam workload-identity-pools create-cred-config \ - $FAST_WIF_PROVIDER \ - --service-account=$FAST_SERVICE_ACCOUNT \ - --service-account-token-lifetime-seconds=900 \ - --output-file=$GOOGLE_CREDENTIALS \ - --credential-source-file=token.txt - - gcloud config set auth/credential_file_override $GOOGLE_CREDENTIALS - - gcloud storage cp -r "gs://$FAST_OUTPUTS_BUCKET/providers/$TF_PROVIDERS_FILE" ./providers.tf - %{~ for f in tf_var_files ~} - - gcloud storage cp gs://$FAST_OUTPUTS_BUCKET/tfvars/${f} ./ - %{~ endfor ~} - - -tf-plan-apply: - stage: tf-plan-apply - dependencies: - - gcp-setup - id_tokens: - GITLAB_TOKEN: - aud: - %{~ for aud in audiences ~} - - ${aud} - %{~ endfor ~} - image: - name: hashicorp/terraform - entrypoint: - - "/usr/bin/env" - variables: - SSH_AUTH_SOCK: /tmp/ssh-agent.sock - script: - - | - ssh-agent -a $SSH_AUTH_SOCK - echo "$CICD_MODULES_KEY" | ssh-add - - mkdir -p ~/.ssh - ssh-keyscan -H 'gitlab.com' >> ~/.ssh/known_hosts - ssh-keyscan gitlab.com | sort -u - ~/.ssh/known_hosts -o ~/.ssh/known_hosts - - echo "$GITLAB_TOKEN" > token.txt - - terraform init - - terraform validate - - "if [ $COMMAND == 'plan' ]; then terraform plan -input=false -no-color -lock=false; fi" - - "if [ $COMMAND == 'apply' ]; then terraform apply -input=false -no-color -auto-approve; fi" diff --git a/fast/stages/2-networking-a-simple/README.md b/fast/stages/2-networking-a-simple/README.md index 6a4b08f06..befea92c7 100644 --- a/fast/stages/2-networking-a-simple/README.md +++ b/fast/stages/2-networking-a-simple/README.md @@ -272,7 +272,7 @@ The [Inbound DNS Policy](https://cloud.google.com/dns/docs/server-policies-overv ## How to run this stage -This stage is meant to be executed after the [resource management](../1-resman) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured in the [bootstrap](../0-bootstrap) stage. +This stage is meant to be executed after the [bootstrap](../0-bootstrap) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured there. It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the previous stages for the environmental requirements. diff --git a/fast/stages/2-networking-b-nva/README.md b/fast/stages/2-networking-b-nva/README.md index 779a903dd..23a6618b7 100644 --- a/fast/stages/2-networking-b-nva/README.md +++ b/fast/stages/2-networking-b-nva/README.md @@ -339,7 +339,7 @@ The [Inbound DNS Policy](https://cloud.google.com/dns/docs/server-policies-overv ## How to run this stage -This stage is meant to be executed after the [resource management](../1-resman) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured in the [bootstrap](../0-bootstrap) stage. +This stage is meant to be executed after the [bootstrap](../0-bootstrap) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured there. It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the previous stages for the environmental requirements. diff --git a/fast/stages/2-networking-c-separate-envs/README.md b/fast/stages/2-networking-c-separate-envs/README.md index 6e3c7db7b..74632f3de 100644 --- a/fast/stages/2-networking-c-separate-envs/README.md +++ b/fast/stages/2-networking-c-separate-envs/README.md @@ -165,7 +165,7 @@ The [Inbound DNS Policy](https://cloud.google.com/dns/docs/server-policies-overv ## How to run this stage -This stage is meant to be executed after the [resource management](../1-resman) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured in the [bootstrap](../0-bootstrap) stage. +This stage is meant to be executed after the [bootstrap](../0-bootstrap) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured there. It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the previous stages for the environmental requirements. diff --git a/fast/stages/2-project-factory-experimental/main.tf b/fast/stages/2-project-factory-experimental/main.tf deleted file mode 100644 index bcea2574f..000000000 --- a/fast/stages/2-project-factory-experimental/main.tf +++ /dev/null @@ -1,77 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Project factory. - -module "factory" { - source = "../../../modules/project-factory-experimental" - context = { - custom_roles = merge( - var.custom_roles, var.context.custom_roles - ) - folder_ids = merge( - var.folder_ids, var.context.folder_ids - ) - iam_principals = merge( - var.iam_principals, - { - for k, v in var.service_accounts : - k => "serviceAccount:${v}" if v != null - }, - var.context.iam_principals - ) - kms_keys = merge( - var.kms_keys, var.context.kms_keys - ) - locations = merge( - var.locations, var.context.locations - ) - notification_channels = var.context.notification_channels - project_ids = merge( - var.project_ids, var.host_project_ids, var.context.project_ids - ) - tag_values = merge( - var.tag_values, var.context.tag_values - ) - vpc_sc_perimeters = merge( - var.perimeters, var.context.vpc_sc_perimeters - ) - } - data_defaults = { - # more defaults are available, check the project factory variables - billing_account = var.billing_account.id - storage_location = var.locations.storage - } - data_merges = { - services = [ - "logging.googleapis.com", - "monitoring.googleapis.com" - ] - } - data_overrides = { - prefix = var.prefix - } - factories_config = merge(var.factories_config, { - budgets = { - billing_account_id = try( - var.factories_config.budgets.billing_account_id, var.billing_account.id - ) - data = try( - var.factories_config.budgets.data, "data/budgets" - ) - } - }) -} diff --git a/fast/stages/2-project-factory-experimental/variables.tf b/fast/stages/2-project-factory-experimental/variables.tf deleted file mode 100644 index ec57ebdfe..000000000 --- a/fast/stages/2-project-factory-experimental/variables.tf +++ /dev/null @@ -1,60 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -variable "context" { - description = "Context-specific interpolations." - type = object({ - custom_roles = optional(map(string), {}) - folder_ids = optional(map(string), {}) - iam_principals = optional(map(string), {}) - kms_keys = optional(map(string), {}) - locations = optional(map(string), {}) - notification_channels = optional(map(string), {}) - project_ids = optional(map(string), {}) - tag_values = optional(map(string), {}) - vpc_host_projects = optional(map(string), {}) - vpc_sc_perimeters = optional(map(string), {}) - }) - default = {} - nullable = false -} - -variable "factories_config" { - description = "Path to folder with YAML resource description data files." - type = object({ - folders = optional(string, "data/folders") - projects = optional(string, "data/projects") - budgets = optional(object({ - billing_account_id = string - data = string - })) - }) - nullable = false - default = {} -} - -# variable "outputs_location" { -# description = "Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable." -# type = string -# default = null -# } - -variable "stage_name" { - description = "FAST stage name. Used to separate output files across different factories." - type = string - nullable = false - default = "2-project-factory" -} diff --git a/fast/stages/2-project-factory-experimental/.fast-stage.env b/fast/stages/2-project-factory-legacy/.fast-stage.env similarity index 76% rename from fast/stages/2-project-factory-experimental/.fast-stage.env rename to fast/stages/2-project-factory-legacy/.fast-stage.env index 9ef2979f5..24820e777 100644 --- a/fast/stages/2-project-factory-experimental/.fast-stage.env +++ b/fast/stages/2-project-factory-legacy/.fast-stage.env @@ -1,5 +1,5 @@ FAST_STAGE_DESCRIPTION="project factory (org level)" FAST_STAGE_LEVEL=2 FAST_STAGE_NAME=project-factory -FAST_STAGE_DEPS="0-globals 0-bootstrap" +FAST_STAGE_DEPS="0-globals 0-bootstrap 1-resman" FAST_STAGE_OPTIONAL="1-vpcsc 2-networking 2-security" \ No newline at end of file diff --git a/fast/stages/2-project-factory-experimental/README.md b/fast/stages/2-project-factory-legacy/README.md similarity index 53% rename from fast/stages/2-project-factory-experimental/README.md rename to fast/stages/2-project-factory-legacy/README.md index 1f3826c4d..544552344 100644 --- a/fast/stages/2-project-factory-experimental/README.md +++ b/fast/stages/2-project-factory-legacy/README.md @@ -3,11 +3,7 @@ - [Design overview and choices](#design-overview-and-choices) - [How to run this stage](#how-to-run-this-stage) - - [Bootstrap stage configuration](#bootstrap-stage-configuration) - - [Automation resources](#automation-resources) - - [Billing account](#billing-account) - - [Organization IAM](#organization-iam) - - [Parent folder](#parent-folder) + - [Resource Management stage configuration](#resource-management-stage-configuration) - [Factory configuration](#factory-configuration) - [Stage provider and Terraform variables](#stage-provider-and-terraform-variables) - [Managing folders and projects](#managing-folders-and-projects) @@ -15,6 +11,8 @@ - [Folder parent-child relationship and variable substitutions](#folder-parent-child-relationship-and-variable-substitutions) - [Project Creation](#project-creation) - [Automation Resources for Projects](#automation-resources-for-projects) +- [Alternative patterns](#alternative-patterns) + - [Per-environment Factories](#per-environment-factories) - [Files](#files) - [Variables](#variables) - [Outputs](#outputs) @@ -24,123 +22,76 @@ The Project Factory stage allows simplified management of folder hierarchies and The pattern implemented here by default allows management of a teams (or business units, applications, etc.) hierarchy. Different patterns are possible, and this document also tries to provide some guidance on how to implement them. +

+ Project factory teams pattern +

+ ## Design overview and choices -The project factory optionally "consumes" resources created by preceding stages, by using their outputs as a source for [context interpolation](../../../modules/project-factory-experimental/README.md#context-based-interpolation): +The project factory is "primed" by the resource management stage via -- folder ids from the bootstrap stage and via `var.context.folder_ids` -- project ids from the bootstrap and networking stages and via `var.context.project_ids` -- IAM principals from the bootstrap stage and via `var.context.iam_principals` -- tag values from the bootstrap stage and via `var.context.tag_values` -- KMS keys from the security stage and via `var.context.kms_keys` -- VPC SC perimeters from the VPC SC stage and via `var.context.vpc_sc_perimeters` +- a set of service accounts with different scopes +- one or more user-defined top-level folders where those service accounts operate -Additionally, some of the values defined earlier in the FAST apply cycle are set here as project defaults: +This stage does not directly depend on other stage 2 like networking and security, but it can optionally leverage resources created there like Shared VPC host projects, which are used to define service projects. -- prefix (as override) -- billing account -- storage location - -The project factory stage is a thin wrapper of the underlying [project-factory module](../../../modules/project-factory-experimental/), which in turn exposes the full interface of the [project](../../../modules/project/) and [folder](../../../modules/folder/) modules. +The project factory stage is a thin wrapper of the underlying [project-factory module](../../../modules/project-factory/), which in turn exposes the full interface of the [project](../../../modules/project/) and [folder](../../../modules/folder/) modules. ## How to run this stage -This stage is meant to be executed after the [bootstrap](../0-bootstrap/) stage. If any of the VPC SC, networking, and security stages have been applied, their resources can be directly leveraged via context interpolation as explained above. +This stage is meant to be executed after the [bootstrap](../0-bootstrap) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured there. -### Bootstrap stage configuration +### Resource Management stage configuration -The bootstrap stage already contains the project factory automation resources, a sample "Teams" folder defined via YAML, and all the required IAM wiring to make this stage functional. The default "Teams" setup can be extended, or used as an example to implement different designs. +The resource management stage already contains a sample "Teams" folder defined via YAML, which can be used as-is or modified to provide a top-level folder for the project factory. More folders can of course be added, and Terraform variables used instead of or in addition to YAML files in the resource management stage. -The bootstrap-specific setup is reproduced here to aid using it as a starting point. Only snippets relevant to this stage are shown below for simplicity. - -#### Automation resources - -The default design uses two service accounts (read-write and read-only) and a Cloud Storage folder in a pre-existing bucket, to enable this stage for Infrastructure as Code. +This is the teams YAML in resource management, leveraging attribute substitutions from provided context for the project factory service account and tag value. ```yaml -# data/projects/core/iac-0.yaml -buckets: - iac-stage-state: - description: Terraform state for stage automation. - managed_folders: - 2-project-factory: - iam: - roles/storage.admin: - - $iam_principals:service_accounts/iac-0/iac-pf-rw - $custom_roles:storage_viewer: - - $iam_principals:service_accounts/iac-0/iac-pf-ro - iac-outputs: - description: Terraform state for the org-level automation. - iam: - roles/storage.admin: - - $iam_principals:service_accounts/iac-0/iac-pf-rw - $custom_roles:storage_viewer: - - $iam_principals:service_accounts/iac-0/iac-pf-ro -service_accounts: - iac-pf-ro: - display_name: IaC service account for project factory (read-only). - iac-pf-rw: - display_name: IaC service account for project factory (read-write). -``` - -#### Billing account - -If an externally managed billing account is used, billing user permissions need to be assigned to the project factory service account. - -```yaml -# data/billing-accounts/default.yaml -id: $defaults:billing_account -iam_bindings_additive: - billing_user_pf_sa: - role: roles/billing.user - member: $iam_principals:service_accounts/iac-0/iac-pf-rw -``` - -#### Organization IAM - -This stage only needs conditional grants for organization policy management at the organization level. Additionally, if an organization-managed billing account is used the IAM bindings described in the section above can be omitted, and moved to the organization. - -```yaml -# data/organization/.config.yaml -iam_bindings: - pf_org_policy_admin: - role: roles/orgpolicy.policyAdmin - members: - - $iam_principals:service_accounts/iac-0/iac-pf-rw - condition: - expression: resource.matchTag('${organization}/context', 'project-factory') - title: Project factory org policy admin - pf_org_policy_viewer: - role: roles/orgpolicy.policyViewer - members: - - $iam_principals:service_accounts/iac-0/iac-pf-ro - condition: - expression: resource.matchTag('${organization}/context', 'project-factory') - title: Project factory org policy viewer -``` - -#### Parent folder - -A single "Teams" folder is created here. Multiple folders (or sub-folders) can of course be created by replicating the IAM configuration below for each. - -```yaml -# data/folders/teams/.config.yaml name: Teams -iam_by_principals: - $iam_principals:service_accounts/iac-0/iac-pf-rw: - - roles/owner - - roles/resourcemanager.folderAdmin - - roles/resourcemanager.projectCreator - - roles/resourcemanager.tagUser - - $custom_roles:service_project_network_admin - $iam_principals:service_accounts/iac-0/iac-pf-ro: - - roles/viewer - - roles/resourcemanager.folderViewer - - roles/resourcemanager.tagViewer +automation: + enable: false +iam: + "roles/owner": + - project-factory + "roles/resourcemanager.folderAdmin": + - project-factory + "roles/resourcemanager.projectCreator": + - project-factory + "roles/resourcemanager.tagUser": + - project-factory + "service_project_network_admin": + - project-factory tag_bindings: - context: $tag_values:context/project-factory + context: context/project-factory ``` +This is the alternative version that can be used instead of the YAML file above. + +```tfvars +top_level_folders = { + # more top-level folders might be present here + teams = { + name = "Teams" + iam = { + "roles/owner" = ["project-factory"] + "roles/resourcemanager.folderAdmin" = ["project-factory"] + "roles/resourcemanager.projectCreator" = ["project-factory"] + "roles/resourcemanager.tagUser" = ["project-factory"] + "service_project_network_admin" = ["project-factory"] + } + tag_bindings = { + context = "context/project-factory" + } + } +} +# tftest skip +``` + +You can of course extend these snippets to grant additional roles to groups or other service accounts via the `iam`, `iam_by_principals`, and `iam_bindings` folder-level variables. + +The project factory tag binding on the folder allows management of organization policies in the Teams hierarchy. If this functionality is not needed, the tag binding can be safely omitted. + ### Factory configuration The `data` folder in this stage contains factory files that can be used as examples to implement the team-based design shown above. Before running `terraform apply` check the YAML files, as project names and other attributes will need basic editing to match your desired setup. @@ -162,6 +113,7 @@ ln -s ~/fast-config/fast-test-00/providers/2-project-factory-providers.tf ./ # input files from other stages ln -s ~/fast-config/fast-test-00/tfvars/0-globals.auto.tfvars.json ./ ln -s ~/fast-config/fast-test-00/tfvars/0-bootstrap.auto.tfvars.json ./ +ln -s ~/fast-config/fast-test-00/tfvars/1-resman.auto.tfvars.json ./ # conventional place for stage tfvars (manually created) ln -s ~/fast-config/fast-test-00/2-project-factory.auto.tfvars ./ @@ -169,7 +121,6 @@ ln -s ~/fast-config/fast-test-00/2-project-factory.auto.tfvars ./ # optional files ln -s ~/fast-config/fast-test-00/2-networking.auto.tfvars.json ./ ln -s ~/fast-config/fast-test-00/2-security.auto.tfvars.json ./ -ln -s ~/fast-config/fast-test-00/2-vpcsc.auto.tfvars.json ./ ``` ```bash @@ -183,6 +134,7 @@ gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/2-project-factory-p # input files from other stages gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json ./ gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./ +gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json ./ # conventional place for stage tfvars (manually created) gcloud storage cp gs://xxx-prod-iac-core-outputs-0/2-project-factory.auto.tfvars ./ @@ -190,7 +142,6 @@ gcloud storage cp gs://xxx-prod-iac-core-outputs-0/2-project-factory.auto.tfvars # optional files gcloud storage cp gs://xxx-prod-iac-core-outputs-0/2-networking.auto.tfvars.json ./ gcloud storage cp gs://xxx-prod-iac-core-outputs-0/2-security.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/2-vpcsc.auto.tfvars.json ./ ``` If you're not using FAST, refer to the [Variables](#variables) table at the bottom of this document for a full list of variables, their origin (e.g., a stage or specific to this one), and descriptions explaining their meaning. @@ -210,26 +161,26 @@ The YAML data files are self-explanatory and the included [schema files](./schem ### Folder and hierarchy management -The project factory manages its folder hierarchy via a filesystem tree, rooted in the path defined via the `factories_config.folders` variable. +The project factory manages its folder hierarchy via a filesystem tree, rooted in the path defined via the `factories_config.hierarchy_data` variable. -Filesystem folders which contain a `.config.yaml` file are mapped to folders in the resource management hierarchy. Their YAML configuration files allow defining folder attributes like descriptive name, IAM bindings, organization policies, tag bindings. +Filesystem folders which contain a `_config.yaml` file are mapped to folders in the resource management hierarchy. Their YAML configuration files allow defining folder attributes like descriptive name, IAM bindings, organization policies, tag bindings. This is the simple filesystem hierarchy provided here as an example. ```bash hierarchy ├── team-a -│   ├── .config.yaml +│   ├── _config.yaml │   ├── dev -│   │   └── .config.yaml +│   │   └── _config.yaml │   └── prod -│   └── .config.yaml +│   └── _config.yaml └── team-b - ├── .config.yaml + ├── _config.yaml ├── dev - │   └── .config.yaml + │   └── _config.yaml └── prod - └── .config.yaml + └── _config.yaml ``` The approach is intentionally explicit and repetitive in order to simplify operations: copy/pasting an existing set of folders (or an ad hoc template) and changing a few YAML variables allows to quickly define new sub-hierarchy branches. Mass editing via search and replace functionality allows sweeping changes across the whole hierarchy. @@ -239,7 +190,7 @@ Where inheritance is leveraged in the overall design config files can be decepti ```yaml name: Development tag_bindings: - environment: $tag_values:environment/development + environment: environment/development iam_by_principals: "group:team-a-admins@example.com": - roles/editor @@ -253,10 +204,11 @@ In the example YAML configuration above there's no explicitly specified folder p But what about the "Team A" folder itself? From the point of view of the project factory it's a top-level folder attached to the root of its hierarchy (the "Teams" folder), so how does it know where to create it in the GCP hierarchy? -There are two different ways to pass this information to the project factory: +There are three different ways to pass this information to the project factory: -- in the YAML file itself, by explicitly setting the folder's `parent` attribute to the explicit numeric id of the "Teams" folder (e.g. `folders/1234567890`) -- in the YAML file itself, by using explicit context interpolation (e.g. `$folder_ids:teams`) +- in the YAML file itself, by explicitly setting the folder's `parent` attribute to the explicit numeric id of the "Teams" folder +- in the YAML file itself, by explicitly setting the folder's `parent` attribute to the short name of the "Teams" folder in the resource management stage's outputs +- in the stage Terraform variables, by setting the `default` folder for the project factory to the numeric id of the "Teams" folder This flexibility is what allows the project factory to manage folders under multiple roots, and to also be used for folders created outside of FAST. Imagine a scenario where there's no single "Teams" folder, but multiple ones for different subsidiaries, or for internal and external teams, etc. @@ -270,15 +222,30 @@ parent: folders/1234567890 ```yaml name: Team A -# use context interpolation from stage 0 tfvars (preferred approach) -parent: $folder_ids:teams +# use variable substitution from stage 1 tfvars (preferred approach) +parent: teams +``` + +The third way explained above does not explicitly define a root folder in the YAML files, but sets a default folder in the Terraform variables for the stage via the `factories_config.substitutions.folder_ids`, by adding a `default` key pointing to the folder id of the root ("Teams") folder. + +```tfvars +factories_config = { + substitutions = { + folder_ids = { + # id of the top-level Teams folder + # derived from the 1-resman.auto.tfvars.json file + default = "folders/12345678" + } + } +} +# tftest skip ``` ### Project Creation Project YAML files can be created in two different filesystem paths: -- in the filesystem folder defined via the `factories_config.projects` variable, and then explicitly setting their `parent` attribute in YAML files, or +- in the filesystem folder defined via the `factories_config.project_data` variable, and then explicitly setting their `parent` attribute in YAML files, or - in the filesystem hierarchy discussed above, so that their `parent` attribute is automatically derived from the containing folder The two approaches can be mixed and matched, but the first approach is safer as is avoids potentially dangerous situations when folders are deleted with project configuration files still inside. @@ -291,8 +258,8 @@ parent: folders/1234509876 ``` ```yaml -# use context interpolation from managed folders (preferred approach) -parent: $folder_ids:team-a/dev +# use variable substitution from managed folders (preferred approach) +parent: team-a/dev ``` All of the [project module](../../../modules/project/) attributes (and some service account attributes) can of course be leveraged in the configuration files. Refer to the [project schema](./schemas/folder.schema.json) for the complete set of available attributes. @@ -303,8 +270,8 @@ When created projects are meant to be managed via IaC downstream, an initial set ```yaml # controlling project shown in the diagram above -parent: $folder_ids:teams -name: $project_ids:iac-core-0 +parent: teams +name: xxx-prod-iac-teams-0 services: - compute.googleapis.com - storage.googleapis.com @@ -315,43 +282,69 @@ services: Once a controlling project is in place, it can be used in any other project declaration to host service accounts and bucket for automation. The service accounts can be used in IAM bindings in the same file by referring to their name via substitutions, as shown here. ```yaml -# file name: dev-ta-app-0.yaml (implicitly used for project id) # team or application-level project with automation resources -parent: $folder_ids:team-a/dev +parent: team-a/dev # project prefix is forced via override in `main.tf` +name: dev-ta-app-0 iam: roles/owner: # refer to the rw service account defined below - - $iam_principals:service_accounts/dev-ta-app-0/rw + - rw roles/viewer: # refer to the ro service account defined below - - $iam_principals:service_accounts/dev-ta-app-0/ro + - ro automation: - project: $project_ids:iac-core-0 + # no context is possible here + # use the complete project id + project: xxx-prod-iac-teams-0 service_accounts: + # resulting sa name: xxx-dev-ta-app-0-rw rw: description: Read/write automation sa for team a app 0. + # resulting sa name: xxx-dev-ta-app-0-ro ro: description: Read-only automation sa for team a app 0. bucket: + # resulting bucket name: xxx-dev-ta-app-0-state description: Terraform state bucket for team a app 0. iam: + # service accounts can use short name substitutions from context roles/storage.objectCreator: - - $iam_principals:service_accounts/dev-ta-app-0/rw + - rw roles/storage.objectViewer: - - $iam_principals:service_accounts/dev-ta-app-0/rw - - $iam_principals:service_accounts/dev-ta-app-0/ro + - rw + - ro - group:devops@example.org ``` +## Alternative patterns + +Some alternative patterns are captured here, the list will grow as we generalize approaches seen in the field. + +### Per-environment Factories + +A variation of this pattern uses separate project factories for each environment, as in the following diagram. + +

+ Project factory team-level per environment. +

+ +This approach leverages the per-environment project factory service accounts and tags created by the resource management stage, so that + +- the Teams folder hierarchy and IaC project are managed by a cross-environment factory using the "main" project factory service account +- IAM permissions are set on the environment folders to grant control to the prod and dev project factory service accounts +- one additional factory per environment manages project creation leveraging the folders created above + +The approach is not shown here but reasonably easy to implement. The main project factory output file can also be used to set up folder id susbtitution in the per-environment factories. + ## Files | name | description | modules | resources | |---|---|---|---| -| [main.tf](./main.tf) | Project factory. | project-factory-experimental | | -| [outputs.tf](./outputs.tf) | Module outputs. | | google_storage_bucket_object | +| [main.tf](./main.tf) | Project factory. | project-factory-legacy | | +| [outputs.tf](./outputs.tf) | Module outputs. | | google_storage_bucket_object · local_file | | [variables-fast.tf](./variables-fast.tf) | None | | | | [variables.tf](./variables.tf) | Module variables. | | | @@ -360,25 +353,27 @@ automation: | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| | [automation](variables-fast.tf#L17) | Automation resources created by the bootstrap stage. | object({…}) | ✓ | | 0-bootstrap | -| [billing_account](variables-fast.tf#L26) | Billing account id. | object({…}) | ✓ | | 0-bootstrap | -| [prefix](variables-fast.tf#L92) | Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants. | string | ✓ | | 0-bootstrap | -| [context](variables.tf#L17) | Context-specific interpolations. | object({…}) | | {} | | -| [custom_roles](variables-fast.tf#L34) | Custom roles defined at the org level, in key => id format. | map(string) | | {} | 0-bootstrap | -| [factories_config](variables.tf#L35) | Path to folder with YAML resource description data files. | object({…}) | | {} | | -| [folder_ids](variables-fast.tf#L42) | Folders created in the bootstrap stage. | map(string) | | {} | 0-bootstrap | -| [host_project_ids](variables-fast.tf#L58) | Host project for the shared VPC. | map(string) | | {} | 2-networking | -| [iam_principals](variables-fast.tf#L50) | IAM-format principals. | map(string) | | {} | 0-bootstrap | -| [kms_keys](variables-fast.tf#L66) | KMS key ids. | map(string) | | {} | 2-security | -| [locations](variables-fast.tf#L74) | Optional locations for GCS, BigQuery, and logging buckets created here. | object({…}) | | {} | 0-bootstrap | -| [perimeters](variables-fast.tf#L84) | Optional VPC-SC perimeter ids. | map(string) | | {} | 1-vpcsc | -| [project_ids](variables-fast.tf#L102) | Projects created in the bootstrap stage. | map(string) | | {} | 0-bootstrap | -| [service_accounts](variables-fast.tf#L110) | Service accounts created in the bootstrap stage. | map(string) | | {} | 0-bootstrap | -| [stage_name](variables.tf#L55) | FAST stage name. Used to separate output files across different factories. | string | | "2-project-factory" | | -| [tag_values](variables-fast.tf#L118) | FAST-managed resource manager tag values. | map(string) | | {} | 0-bootstrap | +| [billing_account](variables-fast.tf#L26) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | object({…}) | ✓ | | 0-bootstrap | +| [prefix](variables-fast.tf#L109) | Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants. | string | ✓ | | 0-bootstrap | +| [custom_roles](variables-fast.tf#L39) | Custom roles defined at the org level, in key => id format. | map(string) | | {} | 0-bootstrap | +| [factories_config](variables.tf#L17) | Configuration for YAML-based factories. | object({…}) | | {} | | +| [folder_ids](variables-fast.tf#L47) | Folders created in the resource management stage. | map(string) | | {} | 1-resman | +| [groups](variables-fast.tf#L55) | Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated. | map(string) | | {} | 0-bootstrap | +| [host_project_ids](variables-fast.tf#L64) | Host project for the shared VPC. | map(string) | | {} | 2-networking | +| [kms_keys](variables-fast.tf#L72) | KMS key ids. | map(string) | | {} | 2-security | +| [locations](variables-fast.tf#L80) | Optional locations for GCS, BigQuery, and logging buckets created here. | object({…}) | | {} | 0-bootstrap | +| [org_policy_tags](variables-fast.tf#L98) | Optional organization policy tag values. | object({…}) | | {} | 0-bootstrap | +| [outputs_location](variables.tf#L43) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | string | | null | | +| [perimeters](variables-fast.tf#L90) | Optional VPC-SC perimeter ids. | map(string) | | {} | 1-vpcsc | +| [service_accounts](variables-fast.tf#L119) | Automation service accounts in name => email format. | map(string) | | {} | 1-resman | +| [stage_name](variables.tf#L49) | FAST stage name. Used to separate output files across different factories. | string | | "2-project-factory" | | +| [tag_values](variables-fast.tf#L127) | FAST-managed resource manager tag values. | map(string) | | {} | 1-resman | ## Outputs | name | description | sensitive | consumers | |---|---|:---:|---| -| [projects](outputs.tf#L17) | Attributes for managed projects. | | | +| [buckets](outputs.tf#L31) | Created buckets. | | | +| [projects](outputs.tf#L38) | Created projects. | | | +| [service_accounts](outputs.tf#L50) | Created service accounts. | | | diff --git a/fast/stages/2-project-factory/data/hierarchy/team-a/_config.yaml b/fast/stages/2-project-factory-legacy/data/hierarchy/team-a/_config.yaml similarity index 100% rename from fast/stages/2-project-factory/data/hierarchy/team-a/_config.yaml rename to fast/stages/2-project-factory-legacy/data/hierarchy/team-a/_config.yaml diff --git a/fast/stages/2-project-factory/data/hierarchy/team-a/dev/_config.yaml b/fast/stages/2-project-factory-legacy/data/hierarchy/team-a/dev/_config.yaml similarity index 100% rename from fast/stages/2-project-factory/data/hierarchy/team-a/dev/_config.yaml rename to fast/stages/2-project-factory-legacy/data/hierarchy/team-a/dev/_config.yaml diff --git a/fast/stages/2-project-factory/data/hierarchy/team-a/prod/_config.yaml b/fast/stages/2-project-factory-legacy/data/hierarchy/team-a/prod/_config.yaml similarity index 100% rename from fast/stages/2-project-factory/data/hierarchy/team-a/prod/_config.yaml rename to fast/stages/2-project-factory-legacy/data/hierarchy/team-a/prod/_config.yaml diff --git a/fast/stages/2-project-factory/data/hierarchy/team-b/_config.yaml b/fast/stages/2-project-factory-legacy/data/hierarchy/team-b/_config.yaml similarity index 100% rename from fast/stages/2-project-factory/data/hierarchy/team-b/_config.yaml rename to fast/stages/2-project-factory-legacy/data/hierarchy/team-b/_config.yaml diff --git a/fast/stages/2-project-factory/data/hierarchy/team-b/dev/_config.yaml b/fast/stages/2-project-factory-legacy/data/hierarchy/team-b/dev/_config.yaml similarity index 100% rename from fast/stages/2-project-factory/data/hierarchy/team-b/dev/_config.yaml rename to fast/stages/2-project-factory-legacy/data/hierarchy/team-b/dev/_config.yaml diff --git a/fast/stages/2-project-factory/data/hierarchy/team-b/prod/_config.yaml b/fast/stages/2-project-factory-legacy/data/hierarchy/team-b/prod/_config.yaml similarity index 100% rename from fast/stages/2-project-factory/data/hierarchy/team-b/prod/_config.yaml rename to fast/stages/2-project-factory-legacy/data/hierarchy/team-b/prod/_config.yaml diff --git a/fast/stages/2-project-factory/data/projects/dev-ta-0.yaml b/fast/stages/2-project-factory-legacy/data/projects/dev-ta-0.yaml similarity index 100% rename from fast/stages/2-project-factory/data/projects/dev-ta-0.yaml rename to fast/stages/2-project-factory-legacy/data/projects/dev-ta-0.yaml diff --git a/fast/stages/2-project-factory/data/projects/dev-tb-0.yaml b/fast/stages/2-project-factory-legacy/data/projects/dev-tb-0.yaml similarity index 100% rename from fast/stages/2-project-factory/data/projects/dev-tb-0.yaml rename to fast/stages/2-project-factory-legacy/data/projects/dev-tb-0.yaml diff --git a/fast/stages/2-project-factory/data/projects/prod-ta-0.yaml b/fast/stages/2-project-factory-legacy/data/projects/prod-ta-0.yaml similarity index 100% rename from fast/stages/2-project-factory/data/projects/prod-ta-0.yaml rename to fast/stages/2-project-factory-legacy/data/projects/prod-ta-0.yaml diff --git a/fast/stages/2-project-factory/data/projects/prod-tb-0.yaml b/fast/stages/2-project-factory-legacy/data/projects/prod-tb-0.yaml similarity index 100% rename from fast/stages/2-project-factory/data/projects/prod-tb-0.yaml rename to fast/stages/2-project-factory-legacy/data/projects/prod-tb-0.yaml diff --git a/fast/stages/2-project-factory/diagram-env.png b/fast/stages/2-project-factory-legacy/diagram-env.png similarity index 100% rename from fast/stages/2-project-factory/diagram-env.png rename to fast/stages/2-project-factory-legacy/diagram-env.png diff --git a/fast/stages/2-project-factory/diagram.png b/fast/stages/2-project-factory-legacy/diagram.png similarity index 100% rename from fast/stages/2-project-factory/diagram.png rename to fast/stages/2-project-factory-legacy/diagram.png diff --git a/fast/stages/2-project-factory-experimental/fast_version.txt b/fast/stages/2-project-factory-legacy/fast_version.txt similarity index 100% rename from fast/stages/2-project-factory-experimental/fast_version.txt rename to fast/stages/2-project-factory-legacy/fast_version.txt diff --git a/fast/stages/2-project-factory-legacy/main.tf b/fast/stages/2-project-factory-legacy/main.tf new file mode 100644 index 000000000..22f6c9b79 --- /dev/null +++ b/fast/stages/2-project-factory-legacy/main.tf @@ -0,0 +1,70 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Project factory. + +module "projects" { + source = "../../../modules/project-factory-legacy" + data_defaults = { + # more defaults are available, check the project factory variables + billing_account = var.billing_account.id + storage_location = var.locations.gcs + } + data_merges = { + services = [ + "stackdriver.googleapis.com" + ] + } + data_overrides = { + prefix = var.prefix + } + factories_config = merge(var.factories_config, { + context = { + custom_roles = merge( + var.custom_roles, var.factories_config.context.custom_roles + ) + folder_ids = merge( + { for k, v in var.folder_ids : k => v if v != null }, + var.factories_config.context.folder_ids + ) + iam_principals = merge( + { + for k, v in var.service_accounts : + k => "serviceAccount:${v}" if v != null + }, + var.groups, + var.factories_config.context.iam_principals + ) + kms_keys = merge( + var.kms_keys, + var.factories_config.context.kms_keys + ) + perimeters = var.perimeters + tag_values = merge( + { + for k, v in var.org_policy_tags.values : + "${var.org_policy_tags.key_name}/${k}" => v + }, + var.tag_values, + var.factories_config.context.tag_values + ) + vpc_host_projects = merge( + var.host_project_ids, + var.factories_config.context.vpc_host_projects + ) + } + }) +} diff --git a/fast/stages/2-project-factory-legacy/outputs.tf b/fast/stages/2-project-factory-legacy/outputs.tf new file mode 100644 index 000000000..4c6359f6d --- /dev/null +++ b/fast/stages/2-project-factory-legacy/outputs.tf @@ -0,0 +1,81 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +locals { + project_provider_data = flatten([ + for k, v in module.projects.projects : [ + for sk, sv in try(v.automation.service_accounts) : { + key = "${k}-${sk}" + bucket = try(v.automation.bucket, null) + project_id = v.project_id + project_number = v.number + service_account = sv + } + ] if try(v.automation.bucket, null) != null + ]) +} + +output "buckets" { + description = "Created buckets." + value = { + for k, v in module.projects.buckets : k => v + } +} + +output "projects" { + description = "Created projects." + value = { + for k, v in module.projects.projects : k => { + id = v.project_id + number = v.number + automation = v.automation + service_agents = v.service_agents + } + } +} + +output "service_accounts" { + description = "Created service accounts." + value = { + for k, v in module.projects.service_accounts : k => { + email = v.email + iam_email = v.iam_email + } + } +} + +resource "google_storage_bucket_object" "version" { + count = fileexists("fast_version.txt") ? 1 : 0 + bucket = var.automation.outputs_bucket + name = "versions/2-project-factory-version.txt" + source = "fast_version.txt" +} + +# generate tfvars file for subsequent stages + +resource "local_file" "providers" { + for_each = var.outputs_location == null ? {} : { for v in local.project_provider_data : v.key => v } + file_permission = "0644" + filename = "${pathexpand(var.outputs_location)}/providers/${var.stage_name}/${each.key}-providers.tf" + content = templatefile("templates/providers.tf.tpl", each.value) +} + +resource "google_storage_bucket_object" "tfvars" { + for_each = { for v in local.project_provider_data : v.key => v } + bucket = var.automation.outputs_bucket + name = "providers/${var.stage_name}/${each.key}-providers.tf" + content = templatefile("templates/providers.tf.tpl", each.value) +} diff --git a/fast/stages/2-project-factory-experimental/schemas/budget.schema.json b/fast/stages/2-project-factory-legacy/schemas/budget.schema.json similarity index 100% rename from fast/stages/2-project-factory-experimental/schemas/budget.schema.json rename to fast/stages/2-project-factory-legacy/schemas/budget.schema.json diff --git a/fast/stages/2-project-factory-experimental/schemas/budget.schema.md b/fast/stages/2-project-factory-legacy/schemas/budget.schema.md similarity index 100% rename from fast/stages/2-project-factory-experimental/schemas/budget.schema.md rename to fast/stages/2-project-factory-legacy/schemas/budget.schema.md diff --git a/fast/stages/2-project-factory-experimental/schemas/folder.schema.json b/fast/stages/2-project-factory-legacy/schemas/folder.schema.json similarity index 100% rename from fast/stages/2-project-factory-experimental/schemas/folder.schema.json rename to fast/stages/2-project-factory-legacy/schemas/folder.schema.json diff --git a/fast/stages/0-bootstrap-experimental/schemas/folder.schema.md b/fast/stages/2-project-factory-legacy/schemas/folder.schema.md similarity index 100% rename from fast/stages/0-bootstrap-experimental/schemas/folder.schema.md rename to fast/stages/2-project-factory-legacy/schemas/folder.schema.md diff --git a/fast/stages/2-project-factory-experimental/schemas/project.schema.json b/fast/stages/2-project-factory-legacy/schemas/project.schema.json similarity index 100% rename from fast/stages/2-project-factory-experimental/schemas/project.schema.json rename to fast/stages/2-project-factory-legacy/schemas/project.schema.json diff --git a/modules/project-factory-experimental/schemas/project.schema.md b/fast/stages/2-project-factory-legacy/schemas/project.schema.md similarity index 100% rename from modules/project-factory-experimental/schemas/project.schema.md rename to fast/stages/2-project-factory-legacy/schemas/project.schema.md diff --git a/fast/stages/2-project-factory-experimental/templates/providers.tf.tpl b/fast/stages/2-project-factory-legacy/templates/providers.tf.tpl similarity index 100% rename from fast/stages/2-project-factory-experimental/templates/providers.tf.tpl rename to fast/stages/2-project-factory-legacy/templates/providers.tf.tpl diff --git a/fast/stages/2-project-factory-experimental/variables-fast.tf b/fast/stages/2-project-factory-legacy/variables-fast.tf similarity index 71% rename from fast/stages/2-project-factory-experimental/variables-fast.tf rename to fast/stages/2-project-factory-legacy/variables-fast.tf index 19534ef3a..f29bd4a53 100644 --- a/fast/stages/2-project-factory-experimental/variables-fast.tf +++ b/fast/stages/2-project-factory-legacy/variables-fast.tf @@ -25,10 +25,15 @@ variable "automation" { variable "billing_account" { # tfdoc:variable:source 0-bootstrap - description = "Billing account id." + description = "Billing account id. If billing account is not part of the same org set `is_org_level` to false." type = object({ - id = string + id = string + is_org_level = optional(bool, true) }) + validation { + condition = var.billing_account.is_org_level != null + error_message = "Invalid `null` value for `billing_account.is_org_level`." + } } variable "custom_roles" { @@ -40,16 +45,17 @@ variable "custom_roles" { } variable "folder_ids" { - # tfdoc:variable:source 0-bootstrap - description = "Folders created in the bootstrap stage." + # tfdoc:variable:source 1-resman + description = "Folders created in the resource management stage." type = map(string) nullable = false default = {} } -variable "iam_principals" { +variable "groups" { # tfdoc:variable:source 0-bootstrap - description = "IAM-format principals." + # https://cloud.google.com/docs/enterprise/setup-checklist + description = "Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated." type = map(string) nullable = false default = {} @@ -75,7 +81,7 @@ variable "locations" { # tfdoc:variable:source 0-bootstrap description = "Optional locations for GCS, BigQuery, and logging buckets created here." type = object({ - storage = optional(string, "eu") + gcs = optional(string) }) nullable = false default = {} @@ -89,6 +95,17 @@ variable "perimeters" { default = {} } +variable "org_policy_tags" { + # tfdoc:variable:source 0-bootstrap + description = "Optional organization policy tag values." + type = object({ + key_name = optional(string, "org-policies") + values = optional(map(string), {}) + }) + nullable = false + default = {} +} + variable "prefix" { # tfdoc:variable:source 0-bootstrap description = "Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants." @@ -99,24 +116,16 @@ variable "prefix" { } } -variable "project_ids" { - # tfdoc:variable:source 0-bootstrap - description = "Projects created in the bootstrap stage." - type = map(string) - nullable = false - default = {} -} - variable "service_accounts" { - # tfdoc:variable:source 0-bootstrap - description = "Service accounts created in the bootstrap stage." + # tfdoc:variable:source 1-resman + description = "Automation service accounts in name => email format." type = map(string) nullable = false default = {} } variable "tag_values" { - # tfdoc:variable:source 0-bootstrap + # tfdoc:variable:source 1-resman description = "FAST-managed resource manager tag values." type = map(string) nullable = false diff --git a/fast/stages/2-project-factory-legacy/variables.tf b/fast/stages/2-project-factory-legacy/variables.tf new file mode 100644 index 000000000..3c2996caf --- /dev/null +++ b/fast/stages/2-project-factory-legacy/variables.tf @@ -0,0 +1,54 @@ +/** + * Copyright 2024 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +variable "factories_config" { + description = "Configuration for YAML-based factories." + type = object({ + folders_data_path = optional(string, "data/hierarchy") + projects_data_path = optional(string, "data/projects") + budgets = optional(object({ + billing_account = string + budgets_data_path = optional(string, "data/budgets") + notification_channels = optional(map(any), {}) + })) + context = optional(object({ + custom_roles = optional(map(string), {}) + folder_ids = optional(map(string), {}) + kms_keys = optional(map(string), {}) + iam_principals = optional(map(string), {}) + tag_values = optional(map(string), {}) + vpc_host_projects = optional(map(string), {}) + }), {}) + projects_config = optional(object({ + key_ignores_path = optional(bool, false) + }), {}) + }) + nullable = false + default = {} +} + +variable "outputs_location" { + description = "Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable." + type = string + default = null +} + +variable "stage_name" { + description = "FAST stage name. Used to separate output files across different factories." + type = string + nullable = false + default = "2-project-factory" +} diff --git a/fast/stages/2-project-factory/.fast-stage.env b/fast/stages/2-project-factory/.fast-stage.env index 24820e777..9ef2979f5 100644 --- a/fast/stages/2-project-factory/.fast-stage.env +++ b/fast/stages/2-project-factory/.fast-stage.env @@ -1,5 +1,5 @@ FAST_STAGE_DESCRIPTION="project factory (org level)" FAST_STAGE_LEVEL=2 FAST_STAGE_NAME=project-factory -FAST_STAGE_DEPS="0-globals 0-bootstrap 1-resman" +FAST_STAGE_DEPS="0-globals 0-bootstrap" FAST_STAGE_OPTIONAL="1-vpcsc 2-networking 2-security" \ No newline at end of file diff --git a/fast/stages/2-project-factory/README.md b/fast/stages/2-project-factory/README.md index c218548d3..e120974db 100644 --- a/fast/stages/2-project-factory/README.md +++ b/fast/stages/2-project-factory/README.md @@ -3,7 +3,11 @@ - [Design overview and choices](#design-overview-and-choices) - [How to run this stage](#how-to-run-this-stage) - - [Resource Management stage configuration](#resource-management-stage-configuration) + - [Bootstrap stage configuration](#bootstrap-stage-configuration) + - [Automation resources](#automation-resources) + - [Billing account](#billing-account) + - [Organization IAM](#organization-iam) + - [Parent folder](#parent-folder) - [Factory configuration](#factory-configuration) - [Stage provider and Terraform variables](#stage-provider-and-terraform-variables) - [Managing folders and projects](#managing-folders-and-projects) @@ -11,8 +15,6 @@ - [Folder parent-child relationship and variable substitutions](#folder-parent-child-relationship-and-variable-substitutions) - [Project Creation](#project-creation) - [Automation Resources for Projects](#automation-resources-for-projects) -- [Alternative patterns](#alternative-patterns) - - [Per-environment Factories](#per-environment-factories) - [Files](#files) - [Variables](#variables) - [Outputs](#outputs) @@ -22,76 +24,123 @@ The Project Factory stage allows simplified management of folder hierarchies and The pattern implemented here by default allows management of a teams (or business units, applications, etc.) hierarchy. Different patterns are possible, and this document also tries to provide some guidance on how to implement them. -

- Project factory teams pattern -

- ## Design overview and choices -The project factory is "primed" by the resource management stage via +The project factory optionally "consumes" resources created by preceding stages, by using their outputs as a source for [context interpolation](../../../modules/project-factory/README.md#context-based-interpolation): -- a set of service accounts with different scopes -- one or more user-defined top-level folders where those service accounts operate +- folder ids from the bootstrap stage and via `var.context.folder_ids` +- project ids from the bootstrap and networking stages and via `var.context.project_ids` +- IAM principals from the bootstrap stage and via `var.context.iam_principals` +- tag values from the bootstrap stage and via `var.context.tag_values` +- KMS keys from the security stage and via `var.context.kms_keys` +- VPC SC perimeters from the VPC SC stage and via `var.context.vpc_sc_perimeters` -This stage does not directly depend on other stage 2 like networking and security, but it can optionally leverage resources created there like Shared VPC host projects, which are used to define service projects. +Additionally, some of the values defined earlier in the FAST apply cycle are set here as project defaults: + +- prefix (as override) +- billing account +- storage location The project factory stage is a thin wrapper of the underlying [project-factory module](../../../modules/project-factory/), which in turn exposes the full interface of the [project](../../../modules/project/) and [folder](../../../modules/folder/) modules. ## How to run this stage -This stage is meant to be executed after the [bootstrap](../0-bootstrap/) and [resource management](../1-resman/) "foundational stages". As mentioned above it runs in parallel with other stage 2 and can leverage resources they create but does not depend on them. +This stage is meant to be executed after the [bootstrap](../0-bootstrap/) stage. If any of the VPC SC, networking, and security stages have been applied, their resources can be directly leveraged via context interpolation as explained above. -### Resource Management stage configuration +### Bootstrap stage configuration -The resource management stage already contains a sample "Teams" folder defined via YAML, which can be used as-is or modified to provide a top-level folder for the project factory. More folders can of course be added, and Terraform variables used instead of or in addition to YAML files in the resource management stage. +The bootstrap stage already contains the project factory automation resources, a sample "Teams" folder defined via YAML, and all the required IAM wiring to make this stage functional. The default "Teams" setup can be extended, or used as an example to implement different designs. -This is the teams YAML in resource management, leveraging attribute substitutions from provided context for the project factory service account and tag value. +The bootstrap-specific setup is reproduced here to aid using it as a starting point. Only snippets relevant to this stage are shown below for simplicity. + +#### Automation resources + +The default design uses two service accounts (read-write and read-only) and a Cloud Storage folder in a pre-existing bucket, to enable this stage for Infrastructure as Code. ```yaml +# data/projects/core/iac-0.yaml +buckets: + iac-stage-state: + description: Terraform state for stage automation. + managed_folders: + 2-project-factory: + iam: + roles/storage.admin: + - $iam_principals:service_accounts/iac-0/iac-pf-rw + $custom_roles:storage_viewer: + - $iam_principals:service_accounts/iac-0/iac-pf-ro + iac-outputs: + description: Terraform state for the org-level automation. + iam: + roles/storage.admin: + - $iam_principals:service_accounts/iac-0/iac-pf-rw + $custom_roles:storage_viewer: + - $iam_principals:service_accounts/iac-0/iac-pf-ro +service_accounts: + iac-pf-ro: + display_name: IaC service account for project factory (read-only). + iac-pf-rw: + display_name: IaC service account for project factory (read-write). +``` + +#### Billing account + +If an externally managed billing account is used, billing user permissions need to be assigned to the project factory service account. + +```yaml +# data/billing-accounts/default.yaml +id: $defaults:billing_account +iam_bindings_additive: + billing_user_pf_sa: + role: roles/billing.user + member: $iam_principals:service_accounts/iac-0/iac-pf-rw +``` + +#### Organization IAM + +This stage only needs conditional grants for organization policy management at the organization level. Additionally, if an organization-managed billing account is used the IAM bindings described in the section above can be omitted, and moved to the organization. + +```yaml +# data/organization/.config.yaml +iam_bindings: + pf_org_policy_admin: + role: roles/orgpolicy.policyAdmin + members: + - $iam_principals:service_accounts/iac-0/iac-pf-rw + condition: + expression: resource.matchTag('${organization}/context', 'project-factory') + title: Project factory org policy admin + pf_org_policy_viewer: + role: roles/orgpolicy.policyViewer + members: + - $iam_principals:service_accounts/iac-0/iac-pf-ro + condition: + expression: resource.matchTag('${organization}/context', 'project-factory') + title: Project factory org policy viewer +``` + +#### Parent folder + +A single "Teams" folder is created here. Multiple folders (or sub-folders) can of course be created by replicating the IAM configuration below for each. + +```yaml +# data/folders/teams/.config.yaml name: Teams -automation: - enable: false -iam: - "roles/owner": - - project-factory - "roles/resourcemanager.folderAdmin": - - project-factory - "roles/resourcemanager.projectCreator": - - project-factory - "roles/resourcemanager.tagUser": - - project-factory - "service_project_network_admin": - - project-factory +iam_by_principals: + $iam_principals:service_accounts/iac-0/iac-pf-rw: + - roles/owner + - roles/resourcemanager.folderAdmin + - roles/resourcemanager.projectCreator + - roles/resourcemanager.tagUser + - $custom_roles:service_project_network_admin + $iam_principals:service_accounts/iac-0/iac-pf-ro: + - roles/viewer + - roles/resourcemanager.folderViewer + - roles/resourcemanager.tagViewer tag_bindings: - context: context/project-factory + context: $tag_values:context/project-factory ``` -This is the alternative version that can be used instead of the YAML file above. - -```tfvars -top_level_folders = { - # more top-level folders might be present here - teams = { - name = "Teams" - iam = { - "roles/owner" = ["project-factory"] - "roles/resourcemanager.folderAdmin" = ["project-factory"] - "roles/resourcemanager.projectCreator" = ["project-factory"] - "roles/resourcemanager.tagUser" = ["project-factory"] - "service_project_network_admin" = ["project-factory"] - } - tag_bindings = { - context = "context/project-factory" - } - } -} -# tftest skip -``` - -You can of course extend these snippets to grant additional roles to groups or other service accounts via the `iam`, `iam_by_principals`, and `iam_bindings` folder-level variables. - -The project factory tag binding on the folder allows management of organization policies in the Teams hierarchy. If this functionality is not needed, the tag binding can be safely omitted. - ### Factory configuration The `data` folder in this stage contains factory files that can be used as examples to implement the team-based design shown above. Before running `terraform apply` check the YAML files, as project names and other attributes will need basic editing to match your desired setup. @@ -113,7 +162,6 @@ ln -s ~/fast-config/fast-test-00/providers/2-project-factory-providers.tf ./ # input files from other stages ln -s ~/fast-config/fast-test-00/tfvars/0-globals.auto.tfvars.json ./ ln -s ~/fast-config/fast-test-00/tfvars/0-bootstrap.auto.tfvars.json ./ -ln -s ~/fast-config/fast-test-00/tfvars/1-resman.auto.tfvars.json ./ # conventional place for stage tfvars (manually created) ln -s ~/fast-config/fast-test-00/2-project-factory.auto.tfvars ./ @@ -121,6 +169,7 @@ ln -s ~/fast-config/fast-test-00/2-project-factory.auto.tfvars ./ # optional files ln -s ~/fast-config/fast-test-00/2-networking.auto.tfvars.json ./ ln -s ~/fast-config/fast-test-00/2-security.auto.tfvars.json ./ +ln -s ~/fast-config/fast-test-00/2-vpcsc.auto.tfvars.json ./ ``` ```bash @@ -134,7 +183,6 @@ gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/2-project-factory-p # input files from other stages gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json ./ gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json ./ # conventional place for stage tfvars (manually created) gcloud storage cp gs://xxx-prod-iac-core-outputs-0/2-project-factory.auto.tfvars ./ @@ -142,6 +190,7 @@ gcloud storage cp gs://xxx-prod-iac-core-outputs-0/2-project-factory.auto.tfvars # optional files gcloud storage cp gs://xxx-prod-iac-core-outputs-0/2-networking.auto.tfvars.json ./ gcloud storage cp gs://xxx-prod-iac-core-outputs-0/2-security.auto.tfvars.json ./ +gcloud storage cp gs://xxx-prod-iac-core-outputs-0/2-vpcsc.auto.tfvars.json ./ ``` If you're not using FAST, refer to the [Variables](#variables) table at the bottom of this document for a full list of variables, their origin (e.g., a stage or specific to this one), and descriptions explaining their meaning. @@ -161,26 +210,26 @@ The YAML data files are self-explanatory and the included [schema files](./schem ### Folder and hierarchy management -The project factory manages its folder hierarchy via a filesystem tree, rooted in the path defined via the `factories_config.hierarchy_data` variable. +The project factory manages its folder hierarchy via a filesystem tree, rooted in the path defined via the `factories_config.folders` variable. -Filesystem folders which contain a `_config.yaml` file are mapped to folders in the resource management hierarchy. Their YAML configuration files allow defining folder attributes like descriptive name, IAM bindings, organization policies, tag bindings. +Filesystem folders which contain a `.config.yaml` file are mapped to folders in the resource management hierarchy. Their YAML configuration files allow defining folder attributes like descriptive name, IAM bindings, organization policies, tag bindings. This is the simple filesystem hierarchy provided here as an example. ```bash hierarchy ├── team-a -│   ├── _config.yaml +│   ├── .config.yaml │   ├── dev -│   │   └── _config.yaml +│   │   └── .config.yaml │   └── prod -│   └── _config.yaml +│   └── .config.yaml └── team-b - ├── _config.yaml + ├── .config.yaml ├── dev - │   └── _config.yaml + │   └── .config.yaml └── prod - └── _config.yaml + └── .config.yaml ``` The approach is intentionally explicit and repetitive in order to simplify operations: copy/pasting an existing set of folders (or an ad hoc template) and changing a few YAML variables allows to quickly define new sub-hierarchy branches. Mass editing via search and replace functionality allows sweeping changes across the whole hierarchy. @@ -190,7 +239,7 @@ Where inheritance is leveraged in the overall design config files can be decepti ```yaml name: Development tag_bindings: - environment: environment/development + environment: $tag_values:environment/development iam_by_principals: "group:team-a-admins@example.com": - roles/editor @@ -204,11 +253,10 @@ In the example YAML configuration above there's no explicitly specified folder p But what about the "Team A" folder itself? From the point of view of the project factory it's a top-level folder attached to the root of its hierarchy (the "Teams" folder), so how does it know where to create it in the GCP hierarchy? -There are three different ways to pass this information to the project factory: +There are two different ways to pass this information to the project factory: -- in the YAML file itself, by explicitly setting the folder's `parent` attribute to the explicit numeric id of the "Teams" folder -- in the YAML file itself, by explicitly setting the folder's `parent` attribute to the short name of the "Teams" folder in the resource management stage's outputs -- in the stage Terraform variables, by setting the `default` folder for the project factory to the numeric id of the "Teams" folder +- in the YAML file itself, by explicitly setting the folder's `parent` attribute to the explicit numeric id of the "Teams" folder (e.g. `folders/1234567890`) +- in the YAML file itself, by using explicit context interpolation (e.g. `$folder_ids:teams`) This flexibility is what allows the project factory to manage folders under multiple roots, and to also be used for folders created outside of FAST. Imagine a scenario where there's no single "Teams" folder, but multiple ones for different subsidiaries, or for internal and external teams, etc. @@ -222,30 +270,15 @@ parent: folders/1234567890 ```yaml name: Team A -# use variable substitution from stage 1 tfvars (preferred approach) -parent: teams -``` - -The third way explained above does not explicitly define a root folder in the YAML files, but sets a default folder in the Terraform variables for the stage via the `factories_config.substitutions.folder_ids`, by adding a `default` key pointing to the folder id of the root ("Teams") folder. - -```tfvars -factories_config = { - substitutions = { - folder_ids = { - # id of the top-level Teams folder - # derived from the 1-resman.auto.tfvars.json file - default = "folders/12345678" - } - } -} -# tftest skip +# use context interpolation from stage 0 tfvars (preferred approach) +parent: $folder_ids:teams ``` ### Project Creation Project YAML files can be created in two different filesystem paths: -- in the filesystem folder defined via the `factories_config.project_data` variable, and then explicitly setting their `parent` attribute in YAML files, or +- in the filesystem folder defined via the `factories_config.projects` variable, and then explicitly setting their `parent` attribute in YAML files, or - in the filesystem hierarchy discussed above, so that their `parent` attribute is automatically derived from the containing folder The two approaches can be mixed and matched, but the first approach is safer as is avoids potentially dangerous situations when folders are deleted with project configuration files still inside. @@ -258,8 +291,8 @@ parent: folders/1234509876 ``` ```yaml -# use variable substitution from managed folders (preferred approach) -parent: team-a/dev +# use context interpolation from managed folders (preferred approach) +parent: $folder_ids:team-a/dev ``` All of the [project module](../../../modules/project/) attributes (and some service account attributes) can of course be leveraged in the configuration files. Refer to the [project schema](./schemas/folder.schema.json) for the complete set of available attributes. @@ -270,8 +303,8 @@ When created projects are meant to be managed via IaC downstream, an initial set ```yaml # controlling project shown in the diagram above -parent: teams -name: xxx-prod-iac-teams-0 +parent: $folder_ids:teams +name: $project_ids:iac-core-0 services: - compute.googleapis.com - storage.googleapis.com @@ -282,61 +315,35 @@ services: Once a controlling project is in place, it can be used in any other project declaration to host service accounts and bucket for automation. The service accounts can be used in IAM bindings in the same file by referring to their name via substitutions, as shown here. ```yaml +# file name: dev-ta-app-0.yaml (implicitly used for project id) # team or application-level project with automation resources -parent: team-a/dev +parent: $folder_ids:team-a/dev # project prefix is forced via override in `main.tf` -name: dev-ta-app-0 iam: roles/owner: # refer to the rw service account defined below - - rw + - $iam_principals:service_accounts/dev-ta-app-0/rw roles/viewer: # refer to the ro service account defined below - - ro + - $iam_principals:service_accounts/dev-ta-app-0/ro automation: - # no context is possible here - # use the complete project id - project: xxx-prod-iac-teams-0 + project: $project_ids:iac-core-0 service_accounts: - # resulting sa name: xxx-dev-ta-app-0-rw rw: description: Read/write automation sa for team a app 0. - # resulting sa name: xxx-dev-ta-app-0-ro ro: description: Read-only automation sa for team a app 0. bucket: - # resulting bucket name: xxx-dev-ta-app-0-state description: Terraform state bucket for team a app 0. iam: - # service accounts can use short name substitutions from context roles/storage.objectCreator: - - rw + - $iam_principals:service_accounts/dev-ta-app-0/rw roles/storage.objectViewer: - - rw - - ro + - $iam_principals:service_accounts/dev-ta-app-0/rw + - $iam_principals:service_accounts/dev-ta-app-0/ro - group:devops@example.org ``` -## Alternative patterns - -Some alternative patterns are captured here, the list will grow as we generalize approaches seen in the field. - -### Per-environment Factories - -A variation of this pattern uses separate project factories for each environment, as in the following diagram. - -

- Project factory team-level per environment. -

- -This approach leverages the per-environment project factory service accounts and tags created by the resource management stage, so that - -- the Teams folder hierarchy and IaC project are managed by a cross-environment factory using the "main" project factory service account -- IAM permissions are set on the environment folders to grant control to the prod and dev project factory service accounts -- one additional factory per environment manages project creation leveraging the folders created above - -The approach is not shown here but reasonably easy to implement. The main project factory output file can also be used to set up folder id susbtitution in the per-environment factories. - ## Files @@ -344,7 +351,7 @@ The approach is not shown here but reasonably easy to implement. The main projec | name | description | modules | resources | |---|---|---|---| | [main.tf](./main.tf) | Project factory. | project-factory | | -| [outputs.tf](./outputs.tf) | Module outputs. | | google_storage_bucket_object · local_file | +| [outputs.tf](./outputs.tf) | Module outputs. | | google_storage_bucket_object | | [variables-fast.tf](./variables-fast.tf) | None | | | | [variables.tf](./variables.tf) | Module variables. | | | @@ -353,27 +360,25 @@ The approach is not shown here but reasonably easy to implement. The main projec | name | description | type | required | default | producer | |---|---|:---:|:---:|:---:|:---:| | [automation](variables-fast.tf#L17) | Automation resources created by the bootstrap stage. | object({…}) | ✓ | | 0-bootstrap | -| [billing_account](variables-fast.tf#L26) | Billing account id. If billing account is not part of the same org set `is_org_level` to false. | object({…}) | ✓ | | 0-bootstrap | -| [prefix](variables-fast.tf#L109) | Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants. | string | ✓ | | 0-bootstrap | -| [custom_roles](variables-fast.tf#L39) | Custom roles defined at the org level, in key => id format. | map(string) | | {} | 0-bootstrap | -| [factories_config](variables.tf#L17) | Configuration for YAML-based factories. | object({…}) | | {} | | -| [folder_ids](variables-fast.tf#L47) | Folders created in the resource management stage. | map(string) | | {} | 1-resman | -| [groups](variables-fast.tf#L55) | Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated. | map(string) | | {} | 0-bootstrap | -| [host_project_ids](variables-fast.tf#L64) | Host project for the shared VPC. | map(string) | | {} | 2-networking | -| [kms_keys](variables-fast.tf#L72) | KMS key ids. | map(string) | | {} | 2-security | -| [locations](variables-fast.tf#L80) | Optional locations for GCS, BigQuery, and logging buckets created here. | object({…}) | | {} | 0-bootstrap | -| [org_policy_tags](variables-fast.tf#L98) | Optional organization policy tag values. | object({…}) | | {} | 0-bootstrap | -| [outputs_location](variables.tf#L43) | Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable. | string | | null | | -| [perimeters](variables-fast.tf#L90) | Optional VPC-SC perimeter ids. | map(string) | | {} | 1-vpcsc | -| [service_accounts](variables-fast.tf#L119) | Automation service accounts in name => email format. | map(string) | | {} | 1-resman | -| [stage_name](variables.tf#L49) | FAST stage name. Used to separate output files across different factories. | string | | "2-project-factory" | | -| [tag_values](variables-fast.tf#L127) | FAST-managed resource manager tag values. | map(string) | | {} | 1-resman | +| [billing_account](variables-fast.tf#L26) | Billing account id. | object({…}) | ✓ | | 0-bootstrap | +| [prefix](variables-fast.tf#L92) | Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants. | string | ✓ | | 0-bootstrap | +| [context](variables.tf#L17) | Context-specific interpolations. | object({…}) | | {} | | +| [custom_roles](variables-fast.tf#L34) | Custom roles defined at the org level, in key => id format. | map(string) | | {} | 0-bootstrap | +| [factories_config](variables.tf#L35) | Path to folder with YAML resource description data files. | object({…}) | | {} | | +| [folder_ids](variables-fast.tf#L42) | Folders created in the bootstrap stage. | map(string) | | {} | 0-bootstrap | +| [host_project_ids](variables-fast.tf#L58) | Host project for the shared VPC. | map(string) | | {} | 2-networking | +| [iam_principals](variables-fast.tf#L50) | IAM-format principals. | map(string) | | {} | 0-bootstrap | +| [kms_keys](variables-fast.tf#L66) | KMS key ids. | map(string) | | {} | 2-security | +| [locations](variables-fast.tf#L74) | Optional locations for GCS, BigQuery, and logging buckets created here. | object({…}) | | {} | 0-bootstrap | +| [perimeters](variables-fast.tf#L84) | Optional VPC-SC perimeter ids. | map(string) | | {} | 1-vpcsc | +| [project_ids](variables-fast.tf#L102) | Projects created in the bootstrap stage. | map(string) | | {} | 0-bootstrap | +| [service_accounts](variables-fast.tf#L110) | Service accounts created in the bootstrap stage. | map(string) | | {} | 0-bootstrap | +| [stage_name](variables.tf#L55) | FAST stage name. Used to separate output files across different factories. | string | | "2-project-factory" | | +| [tag_values](variables-fast.tf#L118) | FAST-managed resource manager tag values. | map(string) | | {} | 0-bootstrap | ## Outputs | name | description | sensitive | consumers | |---|---|:---:|---| -| [buckets](outputs.tf#L31) | Created buckets. | | | -| [projects](outputs.tf#L38) | Created projects. | | | -| [service_accounts](outputs.tf#L50) | Created service accounts. | | | +| [projects](outputs.tf#L17) | Attributes for managed projects. | | | diff --git a/fast/stages/2-project-factory-experimental/data/folders/team-a/.config.yaml b/fast/stages/2-project-factory/data/folders/team-a/.config.yaml similarity index 100% rename from fast/stages/2-project-factory-experimental/data/folders/team-a/.config.yaml rename to fast/stages/2-project-factory/data/folders/team-a/.config.yaml diff --git a/fast/stages/2-project-factory-experimental/data/folders/team-a/dev/.config.yaml b/fast/stages/2-project-factory/data/folders/team-a/dev/.config.yaml similarity index 100% rename from fast/stages/2-project-factory-experimental/data/folders/team-a/dev/.config.yaml rename to fast/stages/2-project-factory/data/folders/team-a/dev/.config.yaml diff --git a/fast/stages/2-project-factory-experimental/data/folders/team-a/prod/.config.yaml b/fast/stages/2-project-factory/data/folders/team-a/prod/.config.yaml similarity index 100% rename from fast/stages/2-project-factory-experimental/data/folders/team-a/prod/.config.yaml rename to fast/stages/2-project-factory/data/folders/team-a/prod/.config.yaml diff --git a/fast/stages/2-project-factory-experimental/data/projects/dev-app-a-0.yaml b/fast/stages/2-project-factory/data/projects/dev-app-a-0.yaml similarity index 100% rename from fast/stages/2-project-factory-experimental/data/projects/dev-app-a-0.yaml rename to fast/stages/2-project-factory/data/projects/dev-app-a-0.yaml diff --git a/fast/stages/2-project-factory-experimental/data/projects/prod-app-a-0.yaml b/fast/stages/2-project-factory/data/projects/prod-app-a-0.yaml similarity index 100% rename from fast/stages/2-project-factory-experimental/data/projects/prod-app-a-0.yaml rename to fast/stages/2-project-factory/data/projects/prod-app-a-0.yaml diff --git a/fast/stages/2-project-factory/main.tf b/fast/stages/2-project-factory/main.tf index f6ea11cf6..5de1109bc 100644 --- a/fast/stages/2-project-factory/main.tf +++ b/fast/stages/2-project-factory/main.tf @@ -16,54 +16,61 @@ # tfdoc:file:description Project factory. -module "projects" { +module "factory" { source = "../../../modules/project-factory" + context = { + custom_roles = merge( + var.custom_roles, var.context.custom_roles + ) + folder_ids = merge( + var.folder_ids, var.context.folder_ids + ) + iam_principals = merge( + var.iam_principals, + { + for k, v in var.service_accounts : + k => "serviceAccount:${v}" if v != null + }, + var.context.iam_principals + ) + kms_keys = merge( + var.kms_keys, var.context.kms_keys + ) + locations = merge( + var.locations, var.context.locations + ) + notification_channels = var.context.notification_channels + project_ids = merge( + var.project_ids, var.host_project_ids, var.context.project_ids + ) + tag_values = merge( + var.tag_values, var.context.tag_values + ) + vpc_sc_perimeters = merge( + var.perimeters, var.context.vpc_sc_perimeters + ) + } data_defaults = { # more defaults are available, check the project factory variables billing_account = var.billing_account.id - storage_location = var.locations.gcs + storage_location = var.locations.storage } data_merges = { services = [ - "stackdriver.googleapis.com" + "logging.googleapis.com", + "monitoring.googleapis.com" ] } data_overrides = { prefix = var.prefix } factories_config = merge(var.factories_config, { - context = { - custom_roles = merge( - var.custom_roles, var.factories_config.context.custom_roles + budgets = { + billing_account_id = try( + var.factories_config.budgets.billing_account_id, var.billing_account.id ) - folder_ids = merge( - { for k, v in var.folder_ids : k => v if v != null }, - var.factories_config.context.folder_ids - ) - iam_principals = merge( - { - for k, v in var.service_accounts : - k => "serviceAccount:${v}" if v != null - }, - var.groups, - var.factories_config.context.iam_principals - ) - kms_keys = merge( - var.kms_keys, - var.factories_config.context.kms_keys - ) - perimeters = var.perimeters - tag_values = merge( - { - for k, v in var.org_policy_tags.values : - "${var.org_policy_tags.key_name}/${k}" => v - }, - var.tag_values, - var.factories_config.context.tag_values - ) - vpc_host_projects = merge( - var.host_project_ids, - var.factories_config.context.vpc_host_projects + data = try( + var.factories_config.budgets.data, "data/budgets" ) } }) diff --git a/fast/stages/2-project-factory/outputs.tf b/fast/stages/2-project-factory/outputs.tf index 4c6359f6d..4d7c73f06 100644 --- a/fast/stages/2-project-factory/outputs.tf +++ b/fast/stages/2-project-factory/outputs.tf @@ -14,68 +14,14 @@ * limitations under the License. */ -locals { - project_provider_data = flatten([ - for k, v in module.projects.projects : [ - for sk, sv in try(v.automation.service_accounts) : { - key = "${k}-${sk}" - bucket = try(v.automation.bucket, null) - project_id = v.project_id - project_number = v.number - service_account = sv - } - ] if try(v.automation.bucket, null) != null - ]) -} - -output "buckets" { - description = "Created buckets." - value = { - for k, v in module.projects.buckets : k => v - } -} - output "projects" { - description = "Created projects." - value = { - for k, v in module.projects.projects : k => { - id = v.project_id - number = v.number - automation = v.automation - service_agents = v.service_agents - } - } -} - -output "service_accounts" { - description = "Created service accounts." - value = { - for k, v in module.projects.service_accounts : k => { - email = v.email - iam_email = v.iam_email - } - } + description = "Attributes for managed projects." + value = module.factory.projects } resource "google_storage_bucket_object" "version" { count = fileexists("fast_version.txt") ? 1 : 0 bucket = var.automation.outputs_bucket - name = "versions/2-project-factory-version.txt" + name = "versions/2-${var.stage_name}-version.txt" source = "fast_version.txt" } - -# generate tfvars file for subsequent stages - -resource "local_file" "providers" { - for_each = var.outputs_location == null ? {} : { for v in local.project_provider_data : v.key => v } - file_permission = "0644" - filename = "${pathexpand(var.outputs_location)}/providers/${var.stage_name}/${each.key}-providers.tf" - content = templatefile("templates/providers.tf.tpl", each.value) -} - -resource "google_storage_bucket_object" "tfvars" { - for_each = { for v in local.project_provider_data : v.key => v } - bucket = var.automation.outputs_bucket - name = "providers/${var.stage_name}/${each.key}-providers.tf" - content = templatefile("templates/providers.tf.tpl", each.value) -} diff --git a/fast/stages/2-project-factory/schemas/folder.schema.md b/fast/stages/2-project-factory/schemas/folder.schema.md index 7ea4d8c0c..4c5fac144 100644 --- a/fast/stages/2-project-factory/schemas/folder.schema.md +++ b/fast/stages/2-project-factory/schemas/folder.schema.md @@ -6,6 +6,25 @@ *additional properties: false* +- **automation**: *object* +
*additional properties: false* + - **prefix**: *string* + - ⁺**project**: *string* + - **bucket**: *reference([bucket](#refs-bucket))* + - **service_accounts**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *object* +
*additional properties: false* + - **description**: *string* + - **iam**: *reference([iam](#refs-iam))* + - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* + - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* + - **iam_billing_roles**: *reference([iam_billing_roles](#refs-iam_billing_roles))* + - **iam_folder_roles**: *reference([iam_folder_roles](#refs-iam_folder_roles))* + - **iam_organization_roles**: *reference([iam_organization_roles](#refs-iam_organization_roles))* + - **iam_project_roles**: *reference([iam_project_roles](#refs-iam_project_roles))* + - **iam_sa_roles**: *reference([iam_sa_roles](#refs-iam_sa_roles))* + - **iam_storage_roles**: *reference([iam_storage_roles](#refs-iam_storage_roles))* - **iam**: *reference([iam](#refs-iam))* - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* @@ -37,24 +56,48 @@ - **location**: *string* - **title**: *string* - **parent**: *string* +
*pattern: ^(?:folders/[0-9]+|organizations/[0-9]+|\$folder_ids:[a-z0-9_-]+)$* - **tag_bindings**: *object*
*additional properties: false* - **`^[a-z0-9_-]+$`**: *string* ## Definitions +- **bucket**: *object* +
*additional properties: false* + - **name**: *string* + - **description**: *string* + - **iam**: *reference([iam](#refs-iam))* + - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* + - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* + - **force_destroy**: *boolean* + - **labels**: *object* + *additional properties: String* + - **location**: *string* + - **managed_folders**: *object* +
*additional properties: false* + - **`^[a-zA-Z0-9][a-zA-Z0-9_/-]+$`**: *object* +
*additional properties: false* + - **force_destroy**: *boolean* + - **iam**: *reference([iam](#refs-iam))* + - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* + - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* + - **prefix**: *string* + - **storage_class**: *string* + - **uniform_bucket_level_access**: *boolean* + - **versioning**: *boolean* - **iam**: *object*
*additional properties: false* - - **`^roles/`**: *array* + - **`^(?:roles/|\$custom_roles:)`**: *array* - items: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:)* - **iam_bindings**: *object*
*additional properties: false* - **`^[a-z0-9_-]+$`**: *object*
*additional properties: false* - **members**: *array* - items: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:)* - **role**: *string*
*pattern: ^roles/* - **condition**: *object* @@ -67,7 +110,7 @@ - **`^[a-z0-9_-]+$`**: *object*
*additional properties: false* - **member**: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:)* - **role**: *string*
*pattern: ^roles/* - **condition**: *object* @@ -77,6 +120,30 @@ - **description**: *string* - **iam_by_principals**: *object*
*additional properties: false* - - **`^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])`**: *array* + - **`^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:)`**: *array* + - items: *string* +
*pattern: ^(?:roles/|\$custom_roles:)* +- **iam_billing_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* + - items: *string* +- **iam_folder_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* + - items: *string* +- **iam_organization_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* + - items: *string* +- **iam_project_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* + - items: *string* +- **iam_sa_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* + - items: *string* +- **iam_storage_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* - items: *string* -
*pattern: ^roles/* diff --git a/fast/stages/2-project-factory/schemas/project.schema.md b/fast/stages/2-project-factory/schemas/project.schema.md index 8ff5b23c6..900523134 100644 --- a/fast/stages/2-project-factory/schemas/project.schema.md +++ b/fast/stages/2-project-factory/schemas/project.schema.md @@ -40,6 +40,9 @@ - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* - **iam_by_principals**: *reference([iam_by_principals](#refs-iam_by_principals))* - **labels**: *object* +- **log_buckets**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *reference([log_bucket](#refs-log_bucket))* - **metric_scopes**: *array* - items: *string* - **name**: *string* @@ -137,6 +140,9 @@ - **`^[a-z0-9_-]+$`**: *string* - **tags**: *object* *additional properties: Object* +- **universe**: *object* +
*additional properties: false* + - **prefix**: *string* - **vpc_sc**: *object* - ⁺**perimeter_name**: *string* - **is_dry_run**: *boolean* @@ -145,6 +151,7 @@ - **bucket**: *object*
*additional properties: false* + - **name**: *string* - **description**: *string* - **iam**: *reference([iam](#refs-iam))* - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* @@ -153,6 +160,14 @@ - **labels**: *object* *additional properties: String* - **location**: *string* + - **managed_folders**: *object* +
*additional properties: false* + - **`^[a-zA-Z0-9][a-zA-Z0-9_/-]+$`**: *object* +
*additional properties: false* + - **force_destroy**: *boolean* + - **iam**: *reference([iam](#refs-iam))* + - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* + - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* - **prefix**: *string* - **storage_class**: *string* - **uniform_bucket_level_access**: *boolean* @@ -162,18 +177,18 @@ - **`^[a-z0-9-]+$`**: *reference([bucket](#refs-bucket))* - **iam**: *object*
*additional properties: false* - - **`^roles/`**: *array* + - **`^(?:roles/|\$custom_roles:)`**: *array* - items: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:||\$iam_principals:[a-z0-9_-]+)* - **iam_bindings**: *object*
*additional properties: false* - **`^[a-z0-9_-]+$`**: *object*
*additional properties: false* - **members**: *array* - items: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:[a-z0-9_-]+)* - **role**: *string* -
*pattern: ^roles/* +
*pattern: ^(?:roles/|\$custom_roles:)* - **condition**: *object*
*additional properties: false* - ⁺**expression**: *string* @@ -184,9 +199,9 @@ - **`^[a-z0-9_-]+$`**: *object*
*additional properties: false* - **member**: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:[a-z0-9_-]+)* - **role**: *string* -
*pattern: ^[a-zA-Z0-9_/.]+$* +
*pattern: ^(?:roles/|\$custom_roles:)* - **condition**: *object*
*additional properties: false* - ⁺**expression**: *string* @@ -194,9 +209,9 @@ - **description**: *string* - **iam_by_principals**: *object*
*additional properties: false* - - **`^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])`**: *array* + - **`^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:[a-z0-9_-]+)`**: *array* - items: *string* -
*pattern: ^roles/* +
*pattern: ^(?:roles/|\$custom_roles:)* - **iam_billing_roles**: *object*
*additional properties: false* - **`^[a-z0-9-]+$`**: *array* @@ -211,13 +226,24 @@ - items: *string* - **iam_project_roles**: *object*
*additional properties: false* - - **`^[a-z0-9-]+$`**: *array* + - **`^(?:[a-z0-9-]|\$project_ids:[a-z0-9_-])+$`**: *array* - items: *string* - **iam_sa_roles**: *object*
*additional properties: false* - - **`^[a-z0-9-]+$`**: *array* + - **`^(?:\$service_account_ids:|projects/)`**: *array* - items: *string* - **iam_storage_roles**: *object*
*additional properties: false* - **`^[a-z0-9-]+$`**: *array* - items: *string* +- **log_bucket**: *object* +
*additional properties: false* + - **description**: *string* + - **kms_key_name**: *string* + - **location**: *string* + - **log_analytics**: *object* +
*additional properties: false* + - **enable**: *boolean* + - **dataset_link_id**: *string* + - **description**: *string* + - **retention**: *number* diff --git a/fast/stages/2-project-factory/variables-fast.tf b/fast/stages/2-project-factory/variables-fast.tf index f29bd4a53..19534ef3a 100644 --- a/fast/stages/2-project-factory/variables-fast.tf +++ b/fast/stages/2-project-factory/variables-fast.tf @@ -25,15 +25,10 @@ variable "automation" { variable "billing_account" { # tfdoc:variable:source 0-bootstrap - description = "Billing account id. If billing account is not part of the same org set `is_org_level` to false." + description = "Billing account id." type = object({ - id = string - is_org_level = optional(bool, true) + id = string }) - validation { - condition = var.billing_account.is_org_level != null - error_message = "Invalid `null` value for `billing_account.is_org_level`." - } } variable "custom_roles" { @@ -45,17 +40,16 @@ variable "custom_roles" { } variable "folder_ids" { - # tfdoc:variable:source 1-resman - description = "Folders created in the resource management stage." + # tfdoc:variable:source 0-bootstrap + description = "Folders created in the bootstrap stage." type = map(string) nullable = false default = {} } -variable "groups" { +variable "iam_principals" { # tfdoc:variable:source 0-bootstrap - # https://cloud.google.com/docs/enterprise/setup-checklist - description = "Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated." + description = "IAM-format principals." type = map(string) nullable = false default = {} @@ -81,7 +75,7 @@ variable "locations" { # tfdoc:variable:source 0-bootstrap description = "Optional locations for GCS, BigQuery, and logging buckets created here." type = object({ - gcs = optional(string) + storage = optional(string, "eu") }) nullable = false default = {} @@ -95,17 +89,6 @@ variable "perimeters" { default = {} } -variable "org_policy_tags" { - # tfdoc:variable:source 0-bootstrap - description = "Optional organization policy tag values." - type = object({ - key_name = optional(string, "org-policies") - values = optional(map(string), {}) - }) - nullable = false - default = {} -} - variable "prefix" { # tfdoc:variable:source 0-bootstrap description = "Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants." @@ -116,16 +99,24 @@ variable "prefix" { } } +variable "project_ids" { + # tfdoc:variable:source 0-bootstrap + description = "Projects created in the bootstrap stage." + type = map(string) + nullable = false + default = {} +} + variable "service_accounts" { - # tfdoc:variable:source 1-resman - description = "Automation service accounts in name => email format." + # tfdoc:variable:source 0-bootstrap + description = "Service accounts created in the bootstrap stage." type = map(string) nullable = false default = {} } variable "tag_values" { - # tfdoc:variable:source 1-resman + # tfdoc:variable:source 0-bootstrap description = "FAST-managed resource manager tag values." type = map(string) nullable = false diff --git a/fast/stages/2-project-factory/variables.tf b/fast/stages/2-project-factory/variables.tf index 3c2996caf..ec57ebdfe 100644 --- a/fast/stages/2-project-factory/variables.tf +++ b/fast/stages/2-project-factory/variables.tf @@ -14,37 +14,43 @@ * limitations under the License. */ -variable "factories_config" { - description = "Configuration for YAML-based factories." +variable "context" { + description = "Context-specific interpolations." type = object({ - folders_data_path = optional(string, "data/hierarchy") - projects_data_path = optional(string, "data/projects") + custom_roles = optional(map(string), {}) + folder_ids = optional(map(string), {}) + iam_principals = optional(map(string), {}) + kms_keys = optional(map(string), {}) + locations = optional(map(string), {}) + notification_channels = optional(map(string), {}) + project_ids = optional(map(string), {}) + tag_values = optional(map(string), {}) + vpc_host_projects = optional(map(string), {}) + vpc_sc_perimeters = optional(map(string), {}) + }) + default = {} + nullable = false +} + +variable "factories_config" { + description = "Path to folder with YAML resource description data files." + type = object({ + folders = optional(string, "data/folders") + projects = optional(string, "data/projects") budgets = optional(object({ - billing_account = string - budgets_data_path = optional(string, "data/budgets") - notification_channels = optional(map(any), {}) + billing_account_id = string + data = string })) - context = optional(object({ - custom_roles = optional(map(string), {}) - folder_ids = optional(map(string), {}) - kms_keys = optional(map(string), {}) - iam_principals = optional(map(string), {}) - tag_values = optional(map(string), {}) - vpc_host_projects = optional(map(string), {}) - }), {}) - projects_config = optional(object({ - key_ignores_path = optional(bool, false) - }), {}) }) nullable = false default = {} } -variable "outputs_location" { - description = "Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable." - type = string - default = null -} +# variable "outputs_location" { +# description = "Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable." +# type = string +# default = null +# } variable "stage_name" { description = "FAST stage name. Used to separate output files across different factories." diff --git a/fast/stages/2-secops/README.md b/fast/stages/2-secops/README.md index 852c50a50..8cc69e618 100644 --- a/fast/stages/2-secops/README.md +++ b/fast/stages/2-secops/README.md @@ -57,7 +57,7 @@ workforce_identity_providers = { ## How to run this stage -This stage is meant to be executed after the [resource management](../1-resman) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured in the [bootstrap](../0-bootstrap) stage. +This stage is meant to be executed after the [bootstrap](../0-bootstrap) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured there. It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the previous stages for the environmental requirements. diff --git a/fast/stages/2-security/README.md b/fast/stages/2-security/README.md index 50d34149b..e78c9b0d2 100644 --- a/fast/stages/2-security/README.md +++ b/fast/stages/2-security/README.md @@ -52,7 +52,7 @@ With this stage you can leverage Certificate Authority Services (CAS) and create ## How to run this stage -This stage is meant to be executed after the [resource management](../1-resman) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured in the [bootstrap](../0-bootstrap) stage. +This stage is meant to be executed after the [bootstrap](../0-bootstrap) stage has run, as it leverages the automation service account and bucket created there, and additional resources configured there. It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the previous stages for the environmental requirements. diff --git a/fast/stages/3-data-platform-dev/README.md b/fast/stages/3-data-platform-dev/README.md index d9724ba75..3b7f7e706 100644 --- a/fast/stages/3-data-platform-dev/README.md +++ b/fast/stages/3-data-platform-dev/README.md @@ -9,7 +9,7 @@ While our solution is conceptually guided by [Data Mesh principles on Google Clo - [Design Overview and Choices](#design-overview-and-choices) - [Data Platform Architecture](#data-platform-architecture) - - [Folder & Project Structure](#folder-project-structure) + - [Folder and Project Structure](#folder-and-project-structure) - [Central Shared Services (Federated Governance)](#central-shared-services-federated-governance) - [Data Domains (Domain-Driven Ownership)](#data-domains-domain-driven-ownership) - [Data Products (DaaP)](#data-products-daap) @@ -38,7 +38,7 @@ The following diagram represent the high-level architecture of the Data Platform High level diagram.

-### Folder & Project Structure +### Folder and Project Structure The stage manages the following three high-level logical components implemented via GCP folders and projects: @@ -69,7 +69,7 @@ To support this ownership model and ensure clear separation, each logical Data D Within each Data Domain, a corresponding Google Cloud "Data Domain" project serves as the primary container for all its specific services and resources. A dedicated Cloud Composer environment is provisioned within this project for orchestrating the domain's data workflows. To adhere to the principle of least privilege, this Composer environment operates with a dedicated IAM Service Account capable of impersonating the necessary Data Product-specific service accounts within that domain. -Define data domains by creating individual sub-folders within the `data/data-domains` directory. Each domain's configuration, including IAM permissions, services to enable in its shared folder, and settings for its Cloud Composer instance, should be specified in a `_config.yaml` file within its respective subfolder. Refer to the +Define data domains by creating individual sub-folders within the `data/data-domains` directory. Each domain's configuration, including IAM permissions, services to enable in its shared folder, and settings for its Cloud Composer instance, should be specified in a `_config.yaml` file within its respective subfolder. Refer to the We recommend granting data consumers access to exposed data product metadata through IAM Secure Tags created in the central project. @@ -136,7 +136,6 @@ Please note that the above access scopes and the example configurations provided Refer to the [terraform.tfvars.sample](terraform.tfvars.sample), ["domain-0" _config.yaml](./data/data-domains/domain-0/_config.yaml) and [."domain-0" product-0.yaml](./data/data-domains/domain-0/product-0.yaml) files as a starting point for managing IAM. - #### Central Data Platform Team This team defines the overall data platform architecture, establishes shared infrastructure, and enforces central data governance policies and standards across the data mesh. It empowers Data Producers with building blocks and best practices, ensuring high data quality, security, and trustworthiness for consumers. The primary focus is on providing the foundations for a self-serve data platform and universal governance standards for all users. The Central Data Platform team often collaborates with Data Governance functions within the enterprise. @@ -163,6 +162,7 @@ This team is responsible for the end-to-end lifecycle of a specific Data Product Typically, this group has `ADMIN` access to resources within their Data Product project(s). They also usually have `READ/USAGE` access to relevant resources in the Central Shared Services project (e.g., Aspect Types) and the Data Domain's top-level shared project (e.g., Cloud Composer). This team is generally not the primary owner for configuring overarching IAM bindings, as that responsibility often lies elsewhere. Key responsibilities for the Data Product Team include: + - Identifying and configuring the necessary resources within their data product project to perform ETL operations for the exposure layer. These resources should be deployed in a separate Terraform state, using the dedicated automation service account created for each data product. - Configuring Policy Tags to protect PII and sensitive data. - Defining and managing metadata for aspects related to tables and other resources within their data product's exposure layer. @@ -171,67 +171,17 @@ When using BigQuery in the exposure layer, we recommend using [authorized datase ## How to run this stage -If this stage is deployed within a FAST-based GCP organization, we recommend executing it after foundational FAST `stage-2` components like `networking` and `security`. This sequencing is advisable as specific data platform features in this stage might depend on configurations from these earlier stages. Although this stage can be run independently, instructions for such a standalone setup are beyond the scope of this document. +If this stage is deployed within a FAST-based GCP organization, we recommend executing it after foundational FAST `stage-2` components like `networking` and `security`. This is the recommended flow as specific data platform features in this stage might depend on configurations from these earlier stages. Although this stage can be run independently, instructions for such a standalone setup are beyond the scope of this document. ### FAST prerequisites -This stage needs specific permissions granted to its automation service accounts on networking and security resources. +This stage needs specific automation resources, and permissions granted on those that allow control of selective IAM roles on specific networking and security resources. Network permissions are needed to associate data domain or product projects to Shared VPC hosts and grant network permissions to data platform managed service accounts. They are mandatory when deploying Composer. Security permissions are only needed when using CMEK encryption, to grant the relevant IAM roles to data platform service agents on the encryption keys used. -The networking and security configuration need to be defined in the resource management stage via specific YAML code blocks: two are needed for networking, and one for security. - -The first networking code block grants the relevant roles on the Networking folder to the Data Platform service accounts, with a condition on the environment tag. - -```yaml -# make sure this block exists in the data/stage-2/networking.yaml file - iam_bindings_additive: - # Data Platform (dev) - dp_dev_net_admin: - role: service_project_network_admin - member: data-platform-dev-rw - condition: - title: Data platform dev service project admin. - expression: | - resource.matchTag('${organization.id}/${tag_names.environment}', 'development') - dp_dev_net_viewer: - role: roles/compute.networkViewer - member: data-platform-dev-ro - condition: - title: Data platform dev network viewer. - expression: | - resource.matchTag('${organization.id}/${tag_names.environment}', 'development') -``` - -The second networking code block signals the networking stage that the Data Platform service accounts need delegated IAM grants on the dev network project, in order to be able to assign specific roles on it. - -```yaml -# make sure this block exists in the data/stage-2/networking.yaml file -stage3_config: - iam_admin_delegated: - - environment: dev - principal: data-platform-dev-rw - iam_viewer: - - environment: dev - principal: data-platform-dev-ro -``` - -For security, a block similar to the one above is needed. - -```yaml -# make sure this block exists in the data/stage-2/security.yaml file -stage3_config: - iam_admin_delegated: - - environment: dev - principal: data-platform-dev-rw - iam_viewer: - - environment: dev - principal: data-platform-dev-ro -``` - -Once the two above configurations are in place, apply the resource management, networking and security stages in succession. Be sure to refresh the tfvars files in the network and security stages if needed (e.g. by re-running `fast-links.sh`). +The ["Classic FAST" dataset](../0-bootstrap/README.md#classic-fast-dataset) in the bootstrap stage already contains the configuration for a development Data Platform. Adapting it to multiple environments, or for a multi-environment setup is relatively trivial and left as an exercise to the user. ### Provider and Terraform variables diff --git a/fast/stages/3-gcve-dev/README.md b/fast/stages/3-gcve-dev/README.md index a2a008cc0..b22011e4f 100644 --- a/fast/stages/3-gcve-dev/README.md +++ b/fast/stages/3-gcve-dev/README.md @@ -13,7 +13,7 @@ The setup configured here is for a single environment in a single region, and is - [Single-region per-environment GCVE deployment](#single-region-per-environment-gcve-deployment) - [Multi-regional deployments](#multi-regional-deployments) - [How to run this stage](#how-to-run-this-stage) - - [Resource management configuration](#resource-management-configuration) + - [FAST prerequisites](#fast-prerequisites) - [Provider and Terraform variables](#provider-and-terraform-variables) - [Impersonating the automation service account](#impersonating-the-automation-service-account) - [Variable configuration](#variable-configuration) @@ -81,72 +81,17 @@ A design for a multi-regional deployment with the NVA FAST networking stage is s ## How to run this stage -This stage is meant to be executed after the FAST "foundational" stages: bootstrap, resource management, security and networking stages. +If this stage is deployed within a FAST-based GCP organization, we recommend executing it after foundational FAST `stage-2` components like `networking` and `security`. This is the recommended flow as specific data platform features in this stage might depend on configurations from these earlier stages. Although this stage can be run independently, instructions for such a standalone setup are beyond the scope of this document. -It is also possible to run this stage in isolation. Refer to the *[Running in isolation](#running-in-isolation)* section below for details. +### FAST prerequisites -Before running this stage, you need to make sure you have the correct credentials and permissions, and localize variables by assigning values that match your configuration. +This stage needs specific automation resources, and permissions granted on those that allow control of selective IAM roles on specific networking and security resources. -### Resource management configuration +Network permissions are needed to associate data domain or product projects to Shared VPC hosts and grant network permissions to data platform managed service accounts. They are mandatory when deploying Composer. -Some configuration changes are needed in resource management before this stage can be run. +Security permissions are only needed when using CMEK encryption, to grant the relevant IAM roles to data platform service agents on the encryption keys used. -First, define a parent folder for each stage environment folder in the `data/top-level-folder` folder [in the resource management stage](../1-resman/data/top-level-folders/). As an example, this YAML definition creates a `GCVE` folder under the organization: - -```yaml -# yaml-language-server: $schema=../../schemas/top-level-folder.schema.json - -name: GCVE - -# IAM bindings and organization policies can also be defined here -``` - -Then, make sure the stage 3 is enabled in the `data/stage-3` folder [in the resource management stage](../1-resman/data/stage-3/). As an example, this YAML definition saved as `gcve-dev.yaml` enables this stage 3 for the development environment: - -```yaml -# yaml-language-server: $schema=../../schemas/fast-stage3.schema.json - -short_name: gcve -environment: dev -folder_config: - name: Development - parent_id: gcve -``` - -Then edit the definition of the networking stage 2 in the `data/stage2` folder [in the resource management stage](../1-resman/data/stage-2/) to include the IAM configuration for GCVE. The following are example snippets for GCVE dev, make sure they match the `short_name` and `environment` configured above. - -In `folder_config.iam_bindings_additive` add: - -```yaml -# folder_config: - # iam_bindings_additive: - gcve_dev_net_admin: - role: gcve_network_admin - member: gcve-dev-rw - condition: - title: GCVE dev network admin. - expression: | - resource.matchTag('${organization.id}/${tag_names.environment}', 'development') - gcve_dev_net_viewer: - role: gcve_network_viewer - member: gcve-dev-ro - condition: - title: GCVE dev network viewer. - expression: | - resource.matchTag('${organization.id}/${tag_names.environment}', 'development') -``` - -In `stage3_config` add the following so that the networking stage grants IAM delegated permissions to this stage's service accounts: - -```yaml -# stage3_config: - iam_admin_delegated: - - environment: dev - principal: gcve-dev-rw - iam_viewer: - - environment: dev - principal: gcve-dev-ro -``` +The ["Classic FAST" dataset](../0-bootstrap/README.md#classic-fast-dataset) in the bootstrap stage contains the configuration for a development Data Platform that can be easily adapted to serve for this stage. ### Provider and Terraform variables diff --git a/fast/stages/3-gke-dev/README.md b/fast/stages/3-gke-dev/README.md index bf95e9bcf..bb9761df0 100644 --- a/fast/stages/3-gke-dev/README.md +++ b/fast/stages/3-gke-dev/README.md @@ -11,11 +11,7 @@ The following diagram illustrates the high-level design of created resources, wh - [Design overview and choices](#design-overview-and-choices) - [How to run this stage](#how-to-run-this-stage) - - [Resource management configuration](#resource-management-configuration) - - [Provider and Terraform variables](#provider-and-terraform-variables) - - [Impersonating the automation service account](#impersonating-the-automation-service-account) - - [Variable configuration](#variable-configuration) - - [Running the stage](#running-the-stage) + - [FAST prerequisites](#fast-prerequisites) - [Customizations](#customizations) - [Clusters and node pools](#clusters-and-node-pools) - [Fleet management](#fleet-management) @@ -54,113 +50,17 @@ Some high level choices applied here: ## How to run this stage -This stage is meant to be executed after the FAST "foundational" stages: bootstrap, resource management, security and networking stages. +If this stage is deployed within a FAST-based GCP organization, we recommend executing it after foundational FAST `stage-2` components like `networking` and `security`. This is the recommended flow as specific data platform features in this stage might depend on configurations from these earlier stages. Although this stage can be run independently, instructions for such a standalone setup are beyond the scope of this document. -It's of course possible to run this stage in isolation, refer to the *[Running in isolation](#running-in-isolation)* section below for details. +### FAST prerequisites -Before running this stage, you need to make sure you have the correct credentials and permissions, and localize variables by assigning values that match your configuration. +This stage needs specific automation resources, and permissions granted on those that allow control of selective IAM roles on specific networking and security resources. -### Resource management configuration +Network permissions are needed to associate data domain or product projects to Shared VPC hosts and grant network permissions to data platform managed service accounts. They are mandatory when deploying Composer. -Some configuration changes are needed in resource management before this stage can be run. +Security permissions are only needed when using CMEK encryption, to grant the relevant IAM roles to data platform service agents on the encryption keys used. -First, define a parent folder for each stage environment folder in the `data/top-level-folder` folder [in the resource management stage](../1-resman/data/top-level-folders/). As an example, this YAML definition creates a `GKE` folder under the organization: - -```yaml -# yaml-language-server: $schema=../../schemas/top-level-folder.schema.json - -name: GKE - -# IAM bindings and organization policies can also be defined here -``` - -Then, edit the definition of the networking stage 2 in the `data/stage2` folder [in the resource management stage](../1-resman/data/stage-2/) to include the IAM configuration for GKE. The following are example snippets for GKE dev, make sure they match the `short_name` and `environment` configured above. - -In `folder_config.iam_bindings_additive` add: - -```yaml -# folder_config: - # iam_bindings_additive: - gke_dns_admin: - role: roles/dns.admin - member: gke-dev-ro - condition: - title: GKE dev DNS admin. - expression: | - resource.matchTag('${organization.id}/${tag_names.environment}', 'development') - gke_dns_reader: - role: roles/dns.reader - member: gke-dev-ro - condition: - title: GKE dev DNS reader. - expression: | - resource.matchTag('${organization.id}/${tag_names.environment}', 'development') -``` - -### Provider and Terraform variables - -As all other FAST stages, the [mechanism used to pass variable values and pre-built provider files from one stage to the next](../0-bootstrap/README.md#output-files-and-cross-stage-variables) is also leveraged here. - -The commands to link or copy the provider and terraform variable files can be easily derived from the `fast-links.sh` script in the FAST root folder, passing it a single argument with the local output files folder (if configured) or the GCS output bucket in the automation project (derived from stage 0 outputs). The following examples demonstrate both cases, and the resulting commands that then need to be copy/pasted and run. - -```bash -../fast-links.sh ~/fast-config - -# File linking commands for GKE (dev) stage - -# provider file -ln -s ~/fast-config/providers/3-gke-dev-providers.tf ./ - -# input files from other stages -ln -s ~/fast-config/tfvars/0-globals.auto.tfvars.json ./ -ln -s ~/fast-config/tfvars/0-bootstrap.auto.tfvars.json ./ -ln -s ~/fast-config/tfvars/1-resman.auto.tfvars.json ./ -ln -s ~/fast-config/tfvars/2-networking.auto.tfvars.json ./ - -# conventional place for stage tfvars (manually created) -ln -s ~/fast-config/3-gke-dev.auto.tfvars ./ -``` - -```bash -../fast-links.sh gs://xxx-prod-iac-core-outputs-0 - -# File linking commands for GKE (dev) stage - -# provider file -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/3-gke-dev-providers.tf ./ - -# input files from other stages -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/2-networking.auto.tfvars.json ./ - -# conventional place for stage tfvars (manually created) -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/3-gke-dev.auto.tfvars ./ -``` - -### Impersonating the automation service account - -The preconfigured provider file uses impersonation to run with this stage's automation service account's credentials. The `gcp-devops` and `organization-admins` groups have the necessary IAM bindings in place to do that, so make sure the current user is a member of one of those groups. - -### Variable configuration - -Variables in this stage -- like most other FAST stages -- are broadly divided into three separate sets: - -- variables which refer to global values for the whole organization (org id, billing account id, prefix, etc.), which are pre-populated via the `0-globals.auto.tfvars.json` file linked or copied above -- variables which refer to resources managed by previous stage, which are prepopulated here via the `*.auto.tfvars.json` files linked or copied above -- and finally variables that optionally control this stage's behaviour and customizations, and can to be set in a custom `terraform.tfvars` file - -The latter set is explained in the [Customization](#customizations) sections below, and the full list can be found in the [Variables](#variables) table at the bottom of this document. - -### Running the stage - -Once provider and variable values are in place and the correct user is configured, the stage can be run: - -```bash -terraform init -terraform apply -``` +The ["Classic FAST" dataset](../0-bootstrap/README.md#classic-fast-dataset) in the bootstrap stage contains the configuration for a development Data Platform that can be easily adapted to serve for this stage. ## Customizations diff --git a/fast/stages/3-secops-dev/README.md b/fast/stages/3-secops-dev/README.md index 70c100b01..e31897d06 100644 --- a/fast/stages/3-secops-dev/README.md +++ b/fast/stages/3-secops-dev/README.md @@ -11,11 +11,7 @@ The following diagram illustrates the high-level design of SecOps instance confi - [Design overview and choices](#design-overview-and-choices) - [How to run this stage](#how-to-run-this-stage) - - [Resource management configuration](#resource-management-configuration) - - [Provider and Terraform variables](#provider-and-terraform-variables) - - [Impersonating the automation service account](#impersonating-the-automation-service-account) - - [Variable configuration](#variable-configuration) - - [Running the stage](#running-the-stage) + - [FAST prerequisites](#fast-prerequisites) - [Customizations](#customizations) - [Data RBAC](#data-rbac) - [SecOps rules and reference list management](#secops-rules-and-reference-list-management) @@ -35,99 +31,22 @@ Some high level features of the current version of the stage are: - Data RBAC configuration with labels and scopes - IAM setup for the SecOps instance based on groups from Cloud Identity or WIF (with supports for Data RBAC) - Detection Rules and reference lists management via terraform (leveraging [secops-rules](../../../modules/secops-rules) module) -- API Key setup for Webhook feeds +- API Key setup for Webhook feeds - Integration with Workspace for alerts and logs ingestion via SecOps Feeds ## How to run this stage -This stage is meant to be executed after the FAST "foundational" stages: bootstrap, resource management, secops stages. +If this stage is deployed within a FAST-based GCP organization, we recommend executing it after foundational FAST `stage-2` components like `networking` and `security`. This is the recommended flow as specific data platform features in this stage might depend on configurations from these earlier stages. Although this stage can be run independently, instructions for such a standalone setup are beyond the scope of this document. -It's of course possible to run this stage in isolation, refer to the *[Running in isolation](#running-in-isolation)* section below for details. +### FAST prerequisites -Before running this stage, you need to make sure you have the correct credentials and permissions, and localize variables by assigning values that match your configuration. +This stage needs specific automation resources, and permissions granted on those that allow control of selective IAM roles on specific networking and security resources. -### Resource management configuration +Network permissions are needed to associate data domain or product projects to Shared VPC hosts and grant network permissions to data platform managed service accounts. They are mandatory when deploying Composer. -Some configuration changes are needed in resource management before this stage can be run. +Security permissions are only needed when using CMEK encryption, to grant the relevant IAM roles to data platform service agents on the encryption keys used. -Make sure the stage 3 is enabled in the `data/stage-3` folder [in the resource management stage](../1-resman/data/stage-3/). As an example, this YAML definition saved as `secops-dev.yaml` enables this stage 3 for the development environment: - -```yaml -# yaml-language-server: $schema=../../schemas/fast-stage3.schema.json - -short_name: secops -environment: dev -folder_config: - name: Development - parent_id: secops -``` - -Make sure the stage 3 definitions are aligned with the environments you would like to setup for SecOps and coherent with the environments definitions in the stage 2 [2-secops](../2-secops) in order to have a dedicated stage 3 for SecOps for each environment (dev and prod as an example). - -### Provider and Terraform variables - -As all other FAST stages, the [mechanism used to pass variable values and pre-built provider files from one stage to the next](../0-bootstrap/README.md#output-files-and-cross-stage-variables) is also leveraged here. - -The commands to link or copy the provider and terraform variable files can be easily derived from the `fast-links.sh` script in the FAST root folder, passing it a single argument with the local output files folder (if configured) or the GCS output bucket in the automation project (derived from stage 0 outputs). The following examples demonstrate both cases, and the resulting commands that then need to be copy/pasted and run. - -```bash -../fast-links.sh ~/fast-config - -# File linking commands for GKE (dev) stage - -# provider file -ln -s ~/fast-config/providers/3-secops-dev-providers.tf ./ - -# input files from other stages -ln -s ~/fast-config/tfvars/0-globals.auto.tfvars.json ./ -ln -s ~/fast-config/tfvars/0-bootstrap.auto.tfvars.json ./ -ln -s ~/fast-config/tfvars/1-resman.auto.tfvars.json ./ -ln -s ~/fast-config/tfvars/2-secops.auto.tfvars.json ./ - -# conventional place for stage tfvars (manually created) -ln -s ~/fast-config/3-secops-dev.auto.tfvars ./ -``` - -```bash -../fast-links.sh gs://xxx-prod-iac-core-outputs-0 - -# File linking commands for GKE (dev) stage - -# provider file -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/3-secops-dev-providers.tf ./ - -# input files from other stages -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json ./ -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/2-secops.auto.tfvars.json ./ - -# conventional place for stage tfvars (manually created) -gcloud storage cp gs://xxx-prod-iac-core-outputs-0/3-secops-dev.auto.tfvars ./ -``` - -### Impersonating the automation service account - -The preconfigured provider file uses impersonation to run with this stage's automation service account's credentials. The `gcp-devops` and `organization-admins` groups have the necessary IAM bindings in place to do that, so make sure the current user is a member of one of those groups. - -### Variable configuration - -Variables in this stage -- like most other FAST stages -- are broadly divided into three separate sets: - -- variables which refer to global values for the whole organization (org id, billing account id, prefix, etc.), which are pre-populated via the `0-globals.auto.tfvars.json` file linked or copied above -- variables which refer to resources managed by previous stage, which are prepopulated here via the `*.auto.tfvars.json` files linked or copied above -- and finally variables that optionally control this stage's behaviour and customizations, and can to be set in a custom `terraform.tfvars` file - -The latter set is explained in the [Customization](#customizations) sections below, and the full list can be found in the [Variables](#variables) table at the bottom of this document. - -### Running the stage - -Once provider and variable values are in place and the correct user is configured, the stage can be run: - -```bash -terraform init -terraform apply -``` +The ["Classic FAST" dataset](../0-bootstrap/README.md#classic-fast-dataset) in the bootstrap stage contains the configuration for a development Data Platform that can be easily adapted to serve for this stage. ## Customizations @@ -137,7 +56,7 @@ This stage is designed with few basic integrations provided out of the box which This stage supports configuration of [SecOps Data RBAC](https://cloud.google.com/chronicle/docs/administration/datarbac-overview) using two separate variables: -- `secops_data_rbac_config`: specifies Data RBAC [label and scopes](https://cloud.google.com/chronicle/docs/administration/configure-datarbac-users) in Google SecOps +- `secops_data_rbac_config`: specifies Data RBAC [label and scopes](https://cloud.google.com/chronicle/docs/administration/configure-datarbac-users) in Google SecOps - `secops_iam`: defines SecOps IAM configuration in {PRINCIPAL => {roles => [ROLES], scopes => [SCOPES]}} format referencing previously defined scopes. When scope is populated a [IAM condition](https://cloud.google.com/chronicle/docs/administration/configure-datarbac-users#assign-scope-to-users) restrict access to those scopes. Example of a Data RBAC configuration is reported below. diff --git a/fast/stages/README.md b/fast/stages/README.md index b5d7b869a..8333c61be 100644 --- a/fast/stages/README.md +++ b/fast/stages/README.md @@ -21,25 +21,23 @@ Stages encapsulate core designs and functionality that is common in most type of To destroy a previous FAST deployment follow the instructions detailed in [cleanup](CLEANUP.md). -## Organization (experimental) +## Organization (0) -- [Experimental Bootstrap](./0-bootstrap-experimental/README.md) +- [Bootstrap](./0-bootstrap/README.md) This stage combines the legacy bootstrap and resource management stages described below, allowing easy configuration of all related resources via factories. Its flexibility supports any type of organizational design, while still supporting traditional FAST stages like VPC Service Controls, security, networking, and any stage 3. - -## Organization (0 and 1) - -- [Bootstrap](0-bootstrap/README.md) - Enables critical organization-level functionality that depends on broad permissions. It has two primary purposes. The first is to bootstrap the resources needed for automation of this and the following stages (service accounts, GCS buckets). And secondly, it applies the minimum amount of configuration needed at the organization level to avoid the need of broad permissions later on, and to implement from the start critical auditing or security features like organization policies, sinks and exports.\ - Exports: automation variables, organization-level custom roles -- [Resource Management](1-resman/README.md) - Creates the base resource hierarchy (folders) and the automation resources that will be required later to delegate deployment of each part of the hierarchy to separate stages. This stage also configures resource management tags used in scoping specific IAM roles on the resource hierarchy.\ - Exports: folder ids, automation service account emails, tags - [VPC Service Controls](./1-vpcsc/README.md) Optionally configures VPC Service Controls protection for the organization. + +## Legacy Organization (0 and 1) -## Multitenancy +These stages are considered legacy, and kept in this release to simplify migration to the new bootstrap stage. They will be dropped from the next release. -Implemented as an [add-on stage 1](../addons/1-resman-tenants/), with optional FAST compatibility for tenants. +- [Bootstrap](0-bootstrap-legacy/README.md) + Enables critical organization-level functionality that depends on broad permissions. It has two primary purposes. The first is to bootstrap the resources needed for automation of this and the following stages (service accounts, GCS buckets). And secondly, it applies the minimum amount of configuration needed at the organization level to avoid the need of broad permissions later on, and to implement from the start critical auditing or security features like organization policies, sinks and exports.\ + Exports: automation variables, organization-level custom roles +- [Resource Management](1-resman-legacy/README.md) + Creates the base resource hierarchy (folders) and the automation resources that will be required later to delegate deployment of each part of the hierarchy to separate stages. This stage also configures resource management tags used in scoping specific IAM roles on the resource hierarchy.\ + Exports: folder ids, automation service account emails, tags ## Shared resources (2) @@ -51,6 +49,8 @@ Implemented as an [add-on stage 1](../addons/1-resman-tenants/), with optional F Exports: host project ids and numbers, vpc self links - [Project Factory](./2-project-factory/) YAML-based factory to create and configure application or team-level projects. Configuration includes VPC-level settings for Shared VPC, service-level configuration for CMEK encryption via centralized keys, and service account creation for workloads and applications. This stage can be cloned if an org-wide or dedicated per-environment factories are needed. +- [Legacy Project Factory](./2-project-factory-legacy/) + More limited version of the project factory, that can be used for backward compatibility. Will be dropped in the next major release. ## Environment-level resources (3) diff --git a/fast/stages/UPGRADING.md b/fast/stages/UPGRADING.md index c6fde1328..39d1d92fa 100644 --- a/fast/stages/UPGRADING.md +++ b/fast/stages/UPGRADING.md @@ -40,7 +40,7 @@ As usual, consider this a guideline with no guarantees. Migrations between FAST The Resource Management stage has been largely refactored, adopting factories to simplify the creation of multiple environments and the creation and deployment of new "Stage 3" stages. Before upgrading it's highly recommended to familiarize yourself with the documentation, to assess whether your specific configurations need to be migrated to the new variables. -The [file containing moved blocks](./1-resman/moved/v35.1.0-v36.0.0.tf) for this release can be used to preserve most of the important resources which changed from the previous release. Just link it in the stage and plan/apply to see the remaining changes. +The [file containing moved blocks](./1-resman-legacy/moved/v35.1.0-v36.0.0.tf) for this release can be used to preserve most of the important resources which changed from the previous release. Just link it in the stage and plan/apply to see the remaining changes. The moved blocks are not exhaustive and do not include resources that can be dropped and recreated with limited impact like IAM and tag bindings. As usual, proceed with care as we provide no guarantee, just a starting point. diff --git a/modules/folder/README.md b/modules/folder/README.md index 01b7b0dee..05dfdbc9a 100644 --- a/modules/folder/README.md +++ b/modules/folder/README.md @@ -171,7 +171,7 @@ module "folder" { org_policies = "configs/org-policies/" } context = { - org_policies = { + condition_vars = { tags = { my_conditional_tag = "tagKeys/1234" } @@ -425,7 +425,7 @@ module "folder" { |---|---|:---:|:---:|:---:| | [assured_workload_config](variables.tf#L17) | Create AssuredWorkloads folder instead of regular folder when value is provided. Incompatible with folder_create=false. | object({…}) | | null | | [contacts](variables.tf#L70) | List of essential contacts for this resource. Must be in the form EMAIL -> [NOTIFICATION_TYPES]. Valid notification types are ALL, SUSPENSION, SECURITY, TECHNICAL, BILLING, LEGAL, PRODUCT_UPDATES. | map(list(string)) | | {} | -| [context](variables.tf#L78) | Context-specific interpolations. | object({…}) | | {} | +| [context](variables.tf#L78) | Context-specific interpolations. | object({…}) | | {} | | [deletion_protection](variables.tf#L91) | Deletion protection setting for this folder. | bool | | false | | [factories_config](variables.tf#L97) | Paths to data files and folders that enable factory functionality. | object({…}) | | {} | | [firewall_policy](variables.tf#L106) | Hierarchical firewall policy to associate to this folder. | object({…}) | | null | diff --git a/modules/folder/iam.tf b/modules/folder/iam.tf index 1c96a9db7..5ec8e46ec 100644 --- a/modules/folder/iam.tf +++ b/modules/folder/iam.tf @@ -66,7 +66,9 @@ resource "google_folder_iam_binding" "bindings" { dynamic "condition" { for_each = each.value.condition == null ? [] : [""] content { - expression = each.value.condition.expression + expression = templatestring( + each.value.condition.expression, var.context.condition_vars + ) title = each.value.condition.title description = each.value.condition.description } @@ -83,7 +85,9 @@ resource "google_folder_iam_member" "bindings" { dynamic "condition" { for_each = each.value.condition == null ? [] : [""] content { - expression = each.value.condition.expression + expression = templatestring( + each.value.condition.expression, var.context.condition_vars + ) title = each.value.condition.title description = each.value.condition.description } diff --git a/modules/folder/main.tf b/modules/folder/main.tf index 70054658b..75d099846 100644 --- a/modules/folder/main.tf +++ b/modules/folder/main.tf @@ -18,7 +18,7 @@ locals { ctx = { for k, v in var.context : k => { for kk, vv in v : "${local.ctx_p}${k}:${kk}" => vv - } + } if k != "condition_vars" } ctx_p = "$" folder_id = ( diff --git a/modules/folder/organization-policies.tf b/modules/folder/organization-policies.tf index 2b546e699..0bfa8074f 100644 --- a/modules/folder/organization-policies.tf +++ b/modules/folder/organization-policies.tf @@ -34,7 +34,7 @@ locals { all = try(r.allow.all, null) values = ( can(r.allow.values) - ? [for x in r.allow.values : templatestring(x, var.context.org_policies)] + ? [for x in r.allow.values : templatestring(x, var.context.condition_vars)] : null ) } : null @@ -42,7 +42,7 @@ locals { all = try(r.deny.all, null) values = ( can(r.deny.values) - ? [for x in r.deny.values : templatestring(x, var.context.org_policies)] + ? [for x in r.deny.values : templatestring(x, var.context.condition_vars)] : null ) } : null @@ -50,28 +50,28 @@ locals { condition = { description = ( can(r.condition.description) - ? templatestring(r.condition.description, var.context.org_policies) + ? templatestring(r.condition.description, var.context.condition_vars) : null ) expression = ( can(r.condition.expression) - ? templatestring(r.condition.expression, var.context.org_policies) + ? templatestring(r.condition.expression, var.context.condition_vars) : null ) location = ( can(r.condition.location) - ? templatestring(r.condition.location, var.context.org_policies) + ? templatestring(r.condition.location, var.context.condition_vars) : null ) title = ( can(r.condition.title) - ? templatestring(r.condition.title, var.context.org_policies) + ? templatestring(r.condition.title, var.context.condition_vars) : null ) } parameters = ( can(r.parameters) - ? templatestring(r.parameters, var.context.org_policies) + ? templatestring(r.parameters, var.context.condition_vars) : null ) } diff --git a/modules/folder/variables.tf b/modules/folder/variables.tf index e62acbd32..94084f243 100644 --- a/modules/folder/variables.tf +++ b/modules/folder/variables.tf @@ -78,10 +78,10 @@ variable "contacts" { variable "context" { description = "Context-specific interpolations." type = object({ + condition_vars = optional(map(map(string)), {}) custom_roles = optional(map(string), {}) folder_ids = optional(map(string), {}) iam_principals = optional(map(string), {}) - org_policies = optional(map(map(string)), {}) tag_values = optional(map(string), {}) }) default = {} diff --git a/modules/gcs/README.md b/modules/gcs/README.md index 1c64b5f97..4985f26ad 100644 --- a/modules/gcs/README.md +++ b/modules/gcs/README.md @@ -370,41 +370,41 @@ module "bucket" { | name | description | type | required | default | |---|---|:---:|:---:|:---:| -| [name](variables.tf#L202) | Bucket name suffix. | string | ✓ | | -| [project_id](variables.tf#L260) | Bucket project id. | string | ✓ | | +| [name](variables.tf#L203) | Bucket name suffix. | string | ✓ | | +| [project_id](variables.tf#L261) | Bucket project id. | string | ✓ | | | [autoclass](variables.tf#L17) | Enable autoclass to automatically transition objects to appropriate storage classes based on their access pattern. If set to true, storage_class must be set to STANDARD. Defaults to false. | bool | | null | | [bucket_create](variables.tf#L23) | Create bucket. | bool | | true | -| [context](variables.tf#L29) | Context-specific interpolations. | object({…}) | | {} | -| [cors](variables.tf#L42) | CORS configuration for the bucket. Defaults to null. | object({…}) | | null | -| [custom_placement_config](variables.tf#L53) | The bucket's custom location configuration, which specifies the individual regions that comprise a dual-region bucket. If the bucket is designated as REGIONAL or MULTI_REGIONAL, the parameters are empty. | list(string) | | null | -| [default_event_based_hold](variables.tf#L59) | Enable event based hold to new objects added to specific bucket, defaults to false. | bool | | null | -| [enable_hierarchical_namespace](variables.tf#L65) | Enables hierarchical namespace. | bool | | null | -| [enable_object_retention](variables.tf#L71) | Enables object retention on a storage bucket. | bool | | null | -| [encryption_key](variables.tf#L77) | KMS key that will be used for encryption. | string | | null | -| [force_destroy](variables.tf#L83) | Optional map to set force destroy keyed by name, defaults to false. | bool | | false | +| [context](variables.tf#L29) | Context-specific interpolations. | object({…}) | | {} | +| [cors](variables.tf#L43) | CORS configuration for the bucket. Defaults to null. | object({…}) | | null | +| [custom_placement_config](variables.tf#L54) | The bucket's custom location configuration, which specifies the individual regions that comprise a dual-region bucket. If the bucket is designated as REGIONAL or MULTI_REGIONAL, the parameters are empty. | list(string) | | null | +| [default_event_based_hold](variables.tf#L60) | Enable event based hold to new objects added to specific bucket, defaults to false. | bool | | null | +| [enable_hierarchical_namespace](variables.tf#L66) | Enables hierarchical namespace. | bool | | null | +| [enable_object_retention](variables.tf#L72) | Enables object retention on a storage bucket. | bool | | null | +| [encryption_key](variables.tf#L78) | KMS key that will be used for encryption. | string | | null | +| [force_destroy](variables.tf#L84) | Optional map to set force destroy keyed by name, defaults to false. | bool | | false | | [iam](variables-iam.tf#L17) | IAM bindings in {ROLE => [MEMBERS]} format. | map(list(string)) | | {} | | [iam_bindings](variables-iam.tf#L23) | Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. | map(object({…})) | | {} | | [iam_bindings_additive](variables-iam.tf#L38) | Individual additive IAM bindings. Keys are arbitrary. | map(object({…})) | | {} | | [iam_by_principals](variables-iam.tf#L53) | Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the `iam` variable. | map(list(string)) | | {} | -| [ip_filter](variables.tf#L89) | The bucket's IP filter configuration. | object({…}) | | null | -| [labels](variables.tf#L100) | Labels to be attached to all buckets. | map(string) | | {} | -| [lifecycle_rules](variables.tf#L106) | Bucket lifecycle rule. | map(object({…})) | | {} | -| [location](variables.tf#L155) | Bucket location. | string | | null | -| [logging_config](variables.tf#L165) | Bucket logging configuration. | object({…}) | | null | -| [managed_folders](variables.tf#L174) | Managed folders to create within the bucket in {PATH => CONFIG} format. | map(object({…})) | | {} | -| [notification_config](variables.tf#L207) | GCS Notification configuration. | object({…}) | | null | -| [objects_to_upload](variables.tf#L224) | Objects to be uploaded to bucket. | map(object({…})) | | {} | -| [prefix](variables.tf#L250) | Optional prefix used to generate the bucket name. | string | | null | -| [public_access_prevention](variables.tf#L265) | Prevents public access to the bucket. | string | | null | -| [requester_pays](variables.tf#L275) | Enables Requester Pays on a storage bucket. | bool | | null | -| [retention_policy](variables.tf#L281) | Bucket retention policy. | object({…}) | | null | -| [rpo](variables.tf#L290) | Bucket recovery point objective. | string | | null | -| [soft_delete_retention](variables.tf#L300) | The duration in seconds that soft-deleted objects in the bucket will be retained and cannot be permanently deleted. Set to 0 to override the default and disable. | number | | null | -| [storage_class](variables.tf#L306) | Bucket storage class. | string | | "STANDARD" | -| [tag_bindings](variables.tf#L316) | Tag bindings for this folder, in key => tag value id format. | map(string) | | {} | -| [uniform_bucket_level_access](variables.tf#L323) | Allow using object ACLs (false) or not (true, this is the recommended behavior) , defaults to true (which is the recommended practice, but not the behavior of storage API). | bool | | true | -| [versioning](variables.tf#L329) | Enable versioning, defaults to false. | bool | | null | -| [website](variables.tf#L335) | Bucket website. | object({…}) | | null | +| [ip_filter](variables.tf#L90) | The bucket's IP filter configuration. | object({…}) | | null | +| [labels](variables.tf#L101) | Labels to be attached to all buckets. | map(string) | | {} | +| [lifecycle_rules](variables.tf#L107) | Bucket lifecycle rule. | map(object({…})) | | {} | +| [location](variables.tf#L156) | Bucket location. | string | | null | +| [logging_config](variables.tf#L166) | Bucket logging configuration. | object({…}) | | null | +| [managed_folders](variables.tf#L175) | Managed folders to create within the bucket in {PATH => CONFIG} format. | map(object({…})) | | {} | +| [notification_config](variables.tf#L208) | GCS Notification configuration. | object({…}) | | null | +| [objects_to_upload](variables.tf#L225) | Objects to be uploaded to bucket. | map(object({…})) | | {} | +| [prefix](variables.tf#L251) | Optional prefix used to generate the bucket name. | string | | null | +| [public_access_prevention](variables.tf#L266) | Prevents public access to the bucket. | string | | null | +| [requester_pays](variables.tf#L276) | Enables Requester Pays on a storage bucket. | bool | | null | +| [retention_policy](variables.tf#L282) | Bucket retention policy. | object({…}) | | null | +| [rpo](variables.tf#L291) | Bucket recovery point objective. | string | | null | +| [soft_delete_retention](variables.tf#L301) | The duration in seconds that soft-deleted objects in the bucket will be retained and cannot be permanently deleted. Set to 0 to override the default and disable. | number | | null | +| [storage_class](variables.tf#L307) | Bucket storage class. | string | | "STANDARD" | +| [tag_bindings](variables.tf#L317) | Tag bindings for this folder, in key => tag value id format. | map(string) | | {} | +| [uniform_bucket_level_access](variables.tf#L324) | Allow using object ACLs (false) or not (true, this is the recommended behavior) , defaults to true (which is the recommended practice, but not the behavior of storage API). | bool | | true | +| [versioning](variables.tf#L330) | Enable versioning, defaults to false. | bool | | null | +| [website](variables.tf#L336) | Bucket website. | object({…}) | | null | ## Outputs diff --git a/modules/gcs/iam.tf b/modules/gcs/iam.tf index f90ba1626..9fd507573 100644 --- a/modules/gcs/iam.tf +++ b/modules/gcs/iam.tf @@ -52,7 +52,9 @@ resource "google_storage_bucket_iam_binding" "bindings" { dynamic "condition" { for_each = each.value.condition == null ? [] : [""] content { - expression = each.value.condition.expression + expression = templatestring( + each.value.condition.expression, var.context.condition_vars + ) title = each.value.condition.title description = each.value.condition.description } @@ -69,7 +71,9 @@ resource "google_storage_bucket_iam_member" "bindings" { dynamic "condition" { for_each = each.value.condition == null ? [] : [""] content { - expression = each.value.condition.expression + expression = templatestring( + each.value.condition.expression, var.context.condition_vars + ) title = each.value.condition.title description = each.value.condition.description } diff --git a/modules/gcs/main.tf b/modules/gcs/main.tf index 46d292210..440db500e 100644 --- a/modules/gcs/main.tf +++ b/modules/gcs/main.tf @@ -19,7 +19,7 @@ locals { ctx = { for k, v in var.context : k => { for kk, vv in v : "${local.ctx_p}${k}:${kk}" => vv - } + } if k != "condition_vars" } ctx_p = "$" prefix = var.prefix == null ? "" : "${var.prefix}-" diff --git a/modules/gcs/variables.tf b/modules/gcs/variables.tf index 243b9eab6..08ec0366b 100644 --- a/modules/gcs/variables.tf +++ b/modules/gcs/variables.tf @@ -29,6 +29,7 @@ variable "bucket_create" { variable "context" { description = "Context-specific interpolations." type = object({ + condition_vars = optional(map(map(string)), {}) custom_roles = optional(map(string), {}) iam_principals = optional(map(string), {}) locations = optional(map(string), {}) diff --git a/modules/iam-service-account/README.md b/modules/iam-service-account/README.md index ef882acca..e07d66dd7 100644 --- a/modules/iam-service-account/README.md +++ b/modules/iam-service-account/README.md @@ -119,12 +119,12 @@ module "service-account-with-tags" { | name | description | type | required | default | |---|---|:---:|:---:|:---:| -| [name](variables.tf#L54) | Name of the service account to create. | string | ✓ | | -| [project_id](variables.tf#L69) | Project id where service account will be created. | string | ✓ | | -| [context](variables.tf#L17) | External context used in replacements. | object({…}) | | {} | -| [create_ignore_already_exists](variables.tf#L32) | If set to true, skip service account creation if a service account with the same email already exists. | bool | | null | -| [description](variables.tf#L42) | Optional description. | string | | null | -| [display_name](variables.tf#L48) | Display name of the service account to create. | string | | "Terraform-managed." | +| [name](variables.tf#L55) | Name of the service account to create. | string | ✓ | | +| [project_id](variables.tf#L70) | Project id where service account will be created. | string | ✓ | | +| [context](variables.tf#L17) | External context used in replacements. | object({…}) | | {} | +| [create_ignore_already_exists](variables.tf#L33) | If set to true, skip service account creation if a service account with the same email already exists. | bool | | null | +| [description](variables.tf#L43) | Optional description. | string | | null | +| [display_name](variables.tf#L49) | Display name of the service account to create. | string | | "Terraform-managed." | | [iam](variables-iam.tf#L17) | IAM bindings in {ROLE => [MEMBERS]} format. | map(list(string)) | | {} | | [iam_billing_roles](variables-iam.tf#L24) | Billing account roles granted to this service account, by billing account id. Non-authoritative. | map(list(string)) | | {} | | [iam_bindings](variables-iam.tf#L31) | Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. | map(object({…})) | | {} | @@ -136,10 +136,10 @@ module "service-account-with-tags" { | [iam_project_roles](variables-iam.tf#L89) | Project roles granted to this service account, by project id. | map(list(string)) | | {} | | [iam_sa_roles](variables-iam.tf#L96) | Service account roles granted to this service account, by service account name. | map(list(string)) | | {} | | [iam_storage_roles](variables-iam.tf#L103) | Storage roles granted to this service account, by bucket name. | map(list(string)) | | {} | -| [prefix](variables.tf#L59) | Prefix applied to service account names. | string | | null | -| [project_number](variables.tf#L74) | Project number of var.project_id. Set this to avoid permadiffs when creating tag bindings. | string | | null | -| [service_account_create](variables.tf#L80) | Create service account. When set to false, uses a data source to reference an existing service account. | bool | | true | -| [tag_bindings](variables.tf#L87) | Tag bindings for this service accounts, in key => tag value id format. | map(string) | | {} | +| [prefix](variables.tf#L60) | Prefix applied to service account names. | string | | null | +| [project_number](variables.tf#L75) | Project number of var.project_id. Set this to avoid permadiffs when creating tag bindings. | string | | null | +| [service_account_create](variables.tf#L81) | Create service account. When set to false, uses a data source to reference an existing service account. | bool | | true | +| [tag_bindings](variables.tf#L88) | Tag bindings for this service accounts, in key => tag value id format. | map(string) | | {} | ## Outputs diff --git a/modules/iam-service-account/iam.tf b/modules/iam-service-account/iam.tf index a5ed01f0b..f993764d6 100644 --- a/modules/iam-service-account/iam.tf +++ b/modules/iam-service-account/iam.tf @@ -110,7 +110,9 @@ resource "google_service_account_iam_binding" "bindings" { dynamic "condition" { for_each = each.value.condition == null ? [] : [""] content { - expression = each.value.condition.expression + expression = templatestring( + each.value.condition.expression, var.context.condition_vars + ) title = each.value.condition.title description = each.value.condition.description } @@ -129,7 +131,9 @@ resource "google_service_account_iam_member" "bindings" { dynamic "condition" { for_each = each.value.condition == null ? [] : [""] content { - expression = each.value.condition.expression + expression = templatestring( + each.value.condition.expression, var.context.condition_vars + ) title = each.value.condition.title description = each.value.condition.description } diff --git a/modules/iam-service-account/main.tf b/modules/iam-service-account/main.tf index 3fe81052f..cf7eb1b96 100644 --- a/modules/iam-service-account/main.tf +++ b/modules/iam-service-account/main.tf @@ -18,7 +18,7 @@ locals { ctx = { for k, v in var.context : k => { for kk, vv in v : "${local.ctx_p}${k}:${kk}" => vv - } + } if k != "condition_vars" } ctx_p = "$" iam_email = ( diff --git a/modules/iam-service-account/variables.tf b/modules/iam-service-account/variables.tf index 33f8af819..45f64f3ce 100644 --- a/modules/iam-service-account/variables.tf +++ b/modules/iam-service-account/variables.tf @@ -17,6 +17,7 @@ variable "context" { description = "External context used in replacements." type = object({ + condition_vars = optional(map(map(string)), {}) custom_roles = optional(map(string), {}) folder_ids = optional(map(string), {}) iam_principals = optional(map(string), {}) diff --git a/modules/organization/README.md b/modules/organization/README.md index 7e4273396..35b96726f 100644 --- a/modules/organization/README.md +++ b/modules/organization/README.md @@ -607,7 +607,7 @@ values: |---|---|:---:|:---:|:---:| | [organization_id](variables.tf#L113) | Organization id in organizations/nnnnnn format. | string | ✓ | | | [contacts](variables.tf#L17) | List of essential contacts for this resource. Must be in the form EMAIL -> [NOTIFICATION_TYPES]. Valid notification types are ALL, SUSPENSION, SECURITY, TECHNICAL, BILLING, LEGAL, PRODUCT_UPDATES. | map(list(string)) | | {} | -| [context](variables.tf#L24) | Context-specific interpolations. | object({…}) | | {} | +| [context](variables.tf#L24) | Context-specific interpolations. | object({…}) | | {} | | [custom_roles](variables.tf#L43) | Map of role name => list of permissions to create in this project. | map(list(string)) | | {} | | [factories_config](variables.tf#L50) | Paths to data files and folders that enable factory functionality. | object({…}) | | {} | | [firewall_policy](variables.tf#L62) | Hierarchical firewall policies to associate to the organization. | object({…}) | | null | diff --git a/modules/organization/iam.tf b/modules/organization/iam.tf index e39c616fa..0a5839de4 100644 --- a/modules/organization/iam.tf +++ b/modules/organization/iam.tf @@ -75,7 +75,6 @@ locals { } ]... ) - iam_condition_context = { organization = local.organization_id_numeric } } # we use a different key for custom roles to allow referring to the role alias @@ -121,7 +120,7 @@ resource "google_organization_iam_binding" "bindings" { dynamic "condition" { for_each = each.value.condition == null ? [] : [""] content { - expression = templatestring(each.value.condition.expression, local.iam_condition_context) + expression = templatestring(each.value.condition.expression, var.context.condition_vars) title = each.value.condition.title description = each.value.condition.description } @@ -137,7 +136,7 @@ resource "google_organization_iam_member" "bindings" { dynamic "condition" { for_each = each.value.condition == null ? [] : [""] content { - expression = templatestring(each.value.condition.expression, local.iam_condition_context) + expression = templatestring(each.value.condition.expression, var.context.condition_vars) title = each.value.condition.title description = each.value.condition.description } diff --git a/modules/organization/main.tf b/modules/organization/main.tf index e02f85935..5b8f341d3 100644 --- a/modules/organization/main.tf +++ b/modules/organization/main.tf @@ -18,7 +18,7 @@ locals { ctx = { for k, v in var.context : k => { for kk, vv in v : "${local.ctx_p}${k}:${kk}" => vv - } + } if k != "condition_vars" } ctx_p = "$" organization_id_numeric = split("/", var.organization_id)[1] diff --git a/modules/organization/organization-policies.tf b/modules/organization/organization-policies.tf index 3fae3f551..5adcf9403 100644 --- a/modules/organization/organization-policies.tf +++ b/modules/organization/organization-policies.tf @@ -34,7 +34,7 @@ locals { all = try(r.allow.all, null) values = ( can(r.allow.values) - ? [for x in r.allow.values : templatestring(x, var.context.org_policies)] + ? [for x in r.allow.values : templatestring(x, var.context.condition_vars)] : null ) } : null @@ -42,7 +42,7 @@ locals { all = try(r.deny.all, null) values = ( can(r.deny.values) - ? [for x in r.deny.values : templatestring(x, var.context.org_policies)] + ? [for x in r.deny.values : templatestring(x, var.context.condition_vars)] : null ) } : null @@ -50,28 +50,28 @@ locals { condition = { description = ( can(r.condition.description) - ? templatestring(r.condition.description, var.context.org_policies) + ? templatestring(r.condition.description, var.context.condition_vars) : null ) expression = ( can(r.condition.expression) - ? templatestring(r.condition.expression, var.context.org_policies) + ? templatestring(r.condition.expression, var.context.condition_vars) : null ) location = ( can(r.condition.location) - ? templatestring(r.condition.location, var.context.org_policies) + ? templatestring(r.condition.location, var.context.condition_vars) : null ) title = ( can(r.condition.title) - ? templatestring(r.condition.title, var.context.org_policies) + ? templatestring(r.condition.title, var.context.condition_vars) : null ) } parameters = ( can(r.parameters) - ? templatestring(r.parameters, var.context.org_policies) + ? templatestring(r.parameters, var.context.condition_vars) : null ) } diff --git a/modules/organization/variables.tf b/modules/organization/variables.tf index 9c0d313a9..110417d6c 100644 --- a/modules/organization/variables.tf +++ b/modules/organization/variables.tf @@ -25,11 +25,11 @@ variable "context" { description = "Context-specific interpolations." type = object({ bigquery_datasets = optional(map(string), {}) + condition_vars = optional(map(map(string)), {}) custom_roles = optional(map(string), {}) iam_principals = optional(map(string), {}) locations = optional(map(string), {}) log_buckets = optional(map(string), {}) - org_policies = optional(map(map(string)), {}) project_ids = optional(map(string), {}) pubsub_topics = optional(map(string), {}) storage_buckets = optional(map(string), {}) diff --git a/modules/project-factory-experimental/README.md b/modules/project-factory-experimental/README.md deleted file mode 100644 index 702142c09..000000000 --- a/modules/project-factory-experimental/README.md +++ /dev/null @@ -1,661 +0,0 @@ -# Project and Folder Factory - -This module implements end-to-end creation processes for a folder hierarchy, projects and billing budgets via YAML data configurations. - -It supports - -- filesystem-driven folder hierarchy exposing the full configuration options available in the [folder module](../folder/) -- multiple project creation and management exposing the full configuration options available in the [project module](../project/), including KMS key grants and VPC-SC perimeter membership -- optional per-project [service accounts and buckets management](#service-accounts-and-buckets) including basic IAM grants -- optional [billing budgets](#billing-budgets) factory and budget/project associations -- cross-referencing of hierarchy folders in projects -- optional per-project IaC configuration -- global defaults or overrides for most project configurations -- extensive support of [context-based interpolation](#context-based-interpolation) - -The factory is implemented as a thin data translation layer over the underlying modules, so that no "magic" or hidden side effects are implemented in code, and debugging or integration of new features are simple. - -The code is meant to be executed by a high level service account with powerful permissions: - -- folder admin permissions for the hierarchy -- project creation on the nodes (folder or org) where projects will be defined -- Shared VPC connection if service project attachment is desired -- VPC Service Controls perimeter management if project inclusion is desired -- billing cost manager permissions to manage budgets and monitoring permissions if notifications should also be managed here - -## Contents - - -- [Folder hierarchy](#folder-hierarchy) -- [Projects](#projects) - - [Factory-wide project defaults, merges, optionals](#factory-wide-project-defaults-merges-optionals) - - [Service accounts and buckets](#service-accounts-and-buckets) - - [Automation project and resources](#automation-project-and-resources) -- [Billing budgets](#billing-budgets) -- [Context-based interpolation](#context-based-interpolation) - - [Folder context ids](#folder-context-ids) - - [Project context ids](#project-context-ids) - - [Service account context ids](#service-account-context-ids) - - [Other context ids](#other-context-ids) -- [Example](#example) -- [Files](#files) -- [Variables](#variables) -- [Outputs](#outputs) -- [Tests](#tests) - - -## Folder hierarchy - -The hierarchy supports up to three levels of folders, which are defined via filesystem directories each including a `.config.yaml` files detailing their attributes. - -The filesystem tree containing folder definitions is configured via the `factories_config.folders` variable, which sets the the path containing the YAML definitions for folders. It's also possible to configure the hierarchy via the `folders` variable, which is internally merged in with the factory definitions. - -Parent ids for top-level folders can either be set explicitly (e.g. `folders/12345678`), or via [context interpolation](#context-based-interpolation) by referring to keys in the `context.folder_ids` variable. The special `default` key in the substitutions folder variable is used if present and no folder id/key has been specified in the YAML. - -Filesystem directories can also contain project definitions in the same YAML format described below. This approach must be used with caution and is best adopted for stable scenarios, as problems in the filesystem hierarchy definitions might result in the project files not being read and the resources being deleted by Terraform. - -Refer to the [example](#example) below for actual examples of the YAML definitions. - -## Projects - -The project factory is configured in three ia the `factories_config.projects` variable, and project files are also additionally read from the folder tree described in the previous section. It's best to limit project definition via the hierarchy tree to a minimum to avoid cross dependencies between folders and projects, which could complicate their lifecycle. - -Projects can also be configured via the `projects` variable, which is internally merged in with the factory definitions. - -The YAML format mirrors the project module, refer to the [example](#example) below for actual examples of the YAML definitions. - -### Factory-wide project defaults, merges, optionals - -In addition to the YAML-based project configurations, the factory accepts three additional sets of inputs via Terraform variables: - -- the `data_defaults` variable allows defining defaults for specific project attributes, which are only used if the attributes are not passed in via YAML -- the `data_overrides` variable works similarly to defaults, but the values specified here take precedence over those in YAML files -- the `data_merges` variable allows specifying additional values for map or set based variables, which are merged with the data coming from YAML - -Some examples on where to use each of the three sets are [provided below](#example). - -### Service accounts and buckets - -Service accounts and GCS buckets can be managed as part of each project's YAML configuration. This allows creation of default service accounts used for GCE instances, in firewall rules, or for application-level credentials without resorting to a separate Terraform configuration. - -Each service account is represented by one key and a set of optional key/value pairs in the `service_accounts` top-level YAML map, which exposes most of the variables available in the `iam-service-account` module. Most of the service accounts attributes are optional. - -```yaml -service_accounts: - be-0: {} - fe-1: - display_name: GCE frontend service account. - iam_self_roles: - - roles/storage.objectViewer - iam_project_roles: - $project_ids:my-host-project: - - roles/compute.networkUser - iam_sa_roles: - $iam_principals:service_accounts/my-project/be-0: - - roles/iam.serviceAccountUser - terraform-rw: {} -``` - -Each bucket is represented by one key and a set of optional key/value pairs in the `buckets` top-level YAML map, which exposes most of the variables available in the `gcs` module. Bucket location, storage class and a few other attributes can be defaulted/enforced via project factory level variables. - -```yaml -buckets: - state: - location: europe-west8 - iam: - roles/storage.admin: - - $iam_principals:service_accounts/my-project/terraform-rw -``` - -### Automation project and resources - -Other than creating automation resources within the project via the `service_accounts` and `buckets` attributes, this module also support management of automation resources created in a separate controlling project. This allows grating broad roles on the project, while still making sure that the automation resources used for Terraform cannot be manipulated from the same identities. - -Automation resources are defined via the `automation` attribute in project configurations, which supports: - -- a mandatory `project` attribute to define the external controlling project; this attribute does not support interpolation and needs to be explicit -- an optional `service_accounts` list where each element defines a service account in the controlling project -- an optional `bucket` which defines a bucket in the controlling project, and the map of roles/principals in the corresponding value assigned on the created bucket; principals can refer to the created service accounts by key - -Service accounts and buckets are prefixed with the project name. Service accounts use the key specified in the YAML file as a suffix, while buckets use a default `tf-state` suffix. - -```yaml -# file name: prod-app-example-0 -# prefix via factory defaults: foo -# project id: foo-prod-app-example-0 -billing_account: 012345-67890A-BCDEF0 -parent: folders/12345678 -services: - - compute.googleapis.com - - stackdriver.googleapis.com -iam: - roles/owner: - - $iam_principals:service_accounts/iac-core-0/rw - roles/viewer: - - $iam_principals:service_accounts/iac-core-0/ro -automation: - project: $project_ids:iac-core-0 - service_accounts: - # sa name: foo-prod-app-example-0-rw - rw: - description: Read/write automation sa for app example 0. - # sa name: foo-prod-app-example-0-ro - ro: - description: Read-only automation sa for app example 0. - bucket: - # bucket name: foo-prod-app-example-0-tf-state - description: Terraform state bucket for app example 0. - iam: - roles/storage.objectCreator: - - $iam_principals:service_accounts/iac-core-0/rw - roles/storage.objectViewer: - - $iam_principals:service_accounts/iac-core-0/rw - - $iam_principals:service_accounts/iac-core-0/ro - - group:devops@example.org -``` - -## Billing budgets - -The billing budgets factory integrates the `[`billing-account`](../billing-account/) module functionality, and adds support for easy referencing budgets in project files. - -To enable support for billing budgets, set the billing account id, optional notification channels, and the data folder for budgets in the `factories_config.budgets` variable, then create billing budgets using YAML definitions following the format described in the `billing-account` module. - -Once budgets are defined, they can be referenced in a project file using their file name: - -```yaml -billing_account: 012345-67890A-BCDEF0 -labels: - app: app-1 - team: foo -parent: folders/12345678 -services: - - container.googleapis.com - - storage.googleapis.com -billing_budgets: - - test-100 -``` - -A simple billing budget example is show in the [example](#example) below. - -## Context-based interpolation - -Interpolation allow referring to resources which are either created at runtime, or externally managed via short aliases. - -This feature has two main benefits: - -- being able to refer to resource ids which cannot be known before creation, for example project automation service accounts in IAM bindings -- making YAML configuration files more easily readable and portable, by using mnemonic keys which are not specific to an organization or project - -One example of both types of contexts is in this project snippet. The automation service account is used in IAM bindings via its key, while the parent folder is set by referring to its path in the hierarchy factory. - -```yaml -# file name: my-project -parent: $folder_ids:teams/team-a -iam: - "roles/owner": - - $iam_principals:service_accounts/my-project/rw -automation: - project: $project_ids:ta-app0-0 - service_accounts: - rw: - description: Read/write automation sa for team a app 0. - buckets: - state: - description: Terraform state bucket for team a app 0. - iam: - roles/storage.objectCreator: - - $iam_principals:service_accounts/my-project/rw -``` - -Interpolations leverage contexts from two separate sources: resources managed by the project factory (folders, service accounts, etc.), and user-defined resource ids passed in via the `context` variable. - -Context replacements use the `$` prefix and are accessible via namespaces that match the attributes in the context variable: - -- `$custom_roles:foo` -- `$folder_ids:foo` -- `$iam_principals:foo` -- `$kms_keys:foo` -- `$locations:foo` -- `$notification_channels:foo` -- `$project_ids:foo` -- `$tag_values:foo` -- `$vpc_host_projects:foo` -- `$vpc_sc_perimeters:foo` - -Internally created resources are mapped to context namespaces, and use specific prefixes to express the relationship with their container folder/project where necessary, as shown in the following examples. - -### Folder context ids - -Folders ids use the `$folder_ids` namespace, with ids derived from the full filesystem path to express the hierarchy. - -As an example, the id of the folder defined in `folders/networking/prod/.config.yaml` file will be accessible via `$folder_ids:networking/prod`. - -### Project context ids - -Project ids ise the `$project_ids:` namespace, with ids defined in two different ways: - -- projects defined in the `var.factories_config.project` tree use the filename (dirname is stripped) -- projects defined in the `var.factories_config.folders` tree use the full path (dirname is kept) - -As an example, the id of the project defined in the `projects/team-0/app-0-0.yaml` file will be accessible via `$project_ids:app-0-0`. The id of the project defined in the `folders/shared/iac-core-0.yaml` file will be accessible via `$project_ids:shared/iac-core-0`. - -### Service account context ids - -Service accounts use the `$iam_principals:` namespace, with ids that allow referring to their parent project. - -As an example, the `rw` service account defined in the `projects/team-0/app-0-0.yaml` file will be accessible via `$iam_principals:service_accounts/app-0-0/rw`. - -### Other context ids - -Other context ids simply match whatever was passed in via the `var.contexts` variable. The following is a short example. - -```hcl -context = { - custom_roles = { - myrole = "organizations/1234567890/roles/myRoleOne" - } - folder_ids = { - "test/prod" = "folders/1234567890" - } - iam_principals = { - mysa = "serviceAccount:test@test-project.iam.gserviceaccount.com" - } - project_ids = { - vpc-host = "test-vpc-host" - } - tag_values = { - "test/one" = "tagValues/1234567890" - } - vpc_sc_perimeters = { - default = "accessPolicies/888933661165/servicePerimeters/default" - } -} -# tftest: skip -``` - -```yaml -parent: $folder_ids/test/prod -iam: - $custom_roles:myrole: - - $iam_principals:mygroup -shared_vpc_service_config: - host_project: $project_ids:vpc-host -tag_bindings: - foo: $tag_values:test/one -vpc_sc: - perimeter_name: $vpc_sc_perimeters:default -``` - -## Example - -This show a module invocation using all optional features: - -```hcl -module "project-factory" { - source = "./fabric/modules/project-factory-experimental" - context = { - folder_ids = { - default = "folders/5678901234" - teams = "folders/5678901234" - } - kms_keys = { - compute-prod-ew1 = "projects/kms-central-prj/locations/europe-west1/keyRings/my-keyring/cryptoKeys/ew1-compute" - } - iam_principals = { - gcp-devops = "group:gcp-devops@example.org" - } - tag_values = { - "org-policies/drs-allow-all" = "tagValues/123456" - } - vpc_host_projects = { - dev-spoke-0 = "test-pf-dev-net-spoke-0" - } - } - # use a default billing account if none is specified via yaml - data_defaults = { - billing_account = var.billing_account_id - storage_location = "EU" - } - # make sure the environment label and stackdriver service are always added - data_merges = { - labels = { - environment = "test" - } - services = [ - "stackdriver.googleapis.com" - ] - } - # always use this contacts and prefix, regardless of what is in the yaml file - data_overrides = { - contacts = { - "admin@example.org" = ["ALL"] - } - prefix = "test-pf" - } - # location where the yaml files are read from - factories_config = { - budgets = { - billing_account_id = var.billing_account_id - data = "data/budgets" - } - folders = "data/hierarchy" - projects = "data/projects" - } - notification_channels = { - billing-default = { - project_id = "foo-billing-audit" - type = "email" - labels = { - email_address = "gcp-billing-admins@example.org" - } - } - } -} -# tftest files=0,1,2,3,4,5,6,7,8,9 inventory=example.yaml -``` - -A simple hierarchy of folders: - -```yaml -name: Team A -# implicit parent definition via 'default' key -iam: - roles/viewer: - - group:team-a-admins@example.org - - $iam_principals:gcp-devops -# tftest-file id=0 path=data/hierarchy/team-a/.config.yaml schema=folder.schema.json -``` - -```yaml -name: Team B -# explicit parent definition via key -parent: $folder_ids:teams -# tftest-file id=1 path=data/hierarchy/team-b/.config.yaml schema=folder.schema.json -``` - -```yaml -name: Team C -# explicit parent definition via folder id -parent: folders/5678901234 -# tftest-file id=2 path=data/hierarchy/team-c/.config.yaml schema=folder.schema.json -``` - -```yaml -name: App 0 -# tftest-file id=3 path=data/hierarchy/team-a/app-0/.config.yaml schema=folder.schema.json -``` - -```yaml -name: App 0 -tag_bindings: - drs-allow-all: $tag_values:org-policies/drs-allow-all -# tftest-file id=4 path=data/hierarchy/team-b/app-0/.config.yaml schema=folder.schema.json -``` - -One project defined within the folder hierarchy: - -```yaml -billing_account: 012345-67890A-BCDEF0 -services: - - container.googleapis.com - - storage.googleapis.com -# tftest-file id=5 path=data/hierarchy/teams-iac-0.yaml schema=project.schema.json -``` - -More traditional project definitions via the project factory data: - -```yaml -billing_account: 012345-67890A-BCDEF0 -labels: - app: app-0 - team: team-a -parent: $folder_ids:team-a/app-0 -service_encryption_key_ids: - storage.googleapis.com: - - projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce - compute.googleapis.com: - - $kms_keys:compute-prod-ew1 -services: - - compute.googleapis.com - - container.googleapis.com - - storage.googleapis.com -iam_by_principals: - $iam_principals:service_accounts/dev-ta-app0-be/app-0-be: - - roles/storage.objectViewer -iam: - roles/cloudkms.cryptoKeyEncrypterDecrypter: - - $service_agents:storage -service_accounts: - app-0-be: - display_name: "Backend instances." - iam_project_roles: - $project_ids:dev-spoke-0: - - roles/compute.networkUser - iam_self_roles: - - roles/logging.logWriter - - roles/monitoring.metricWriter - app-0-fe: - display_name: "Frontend instances." - iam_project_roles: - $project_ids:dev-spoke-0: - - roles/compute.networkUser - iam_self_roles: - - roles/logging.logWriter - - roles/monitoring.metricWriter -shared_vpc_service_config: - host_project: $project_ids:dev-spoke-0 - network_users: - - $iam_principals:gcp-devops - service_agent_iam: - "roles/container.hostServiceAgentUser": - - $service_agents:container-engine - "roles/compute.networkUser": - - $service_agents:container-engine -billing_budgets: - - $billing_budgets:test-100 -tags: - my-tag-key-1: - values: - my-value-1: - description: My value 1 - my-value-2: - description: My value 3 - iam: - roles/resourcemanager.tagUser: - - user:user@example.com -# tftest-file id=6 path=data/projects/dev-ta-app0-be.yaml schema=project.schema.json -``` - -This project defines a controlling project via the `automation` attributes: - -```yaml -parent: $folder_ids:team-b/app-0 -services: -- run.googleapis.com -- storage.googleapis.com -iam: - "roles/owner": - - $iam_principals:service_accounts/dev-tb-app0-0/rw - "roles/viewer": - - $iam_principals:service_accounts/dev-tb-app0-0/ro -shared_vpc_host_config: - enabled: true -service_accounts: - vm-default: - display_name: "VM default service account." - iam_self_roles: - - roles/logging.logWriter - - roles/monitoring.metricWriter - iam: - roles/iam.serviceAccountTokenCreator: - - $iam_principals:service_accounts/dev-tb-app0-0/rw -automation: - project: test-pf-teams-iac-0 - # prefix used for automation resources can be explicitly set if needed - # prefix: test-pf-dev-tb-0-0 - service_accounts: - rw: - description: Team B app 0 read/write automation sa. - ro: - description: Team B app 0 read-only automation sa. - bucket: - description: Team B app 0 Terraform state bucket. - iam: - roles/storage.objectCreator: - - $iam_principals:service_accounts/dev-tb-app0-0/rw - roles/storage.objectViewer: - - $iam_principals:gcp-devops - - group:team-b-admins@example.org - - $iam_principals:service_accounts/dev-tb-app0-0/rw - - $iam_principals:service_accounts/dev-tb-app0-0/ro - -# tftest-file id=7 path=data/projects/dev-tb-app0-0.yaml schema=project.schema.json -``` - -A billing budget: - -```yaml -# billing budget test-100 -display_name: 100 dollars in current spend -amount: - units: 100 -filter: - period: - calendar: MONTH - resource_ancestors: - - folders/1234567890 -threshold_rules: -- percent: 0.5 -- percent: 0.75 -update_rules: - default: - disable_default_iam_recipients: true - monitoring_notification_channels: - - $notification_channels:billing-default -# tftest-file id=8 path=data/budgets/test-100.yaml schema=budget.schema.json -``` - -Granting permissions to service accounts defined in other project through interpolation: - -```yaml -billing_account: 012345-67890A-BCDEF0 -labels: - app: app-0 - team: team-b -parent: $folder_ids:team-b/app-0 -services: - - container.googleapis.com - - storage.googleapis.com -iam: - "roles/run.admin": - - $iam_principals:service_accounts/dev-ta-app0-be/app-0-be - "roles/run.developer": - - $iam_principals:service_accounts/dev-tb-app0-1/app-0-be -service_accounts: - app-0-be: - display_name: "Backend instances." - iam_self_roles: - - roles/logging.logWriter - - roles/monitoring.metricWriter -# tftest-file id=9 path=data/projects/dev-tb-app0-1.yaml schema=project.schema.json -``` - - - -## Files - -| name | description | modules | resources | -|---|---|---|---| -| [automation.tf](./automation.tf) | None | gcs · iam-service-account | | -| [budgets.tf](./budgets.tf) | Billing budget factory locals. | billing-account | | -| [folders.tf](./folders.tf) | Folder hierarchy factory resources. | folder | | -| [main.tf](./main.tf) | Projects and billing budgets factory resources. | | terraform_data | -| [outputs.tf](./outputs.tf) | Module outputs. | | | -| [projects-buckets.tf](./projects-buckets.tf) | None | gcs | | -| [projects-defaults.tf](./projects-defaults.tf) | None | | | -| [projects-log-buckets.tf](./projects-log-buckets.tf) | None | logging-bucket | | -| [projects-service-accounts.tf](./projects-service-accounts.tf) | None | iam-service-account | | -| [projects.tf](./projects.tf) | None | project | | -| [variables-billing.tf](./variables-billing.tf) | None | | | -| [variables-folders.tf](./variables-folders.tf) | None | | | -| [variables-projects.tf](./variables-projects.tf) | None | | | -| [variables.tf](./variables.tf) | Module variables. | | | - -## Variables - -| name | description | type | required | default | -|---|---|:---:|:---:|:---:| -| [factories_config](variables.tf#L172) | Path to folder with YAML resource description data files. | object({…}) | ✓ | | -| [context](variables.tf#L17) | Context-specific interpolations. | object({…}) | | {} | -| [data_defaults](variables.tf#L35) | Optional default values used when corresponding project or folder data from files are missing. | object({…}) | | {} | -| [data_merges](variables.tf#L107) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | object({…}) | | {} | -| [data_overrides](variables.tf#L126) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | object({…}) | | {} | -| [folders](variables-folders.tf#L17) | Folders data merged with factory data. | map(object({…})) | | {} | -| [notification_channels](variables-billing.tf#L17) | Notification channels used by budget alerts. | map(object({…})) | | {} | -| [projects](variables-projects.tf#L17) | Projects data merged with factory data. | map(object({…})) | | {} | - -## Outputs - -| name | description | sensitive | -|---|---|:---:| -| [folder_ids](outputs.tf#L44) | Folder ids. | | -| [iam_principals](outputs.tf#L49) | IAM principals mappings. | | -| [log_buckets](outputs.tf#L54) | Log bucket ids. | | -| [project_ids](outputs.tf#L61) | Project ids. | | -| [project_numbers](outputs.tf#L66) | Project numbers. | | -| [projects](outputs.tf#L73) | Project attributes. | | -| [service_accounts](outputs.tf#L78) | Service account emails. | | -| [storage_buckets](outputs.tf#L85) | Bucket names. | | - -## Tests - -These tests validate fixes to the project factory. - -```hcl -module "project-factory" { - source = "./fabric/modules/project-factory" - data_defaults = { - billing_account = "012345-67890A-ABCDEF" - } - data_merges = { - labels = { - owner = "foo" - } - services = [ - "compute.googleapis.com" - ] - } - data_overrides = { - prefix = "foo" - } - factories_config = { - projects_data_path = "data/projects" - } -} -# tftest modules=4 resources=22 files=test-0,test-1,test-2 -``` - -```yaml -parent: folders/1234567890 -services: - - iam.googleapis.com - - contactcenteraiplatform.googleapis.com - - container.googleapis.com -# tftest-file id=test-0 path=data/projects/test-0.yaml -``` - -```yaml -parent: folders/1234567890 -services: - - iam.googleapis.com - - contactcenteraiplatform.googleapis.com -# tftest-file id=test-1 path=data/projects/test-1.yaml -``` - -```yaml -parent: folders/1234567890 -services: - - iam.googleapis.com - - storage.googleapis.com -# tftest-file id=test-2 path=data/projects/test-2.yaml -``` diff --git a/modules/project-factory-experimental/automation.tf b/modules/project-factory-experimental/automation.tf deleted file mode 100644 index ca0c45a92..000000000 --- a/modules/project-factory-experimental/automation.tf +++ /dev/null @@ -1,145 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - _automation = merge( - { - for k, v in local.folders_input : k => { - bucket = try(v.automation.bucket, {}) - # name = replace(k, "/", "-") - parent_type = "folder" - prefix = try(v.automation.prefix, null) - project = try(v.automation.project, null) - service_accounts = try(v.automation.service_accounts, {}) - } if try(v.automation.bucket, null) != null - }, - { - for k, v in local.projects_input : k => { - bucket = try(v.automation.bucket, {}) - # name = v.name - parent_type = "project" - prefix = coalesce( - try(v.automation.prefix, null), - v.prefix == null ? v.name : "${v.prefix}-${v.name}" - ) - project = try(v.automation.project, null) - service_accounts = try(v.automation.service_accounts, {}) - } if try(v.automation.bucket, null) != null - } - ) - _automation_buckets = { - for k, v in local._automation : k => merge(v.bucket, { - automation_project = v.project - name = lookup(v, "name", "tf-state") - # project automation always has a prefix - prefix = try(coalesce( - v.prefix, - local.data_defaults.overrides.prefix, - local.data_defaults.defaults.prefix - ), null) - }) - } - _automation_sas = flatten(concat([ - for k, v in local._automation : [ - for sk, sv in v.service_accounts : merge(sv, { - automation_project = v.project - name = sk - parent = k - prefix = v.prefix - }) - ] - ])) - automation_buckets = { - for k, v in local._automation_buckets : - "${k}/${v.name}" => v - } - automation_sas = { - for k in local._automation_sas : - "${k.parent}/${k.name}" => k - } - automation_sas_iam_emails = { - for k, v in local.automation_sas : - "service_accounts/${v.parent}/${v.name}" => module.automation-service-accounts[k].iam_email - } -} - -module "automation-bucket" { - source = "../gcs" - for_each = local.automation_buckets - project_id = each.value.automation_project - prefix = each.value.prefix - name = each.value.name - encryption_key = lookup(each.value, "encryption_key", null) - force_destroy = try(coalesce( - local.data_defaults.overrides.bucket.force_destroy, - each.value.force_destroy, - local.data_defaults.defaults.force_destroy, - ), null) - context = merge(local.ctx, { - project_ids = local.ctx_project_ids - iam_principals = local.ctx_iam_principals - }) - iam = lookup(each.value, "iam", {}) - iam_bindings = lookup(each.value, "iam_bindings", {}) - iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) - labels = lookup(each.value, "labels", {}) - managed_folders = lookup(each.value, "managed_folders", {}) - location = coalesce( - local.data_defaults.overrides.storage_location, - lookup(each.value, "location", null), - local.data_defaults.defaults.storage_location - ) - storage_class = lookup( - each.value, "storage_class", "STANDARD" - ) - uniform_bucket_level_access = lookup( - each.value, "uniform_bucket_level_access", true - ) - versioning = lookup( - each.value, "versioning", false - ) -} - -module "automation-service-accounts" { - source = "../iam-service-account" - for_each = local.automation_sas - project_id = each.value.automation_project - prefix = each.value.prefix - name = each.value.name - description = lookup(each.value, "description", null) - display_name = lookup( - each.value, - "display_name", - "Service account ${each.value.name} for ${each.value.parent}." - ) - context = merge(local.ctx, { - project_ids = local.ctx_project_ids - iam_principals = merge( - local.ctx.iam_principals, - local.projects_sas_iam_emails - ) - }) - iam = lookup(each.value, "iam", {}) - iam_bindings = lookup(each.value, "iam_bindings", {}) - iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) - iam_billing_roles = lookup(each.value, "iam_billing_roles", {}) - iam_folder_roles = lookup(each.value, "iam_folder_roles", {}) - iam_organization_roles = lookup(each.value, "iam_organization_roles", {}) - iam_project_roles = lookup(each.value, "iam_project_roles", {}) - iam_sa_roles = lookup(each.value, "iam_sa_roles", {}) - # we don't interpolate buckets here as we can't use a dynamic key - iam_storage_roles = lookup(each.value, "iam_storage_roles", {}) -} diff --git a/modules/project-factory-experimental/folders.tf b/modules/project-factory-experimental/folders.tf deleted file mode 100644 index 8d57ab85b..000000000 --- a/modules/project-factory-experimental/folders.tf +++ /dev/null @@ -1,160 +0,0 @@ -/** - * Copyright 2024 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Folder hierarchy factory resources. - -# TODO: folder automation - -locals { - _folders_path = try( - pathexpand(var.factories_config.folders), null - ) - _folders_files = try( - fileset(local._folders_path, "**/**/.config.yaml"), - [] - ) - _folders_raw = merge( - var.folders, - { - for f in local._folders_files : dirname(f) => yamldecode(file( - "${coalesce(local._folders_path, "-")}/${f}" - )) - } - ) - ctx_folder_ids = merge(local.ctx.folder_ids, local.folder_ids) - folder_ids = merge( - { for k, v in module.folder-1 : k => v.id }, - { for k, v in module.folder-2 : k => v.id }, - { for k, v in module.folder-3 : k => v.id } - ) - folders_input = { - for key, data in local._folders_raw : key => merge(data, { - key = key - level = length(split("/", key)) - parent_key = dirname(key) - # do not enforce overrides / defaults on folders - parent = lookup(data, "parent", null) - }) - } -} - -module "folder-1" { - source = "../folder" - for_each = { - for k, v in local.folders_input : k => v if v.level == 1 - } - parent = coalesce(each.value.parent, "$folder_ids:default") - name = each.value.name - org_policies = lookup(each.value, "org_policies", {}) - tag_bindings = lookup(each.value, "tag_bindings", {}) - logging_data_access = lookup(each.value, "logging_data_access", {}) - context = local.ctx -} - -module "folder-1-iam" { - source = "../folder" - for_each = { - for k, v in local.folders_input : k => v if v.level == 1 - } - id = module.folder-1[each.key].id - folder_create = false - iam = lookup(each.value, "iam", {}) - iam_bindings = lookup(each.value, "iam_bindings", {}) - iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) - iam_by_principals = lookup(each.value, "iam_by_principals", {}) - context = merge(local.ctx, { - iam_principals = local.ctx_iam_principals - }) -} - -module "folder-2" { - source = "../folder" - for_each = { - for k, v in local.folders_input : k => v if v.level == 2 - } - parent = coalesce( - each.value.parent, "$folder_ids:${each.value.parent_key}" - ) - name = each.value.name - org_policies = lookup(each.value, "org_policies", {}) - tag_bindings = lookup(each.value, "tag_bindings", {}) - logging_data_access = lookup(each.value, "logging_data_access", {}) - context = merge(local.ctx, { - folder_ids = merge(local.ctx.folder_ids, { - for k, v in module.folder-1 : k => v.id - }) - }) - depends_on = [module.folder-1] -} - -module "folder-2-iam" { - source = "../folder" - for_each = { - for k, v in local.folders_input : k => v if v.level == 2 - } - id = module.folder-2[each.key].id - folder_create = false - iam = lookup(each.value, "iam", {}) - iam_bindings = lookup(each.value, "iam_bindings", {}) - iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) - iam_by_principals = lookup(each.value, "iam_by_principals", {}) - context = merge(local.ctx, { - folder_ids = merge(local.ctx.folder_ids, { - for k, v in module.folder-1 : k => v.id - }) - iam_principals = local.ctx_iam_principals - }) -} - -module "folder-3" { - source = "../folder" - for_each = { - for k, v in local.folders_input : k => v if v.level == 3 - } - parent = coalesce( - each.value.parent, "$folder_ids:${each.value.parent_key}" - ) - name = each.value.name - org_policies = lookup(each.value, "org_policies", {}) - tag_bindings = lookup(each.value, "tag_bindings", {}) - logging_data_access = lookup(each.value, "logging_data_access", {}) - context = merge(local.ctx, { - folder_ids = merge(local.ctx.folder_ids, { - for k, v in module.folder-2 : k => v.id - }) - }) - depends_on = [module.folder-2] -} - -module "folder-3-iam" { - source = "../folder" - for_each = { - for k, v in local.folders_input : k => v if v.level == 3 - } - id = module.folder-3[each.key].id - folder_create = false - iam = lookup(each.value, "iam", {}) - iam_bindings = lookup(each.value, "iam_bindings", {}) - iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) - iam_by_principals = lookup(each.value, "iam_by_principals", {}) - context = merge(local.ctx, { - folder_ids = merge(local.ctx.folder_ids, { - for k, v in module.folder-2 : k => v.id - }) - iam_principals = local.ctx_iam_principals - }) -} - diff --git a/modules/project-factory-experimental/main.tf b/modules/project-factory-experimental/main.tf deleted file mode 100644 index ea74e1d63..000000000 --- a/modules/project-factory-experimental/main.tf +++ /dev/null @@ -1,41 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -# tfdoc:file:description Projects and billing budgets factory resources. - -locals { - ctx = var.context - ctx_iam_principals = merge( - local.ctx.iam_principals, - local.iam_principals - ) - iam_principals = merge( - local.projects_sas_iam_emails, - local.automation_sas_iam_emails - ) -} - -resource "terraform_data" "defaults_preconditions" { - lifecycle { - precondition { - condition = ( - var.data_defaults.storage_location != null || - var.data_overrides.storage_location != null - ) - error_message = "No default storage location defined in defaults or overides variables." - } - } -} diff --git a/modules/project-factory-experimental/outputs.tf b/modules/project-factory-experimental/outputs.tf deleted file mode 100644 index a30e92a29..000000000 --- a/modules/project-factory-experimental/outputs.tf +++ /dev/null @@ -1,90 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - outputs_projects = { - for k, v in local.projects_input : k => { - number = module.projects[k].number - project_id = module.projects[k].project_id - log_buckets = { - for sk, sv in lookup(v, "log_buckets", {}) : - "${k}/${sk}" => ( - module.log-buckets["${k}/${sk}"].id - ) - } - service_accounts = { - for sk, sv in lookup(v, "service_accounts", {}) : - "${k}/${sk}" => ( - module.service-accounts["${k}/${sk}"].email - ) - } - storage_buckets = { - for sk, sv in lookup(v, "buckets", {}) : - "${k}/${sk}" => ( - module.buckets["${k}/${sk}"].name - ) - } - } - } -} - -output "folder_ids" { - description = "Folder ids." - value = local.folder_ids -} - -output "iam_principals" { - description = "IAM principals mappings." - value = local.iam_principals -} - -output "log_buckets" { - description = "Log bucket ids." - value = merge([ - for k, v in local.outputs_projects : v.log_buckets - ]...) -} - -output "project_ids" { - description = "Project ids." - value = local.project_ids -} - -output "project_numbers" { - description = "Project numbers." - value = { - for k, v in local.outputs_projects : k => v.number - } -} - -output "projects" { - description = "Project attributes." - value = local.outputs_projects -} - -output "service_accounts" { - description = "Service account emails." - value = merge([ - for k, v in local.outputs_projects : v.service_accounts - ]...) -} - -output "storage_buckets" { - description = "Bucket names." - value = merge([ - for k, v in local.outputs_projects : v.storage_buckets - ]...) -} diff --git a/modules/project-factory-experimental/variables.tf b/modules/project-factory-experimental/variables.tf deleted file mode 100644 index 133371741..000000000 --- a/modules/project-factory-experimental/variables.tf +++ /dev/null @@ -1,183 +0,0 @@ -/** - * Copyright 2025 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -variable "context" { - description = "Context-specific interpolations." - type = object({ - custom_roles = optional(map(string), {}) - folder_ids = optional(map(string), {}) - iam_principals = optional(map(string), {}) - kms_keys = optional(map(string), {}) - locations = optional(map(string), {}) - notification_channels = optional(map(string), {}) - project_ids = optional(map(string), {}) - tag_values = optional(map(string), {}) - vpc_host_projects = optional(map(string), {}) - vpc_sc_perimeters = optional(map(string), {}) - }) - default = {} - nullable = false -} - -variable "data_defaults" { - description = "Optional default values used when corresponding project or folder data from files are missing." - type = object({ - billing_account = optional(string) - bucket = optional(object({ - force_destroy = optional(bool) - }), {}) - contacts = optional(map(list(string)), {}) - deletion_policy = optional(string) - factories_config = optional(object({ - custom_roles = optional(string) - observability = optional(string) - org_policies = optional(string) - quotas = optional(string) - }), {}) - labels = optional(map(string), {}) - metric_scopes = optional(list(string), []) - parent = optional(string) - prefix = optional(string) - project_reuse = optional(object({ - use_data_source = optional(bool, true) - attributes = optional(object({ - name = string - number = number - services_enabled = optional(list(string), []) - })) - })) - service_encryption_key_ids = optional(map(list(string)), {}) - services = optional(list(string), []) - shared_vpc_service_config = optional(object({ - host_project = string - iam_bindings_additive = optional(map(object({ - member = string - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - network_users = optional(list(string), []) - service_agent_iam = optional(map(list(string)), {}) - service_agent_subnet_iam = optional(map(list(string)), {}) - service_iam_grants = optional(list(string), []) - network_subnet_users = optional(map(list(string)), {}) - })) - storage_location = optional(string) - tag_bindings = optional(map(string), {}) - # non-project resources - service_accounts = optional(map(object({ - display_name = optional(string, "Terraform-managed.") - iam_self_roles = optional(list(string)) - })), {}) - universe = optional(object({ - prefix = string - unavailable_service_identities = optional(list(string), []) - unavailable_services = optional(list(string), []) - })) - vpc_sc = optional(object({ - perimeter_name = string - is_dry_run = optional(bool, false) - })) - logging_data_access = optional(map(object({ - ADMIN_READ = optional(object({ exempted_members = optional(list(string)) })), - DATA_READ = optional(object({ exempted_members = optional(list(string)) })), - DATA_WRITE = optional(object({ exempted_members = optional(list(string)) })) - })), {}) - }) - nullable = false - default = {} -} - -variable "data_merges" { - description = "Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`." - type = object({ - contacts = optional(map(list(string)), {}) - labels = optional(map(string), {}) - metric_scopes = optional(list(string), []) - service_encryption_key_ids = optional(map(list(string)), {}) - services = optional(list(string), []) - tag_bindings = optional(map(string), {}) - # non-project resources - service_accounts = optional(map(object({ - display_name = optional(string, "Terraform-managed.") - iam_self_roles = optional(list(string)) - })), {}) - }) - nullable = false - default = {} -} - -variable "data_overrides" { - description = "Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`." - type = object({ - # data overrides default to null to mark that they should not override - billing_account = optional(string) - bucket = optional(object({ - force_destroy = optional(bool) - }), {}) - contacts = optional(map(list(string))) - deletion_policy = optional(string) - factories_config = optional(object({ - custom_roles = optional(string) - observability = optional(string) - org_policies = optional(string) - quotas = optional(string) - }), {}) - parent = optional(string) - prefix = optional(string) - service_encryption_key_ids = optional(map(list(string))) - storage_location = optional(string) - tag_bindings = optional(map(string)) - services = optional(list(string)) - # non-project resources - service_accounts = optional(map(object({ - display_name = optional(string, "Terraform-managed.") - iam_self_roles = optional(list(string)) - }))) - universe = optional(object({ - prefix = string - unavailable_service_identities = optional(list(string), []) - unavailable_services = optional(list(string), []) - })) - vpc_sc = optional(object({ - perimeter_name = string - is_dry_run = optional(bool, false) - })) - logging_data_access = optional(map(object({ - ADMIN_READ = optional(object({ exempted_members = optional(list(string)) })), - DATA_READ = optional(object({ exempted_members = optional(list(string)) })), - DATA_WRITE = optional(object({ exempted_members = optional(list(string)) })) - }))) - }) - nullable = false - default = {} -} - -variable "factories_config" { - description = "Path to folder with YAML resource description data files." - type = object({ - folders = optional(string) - projects = optional(string) - budgets = optional(object({ - billing_account_id = string - data = string - })) - }) - nullable = false -} diff --git a/modules/project-factory/.gitignore b/modules/project-factory-legacy/.gitignore similarity index 100% rename from modules/project-factory/.gitignore rename to modules/project-factory-legacy/.gitignore diff --git a/modules/project-factory-legacy/README.md b/modules/project-factory-legacy/README.md new file mode 100644 index 000000000..cdc877267 --- /dev/null +++ b/modules/project-factory-legacy/README.md @@ -0,0 +1,595 @@ +# Project and Folder Factory + +This module implements end-to-end creation processes for a folder hierarchy, projects and billing budgets via YAML data configurations. + +It supports + +- filesystem-driven folder hierarchy exposing the full configuration options available in the [folder module](../folder/) +- multiple project creation and management exposing the full configuration options available in the [project module](../project/), including KMS key grants and VPC-SC perimeter membership +- optional per-project [service account and bucket management](#service-accounts-and-buckets) including basic IAM grants +- optional [billing budgets](#billing-budgets) factory and budget/project associations +- cross-referencing of hierarchy folders in projects +- optional per-project IaC configuration + +The factory is implemented as a thin data translation layer for the underlying modules, so that no "magic" or hidden side effects are implemented in code, and debugging or integration of new features are simple. + +The code is meant to be executed by a high level service accounts with powerful permissions: + +- folder admin permissions for the hierarchy +- project creation on the nodes (folder or org) where projects will be defined +- Shared VPC connection if service project attachment is desired +- billing cost manager permissions to manage budgets and monitoring permissions if notifications should also be managed here + +## Contents + + +- [Folder hierarchy](#folder-hierarchy) +- [Projects](#projects) + - [Factory-wide project defaults, merges, optionals](#factory-wide-project-defaults-merges-optionals) + - [Service accounts and buckets](#service-accounts-and-buckets) + - [Automation project and resources](#automation-project-and-resources) +- [Billing budgets](#billing-budgets) +- [Interpolation in YAML configuration attributes](#interpolation-in-yaml-configuration-attributes) +- [Example](#example) +- [Files](#files) +- [Variables](#variables) +- [Outputs](#outputs) +- [Tests](#tests) + + +## Folder hierarchy + +The hierarchy supports up to three levels of folders, which are defined via filesystem directories each including a `_config.yaml` files detailing their attributes. + +The hierarchy factory is configured via the `factories_config.folders_data_path` variable, which sets the the path containing the YAML definitions for folders. + +Parent ids for top-level folders can either be set explicitly (e.g. `folders/12345678`) or via substitutions, by referring to keys in the `context.folder_ids` variable. The special `default` key in the substitutions folder variable is used if present and no folder id/key has been specified in the YAML. + +Filesystem directories can also contain project definitions in the same YAML format described below. This approach must be used with caution and is best adopted for stable scenarios, as problems in the filesystem hierarchy definitions might result in the project files not being read and the resources being deleted by Terraform. + +Refer to the [example](#example) below for actual examples of the YAML definitions. + +## Projects + +The project factory is configured via the `factories_config.projects_data_path` variable, and project files are also read from the hierarchy describe in the previous section when enabled. The YAML format mirrors the project module, refer to the [example](#example) below for actual examples of the YAML definitions. + +### Factory-wide project defaults, merges, optionals + +In addition to the YAML-based project configurations, the factory accepts three additional sets of inputs via Terraform variables: + +- the `data_defaults` variable allows defining defaults for specific project attributes, which are only used if the attributes are not passed in via YAML +- the `data_overrides` variable works similarly to defaults, but the values specified here take precedence over those in YAML files +- the `data_merges` variable allows specifying additional values for map or set based variables, which are merged with the data coming from YAML + +Some examples on where to use each of the three sets are [provided below](#example). + +### Service accounts and buckets + +Service accounts and GCS buckets can be managed as part of each project's YAML configuration. This allows creation of default service accounts used for GCE instances, in firewall rules, or for application-level credentials without resorting to a separate Terraform configuration. + +Each service account is represented by one key and a set of optional key/value pairs in the `service_accounts` top-level YAML map, which exposes most of the variables available in the `iam-service-account` module. Most of the service accounts attributes are optional. + +```yaml +service_accounts: + be-0: {} + fe-1: + display_name: GCE frontend service account. + iam_self_roles: + - roles/storage.objectViewer + iam_project_roles: + my-host-project: + - roles/compute.networkUser + iam_sa_roles: + be-0: + - roles/iam.serviceAccountUser + terraform-rw: {} +``` + +Each bucket is represented by one key and a set of optional key/value pairs in the `buckets` top-level YAML map, which exposes most of the variables available in the `gcs` module. Bucket location, storage class and a few other attributes can be defaulted/enforced via project factory level variables. + +```yaml +buckets: + state: + location: europe-west8 + iam: + roles/storage.admin: + - terraform-rw +``` + +### Automation project and resources + +Other than creating automation resources within the project via the `service_accounts` and `buckets` attributes, this module also support management of automation resources created in a separate controlling project. This allows grating broad roles on the project, while still making sure that the automation resources used for Terraform cannot be manipulated from the same identities. + +Automation resources are defined via the `automation` attribute in project configurations, which supports: + +- a mandatory `project` attribute to define the external controlling project; this attribute does not support interpolation and needs to be explicit +- an optional `service_accounts` list where each element defines a service account in the controlling project +- an optional `bucket` which defines a bucket in the controlling project, and the map of roles/principals in the corresponding value assigned on the created bucket; principals can refer to the created service accounts by key + +Service accounts and buckets are prefixed with the project name. Service accounts use the key specified in the YAML file as a suffix, while buckets use a default `tf-state` suffix. + +```yaml +# file name: prod-app-example-0 +# prefix via factory defaults: foo +# project id: foo-prod-app-example-0 +billing_account: 012345-67890A-BCDEF0 +parent: folders/12345678 +services: + - compute.googleapis.com + - stackdriver.googleapis.com +iam: + roles/owner: + - rw + roles/viewer: + - ro +automation: + project: foo-prod-iac-core-0 + service_accounts: + # sa name: foo-prod-app-example-0-rw + rw: + description: Read/write automation sa for app example 0. + # sa name: foo-prod-app-example-0-ro + ro: + description: Read-only automation sa for app example 0. + bucket: + # bucket name: foo-prod-app-example-0-tf-state + description: Terraform state bucket for app example 0. + iam: + roles/storage.objectCreator: + - rw + roles/storage.objectViewer: + - rw + - ro + - group:devops@example.org +``` + +## Billing budgets + +The billing budgets factory integrates the `[`billing-account`](../billing-account/) module functionality, and adds support for easy referencing budgets in project files. + +To enable support for billing budgets, set the billing account id, optional notification channels, and the data folder for budgets in the `factories_config.budgets` variable, then create billing budgets using YAML definitions following the format described in the `billing-account` module. + +Once budgets are defined, they can be referenced in a project file using their file name: + +```yaml +billing_account: 012345-67890A-BCDEF0 +labels: + app: app-1 + team: foo +parent: folders/12345678 +services: + - container.googleapis.com + - storage.googleapis.com +billing_budgets: + - test-100 +``` + +A simple billing budget example is show in the [example](#example) below. + +## Interpolation in YAML configuration attributes + +Interpolation allow referring via short mnemonic names to resources which are either created at runtime, or externally managed. + +This feature has two main benefits: + +- being able to refer to resource ids which cannot be known before creation, for example project automation service accounts in IAM bindings +- making YAML configuration files more easily readable and portable, by using mnemonic keys which are not specific to an organization or project + +One example of both types of contexts is in this project snippet. The automation service account is used in IAM bindings via its `rw` key, while the parent folder is set by referring to its path in the hierarchy factory. + +```yaml +parent: teams/team-a +iam: + "roles/owner": + - rw +automation: + project: ta-app0-0 + service_accounts: + rw: + description: Read/write automation sa for team a app 0. + buckets: + state: + description: Terraform state bucket for team a app 0. + iam: + roles/storage.objectCreator: + - rw +``` + +Interpolations leverage contexts from two separate sources: an internal set for resources managed by the project factory (folders, service accounts, etc.), and an external user-defined set passed in via the `factories_config.context` variable. + +The following table lists the available context interpolations. External contexts are passed in via the `factories_config.contexts` variable. IAM principals are interpolated in all IAM attributes except `iam_by_principal`. First two columns show for which attribute of which resource context is interpolated. `external contexts` column show in which map passed as `var.factories_config.context` key will be looked up. + +- Internally created folders creates keys under `${folder_name_1}[/${folder_name_2}/${folder_name_3}]` +- IAM principals are resolved within context of managed project or use `${project}/${service_account}` to refer service account from other projects managed by the same project factory instance. + +| resource | attribute | external contexts | internal contexts | +| ------------------- | -------------------- | ------------------- | ---------------------------------- | +| folder | parent | `folder_ids` | implicit through folder structure | +| folder | IAM principals | `iam_principals` | | +| folder | tag bindings | `tag_values` | | +| project | parent | `folder_ids` | internally created folders | +| project | Shared VPC host | `vpc_host_projects` | | +| project | Shared VPC IAM | `iam_principals` | project service accounts | +| | | | project service agents | +| | | | IaC service accounts | +| | | | other project service accounts | +| | | | other project IaC service accounts | +| | | | project number in principals | +| project | tag bindings | `tag_values` | | +| project | IAM principals | `iam_principals` | project service accounts | +| | | | IaC service accounts | +| | | | other project service accounts | +| | | | other project service agents | +| | | | other project IaC service accounts | +| | | | project number in principals | +| bucket | IAM principals | `iam_principals` | project service accounts | +| | | | IaC service accounts | +| | | | other project service accounts | +| | | | other project IaC service accounts | +| | | | project number in principals | +| service account | IAM projects | `vpc_host_projects` | | +| service account | `iam_sa_roles` | | service accounts in the same project | +| IaC bucket | IAM principals | `iam_principals` | IaC service accounts | +| IaC service account | IAM principals | `iam_principals` | | + +## Example + +The module invocation using all optional features: + +```hcl +module "project-factory" { + source = "./fabric/modules/project-factory-legacy" + # use a default billing account if none is specified via yaml + data_defaults = { + billing_account = var.billing_account_id + storage_location = "EU" + } + # make sure the environment label and stackdriver service are always added + data_merges = { + labels = { + environment = "test" + } + services = [ + "stackdriver.googleapis.com" + ] + } + # always use this contacts and prefix, regardless of what is in the yaml file + data_overrides = { + contacts = { + "admin@example.org" = ["ALL"] + } + prefix = "test-pf" + } + # location where the yaml files are read from + factories_config = { + budgets = { + billing_account = var.billing_account_id + budgets_data_path = "data/budgets" + notification_channels = { + billing-default = { + project_id = "foo-billing-audit" + type = "email" + labels = { + email_address = "gcp-billing-admins@example.org" + } + } + } + } + folders_data_path = "data/hierarchy" + projects_data_path = "data/projects" + context = { + folder_ids = { + default = "folders/5678901234" + teams = "folders/5678901234" + } + kms_keys = { + compute-prod-ew1 = "projects/kms-central-prj/locations/europe-west1/keyRings/my-keyring/cryptoKeys/ew1-compute" + } + iam_principals = { + gcp-devops = "group:gcp-devops@example.org" + } + tag_values = { + "org-policies/drs-allow-all" = "tagValues/123456" + } + vpc_host_projects = { + dev-spoke-0 = "test-pf-dev-net-spoke-0" + } + } + } +} +# tftest files=0,1,2,3,4,5,6,7,8,9 inventory=example.yaml +``` + +A simple hierarchy of folders: + +```yaml +name: Team A +# implicit parent definition via 'default' key +iam: + roles/viewer: + - group:team-a-admins@example.org + - gcp-devops +# tftest-file id=0 path=data/hierarchy/team-a/_config.yaml schema=folder.schema.json +``` + +```yaml +name: Team B +# explicit parent definition via key +parent: teams +# tftest-file id=1 path=data/hierarchy/team-b/_config.yaml schema=folder.schema.json +``` + +```yaml +name: Team C +# explicit parent definition via folder id +parent: folders/5678901234 +# tftest-file id=2 path=data/hierarchy/team-c/_config.yaml schema=folder.schema.json +``` + +```yaml +name: App 0 +# tftest-file id=3 path=data/hierarchy/team-a/app-0/_config.yaml schema=folder.schema.json +``` + +```yaml +name: App 0 +tag_bindings: + drs-allow-all: org-policies/drs-allow-all +# tftest-file id=4 path=data/hierarchy/team-b/app-0/_config.yaml schema=folder.schema.json +``` + +One project defined within the folder hierarchy: + +```yaml +billing_account: 012345-67890A-BCDEF0 +services: + - container.googleapis.com + - storage.googleapis.com +# tftest-file id=5 path=data/hierarchy/teams-iac-0.yaml schema=project.schema.json +``` + +More traditional project definitions via the project factory data: + +```yaml +billing_account: 012345-67890A-BCDEF0 +labels: + app: app-0 + team: team-a +parent: team-a/app-0 +service_encryption_key_ids: + storage.googleapis.com: + - projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce + compute.googleapis.com: + - compute-prod-ew1 +services: + - compute.googleapis.com + - container.googleapis.com + - storage.googleapis.com +iam_by_principals: + app-0-be: + - roles/storage.objectViewer +iam: + roles/cloudkms.cryptoKeyEncrypterDecrypter: + - storage +service_accounts: + app-0-be: + display_name: "Backend instances." + iam_project_roles: + dev-spoke-0: + - roles/compute.networkUser + iam_self_roles: + - roles/logging.logWriter + - roles/monitoring.metricWriter + app-0-fe: + display_name: "Frontend instances." + iam_project_roles: + dev-spoke-0: + - roles/compute.networkUser + iam_self_roles: + - roles/logging.logWriter + - roles/monitoring.metricWriter +shared_vpc_service_config: + host_project: dev-spoke-0 + network_users: + - gcp-devops + service_agent_iam: + "roles/container.hostServiceAgentUser": + - container-engine + "roles/compute.networkUser": + - container-engine +billing_budgets: + - test-100 +tags: + my-tag-key-1: + values: + my-value-1: + description: My value 1 + my-value-2: + description: My value 3 + iam: + roles/resourcemanager.tagUser: + - user:user@example.com +# tftest-file id=6 path=data/projects/dev-ta-app0-be.yaml schema=project.schema.json +``` + +This project defines a controlling project via the `automation` attributes: + +```yaml +parent: team-b/app-0 +services: +- run.googleapis.com +- storage.googleapis.com +iam: + "roles/owner": + - automation/rw + "roles/viewer": + - automation/ro +shared_vpc_host_config: + enabled: true +service_accounts: + vm-default: + display_name: "VM default service account." + iam_self_roles: + - roles/logging.logWriter + - roles/monitoring.metricWriter + iam: + "roles/iam.serviceAccountTokenCreator": + - automation/rw +automation: + project: test-pf-teams-iac-0 + # prefix used for automation resources can be explicitly set if needed + # prefix: test-pf-dev-tb-0-0 + service_accounts: + rw: + description: Team B app 0 read/write automation sa. + ro: + description: Team B app 0 read-only automation sa. + bucket: + description: Team B app 0 Terraform state bucket. + iam: + roles/storage.objectCreator: + - automation/rw + roles/storage.objectViewer: + - gcp-devops + - group:team-b-admins@example.org + - automation/rw + - automation/ro + +# tftest-file id=7 path=data/projects/dev-tb-app0-0.yaml schema=project.schema.json +``` + +A billing budget: + +```yaml +# billing budget test-100 +display_name: 100 dollars in current spend +amount: + units: 100 +filter: + period: + calendar: MONTH + resource_ancestors: + - folders/1234567890 +threshold_rules: +- percent: 0.5 +- percent: 0.75 +update_rules: + default: + disable_default_iam_recipients: true + monitoring_notification_channels: + - billing-default +# tftest-file id=8 path=data/budgets/test-100.yaml schema=budget.schema.json +``` + +Granting permissions to service accounts defined in other project through interpolation: + +```yaml +billing_account: 012345-67890A-BCDEF0 +labels: + app: app-0 + team: team-b +parent: team-b/app-0 +services: + - container.googleapis.com + - storage.googleapis.com +iam: + "roles/run.admin": + - dev-ta-app0-be/app-0-be # interpolate to app-0-be service account in project defined in file dev-ta-app0-be + "roles/run.developer": + - app-0-be # interpolate to app-0-be service account within the same project +service_accounts: + app-0-be: + display_name: "Backend instances." + iam_self_roles: + - roles/logging.logWriter + - roles/monitoring.metricWriter +# tftest-file id=9 path=data/projects/dev-tb-app0-1.yaml schema=project.schema.json +``` + + + +## Files + +| name | description | modules | +|---|---|---| +| [automation.tf](./automation.tf) | Automation projects locals and resources. | gcs · iam-service-account | +| [factory-budgets.tf](./factory-budgets.tf) | Billing budget factory locals. | | +| [factory-folders.tf](./factory-folders.tf) | Folder hierarchy factory locals. | | +| [factory-projects-object.tf](./factory-projects-object.tf) | None | | +| [factory-projects.tf](./factory-projects.tf) | Projects factory locals. | | +| [folders.tf](./folders.tf) | Folder hierarchy factory resources. | folder | +| [main.tf](./main.tf) | Projects and billing budgets factory resources. | billing-account · gcs · iam-service-account · project | +| [outputs.tf](./outputs.tf) | Module outputs. | | +| [variables.tf](./variables.tf) | Module variables. | | + +## Variables + +| name | description | type | required | default | +|---|---|:---:|:---:|:---:| +| [factories_config](variables.tf#L144) | Path to folder with YAML resource description data files. | object({…}) | ✓ | | +| [data_defaults](variables.tf#L17) | Optional default values used when corresponding project data from files are missing. | object({…}) | | {} | +| [data_merges](variables.tf#L84) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | object({…}) | | {} | +| [data_overrides](variables.tf#L103) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | object({…}) | | {} | +| [factories_data](variables.tf#L172) | Alternate factory data input allowing to use this module as a library. Merged with local YAML data. | object({…}) | | {} | + +## Outputs + +| name | description | sensitive | +|---|---|:---:| +| [buckets](outputs.tf#L17) | Bucket names. | | +| [folders](outputs.tf#L24) | Folder ids. | | +| [projects](outputs.tf#L29) | Created projects. | | +| [service_accounts](outputs.tf#L55) | Service account emails. | | + +## Tests + +These tests validate fixes to the project factory. + +```hcl +module "project-factory" { + source = "./fabric/modules/project-factory-legacy" + data_defaults = { + billing_account = "012345-67890A-ABCDEF" + } + data_merges = { + labels = { + owner = "foo" + } + services = [ + "compute.googleapis.com" + ] + } + data_overrides = { + prefix = "foo" + } + factories_config = { + projects_data_path = "data/projects" + } +} +# tftest modules=4 resources=22 files=test-0,test-1,test-2 +``` + +```yaml +parent: folders/1234567890 +services: + - iam.googleapis.com + - contactcenteraiplatform.googleapis.com + - container.googleapis.com +# tftest-file id=test-0 path=data/projects/test-0.yaml +``` + +```yaml +parent: folders/1234567890 +services: + - iam.googleapis.com + - contactcenteraiplatform.googleapis.com +# tftest-file id=test-1 path=data/projects/test-1.yaml +``` + +```yaml +parent: folders/1234567890 +services: + - iam.googleapis.com + - storage.googleapis.com +# tftest-file id=test-2 path=data/projects/test-2.yaml +``` diff --git a/modules/project-factory-legacy/automation.tf b/modules/project-factory-legacy/automation.tf new file mode 100644 index 000000000..e8d301999 --- /dev/null +++ b/modules/project-factory-legacy/automation.tf @@ -0,0 +1,167 @@ +/** + * Copyright 2024 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Automation projects locals and resources. + +locals { + automation_buckets = { + for k, v in local.projects : + k => merge(try(v.automation.bucket, {}), { + automation_project = v.automation.project + prefix = coalesce( + try(v.automation.prefix, null), + "${v.prefix}-${v.name}" + ) + project_name = v.name + }) if try(v.automation.bucket, null) != null + } + automation_sa = flatten([ + for k, v in local.projects : [ + for ks, kv in try(v.automation.service_accounts, {}) : merge(kv, { + automation_project = v.automation.project + name = ks + prefix = coalesce( + try(v.automation.prefix, null), + "${v.prefix}-${v.name}" + ) + project = k + project_name = v.name + }) + ] + ]) +} + +module "automation-bucket" { + source = "../gcs" + for_each = local.automation_buckets + # we cannot use interpolation here as we would get a cycle + # from the IAM dependency in the outputs of the main project + project_id = each.value.automation_project + prefix = each.value.prefix + name = "tf-state" + encryption_key = lookup(each.value, "encryption_key", null) + force_destroy = try(coalesce( + var.data_overrides.bucket.force_destroy, + each.value.force_destroy, + var.data_defaults.bucket.force_destroy, + ), null) + iam = { + for k, v in lookup(each.value, "iam", {}) : k => [ + for vv in v : try( + module.automation-service-accounts["${each.key}/automation/${vv}"].iam_email, + module.automation-service-accounts["${each.key}/${vv}"].iam_email, + var.factories_config.context.iam_principals[vv], + vv + ) + ] + } + iam_bindings = { + for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { + members = [ + for vv in v.members : try( + # rw (infer local project and automation prefix) + module.automation-service-accounts["${each.key}/automation/${vv}"].iam_email, + # automation/rw or sa (infer local project) + module.automation-service-accounts["${each.key}/${vv}"].iam_email, + # project/automation/rw project/sa + var.factories_config.context.iam_principals[vv], + # fully specified principal + vv, + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(vv, ":") + ? vv + : tonumber("[Error] Invalid member: '${vv}' in automation bucket '${each.key}'") + ) + ) + ] + }) + } + iam_bindings_additive = { + for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { + member = try( + module.automation-service-accounts["${each.key}/automation/${v.member}"].iam_email, + module.automation-service-accounts["${each.key}/${v.member}"].iam_email, + var.factories_config.context.iam_principals[v.member], + v.member + ) + }) + } + labels = lookup(each.value, "labels", {}) + location = coalesce( + var.data_overrides.storage_location, + lookup(each.value, "location", null), + var.data_defaults.storage_location + ) + storage_class = lookup( + each.value, "storage_class", "STANDARD" + ) + uniform_bucket_level_access = lookup( + each.value, "uniform_bucket_level_access", true + ) + versioning = lookup( + each.value, "versioning", false + ) +} + +module "automation-service-accounts" { + source = "../iam-service-account" + for_each = { + for k in local.automation_sa : "${k.project}/automation/${k.name}" => k + } + # we cannot use interpolation here as we would get a cycle + # from the IAM dependency in the outputs of the main project + project_id = each.value.automation_project + prefix = each.value.prefix + name = each.value.name + description = lookup(each.value, "description", null) + display_name = lookup( + each.value, + "display_name", + "Service account ${each.value.name} for ${each.value.project}." + ) + # TODO: also support short form for service accounts in this project + iam = { + for k, v in lookup(each.value, "iam", {}) : k => [ + for vv in v : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + } + iam_bindings = { + for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { + members = [ + for vv in v.members : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + }) + } + iam_bindings_additive = { + for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { + member = lookup( + var.factories_config.context.iam_principals, v.member, v.member + ) + }) + } + iam_billing_roles = lookup(each.value, "iam_billing_roles", {}) + iam_folder_roles = lookup(each.value, "iam_folder_roles", {}) + iam_organization_roles = lookup(each.value, "iam_organization_roles", {}) + iam_project_roles = lookup(each.value, "iam_project_roles", {}) + iam_sa_roles = lookup(each.value, "iam_sa_roles", {}) + # we don't interpolate buckets here as we can't use a dynamic key + iam_storage_roles = lookup(each.value, "iam_storage_roles", {}) +} diff --git a/modules/project-factory/factory-budgets.tf b/modules/project-factory-legacy/factory-budgets.tf similarity index 100% rename from modules/project-factory/factory-budgets.tf rename to modules/project-factory-legacy/factory-budgets.tf diff --git a/modules/project-factory/factory-folders.tf b/modules/project-factory-legacy/factory-folders.tf similarity index 100% rename from modules/project-factory/factory-folders.tf rename to modules/project-factory-legacy/factory-folders.tf diff --git a/modules/project-factory/factory-projects-object.tf b/modules/project-factory-legacy/factory-projects-object.tf similarity index 100% rename from modules/project-factory/factory-projects-object.tf rename to modules/project-factory-legacy/factory-projects-object.tf diff --git a/modules/project-factory/factory-projects.tf b/modules/project-factory-legacy/factory-projects.tf similarity index 100% rename from modules/project-factory/factory-projects.tf rename to modules/project-factory-legacy/factory-projects.tf diff --git a/modules/project-factory-legacy/folders.tf b/modules/project-factory-legacy/folders.tf new file mode 100644 index 000000000..6f4710853 --- /dev/null +++ b/modules/project-factory-legacy/folders.tf @@ -0,0 +1,158 @@ +/** + * Copyright 2024 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Folder hierarchy factory resources. + +locals { + folder_parent_default = try( + var.factories_config.context.folder_ids.default, null + ) +} + +module "hierarchy-folder-lvl-1" { + source = "../folder" + for_each = { for k, v in local.folders : k => v if v.level == 1 } + parent = try( + # allow the YAML data to set the parent for this level + lookup( + var.factories_config.context.folder_ids, + each.value.parent, + each.value.parent + ), + # use the default value in the initial parents map + local.folder_parent_default + # fail if we don't have an explicit parent + ) + name = each.value.name + iam = { + for k, v in lookup(each.value, "iam", {}) : k => [ + # don't interpolate automation service account to prevent cycles + for vv in v : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + } + iam_bindings = { + for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { + members = [ + # don't interpolate automation service account to prevent cycles + for vv in v.members : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + }) + } + iam_bindings_additive = { + for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { + # don't interpolate automation service account to prevent cycles + member = lookup( + var.factories_config.context.iam_principals, v.member, v.member + ) + }) + } + iam_by_principals = { + for k, v in lookup(each.value, "iam_by_principals", {}) : + lookup( + var.factories_config.context.iam_principals, k, k + ) => v + } + org_policies = lookup(each.value, "org_policies", {}) + tag_bindings = { + for k, v in lookup(each.value, "tag_bindings", {}) : + k => lookup(var.factories_config.context.tag_values, v, v) + } + logging_data_access = lookup(each.value, "logging_data_access", {}) +} + +module "hierarchy-folder-lvl-2" { + source = "../folder" + for_each = { for k, v in local.folders : k => v if v.level == 2 } + parent = module.hierarchy-folder-lvl-1[each.value.parent_key].id + name = each.value.name + iam = { + for k, v in lookup(each.value, "iam", {}) : k => [ + # don't interpolate automation service account to prevent cycles + for vv in v : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + } + iam_bindings = { + for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { + members = [ + # don't interpolate automation service account to prevent cycles + for vv in v.members : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + }) + } + iam_bindings_additive = { + for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { + # don't interpolate automation service account to prevent cycles + member = lookup( + var.factories_config.context.iam_principals, v.member, v.member + ) + }) + } + iam_by_principals = lookup(each.value, "iam_by_principals", {}) + org_policies = lookup(each.value, "org_policies", {}) + tag_bindings = { + for k, v in lookup(each.value, "tag_bindings", {}) : + k => lookup(var.factories_config.context.tag_values, v, v) + } + logging_data_access = lookup(each.value, "logging_data_access", {}) +} + +module "hierarchy-folder-lvl-3" { + source = "../folder" + for_each = { for k, v in local.folders : k => v if v.level == 3 } + parent = module.hierarchy-folder-lvl-2[each.value.parent_key].id + name = each.value.name + iam = { + for k, v in lookup(each.value, "iam", {}) : k => [ + # don't interpolate automation service account to prevent cycles + for vv in v : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + } + iam_bindings = { + for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { + members = [ + # don't interpolate automation service account to prevent cycles + for vv in v.members : lookup( + var.factories_config.context.iam_principals, vv, vv + ) + ] + }) + } + iam_bindings_additive = { + for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { + # don't interpolate automation service account to prevent cycles + member = lookup( + var.factories_config.context.iam_principals, v.member, v.member + ) + }) + } + iam_by_principals = lookup(each.value, "iam_by_principals", {}) + org_policies = lookup(each.value, "org_policies", {}) + tag_bindings = { + for k, v in lookup(each.value, "tag_bindings", {}) : + k => lookup(var.factories_config.context.tag_values, v, v) + } + logging_data_access = lookup(each.value, "logging_data_access", {}) +} diff --git a/modules/project-factory-legacy/main.tf b/modules/project-factory-legacy/main.tf new file mode 100644 index 000000000..29d15f22c --- /dev/null +++ b/modules/project-factory-legacy/main.tf @@ -0,0 +1,490 @@ +/** + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Projects and billing budgets factory resources. + +locals { + _service_agent_emails = flatten([ + for k, v in module.projects : [ + for kk, vv in v.service_agents : { + key = "${k}/${kk}" + value = "serviceAccount:${vv.email}" + } + ] + ]) + context = { + folder_ids = merge( + var.factories_config.context.folder_ids, + local.hierarchy + ) + iam_principals = merge( + var.factories_config.context.iam_principals, + { + for k, v in module.automation-service-accounts : + k => v.iam_email + }, + # module.service-accounts are excluded here, as adding them here results in dependency cycles + ) + } + service_accounts_names = { + for k, v in module.service-accounts : k => v.name + } + service_agents_email = { + for v in local._service_agent_emails : v.key => v.value + } +} + +module "projects" { + source = "../project" + for_each = local.projects + billing_account = each.value.billing_account + deletion_policy = each.value.deletion_policy + name = each.value.name + parent = lookup( + local.context.folder_ids, each.value.parent, each.value.parent + ) + prefix = each.value.prefix + project_reuse = each.value.project_reuse + alerts = try(each.value.alerts, null) + auto_create_network = try(each.value.auto_create_network, false) + compute_metadata = try(each.value.compute_metadata, {}) + # TODO: concat lists for each key + contacts = merge( + each.value.contacts, var.data_merges.contacts + ) + default_service_account = try(each.value.default_service_account, "keep") + descriptive_name = try(each.value.descriptive_name, null) + factories_config = { + custom_roles = each.value.factories_config.custom_roles + observability = each.value.factories_config.observability + org_policies = each.value.factories_config.org_policies + quotas = each.value.factories_config.quotas + context = { + notification_channels = var.factories_config.context.notification_channels + } + } + labels = merge( + each.value.labels, var.data_merges.labels + ) + lien_reason = try(each.value.lien_reason, null) + log_scopes = try(each.value.log_scopes, null) + logging_data_access = try(each.value.logging_data_access, {}) + logging_exclusions = try(each.value.logging_exclusions, {}) + logging_metrics = try(each.value.logging_metrics, null) + logging_sinks = try(each.value.logging_sinks, {}) + metric_scopes = distinct(concat( + each.value.metric_scopes, var.data_merges.metric_scopes + )) + notification_channels = try(each.value.notification_channels, null) + org_policies = each.value.org_policies + service_encryption_key_ids = { + for k, v in merge( + each.value.service_encryption_key_ids, + var.data_merges.service_encryption_key_ids + ) : k => [ + for key in v : lookup(var.factories_config.context.kms_keys, key, key) + ] + } + services = distinct(concat( + each.value.services, + var.data_merges.services + )) + shared_vpc_host_config = each.value.shared_vpc_host_config + tag_bindings = { + for k, v in merge(each.value.tag_bindings, var.data_merges.tag_bindings) : + k => lookup(var.factories_config.context.tag_values, v, v) + } + tags = each.value.tags + vpc_sc = each.value.vpc_sc == null ? null : { + perimeter_name = ( + each.value.vpc_sc.perimeter_name == null + ? null + : lookup( + var.factories_config.context.perimeters, + each.value.vpc_sc.perimeter_name, + each.value.vpc_sc.perimeter_name + ) + ) + is_dry_run = each.value.vpc_sc.is_dry_run + } + quotas = each.value.quotas +} + +module "projects-iam" { + source = "../project" + for_each = local.projects + name = module.projects[each.key].project_id + project_reuse = { + use_data_source = false + attributes = { + name = module.projects[each.key].name + number = module.projects[each.key].number + services_enabled = module.projects[each.key].services + } + } + iam = { + for k, v in lookup(each.value, "iam", {}) : + lookup(var.factories_config.context.custom_roles, k, k) => [ + for vv in v : try( + # project service accounts (sa) + module.service-accounts["${each.key}/${vv}"].iam_email, + # automation service account (rw) + local.context.iam_principals["${each.key}/automation/${vv}"], + # automation service account (automation/rw) + local.context.iam_principals["${each.key}/${vv}"], + # other projects service accounts (project/sa) + module.service-accounts[vv].iam_email, + # other automation service account (project/automation/rw) + local.context.iam_principals[vv], + # project's service identities + local.service_agents_email["${each.key}/${vv}"], + local.service_agents_email[vv], + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(vv, ":") + ? templatestring( + vv, { project_number = module.projects[each.key].number } + ) + : tonumber("[Error] Invalid member: '${vv}' in project '${each.key}'") + ) + ) + ] + } + iam_bindings = { + for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { + members = [ + for vv in v.members : try( + # project service accounts (sa) + module.service-accounts["${each.key}/${vv}"].iam_email, + # automation service account (rw) + local.context.iam_principals["${each.key}/automation/${vv}"], + # automation service account (automation/rw) + local.context.iam_principals["${each.key}/${vv}"], + # other projects service accounts (project/sa) + module.service-accounts[vv].iam_email, + # other automation service account (project/automation/rw) + local.context.iam_principals[vv], + # project's service identities + local.service_agents_email["${each.key}/${vv}"], + local.service_agents_email[vv], + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(vv, ":") + ? templatestring( + vv, { project_number = module.projects[each.key].number } + ) + : tonumber("[Error] Invalid member: '${vv}' in project '${each.key}'") + ) + ) + ] + role = lookup(var.factories_config.context.custom_roles, v.role, v.role) + }) + } + iam_bindings_additive = { + for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { + member = try( + # project service accounts (sa) + module.service-accounts["${each.key}/${v.member}"].iam_email, + # automation service account (rw) + local.context.iam_principals["${each.key}/automation/${v.member}"], + # automation service account (automation/rw) + local.context.iam_principals["${each.key}/${v.member}"], + # other projects service accounts (project/sa) + module.service-accounts[v.member].iam_email, + # other automation service account (project/automation/rw) + local.context.iam_principals[v.member], + # project's service identities + local.service_agents_email["${each.key}/${v.member}"], + local.service_agents_email[v.member], + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(v.member, ":") + ? templatestring( + v.member, { project_number = module.projects[each.key].number } + ) + : tonumber("[Error] Invalid member: '${v.member}' in project '${each.key}'") + ) + ) + role = lookup(var.factories_config.context.custom_roles, v.role, v.role) + }) + } + # IAM by principals would trigger dynamic key errors so we don't interpolate + # iam_by_principals = try(each.value.iam_by_principals, {}) + iam_by_principals = { + for k, v in try(each.value.iam_by_principals, {}) : + try( + # project service accounts (sa) + module.service-accounts["${each.key}/${k}"].iam_email, + # automation service account (rw) + local.context.iam_principals["${each.key}/automation/${k}"], + # automation service account (automation/rw) + local.context.iam_principals["${each.key}/${k}"], + # other projects service accounts (project/sa) + module.service-accounts[k].iam_email, + # other automation service account (project/automation/rw) + local.context.iam_principals[k], + # project's service identities + local.service_agents_email["${each.key}/${k}"], + local.service_agents_email[k], + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(k, ":") + ? templatestring( + k, { project_number = module.projects[each.key].number } + ) + : tonumber("[Error] Invalid member: '${k}' in project '${each.key}'") + ) + ) => [ + for vv in v : lookup(var.factories_config.context.custom_roles, vv, vv) + ] + } + # Shared VPC configuration is done at stage 2, to avoid dependency cycle between project service accounts and + # IAM grants done for those service accounts + shared_vpc_service_config = ( + try(each.value.shared_vpc_service_config.host_project, null) == null + ? null + : merge(each.value.shared_vpc_service_config, { + host_project = try( + var.factories_config.context.vpc_host_projects[each.value.shared_vpc_service_config.host_project], + module.projects[each.value.shared_vpc_service_config.host_project].project_id, + each.value.shared_vpc_service_config.host_project + ) + iam_bindings_additive = { + for k, v in try(each.value.shared_vpc_service_config.iam_bindings_additive, {}) : k => merge(v, { + member = try( + # project service accounts (sa) + module.service-accounts["${each.key}/${v.member}"].iam_email, + # automation service account (rw) + local.context.iam_principals["${each.key}/automation/${v.member}"], + # automation service account (automation/rw) + local.context.iam_principals["${each.key}/${v.member}"], + # other projects service accounts (project/sa) + module.service-accounts[v.member].iam_email, + # other automation service account (project/automation/rw) + local.context.iam_principals[v.member], + # project's service identities + local.service_agents_email["${each.key}/${v.member}"], + local.service_agents_email[v.member], + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(v.member, ":") + ? templatestring( + v.member, { project_number = module.projects[each.key].number } + ) + : tonumber("[Error] Invalid member: '${v.member}' in project '${each.key}'") + ) + ) + role = lookup(var.factories_config.context.custom_roles, v.role, v.role) + }) + } + network_users = [ + for vv in try(each.value.shared_vpc_service_config.network_users, []) : + try( + # project service accounts (sa) + module.service-accounts["${each.key}/${vv}"].iam_email, + # automation service account (rw) + local.context.iam_principals["${each.key}/automation/${vv}"], + # automation service account (automation/rw) + local.context.iam_principals["${each.key}/${vv}"], + # other projects service accounts (project/sa) + module.service-accounts[vv].iam_email, + # other automation service account (project/automation/rw) + local.context.iam_principals[vv], + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(vv, ":") + ? templatestring( + vv, { project_number = module.projects[each.key].number } + ) + : tonumber("[Error] Invalid member: '${vv}' in project '${each.key}'") + ) + ) + ] + }) + ) + # add service agents config, so Service Agents can be referred in the IAM grants + service_agents_config = { + # default roles are granted in module.project + grant_default_roles = false + } +} + +module "buckets" { + source = "../gcs" + for_each = { + for k in local.buckets : "${k.project_key}/${k.name}" => k + } + project_id = module.projects[each.value.project_key].project_id + prefix = each.value.prefix + name = "${each.value.project_name}-${each.value.name}" + encryption_key = each.value.encryption_key + force_destroy = each.value.force_destroy + iam = { + for k, v in each.value.iam : k => [ + for vv in v : try( + # project service accounts (sa) + module.service-accounts["${each.value.project_key}/${vv}"].iam_email, + # automation service account (rw) + local.context.iam_principals["${each.value.project_key}/automation/${vv}"], + # automation service account (automation/rw) + local.context.iam_principals["${each.value.project_key}/${vv}"], + # other projects service accounts (project/sa) + module.service-accounts[vv].iam_email, + # other automation service account (project/automation/rw) + local.context.iam_principals[vv], + # project's service identities + local.service_agents_email["${each.value.project_key}/${vv}"], + local.service_agents_email[vv], + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(vv, ":") + ? templatestring( + vv, { project_number = module.projects[each.value.project_key].number } + ) + : tonumber("[Error] Invalid member: '${vv}' in bucket '${each.key}'") + ) + ) + ] + } + iam_bindings = { + for k, v in each.value.iam_bindings : k => merge(v, { + members = [ + for vv in v.members : try( + # project service accounts (sa) + module.service-accounts["${each.value.project_key}/${vv}"].iam_email, + # automation service account (rw) + local.context.iam_principals["${each.value.project_key}/automation/${vv}"], + # automation service account (automation/rw) + local.context.iam_principals["${each.value.project_key}/${vv}"], + # other projects service accounts (project/sa) + module.service-accounts[vv].iam_email, + # other automation service account (project/automation/rw) + local.context.iam_principals[vv], + # project's service identities + local.service_agents_email["${each.value.project_key}/${vv}"], + local.service_agents_email[vv], + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(vv, ":") + ? templatestring( + vv, { project_number = module.projects[each.value.project_key].number } + ) + : tonumber("[Error] Invalid member: '${vv}' in bucket '${each.key}'") + ) + ) + ] + }) + } + iam_bindings_additive = { + for k, v in each.value.iam_bindings_additive : k => merge(v, { + member = try( + # project service accounts (sa) + module.service-accounts["${each.value.project_key}/${v.member}"].iam_email, + # automation service account (rw) + local.context.iam_principals["${each.value.project_key}/automation/${v.member}"], + # automation service account (automation/rw) + local.context.iam_principals["${each.value.project_key}/${v.member}"], + # other projects service accounts (project/sa) + module.service-accounts[v.member].iam_email, + # other automation service account (project/automation/rw) + local.context.iam_principals[v.member], + # project's service identities + local.service_agents_email["${each.value.project_key}/${v.member}"], + local.service_agents_email[v.member], + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(v.member, ":") + ? templatestring( + v.member, { project_number = module.projects[each.value.project_key].number } + ) + : tonumber("[Error] Invalid member: '${v.member}' in bucket '${each.key}'") + ) + ) + }) + } + labels = each.value.labels + location = coalesce( + var.data_overrides.storage_location, + lookup(each.value, "location", null), + var.data_defaults.storage_location + ) + storage_class = each.value.storage_class + uniform_bucket_level_access = each.value.uniform_bucket_level_access + versioning = each.value.versioning +} + +module "service-accounts" { + source = "../iam-service-account" + for_each = { + for k in local.service_accounts : "${k.project_key}/${k.name}" => k + } + project_id = module.projects[each.value.project_key].project_id + name = each.value.name + display_name = each.value.display_name + iam = { + for k, v in lookup(each.value, "iam", {}) : k => [ + for vv in v : try( + # automation service account (rw) + local.context.iam_principals["${each.value.project_key}/automation/${vv}"], + # automation service account (automation/rw) + local.context.iam_principals["${each.value.project_key}/${vv}"], + # other automation service account (project/automation/rw) + local.context.iam_principals[vv], + # passthrough + error handling using tonumber until Terraform gets fail/raise function + ( + strcontains(vv, ":") + ? vv + : tonumber("[Error] Invalid member: '${vv}' in project '${each.value.project_key}'") + ) + ) + ] + } + iam_project_roles = merge( + { + for k, v in each.value.iam_project_roles : + lookup(var.factories_config.context.vpc_host_projects, k, k) => v + }, + each.value.iam_self_roles == null ? {} : { + (module.projects[each.value.project_key].project_id) = each.value.iam_self_roles + } + ) +} + +module "service_accounts-iam" { + source = "../iam-service-account" + for_each = { + for k in local.service_accounts : "${k.project_key}/${k.name}" => k + if k.iam_sa_roles != {} + } + project_id = module.service-accounts[each.key].service_account.project + name = each.value.name + service_account_create = false + iam_sa_roles = { + for k, v in each.value.iam_sa_roles : lookup( + local.service_accounts_names, "${each.value.project_key}/${k}", k + ) => v + } +} + +module "billing-account" { + source = "../billing-account" + count = var.factories_config.budgets == null ? 0 : 1 + id = var.factories_config.budgets.billing_account + budget_notification_channels = ( + var.factories_config.budgets.notification_channels + ) + budgets = local.budgets +} diff --git a/modules/project-factory-legacy/outputs.tf b/modules/project-factory-legacy/outputs.tf new file mode 100644 index 000000000..770b89518 --- /dev/null +++ b/modules/project-factory-legacy/outputs.tf @@ -0,0 +1,58 @@ +/** + * Copyright 2024 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +output "buckets" { + description = "Bucket names." + value = { + for k, v in module.buckets : k => v.name + } +} + +output "folders" { + description = "Folder ids." + value = local.hierarchy +} + +output "projects" { + description = "Created projects." + value = { + for k, v in module.projects : k => { + number = v.number + project_id = v.id + project = v + automation = ( + lookup(local.projects[k], "automation", null) == null + ? null + : { + bucket = try(module.automation-bucket[k].name, null) + service_accounts = { + for kk, vv in module.automation-service-accounts : + trimprefix(kk, "${k}/") => vv.email + if startswith(kk, "${k}/") + } + } + ) + service_agents = { + for k, v in v.service_agents : k => v.email if v.is_primary + } + } + } +} + +output "service_accounts" { + description = "Service account emails." + value = module.service-accounts +} diff --git a/modules/project-factory-experimental/schemas/budget.schema.json b/modules/project-factory-legacy/schemas/budget.schema.json similarity index 100% rename from modules/project-factory-experimental/schemas/budget.schema.json rename to modules/project-factory-legacy/schemas/budget.schema.json diff --git a/modules/project-factory-experimental/schemas/budget.schema.md b/modules/project-factory-legacy/schemas/budget.schema.md similarity index 100% rename from modules/project-factory-experimental/schemas/budget.schema.md rename to modules/project-factory-legacy/schemas/budget.schema.md diff --git a/modules/project-factory-experimental/schemas/folder.schema.json b/modules/project-factory-legacy/schemas/folder.schema.json similarity index 51% rename from modules/project-factory-experimental/schemas/folder.schema.json rename to modules/project-factory-legacy/schemas/folder.schema.json index 5fbe2769c..1e87c94c6 100644 --- a/modules/project-factory-experimental/schemas/folder.schema.json +++ b/modules/project-factory-legacy/schemas/folder.schema.json @@ -4,66 +4,6 @@ "type": "object", "additionalProperties": false, "properties": { - "automation": { - "type": "object", - "additionalProperties": false, - "required": [ - "project" - ], - "properties": { - "prefix": { - "type": "string" - }, - "project": { - "type": "string" - }, - "bucket": { - "$ref": "#/$defs/bucket" - }, - "service_accounts": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-z0-9-]+$": { - "type": "object", - "additionalProperties": false, - "properties": { - "description": { - "type": "string" - }, - "iam": { - "$ref": "#/$defs/iam" - }, - "iam_bindings": { - "$ref": "#/$defs/iam_bindings" - }, - "iam_bindings_additive": { - "$ref": "#/$defs/iam_bindings_additive" - }, - "iam_billing_roles": { - "$ref": "#/$defs/iam_billing_roles" - }, - "iam_folder_roles": { - "$ref": "#/$defs/iam_folder_roles" - }, - "iam_organization_roles": { - "$ref": "#/$defs/iam_organization_roles" - }, - "iam_project_roles": { - "$ref": "#/$defs/iam_project_roles" - }, - "iam_sa_roles": { - "$ref": "#/$defs/iam_sa_roles" - }, - "iam_storage_roles": { - "$ref": "#/$defs/iam_storage_roles" - } - } - } - } - } - } - }, "iam": { "$ref": "#/$defs/iam" }, @@ -157,8 +97,7 @@ } }, "parent": { - "type": "string", - "pattern": "^(?:folders/[0-9]+|organizations/[0-9]+|\\$folder_ids:[a-z0-9_-]+)$" + "type": "string" }, "tag_bindings": { "type": "object", @@ -171,84 +110,15 @@ } }, "$defs": { - "bucket": { - "type": "object", - "additionalProperties": false, - "properties": { - "name": { - "type": "string" - }, - "description": { - "type": "string" - }, - "iam": { - "$ref": "#/$defs/iam" - }, - "iam_bindings": { - "$ref": "#/$defs/iam_bindings" - }, - "iam_bindings_additive": { - "$ref": "#/$defs/iam_bindings_additive" - }, - "force_destroy": { - "type": "boolean" - }, - "labels": { - "type": "object", - "additionalProperties": { - "type": "string" - } - }, - "location": { - "type": "string" - }, - "managed_folders": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-zA-Z0-9][a-zA-Z0-9_/-]+$": { - "type": "object", - "additionalProperties": false, - "properties": { - "force_destroy": { - "type": "boolean" - }, - "iam": { - "$ref": "#/$defs/iam" - }, - "iam_bindings": { - "$ref": "#/$defs/iam_bindings" - }, - "iam_bindings_additive": { - "$ref": "#/$defs/iam_bindings_additive" - } - } - } - } - }, - "prefix": { - "type": "string" - }, - "storage_class": { - "type": "string" - }, - "uniform_bucket_level_access": { - "type": "boolean" - }, - "versioning": { - "type": "boolean" - } - } - }, "iam": { "type": "object", "additionalProperties": false, "patternProperties": { - "^(?:roles/|\\$custom_roles:)": { + "^roles/": { "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" } } } @@ -265,7 +135,7 @@ "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" } }, "role": { @@ -305,7 +175,7 @@ "properties": { "member": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" }, "role": { "type": "string", @@ -338,83 +208,11 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:)": { + "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])": { "type": "array", "items": { "type": "string", - "pattern": "^(?:roles/|\\$custom_roles:)" - } - } - } - }, - "iam_billing_roles": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-z0-9-]+$": { - "type": "array", - "items": { - "type": "string" - } - } - } - }, - "iam_folder_roles": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-z0-9-]+$": { - "type": "array", - "items": { - "type": "string" - } - } - } - }, - "iam_organization_roles": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-z0-9-]+$": { - "type": "array", - "items": { - "type": "string" - } - } - } - }, - "iam_project_roles": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-z0-9-]+$": { - "type": "array", - "items": { - "type": "string" - } - } - } - }, - "iam_sa_roles": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-z0-9-]+$": { - "type": "array", - "items": { - "type": "string" - } - } - } - }, - "iam_storage_roles": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-z0-9-]+$": { - "type": "array", - "items": { - "type": "string" + "pattern": "^roles/" } } } diff --git a/fast/stages/2-project-factory-experimental/schemas/folder.schema.md b/modules/project-factory-legacy/schemas/folder.schema.md similarity index 100% rename from fast/stages/2-project-factory-experimental/schemas/folder.schema.md rename to modules/project-factory-legacy/schemas/folder.schema.md diff --git a/modules/project-factory-experimental/schemas/project.schema.json b/modules/project-factory-legacy/schemas/project.schema.json similarity index 86% rename from modules/project-factory-experimental/schemas/project.schema.json rename to modules/project-factory-legacy/schemas/project.schema.json index 626dfcc2a..8f255e909 100644 --- a/modules/project-factory-experimental/schemas/project.schema.json +++ b/modules/project-factory-legacy/schemas/project.schema.json @@ -111,15 +111,6 @@ "labels": { "type": "object" }, - "log_buckets": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-z0-9-]+$": { - "$ref": "#/$defs/log_bucket" - } - } - }, "metric_scopes": { "type": "array", "items": { @@ -482,27 +473,6 @@ } } }, - "universe": { - "type": "object", - "additionalProperties": false, - "properties": { - "prefix": { - "type": "string", - "unavailable_services": { - "type": "array", - "items": { - "type": "string" - } - }, - "unavailable_service_identities": { - "type": "array", - "items": { - "type": "string" - } - } - } - } - }, "vpc_sc": { "type": "object", "additionalItems": false, @@ -524,9 +494,6 @@ "type": "object", "additionalProperties": false, "properties": { - "name": { - "type": "string" - }, "description": { "type": "string" }, @@ -551,30 +518,6 @@ "location": { "type": "string" }, - "managed_folders": { - "type": "object", - "additionalProperties": false, - "patternProperties": { - "^[a-zA-Z0-9][a-zA-Z0-9_/-]+$": { - "type": "object", - "additionalProperties": false, - "properties": { - "force_destroy": { - "type": "boolean" - }, - "iam": { - "$ref": "#/$defs/iam" - }, - "iam_bindings": { - "$ref": "#/$defs/iam_bindings" - }, - "iam_bindings_additive": { - "$ref": "#/$defs/iam_bindings_additive" - } - } - } - } - }, "prefix": { "type": "string" }, @@ -602,11 +545,11 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^(?:roles/|\\$custom_roles:)": { + "^roles/": { "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:||\\$iam_principals:[a-z0-9_-]+)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" } } } @@ -623,12 +566,12 @@ "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:[a-z0-9_-]+)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" } }, "role": { "type": "string", - "pattern": "^(?:roles/|\\$custom_roles:)" + "pattern": "^roles/" }, "condition": { "type": "object", @@ -663,11 +606,11 @@ "properties": { "member": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:[a-z0-9_-]+)" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" }, "role": { "type": "string", - "pattern": "^(?:roles/|\\$custom_roles:)" + "pattern": "^[a-zA-Z0-9_/.]+$" }, "condition": { "type": "object", @@ -696,11 +639,11 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:[a-z0-9_-]+)": { + "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])": { "type": "array", "items": { "type": "string", - "pattern": "^(?:roles/|\\$custom_roles:)" + "pattern": "^roles/" } } } @@ -745,7 +688,7 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^(?:[a-z0-9-]|\\$project_ids:[a-z0-9_-])+$": { + "^[a-z0-9-]+$": { "type": "array", "items": { "type": "string" @@ -757,7 +700,7 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^(?:\\$service_account_ids:|projects/)": { + "^[a-z0-9-]+$": { "type": "array", "items": { "type": "string" @@ -776,40 +719,6 @@ } } } - }, - "log_bucket": { - "type": "object", - "additionalProperties": false, - "properties": { - "description": { - "type": "string" - }, - "kms_key_name": { - "type": "string" - }, - "location": { - "type": "string" - }, - "log_analytics": { - "type": "object", - "additionalProperties": false, - "properties": { - "enable": { - "type": "boolean", - "default": false - }, - "dataset_link_id": { - "type": "string" - }, - "description": { - "type": "string" - } - } - }, - "retention": { - "type": "number" - } - } } } -} \ No newline at end of file +} diff --git a/fast/stages/2-project-factory-experimental/schemas/project.schema.md b/modules/project-factory-legacy/schemas/project.schema.md similarity index 100% rename from fast/stages/2-project-factory-experimental/schemas/project.schema.md rename to modules/project-factory-legacy/schemas/project.schema.md diff --git a/modules/project-factory-legacy/variables.tf b/modules/project-factory-legacy/variables.tf new file mode 100644 index 000000000..5dbc10c6f --- /dev/null +++ b/modules/project-factory-legacy/variables.tf @@ -0,0 +1,415 @@ +/** + * Copyright 2025 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +variable "data_defaults" { + description = "Optional default values used when corresponding project data from files are missing." + type = object({ + billing_account = optional(string) + bucket = optional(object({ + force_destroy = optional(bool) + }), {}) + contacts = optional(map(list(string)), {}) + deletion_policy = optional(string) + factories_config = optional(object({ + custom_roles = optional(string) + observability = optional(string) + org_policies = optional(string) + quotas = optional(string) + }), {}) + labels = optional(map(string), {}) + metric_scopes = optional(list(string), []) + parent = optional(string) + prefix = optional(string) + project_reuse = optional(object({ + use_data_source = optional(bool, true) + attributes = optional(object({ + name = string + number = number + services_enabled = optional(list(string), []) + })) + })) + service_encryption_key_ids = optional(map(list(string)), {}) + services = optional(list(string), []) + shared_vpc_service_config = optional(object({ + host_project = string + iam_bindings_additive = optional(map(object({ + member = string + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + network_users = optional(list(string), []) + service_agent_iam = optional(map(list(string)), {}) + service_agent_subnet_iam = optional(map(list(string)), {}) + service_iam_grants = optional(list(string), []) + network_subnet_users = optional(map(list(string)), {}) + })) + storage_location = optional(string) + tag_bindings = optional(map(string), {}) + # non-project resources + service_accounts = optional(map(object({ + display_name = optional(string, "Terraform-managed.") + iam_self_roles = optional(list(string)) + })), {}) + vpc_sc = optional(object({ + perimeter_name = string + is_dry_run = optional(bool, false) + })) + logging_data_access = optional(map(object({ + ADMIN_READ = optional(object({ exempted_members = optional(list(string)) })), + DATA_READ = optional(object({ exempted_members = optional(list(string)) })), + DATA_WRITE = optional(object({ exempted_members = optional(list(string)) })) + })), {}) + }) + nullable = false + default = {} +} + +variable "data_merges" { + description = "Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`." + type = object({ + contacts = optional(map(list(string)), {}) + labels = optional(map(string), {}) + metric_scopes = optional(list(string), []) + service_encryption_key_ids = optional(map(list(string)), {}) + services = optional(list(string), []) + tag_bindings = optional(map(string), {}) + # non-project resources + service_accounts = optional(map(object({ + display_name = optional(string, "Terraform-managed.") + iam_self_roles = optional(list(string)) + })), {}) + }) + nullable = false + default = {} +} + +variable "data_overrides" { + description = "Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`." + type = object({ + # data overrides default to null to mark that they should not override + billing_account = optional(string) + bucket = optional(object({ + force_destroy = optional(bool) + }), {}) + contacts = optional(map(list(string))) + deletion_policy = optional(string) + factories_config = optional(object({ + custom_roles = optional(string) + observability = optional(string) + org_policies = optional(string) + quotas = optional(string) + }), {}) + parent = optional(string) + prefix = optional(string) + service_encryption_key_ids = optional(map(list(string))) + storage_location = optional(string) + tag_bindings = optional(map(string)) + services = optional(list(string)) + # non-project resources + service_accounts = optional(map(object({ + display_name = optional(string, "Terraform-managed.") + iam_self_roles = optional(list(string)) + }))) + vpc_sc = optional(object({ + perimeter_name = string + is_dry_run = optional(bool, false) + })) + logging_data_access = optional(map(object({ + ADMIN_READ = optional(object({ exempted_members = optional(list(string)) })), + DATA_READ = optional(object({ exempted_members = optional(list(string)) })), + DATA_WRITE = optional(object({ exempted_members = optional(list(string)) })) + }))) + }) + nullable = false + default = {} +} + +variable "factories_config" { + description = "Path to folder with YAML resource description data files." + type = object({ + folders_data_path = optional(string) + projects_data_path = optional(string) + budgets = optional(object({ + billing_account = string + budgets_data_path = string + # TODO: allow defining notification channels via YAML files + notification_channels = optional(map(any), {}) + })) + context = optional(object({ + custom_roles = optional(map(string), {}) + folder_ids = optional(map(string), {}) + iam_principals = optional(map(string), {}) + kms_keys = optional(map(string), {}) + perimeters = optional(map(string), {}) + tag_values = optional(map(string), {}) + vpc_host_projects = optional(map(string), {}) + notification_channels = optional(map(string), {}) + }), {}) + projects_config = optional(object({ + key_ignores_path = optional(bool, false) + }), {}) + }) + nullable = false +} + +variable "factories_data" { + description = "Alternate factory data input allowing to use this module as a library. Merged with local YAML data." + type = object({ + budgets = optional(map(object({ + amount = object({ + currency_code = optional(string) + nanos = optional(number) + units = optional(number) + use_last_period = optional(bool) + }) + display_name = optional(string) + filter = optional(object({ + credit_types_treatment = optional(object({ + exclude_all = optional(bool) + include_specified = optional(list(string)) + })) + label = optional(object({ + key = string + value = string + })) + period = optional(object({ + calendar = optional(string) + custom = optional(object({ + start_date = object({ + day = number + month = number + year = number + }) + end_date = optional(object({ + day = number + month = number + year = number + })) + })) + })) + projects = optional(list(string)) + resource_ancestors = optional(list(string)) + services = optional(list(string)) + subaccounts = optional(list(string)) + })) + threshold_rules = optional(list(object({ + percent = number + forecasted_spend = optional(bool) + })), []) + update_rules = optional(map(object({ + disable_default_iam_recipients = optional(bool) + monitoring_notification_channels = optional(list(string)) + pubsub_topic = optional(string) + })), {}) + })), {}) + hierarchy = optional(map(object({ + name = optional(string) + parent = optional(string) + iam = optional(map(list(string)), {}) + iam_bindings = optional(map(object({ + members = list(string) + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + iam_bindings_additive = optional(map(object({ + member = string + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + iam_by_principals = optional(map(list(string)), {}) + tag_bindings = optional(map(string), {}) + })), {}) + projects = optional(map(object({ + automation = optional(object({ + project = string + bucket = optional(object({ + location = string + description = optional(string) + force_destroy = optional(bool) + prefix = optional(string) + storage_class = optional(string, "STANDARD") + uniform_bucket_level_access = optional(bool, true) + versioning = optional(bool) + iam = optional(map(list(string)), {}) + iam_bindings = optional(map(object({ + members = list(string) + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + iam_bindings_additive = optional(map(object({ + member = string + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + labels = optional(map(string), {}) + })) + service_accounts = optional(map(object({ + description = optional(string) + iam = optional(map(list(string)), {}) + iam_bindings = optional(map(object({ + members = list(string) + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + iam_bindings_additive = optional(map(object({ + member = string + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + iam_billing_roles = optional(map(list(string)), {}) + iam_folder_roles = optional(map(list(string)), {}) + iam_organization_roles = optional(map(list(string)), {}) + iam_project_roles = optional(map(list(string)), {}) + iam_sa_roles = optional(map(list(string)), {}) + iam_storage_roles = optional(map(list(string)), {}) + })), {}) + })) + billing_account = optional(string) + billing_budgets = optional(list(string), []) + buckets = optional(map(object({ + location = string + description = optional(string) + force_destroy = optional(bool) + prefix = optional(string) + storage_class = optional(string, "STANDARD") + uniform_bucket_level_access = optional(bool, true) + versioning = optional(bool) + iam = optional(map(list(string)), {}) + iam_bindings = optional(map(object({ + members = list(string) + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + iam_bindings_additive = optional(map(object({ + member = string + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + labels = optional(map(string), {}) + })), {}) + contacts = optional(map(list(string)), {}) + iam = optional(map(list(string)), {}) + iam_bindings = optional(map(object({ + members = list(string) + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + iam_bindings_additive = optional(map(object({ + member = string + role = string + condition = optional(object({ + expression = string + title = string + description = optional(string) + })) + })), {}) + iam_by_principals = optional(map(list(string)), {}) + labels = optional(map(string), {}) + metric_scopes = optional(list(string), []) + name = optional(string) + org_policies = optional(map(object({ + inherit_from_parent = optional(bool) # for list policies only. + reset = optional(bool) + rules = optional(list(object({ + allow = optional(object({ + all = optional(bool) + values = optional(list(string)) + })) + deny = optional(object({ + all = optional(bool) + values = optional(list(string)) + })) + enforce = optional(bool) # for boolean policies only. + condition = optional(object({ + description = optional(string) + expression = optional(string) + location = optional(string) + title = optional(string) + }), {}) + parameters = optional(string) + })), []) + })), {}) + parent = optional(string) + prefix = optional(string) + service_accounts = optional(map(object({ + display_name = optional(string) + iam_self_roles = optional(list(string), []) + iam_project_roles = optional(map(list(string)), {}) + })), {}) + service_encryption_key_ids = optional(map(list(string)), {}) + services = optional(list(string), []) + shared_vpc_host_config = optional(object({ + enabled = bool + service_projects = optional(list(string), []) + })) + shared_vpc_service_config = optional(object({ + host_project = string + network_users = optional(list(string), []) + service_agent_iam = optional(map(list(string)), {}) + service_agent_subnet_iam = optional(map(list(string)), {}) + service_iam_grants = optional(list(string), []) + network_subnet_users = optional(map(list(string)), {}) + })) + tag_bindings = optional(map(string), {}) + vpc_sc = optional(object({ + perimeter_name = string + is_dry_run = optional(bool, false) + })) + })), {}) + }) + nullable = false + default = {} +} diff --git a/modules/project-factory/README.md b/modules/project-factory/README.md index 87b90eecb..91951db98 100644 --- a/modules/project-factory/README.md +++ b/modules/project-factory/README.md @@ -6,18 +6,21 @@ It supports - filesystem-driven folder hierarchy exposing the full configuration options available in the [folder module](../folder/) - multiple project creation and management exposing the full configuration options available in the [project module](../project/), including KMS key grants and VPC-SC perimeter membership -- optional per-project [service account and bucket management](#service-accounts-and-buckets) including basic IAM grants +- optional per-project [service accounts and buckets management](#service-accounts-and-buckets) including basic IAM grants - optional [billing budgets](#billing-budgets) factory and budget/project associations - cross-referencing of hierarchy folders in projects - optional per-project IaC configuration +- global defaults or overrides for most project configurations +- extensive support of [context-based interpolation](#context-based-interpolation) -The factory is implemented as a thin data translation layer for the underlying modules, so that no "magic" or hidden side effects are implemented in code, and debugging or integration of new features are simple. +The factory is implemented as a thin data translation layer over the underlying modules, so that no "magic" or hidden side effects are implemented in code, and debugging or integration of new features are simple. -The code is meant to be executed by a high level service accounts with powerful permissions: +The code is meant to be executed by a high level service account with powerful permissions: - folder admin permissions for the hierarchy - project creation on the nodes (folder or org) where projects will be defined - Shared VPC connection if service project attachment is desired +- VPC Service Controls perimeter management if project inclusion is desired - billing cost manager permissions to manage budgets and monitoring permissions if notifications should also be managed here ## Contents @@ -29,7 +32,11 @@ The code is meant to be executed by a high level service accounts with powerful - [Service accounts and buckets](#service-accounts-and-buckets) - [Automation project and resources](#automation-project-and-resources) - [Billing budgets](#billing-budgets) -- [Interpolation in YAML configuration attributes](#interpolation-in-yaml-configuration-attributes) +- [Context-based interpolation](#context-based-interpolation) + - [Folder context ids](#folder-context-ids) + - [Project context ids](#project-context-ids) + - [Service account context ids](#service-account-context-ids) + - [Other context ids](#other-context-ids) - [Example](#example) - [Files](#files) - [Variables](#variables) @@ -39,11 +46,11 @@ The code is meant to be executed by a high level service accounts with powerful ## Folder hierarchy -The hierarchy supports up to three levels of folders, which are defined via filesystem directories each including a `_config.yaml` files detailing their attributes. +The hierarchy supports up to three levels of folders, which are defined via filesystem directories each including a `.config.yaml` files detailing their attributes. -The hierarchy factory is configured via the `factories_config.folders_data_path` variable, which sets the the path containing the YAML definitions for folders. +The filesystem tree containing folder definitions is configured via the `factories_config.folders` variable, which sets the the path containing the YAML definitions for folders. It's also possible to configure the hierarchy via the `folders` variable, which is internally merged in with the factory definitions. -Parent ids for top-level folders can either be set explicitly (e.g. `folders/12345678`) or via substitutions, by referring to keys in the `context.folder_ids` variable. The special `default` key in the substitutions folder variable is used if present and no folder id/key has been specified in the YAML. +Parent ids for top-level folders can either be set explicitly (e.g. `folders/12345678`), or via [context interpolation](#context-based-interpolation) by referring to keys in the `context.folder_ids` variable. The special `default` key in the substitutions folder variable is used if present and no folder id/key has been specified in the YAML. Filesystem directories can also contain project definitions in the same YAML format described below. This approach must be used with caution and is best adopted for stable scenarios, as problems in the filesystem hierarchy definitions might result in the project files not being read and the resources being deleted by Terraform. @@ -51,7 +58,11 @@ Refer to the [example](#example) below for actual examples of the YAML definitio ## Projects -The project factory is configured via the `factories_config.projects_data_path` variable, and project files are also read from the hierarchy describe in the previous section when enabled. The YAML format mirrors the project module, refer to the [example](#example) below for actual examples of the YAML definitions. +The project factory is configured in three ia the `factories_config.projects` variable, and project files are also additionally read from the folder tree described in the previous section. It's best to limit project definition via the hierarchy tree to a minimum to avoid cross dependencies between folders and projects, which could complicate their lifecycle. + +Projects can also be configured via the `projects` variable, which is internally merged in with the factory definitions. + +The YAML format mirrors the project module, refer to the [example](#example) below for actual examples of the YAML definitions. ### Factory-wide project defaults, merges, optionals @@ -77,10 +88,10 @@ service_accounts: iam_self_roles: - roles/storage.objectViewer iam_project_roles: - my-host-project: + $project_ids:my-host-project: - roles/compute.networkUser iam_sa_roles: - be-0: + $iam_principals:service_accounts/my-project/be-0: - roles/iam.serviceAccountUser terraform-rw: {} ``` @@ -93,7 +104,7 @@ buckets: location: europe-west8 iam: roles/storage.admin: - - terraform-rw + - $iam_principals:service_accounts/my-project/terraform-rw ``` ### Automation project and resources @@ -119,11 +130,11 @@ services: - stackdriver.googleapis.com iam: roles/owner: - - rw + - $iam_principals:service_accounts/iac-core-0/rw roles/viewer: - - ro + - $iam_principals:service_accounts/iac-core-0/ro automation: - project: foo-prod-iac-core-0 + project: $project_ids:iac-core-0 service_accounts: # sa name: foo-prod-app-example-0-rw rw: @@ -136,10 +147,10 @@ automation: description: Terraform state bucket for app example 0. iam: roles/storage.objectCreator: - - rw + - $iam_principals:service_accounts/iac-core-0/rw roles/storage.objectViewer: - - rw - - ro + - $iam_principals:service_accounts/iac-core-0/rw + - $iam_principals:service_accounts/iac-core-0/ro - group:devops@example.org ``` @@ -166,24 +177,25 @@ billing_budgets: A simple billing budget example is show in the [example](#example) below. -## Interpolation in YAML configuration attributes +## Context-based interpolation -Interpolation allow referring via short mnemonic names to resources which are either created at runtime, or externally managed. +Interpolation allow referring to resources which are either created at runtime, or externally managed via short aliases. This feature has two main benefits: - being able to refer to resource ids which cannot be known before creation, for example project automation service accounts in IAM bindings - making YAML configuration files more easily readable and portable, by using mnemonic keys which are not specific to an organization or project -One example of both types of contexts is in this project snippet. The automation service account is used in IAM bindings via its `rw` key, while the parent folder is set by referring to its path in the hierarchy factory. +One example of both types of contexts is in this project snippet. The automation service account is used in IAM bindings via its key, while the parent folder is set by referring to its path in the hierarchy factory. ```yaml -parent: teams/team-a +# file name: my-project +parent: $folder_ids:teams/team-a iam: "roles/owner": - - rw + - $iam_principals:service_accounts/my-project/rw automation: - project: ta-app0-0 + project: $project_ids:ta-app0-0 service_accounts: rw: description: Read/write automation sa for team a app 0. @@ -192,53 +204,113 @@ automation: description: Terraform state bucket for team a app 0. iam: roles/storage.objectCreator: - - rw + - $iam_principals:service_accounts/my-project/rw ``` -Interpolations leverage contexts from two separate sources: an internal set for resources managed by the project factory (folders, service accounts, etc.), and an external user-defined set passed in via the `factories_config.context` variable. +Interpolations leverage contexts from two separate sources: resources managed by the project factory (folders, service accounts, etc.), and user-defined resource ids passed in via the `context` variable. -The following table lists the available context interpolations. External contexts are passed in via the `factories_config.contexts` variable. IAM principals are interpolated in all IAM attributes except `iam_by_principal`. First two columns show for which attribute of which resource context is interpolated. `external contexts` column show in which map passed as `var.factories_config.context` key will be looked up. +Context replacements use the `$` prefix and are accessible via namespaces that match the attributes in the context variable: -- Internally created folders creates keys under `${folder_name_1}[/${folder_name_2}/${folder_name_3}]` -- IAM principals are resolved within context of managed project or use `${project}/${service_account}` to refer service account from other projects managed by the same project factory instance. +- `$custom_roles:foo` +- `$folder_ids:foo` +- `$iam_principals:foo` +- `$kms_keys:foo` +- `$locations:foo` +- `$notification_channels:foo` +- `$project_ids:foo` +- `$tag_values:foo` +- `$vpc_host_projects:foo` +- `$vpc_sc_perimeters:foo` -| resource | attribute | external contexts | internal contexts | -| ------------------- | -------------------- | ------------------- | ---------------------------------- | -| folder | parent | `folder_ids` | implicit through folder structure | -| folder | IAM principals | `iam_principals` | | -| folder | tag bindings | `tag_values` | | -| project | parent | `folder_ids` | internally created folders | -| project | Shared VPC host | `vpc_host_projects` | | -| project | Shared VPC IAM | `iam_principals` | project service accounts | -| | | | project service agents | -| | | | IaC service accounts | -| | | | other project service accounts | -| | | | other project IaC service accounts | -| | | | project number in principals | -| project | tag bindings | `tag_values` | | -| project | IAM principals | `iam_principals` | project service accounts | -| | | | IaC service accounts | -| | | | other project service accounts | -| | | | other project service agents | -| | | | other project IaC service accounts | -| | | | project number in principals | -| bucket | IAM principals | `iam_principals` | project service accounts | -| | | | IaC service accounts | -| | | | other project service accounts | -| | | | other project IaC service accounts | -| | | | project number in principals | -| service account | IAM projects | `vpc_host_projects` | | -| service account | `iam_sa_roles` | | service accounts in the same project | -| IaC bucket | IAM principals | `iam_principals` | IaC service accounts | -| IaC service account | IAM principals | `iam_principals` | | +Internally created resources are mapped to context namespaces, and use specific prefixes to express the relationship with their container folder/project where necessary, as shown in the following examples. + +### Folder context ids + +Folders ids use the `$folder_ids` namespace, with ids derived from the full filesystem path to express the hierarchy. + +As an example, the id of the folder defined in `folders/networking/prod/.config.yaml` file will be accessible via `$folder_ids:networking/prod`. + +### Project context ids + +Project ids ise the `$project_ids:` namespace, with ids defined in two different ways: + +- projects defined in the `var.factories_config.project` tree use the filename (dirname is stripped) +- projects defined in the `var.factories_config.folders` tree use the full path (dirname is kept) + +As an example, the id of the project defined in the `projects/team-0/app-0-0.yaml` file will be accessible via `$project_ids:app-0-0`. The id of the project defined in the `folders/shared/iac-core-0.yaml` file will be accessible via `$project_ids:shared/iac-core-0`. + +### Service account context ids + +Service accounts use the `$iam_principals:` namespace, with ids that allow referring to their parent project. + +As an example, the `rw` service account defined in the `projects/team-0/app-0-0.yaml` file will be accessible via `$iam_principals:service_accounts/app-0-0/rw`. + +### Other context ids + +Other context ids simply match whatever was passed in via the `var.contexts` variable. The following is a short example. + +```hcl +context = { + custom_roles = { + myrole = "organizations/1234567890/roles/myRoleOne" + } + folder_ids = { + "test/prod" = "folders/1234567890" + } + iam_principals = { + mysa = "serviceAccount:test@test-project.iam.gserviceaccount.com" + } + project_ids = { + vpc-host = "test-vpc-host" + } + tag_values = { + "test/one" = "tagValues/1234567890" + } + vpc_sc_perimeters = { + default = "accessPolicies/888933661165/servicePerimeters/default" + } +} +# tftest: skip +``` + +```yaml +parent: $folder_ids/test/prod +iam: + $custom_roles:myrole: + - $iam_principals:mygroup +shared_vpc_service_config: + host_project: $project_ids:vpc-host +tag_bindings: + foo: $tag_values:test/one +vpc_sc: + perimeter_name: $vpc_sc_perimeters:default +``` ## Example -The module invocation using all optional features: +This show a module invocation using all optional features: ```hcl module "project-factory" { source = "./fabric/modules/project-factory" + context = { + folder_ids = { + default = "folders/5678901234" + teams = "folders/5678901234" + } + kms_keys = { + compute-prod-ew1 = "projects/kms-central-prj/locations/europe-west1/keyRings/my-keyring/cryptoKeys/ew1-compute" + } + iam_principals = { + gcp-devops = "group:gcp-devops@example.org" + } + tag_values = { + "org-policies/drs-allow-all" = "tagValues/123456" + } + vpc_host_projects = { + dev-spoke-0 = "test-pf-dev-net-spoke-0" + } + } # use a default billing account if none is specified via yaml data_defaults = { billing_account = var.billing_account_id @@ -263,36 +335,18 @@ module "project-factory" { # location where the yaml files are read from factories_config = { budgets = { - billing_account = var.billing_account_id - budgets_data_path = "data/budgets" - notification_channels = { - billing-default = { - project_id = "foo-billing-audit" - type = "email" - labels = { - email_address = "gcp-billing-admins@example.org" - } - } - } + billing_account_id = var.billing_account_id + data = "data/budgets" } - folders_data_path = "data/hierarchy" - projects_data_path = "data/projects" - context = { - folder_ids = { - default = "folders/5678901234" - teams = "folders/5678901234" - } - kms_keys = { - compute-prod-ew1 = "projects/kms-central-prj/locations/europe-west1/keyRings/my-keyring/cryptoKeys/ew1-compute" - } - iam_principals = { - gcp-devops = "group:gcp-devops@example.org" - } - tag_values = { - "org-policies/drs-allow-all" = "tagValues/123456" - } - vpc_host_projects = { - dev-spoke-0 = "test-pf-dev-net-spoke-0" + folders = "data/hierarchy" + projects = "data/projects" + } + notification_channels = { + billing-default = { + project_id = "foo-billing-audit" + type = "email" + labels = { + email_address = "gcp-billing-admins@example.org" } } } @@ -308,34 +362,34 @@ name: Team A iam: roles/viewer: - group:team-a-admins@example.org - - gcp-devops -# tftest-file id=0 path=data/hierarchy/team-a/_config.yaml schema=folder.schema.json + - $iam_principals:gcp-devops +# tftest-file id=0 path=data/hierarchy/team-a/.config.yaml schema=folder.schema.json ``` ```yaml name: Team B # explicit parent definition via key -parent: teams -# tftest-file id=1 path=data/hierarchy/team-b/_config.yaml schema=folder.schema.json +parent: $folder_ids:teams +# tftest-file id=1 path=data/hierarchy/team-b/.config.yaml schema=folder.schema.json ``` ```yaml name: Team C # explicit parent definition via folder id parent: folders/5678901234 -# tftest-file id=2 path=data/hierarchy/team-c/_config.yaml schema=folder.schema.json +# tftest-file id=2 path=data/hierarchy/team-c/.config.yaml schema=folder.schema.json ``` ```yaml name: App 0 -# tftest-file id=3 path=data/hierarchy/team-a/app-0/_config.yaml schema=folder.schema.json +# tftest-file id=3 path=data/hierarchy/team-a/app-0/.config.yaml schema=folder.schema.json ``` ```yaml name: App 0 tag_bindings: - drs-allow-all: org-policies/drs-allow-all -# tftest-file id=4 path=data/hierarchy/team-b/app-0/_config.yaml schema=folder.schema.json + drs-allow-all: $tag_values:org-policies/drs-allow-all +# tftest-file id=4 path=data/hierarchy/team-b/app-0/.config.yaml schema=folder.schema.json ``` One project defined within the folder hierarchy: @@ -355,27 +409,27 @@ billing_account: 012345-67890A-BCDEF0 labels: app: app-0 team: team-a -parent: team-a/app-0 +parent: $folder_ids:team-a/app-0 service_encryption_key_ids: storage.googleapis.com: - projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce compute.googleapis.com: - - compute-prod-ew1 + - $kms_keys:compute-prod-ew1 services: - compute.googleapis.com - container.googleapis.com - storage.googleapis.com iam_by_principals: - app-0-be: + $iam_principals:service_accounts/dev-ta-app0-be/app-0-be: - roles/storage.objectViewer iam: roles/cloudkms.cryptoKeyEncrypterDecrypter: - - storage + - $service_agents:storage service_accounts: app-0-be: display_name: "Backend instances." iam_project_roles: - dev-spoke-0: + $project_ids:dev-spoke-0: - roles/compute.networkUser iam_self_roles: - roles/logging.logWriter @@ -383,22 +437,22 @@ service_accounts: app-0-fe: display_name: "Frontend instances." iam_project_roles: - dev-spoke-0: + $project_ids:dev-spoke-0: - roles/compute.networkUser iam_self_roles: - roles/logging.logWriter - roles/monitoring.metricWriter shared_vpc_service_config: - host_project: dev-spoke-0 + host_project: $project_ids:dev-spoke-0 network_users: - - gcp-devops + - $iam_principals:gcp-devops service_agent_iam: "roles/container.hostServiceAgentUser": - - container-engine + - $service_agents:container-engine "roles/compute.networkUser": - - container-engine + - $service_agents:container-engine billing_budgets: - - test-100 + - $billing_budgets:test-100 tags: my-tag-key-1: values: @@ -415,15 +469,15 @@ tags: This project defines a controlling project via the `automation` attributes: ```yaml -parent: team-b/app-0 +parent: $folder_ids:team-b/app-0 services: - run.googleapis.com - storage.googleapis.com iam: "roles/owner": - - automation/rw + - $iam_principals:service_accounts/dev-tb-app0-0/rw "roles/viewer": - - automation/ro + - $iam_principals:service_accounts/dev-tb-app0-0/ro shared_vpc_host_config: enabled: true service_accounts: @@ -433,8 +487,8 @@ service_accounts: - roles/logging.logWriter - roles/monitoring.metricWriter iam: - "roles/iam.serviceAccountTokenCreator": - - automation/rw + roles/iam.serviceAccountTokenCreator: + - $iam_principals:service_accounts/dev-tb-app0-0/rw automation: project: test-pf-teams-iac-0 # prefix used for automation resources can be explicitly set if needed @@ -448,12 +502,12 @@ automation: description: Team B app 0 Terraform state bucket. iam: roles/storage.objectCreator: - - automation/rw + - $iam_principals:service_accounts/dev-tb-app0-0/rw roles/storage.objectViewer: - - gcp-devops + - $iam_principals:gcp-devops - group:team-b-admins@example.org - - automation/rw - - automation/ro + - $iam_principals:service_accounts/dev-tb-app0-0/rw + - $iam_principals:service_accounts/dev-tb-app0-0/ro # tftest-file id=7 path=data/projects/dev-tb-app0-0.yaml schema=project.schema.json ``` @@ -477,7 +531,7 @@ update_rules: default: disable_default_iam_recipients: true monitoring_notification_channels: - - billing-default + - $notification_channels:billing-default # tftest-file id=8 path=data/budgets/test-100.yaml schema=budget.schema.json ``` @@ -488,15 +542,15 @@ billing_account: 012345-67890A-BCDEF0 labels: app: app-0 team: team-b -parent: team-b/app-0 +parent: $folder_ids:team-b/app-0 services: - container.googleapis.com - storage.googleapis.com iam: "roles/run.admin": - - dev-ta-app0-be/app-0-be # interpolate to app-0-be service account in project defined in file dev-ta-app0-be + - $iam_principals:service_accounts/dev-ta-app0-be/app-0-be "roles/run.developer": - - app-0-be # interpolate to app-0-be service account within the same project + - $iam_principals:service_accounts/dev-tb-app0-1/app-0-be service_accounts: app-0-be: display_name: "Backend instances." @@ -510,36 +564,48 @@ service_accounts: ## Files -| name | description | modules | -|---|---|---| -| [automation.tf](./automation.tf) | Automation projects locals and resources. | gcs · iam-service-account | -| [factory-budgets.tf](./factory-budgets.tf) | Billing budget factory locals. | | -| [factory-folders.tf](./factory-folders.tf) | Folder hierarchy factory locals. | | -| [factory-projects-object.tf](./factory-projects-object.tf) | None | | -| [factory-projects.tf](./factory-projects.tf) | Projects factory locals. | | -| [folders.tf](./folders.tf) | Folder hierarchy factory resources. | folder | -| [main.tf](./main.tf) | Projects and billing budgets factory resources. | billing-account · gcs · iam-service-account · project | -| [outputs.tf](./outputs.tf) | Module outputs. | | -| [variables.tf](./variables.tf) | Module variables. | | +| name | description | modules | resources | +|---|---|---|---| +| [automation.tf](./automation.tf) | None | gcs · iam-service-account | | +| [budgets.tf](./budgets.tf) | Billing budget factory locals. | billing-account | | +| [folders.tf](./folders.tf) | Folder hierarchy factory resources. | folder | | +| [main.tf](./main.tf) | Projects and billing budgets factory resources. | | terraform_data | +| [outputs.tf](./outputs.tf) | Module outputs. | | | +| [projects-buckets.tf](./projects-buckets.tf) | None | gcs | | +| [projects-defaults.tf](./projects-defaults.tf) | None | | | +| [projects-log-buckets.tf](./projects-log-buckets.tf) | None | logging-bucket | | +| [projects-service-accounts.tf](./projects-service-accounts.tf) | None | iam-service-account | | +| [projects.tf](./projects.tf) | None | project | | +| [variables-billing.tf](./variables-billing.tf) | None | | | +| [variables-folders.tf](./variables-folders.tf) | None | | | +| [variables-projects.tf](./variables-projects.tf) | None | | | +| [variables.tf](./variables.tf) | Module variables. | | | ## Variables | name | description | type | required | default | |---|---|:---:|:---:|:---:| -| [factories_config](variables.tf#L144) | Path to folder with YAML resource description data files. | object({…}) | ✓ | | -| [data_defaults](variables.tf#L17) | Optional default values used when corresponding project data from files are missing. | object({…}) | | {} | -| [data_merges](variables.tf#L84) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | object({…}) | | {} | -| [data_overrides](variables.tf#L103) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | object({…}) | | {} | -| [factories_data](variables.tf#L172) | Alternate factory data input allowing to use this module as a library. Merged with local YAML data. | object({…}) | | {} | +| [factories_config](variables.tf#L173) | Path to folder with YAML resource description data files. | object({…}) | ✓ | | +| [context](variables.tf#L17) | Context-specific interpolations. | object({…}) | | {} | +| [data_defaults](variables.tf#L36) | Optional default values used when corresponding project or folder data from files are missing. | object({…}) | | {} | +| [data_merges](variables.tf#L108) | Optional values that will be merged with corresponding data from files. Combines with `data_defaults`, file data, and `data_overrides`. | object({…}) | | {} | +| [data_overrides](variables.tf#L127) | Optional values that override corresponding data from files. Takes precedence over file data and `data_defaults`. | object({…}) | | {} | +| [folders](variables-folders.tf#L17) | Folders data merged with factory data. | map(object({…})) | | {} | +| [notification_channels](variables-billing.tf#L17) | Notification channels used by budget alerts. | map(object({…})) | | {} | +| [projects](variables-projects.tf#L17) | Projects data merged with factory data. | map(object({…})) | | {} | ## Outputs | name | description | sensitive | |---|---|:---:| -| [buckets](outputs.tf#L17) | Bucket names. | | -| [folders](outputs.tf#L24) | Folder ids. | | -| [projects](outputs.tf#L29) | Created projects. | | -| [service_accounts](outputs.tf#L55) | Service account emails. | | +| [folder_ids](outputs.tf#L44) | Folder ids. | | +| [iam_principals](outputs.tf#L49) | IAM principals mappings. | | +| [log_buckets](outputs.tf#L54) | Log bucket ids. | | +| [project_ids](outputs.tf#L61) | Project ids. | | +| [project_numbers](outputs.tf#L66) | Project numbers. | | +| [projects](outputs.tf#L73) | Project attributes. | | +| [service_accounts](outputs.tf#L78) | Service account emails. | | +| [storage_buckets](outputs.tf#L85) | Bucket names. | | ## Tests @@ -549,7 +615,8 @@ These tests validate fixes to the project factory. module "project-factory" { source = "./fabric/modules/project-factory" data_defaults = { - billing_account = "012345-67890A-ABCDEF" + billing_account = "012345-67890A-ABCDEF" + storage_location = "eu" } data_merges = { labels = { @@ -563,10 +630,10 @@ module "project-factory" { prefix = "foo" } factories_config = { - projects_data_path = "data/projects" + projects = "data/projects" } } -# tftest modules=4 resources=22 files=test-0,test-1,test-2 +# tftest modules=4 resources=23 files=test-0,test-1,test-2 ``` ```yaml diff --git a/modules/project-factory/automation.tf b/modules/project-factory/automation.tf index e8d301999..ca0c45a92 100644 --- a/modules/project-factory/automation.tf +++ b/modules/project-factory/automation.tf @@ -1,5 +1,5 @@ /** - * Copyright 2024 Google LLC + * Copyright 2025 Google LLC * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -14,97 +14,93 @@ * limitations under the License. */ -# tfdoc:file:description Automation projects locals and resources. - locals { - automation_buckets = { - for k, v in local.projects : - k => merge(try(v.automation.bucket, {}), { - automation_project = v.automation.project - prefix = coalesce( - try(v.automation.prefix, null), - "${v.prefix}-${v.name}" - ) - project_name = v.name - }) if try(v.automation.bucket, null) != null - } - automation_sa = flatten([ - for k, v in local.projects : [ - for ks, kv in try(v.automation.service_accounts, {}) : merge(kv, { - automation_project = v.automation.project - name = ks + _automation = merge( + { + for k, v in local.folders_input : k => { + bucket = try(v.automation.bucket, {}) + # name = replace(k, "/", "-") + parent_type = "folder" + prefix = try(v.automation.prefix, null) + project = try(v.automation.project, null) + service_accounts = try(v.automation.service_accounts, {}) + } if try(v.automation.bucket, null) != null + }, + { + for k, v in local.projects_input : k => { + bucket = try(v.automation.bucket, {}) + # name = v.name + parent_type = "project" prefix = coalesce( try(v.automation.prefix, null), - "${v.prefix}-${v.name}" + v.prefix == null ? v.name : "${v.prefix}-${v.name}" ) - project = k - project_name = v.name + project = try(v.automation.project, null) + service_accounts = try(v.automation.service_accounts, {}) + } if try(v.automation.bucket, null) != null + } + ) + _automation_buckets = { + for k, v in local._automation : k => merge(v.bucket, { + automation_project = v.project + name = lookup(v, "name", "tf-state") + # project automation always has a prefix + prefix = try(coalesce( + v.prefix, + local.data_defaults.overrides.prefix, + local.data_defaults.defaults.prefix + ), null) + }) + } + _automation_sas = flatten(concat([ + for k, v in local._automation : [ + for sk, sv in v.service_accounts : merge(sv, { + automation_project = v.project + name = sk + parent = k + prefix = v.prefix }) ] - ]) + ])) + automation_buckets = { + for k, v in local._automation_buckets : + "${k}/${v.name}" => v + } + automation_sas = { + for k in local._automation_sas : + "${k.parent}/${k.name}" => k + } + automation_sas_iam_emails = { + for k, v in local.automation_sas : + "service_accounts/${v.parent}/${v.name}" => module.automation-service-accounts[k].iam_email + } } module "automation-bucket" { - source = "../gcs" - for_each = local.automation_buckets - # we cannot use interpolation here as we would get a cycle - # from the IAM dependency in the outputs of the main project + source = "../gcs" + for_each = local.automation_buckets project_id = each.value.automation_project prefix = each.value.prefix - name = "tf-state" + name = each.value.name encryption_key = lookup(each.value, "encryption_key", null) force_destroy = try(coalesce( - var.data_overrides.bucket.force_destroy, + local.data_defaults.overrides.bucket.force_destroy, each.value.force_destroy, - var.data_defaults.bucket.force_destroy, + local.data_defaults.defaults.force_destroy, ), null) - iam = { - for k, v in lookup(each.value, "iam", {}) : k => [ - for vv in v : try( - module.automation-service-accounts["${each.key}/automation/${vv}"].iam_email, - module.automation-service-accounts["${each.key}/${vv}"].iam_email, - var.factories_config.context.iam_principals[vv], - vv - ) - ] - } - iam_bindings = { - for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { - members = [ - for vv in v.members : try( - # rw (infer local project and automation prefix) - module.automation-service-accounts["${each.key}/automation/${vv}"].iam_email, - # automation/rw or sa (infer local project) - module.automation-service-accounts["${each.key}/${vv}"].iam_email, - # project/automation/rw project/sa - var.factories_config.context.iam_principals[vv], - # fully specified principal - vv, - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(vv, ":") - ? vv - : tonumber("[Error] Invalid member: '${vv}' in automation bucket '${each.key}'") - ) - ) - ] - }) - } - iam_bindings_additive = { - for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { - member = try( - module.automation-service-accounts["${each.key}/automation/${v.member}"].iam_email, - module.automation-service-accounts["${each.key}/${v.member}"].iam_email, - var.factories_config.context.iam_principals[v.member], - v.member - ) - }) - } - labels = lookup(each.value, "labels", {}) + context = merge(local.ctx, { + project_ids = local.ctx_project_ids + iam_principals = local.ctx_iam_principals + }) + iam = lookup(each.value, "iam", {}) + iam_bindings = lookup(each.value, "iam_bindings", {}) + iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) + labels = lookup(each.value, "labels", {}) + managed_folders = lookup(each.value, "managed_folders", {}) location = coalesce( - var.data_overrides.storage_location, + local.data_defaults.overrides.storage_location, lookup(each.value, "location", null), - var.data_defaults.storage_location + local.data_defaults.defaults.storage_location ) storage_class = lookup( each.value, "storage_class", "STANDARD" @@ -118,12 +114,8 @@ module "automation-bucket" { } module "automation-service-accounts" { - source = "../iam-service-account" - for_each = { - for k in local.automation_sa : "${k.project}/automation/${k.name}" => k - } - # we cannot use interpolation here as we would get a cycle - # from the IAM dependency in the outputs of the main project + source = "../iam-service-account" + for_each = local.automation_sas project_id = each.value.automation_project prefix = each.value.prefix name = each.value.name @@ -131,32 +123,18 @@ module "automation-service-accounts" { display_name = lookup( each.value, "display_name", - "Service account ${each.value.name} for ${each.value.project}." + "Service account ${each.value.name} for ${each.value.parent}." ) - # TODO: also support short form for service accounts in this project - iam = { - for k, v in lookup(each.value, "iam", {}) : k => [ - for vv in v : lookup( - var.factories_config.context.iam_principals, vv, vv - ) - ] - } - iam_bindings = { - for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { - members = [ - for vv in v.members : lookup( - var.factories_config.context.iam_principals, vv, vv - ) - ] - }) - } - iam_bindings_additive = { - for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { - member = lookup( - var.factories_config.context.iam_principals, v.member, v.member - ) - }) - } + context = merge(local.ctx, { + project_ids = local.ctx_project_ids + iam_principals = merge( + local.ctx.iam_principals, + local.projects_sas_iam_emails + ) + }) + iam = lookup(each.value, "iam", {}) + iam_bindings = lookup(each.value, "iam_bindings", {}) + iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) iam_billing_roles = lookup(each.value, "iam_billing_roles", {}) iam_folder_roles = lookup(each.value, "iam_folder_roles", {}) iam_organization_roles = lookup(each.value, "iam_organization_roles", {}) diff --git a/modules/project-factory-experimental/budgets.tf b/modules/project-factory/budgets.tf similarity index 100% rename from modules/project-factory-experimental/budgets.tf rename to modules/project-factory/budgets.tf diff --git a/modules/project-factory/folders.tf b/modules/project-factory/folders.tf index 6f4710853..8d57ab85b 100644 --- a/modules/project-factory/folders.tf +++ b/modules/project-factory/folders.tf @@ -16,143 +16,145 @@ # tfdoc:file:description Folder hierarchy factory resources. +# TODO: folder automation + locals { - folder_parent_default = try( - var.factories_config.context.folder_ids.default, null + _folders_path = try( + pathexpand(var.factories_config.folders), null ) -} - -module "hierarchy-folder-lvl-1" { - source = "../folder" - for_each = { for k, v in local.folders : k => v if v.level == 1 } - parent = try( - # allow the YAML data to set the parent for this level - lookup( - var.factories_config.context.folder_ids, - each.value.parent, - each.value.parent - ), - # use the default value in the initial parents map - local.folder_parent_default - # fail if we don't have an explicit parent + _folders_files = try( + fileset(local._folders_path, "**/**/.config.yaml"), + [] ) - name = each.value.name - iam = { - for k, v in lookup(each.value, "iam", {}) : k => [ - # don't interpolate automation service account to prevent cycles - for vv in v : lookup( - var.factories_config.context.iam_principals, vv, vv - ) - ] - } - iam_bindings = { - for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { - members = [ - # don't interpolate automation service account to prevent cycles - for vv in v.members : lookup( - var.factories_config.context.iam_principals, vv, vv - ) - ] + _folders_raw = merge( + var.folders, + { + for f in local._folders_files : dirname(f) => yamldecode(file( + "${coalesce(local._folders_path, "-")}/${f}" + )) + } + ) + ctx_folder_ids = merge(local.ctx.folder_ids, local.folder_ids) + folder_ids = merge( + { for k, v in module.folder-1 : k => v.id }, + { for k, v in module.folder-2 : k => v.id }, + { for k, v in module.folder-3 : k => v.id } + ) + folders_input = { + for key, data in local._folders_raw : key => merge(data, { + key = key + level = length(split("/", key)) + parent_key = dirname(key) + # do not enforce overrides / defaults on folders + parent = lookup(data, "parent", null) }) } - iam_bindings_additive = { - for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { - # don't interpolate automation service account to prevent cycles - member = lookup( - var.factories_config.context.iam_principals, v.member, v.member - ) - }) - } - iam_by_principals = { - for k, v in lookup(each.value, "iam_by_principals", {}) : - lookup( - var.factories_config.context.iam_principals, k, k - ) => v - } - org_policies = lookup(each.value, "org_policies", {}) - tag_bindings = { - for k, v in lookup(each.value, "tag_bindings", {}) : - k => lookup(var.factories_config.context.tag_values, v, v) - } - logging_data_access = lookup(each.value, "logging_data_access", {}) } -module "hierarchy-folder-lvl-2" { - source = "../folder" - for_each = { for k, v in local.folders : k => v if v.level == 2 } - parent = module.hierarchy-folder-lvl-1[each.value.parent_key].id - name = each.value.name - iam = { - for k, v in lookup(each.value, "iam", {}) : k => [ - # don't interpolate automation service account to prevent cycles - for vv in v : lookup( - var.factories_config.context.iam_principals, vv, vv - ) - ] - } - iam_bindings = { - for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { - members = [ - # don't interpolate automation service account to prevent cycles - for vv in v.members : lookup( - var.factories_config.context.iam_principals, vv, vv - ) - ] - }) - } - iam_bindings_additive = { - for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { - # don't interpolate automation service account to prevent cycles - member = lookup( - var.factories_config.context.iam_principals, v.member, v.member - ) - }) - } - iam_by_principals = lookup(each.value, "iam_by_principals", {}) - org_policies = lookup(each.value, "org_policies", {}) - tag_bindings = { - for k, v in lookup(each.value, "tag_bindings", {}) : - k => lookup(var.factories_config.context.tag_values, v, v) +module "folder-1" { + source = "../folder" + for_each = { + for k, v in local.folders_input : k => v if v.level == 1 } + parent = coalesce(each.value.parent, "$folder_ids:default") + name = each.value.name + org_policies = lookup(each.value, "org_policies", {}) + tag_bindings = lookup(each.value, "tag_bindings", {}) logging_data_access = lookup(each.value, "logging_data_access", {}) + context = local.ctx } -module "hierarchy-folder-lvl-3" { - source = "../folder" - for_each = { for k, v in local.folders : k => v if v.level == 3 } - parent = module.hierarchy-folder-lvl-2[each.value.parent_key].id - name = each.value.name - iam = { - for k, v in lookup(each.value, "iam", {}) : k => [ - # don't interpolate automation service account to prevent cycles - for vv in v : lookup( - var.factories_config.context.iam_principals, vv, vv - ) - ] +module "folder-1-iam" { + source = "../folder" + for_each = { + for k, v in local.folders_input : k => v if v.level == 1 } - iam_bindings = { - for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { - members = [ - # don't interpolate automation service account to prevent cycles - for vv in v.members : lookup( - var.factories_config.context.iam_principals, vv, vv - ) - ] - }) - } - iam_bindings_additive = { - for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { - # don't interpolate automation service account to prevent cycles - member = lookup( - var.factories_config.context.iam_principals, v.member, v.member - ) - }) - } - iam_by_principals = lookup(each.value, "iam_by_principals", {}) - org_policies = lookup(each.value, "org_policies", {}) - tag_bindings = { - for k, v in lookup(each.value, "tag_bindings", {}) : - k => lookup(var.factories_config.context.tag_values, v, v) - } - logging_data_access = lookup(each.value, "logging_data_access", {}) + id = module.folder-1[each.key].id + folder_create = false + iam = lookup(each.value, "iam", {}) + iam_bindings = lookup(each.value, "iam_bindings", {}) + iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) + iam_by_principals = lookup(each.value, "iam_by_principals", {}) + context = merge(local.ctx, { + iam_principals = local.ctx_iam_principals + }) } + +module "folder-2" { + source = "../folder" + for_each = { + for k, v in local.folders_input : k => v if v.level == 2 + } + parent = coalesce( + each.value.parent, "$folder_ids:${each.value.parent_key}" + ) + name = each.value.name + org_policies = lookup(each.value, "org_policies", {}) + tag_bindings = lookup(each.value, "tag_bindings", {}) + logging_data_access = lookup(each.value, "logging_data_access", {}) + context = merge(local.ctx, { + folder_ids = merge(local.ctx.folder_ids, { + for k, v in module.folder-1 : k => v.id + }) + }) + depends_on = [module.folder-1] +} + +module "folder-2-iam" { + source = "../folder" + for_each = { + for k, v in local.folders_input : k => v if v.level == 2 + } + id = module.folder-2[each.key].id + folder_create = false + iam = lookup(each.value, "iam", {}) + iam_bindings = lookup(each.value, "iam_bindings", {}) + iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) + iam_by_principals = lookup(each.value, "iam_by_principals", {}) + context = merge(local.ctx, { + folder_ids = merge(local.ctx.folder_ids, { + for k, v in module.folder-1 : k => v.id + }) + iam_principals = local.ctx_iam_principals + }) +} + +module "folder-3" { + source = "../folder" + for_each = { + for k, v in local.folders_input : k => v if v.level == 3 + } + parent = coalesce( + each.value.parent, "$folder_ids:${each.value.parent_key}" + ) + name = each.value.name + org_policies = lookup(each.value, "org_policies", {}) + tag_bindings = lookup(each.value, "tag_bindings", {}) + logging_data_access = lookup(each.value, "logging_data_access", {}) + context = merge(local.ctx, { + folder_ids = merge(local.ctx.folder_ids, { + for k, v in module.folder-2 : k => v.id + }) + }) + depends_on = [module.folder-2] +} + +module "folder-3-iam" { + source = "../folder" + for_each = { + for k, v in local.folders_input : k => v if v.level == 3 + } + id = module.folder-3[each.key].id + folder_create = false + iam = lookup(each.value, "iam", {}) + iam_bindings = lookup(each.value, "iam_bindings", {}) + iam_bindings_additive = lookup(each.value, "iam_bindings_additive", {}) + iam_by_principals = lookup(each.value, "iam_by_principals", {}) + context = merge(local.ctx, { + folder_ids = merge(local.ctx.folder_ids, { + for k, v in module.folder-2 : k => v.id + }) + iam_principals = local.ctx_iam_principals + }) +} + diff --git a/modules/project-factory-experimental/ludo.tfvars b/modules/project-factory/ludo.tfvars similarity index 100% rename from modules/project-factory-experimental/ludo.tfvars rename to modules/project-factory/ludo.tfvars diff --git a/modules/project-factory/main.tf b/modules/project-factory/main.tf index 29d15f22c..ea74e1d63 100644 --- a/modules/project-factory/main.tf +++ b/modules/project-factory/main.tf @@ -17,474 +17,25 @@ # tfdoc:file:description Projects and billing budgets factory resources. locals { - _service_agent_emails = flatten([ - for k, v in module.projects : [ - for kk, vv in v.service_agents : { - key = "${k}/${kk}" - value = "serviceAccount:${vv.email}" - } - ] - ]) - context = { - folder_ids = merge( - var.factories_config.context.folder_ids, - local.hierarchy - ) - iam_principals = merge( - var.factories_config.context.iam_principals, - { - for k, v in module.automation-service-accounts : - k => v.iam_email - }, - # module.service-accounts are excluded here, as adding them here results in dependency cycles - ) - } - service_accounts_names = { - for k, v in module.service-accounts : k => v.name - } - service_agents_email = { - for v in local._service_agent_emails : v.key => v.value - } + ctx = var.context + ctx_iam_principals = merge( + local.ctx.iam_principals, + local.iam_principals + ) + iam_principals = merge( + local.projects_sas_iam_emails, + local.automation_sas_iam_emails + ) } -module "projects" { - source = "../project" - for_each = local.projects - billing_account = each.value.billing_account - deletion_policy = each.value.deletion_policy - name = each.value.name - parent = lookup( - local.context.folder_ids, each.value.parent, each.value.parent - ) - prefix = each.value.prefix - project_reuse = each.value.project_reuse - alerts = try(each.value.alerts, null) - auto_create_network = try(each.value.auto_create_network, false) - compute_metadata = try(each.value.compute_metadata, {}) - # TODO: concat lists for each key - contacts = merge( - each.value.contacts, var.data_merges.contacts - ) - default_service_account = try(each.value.default_service_account, "keep") - descriptive_name = try(each.value.descriptive_name, null) - factories_config = { - custom_roles = each.value.factories_config.custom_roles - observability = each.value.factories_config.observability - org_policies = each.value.factories_config.org_policies - quotas = each.value.factories_config.quotas - context = { - notification_channels = var.factories_config.context.notification_channels +resource "terraform_data" "defaults_preconditions" { + lifecycle { + precondition { + condition = ( + var.data_defaults.storage_location != null || + var.data_overrides.storage_location != null + ) + error_message = "No default storage location defined in defaults or overides variables." } } - labels = merge( - each.value.labels, var.data_merges.labels - ) - lien_reason = try(each.value.lien_reason, null) - log_scopes = try(each.value.log_scopes, null) - logging_data_access = try(each.value.logging_data_access, {}) - logging_exclusions = try(each.value.logging_exclusions, {}) - logging_metrics = try(each.value.logging_metrics, null) - logging_sinks = try(each.value.logging_sinks, {}) - metric_scopes = distinct(concat( - each.value.metric_scopes, var.data_merges.metric_scopes - )) - notification_channels = try(each.value.notification_channels, null) - org_policies = each.value.org_policies - service_encryption_key_ids = { - for k, v in merge( - each.value.service_encryption_key_ids, - var.data_merges.service_encryption_key_ids - ) : k => [ - for key in v : lookup(var.factories_config.context.kms_keys, key, key) - ] - } - services = distinct(concat( - each.value.services, - var.data_merges.services - )) - shared_vpc_host_config = each.value.shared_vpc_host_config - tag_bindings = { - for k, v in merge(each.value.tag_bindings, var.data_merges.tag_bindings) : - k => lookup(var.factories_config.context.tag_values, v, v) - } - tags = each.value.tags - vpc_sc = each.value.vpc_sc == null ? null : { - perimeter_name = ( - each.value.vpc_sc.perimeter_name == null - ? null - : lookup( - var.factories_config.context.perimeters, - each.value.vpc_sc.perimeter_name, - each.value.vpc_sc.perimeter_name - ) - ) - is_dry_run = each.value.vpc_sc.is_dry_run - } - quotas = each.value.quotas -} - -module "projects-iam" { - source = "../project" - for_each = local.projects - name = module.projects[each.key].project_id - project_reuse = { - use_data_source = false - attributes = { - name = module.projects[each.key].name - number = module.projects[each.key].number - services_enabled = module.projects[each.key].services - } - } - iam = { - for k, v in lookup(each.value, "iam", {}) : - lookup(var.factories_config.context.custom_roles, k, k) => [ - for vv in v : try( - # project service accounts (sa) - module.service-accounts["${each.key}/${vv}"].iam_email, - # automation service account (rw) - local.context.iam_principals["${each.key}/automation/${vv}"], - # automation service account (automation/rw) - local.context.iam_principals["${each.key}/${vv}"], - # other projects service accounts (project/sa) - module.service-accounts[vv].iam_email, - # other automation service account (project/automation/rw) - local.context.iam_principals[vv], - # project's service identities - local.service_agents_email["${each.key}/${vv}"], - local.service_agents_email[vv], - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(vv, ":") - ? templatestring( - vv, { project_number = module.projects[each.key].number } - ) - : tonumber("[Error] Invalid member: '${vv}' in project '${each.key}'") - ) - ) - ] - } - iam_bindings = { - for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, { - members = [ - for vv in v.members : try( - # project service accounts (sa) - module.service-accounts["${each.key}/${vv}"].iam_email, - # automation service account (rw) - local.context.iam_principals["${each.key}/automation/${vv}"], - # automation service account (automation/rw) - local.context.iam_principals["${each.key}/${vv}"], - # other projects service accounts (project/sa) - module.service-accounts[vv].iam_email, - # other automation service account (project/automation/rw) - local.context.iam_principals[vv], - # project's service identities - local.service_agents_email["${each.key}/${vv}"], - local.service_agents_email[vv], - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(vv, ":") - ? templatestring( - vv, { project_number = module.projects[each.key].number } - ) - : tonumber("[Error] Invalid member: '${vv}' in project '${each.key}'") - ) - ) - ] - role = lookup(var.factories_config.context.custom_roles, v.role, v.role) - }) - } - iam_bindings_additive = { - for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, { - member = try( - # project service accounts (sa) - module.service-accounts["${each.key}/${v.member}"].iam_email, - # automation service account (rw) - local.context.iam_principals["${each.key}/automation/${v.member}"], - # automation service account (automation/rw) - local.context.iam_principals["${each.key}/${v.member}"], - # other projects service accounts (project/sa) - module.service-accounts[v.member].iam_email, - # other automation service account (project/automation/rw) - local.context.iam_principals[v.member], - # project's service identities - local.service_agents_email["${each.key}/${v.member}"], - local.service_agents_email[v.member], - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(v.member, ":") - ? templatestring( - v.member, { project_number = module.projects[each.key].number } - ) - : tonumber("[Error] Invalid member: '${v.member}' in project '${each.key}'") - ) - ) - role = lookup(var.factories_config.context.custom_roles, v.role, v.role) - }) - } - # IAM by principals would trigger dynamic key errors so we don't interpolate - # iam_by_principals = try(each.value.iam_by_principals, {}) - iam_by_principals = { - for k, v in try(each.value.iam_by_principals, {}) : - try( - # project service accounts (sa) - module.service-accounts["${each.key}/${k}"].iam_email, - # automation service account (rw) - local.context.iam_principals["${each.key}/automation/${k}"], - # automation service account (automation/rw) - local.context.iam_principals["${each.key}/${k}"], - # other projects service accounts (project/sa) - module.service-accounts[k].iam_email, - # other automation service account (project/automation/rw) - local.context.iam_principals[k], - # project's service identities - local.service_agents_email["${each.key}/${k}"], - local.service_agents_email[k], - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(k, ":") - ? templatestring( - k, { project_number = module.projects[each.key].number } - ) - : tonumber("[Error] Invalid member: '${k}' in project '${each.key}'") - ) - ) => [ - for vv in v : lookup(var.factories_config.context.custom_roles, vv, vv) - ] - } - # Shared VPC configuration is done at stage 2, to avoid dependency cycle between project service accounts and - # IAM grants done for those service accounts - shared_vpc_service_config = ( - try(each.value.shared_vpc_service_config.host_project, null) == null - ? null - : merge(each.value.shared_vpc_service_config, { - host_project = try( - var.factories_config.context.vpc_host_projects[each.value.shared_vpc_service_config.host_project], - module.projects[each.value.shared_vpc_service_config.host_project].project_id, - each.value.shared_vpc_service_config.host_project - ) - iam_bindings_additive = { - for k, v in try(each.value.shared_vpc_service_config.iam_bindings_additive, {}) : k => merge(v, { - member = try( - # project service accounts (sa) - module.service-accounts["${each.key}/${v.member}"].iam_email, - # automation service account (rw) - local.context.iam_principals["${each.key}/automation/${v.member}"], - # automation service account (automation/rw) - local.context.iam_principals["${each.key}/${v.member}"], - # other projects service accounts (project/sa) - module.service-accounts[v.member].iam_email, - # other automation service account (project/automation/rw) - local.context.iam_principals[v.member], - # project's service identities - local.service_agents_email["${each.key}/${v.member}"], - local.service_agents_email[v.member], - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(v.member, ":") - ? templatestring( - v.member, { project_number = module.projects[each.key].number } - ) - : tonumber("[Error] Invalid member: '${v.member}' in project '${each.key}'") - ) - ) - role = lookup(var.factories_config.context.custom_roles, v.role, v.role) - }) - } - network_users = [ - for vv in try(each.value.shared_vpc_service_config.network_users, []) : - try( - # project service accounts (sa) - module.service-accounts["${each.key}/${vv}"].iam_email, - # automation service account (rw) - local.context.iam_principals["${each.key}/automation/${vv}"], - # automation service account (automation/rw) - local.context.iam_principals["${each.key}/${vv}"], - # other projects service accounts (project/sa) - module.service-accounts[vv].iam_email, - # other automation service account (project/automation/rw) - local.context.iam_principals[vv], - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(vv, ":") - ? templatestring( - vv, { project_number = module.projects[each.key].number } - ) - : tonumber("[Error] Invalid member: '${vv}' in project '${each.key}'") - ) - ) - ] - }) - ) - # add service agents config, so Service Agents can be referred in the IAM grants - service_agents_config = { - # default roles are granted in module.project - grant_default_roles = false - } -} - -module "buckets" { - source = "../gcs" - for_each = { - for k in local.buckets : "${k.project_key}/${k.name}" => k - } - project_id = module.projects[each.value.project_key].project_id - prefix = each.value.prefix - name = "${each.value.project_name}-${each.value.name}" - encryption_key = each.value.encryption_key - force_destroy = each.value.force_destroy - iam = { - for k, v in each.value.iam : k => [ - for vv in v : try( - # project service accounts (sa) - module.service-accounts["${each.value.project_key}/${vv}"].iam_email, - # automation service account (rw) - local.context.iam_principals["${each.value.project_key}/automation/${vv}"], - # automation service account (automation/rw) - local.context.iam_principals["${each.value.project_key}/${vv}"], - # other projects service accounts (project/sa) - module.service-accounts[vv].iam_email, - # other automation service account (project/automation/rw) - local.context.iam_principals[vv], - # project's service identities - local.service_agents_email["${each.value.project_key}/${vv}"], - local.service_agents_email[vv], - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(vv, ":") - ? templatestring( - vv, { project_number = module.projects[each.value.project_key].number } - ) - : tonumber("[Error] Invalid member: '${vv}' in bucket '${each.key}'") - ) - ) - ] - } - iam_bindings = { - for k, v in each.value.iam_bindings : k => merge(v, { - members = [ - for vv in v.members : try( - # project service accounts (sa) - module.service-accounts["${each.value.project_key}/${vv}"].iam_email, - # automation service account (rw) - local.context.iam_principals["${each.value.project_key}/automation/${vv}"], - # automation service account (automation/rw) - local.context.iam_principals["${each.value.project_key}/${vv}"], - # other projects service accounts (project/sa) - module.service-accounts[vv].iam_email, - # other automation service account (project/automation/rw) - local.context.iam_principals[vv], - # project's service identities - local.service_agents_email["${each.value.project_key}/${vv}"], - local.service_agents_email[vv], - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(vv, ":") - ? templatestring( - vv, { project_number = module.projects[each.value.project_key].number } - ) - : tonumber("[Error] Invalid member: '${vv}' in bucket '${each.key}'") - ) - ) - ] - }) - } - iam_bindings_additive = { - for k, v in each.value.iam_bindings_additive : k => merge(v, { - member = try( - # project service accounts (sa) - module.service-accounts["${each.value.project_key}/${v.member}"].iam_email, - # automation service account (rw) - local.context.iam_principals["${each.value.project_key}/automation/${v.member}"], - # automation service account (automation/rw) - local.context.iam_principals["${each.value.project_key}/${v.member}"], - # other projects service accounts (project/sa) - module.service-accounts[v.member].iam_email, - # other automation service account (project/automation/rw) - local.context.iam_principals[v.member], - # project's service identities - local.service_agents_email["${each.value.project_key}/${v.member}"], - local.service_agents_email[v.member], - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(v.member, ":") - ? templatestring( - v.member, { project_number = module.projects[each.value.project_key].number } - ) - : tonumber("[Error] Invalid member: '${v.member}' in bucket '${each.key}'") - ) - ) - }) - } - labels = each.value.labels - location = coalesce( - var.data_overrides.storage_location, - lookup(each.value, "location", null), - var.data_defaults.storage_location - ) - storage_class = each.value.storage_class - uniform_bucket_level_access = each.value.uniform_bucket_level_access - versioning = each.value.versioning -} - -module "service-accounts" { - source = "../iam-service-account" - for_each = { - for k in local.service_accounts : "${k.project_key}/${k.name}" => k - } - project_id = module.projects[each.value.project_key].project_id - name = each.value.name - display_name = each.value.display_name - iam = { - for k, v in lookup(each.value, "iam", {}) : k => [ - for vv in v : try( - # automation service account (rw) - local.context.iam_principals["${each.value.project_key}/automation/${vv}"], - # automation service account (automation/rw) - local.context.iam_principals["${each.value.project_key}/${vv}"], - # other automation service account (project/automation/rw) - local.context.iam_principals[vv], - # passthrough + error handling using tonumber until Terraform gets fail/raise function - ( - strcontains(vv, ":") - ? vv - : tonumber("[Error] Invalid member: '${vv}' in project '${each.value.project_key}'") - ) - ) - ] - } - iam_project_roles = merge( - { - for k, v in each.value.iam_project_roles : - lookup(var.factories_config.context.vpc_host_projects, k, k) => v - }, - each.value.iam_self_roles == null ? {} : { - (module.projects[each.value.project_key].project_id) = each.value.iam_self_roles - } - ) -} - -module "service_accounts-iam" { - source = "../iam-service-account" - for_each = { - for k in local.service_accounts : "${k.project_key}/${k.name}" => k - if k.iam_sa_roles != {} - } - project_id = module.service-accounts[each.key].service_account.project - name = each.value.name - service_account_create = false - iam_sa_roles = { - for k, v in each.value.iam_sa_roles : lookup( - local.service_accounts_names, "${each.value.project_key}/${k}", k - ) => v - } -} - -module "billing-account" { - source = "../billing-account" - count = var.factories_config.budgets == null ? 0 : 1 - id = var.factories_config.budgets.billing_account - budget_notification_channels = ( - var.factories_config.budgets.notification_channels - ) - budgets = local.budgets } diff --git a/modules/project-factory/outputs.tf b/modules/project-factory/outputs.tf index 770b89518..a30e92a29 100644 --- a/modules/project-factory/outputs.tf +++ b/modules/project-factory/outputs.tf @@ -1,5 +1,5 @@ /** - * Copyright 2024 Google LLC + * Copyright 2025 Google LLC * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -14,45 +14,77 @@ * limitations under the License. */ -output "buckets" { - description = "Bucket names." - value = { - for k, v in module.buckets : k => v.name - } -} - -output "folders" { - description = "Folder ids." - value = local.hierarchy -} - -output "projects" { - description = "Created projects." - value = { - for k, v in module.projects : k => { - number = v.number - project_id = v.id - project = v - automation = ( - lookup(local.projects[k], "automation", null) == null - ? null - : { - bucket = try(module.automation-bucket[k].name, null) - service_accounts = { - for kk, vv in module.automation-service-accounts : - trimprefix(kk, "${k}/") => vv.email - if startswith(kk, "${k}/") - } - } - ) - service_agents = { - for k, v in v.service_agents : k => v.email if v.is_primary +locals { + outputs_projects = { + for k, v in local.projects_input : k => { + number = module.projects[k].number + project_id = module.projects[k].project_id + log_buckets = { + for sk, sv in lookup(v, "log_buckets", {}) : + "${k}/${sk}" => ( + module.log-buckets["${k}/${sk}"].id + ) + } + service_accounts = { + for sk, sv in lookup(v, "service_accounts", {}) : + "${k}/${sk}" => ( + module.service-accounts["${k}/${sk}"].email + ) + } + storage_buckets = { + for sk, sv in lookup(v, "buckets", {}) : + "${k}/${sk}" => ( + module.buckets["${k}/${sk}"].name + ) } } } } +output "folder_ids" { + description = "Folder ids." + value = local.folder_ids +} + +output "iam_principals" { + description = "IAM principals mappings." + value = local.iam_principals +} + +output "log_buckets" { + description = "Log bucket ids." + value = merge([ + for k, v in local.outputs_projects : v.log_buckets + ]...) +} + +output "project_ids" { + description = "Project ids." + value = local.project_ids +} + +output "project_numbers" { + description = "Project numbers." + value = { + for k, v in local.outputs_projects : k => v.number + } +} + +output "projects" { + description = "Project attributes." + value = local.outputs_projects +} + output "service_accounts" { description = "Service account emails." - value = module.service-accounts + value = merge([ + for k, v in local.outputs_projects : v.service_accounts + ]...) +} + +output "storage_buckets" { + description = "Bucket names." + value = merge([ + for k, v in local.outputs_projects : v.storage_buckets + ]...) } diff --git a/modules/project-factory-experimental/projects-buckets.tf b/modules/project-factory/projects-buckets.tf similarity index 100% rename from modules/project-factory-experimental/projects-buckets.tf rename to modules/project-factory/projects-buckets.tf diff --git a/modules/project-factory-experimental/projects-defaults.tf b/modules/project-factory/projects-defaults.tf similarity index 100% rename from modules/project-factory-experimental/projects-defaults.tf rename to modules/project-factory/projects-defaults.tf diff --git a/modules/project-factory-experimental/projects-log-buckets.tf b/modules/project-factory/projects-log-buckets.tf similarity index 100% rename from modules/project-factory-experimental/projects-log-buckets.tf rename to modules/project-factory/projects-log-buckets.tf diff --git a/modules/project-factory-experimental/projects-service-accounts.tf b/modules/project-factory/projects-service-accounts.tf similarity index 100% rename from modules/project-factory-experimental/projects-service-accounts.tf rename to modules/project-factory/projects-service-accounts.tf diff --git a/modules/project-factory-experimental/projects.tf b/modules/project-factory/projects.tf similarity index 100% rename from modules/project-factory-experimental/projects.tf rename to modules/project-factory/projects.tf diff --git a/modules/project-factory-experimental/sample-data-1/folders/applications/.config.yaml b/modules/project-factory/sample-data-1/folders/applications/.config.yaml similarity index 100% rename from modules/project-factory-experimental/sample-data-1/folders/applications/.config.yaml rename to modules/project-factory/sample-data-1/folders/applications/.config.yaml diff --git a/modules/project-factory-experimental/sample-data-1/folders/applications/dev/.config.yaml b/modules/project-factory/sample-data-1/folders/applications/dev/.config.yaml similarity index 100% rename from modules/project-factory-experimental/sample-data-1/folders/applications/dev/.config.yaml rename to modules/project-factory/sample-data-1/folders/applications/dev/.config.yaml diff --git a/modules/project-factory-experimental/sample-data-1/folders/applications/prod/.config.yaml b/modules/project-factory/sample-data-1/folders/applications/prod/.config.yaml similarity index 100% rename from modules/project-factory-experimental/sample-data-1/folders/applications/prod/.config.yaml rename to modules/project-factory/sample-data-1/folders/applications/prod/.config.yaml diff --git a/modules/project-factory-experimental/sample-data-1/folders/networking/.config.yaml b/modules/project-factory/sample-data-1/folders/networking/.config.yaml similarity index 100% rename from modules/project-factory-experimental/sample-data-1/folders/networking/.config.yaml rename to modules/project-factory/sample-data-1/folders/networking/.config.yaml diff --git a/modules/project-factory-experimental/sample-data-1/folders/networking/dev-net-spoke-0.yaml b/modules/project-factory/sample-data-1/folders/networking/dev-net-spoke-0.yaml similarity index 100% rename from modules/project-factory-experimental/sample-data-1/folders/networking/dev-net-spoke-0.yaml rename to modules/project-factory/sample-data-1/folders/networking/dev-net-spoke-0.yaml diff --git a/modules/project-factory-experimental/sample-data-1/folders/networking/prod-net-spoke-0.yaml b/modules/project-factory/sample-data-1/folders/networking/prod-net-spoke-0.yaml similarity index 100% rename from modules/project-factory-experimental/sample-data-1/folders/networking/prod-net-spoke-0.yaml rename to modules/project-factory/sample-data-1/folders/networking/prod-net-spoke-0.yaml diff --git a/modules/project-factory-experimental/sample-data-1/folders/prod-iac-core-0.yaml b/modules/project-factory/sample-data-1/folders/prod-iac-core-0.yaml similarity index 100% rename from modules/project-factory-experimental/sample-data-1/folders/prod-iac-core-0.yaml rename to modules/project-factory/sample-data-1/folders/prod-iac-core-0.yaml diff --git a/modules/project-factory-experimental/sample-data-1/projects/app-0/dev-app0-be-0.yaml b/modules/project-factory/sample-data-1/projects/app-0/dev-app0-be-0.yaml similarity index 100% rename from modules/project-factory-experimental/sample-data-1/projects/app-0/dev-app0-be-0.yaml rename to modules/project-factory/sample-data-1/projects/app-0/dev-app0-be-0.yaml diff --git a/modules/project-factory-experimental/sample-data-1/projects/app-0/prod-app0-be-0.yaml b/modules/project-factory/sample-data-1/projects/app-0/prod-app0-be-0.yaml similarity index 100% rename from modules/project-factory-experimental/sample-data-1/projects/app-0/prod-app0-be-0.yaml rename to modules/project-factory/sample-data-1/projects/app-0/prod-app0-be-0.yaml diff --git a/modules/project-factory/schemas/folder.schema.json b/modules/project-factory/schemas/folder.schema.json index 1e87c94c6..1125a8a97 100644 --- a/modules/project-factory/schemas/folder.schema.json +++ b/modules/project-factory/schemas/folder.schema.json @@ -4,6 +4,66 @@ "type": "object", "additionalProperties": false, "properties": { + "automation": { + "type": "object", + "additionalProperties": false, + "required": [ + "project" + ], + "properties": { + "prefix": { + "type": "string" + }, + "project": { + "type": "string" + }, + "bucket": { + "$ref": "#/$defs/bucket" + }, + "service_accounts": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9-]+$": { + "type": "object", + "additionalProperties": false, + "properties": { + "description": { + "type": "string" + }, + "iam": { + "$ref": "#/$defs/iam" + }, + "iam_bindings": { + "$ref": "#/$defs/iam_bindings" + }, + "iam_bindings_additive": { + "$ref": "#/$defs/iam_bindings_additive" + }, + "iam_billing_roles": { + "$ref": "#/$defs/iam_billing_roles" + }, + "iam_folder_roles": { + "$ref": "#/$defs/iam_folder_roles" + }, + "iam_organization_roles": { + "$ref": "#/$defs/iam_organization_roles" + }, + "iam_project_roles": { + "$ref": "#/$defs/iam_project_roles" + }, + "iam_sa_roles": { + "$ref": "#/$defs/iam_sa_roles" + }, + "iam_storage_roles": { + "$ref": "#/$defs/iam_storage_roles" + } + } + } + } + } + } + }, "iam": { "$ref": "#/$defs/iam" }, @@ -110,15 +170,84 @@ } }, "$defs": { + "bucket": { + "type": "object", + "additionalProperties": false, + "properties": { + "name": { + "type": "string" + }, + "description": { + "type": "string" + }, + "iam": { + "$ref": "#/$defs/iam" + }, + "iam_bindings": { + "$ref": "#/$defs/iam_bindings" + }, + "iam_bindings_additive": { + "$ref": "#/$defs/iam_bindings_additive" + }, + "force_destroy": { + "type": "boolean" + }, + "labels": { + "type": "object", + "additionalProperties": { + "type": "string" + } + }, + "location": { + "type": "string" + }, + "managed_folders": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-zA-Z0-9][a-zA-Z0-9_/-]+$": { + "type": "object", + "additionalProperties": false, + "properties": { + "force_destroy": { + "type": "boolean" + }, + "iam": { + "$ref": "#/$defs/iam" + }, + "iam_bindings": { + "$ref": "#/$defs/iam_bindings" + }, + "iam_bindings_additive": { + "$ref": "#/$defs/iam_bindings_additive" + } + } + } + } + }, + "prefix": { + "type": "string" + }, + "storage_class": { + "type": "string" + }, + "uniform_bucket_level_access": { + "type": "boolean" + }, + "versioning": { + "type": "boolean" + } + } + }, "iam": { "type": "object", "additionalProperties": false, "patternProperties": { - "^roles/": { + "^(?:roles/|\\$custom_roles:)": { "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:)" } } } @@ -135,7 +264,7 @@ "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:)" } }, "role": { @@ -175,7 +304,7 @@ "properties": { "member": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:)" }, "role": { "type": "string", @@ -208,11 +337,83 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])": { + "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:)": { "type": "array", "items": { "type": "string", - "pattern": "^roles/" + "pattern": "^(?:roles/|\\$custom_roles:)" + } + } + } + }, + "iam_billing_roles": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9-]+$": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "iam_folder_roles": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9-]+$": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "iam_organization_roles": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9-]+$": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "iam_project_roles": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9-]+$": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "iam_sa_roles": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9-]+$": { + "type": "array", + "items": { + "type": "string" + } + } + } + }, + "iam_storage_roles": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9-]+$": { + "type": "array", + "items": { + "type": "string" } } } diff --git a/modules/project-factory/schemas/folder.schema.md b/modules/project-factory/schemas/folder.schema.md index 7ea4d8c0c..4c5fac144 100644 --- a/modules/project-factory/schemas/folder.schema.md +++ b/modules/project-factory/schemas/folder.schema.md @@ -6,6 +6,25 @@ *additional properties: false* +- **automation**: *object* +
*additional properties: false* + - **prefix**: *string* + - ⁺**project**: *string* + - **bucket**: *reference([bucket](#refs-bucket))* + - **service_accounts**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *object* +
*additional properties: false* + - **description**: *string* + - **iam**: *reference([iam](#refs-iam))* + - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* + - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* + - **iam_billing_roles**: *reference([iam_billing_roles](#refs-iam_billing_roles))* + - **iam_folder_roles**: *reference([iam_folder_roles](#refs-iam_folder_roles))* + - **iam_organization_roles**: *reference([iam_organization_roles](#refs-iam_organization_roles))* + - **iam_project_roles**: *reference([iam_project_roles](#refs-iam_project_roles))* + - **iam_sa_roles**: *reference([iam_sa_roles](#refs-iam_sa_roles))* + - **iam_storage_roles**: *reference([iam_storage_roles](#refs-iam_storage_roles))* - **iam**: *reference([iam](#refs-iam))* - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* @@ -37,24 +56,48 @@ - **location**: *string* - **title**: *string* - **parent**: *string* +
*pattern: ^(?:folders/[0-9]+|organizations/[0-9]+|\$folder_ids:[a-z0-9_-]+)$* - **tag_bindings**: *object*
*additional properties: false* - **`^[a-z0-9_-]+$`**: *string* ## Definitions +- **bucket**: *object* +
*additional properties: false* + - **name**: *string* + - **description**: *string* + - **iam**: *reference([iam](#refs-iam))* + - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* + - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* + - **force_destroy**: *boolean* + - **labels**: *object* + *additional properties: String* + - **location**: *string* + - **managed_folders**: *object* +
*additional properties: false* + - **`^[a-zA-Z0-9][a-zA-Z0-9_/-]+$`**: *object* +
*additional properties: false* + - **force_destroy**: *boolean* + - **iam**: *reference([iam](#refs-iam))* + - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* + - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* + - **prefix**: *string* + - **storage_class**: *string* + - **uniform_bucket_level_access**: *boolean* + - **versioning**: *boolean* - **iam**: *object*
*additional properties: false* - - **`^roles/`**: *array* + - **`^(?:roles/|\$custom_roles:)`**: *array* - items: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:)* - **iam_bindings**: *object*
*additional properties: false* - **`^[a-z0-9_-]+$`**: *object*
*additional properties: false* - **members**: *array* - items: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:)* - **role**: *string*
*pattern: ^roles/* - **condition**: *object* @@ -67,7 +110,7 @@ - **`^[a-z0-9_-]+$`**: *object*
*additional properties: false* - **member**: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:)* - **role**: *string*
*pattern: ^roles/* - **condition**: *object* @@ -77,6 +120,30 @@ - **description**: *string* - **iam_by_principals**: *object*
*additional properties: false* - - **`^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])`**: *array* + - **`^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:)`**: *array* + - items: *string* +
*pattern: ^(?:roles/|\$custom_roles:)* +- **iam_billing_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* + - items: *string* +- **iam_folder_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* + - items: *string* +- **iam_organization_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* + - items: *string* +- **iam_project_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* + - items: *string* +- **iam_sa_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* + - items: *string* +- **iam_storage_roles**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *array* - items: *string* -
*pattern: ^roles/* diff --git a/modules/project-factory/schemas/project.schema.json b/modules/project-factory/schemas/project.schema.json index 8f255e909..626dfcc2a 100644 --- a/modules/project-factory/schemas/project.schema.json +++ b/modules/project-factory/schemas/project.schema.json @@ -111,6 +111,15 @@ "labels": { "type": "object" }, + "log_buckets": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-z0-9-]+$": { + "$ref": "#/$defs/log_bucket" + } + } + }, "metric_scopes": { "type": "array", "items": { @@ -473,6 +482,27 @@ } } }, + "universe": { + "type": "object", + "additionalProperties": false, + "properties": { + "prefix": { + "type": "string", + "unavailable_services": { + "type": "array", + "items": { + "type": "string" + } + }, + "unavailable_service_identities": { + "type": "array", + "items": { + "type": "string" + } + } + } + } + }, "vpc_sc": { "type": "object", "additionalItems": false, @@ -494,6 +524,9 @@ "type": "object", "additionalProperties": false, "properties": { + "name": { + "type": "string" + }, "description": { "type": "string" }, @@ -518,6 +551,30 @@ "location": { "type": "string" }, + "managed_folders": { + "type": "object", + "additionalProperties": false, + "patternProperties": { + "^[a-zA-Z0-9][a-zA-Z0-9_/-]+$": { + "type": "object", + "additionalProperties": false, + "properties": { + "force_destroy": { + "type": "boolean" + }, + "iam": { + "$ref": "#/$defs/iam" + }, + "iam_bindings": { + "$ref": "#/$defs/iam_bindings" + }, + "iam_bindings_additive": { + "$ref": "#/$defs/iam_bindings_additive" + } + } + } + } + }, "prefix": { "type": "string" }, @@ -545,11 +602,11 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^roles/": { + "^(?:roles/|\\$custom_roles:)": { "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:||\\$iam_principals:[a-z0-9_-]+)" } } } @@ -566,12 +623,12 @@ "type": "array", "items": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:[a-z0-9_-]+)" } }, "role": { "type": "string", - "pattern": "^roles/" + "pattern": "^(?:roles/|\\$custom_roles:)" }, "condition": { "type": "object", @@ -606,11 +663,11 @@ "properties": { "member": { "type": "string", - "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])" + "pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:[a-z0-9_-]+)" }, "role": { "type": "string", - "pattern": "^[a-zA-Z0-9_/.]+$" + "pattern": "^(?:roles/|\\$custom_roles:)" }, "condition": { "type": "object", @@ -639,11 +696,11 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])": { + "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\\$iam_principals:[a-z0-9_-]+)": { "type": "array", "items": { "type": "string", - "pattern": "^roles/" + "pattern": "^(?:roles/|\\$custom_roles:)" } } } @@ -688,7 +745,7 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^[a-z0-9-]+$": { + "^(?:[a-z0-9-]|\\$project_ids:[a-z0-9_-])+$": { "type": "array", "items": { "type": "string" @@ -700,7 +757,7 @@ "type": "object", "additionalProperties": false, "patternProperties": { - "^[a-z0-9-]+$": { + "^(?:\\$service_account_ids:|projects/)": { "type": "array", "items": { "type": "string" @@ -719,6 +776,40 @@ } } } + }, + "log_bucket": { + "type": "object", + "additionalProperties": false, + "properties": { + "description": { + "type": "string" + }, + "kms_key_name": { + "type": "string" + }, + "location": { + "type": "string" + }, + "log_analytics": { + "type": "object", + "additionalProperties": false, + "properties": { + "enable": { + "type": "boolean", + "default": false + }, + "dataset_link_id": { + "type": "string" + }, + "description": { + "type": "string" + } + } + }, + "retention": { + "type": "number" + } + } } } -} +} \ No newline at end of file diff --git a/modules/project-factory/schemas/project.schema.md b/modules/project-factory/schemas/project.schema.md index 8ff5b23c6..900523134 100644 --- a/modules/project-factory/schemas/project.schema.md +++ b/modules/project-factory/schemas/project.schema.md @@ -40,6 +40,9 @@ - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* - **iam_by_principals**: *reference([iam_by_principals](#refs-iam_by_principals))* - **labels**: *object* +- **log_buckets**: *object* +
*additional properties: false* + - **`^[a-z0-9-]+$`**: *reference([log_bucket](#refs-log_bucket))* - **metric_scopes**: *array* - items: *string* - **name**: *string* @@ -137,6 +140,9 @@ - **`^[a-z0-9_-]+$`**: *string* - **tags**: *object* *additional properties: Object* +- **universe**: *object* +
*additional properties: false* + - **prefix**: *string* - **vpc_sc**: *object* - ⁺**perimeter_name**: *string* - **is_dry_run**: *boolean* @@ -145,6 +151,7 @@ - **bucket**: *object*
*additional properties: false* + - **name**: *string* - **description**: *string* - **iam**: *reference([iam](#refs-iam))* - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* @@ -153,6 +160,14 @@ - **labels**: *object* *additional properties: String* - **location**: *string* + - **managed_folders**: *object* +
*additional properties: false* + - **`^[a-zA-Z0-9][a-zA-Z0-9_/-]+$`**: *object* +
*additional properties: false* + - **force_destroy**: *boolean* + - **iam**: *reference([iam](#refs-iam))* + - **iam_bindings**: *reference([iam_bindings](#refs-iam_bindings))* + - **iam_bindings_additive**: *reference([iam_bindings_additive](#refs-iam_bindings_additive))* - **prefix**: *string* - **storage_class**: *string* - **uniform_bucket_level_access**: *boolean* @@ -162,18 +177,18 @@ - **`^[a-z0-9-]+$`**: *reference([bucket](#refs-bucket))* - **iam**: *object*
*additional properties: false* - - **`^roles/`**: *array* + - **`^(?:roles/|\$custom_roles:)`**: *array* - items: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:||\$iam_principals:[a-z0-9_-]+)* - **iam_bindings**: *object*
*additional properties: false* - **`^[a-z0-9_-]+$`**: *object*
*additional properties: false* - **members**: *array* - items: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:[a-z0-9_-]+)* - **role**: *string* -
*pattern: ^roles/* +
*pattern: ^(?:roles/|\$custom_roles:)* - **condition**: *object*
*additional properties: false* - ⁺**expression**: *string* @@ -184,9 +199,9 @@ - **`^[a-z0-9_-]+$`**: *object*
*additional properties: false* - **member**: *string* -
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])* +
*pattern: ^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:[a-z0-9_-]+)* - **role**: *string* -
*pattern: ^[a-zA-Z0-9_/.]+$* +
*pattern: ^(?:roles/|\$custom_roles:)* - **condition**: *object*
*additional properties: false* - ⁺**expression**: *string* @@ -194,9 +209,9 @@ - **description**: *string* - **iam_by_principals**: *object*
*additional properties: false* - - **`^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])`**: *array* + - **`^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|\$iam_principals:[a-z0-9_-]+)`**: *array* - items: *string* -
*pattern: ^roles/* +
*pattern: ^(?:roles/|\$custom_roles:)* - **iam_billing_roles**: *object*
*additional properties: false* - **`^[a-z0-9-]+$`**: *array* @@ -211,13 +226,24 @@ - items: *string* - **iam_project_roles**: *object*
*additional properties: false* - - **`^[a-z0-9-]+$`**: *array* + - **`^(?:[a-z0-9-]|\$project_ids:[a-z0-9_-])+$`**: *array* - items: *string* - **iam_sa_roles**: *object*
*additional properties: false* - - **`^[a-z0-9-]+$`**: *array* + - **`^(?:\$service_account_ids:|projects/)`**: *array* - items: *string* - **iam_storage_roles**: *object*
*additional properties: false* - **`^[a-z0-9-]+$`**: *array* - items: *string* +- **log_bucket**: *object* +
*additional properties: false* + - **description**: *string* + - **kms_key_name**: *string* + - **location**: *string* + - **log_analytics**: *object* +
*additional properties: false* + - **enable**: *boolean* + - **dataset_link_id**: *string* + - **description**: *string* + - **retention**: *number* diff --git a/modules/project-factory-experimental/variables-billing.tf b/modules/project-factory/variables-billing.tf similarity index 100% rename from modules/project-factory-experimental/variables-billing.tf rename to modules/project-factory/variables-billing.tf diff --git a/modules/project-factory-experimental/variables-folders.tf b/modules/project-factory/variables-folders.tf similarity index 100% rename from modules/project-factory-experimental/variables-folders.tf rename to modules/project-factory/variables-folders.tf diff --git a/modules/project-factory-experimental/variables-projects.tf b/modules/project-factory/variables-projects.tf similarity index 100% rename from modules/project-factory-experimental/variables-projects.tf rename to modules/project-factory/variables-projects.tf diff --git a/modules/project-factory/variables.tf b/modules/project-factory/variables.tf index 5dbc10c6f..176d33492 100644 --- a/modules/project-factory/variables.tf +++ b/modules/project-factory/variables.tf @@ -14,8 +14,27 @@ * limitations under the License. */ +variable "context" { + description = "Context-specific interpolations." + type = object({ + condition_vars = optional(map(map(string)), {}) + custom_roles = optional(map(string), {}) + folder_ids = optional(map(string), {}) + iam_principals = optional(map(string), {}) + kms_keys = optional(map(string), {}) + locations = optional(map(string), {}) + notification_channels = optional(map(string), {}) + project_ids = optional(map(string), {}) + tag_values = optional(map(string), {}) + vpc_host_projects = optional(map(string), {}) + vpc_sc_perimeters = optional(map(string), {}) + }) + default = {} + nullable = false +} + variable "data_defaults" { - description = "Optional default values used when corresponding project data from files are missing." + description = "Optional default values used when corresponding project or folder data from files are missing." type = object({ billing_account = optional(string) bucket = optional(object({ @@ -67,6 +86,11 @@ variable "data_defaults" { display_name = optional(string, "Terraform-managed.") iam_self_roles = optional(list(string)) })), {}) + universe = optional(object({ + prefix = string + unavailable_service_identities = optional(list(string), []) + unavailable_services = optional(list(string), []) + })) vpc_sc = optional(object({ perimeter_name = string is_dry_run = optional(bool, false) @@ -127,6 +151,11 @@ variable "data_overrides" { display_name = optional(string, "Terraform-managed.") iam_self_roles = optional(list(string)) }))) + universe = optional(object({ + prefix = string + unavailable_service_identities = optional(list(string), []) + unavailable_services = optional(list(string), []) + })) vpc_sc = optional(object({ perimeter_name = string is_dry_run = optional(bool, false) @@ -144,272 +173,12 @@ variable "data_overrides" { variable "factories_config" { description = "Path to folder with YAML resource description data files." type = object({ - folders_data_path = optional(string) - projects_data_path = optional(string) + folders = optional(string) + projects = optional(string) budgets = optional(object({ - billing_account = string - budgets_data_path = string - # TODO: allow defining notification channels via YAML files - notification_channels = optional(map(any), {}) + billing_account_id = string + data = string })) - context = optional(object({ - custom_roles = optional(map(string), {}) - folder_ids = optional(map(string), {}) - iam_principals = optional(map(string), {}) - kms_keys = optional(map(string), {}) - perimeters = optional(map(string), {}) - tag_values = optional(map(string), {}) - vpc_host_projects = optional(map(string), {}) - notification_channels = optional(map(string), {}) - }), {}) - projects_config = optional(object({ - key_ignores_path = optional(bool, false) - }), {}) }) nullable = false } - -variable "factories_data" { - description = "Alternate factory data input allowing to use this module as a library. Merged with local YAML data." - type = object({ - budgets = optional(map(object({ - amount = object({ - currency_code = optional(string) - nanos = optional(number) - units = optional(number) - use_last_period = optional(bool) - }) - display_name = optional(string) - filter = optional(object({ - credit_types_treatment = optional(object({ - exclude_all = optional(bool) - include_specified = optional(list(string)) - })) - label = optional(object({ - key = string - value = string - })) - period = optional(object({ - calendar = optional(string) - custom = optional(object({ - start_date = object({ - day = number - month = number - year = number - }) - end_date = optional(object({ - day = number - month = number - year = number - })) - })) - })) - projects = optional(list(string)) - resource_ancestors = optional(list(string)) - services = optional(list(string)) - subaccounts = optional(list(string)) - })) - threshold_rules = optional(list(object({ - percent = number - forecasted_spend = optional(bool) - })), []) - update_rules = optional(map(object({ - disable_default_iam_recipients = optional(bool) - monitoring_notification_channels = optional(list(string)) - pubsub_topic = optional(string) - })), {}) - })), {}) - hierarchy = optional(map(object({ - name = optional(string) - parent = optional(string) - iam = optional(map(list(string)), {}) - iam_bindings = optional(map(object({ - members = list(string) - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - iam_bindings_additive = optional(map(object({ - member = string - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - iam_by_principals = optional(map(list(string)), {}) - tag_bindings = optional(map(string), {}) - })), {}) - projects = optional(map(object({ - automation = optional(object({ - project = string - bucket = optional(object({ - location = string - description = optional(string) - force_destroy = optional(bool) - prefix = optional(string) - storage_class = optional(string, "STANDARD") - uniform_bucket_level_access = optional(bool, true) - versioning = optional(bool) - iam = optional(map(list(string)), {}) - iam_bindings = optional(map(object({ - members = list(string) - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - iam_bindings_additive = optional(map(object({ - member = string - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - labels = optional(map(string), {}) - })) - service_accounts = optional(map(object({ - description = optional(string) - iam = optional(map(list(string)), {}) - iam_bindings = optional(map(object({ - members = list(string) - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - iam_bindings_additive = optional(map(object({ - member = string - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - iam_billing_roles = optional(map(list(string)), {}) - iam_folder_roles = optional(map(list(string)), {}) - iam_organization_roles = optional(map(list(string)), {}) - iam_project_roles = optional(map(list(string)), {}) - iam_sa_roles = optional(map(list(string)), {}) - iam_storage_roles = optional(map(list(string)), {}) - })), {}) - })) - billing_account = optional(string) - billing_budgets = optional(list(string), []) - buckets = optional(map(object({ - location = string - description = optional(string) - force_destroy = optional(bool) - prefix = optional(string) - storage_class = optional(string, "STANDARD") - uniform_bucket_level_access = optional(bool, true) - versioning = optional(bool) - iam = optional(map(list(string)), {}) - iam_bindings = optional(map(object({ - members = list(string) - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - iam_bindings_additive = optional(map(object({ - member = string - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - labels = optional(map(string), {}) - })), {}) - contacts = optional(map(list(string)), {}) - iam = optional(map(list(string)), {}) - iam_bindings = optional(map(object({ - members = list(string) - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - iam_bindings_additive = optional(map(object({ - member = string - role = string - condition = optional(object({ - expression = string - title = string - description = optional(string) - })) - })), {}) - iam_by_principals = optional(map(list(string)), {}) - labels = optional(map(string), {}) - metric_scopes = optional(list(string), []) - name = optional(string) - org_policies = optional(map(object({ - inherit_from_parent = optional(bool) # for list policies only. - reset = optional(bool) - rules = optional(list(object({ - allow = optional(object({ - all = optional(bool) - values = optional(list(string)) - })) - deny = optional(object({ - all = optional(bool) - values = optional(list(string)) - })) - enforce = optional(bool) # for boolean policies only. - condition = optional(object({ - description = optional(string) - expression = optional(string) - location = optional(string) - title = optional(string) - }), {}) - parameters = optional(string) - })), []) - })), {}) - parent = optional(string) - prefix = optional(string) - service_accounts = optional(map(object({ - display_name = optional(string) - iam_self_roles = optional(list(string), []) - iam_project_roles = optional(map(list(string)), {}) - })), {}) - service_encryption_key_ids = optional(map(list(string)), {}) - services = optional(list(string), []) - shared_vpc_host_config = optional(object({ - enabled = bool - service_projects = optional(list(string), []) - })) - shared_vpc_service_config = optional(object({ - host_project = string - network_users = optional(list(string), []) - service_agent_iam = optional(map(list(string)), {}) - service_agent_subnet_iam = optional(map(list(string)), {}) - service_iam_grants = optional(list(string), []) - network_subnet_users = optional(map(list(string)), {}) - })) - tag_bindings = optional(map(string), {}) - vpc_sc = optional(object({ - perimeter_name = string - is_dry_run = optional(bool, false) - })) - })), {}) - }) - nullable = false - default = {} -} diff --git a/modules/project/README.md b/modules/project/README.md index 6e4884997..412fa86c4 100644 --- a/modules/project/README.md +++ b/modules/project/README.md @@ -623,7 +623,7 @@ module "project" { org_policies = "configs/org-policies/" } context = { - org_policies = { + condition_vars = { tags = { my_conditional_tag = "tagKeys/1234" } @@ -1806,7 +1806,7 @@ alerts: | [billing_account](variables.tf#L23) | Billing account id. | string | | null | | [compute_metadata](variables.tf#L29) | Optional compute metadata key/values. Only usable if compute API has been enabled. | map(string) | | {} | | [contacts](variables.tf#L36) | List of essential contacts for this resource. Must be in the form EMAIL -> [NOTIFICATION_TYPES]. Valid notification types are ALL, SUSPENSION, SECURITY, TECHNICAL, BILLING, LEGAL, PRODUCT_UPDATES. | map(list(string)) | | {} | -| [context](variables.tf#L43) | Context-specific interpolations. | object({…}) | | {} | +| [context](variables.tf#L43) | Context-specific interpolations. | object({…}) | | {} | | [custom_roles](variables.tf#L61) | Map of role name => list of permissions to create in this project. | map(list(string)) | | {} | | [default_network_tier](variables.tf#L68) | Default compute network tier for the project. | string | | null | | [default_service_account](variables.tf#L74) | Project default service account setting: can be one of `delete`, `deprivilege`, `disable`, or `keep`. | string | | "keep" | diff --git a/modules/project/iam.tf b/modules/project/iam.tf index b36e442ab..d82d91b71 100644 --- a/modules/project/iam.tf +++ b/modules/project/iam.tf @@ -131,7 +131,9 @@ resource "google_project_iam_binding" "bindings" { dynamic "condition" { for_each = each.value.condition == null ? [] : [""] content { - expression = each.value.condition.expression + expression = templatestring( + each.value.condition.expression, var.context.condition_vars + ) title = each.value.condition.title description = each.value.condition.description } @@ -150,7 +152,9 @@ resource "google_project_iam_member" "bindings" { dynamic "condition" { for_each = each.value.condition == null ? [] : [""] content { - expression = each.value.condition.expression + expression = templatestring( + each.value.condition.expression, var.context.condition_vars + ) title = each.value.condition.title description = each.value.condition.description } diff --git a/modules/project/main.tf b/modules/project/main.tf index 35d127b3f..42845fd44 100644 --- a/modules/project/main.tf +++ b/modules/project/main.tf @@ -22,7 +22,7 @@ locals { ctx = { for k, v in var.context : k => { for kk, vv in v : "${local.ctx_p}${k}:${kk}" => vv - } + } if k != "condition_vars" } ctx_p = "$" descriptive_name = ( diff --git a/modules/project/organization-policies.tf b/modules/project/organization-policies.tf index 4138f83ce..ac688bed3 100644 --- a/modules/project/organization-policies.tf +++ b/modules/project/organization-policies.tf @@ -34,7 +34,7 @@ locals { all = try(r.allow.all, null) values = ( can(r.allow.values) - ? [for x in r.allow.values : templatestring(x, var.context.org_policies)] + ? [for x in r.allow.values : templatestring(x, var.context.condition_vars)] : null ) } : null @@ -42,7 +42,7 @@ locals { all = try(r.deny.all, null) values = ( can(r.deny.values) - ? [for x in r.deny.values : templatestring(x, var.context.org_policies)] + ? [for x in r.deny.values : templatestring(x, var.context.condition_vars)] : null ) } : null @@ -50,28 +50,28 @@ locals { condition = { description = ( can(r.condition.description) - ? templatestring(r.condition.description, var.context.org_policies) + ? templatestring(r.condition.description, var.context.condition_vars) : null ) expression = ( can(r.condition.expression) - ? templatestring(r.condition.expression, var.context.org_policies) + ? templatestring(r.condition.expression, var.context.condition_vars) : null ) location = ( can(r.condition.location) - ? templatestring(r.condition.location, var.context.org_policies) + ? templatestring(r.condition.location, var.context.condition_vars) : null ) title = ( can(r.condition.title) - ? templatestring(r.condition.title, var.context.org_policies) + ? templatestring(r.condition.title, var.context.condition_vars) : null ) } parameters = ( can(r.parameters) - ? templatestring(r.parameters, var.context.org_policies) + ? templatestring(r.parameters, var.context.condition_vars) : null ) } diff --git a/modules/project/variables.tf b/modules/project/variables.tf index df325b6d0..75fdfa304 100644 --- a/modules/project/variables.tf +++ b/modules/project/variables.tf @@ -43,12 +43,12 @@ variable "contacts" { variable "context" { description = "Context-specific interpolations." type = object({ + condition_vars = optional(map(map(string)), {}) custom_roles = optional(map(string), {}) folder_ids = optional(map(string), {}) kms_keys = optional(map(string), {}) iam_principals = optional(map(string), {}) notification_channels = optional(map(string), {}) - org_policies = optional(map(map(string)), {}) project_ids = optional(map(string), {}) tag_keys = optional(map(string), {}) tag_values = optional(map(string), {}) diff --git a/tests/fast/addons/a1_resman_tenants/simple.tfvars b/tests/fast/addons/a1_resman_tenants/simple.tfvars deleted file mode 100644 index 4ccc1e002..000000000 --- a/tests/fast/addons/a1_resman_tenants/simple.tfvars +++ /dev/null @@ -1,138 +0,0 @@ -automation = { - cicd_backends = null - federated_identity_pool = null - federated_identity_providers = null - project_id = "fast-prod-automation" - project_number = 123456 - outputs_bucket = "test" - service_accounts = { - resman = "ldj-prod-resman-0@fast2-prod-iac-core-0.iam.gserviceaccount.com" - resman-r = "ldj-prod-resman-0r@fast2-prod-iac-core-0.iam.gserviceaccount.com" - } -} -billing_account = { - id = "000000-111111-222222" -} -custom_roles = { - # organization_iam_admin = "organizations/123456789012/roles/organizationIamAdmin", - billing_viewer = "organizations/123456789012/roles/billingViewer" - dns_zone_binder = "organizations/123456789012/roles/dnsZoneBinder" - gcve_network_admin = "organizations/123456789012/roles/gcveNetworkAdmin" - gcve_network_viewer = "organizations/123456789012/roles/gcveNetworkViewer" - kms_key_encryption_admin = "organizations/123456789012/roles/kmsKeyEncryptionAdmin" - kms_key_viewer = "organizations/123456789012/roles/kmsKeyViewer" - network_firewall_policies_admin = "organizations/123456789012/roles/networkFirewallPoliciesAdmin" - ngfw_enterprise_admin = "organizations/123456789012/roles/ngfwEnterpriseAdmin" - ngfw_enterprise_viewer = "organizations/123456789012/roles/ngfwEnterpriseViewer" - organization_admin_viewer = "organizations/123456789012/roles/organizationAdminViewer" - project_iam_viewer = "organizations/123456789012/roles/projectIamViewer" - service_project_network_admin = "organizations/123456789012/roles/xpnServiceAdmin" - storage_viewer = "organizations/123456789012/roles/storageViewer" - tenant_network_admin = "organizations/123456789012/roles/tenantNetworkAdmin" -} -environments = { - dev = { - is_default = false - name = "Development" - short_name = "dev" - tag_name = "development" - } - prod = { - is_default = true - name = "Production" - short_name = "prod" - tag_name = "production" - } -} -groups = { - gcp-billing-admins = "gcp-billing-admins", - gcp-devops = "gcp-devops", - gcp-network-admins = "gcp-vpc-network-admins", - gcp-organization-admins = "gcp-organization-admins", - gcp-security-admins = "gcp-security-admins", - gcp-support = "gcp-support" -} -logging = { - project_id = "fast-prod-log-audit-0" -} -organization = { - domain = "fast.example.com" - id = 123456789012 - customer_id = "C00000000" -} -org_policy_tags = { - key_id = "tagKeys/281480694641817" - key_name = "org-policies" - values = { - "allowed-policy-member-domains-all" = "tagValues/281480211229353" - "compute-require-oslogin-false" = "tagValues/281476830880807" - } -} -prefix = "fast2" -root_node = "folders/1234567890" -tenant_configs = { - s0 = { - admin_principal = "group:admins@example0.org" - descriptive_name = "Simple 0" - } - s1 = { - admin_principal = "group:admins@example1.org" - billing_account = { - # implicit no-iam - id = "102345-102345-102345" - } - descriptive_name = "Simple 1" - cloud_identity = { - customer_id = "ABCDEFGH" - domain = "example1.org" - id = 1234567890 - } - vpc_sc_policy_create = true - } - f0 = { - admin_principal = "group:gcp-organization-admins@fast-0.example.org" - billing_account = { - # implicit use of org-level BA with IAM roles - no_iam = false - } - descriptive_name = "Fast 0" - cloud_identity = { - domain = "fast-0.example.org" - id = 12345678 - customer_id = "C0C0C0C0" - } - fast_config = { - groups = { - gcp-network-admins = "gcp-network-admins" - } - cicd_config = { - identity_provider = "github" - name = "fast-0/resman" - type = "github" - branch = "main" - } - workload_identity_providers = { - github = { - attribute_condition = "attribute.repository_owner==\"fast-0\"" - issuer = "github" - } - } - } - vpc_sc_policy_create = true - } - f1 = { - admin_principal = "group:gcp-organization-admins@fast-1.example.org" - # implicit use of org-level BA without IAM roles - descriptive_name = "Fast 1" - cloud_identity = { - domain = "fast-1.example.org" - id = 1234567 - customer_id = "D0D0D0D0" - } - fast_config = { - groups = { - gcp-network-admins = "gcp-network-admins" - } - } - } -} diff --git a/tests/fast/addons/a1_resman_tenants/simple.yaml b/tests/fast/addons/a1_resman_tenants/simple.yaml deleted file mode 100644 index 2696be7f4..000000000 --- a/tests/fast/addons/a1_resman_tenants/simple.yaml +++ /dev/null @@ -1,46 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -counts: - google_access_context_manager_access_policy: 2 - google_access_context_manager_access_policy_iam_member: 5 - google_bigquery_default_service_account: 4 - google_essential_contacts_contact: 2 - google_folder: 8 - google_folder_iam_binding: 34 - google_iam_workload_identity_pool: 1 - google_iam_workload_identity_pool_provider: 1 - google_logging_folder_sink: 4 - google_logging_project_bucket_config: 4 - google_org_policy_policy: 6 - google_organization_iam_member: 6 - google_project: 4 - google_project_iam_audit_config: 2 - google_project_iam_binding: 32 - google_project_iam_member: 36 - google_project_service: 56 - google_project_service_identity: 12 - google_service_account: 16 - google_service_account_iam_binding: 6 - google_service_account_iam_member: 2 - google_storage_bucket: 8 - google_storage_bucket_iam_binding: 8 - google_storage_bucket_iam_member: 6 - google_storage_bucket_object: 17 - google_storage_project_service_account: 4 - google_tags_tag_binding: 4 - google_tags_tag_key: 1 - google_tags_tag_value: 4 - modules: 50 - resources: 295 diff --git a/tests/fast/addons/a1_resman_tenants/tftest.yaml b/tests/fast/addons/a1_resman_tenants/tftest.yaml deleted file mode 100644 index e59f77706..000000000 --- a/tests/fast/addons/a1_resman_tenants/tftest.yaml +++ /dev/null @@ -1,18 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -module: fast/addons/1-resman-tenants - -tests: - simple: diff --git a/tests/fast/stages/s0_bootstrap_experimental/data-simple/cicd.yaml b/tests/fast/stages/s0_bootstrap/data-simple/cicd.yaml similarity index 100% rename from tests/fast/stages/s0_bootstrap_experimental/data-simple/cicd.yaml rename to tests/fast/stages/s0_bootstrap/data-simple/cicd.yaml diff --git a/tests/fast/stages/s0_bootstrap_experimental/data-simple/defaults.yaml b/tests/fast/stages/s0_bootstrap/data-simple/defaults.yaml similarity index 100% rename from tests/fast/stages/s0_bootstrap_experimental/data-simple/defaults.yaml rename to tests/fast/stages/s0_bootstrap/data-simple/defaults.yaml diff --git a/tests/fast/stages/s0_bootstrap_experimental/not-simple.tfvars b/tests/fast/stages/s0_bootstrap/not-simple.tfvars similarity index 100% rename from tests/fast/stages/s0_bootstrap_experimental/not-simple.tfvars rename to tests/fast/stages/s0_bootstrap/not-simple.tfvars diff --git a/tests/fast/stages/s0_bootstrap_experimental/not-simple.yaml b/tests/fast/stages/s0_bootstrap/not-simple.yaml similarity index 94% rename from tests/fast/stages/s0_bootstrap_experimental/not-simple.yaml rename to tests/fast/stages/s0_bootstrap/not-simple.yaml index 7e04865ac..7cea8c719 100644 --- a/tests/fast/stages/s0_bootstrap_experimental/not-simple.yaml +++ b/tests/fast/stages/s0_bootstrap/not-simple.yaml @@ -543,6 +543,7 @@ values: condition: [] members: - serviceAccount:iac-bootstrap-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com + - serviceAccount:iac-dp-dev-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com - serviceAccount:iac-networking-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com - serviceAccount:iac-pf-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com - serviceAccount:iac-security-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com @@ -553,6 +554,7 @@ values: condition: [] members: - serviceAccount:iac-bootstrap-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + - serviceAccount:iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com - serviceAccount:iac-networking-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com - serviceAccount:iac-pf-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com - serviceAccount:iac-security-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com @@ -605,6 +607,11 @@ values: force_destroy: false name: 2-security/ timeouts: null + module.factory.module.buckets["iac-0/iac-stage-state"].google_storage_managed_folder.folder["3-data-platform-dev/"]: + bucket: ft0-prod-iac-core-0-iac-stage-state + force_destroy: false + name: 3-data-platform-dev/ + timeouts: null ? module.factory.module.buckets["iac-0/iac-stage-state"].google_storage_managed_folder_iam_binding.authoritative["1-vpcsc/$custom_roles:storage_viewer"] : bucket: ft0-prod-iac-core-0-iac-stage-state condition: [] @@ -661,6 +668,20 @@ values: members: - serviceAccount:iac-security-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com role: roles/storage.admin + ? module.factory.module.buckets["iac-0/iac-stage-state"].google_storage_managed_folder_iam_binding.authoritative["3-data-platform-dev/$custom_roles:storage_viewer"] + : bucket: ft0-prod-iac-core-0-iac-stage-state + condition: [] + managed_folder: 3-data-platform-dev/ + members: + - serviceAccount:iac-dp-dev-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: organizations/1234567890/roles/storageViewer + ? module.factory.module.buckets["iac-0/iac-stage-state"].google_storage_managed_folder_iam_binding.authoritative["3-data-platform-dev/roles/storage.admin"] + : bucket: ft0-prod-iac-core-0-iac-stage-state + condition: [] + managed_folder: 3-data-platform-dev/ + members: + - serviceAccount:iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: roles/storage.admin ? module.factory.module.folder-1-iam["networking"].google_folder_iam_binding.authoritative["$custom_roles:service_project_network_admin"] : condition: [] members: @@ -716,6 +737,26 @@ values: members: - serviceAccount:iac-networking-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com role: roles/viewer + module.factory.module.folder-1-iam["networking"].google_folder_iam_binding.bindings["dp_dev_ro"]: + condition: + - description: null + expression: 'resource.matchTag(''1234567890/environment'', ''development'') + + ' + title: Data platform dev network viewer. + members: + - serviceAccount:iac-dp-dev-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: roles/compute.networkViewer + module.factory.module.folder-1-iam["networking"].google_folder_iam_binding.bindings["dp_dev_rw"]: + condition: + - description: null + expression: 'resource.matchTag(''1234567890/environment'', ''development'') + + ' + title: Data platform dev service project admin. + members: + - serviceAccount:iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: organizations/1234567890/roles/serviceProjectNetworkAdmin module.factory.module.folder-1-iam["security"].google_folder_iam_binding.authoritative["roles/logging.admin"]: condition: [] members: @@ -796,6 +837,12 @@ values: members: - serviceAccount:iac-pf-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com role: roles/viewer + module.factory.module.folder-1["data-platform"].google_folder.folder[0]: + deletion_protection: false + display_name: Data Platform + parent: organizations/1234567890 + tags: null + timeouts: null module.factory.module.folder-1["networking"].google_folder.folder[0]: deletion_protection: false display_name: Networking @@ -816,6 +863,69 @@ values: timeouts: null module.factory.module.folder-1["teams"].google_tags_tag_binding.binding["context"]: timeouts: null + module.factory.module.folder-2-iam["data-platform/dev"].google_folder_iam_binding.authoritative["roles/compute.xpnAdmin"]: + condition: [] + members: + - serviceAccount:iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: roles/compute.xpnAdmin + module.factory.module.folder-2-iam["data-platform/dev"].google_folder_iam_binding.authoritative["roles/logging.admin"]: + condition: [] + members: + - serviceAccount:iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: roles/logging.admin + module.factory.module.folder-2-iam["data-platform/dev"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: roles/owner + ? module.factory.module.folder-2-iam["data-platform/dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderAdmin"] + : condition: [] + members: + - serviceAccount:iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: roles/resourcemanager.folderAdmin + ? module.factory.module.folder-2-iam["data-platform/dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderViewer"] + : condition: [] + members: + - serviceAccount:iac-dp-dev-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: roles/resourcemanager.folderViewer + ? module.factory.module.folder-2-iam["data-platform/dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.projectCreator"] + : condition: [] + members: + - serviceAccount:iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: roles/resourcemanager.projectCreator + module.factory.module.folder-2-iam["data-platform/dev"].google_folder_iam_binding.authoritative["roles/viewer"]: + condition: [] + members: + - serviceAccount:iac-dp-dev-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: roles/viewer + ? module.factory.module.folder-2-iam["networking/dev"].google_folder_iam_binding.authoritative["$custom_roles:project_iam_viewer"] + : condition: [] + members: + - serviceAccount:iac-dp-dev-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: organizations/1234567890/roles/projectIamViewer + module.factory.module.folder-2-iam["networking/dev"].google_folder_iam_binding.bindings["dp_dev"]: + condition: + - description: null + expression: "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([\n\ + \ 'organizations/1234567890/roles/serviceProjectNetworkAdmin'\n])\n" + title: Data platform dev delegated IAM grant. + members: + - serviceAccount:iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + role: roles/resourcemanager.projectIamAdmin + module.factory.module.folder-2["data-platform/dev"].google_folder.folder[0]: + deletion_protection: false + display_name: Development + tags: null + timeouts: null + module.factory.module.folder-2["data-platform/dev"].google_tags_tag_binding.binding["environment"]: + timeouts: null + module.factory.module.folder-2["data-platform/prod"].google_folder.folder[0]: + deletion_protection: false + display_name: Production + tags: null + timeouts: null + module.factory.module.folder-2["data-platform/prod"].google_tags_tag_binding.binding["environment"]: + timeouts: null module.factory.module.folder-2["networking/dev"].google_folder.folder[0]: deletion_protection: false display_name: Development @@ -1296,6 +1406,26 @@ values: member: serviceAccount:iac-bootstrap-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com project: ft0-prod-iac-core-0 timeouts: null + module.factory.module.service-accounts["iac-0/iac-dp-dev-ro"].google_service_account.service_account[0]: + account_id: iac-dp-dev-ro + create_ignore_already_exists: null + description: null + disabled: false + display_name: IaC service account for data platform dev (read-only). + email: iac-dp-dev-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com + member: serviceAccount:iac-dp-dev-ro@ft0-prod-iac-core-0.iam.gserviceaccount.com + project: ft0-prod-iac-core-0 + timeouts: null + module.factory.module.service-accounts["iac-0/iac-dp-dev-rw"].google_service_account.service_account[0]: + account_id: iac-dp-dev-rw + create_ignore_already_exists: null + description: null + disabled: false + display_name: IaC service account for data platform dev (read-write). + email: iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + member: serviceAccount:iac-dp-dev-rw@ft0-prod-iac-core-0.iam.gserviceaccount.com + project: ft0-prod-iac-core-0 + timeouts: null module.factory.module.service-accounts["iac-0/iac-networking-ro"].google_service_account.service_account[0]: account_id: iac-networking-ro create_ignore_already_exists: null @@ -2488,8 +2618,8 @@ values: counts: google_bigquery_default_service_account: 1 google_billing_account_iam_member: 5 - google_folder: 5 - google_folder_iam_binding: 27 + google_folder: 8 + google_folder_iam_binding: 38 google_iam_workload_identity_pool: 1 google_iam_workload_identity_pool_provider: 1 google_logging_organization_settings: 1 @@ -2504,19 +2634,19 @@ counts: google_project_iam_member: 14 google_project_service: 30 google_project_service_identity: 8 - google_service_account: 14 + google_service_account: 16 google_service_account_iam_member: 2 google_storage_bucket: 3 google_storage_bucket_iam_binding: 4 google_storage_bucket_object: 9 - google_storage_managed_folder: 4 - google_storage_managed_folder_iam_binding: 8 + google_storage_managed_folder: 5 + google_storage_managed_folder_iam_binding: 10 google_storage_project_service_account: 2 - google_tags_tag_binding: 3 + google_tags_tag_binding: 5 google_tags_tag_key: 3 google_tags_tag_value: 5 google_tags_tag_value_iam_binding: 4 local_file: 9 - modules: 36 - resources: 265 + modules: 43 + resources: 286 terraform_data: 2 diff --git a/tests/fast/stages/s0_bootstrap/tftest.yaml b/tests/fast/stages/s0_bootstrap/tftest.yaml index 4aea722b0..a1bc8f50e 100644 --- a/tests/fast/stages/s0_bootstrap/tftest.yaml +++ b/tests/fast/stages/s0_bootstrap/tftest.yaml @@ -15,16 +15,9 @@ module: fast/stages/0-bootstrap tests: - simple: + # TODO: rename to simple once fast lint setup accepts extra dirs + not-simple: inventory: - - simple.yaml - - simple_org_policies.yaml - managed_org_policies: - inventory: - - simple.yaml - - managed_org_policies.yaml - external_billing_account: - inventory: - - external_billing_account.yaml - iam_by_principals: - cicd: + - not-simple.yaml + extra_dirs: + - ../../../tests/fast/stages/s0_bootstrap/data-simple diff --git a/tests/fast/addons/a1_resman_tenants/__init__.py b/tests/fast/stages/s0_bootstrap_legacy/__init__.py similarity index 100% rename from tests/fast/addons/a1_resman_tenants/__init__.py rename to tests/fast/stages/s0_bootstrap_legacy/__init__.py diff --git a/tests/fast/stages/s0_bootstrap/cicd.tfvars b/tests/fast/stages/s0_bootstrap_legacy/cicd.tfvars similarity index 100% rename from tests/fast/stages/s0_bootstrap/cicd.tfvars rename to tests/fast/stages/s0_bootstrap_legacy/cicd.tfvars diff --git a/tests/fast/stages/s0_bootstrap/cicd.yaml b/tests/fast/stages/s0_bootstrap_legacy/cicd.yaml similarity index 100% rename from tests/fast/stages/s0_bootstrap/cicd.yaml rename to tests/fast/stages/s0_bootstrap_legacy/cicd.yaml diff --git a/tests/fast/stages/s0_bootstrap/data/checklist-data.json b/tests/fast/stages/s0_bootstrap_legacy/data/checklist-data.json similarity index 100% rename from tests/fast/stages/s0_bootstrap/data/checklist-data.json rename to tests/fast/stages/s0_bootstrap_legacy/data/checklist-data.json diff --git a/tests/fast/stages/s0_bootstrap/data/checklist-org-iam.json b/tests/fast/stages/s0_bootstrap_legacy/data/checklist-org-iam.json similarity index 100% rename from tests/fast/stages/s0_bootstrap/data/checklist-org-iam.json rename to tests/fast/stages/s0_bootstrap_legacy/data/checklist-org-iam.json diff --git a/tests/fast/stages/s0_bootstrap/external_billing_account.tfvars b/tests/fast/stages/s0_bootstrap_legacy/external_billing_account.tfvars similarity index 100% rename from tests/fast/stages/s0_bootstrap/external_billing_account.tfvars rename to tests/fast/stages/s0_bootstrap_legacy/external_billing_account.tfvars diff --git a/tests/fast/stages/s0_bootstrap/external_billing_account.yaml b/tests/fast/stages/s0_bootstrap_legacy/external_billing_account.yaml similarity index 100% rename from tests/fast/stages/s0_bootstrap/external_billing_account.yaml rename to tests/fast/stages/s0_bootstrap_legacy/external_billing_account.yaml diff --git a/tests/fast/stages/s0_bootstrap/iam_by_principals.tfvars b/tests/fast/stages/s0_bootstrap_legacy/iam_by_principals.tfvars similarity index 100% rename from tests/fast/stages/s0_bootstrap/iam_by_principals.tfvars rename to tests/fast/stages/s0_bootstrap_legacy/iam_by_principals.tfvars diff --git a/tests/fast/stages/s0_bootstrap/iam_by_principals.yaml b/tests/fast/stages/s0_bootstrap_legacy/iam_by_principals.yaml similarity index 100% rename from tests/fast/stages/s0_bootstrap/iam_by_principals.yaml rename to tests/fast/stages/s0_bootstrap_legacy/iam_by_principals.yaml diff --git a/tests/fast/stages/s0_bootstrap/managed_org_policies.tfvars b/tests/fast/stages/s0_bootstrap_legacy/managed_org_policies.tfvars similarity index 100% rename from tests/fast/stages/s0_bootstrap/managed_org_policies.tfvars rename to tests/fast/stages/s0_bootstrap_legacy/managed_org_policies.tfvars diff --git a/tests/fast/stages/s0_bootstrap/managed_org_policies.yaml b/tests/fast/stages/s0_bootstrap_legacy/managed_org_policies.yaml similarity index 100% rename from tests/fast/stages/s0_bootstrap/managed_org_policies.yaml rename to tests/fast/stages/s0_bootstrap_legacy/managed_org_policies.yaml diff --git a/tests/fast/stages/s0_bootstrap/simple.tfvars b/tests/fast/stages/s0_bootstrap_legacy/simple.tfvars similarity index 100% rename from tests/fast/stages/s0_bootstrap/simple.tfvars rename to tests/fast/stages/s0_bootstrap_legacy/simple.tfvars diff --git a/tests/fast/stages/s0_bootstrap/simple.yaml b/tests/fast/stages/s0_bootstrap_legacy/simple.yaml similarity index 100% rename from tests/fast/stages/s0_bootstrap/simple.yaml rename to tests/fast/stages/s0_bootstrap_legacy/simple.yaml diff --git a/tests/fast/stages/s0_bootstrap/simple_org_policies.yaml b/tests/fast/stages/s0_bootstrap_legacy/simple_org_policies.yaml similarity index 100% rename from tests/fast/stages/s0_bootstrap/simple_org_policies.yaml rename to tests/fast/stages/s0_bootstrap_legacy/simple_org_policies.yaml diff --git a/tests/fast/stages/s0_bootstrap_experimental/tftest.yaml b/tests/fast/stages/s0_bootstrap_legacy/tftest.yaml similarity index 67% rename from tests/fast/stages/s0_bootstrap_experimental/tftest.yaml rename to tests/fast/stages/s0_bootstrap_legacy/tftest.yaml index c7a7f639b..4c29b4455 100644 --- a/tests/fast/stages/s0_bootstrap_experimental/tftest.yaml +++ b/tests/fast/stages/s0_bootstrap_legacy/tftest.yaml @@ -13,11 +13,18 @@ # See the License for the specific language governing permissions and # limitations under the License. -module: fast/stages/0-bootstrap-experimental +module: fast/stages/0-bootstrap-legacy tests: - # TODO: rename to simple once fast lint setup accepts extra dirs - not-simple: + simple: inventory: - - not-simple.yaml - extra_dirs: - - ../../../tests/fast/stages/s0_bootstrap_experimental/data-simple + - simple.yaml + - simple_org_policies.yaml + managed_org_policies: + inventory: + - simple.yaml + - managed_org_policies.yaml + external_billing_account: + inventory: + - external_billing_account.yaml + iam_by_principals: + cicd: diff --git a/tests/fast/stages/s1_resman/__init__.py b/tests/fast/stages/s1_resman/__init__.py deleted file mode 100644 index c37e93b74..000000000 --- a/tests/fast/stages/s1_resman/__init__.py +++ /dev/null @@ -1,13 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. diff --git a/tests/fast/stages/s1_resman/simple.yaml b/tests/fast/stages/s1_resman/simple.yaml deleted file mode 100644 index e6ab16d82..000000000 --- a/tests/fast/stages/s1_resman/simple.yaml +++ /dev/null @@ -1,175 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - - -counts: - google_folder: 16 - google_folder_iam_binding: 72 - google_org_policy_policy: 2 - google_organization_iam_member: 21 - google_project_iam_member: 19 - google_service_account: 19 - google_service_account_iam_binding: 19 - google_storage_bucket: 9 - google_storage_bucket_iam_binding: 18 - google_storage_bucket_iam_member: 19 - google_storage_bucket_object: 22 - google_tags_tag_binding: 16 - google_tags_tag_key: 2 - google_tags_tag_value: 13 - google_tags_tag_value_iam_binding: 4 - modules: 45 - resources: 271 - -values: - module.top-level-folder["teams"].google_folder_iam_binding.bindings["pf_viewer"]: - condition: - - description: Allow to check buckets and contact policies - expression: 'resource.matchTag(''${organization.id}/${tag_names.context}'', - ''project-factory'') - - ' - title: project-factory-scoped - members: - - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com - role: organizations/123456789012/roles/organizationAdminViewer - - google_storage_bucket_object.workflows["2-project-factory"]: - bucket: fast2-prod-iac-core-outputs - content: "# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License,\ - \ Version 2.0 (the \"License\");\n# you may not use this file except in compliance\ - \ with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n\ - #\n# Unless required by applicable law or agreed to in writing, software\n#\ - \ distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT\ - \ WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the\ - \ License for the specific language governing permissions and\n# limitations\ - \ under the License.\n\nname: \"FAST project-factory stage\"\n\non:\n pull_request:\n\ - \ branches:\n - main\n types:\n - closed\n - opened\n \ - \ - synchronize\n\nenv:\n FAST_SERVICE_ACCOUNT: fast2-prod-resman-pf-1@fast2-prod-automation.iam.gserviceaccount.com\n\ - \ FAST_SERVICE_ACCOUNT_PLAN: fast2-prod-resman-pf-1r@fast2-prod-automation.iam.gserviceaccount.com\n\ - \ FAST_WIF_PROVIDER: projects/1234567890/locations/global/workloadIdentityPools/ldj-bootstrap/providers/ldj-bootstrap-github-ludomagno\n\ - \ SSH_AUTH_SOCK: /tmp/ssh_agent.sock\n TF_PROVIDERS_FILE: 2-project-factory-providers.tf\n\ - \ TF_PROVIDERS_FILE_PLAN: 2-project-factory-r-providers.tf\n TF_VERSION: 1.11.4\n\ - \njobs:\n fast-pr:\n # Skip PRs which are closed without being merged.\n\ - \ if: >-\n github.event.action == 'closed' && \n github.event.pull_request.merged\ - \ == true ||\n github.event.action == 'opened' ||\n github.event.action\ - \ == 'synchronize'\n permissions:\n contents: read\n id-token:\ - \ write\n issues: write\n pull-requests: write\n runs-on: ubuntu-latest\n\ - \ steps:\n - id: checkout\n name: Checkout repository\n \ - \ uses: actions/checkout@v4\n\n # set up SSH key authentication to the\ - \ modules repository\n\n - id: ssh-config\n name: Configure SSH\ - \ authentication\n run: |\n ssh-agent -a \"$SSH_AUTH_SOCK\"\ - \ > /dev/null\n ssh-add - <<< \"${{ secrets.CICD_MODULES_KEY }}\"\n\ - \n # set up step variables for plan / apply\n\n - id: vars-plan\n\ - \ if: github.event.pull_request.merged != true && success()\n \ - \ name: Set up plan variables\n run: |\n echo \"plan_opts=-lock=false\"\ - \ >> \"$GITHUB_ENV\"\n echo \"provider_file=${{env.TF_PROVIDERS_FILE_PLAN}}\"\ - \ >> \"$GITHUB_ENV\"\n echo \"service_account=${{env.FAST_SERVICE_ACCOUNT_PLAN}}\"\ - \ >> \"$GITHUB_ENV\"\n\n - id: vars-apply\n if: github.event.pull_request.merged\ - \ == true && success()\n name: Set up apply variables\n run: |\n\ - \ echo \"provider_file=${{env.TF_PROVIDERS_FILE}}\" >> \"$GITHUB_ENV\"\ - \n echo \"service_account=${{env.FAST_SERVICE_ACCOUNT}}\" >> \"$GITHUB_ENV\"\ - \n\n # set up authentication via Workload identity Federation and gcloud\n\ - \n - id: gcp-auth\n name: Authenticate to Google Cloud\n \ - \ uses: google-github-actions/auth@v2\n with:\n workload_identity_provider:\ - \ ${{env.FAST_WIF_PROVIDER}}\n service_account: ${{env.service_account}}\n\ - \ access_token_lifetime: 900s\n\n - id: gcp-sdk\n name:\ - \ Set up Cloud SDK\n uses: google-github-actions/setup-gcloud@v2\n \ - \ with:\n install_components: alpha\n\n # copy provider file\n\ - \n - id: tf-config-provider\n name: Copy Terraform provider file\n\ - \ run: |\n gcloud storage cp -r \\\n \"gs://fast2-prod-iac-core-outputs/providers/${{env.provider_file}}\"\ - \ ./\n gcloud storage cp -r \\\n \"gs://fast2-prod-iac-core-outputs/tfvars/0-bootstrap.auto.tfvars.json\"\ - \ ./\n gcloud storage cp -r \\\n \"gs://fast2-prod-iac-core-outputs/tfvars/1-resman.auto.tfvars.json\"\ - \ ./\n gcloud storage cp -r \\\n \"gs://fast2-prod-iac-core-outputs/tfvars/0-globals.auto.tfvars.json\"\ - \ ./\n gcloud storage cp -r \\\n \"gs://fast2-prod-iac-core-outputs/tfvars/99-user.auto.tfvars.json\"\ - \ ./\n\n - id: tf-setup\n name: Set up Terraform\n uses:\ - \ hashicorp/setup-terraform@v3\n with:\n terraform_version:\ - \ ${{env.TF_VERSION}}\n\n # run Terraform init/validate/plan\n\n -\ - \ id: tf-init\n name: Terraform init\n continue-on-error: true\n\ - \ run: |\n terraform init -no-color\n\n - id: tf-validate\n\ - \ continue-on-error: true\n name: Terraform validate\n \ - \ run: terraform validate -no-color\n\n - id: tf-plan\n name: Terraform\ - \ plan\n continue-on-error: true\n run: |\n terraform\ - \ plan -input=false -out ../plan.out -no-color ${{env.plan_opts}}\n\n -\ - \ id: tf-apply\n if: github.event.pull_request.merged == true && success()\n\ - \ name: Terraform apply\n continue-on-error: true\n run:\ - \ |\n terraform apply -input=false -auto-approve -no-color ../plan.out\n\ - \n # PR comment with Terraform result from previous steps\n # length\ - \ is checked and trimmed for length so as to stay within the limit\n\n \ - \ - id: pr-comment\n name: Post comment to Pull Request\n continue-on-error:\ - \ true\n uses: actions/github-script@v7\n if: github.event_name\ - \ == 'pull_request'\n env:\n PLAN: ${{steps.tf-plan.outputs.stdout}}\\\ - n${{steps.tf-plan.outputs.stderr}}\n with:\n script: |\n \ - \ const output = `### Terraform Initialization \\`${{steps.tf-init.outcome}}\\\ - `\n\n ### Terraform Validation \\`${{steps.tf-validate.outcome}}\\\ - `\n\n
Validation Output\n\n \ - \ \\`\\`\\`\\n\n ${{steps.tf-validate.outputs.stdout}}\n \ - \ \\`\\`\\`\n\n
\n\n ### Terraform Plan\ - \ \\`${{steps.tf-plan.outcome}}\\`\n\n
Show Plan\n\ - \n \\`\\`\\`\\n\n ${process.env.PLAN.split('\\n').filter(l\ - \ => l.match(/^([A-Z\\s].*|)$$/)).join('\\n')}\n \\`\\`\\`\n\n \ - \
\n\n ### Terraform Apply \\`${{steps.tf-apply.outcome}}\\\ - `\n\n *Pusher: @${{github.actor}}, Action: \\`${{github.event_name}}\\\ - `, Working Directory: \\`${{env.tf_actions_working_dir}}\\`, Workflow: \\`${{github.workflow}}\\\ - `*`;\n\n github.rest.issues.createComment({\n issue_number:\ - \ context.issue.number,\n owner: context.repo.owner,\n \ - \ repo: context.repo.repo,\n body: output\n })\n\ - \n - id: pr-short-comment\n name: Post comment to Pull Request (abbreviated)\n\ - \ uses: actions/github-script@v7\n if: github.event_name == 'pull_request'\ - \ && steps.pr-comment.outcome != 'success'\n with:\n script:\ - \ |\n const output = `### Terraform Initialization \\`${{steps.tf-init.outcome}}\\\ - `\n\n ### Terraform Validation \\`${{steps.tf-validate.outcome}}\\\ - `\n\n ### Terraform Plan \\`${{steps.tf-plan.outcome}}\\`\n\n \ - \ Plan output is in the action log.\n\n ### Terraform Apply\ - \ \\`${{steps.tf-apply.outcome}}\\`\n\n *Pusher: @${{github.actor}},\ - \ Action: \\`${{github.event_name}}\\`, Working Directory: \\`${{env.tf_actions_working_dir}}\\\ - `, Workflow: \\`${{github.workflow}}\\`*`;\n\n github.rest.issues.createComment({\n\ - \ issue_number: context.issue.number,\n owner: context.repo.owner,\n\ - \ repo: context.repo.repo,\n body: output\n \ - \ })\n\n # exit on error from previous steps\n\n - id: check-init\n\ - \ name: Check init failure\n if: steps.tf-init.outcome != 'success'\n\ - \ run: exit 1\n\n - id: check-validate\n name: Check validate\ - \ failure\n if: steps.tf-validate.outcome != 'success'\n run:\ - \ exit 1\n\n - id: check-plan\n name: Check plan failure\n \ - \ if: steps.tf-plan.outcome != 'success'\n run: exit 1\n\n - id:\ - \ check-apply\n name: Check apply failure\n if: github.event.pull_request.merged\ - \ == true && steps.tf-apply.outcome != 'success'\n run: exit 1\n" - name: workflows/2-project-factory-workflow.yaml - -outputs: - cicd_repositories: - project-factory: - provider: projects/1234567890/locations/global/workloadIdentityPools/ldj-bootstrap/providers/ldj-bootstrap-github-ludomagno - repository: - branch: main - name: cloud-foundation-fabric/1-resman - type: github - service_accounts: - data-platform-dev-ro: fast2-dev-resman-dp-0r@fast2-prod-automation.iam.gserviceaccount.com - data-platform-dev-rw: fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com - gcve-dev-ro: fast2-dev-resman-gcve-0r@fast2-prod-automation.iam.gserviceaccount.com - gcve-dev-rw: fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com - gke-dev-ro: fast2-dev-resman-gke-0r@fast2-prod-automation.iam.gserviceaccount.com - gke-dev-rw: fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com - networking-ro: fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com - networking-rw: fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com - project-factory-ro: fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com - project-factory-rw: fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com - sandbox: fast2-dev-resman-sbox-0@fast2-prod-automation.iam.gserviceaccount.com - secops-dev-ro: fast2-dev-resman-secops-0r@fast2-prod-automation.iam.gserviceaccount.com - secops-dev-rw: fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com - secops-ro: fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com - secops-rw: fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com - security-ro: fast2-prod-resman-sec-0r@fast2-prod-automation.iam.gserviceaccount.com - security-rw: fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com diff --git a/tests/fast/stages/s0_bootstrap_experimental/__init__.py b/tests/fast/stages/s1_resman_legacy/__init__.py similarity index 100% rename from tests/fast/stages/s0_bootstrap_experimental/__init__.py rename to tests/fast/stages/s1_resman_legacy/__init__.py diff --git a/tests/fast/stages/s1_resman/simple.tfvars b/tests/fast/stages/s1_resman_legacy/simple.tfvars similarity index 100% rename from tests/fast/stages/s1_resman/simple.tfvars rename to tests/fast/stages/s1_resman_legacy/simple.tfvars diff --git a/tests/fast/stages/s1_resman_legacy/simple.yaml b/tests/fast/stages/s1_resman_legacy/simple.yaml new file mode 100644 index 000000000..0bbc15fac --- /dev/null +++ b/tests/fast/stages/s1_resman_legacy/simple.yaml @@ -0,0 +1,1682 @@ +# Copyright 2025 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +counts: + google_folder: 16 + google_folder_iam_binding: 72 + google_org_policy_policy: 2 + google_organization_iam_member: 21 + google_project_iam_member: 19 + google_service_account: 19 + google_service_account_iam_binding: 19 + google_storage_bucket: 9 + google_storage_bucket_iam_binding: 18 + google_storage_bucket_iam_member: 19 + google_storage_bucket_object: 22 + google_tags_tag_binding: 16 + google_tags_tag_key: 2 + google_tags_tag_value: 13 + google_tags_tag_value_iam_binding: 4 + modules: 45 + resources: 271 + +values: + google_storage_bucket_object.workflows["2-project-factory"]: + bucket: fast2-prod-iac-core-outputs + cache_control: null + content: "# Copyright 2025 Google LLC\n#\n# Licensed under the Apache License,\ + \ Version 2.0 (the \"License\");\n# you may not use this file except in compliance\ + \ with the License.\n# You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n\ + #\n# Unless required by applicable law or agreed to in writing, software\n#\ + \ distributed under the License is distributed on an \"AS IS\" BASIS,\n# WITHOUT\ + \ WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the\ + \ License for the specific language governing permissions and\n# limitations\ + \ under the License.\n\nname: \"FAST project-factory stage\"\n\non:\n pull_request:\n\ + \ branches:\n - main\n types:\n - closed\n - opened\n \ + \ - synchronize\n\nenv:\n FAST_SERVICE_ACCOUNT: fast2-prod-resman-pf-1@fast2-prod-automation.iam.gserviceaccount.com\n\ + \ FAST_SERVICE_ACCOUNT_PLAN: fast2-prod-resman-pf-1r@fast2-prod-automation.iam.gserviceaccount.com\n\ + \ FAST_WIF_PROVIDER: projects/1234567890/locations/global/workloadIdentityPools/ldj-bootstrap/providers/ldj-bootstrap-github-ludomagno\n\ + \ SSH_AUTH_SOCK: /tmp/ssh_agent.sock\n TF_PROVIDERS_FILE: 2-project-factory-providers.tf\n\ + \ TF_PROVIDERS_FILE_PLAN: 2-project-factory-r-providers.tf\n TF_VERSION: 1.11.4\n\ + \njobs:\n fast-pr:\n # Skip PRs which are closed without being merged.\n\ + \ if: >-\n github.event.action == 'closed' && \n github.event.pull_request.merged\ + \ == true ||\n github.event.action == 'opened' ||\n github.event.action\ + \ == 'synchronize'\n permissions:\n contents: read\n id-token:\ + \ write\n issues: write\n pull-requests: write\n runs-on: ubuntu-latest\n\ + \ steps:\n - id: checkout\n name: Checkout repository\n \ + \ uses: actions/checkout@v4\n\n # set up SSH key authentication to the\ + \ modules repository\n\n - id: ssh-config\n name: Configure SSH\ + \ authentication\n run: |\n ssh-agent -a \"$SSH_AUTH_SOCK\"\ + \ > /dev/null\n ssh-add - <<< \"${{ secrets.CICD_MODULES_KEY }}\"\n\ + \n # set up step variables for plan / apply\n\n - id: vars-plan\n\ + \ if: github.event.pull_request.merged != true && success()\n \ + \ name: Set up plan variables\n run: |\n echo \"plan_opts=-lock=false\"\ + \ >> \"$GITHUB_ENV\"\n echo \"provider_file=${{env.TF_PROVIDERS_FILE_PLAN}}\"\ + \ >> \"$GITHUB_ENV\"\n echo \"service_account=${{env.FAST_SERVICE_ACCOUNT_PLAN}}\"\ + \ >> \"$GITHUB_ENV\"\n\n - id: vars-apply\n if: github.event.pull_request.merged\ + \ == true && success()\n name: Set up apply variables\n run: |\n\ + \ echo \"provider_file=${{env.TF_PROVIDERS_FILE}}\" >> \"$GITHUB_ENV\"\ + \n echo \"service_account=${{env.FAST_SERVICE_ACCOUNT}}\" >> \"$GITHUB_ENV\"\ + \n\n # set up authentication via Workload identity Federation and gcloud\n\ + \n - id: gcp-auth\n name: Authenticate to Google Cloud\n \ + \ uses: google-github-actions/auth@v2\n with:\n workload_identity_provider:\ + \ ${{env.FAST_WIF_PROVIDER}}\n service_account: ${{env.service_account}}\n\ + \ access_token_lifetime: 900s\n\n - id: gcp-sdk\n name:\ + \ Set up Cloud SDK\n uses: google-github-actions/setup-gcloud@v2\n \ + \ with:\n install_components: alpha\n\n # copy provider file\n\ + \n - id: tf-config-provider\n name: Copy Terraform provider file\n\ + \ run: |\n gcloud storage cp -r \\\n \"gs://fast2-prod-iac-core-outputs/providers/${{env.provider_file}}\"\ + \ ./\n gcloud storage cp -r \\\n \"gs://fast2-prod-iac-core-outputs/tfvars/0-bootstrap.auto.tfvars.json\"\ + \ ./\n gcloud storage cp -r \\\n \"gs://fast2-prod-iac-core-outputs/tfvars/1-resman.auto.tfvars.json\"\ + \ ./\n gcloud storage cp -r \\\n \"gs://fast2-prod-iac-core-outputs/tfvars/0-globals.auto.tfvars.json\"\ + \ ./\n gcloud storage cp -r \\\n \"gs://fast2-prod-iac-core-outputs/tfvars/99-user.auto.tfvars.json\"\ + \ ./\n\n - id: tf-setup\n name: Set up Terraform\n uses:\ + \ hashicorp/setup-terraform@v3\n with:\n terraform_version:\ + \ ${{env.TF_VERSION}}\n\n # run Terraform init/validate/plan\n\n -\ + \ id: tf-init\n name: Terraform init\n continue-on-error: true\n\ + \ run: |\n terraform init -no-color\n\n - id: tf-validate\n\ + \ continue-on-error: true\n name: Terraform validate\n \ + \ run: terraform validate -no-color\n\n - id: tf-plan\n name: Terraform\ + \ plan\n continue-on-error: true\n run: |\n terraform\ + \ plan -input=false -out ../plan.out -no-color ${{env.plan_opts}}\n\n -\ + \ id: tf-apply\n if: github.event.pull_request.merged == true && success()\n\ + \ name: Terraform apply\n continue-on-error: true\n run:\ + \ |\n terraform apply -input=false -auto-approve -no-color ../plan.out\n\ + \n # PR comment with Terraform result from previous steps\n # length\ + \ is checked and trimmed for length so as to stay within the limit\n\n \ + \ - id: pr-comment\n name: Post comment to Pull Request\n continue-on-error:\ + \ true\n uses: actions/github-script@v7\n if: github.event_name\ + \ == 'pull_request'\n env:\n PLAN: ${{steps.tf-plan.outputs.stdout}}\\\ + n${{steps.tf-plan.outputs.stderr}}\n with:\n script: |\n \ + \ const output = `### Terraform Initialization \\`${{steps.tf-init.outcome}}\\\ + `\n\n ### Terraform Validation \\`${{steps.tf-validate.outcome}}\\\ + `\n\n
Validation Output\n\n \ + \ \\`\\`\\`\\n\n ${{steps.tf-validate.outputs.stdout}}\n \ + \ \\`\\`\\`\n\n
\n\n ### Terraform Plan\ + \ \\`${{steps.tf-plan.outcome}}\\`\n\n
Show Plan\n\ + \n \\`\\`\\`\\n\n ${process.env.PLAN.split('\\n').filter(l\ + \ => l.match(/^([A-Z\\s].*|)$$/)).join('\\n')}\n \\`\\`\\`\n\n \ + \
\n\n ### Terraform Apply \\`${{steps.tf-apply.outcome}}\\\ + `\n\n *Pusher: @${{github.actor}}, Action: \\`${{github.event_name}}\\\ + `, Working Directory: \\`${{env.tf_actions_working_dir}}\\`, Workflow: \\`${{github.workflow}}\\\ + `*`;\n\n github.rest.issues.createComment({\n issue_number:\ + \ context.issue.number,\n owner: context.repo.owner,\n \ + \ repo: context.repo.repo,\n body: output\n })\n\ + \n - id: pr-short-comment\n name: Post comment to Pull Request (abbreviated)\n\ + \ uses: actions/github-script@v7\n if: github.event_name == 'pull_request'\ + \ && steps.pr-comment.outcome != 'success'\n with:\n script:\ + \ |\n const output = `### Terraform Initialization \\`${{steps.tf-init.outcome}}\\\ + `\n\n ### Terraform Validation \\`${{steps.tf-validate.outcome}}\\\ + `\n\n ### Terraform Plan \\`${{steps.tf-plan.outcome}}\\`\n\n \ + \ Plan output is in the action log.\n\n ### Terraform Apply\ + \ \\`${{steps.tf-apply.outcome}}\\`\n\n *Pusher: @${{github.actor}},\ + \ Action: \\`${{github.event_name}}\\`, Working Directory: \\`${{env.tf_actions_working_dir}}\\\ + `, Workflow: \\`${{github.workflow}}\\`*`;\n\n github.rest.issues.createComment({\n\ + \ issue_number: context.issue.number,\n owner: context.repo.owner,\n\ + \ repo: context.repo.repo,\n body: output\n \ + \ })\n\n # exit on error from previous steps\n\n - id: check-init\n\ + \ name: Check init failure\n if: steps.tf-init.outcome != 'success'\n\ + \ run: exit 1\n\n - id: check-validate\n name: Check validate\ + \ failure\n if: steps.tf-validate.outcome != 'success'\n run:\ + \ exit 1\n\n - id: check-plan\n name: Check plan failure\n \ + \ if: steps.tf-plan.outcome != 'success'\n run: exit 1\n\n - id:\ + \ check-apply\n name: Check apply failure\n if: github.event.pull_request.merged\ + \ == true && steps.tf-apply.outcome != 'success'\n run: exit 1\n" + content_disposition: null + content_encoding: null + content_language: null + customer_encryption: [] + deletion_policy: null + detect_md5hash: different hash + event_based_hold: null + force_empty_content_type: null + metadata: null + name: workflows/2-project-factory-workflow.yaml + retention: [] + source: null + source_md5hash: null + temporary_hold: null + timeouts: null + ? module.cicd-sa-ro["project-factory"].google_project_iam_member.project-roles["fast2-prod-automation-roles/logging.logWriter"] + : condition: [] + project: fast2-prod-automation + role: roles/logging.logWriter + module.cicd-sa-ro["project-factory"].google_service_account.service_account[0]: + account_id: fast2-prod-resman-pf-1r + create_ignore_already_exists: null + description: null + disabled: false + display_name: CI/CD 2-pf prod service account (read-only). + email: fast2-prod-resman-pf-1r@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-prod-resman-pf-1r@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.cicd-sa-ro["project-factory"].google_service_account_iam_binding.authoritative["roles/iam.workloadIdentityUser"]: + condition: [] + members: + - principalSet://iam.googleapis.com/projects/1234567890/locations/global/workloadIdentityPools/ldj-bootstrap/attribute.repository/cloud-foundation-fabric/1-resman + role: roles/iam.workloadIdentityUser + ? module.cicd-sa-ro["project-factory"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectViewer"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectViewer + ? module.cicd-sa-rw["project-factory"].google_project_iam_member.project-roles["fast2-prod-automation-roles/logging.logWriter"] + : condition: [] + project: fast2-prod-automation + role: roles/logging.logWriter + module.cicd-sa-rw["project-factory"].google_service_account.service_account[0]: + account_id: fast2-prod-resman-pf-1 + create_ignore_already_exists: null + description: null + disabled: false + display_name: CI/CD 2-pf prod service account. + email: fast2-prod-resman-pf-1@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-prod-resman-pf-1@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.cicd-sa-rw["project-factory"].google_service_account_iam_binding.authoritative["roles/iam.workloadIdentityUser"]: + condition: [] + members: + - principalSet://iam.googleapis.com/projects/1234567890/locations/global/workloadIdentityPools/ldj-bootstrap/attribute.fast_sub/repo:cloud-foundation-fabric/1-resman:ref:refs/heads/main + role: roles/iam.workloadIdentityUser + ? module.cicd-sa-rw["project-factory"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectViewer"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectViewer + module.organization[0].google_organization_iam_member.bindings["data-platform-dev"]: + condition: [] + member: serviceAccount:fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.user + module.organization[0].google_organization_iam_member.bindings["gcve-dev"]: + condition: [] + member: serviceAccount:fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.user + module.organization[0].google_organization_iam_member.bindings["gke-dev"]: + condition: [] + member: serviceAccount:fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.user + module.organization[0].google_organization_iam_member.bindings["sa_net_billing"]: + condition: [] + member: serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.user + module.organization[0].google_organization_iam_member.bindings["sa_net_costs_manager"]: + condition: [] + member: serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.costsManager + module.organization[0].google_organization_iam_member.bindings["sa_net_ro_fw_policy_user"]: + condition: [] + member: serviceAccount:fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/compute.orgFirewallPolicyUser + module.organization[0].google_organization_iam_member.bindings["sa_net_ro_ngfw_enterprise_viewer"]: + condition: [] + member: serviceAccount:fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: organizations/123456789012/roles/ngfwEnterpriseViewer + module.organization[0].google_organization_iam_member.bindings["sa_net_rw_fw_policy_admin"]: + condition: [] + member: serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/compute.orgFirewallPolicyAdmin + module.organization[0].google_organization_iam_member.bindings["sa_net_rw_ngfw_enterprise_admin"]: + condition: [] + member: serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: organizations/123456789012/roles/ngfwEnterpriseAdmin + module.organization[0].google_organization_iam_member.bindings["sa_net_rw_xpn_admin"]: + condition: [] + member: serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/compute.xpnAdmin + module.organization[0].google_organization_iam_member.bindings["sa_pf_billing"]: + condition: [] + member: serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.user + module.organization[0].google_organization_iam_member.bindings["sa_pf_conditional_org_policy"]: + condition: + - description: Org policy tag scoped grant for project factory. + expression: resource.matchTag('123456789012/context', 'project-factory') + title: org_policy_tag_pf_scoped + member: serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/orgpolicy.policyAdmin + module.organization[0].google_organization_iam_member.bindings["sa_pf_costs_manager"]: + condition: [] + member: serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.costsManager + module.organization[0].google_organization_iam_member.bindings["sa_sec_billing"]: + condition: [] + member: serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.user + module.organization[0].google_organization_iam_member.bindings["sa_sec_cloudasset"]: + condition: [] + member: serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/cloudasset.viewer + module.organization[0].google_organization_iam_member.bindings["sa_sec_costs_manager"]: + condition: [] + member: serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.costsManager + module.organization[0].google_organization_iam_member.bindings["sa_so_billing"]: + condition: [] + member: serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.user + module.organization[0].google_organization_iam_member.bindings["sa_so_costs_manager"]: + condition: [] + member: serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.costsManager + module.organization[0].google_organization_iam_member.bindings["sa_so_ro_wif"]: + condition: [] + member: serviceAccount:fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/iam.workforcePoolViewer + module.organization[0].google_organization_iam_member.bindings["sa_so_rw_wif"]: + condition: [] + member: serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/iam.workforcePoolAdmin + module.organization[0].google_organization_iam_member.bindings["secops-dev"]: + condition: [] + member: serviceAccount:fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com + org_id: '123456789012' + role: roles/billing.user + module.organization[0].google_tags_tag_key.default["context"]: + description: Managed by the Terraform organization module. + parent: organizations/123456789012 + purpose: null + purpose_data: null + short_name: context + timeouts: null + module.organization[0].google_tags_tag_key.default["environment"]: + description: Managed by the Terraform organization module. + parent: organizations/123456789012 + purpose: null + purpose_data: null + short_name: environment + timeouts: null + module.organization[0].google_tags_tag_value.default["context/data-platform"]: + description: Managed by the Terraform organization module. + short_name: data-platform + timeouts: null + module.organization[0].google_tags_tag_value.default["context/gcve"]: + description: Managed by the Terraform organization module. + short_name: gcve + timeouts: null + module.organization[0].google_tags_tag_value.default["context/gke"]: + description: Managed by the Terraform organization module. + short_name: gke + timeouts: null + module.organization[0].google_tags_tag_value.default["context/networking"]: + description: Managed by the Terraform organization module. + short_name: networking + timeouts: null + module.organization[0].google_tags_tag_value.default["context/nsec"]: + description: Managed by the Terraform organization module. + short_name: nsec + timeouts: null + module.organization[0].google_tags_tag_value.default["context/project-factory"]: + description: Managed by the Terraform organization module. + short_name: project-factory + timeouts: null + module.organization[0].google_tags_tag_value.default["context/sandbox"]: + description: Managed by the Terraform organization module. + short_name: sandbox + timeouts: null + module.organization[0].google_tags_tag_value.default["context/secops"]: + description: Managed by the Terraform organization module. + short_name: secops + timeouts: null + module.organization[0].google_tags_tag_value.default["context/security"]: + description: Managed by the Terraform organization module. + short_name: security + timeouts: null + module.organization[0].google_tags_tag_value.default["context/shared"]: + description: Managed by the Terraform organization module. + short_name: shared + timeouts: null + module.organization[0].google_tags_tag_value.default["context/tenants"]: + description: Managed by the Terraform organization module. + short_name: tenants + timeouts: null + module.organization[0].google_tags_tag_value.default["environment/development"]: + description: Managed by the Terraform organization module. + short_name: development + timeouts: null + module.organization[0].google_tags_tag_value.default["environment/production"]: + description: Managed by the Terraform organization module. + short_name: production + timeouts: null + module.organization[0].google_tags_tag_value_iam_binding.default["environment/development:roles/resourcemanager.tagUser"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagUser + ? module.organization[0].google_tags_tag_value_iam_binding.default["environment/development:roles/resourcemanager.tagViewer"] + : condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0r@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-sec-0r@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagViewer + module.organization[0].google_tags_tag_value_iam_binding.default["environment/production:roles/resourcemanager.tagUser"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagUser + module.organization[0].google_tags_tag_value_iam_binding.default["environment/production:roles/resourcemanager.tagViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-sec-0r@fast2-prod-automation.iam.gserviceaccount.com + - serviceAccount:fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagViewer + module.stage2-bucket["networking"].google_storage_bucket.bucket[0]: + autoclass: [] + cors: [] + custom_placement_config: [] + default_event_based_hold: null + effective_labels: + goog-terraform-provisioned: 'true' + enable_object_retention: null + encryption: [] + force_destroy: false + hierarchical_namespace: [] + ip_filter: [] + labels: null + lifecycle_rule: [] + location: EU + logging: [] + name: fast2-prod-resman-net-0 + project: fast2-prod-automation + requester_pays: null + retention_policy: [] + storage_class: STANDARD + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + uniform_bucket_level_access: true + versioning: + - enabled: true + module.stage2-bucket["networking"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectAdmin"]: + bucket: fast2-prod-resman-net-0 + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectAdmin + module.stage2-bucket["networking"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"]: + bucket: fast2-prod-resman-net-0 + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectViewer + module.stage2-bucket["project-factory"].google_storage_bucket.bucket[0]: + autoclass: [] + cors: [] + custom_placement_config: [] + default_event_based_hold: null + effective_labels: + goog-terraform-provisioned: 'true' + enable_object_retention: null + encryption: [] + force_destroy: false + hierarchical_namespace: [] + ip_filter: [] + labels: null + lifecycle_rule: [] + location: EU + logging: [] + name: fast2-prod-resman-pf-0 + project: fast2-prod-automation + requester_pays: null + retention_policy: [] + storage_class: STANDARD + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + uniform_bucket_level_access: true + versioning: + - enabled: true + module.stage2-bucket["project-factory"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectAdmin"]: + bucket: fast2-prod-resman-pf-0 + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectAdmin + module.stage2-bucket["project-factory"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"]: + bucket: fast2-prod-resman-pf-0 + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectViewer + module.stage2-bucket["secops"].google_storage_bucket.bucket[0]: + autoclass: [] + cors: [] + custom_placement_config: [] + default_event_based_hold: null + effective_labels: + goog-terraform-provisioned: 'true' + enable_object_retention: null + encryption: [] + force_destroy: false + hierarchical_namespace: [] + ip_filter: [] + labels: null + lifecycle_rule: [] + location: EU + logging: [] + name: fast2-prod-resman-so-0 + project: fast2-prod-automation + requester_pays: null + retention_policy: [] + storage_class: STANDARD + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + uniform_bucket_level_access: true + versioning: + - enabled: true + module.stage2-bucket["secops"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectAdmin"]: + bucket: fast2-prod-resman-so-0 + condition: [] + members: + - serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectAdmin + module.stage2-bucket["secops"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"]: + bucket: fast2-prod-resman-so-0 + condition: [] + members: + - serviceAccount:fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectViewer + module.stage2-bucket["security"].google_storage_bucket.bucket[0]: + autoclass: [] + cors: [] + custom_placement_config: [] + default_event_based_hold: null + effective_labels: + goog-terraform-provisioned: 'true' + enable_object_retention: null + encryption: [] + force_destroy: false + hierarchical_namespace: [] + ip_filter: [] + labels: null + lifecycle_rule: [] + location: EU + logging: [] + name: fast2-prod-resman-sec-0 + project: fast2-prod-automation + requester_pays: null + retention_policy: [] + storage_class: STANDARD + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + uniform_bucket_level_access: true + versioning: + - enabled: true + module.stage2-bucket["security"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectAdmin"]: + bucket: fast2-prod-resman-sec-0 + condition: [] + members: + - serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectAdmin + module.stage2-bucket["security"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"]: + bucket: fast2-prod-resman-sec-0 + condition: [] + members: + - serviceAccount:fast2-prod-resman-sec-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectViewer + module.stage2-folder-env["networking-dev"].google_folder.folder[0]: + deletion_protection: false + display_name: Development + tags: null + timeouts: null + module.stage2-folder-env["networking-dev"].google_tags_tag_binding.binding["environment"]: + timeouts: null + module.stage2-folder-env["networking-prod"].google_folder.folder[0]: + deletion_protection: false + display_name: Production + tags: null + timeouts: null + module.stage2-folder-env["networking-prod"].google_tags_tag_binding.binding["environment"]: + timeouts: null + module.stage2-folder["networking"].google_folder.folder[0]: + deletion_protection: false + display_name: Networking + parent: organizations/123456789012 + tags: null + timeouts: null + ? module.stage2-folder["networking"].google_folder_iam_binding.authoritative["organizations/123456789012/roles/projectIamViewer"] + : condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + role: organizations/123456789012/roles/projectIamViewer + ? module.stage2-folder["networking"].google_folder_iam_binding.authoritative["organizations/123456789012/roles/xpnServiceAdmin"] + : condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + role: organizations/123456789012/roles/xpnServiceAdmin + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/compute.networkViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/compute.networkViewer + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/compute.xpnAdmin"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/compute.xpnAdmin + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/editor"]: + condition: [] + members: + - group:gcp-vpc-network-admins@fast.example.com + role: roles/editor + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/logging.admin"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/logging.admin + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/owner + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderAdmin"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderAdmin + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderViewer + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/resourcemanager.projectCreator"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.projectCreator + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/resourcemanager.tagUser"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagUser + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/resourcemanager.tagViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagViewer + module.stage2-folder["networking"].google_folder_iam_binding.authoritative["roles/viewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/viewer + module.stage2-folder["networking"].google_folder_iam_binding.bindings["project_factory"]: + condition: + - description: null + expression: "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([\n\ + \ 'roles/compute.networkUser', 'roles/composer.sharedVpcAgent',\n 'roles/container.hostServiceAgentUser',\ + \ 'roles/vpcaccess.user',\n 'organizations/123456789012/roles/dnsZoneBinder'\n\ + ])\n" + title: Project factory delegated IAM grant. + members: + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.projectIamAdmin + module.stage2-folder["networking"].google_tags_tag_binding.binding["context"]: + timeouts: null + module.stage2-folder["secops"].google_folder.folder[0]: + deletion_protection: false + display_name: SecOps + parent: organizations/123456789012 + tags: null + timeouts: null + module.stage2-folder["secops"].google_folder_iam_binding.authoritative["roles/editor"]: + condition: [] + members: + - group:gcp-security-admins@fast.example.com + role: roles/editor + module.stage2-folder["secops"].google_folder_iam_binding.authoritative["roles/logging.admin"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/logging.admin + module.stage2-folder["secops"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/owner + module.stage2-folder["secops"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderAdmin"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderAdmin + module.stage2-folder["secops"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderViewer + module.stage2-folder["secops"].google_folder_iam_binding.authoritative["roles/resourcemanager.projectCreator"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.projectCreator + module.stage2-folder["secops"].google_folder_iam_binding.authoritative["roles/resourcemanager.tagUser"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagUser + module.stage2-folder["secops"].google_folder_iam_binding.authoritative["roles/resourcemanager.tagViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagViewer + module.stage2-folder["secops"].google_folder_iam_binding.authoritative["roles/viewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/viewer + module.stage2-folder["secops"].google_tags_tag_binding.binding["context"]: + timeouts: null + module.stage2-folder["security"].google_folder.folder[0]: + deletion_protection: false + display_name: Security + parent: organizations/123456789012 + tags: null + timeouts: null + module.stage2-folder["security"].google_folder_iam_binding.authoritative["organizations/123456789012/roles/kmsKeyViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + role: organizations/123456789012/roles/kmsKeyViewer + module.stage2-folder["security"].google_folder_iam_binding.authoritative["roles/editor"]: + condition: [] + members: + - group:gcp-security-admins@fast.example.com + role: roles/editor + module.stage2-folder["security"].google_folder_iam_binding.authoritative["roles/logging.admin"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/logging.admin + module.stage2-folder["security"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/owner + module.stage2-folder["security"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderAdmin"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderAdmin + module.stage2-folder["security"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-sec-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderViewer + module.stage2-folder["security"].google_folder_iam_binding.authoritative["roles/resourcemanager.projectCreator"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.projectCreator + module.stage2-folder["security"].google_folder_iam_binding.authoritative["roles/resourcemanager.tagUser"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagUser + module.stage2-folder["security"].google_folder_iam_binding.authoritative["roles/resourcemanager.tagViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-sec-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagViewer + module.stage2-folder["security"].google_folder_iam_binding.authoritative["roles/viewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-sec-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/viewer + module.stage2-folder["security"].google_folder_iam_binding.bindings["project_factory"]: + condition: + - description: null + expression: "api.getAttribute('iam.googleapis.com/modifiedGrantsByRole', []).hasOnly([\n\ + \ 'roles/cloudkms.cryptoKeyEncrypterDecrypter'\n])\n" + title: Project factory delegated IAM grant. + members: + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + role: organizations/123456789012/roles/kmsKeyEncryptionAdmin + module.stage2-folder["security"].google_tags_tag_binding.binding["context"]: + timeouts: null + ? module.stage2-sa-ro["networking"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage2-sa-ro["networking"].google_service_account.service_account[0]: + account_id: fast2-prod-resman-net-0r + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman networking service account (read-only). + email: fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage2-sa-ro["networking"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage2-sa-ro["networking"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-organizations/123456789012/roles/storageViewer"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: organizations/123456789012/roles/storageViewer + ? module.stage2-sa-ro["project-factory"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage2-sa-ro["project-factory"].google_service_account.service_account[0]: + account_id: fast2-prod-resman-pf-0r + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman project-factory service account (read-only). + email: fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + ? module.stage2-sa-ro["project-factory"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"] + : condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-1r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/iam.serviceAccountTokenCreator + ? module.stage2-sa-ro["project-factory"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-organizations/123456789012/roles/storageViewer"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: organizations/123456789012/roles/storageViewer + ? module.stage2-sa-ro["secops"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage2-sa-ro["secops"].google_service_account.service_account[0]: + account_id: fast2-prod-resman-so-0r + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman secops service account (read-only). + email: fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage2-sa-ro["secops"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage2-sa-ro["secops"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-organizations/123456789012/roles/storageViewer"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: organizations/123456789012/roles/storageViewer + ? module.stage2-sa-ro["security"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage2-sa-ro["security"].google_service_account.service_account[0]: + account_id: fast2-prod-resman-sec-0r + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman security service account (read-only). + email: fast2-prod-resman-sec-0r@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-prod-resman-sec-0r@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage2-sa-ro["security"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage2-sa-ro["security"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-organizations/123456789012/roles/storageViewer"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: organizations/123456789012/roles/storageViewer + ? module.stage2-sa-rw["networking"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage2-sa-rw["networking"].google_service_account.service_account[0]: + account_id: fast2-prod-resman-net-0 + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman networking service account. + email: fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage2-sa-rw["networking"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage2-sa-rw["networking"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectAdmin"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectAdmin + ? module.stage2-sa-rw["project-factory"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage2-sa-rw["project-factory"].google_service_account.service_account[0]: + account_id: fast2-prod-resman-pf-0 + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman project-factory service account. + email: fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + ? module.stage2-sa-rw["project-factory"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"] + : condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-1@fast2-prod-automation.iam.gserviceaccount.com + role: roles/iam.serviceAccountTokenCreator + ? module.stage2-sa-rw["project-factory"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectAdmin"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectAdmin + ? module.stage2-sa-rw["secops"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage2-sa-rw["secops"].google_service_account.service_account[0]: + account_id: fast2-prod-resman-so-0 + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman secops service account. + email: fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage2-sa-rw["secops"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage2-sa-rw["secops"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectAdmin"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectAdmin + ? module.stage2-sa-rw["security"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage2-sa-rw["security"].google_service_account.service_account[0]: + account_id: fast2-prod-resman-sec-0 + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman security service account. + email: fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage2-sa-rw["security"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage2-sa-rw["security"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectAdmin"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectAdmin + module.stage3-bucket["data-platform-dev"].google_storage_bucket.bucket[0]: + autoclass: [] + cors: [] + custom_placement_config: [] + default_event_based_hold: null + effective_labels: + goog-terraform-provisioned: 'true' + enable_object_retention: null + encryption: [] + force_destroy: false + hierarchical_namespace: [] + ip_filter: [] + labels: null + lifecycle_rule: [] + location: EU + logging: [] + name: fast2-dev-resman-dp-0 + project: fast2-prod-automation + requester_pays: null + retention_policy: [] + storage_class: STANDARD + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + uniform_bucket_level_access: true + versioning: + - enabled: true + module.stage3-bucket["data-platform-dev"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectAdmin"]: + bucket: fast2-dev-resman-dp-0 + condition: [] + members: + - serviceAccount:fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectAdmin + module.stage3-bucket["data-platform-dev"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"]: + bucket: fast2-dev-resman-dp-0 + condition: [] + members: + - serviceAccount:fast2-dev-resman-dp-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectViewer + module.stage3-bucket["gcve-dev"].google_storage_bucket.bucket[0]: + autoclass: [] + cors: [] + custom_placement_config: [] + default_event_based_hold: null + effective_labels: + goog-terraform-provisioned: 'true' + enable_object_retention: null + encryption: [] + force_destroy: false + hierarchical_namespace: [] + ip_filter: [] + labels: null + lifecycle_rule: [] + location: EU + logging: [] + name: fast2-dev-resman-gcve-0 + project: fast2-prod-automation + requester_pays: null + retention_policy: [] + storage_class: STANDARD + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + uniform_bucket_level_access: true + versioning: + - enabled: true + module.stage3-bucket["gcve-dev"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectAdmin"]: + bucket: fast2-dev-resman-gcve-0 + condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectAdmin + module.stage3-bucket["gcve-dev"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"]: + bucket: fast2-dev-resman-gcve-0 + condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectViewer + module.stage3-bucket["gke-dev"].google_storage_bucket.bucket[0]: + autoclass: [] + cors: [] + custom_placement_config: [] + default_event_based_hold: null + effective_labels: + goog-terraform-provisioned: 'true' + enable_object_retention: null + encryption: [] + force_destroy: false + hierarchical_namespace: [] + ip_filter: [] + labels: null + lifecycle_rule: [] + location: EU + logging: [] + name: fast2-dev-resman-gke-0 + project: fast2-prod-automation + requester_pays: null + retention_policy: [] + storage_class: STANDARD + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + uniform_bucket_level_access: true + versioning: + - enabled: true + module.stage3-bucket["gke-dev"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectAdmin"]: + bucket: fast2-dev-resman-gke-0 + condition: [] + members: + - serviceAccount:fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectAdmin + module.stage3-bucket["gke-dev"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"]: + bucket: fast2-dev-resman-gke-0 + condition: [] + members: + - serviceAccount:fast2-dev-resman-gke-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectViewer + module.stage3-bucket["secops-dev"].google_storage_bucket.bucket[0]: + autoclass: [] + cors: [] + custom_placement_config: [] + default_event_based_hold: null + effective_labels: + goog-terraform-provisioned: 'true' + enable_object_retention: null + encryption: [] + force_destroy: false + hierarchical_namespace: [] + ip_filter: [] + labels: null + lifecycle_rule: [] + location: EU + logging: [] + name: fast2-dev-resman-secops-0 + project: fast2-prod-automation + requester_pays: null + retention_policy: [] + storage_class: STANDARD + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + uniform_bucket_level_access: true + versioning: + - enabled: true + module.stage3-bucket["secops-dev"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectAdmin"]: + bucket: fast2-dev-resman-secops-0 + condition: [] + members: + - serviceAccount:fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectAdmin + module.stage3-bucket["secops-dev"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"]: + bucket: fast2-dev-resman-secops-0 + condition: [] + members: + - serviceAccount:fast2-dev-resman-secops-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectViewer + module.stage3-folder["data-platform-dev"].google_folder.folder[0]: + deletion_protection: false + display_name: Development + tags: null + timeouts: null + module.stage3-folder["data-platform-dev"].google_folder_iam_binding.authoritative["roles/compute.xpnAdmin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/compute.xpnAdmin + module.stage3-folder["data-platform-dev"].google_folder_iam_binding.authoritative["roles/logging.admin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/logging.admin + module.stage3-folder["data-platform-dev"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/owner + module.stage3-folder["data-platform-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderAdmin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderAdmin + module.stage3-folder["data-platform-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderViewer"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-dp-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderViewer + module.stage3-folder["data-platform-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.projectCreator"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.projectCreator + module.stage3-folder["data-platform-dev"].google_folder_iam_binding.authoritative["roles/viewer"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-dp-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/viewer + module.stage3-folder["data-platform-dev"].google_tags_tag_binding.binding["environment"]: + timeouts: null + module.stage3-folder["gcve-dev"].google_folder.folder[0]: + deletion_protection: false + display_name: Development + tags: null + timeouts: null + module.stage3-folder["gcve-dev"].google_folder_iam_binding.authoritative["roles/compute.xpnAdmin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/compute.xpnAdmin + module.stage3-folder["gcve-dev"].google_folder_iam_binding.authoritative["roles/logging.admin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/logging.admin + module.stage3-folder["gcve-dev"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/owner + module.stage3-folder["gcve-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderAdmin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderAdmin + module.stage3-folder["gcve-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderViewer"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderViewer + module.stage3-folder["gcve-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.projectCreator"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.projectCreator + module.stage3-folder["gcve-dev"].google_folder_iam_binding.authoritative["roles/viewer"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gcve-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/viewer + module.stage3-folder["gcve-dev"].google_tags_tag_binding.binding["environment"]: + timeouts: null + module.stage3-folder["gke-dev"].google_folder.folder[0]: + deletion_protection: false + display_name: Development + tags: null + timeouts: null + module.stage3-folder["gke-dev"].google_folder_iam_binding.authoritative["roles/compute.xpnAdmin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/compute.xpnAdmin + module.stage3-folder["gke-dev"].google_folder_iam_binding.authoritative["roles/logging.admin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/logging.admin + module.stage3-folder["gke-dev"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/owner + module.stage3-folder["gke-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderAdmin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderAdmin + module.stage3-folder["gke-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderViewer"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gke-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderViewer + module.stage3-folder["gke-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.projectCreator"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.projectCreator + module.stage3-folder["gke-dev"].google_folder_iam_binding.authoritative["roles/viewer"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-gke-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/viewer + module.stage3-folder["gke-dev"].google_tags_tag_binding.binding["environment"]: + timeouts: null + module.stage3-folder["secops-dev"].google_folder.folder[0]: + deletion_protection: false + display_name: Development + tags: null + timeouts: null + module.stage3-folder["secops-dev"].google_folder_iam_binding.authoritative["roles/compute.xpnAdmin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/compute.xpnAdmin + module.stage3-folder["secops-dev"].google_folder_iam_binding.authoritative["roles/logging.admin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/logging.admin + module.stage3-folder["secops-dev"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/owner + module.stage3-folder["secops-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderAdmin"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderAdmin + module.stage3-folder["secops-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderViewer"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-secops-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderViewer + module.stage3-folder["secops-dev"].google_folder_iam_binding.authoritative["roles/resourcemanager.projectCreator"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.projectCreator + module.stage3-folder["secops-dev"].google_folder_iam_binding.authoritative["roles/viewer"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-secops-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/viewer + module.stage3-folder["secops-dev"].google_tags_tag_binding.binding["environment"]: + timeouts: null + ? module.stage3-sa-ro["data-platform-dev"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage3-sa-ro["data-platform-dev"].google_service_account.service_account[0]: + account_id: fast2-dev-resman-dp-0r + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman data-platform-dev service account (read-only). + email: fast2-dev-resman-dp-0r@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-dev-resman-dp-0r@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + ? module.stage3-sa-ro["data-platform-dev"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"] + : condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage3-sa-ro["data-platform-dev"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-organizations/123456789012/roles/storageViewer"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: organizations/123456789012/roles/storageViewer + ? module.stage3-sa-ro["gcve-dev"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage3-sa-ro["gcve-dev"].google_service_account.service_account[0]: + account_id: fast2-dev-resman-gcve-0r + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman gcve-dev service account (read-only). + email: fast2-dev-resman-gcve-0r@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-dev-resman-gcve-0r@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage3-sa-ro["gcve-dev"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage3-sa-ro["gcve-dev"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-organizations/123456789012/roles/storageViewer"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: organizations/123456789012/roles/storageViewer + ? module.stage3-sa-ro["gke-dev"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage3-sa-ro["gke-dev"].google_service_account.service_account[0]: + account_id: fast2-dev-resman-gke-0r + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman gke-dev service account (read-only). + email: fast2-dev-resman-gke-0r@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-dev-resman-gke-0r@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage3-sa-ro["gke-dev"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage3-sa-ro["gke-dev"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-organizations/123456789012/roles/storageViewer"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: organizations/123456789012/roles/storageViewer + ? module.stage3-sa-ro["secops-dev"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage3-sa-ro["secops-dev"].google_service_account.service_account[0]: + account_id: fast2-dev-resman-secops-0r + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman secops-dev service account (read-only). + email: fast2-dev-resman-secops-0r@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-dev-resman-secops-0r@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage3-sa-ro["secops-dev"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage3-sa-ro["secops-dev"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-organizations/123456789012/roles/storageViewer"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: organizations/123456789012/roles/storageViewer + ? module.stage3-sa-rw["data-platform-dev"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage3-sa-rw["data-platform-dev"].google_service_account.service_account[0]: + account_id: fast2-dev-resman-dp-0 + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman data-platform-dev service account. + email: fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + ? module.stage3-sa-rw["data-platform-dev"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"] + : condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage3-sa-rw["data-platform-dev"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectAdmin"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectAdmin + ? module.stage3-sa-rw["gcve-dev"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage3-sa-rw["gcve-dev"].google_service_account.service_account[0]: + account_id: fast2-dev-resman-gcve-0 + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman gcve-dev service account. + email: fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage3-sa-rw["gcve-dev"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage3-sa-rw["gcve-dev"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectAdmin"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectAdmin + ? module.stage3-sa-rw["gke-dev"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage3-sa-rw["gke-dev"].google_service_account.service_account[0]: + account_id: fast2-dev-resman-gke-0 + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman gke-dev service account. + email: fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage3-sa-rw["gke-dev"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage3-sa-rw["gke-dev"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectAdmin"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectAdmin + ? module.stage3-sa-rw["secops-dev"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.stage3-sa-rw["secops-dev"].google_service_account.service_account[0]: + account_id: fast2-dev-resman-secops-0 + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman secops-dev service account. + email: fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.stage3-sa-rw["secops-dev"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.stage3-sa-rw["secops-dev"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectAdmin"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectAdmin + module.top-level-bucket["sandbox"].google_storage_bucket.bucket[0]: + autoclass: [] + cors: [] + custom_placement_config: [] + default_event_based_hold: null + effective_labels: + goog-terraform-provisioned: 'true' + enable_object_retention: null + encryption: [] + force_destroy: false + hierarchical_namespace: [] + ip_filter: [] + labels: null + lifecycle_rule: [] + location: EU + logging: [] + name: fast2-dev-resman-sbox-0 + project: fast2-prod-automation + requester_pays: null + retention_policy: [] + storage_class: STANDARD + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + uniform_bucket_level_access: true + versioning: + - enabled: true + module.top-level-bucket["sandbox"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectAdmin"]: + bucket: fast2-dev-resman-sbox-0 + condition: [] + members: + - serviceAccount:fast2-dev-resman-sbox-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectAdmin + module.top-level-bucket["sandbox"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"]: + bucket: fast2-dev-resman-sbox-0 + condition: [] + members: + - serviceAccount:fast2-dev-resman-sbox-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/storage.objectViewer + module.top-level-folder["data-platform"].google_folder.folder[0]: + deletion_protection: false + display_name: Data Platform + parent: organizations/123456789012 + tags: null + timeouts: null + module.top-level-folder["data-platform"].google_tags_tag_binding.binding["context"]: + timeouts: null + module.top-level-folder["gcve"].google_folder.folder[0]: + deletion_protection: false + display_name: GCVE + parent: organizations/123456789012 + tags: null + timeouts: null + module.top-level-folder["gcve"].google_tags_tag_binding.binding["context"]: + timeouts: null + module.top-level-folder["gke"].google_folder.folder[0]: + deletion_protection: false + display_name: GKE + parent: organizations/123456789012 + tags: null + timeouts: null + module.top-level-folder["gke"].google_tags_tag_binding.binding["context"]: + timeouts: null + module.top-level-folder["sandbox"].google_folder.folder[0]: + deletion_protection: false + display_name: Sandbox + parent: organizations/123456789012 + tags: null + timeouts: null + module.top-level-folder["sandbox"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:fast2-dev-resman-sbox-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/owner + module.top-level-folder["sandbox"].google_org_policy_policy.default["compute.vmExternalIpAccess"]: + dry_run_spec: [] + spec: + - inherit_from_parent: null + reset: null + rules: + - allow_all: 'TRUE' + condition: [] + deny_all: null + enforce: null + parameters: null + values: [] + timeouts: null + module.top-level-folder["sandbox"].google_org_policy_policy.default["sql.restrictPublicIp"]: + dry_run_spec: [] + spec: + - inherit_from_parent: null + reset: null + rules: + - allow_all: null + condition: [] + deny_all: null + enforce: 'TRUE' + parameters: null + values: [] + timeouts: null + module.top-level-folder["sandbox"].google_tags_tag_binding.binding["context"]: + timeouts: null + module.top-level-folder["shared"].google_folder.folder[0]: + deletion_protection: false + display_name: Shared Infrastructure + parent: organizations/123456789012 + tags: null + timeouts: null + module.top-level-folder["shared"].google_tags_tag_binding.binding["context"]: + timeouts: null + module.top-level-folder["teams"].google_folder.folder[0]: + deletion_protection: false + display_name: Teams + parent: organizations/123456789012 + tags: null + timeouts: null + ? module.top-level-folder["teams"].google_folder_iam_binding.authoritative["organizations/123456789012/roles/xpnServiceAdmin"] + : condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + role: organizations/123456789012/roles/xpnServiceAdmin + module.top-level-folder["teams"].google_folder_iam_binding.authoritative["roles/owner"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/owner + module.top-level-folder["teams"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderAdmin"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderAdmin + module.top-level-folder["teams"].google_folder_iam_binding.authoritative["roles/resourcemanager.folderViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.folderViewer + module.top-level-folder["teams"].google_folder_iam_binding.authoritative["roles/resourcemanager.projectCreator"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.projectCreator + module.top-level-folder["teams"].google_folder_iam_binding.authoritative["roles/resourcemanager.tagUser"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagUser + module.top-level-folder["teams"].google_folder_iam_binding.authoritative["roles/resourcemanager.tagViewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/resourcemanager.tagViewer + module.top-level-folder["teams"].google_folder_iam_binding.authoritative["roles/viewer"]: + condition: [] + members: + - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + role: roles/viewer + module.top-level-folder["teams"].google_folder_iam_binding.bindings["pf_viewer"]: + condition: + - description: Allow to check buckets and contact policies + expression: 'resource.matchTag(''123456789012/context'', ''project-factory'') + + ' + title: project-factory-scoped + members: + - serviceAccount:fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + role: organizations/123456789012/roles/organizationAdminViewer + module.top-level-folder["teams"].google_tags_tag_binding.binding["context"]: + timeouts: null + module.top-level-folder["tenants"].google_folder.folder[0]: + deletion_protection: false + display_name: Tenants + parent: organizations/123456789012 + tags: null + timeouts: null + module.top-level-folder["tenants"].google_tags_tag_binding.binding["context"]: + timeouts: null + ? module.top-level-sa["sandbox"].google_project_iam_member.project-roles["fast2-prod-automation-roles/serviceusage.serviceUsageConsumer"] + : condition: [] + project: fast2-prod-automation + role: roles/serviceusage.serviceUsageConsumer + module.top-level-sa["sandbox"].google_service_account.service_account[0]: + account_id: fast2-dev-resman-sbox-0 + create_ignore_already_exists: null + description: null + disabled: false + display_name: Terraform resman sandbox folder service account. + email: fast2-dev-resman-sbox-0@fast2-prod-automation.iam.gserviceaccount.com + member: serviceAccount:fast2-dev-resman-sbox-0@fast2-prod-automation.iam.gserviceaccount.com + project: fast2-prod-automation + timeouts: null + module.top-level-sa["sandbox"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"]: + condition: [] + members: null + role: roles/iam.serviceAccountTokenCreator + ? module.top-level-sa["sandbox"].google_storage_bucket_iam_member.bucket-roles["fast2-prod-iac-core-outputs-roles/storage.objectAdmin"] + : bucket: fast2-prod-iac-core-outputs + condition: [] + role: roles/storage.objectAdmin + +outputs: + cicd_repositories: + project-factory: + provider: projects/1234567890/locations/global/workloadIdentityPools/ldj-bootstrap/providers/ldj-bootstrap-github-ludomagno + repository: + branch: main + name: cloud-foundation-fabric/1-resman + type: github + service_accounts: + data-platform-dev-ro: fast2-dev-resman-dp-0r@fast2-prod-automation.iam.gserviceaccount.com + data-platform-dev-rw: fast2-dev-resman-dp-0@fast2-prod-automation.iam.gserviceaccount.com + gcve-dev-ro: fast2-dev-resman-gcve-0r@fast2-prod-automation.iam.gserviceaccount.com + gcve-dev-rw: fast2-dev-resman-gcve-0@fast2-prod-automation.iam.gserviceaccount.com + gke-dev-ro: fast2-dev-resman-gke-0r@fast2-prod-automation.iam.gserviceaccount.com + gke-dev-rw: fast2-dev-resman-gke-0@fast2-prod-automation.iam.gserviceaccount.com + networking-ro: fast2-prod-resman-net-0r@fast2-prod-automation.iam.gserviceaccount.com + networking-rw: fast2-prod-resman-net-0@fast2-prod-automation.iam.gserviceaccount.com + project-factory-ro: fast2-prod-resman-pf-0r@fast2-prod-automation.iam.gserviceaccount.com + project-factory-rw: fast2-prod-resman-pf-0@fast2-prod-automation.iam.gserviceaccount.com + sandbox: fast2-dev-resman-sbox-0@fast2-prod-automation.iam.gserviceaccount.com + secops-dev-ro: fast2-dev-resman-secops-0r@fast2-prod-automation.iam.gserviceaccount.com + secops-dev-rw: fast2-dev-resman-secops-0@fast2-prod-automation.iam.gserviceaccount.com + secops-ro: fast2-prod-resman-so-0r@fast2-prod-automation.iam.gserviceaccount.com + secops-rw: fast2-prod-resman-so-0@fast2-prod-automation.iam.gserviceaccount.com + security-ro: fast2-prod-resman-sec-0r@fast2-prod-automation.iam.gserviceaccount.com + security-rw: fast2-prod-resman-sec-0@fast2-prod-automation.iam.gserviceaccount.com + diff --git a/tests/fast/stages/s1_resman/tftest.yaml b/tests/fast/stages/s1_resman_legacy/tftest.yaml similarity index 94% rename from tests/fast/stages/s1_resman/tftest.yaml rename to tests/fast/stages/s1_resman_legacy/tftest.yaml index 739d91304..765516efd 100644 --- a/tests/fast/stages/s1_resman/tftest.yaml +++ b/tests/fast/stages/s1_resman_legacy/tftest.yaml @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -module: fast/stages/1-resman +module: fast/stages/1-resman-legacy tests: simple: diff --git a/tests/fast/stages/s2_project_factory/simple.tfvars b/tests/fast/stages/s2_project_factory/simple.tfvars index b50077fda..eaae77f48 100644 --- a/tests/fast/stages/s2_project_factory/simple.tfvars +++ b/tests/fast/stages/s2_project_factory/simple.tfvars @@ -8,9 +8,7 @@ billing_account = { folder_ids = { teams = "folders/1234567890" } -groups = { - gcp-devops = "group:gcp-devops@example.org" -} tag_values = { "environment/development" = "tagValues/1234567890" + "environment/production" = "tagValues/2345678901" } diff --git a/tests/fast/stages/s2_project_factory/simple.yaml b/tests/fast/stages/s2_project_factory/simple.yaml index cd35b778e..cc9f07be2 100644 --- a/tests/fast/stages/s2_project_factory/simple.yaml +++ b/tests/fast/stages/s2_project_factory/simple.yaml @@ -12,180 +12,16 @@ # See the License for the specific language governing permissions and # limitations under the License. -values: - module.projects.module.hierarchy-folder-lvl-1["team-a"].google_folder.folder[0]: - deletion_protection: false - display_name: Team A - parent: folders/1234567890 - tags: null - timeouts: null - module.projects.module.hierarchy-folder-lvl-1["team-b"].google_folder.folder[0]: - deletion_protection: false - display_name: Team B - parent: folders/1234567890 - tags: null - timeouts: null - module.projects.module.hierarchy-folder-lvl-2["team-a/dev"].google_folder.folder[0]: - deletion_protection: false - display_name: Development - tags: null - timeouts: null - module.projects.module.hierarchy-folder-lvl-2["team-a/dev"].google_tags_tag_binding.binding["environment"]: - tag_value: tagValues/1234567890 - timeouts: null - module.projects.module.hierarchy-folder-lvl-2["team-a/prod"].google_folder.folder[0]: - deletion_protection: false - display_name: Production - tags: null - timeouts: null - module.projects.module.hierarchy-folder-lvl-2["team-a/prod"].google_tags_tag_binding.binding["environment"]: - tag_value: environment/production - timeouts: null - module.projects.module.hierarchy-folder-lvl-2["team-b/dev"].google_folder.folder[0]: - deletion_protection: false - display_name: Development - tags: null - timeouts: null - module.projects.module.hierarchy-folder-lvl-2["team-b/dev"].google_tags_tag_binding.binding["environment"]: - tag_value: tagValues/1234567890 - timeouts: null - module.projects.module.hierarchy-folder-lvl-2["team-b/prod"].google_folder.folder[0]: - deletion_protection: false - display_name: Production - tags: null - timeouts: null - module.projects.module.hierarchy-folder-lvl-2["team-b/prod"].google_tags_tag_binding.binding["environment"]: - tag_value: environment/production - timeouts: null - module.projects.module.projects-iam["dev-ta-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: dev-spoke-0 - service_project: test-dev-ta-0 - timeouts: null - ? module.projects.module.projects-iam["dev-ta-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] - : condition: [] - member: group:gcp-devops@example.org - project: dev-spoke-0 - role: roles/compute.networkUser - module.projects.module.projects-iam["dev-tb-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: dev-spoke-0 - service_project: test-dev-tb-0 - timeouts: null - ? module.projects.module.projects-iam["dev-tb-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] - : condition: [] - member: group:gcp-devops@example.org - project: dev-spoke-0 - role: roles/compute.networkUser - module.projects.module.projects-iam["prod-ta-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: prod-spoke-0 - service_project: test-prod-ta-0 - timeouts: null - ? module.projects.module.projects-iam["prod-ta-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] - : condition: [] - member: group:gcp-devops@example.org - project: prod-spoke-0 - role: roles/compute.networkUser - module.projects.module.projects-iam["prod-tb-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: prod-spoke-0 - service_project: test-prod-tb-0 - timeouts: null - ? module.projects.module.projects-iam["prod-tb-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] - : condition: [] - member: group:gcp-devops@example.org - project: prod-spoke-0 - role: roles/compute.networkUser - module.projects.module.projects["dev-ta-0"].google_project.project[0]: - auto_create_network: false - billing_account: 000000-111111-222222 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - labels: null - name: test-dev-ta-0 - project_id: test-dev-ta-0 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects.module.projects["dev-ta-0"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-dev-ta-0 - service: stackdriver.googleapis.com - timeouts: null - module.projects.module.projects["dev-tb-0"].google_project.project[0]: - auto_create_network: false - billing_account: 000000-111111-222222 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - labels: null - name: test-dev-tb-0 - project_id: test-dev-tb-0 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects.module.projects["dev-tb-0"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-dev-tb-0 - service: stackdriver.googleapis.com - timeouts: null - module.projects.module.projects["prod-ta-0"].google_project.project[0]: - auto_create_network: false - billing_account: 000000-111111-222222 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - labels: null - name: test-prod-ta-0 - project_id: test-prod-ta-0 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects.module.projects["prod-ta-0"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-prod-ta-0 - service: stackdriver.googleapis.com - timeouts: null - module.projects.module.projects["prod-tb-0"].google_project.project[0]: - auto_create_network: false - billing_account: 000000-111111-222222 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - labels: null - name: test-prod-tb-0 - project_id: test-prod-tb-0 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects.module.projects["prod-tb-0"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-prod-tb-0 - service: stackdriver.googleapis.com - timeouts: null - +values: {} counts: - google_compute_shared_vpc_service_project: 4 - google_folder: 6 - google_project: 4 - google_project_iam_member: 4 + google_compute_shared_vpc_service_project: 2 + google_folder: 3 + google_project: 2 + google_project_iam_member: 2 google_project_service: 4 + google_project_service_identity: 2 google_storage_bucket_object: 1 - google_tags_tag_binding: 4 - modules: 15 - resources: 27 - -outputs: - buckets: {} - projects: __missing__ - service_accounts: {} + google_tags_tag_binding: 2 + modules: 8 + resources: 19 + terraform_data: 1 diff --git a/tests/fast/stages/s2_project_factory_experimental/simple.yaml b/tests/fast/stages/s2_project_factory_experimental/simple.yaml deleted file mode 100644 index cc9f07be2..000000000 --- a/tests/fast/stages/s2_project_factory_experimental/simple.yaml +++ /dev/null @@ -1,27 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -values: {} -counts: - google_compute_shared_vpc_service_project: 2 - google_folder: 3 - google_project: 2 - google_project_iam_member: 2 - google_project_service: 4 - google_project_service_identity: 2 - google_storage_bucket_object: 1 - google_tags_tag_binding: 2 - modules: 8 - resources: 19 - terraform_data: 1 diff --git a/tests/fast/stages/s2_project_factory_experimental/simple.tfvars b/tests/fast/stages/s2_project_factory_legacy/simple.tfvars similarity index 81% rename from tests/fast/stages/s2_project_factory_experimental/simple.tfvars rename to tests/fast/stages/s2_project_factory_legacy/simple.tfvars index eaae77f48..b50077fda 100644 --- a/tests/fast/stages/s2_project_factory_experimental/simple.tfvars +++ b/tests/fast/stages/s2_project_factory_legacy/simple.tfvars @@ -8,7 +8,9 @@ billing_account = { folder_ids = { teams = "folders/1234567890" } +groups = { + gcp-devops = "group:gcp-devops@example.org" +} tag_values = { "environment/development" = "tagValues/1234567890" - "environment/production" = "tagValues/2345678901" } diff --git a/tests/fast/stages/s2_project_factory_legacy/simple.yaml b/tests/fast/stages/s2_project_factory_legacy/simple.yaml new file mode 100644 index 000000000..cd35b778e --- /dev/null +++ b/tests/fast/stages/s2_project_factory_legacy/simple.yaml @@ -0,0 +1,191 @@ +# Copyright 2025 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +values: + module.projects.module.hierarchy-folder-lvl-1["team-a"].google_folder.folder[0]: + deletion_protection: false + display_name: Team A + parent: folders/1234567890 + tags: null + timeouts: null + module.projects.module.hierarchy-folder-lvl-1["team-b"].google_folder.folder[0]: + deletion_protection: false + display_name: Team B + parent: folders/1234567890 + tags: null + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-a/dev"].google_folder.folder[0]: + deletion_protection: false + display_name: Development + tags: null + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-a/dev"].google_tags_tag_binding.binding["environment"]: + tag_value: tagValues/1234567890 + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-a/prod"].google_folder.folder[0]: + deletion_protection: false + display_name: Production + tags: null + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-a/prod"].google_tags_tag_binding.binding["environment"]: + tag_value: environment/production + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-b/dev"].google_folder.folder[0]: + deletion_protection: false + display_name: Development + tags: null + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-b/dev"].google_tags_tag_binding.binding["environment"]: + tag_value: tagValues/1234567890 + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-b/prod"].google_folder.folder[0]: + deletion_protection: false + display_name: Production + tags: null + timeouts: null + module.projects.module.hierarchy-folder-lvl-2["team-b/prod"].google_tags_tag_binding.binding["environment"]: + tag_value: environment/production + timeouts: null + module.projects.module.projects-iam["dev-ta-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: + deletion_policy: null + host_project: dev-spoke-0 + service_project: test-dev-ta-0 + timeouts: null + ? module.projects.module.projects-iam["dev-ta-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] + : condition: [] + member: group:gcp-devops@example.org + project: dev-spoke-0 + role: roles/compute.networkUser + module.projects.module.projects-iam["dev-tb-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: + deletion_policy: null + host_project: dev-spoke-0 + service_project: test-dev-tb-0 + timeouts: null + ? module.projects.module.projects-iam["dev-tb-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] + : condition: [] + member: group:gcp-devops@example.org + project: dev-spoke-0 + role: roles/compute.networkUser + module.projects.module.projects-iam["prod-ta-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: + deletion_policy: null + host_project: prod-spoke-0 + service_project: test-prod-ta-0 + timeouts: null + ? module.projects.module.projects-iam["prod-ta-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] + : condition: [] + member: group:gcp-devops@example.org + project: prod-spoke-0 + role: roles/compute.networkUser + module.projects.module.projects-iam["prod-tb-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: + deletion_policy: null + host_project: prod-spoke-0 + service_project: test-prod-tb-0 + timeouts: null + ? module.projects.module.projects-iam["prod-tb-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] + : condition: [] + member: group:gcp-devops@example.org + project: prod-spoke-0 + role: roles/compute.networkUser + module.projects.module.projects["dev-ta-0"].google_project.project[0]: + auto_create_network: false + billing_account: 000000-111111-222222 + deletion_policy: DELETE + effective_labels: + goog-terraform-provisioned: 'true' + labels: null + name: test-dev-ta-0 + project_id: test-dev-ta-0 + tags: null + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + module.projects.module.projects["dev-ta-0"].google_project_service.project_services["stackdriver.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-dev-ta-0 + service: stackdriver.googleapis.com + timeouts: null + module.projects.module.projects["dev-tb-0"].google_project.project[0]: + auto_create_network: false + billing_account: 000000-111111-222222 + deletion_policy: DELETE + effective_labels: + goog-terraform-provisioned: 'true' + labels: null + name: test-dev-tb-0 + project_id: test-dev-tb-0 + tags: null + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + module.projects.module.projects["dev-tb-0"].google_project_service.project_services["stackdriver.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-dev-tb-0 + service: stackdriver.googleapis.com + timeouts: null + module.projects.module.projects["prod-ta-0"].google_project.project[0]: + auto_create_network: false + billing_account: 000000-111111-222222 + deletion_policy: DELETE + effective_labels: + goog-terraform-provisioned: 'true' + labels: null + name: test-prod-ta-0 + project_id: test-prod-ta-0 + tags: null + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + module.projects.module.projects["prod-ta-0"].google_project_service.project_services["stackdriver.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-prod-ta-0 + service: stackdriver.googleapis.com + timeouts: null + module.projects.module.projects["prod-tb-0"].google_project.project[0]: + auto_create_network: false + billing_account: 000000-111111-222222 + deletion_policy: DELETE + effective_labels: + goog-terraform-provisioned: 'true' + labels: null + name: test-prod-tb-0 + project_id: test-prod-tb-0 + tags: null + terraform_labels: + goog-terraform-provisioned: 'true' + timeouts: null + module.projects.module.projects["prod-tb-0"].google_project_service.project_services["stackdriver.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-prod-tb-0 + service: stackdriver.googleapis.com + timeouts: null + +counts: + google_compute_shared_vpc_service_project: 4 + google_folder: 6 + google_project: 4 + google_project_iam_member: 4 + google_project_service: 4 + google_storage_bucket_object: 1 + google_tags_tag_binding: 4 + modules: 15 + resources: 27 + +outputs: + buckets: {} + projects: __missing__ + service_accounts: {} diff --git a/tests/fast/stages/s2_project_factory_experimental/tftest.yaml b/tests/fast/stages/s2_project_factory_legacy/tftest.yaml similarity index 92% rename from tests/fast/stages/s2_project_factory_experimental/tftest.yaml rename to tests/fast/stages/s2_project_factory_legacy/tftest.yaml index 6408663ac..6cc4c769e 100644 --- a/tests/fast/stages/s2_project_factory_experimental/tftest.yaml +++ b/tests/fast/stages/s2_project_factory_legacy/tftest.yaml @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -module: fast/stages/2-project-factory-experimental +module: fast/stages/2-project-factory-legacy tests: simple: diff --git a/tests/modules/folder/context.tfvars b/tests/modules/folder/context.tfvars index 624908ed8..75c270a1f 100644 --- a/tests/modules/folder/context.tfvars +++ b/tests/modules/folder/context.tfvars @@ -1,6 +1,11 @@ parent = "$folder_ids:default" name = "Test Context" context = { + condition_vars = { + organization = { + id = 1234567890 + } + } custom_roles = { myrole_one = "organizations/366118655033/roles/myRoleOne" myrole_two = "organizations/366118655033/roles/myRoleTwo" @@ -37,6 +42,10 @@ iam_bindings = { members = [ "$iam_principals:mysa" ] + condition = { + title = "Test" + expression = "resource.matchTag('$${organization.id}/environment', 'development')" + } } } iam_bindings_additive = { diff --git a/tests/modules/folder/context.yaml b/tests/modules/folder/context.yaml index 9e9fccbde..df95744fa 100644 --- a/tests/modules/folder/context.yaml +++ b/tests/modules/folder/context.yaml @@ -36,7 +36,10 @@ values: - serviceAccount:test@test-project.iam.gserviceaccount.com role: roles/viewer google_folder_iam_binding.bindings["myrole_two"]: - condition: [] + condition: + - description: null + expression: resource.matchTag('1234567890/environment', 'development') + title: Test members: - serviceAccount:test@test-project.iam.gserviceaccount.com role: organizations/366118655033/roles/myRoleTwo diff --git a/tests/modules/gcs/context.tfvars b/tests/modules/gcs/context.tfvars index e092b965c..b339fddb7 100644 --- a/tests/modules/gcs/context.tfvars +++ b/tests/modules/gcs/context.tfvars @@ -1,4 +1,9 @@ context = { + condition_vars = { + organization = { + id = 1234567890 + } + } custom_roles = { myrole_one = "organizations/366118655033/roles/myRoleOne" myrole_two = "organizations/366118655033/roles/myRoleTwo" @@ -35,6 +40,10 @@ iam_bindings = { members = [ "$iam_principals:mysa" ] + condition = { + title = "Test" + expression = "resource.matchTag('$${organization.id}/environment', 'development')" + } } } iam_bindings_additive = { diff --git a/tests/modules/gcs/context.yaml b/tests/modules/gcs/context.yaml index f0273e8c6..1773e439d 100644 --- a/tests/modules/gcs/context.yaml +++ b/tests/modules/gcs/context.yaml @@ -70,7 +70,10 @@ values: role: roles/viewer google_storage_bucket_iam_binding.bindings["myrole_two"]: bucket: mybucket - condition: [] + condition: + - description: null + expression: resource.matchTag('1234567890/environment', 'development') + title: Test members: - serviceAccount:test@test-project.iam.gserviceaccount.com role: organizations/366118655033/roles/myRoleTwo diff --git a/tests/modules/iam_service_account/context.tfvars b/tests/modules/iam_service_account/context.tfvars index 4fd9dc816..148200c6b 100644 --- a/tests/modules/iam_service_account/context.tfvars +++ b/tests/modules/iam_service_account/context.tfvars @@ -2,6 +2,11 @@ prefix = "prefix" project_id = "my-project-id" name = "test-sa" context = { + condition_vars = { + organization = { + id = 1234567890 + } + } custom_roles = { myrole_one = "organizations/366118655033/roles/myRoleOne" myrole_two = "organizations/366118655033/roles/myRoleTwo" @@ -47,6 +52,10 @@ iam_bindings = { members = [ "$iam_principals:mysa" ] + condition = { + title = "Test" + expression = "resource.matchTag('$${organization.id}/environment', 'development')" + } } } iam_bindings_additive = { diff --git a/tests/modules/iam_service_account/context.yaml b/tests/modules/iam_service_account/context.yaml index cd2e478ab..a0856f8e5 100644 --- a/tests/modules/iam_service_account/context.yaml +++ b/tests/modules/iam_service_account/context.yaml @@ -48,7 +48,10 @@ values: - serviceAccount:test@test-project.iam.gserviceaccount.com role: roles/viewer google_service_account_iam_binding.bindings["myrole_two"]: - condition: [] + condition: + - description: null + expression: resource.matchTag('1234567890/environment', 'development') + title: Test members: - serviceAccount:test@test-project.iam.gserviceaccount.com role: organizations/366118655033/roles/myRoleTwo diff --git a/tests/modules/organization/context.tfvars b/tests/modules/organization/context.tfvars index 48a1ac3dc..7187be894 100644 --- a/tests/modules/organization/context.tfvars +++ b/tests/modules/organization/context.tfvars @@ -2,6 +2,11 @@ context = { bigquery_datasets = { test = "projects/test-prod-audit-logs-0/datasets/logs" } + condition_vars = { + organization = { + id = 1234567890 + } + } custom_roles = { myrole_one = "organizations/366118655033/roles/myRoleOne" myrole_two = "organizations/366118655033/roles/myRoleTwo" @@ -53,6 +58,10 @@ iam_bindings = { members = [ "$iam_principals:mysa" ] + condition = { + title = "Test" + expression = "resource.matchTag('$${organization.id}/environment', 'development')" + } } } iam_bindings_additive = { diff --git a/tests/modules/organization/context.yaml b/tests/modules/organization/context.yaml index 7ed31fae6..9d23c8b86 100644 --- a/tests/modules/organization/context.yaml +++ b/tests/modules/organization/context.yaml @@ -94,7 +94,10 @@ values: org_id: '1234567890' role: roles/viewer google_organization_iam_binding.bindings["myrole_two"]: - condition: [] + condition: + - description: null + expression: resource.matchTag('1234567890/environment', 'development') + title: Test members: - serviceAccount:test@test-project.iam.gserviceaccount.com org_id: '1234567890' diff --git a/tests/modules/project/context.tfvars b/tests/modules/project/context.tfvars index 7b215e4fa..cc262839b 100644 --- a/tests/modules/project/context.tfvars +++ b/tests/modules/project/context.tfvars @@ -1,4 +1,9 @@ context = { + condition_vars = { + organization = { + id = 1234567890 + } + } custom_roles = { myrole_one = "organizations/366118655033/roles/myRoleOne" myrole_two = "organizations/366118655033/roles/myRoleTwo" @@ -48,6 +53,10 @@ iam_bindings = { members = [ "$iam_principals:mysa" ] + condition = { + title = "Test" + expression = "resource.matchTag('$${organization.id}/environment', 'development')" + } } } iam_bindings_additive = { diff --git a/tests/modules/project/context.yaml b/tests/modules/project/context.yaml index 27aad5ac6..c3c26e504 100644 --- a/tests/modules/project/context.yaml +++ b/tests/modules/project/context.yaml @@ -45,7 +45,10 @@ values: project: my-project role: roles/viewer google_project_iam_binding.bindings["myrole_two"]: - condition: [] + condition: + - description: null + expression: resource.matchTag('1234567890/environment', 'development') + title: Test members: - serviceAccount:test@test-project.iam.gserviceaccount.com project: my-project diff --git a/tests/modules/project_factory/examples/example.yaml b/tests/modules/project_factory/examples/example.yaml index 881813b7a..5828e9832 100644 --- a/tests/modules/project_factory/examples/example.yaml +++ b/tests/modules/project_factory/examples/example.yaml @@ -13,7 +13,7 @@ # limitations under the License. values: - module.project-factory.module.automation-bucket["dev-tb-app0-0"].google_storage_bucket.bucket[0]: + module.project-factory.module.automation-bucket["dev-tb-app0-0/tf-state"].google_storage_bucket.bucket[0]: autoclass: [] cors: [] custom_placement_config: [] @@ -24,6 +24,7 @@ values: encryption: [] force_destroy: false hierarchical_namespace: [] + ip_filter: [] labels: null lifecycle_rule: [] location: EU @@ -39,13 +40,13 @@ values: uniform_bucket_level_access: true versioning: - enabled: false - ? module.project-factory.module.automation-bucket["dev-tb-app0-0"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectCreator"] + ? module.project-factory.module.automation-bucket["dev-tb-app0-0/tf-state"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectCreator"] : bucket: test-pf-dev-tb-app0-0-tf-state condition: [] members: - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com role: roles/storage.objectCreator - ? module.project-factory.module.automation-bucket["dev-tb-app0-0"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"] + ? module.project-factory.module.automation-bucket["dev-tb-app0-0/tf-state"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"] : bucket: test-pf-dev-tb-app0-0-tf-state condition: [] members: @@ -54,8 +55,8 @@ values: - serviceAccount:test-pf-dev-tb-app0-0-ro@test-pf-teams-iac-0.iam.gserviceaccount.com - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com role: roles/storage.objectViewer - ? module.project-factory.module.automation-service-accounts["dev-tb-app0-0/automation/ro"].google_service_account.service_account[0] - : account_id: test-pf-dev-tb-app0-0-ro + module.project-factory.module.automation-service-accounts["dev-tb-app0-0/ro"].google_service_account.service_account[0]: + account_id: test-pf-dev-tb-app0-0-ro create_ignore_already_exists: null description: Team B app 0 read-only automation sa. disabled: false @@ -64,8 +65,8 @@ values: member: serviceAccount:test-pf-dev-tb-app0-0-ro@test-pf-teams-iac-0.iam.gserviceaccount.com project: test-pf-teams-iac-0 timeouts: null - ? module.project-factory.module.automation-service-accounts["dev-tb-app0-0/automation/rw"].google_service_account.service_account[0] - : account_id: test-pf-dev-tb-app0-0-rw + module.project-factory.module.automation-service-accounts["dev-tb-app0-0/rw"].google_service_account.service_account[0]: + account_id: test-pf-dev-tb-app0-0-rw create_ignore_already_exists: null description: Team B app 0 read/write automation sa. disabled: false @@ -74,7 +75,7 @@ values: member: serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com project: test-pf-teams-iac-0 timeouts: null - module.project-factory.module.billing-account[0].google_billing_budget.default["test-100"]: + module.project-factory.module.billing-budgets[0].google_billing_budget.default["test-100"]: all_updates_rule: - disable_default_iam_recipients: true enable_project_level_recipients: false @@ -90,6 +91,7 @@ values: - calendar_period: null credit_types_treatment: INCLUDE_ALL_CREDITS custom_period: [] + projects: null resource_ancestors: - folders/1234567890 display_name: 100 dollars in current spend @@ -100,7 +102,7 @@ values: - spend_basis: CURRENT_SPEND threshold_percent: 0.75 timeouts: null - module.project-factory.module.billing-account[0].google_monitoring_notification_channel.default["billing-default"]: + module.project-factory.module.billing-budgets[0].google_monitoring_notification_channel.default["billing-default"]: description: null display_name: Budget email notification billing-default. enabled: true @@ -112,61 +114,82 @@ values: timeouts: null type: email user_labels: null - module.project-factory.module.hierarchy-folder-lvl-1["team-a"].google_folder.folder[0]: - deletion_protection: false - display_name: Team A - parent: folders/5678901234 - tags: null - timeouts: null - module.project-factory.module.hierarchy-folder-lvl-1["team-a"].google_folder_iam_binding.authoritative["roles/viewer"]: + module.project-factory.module.folder-1-iam["team-a"].google_folder_iam_binding.authoritative["roles/viewer"]: condition: [] members: - group:gcp-devops@example.org - group:team-a-admins@example.org role: roles/viewer - module.project-factory.module.hierarchy-folder-lvl-1["team-b"].google_folder.folder[0]: + module.project-factory.module.folder-1["team-a"].google_folder.folder[0]: + deletion_protection: false + display_name: Team A + parent: folders/5678901234 + tags: null + timeouts: null + module.project-factory.module.folder-1["team-b"].google_folder.folder[0]: deletion_protection: false display_name: Team B parent: folders/5678901234 tags: null timeouts: null - module.project-factory.module.hierarchy-folder-lvl-1["team-c"].google_folder.folder[0]: + module.project-factory.module.folder-1["team-c"].google_folder.folder[0]: deletion_protection: false display_name: Team C parent: folders/5678901234 tags: null timeouts: null - module.project-factory.module.hierarchy-folder-lvl-2["team-a/app-0"].google_folder.folder[0]: + module.project-factory.module.folder-2["team-a/app-0"].google_folder.folder[0]: deletion_protection: false display_name: App 0 tags: null timeouts: null - module.project-factory.module.hierarchy-folder-lvl-2["team-b/app-0"].google_folder.folder[0]: + module.project-factory.module.folder-2["team-b/app-0"].google_folder.folder[0]: deletion_protection: false display_name: App 0 tags: null timeouts: null - module.project-factory.module.hierarchy-folder-lvl-2["team-b/app-0"].google_tags_tag_binding.binding["drs-allow-all"]: + module.project-factory.module.folder-2["team-b/app-0"].google_tags_tag_binding.binding["drs-allow-all"]: tag_value: tagValues/123456 timeouts: null ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_compute_shared_vpc_service_project.shared_vpc_service[0] : deletion_policy: null - host_project: test-pf-dev-net-spoke-0 + host_project: $project_ids:dev-spoke-0 service_project: test-pf-dev-ta-app0-be timeouts: null - ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] + ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["key-0.compute-system"] + : condition: [] + crypto_key_id: projects/kms-central-prj/locations/europe-west1/keyRings/my-keyring/cryptoKeys/ew1-compute + role: roles/cloudkms.cryptoKeyEncrypterDecrypter + ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["key-0.gs-project-accounts"] + : condition: [] + crypto_key_id: projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce + role: roles/cloudkms.cryptoKeyEncrypterDecrypter + ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_binding.authoritative["roles/cloudkms.cryptoKeyEncrypterDecrypter"] + : condition: [] + project: test-pf-dev-ta-app0-be + role: roles/cloudkms.cryptoKeyEncrypterDecrypter + ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_binding.authoritative["roles/storage.objectViewer"] + : condition: [] + members: + - serviceAccount:app-0-be@test-pf-dev-ta-app0-be.iam.gserviceaccount.com + project: test-pf-dev-ta-app0-be + role: roles/storage.objectViewer + ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_iam["$iam_principals:gcp-devops"] : condition: [] member: group:gcp-devops@example.org - project: test-pf-dev-net-spoke-0 + project: $project_ids:dev-spoke-0 role: roles/compute.networkUser ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_robots["roles/compute.networkUser:container-engine"] : condition: [] - project: test-pf-dev-net-spoke-0 + project: $project_ids:dev-spoke-0 role: roles/compute.networkUser ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_robots["roles/container.hostServiceAgentUser:container-engine"] : condition: [] - project: test-pf-dev-net-spoke-0 + project: $project_ids:dev-spoke-0 role: roles/container.hostServiceAgentUser + module.project-factory.module.projects-iam["dev-tb-app0-0"].google_compute_shared_vpc_host_project.shared_vpc_host[0]: + project: test-pf-dev-tb-app0-0 + timeouts: null module.project-factory.module.projects-iam["dev-tb-app0-0"].google_project_iam_binding.authoritative["roles/owner"]: condition: [] members: @@ -201,14 +224,6 @@ values: - ALL parent: projects/test-pf-dev-ta-app0-be timeouts: null - ? module.project-factory.module.projects["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["key-0.compute-system"] - : condition: [] - crypto_key_id: projects/kms-central-prj/locations/europe-west1/keyRings/my-keyring/cryptoKeys/ew1-compute - role: roles/cloudkms.cryptoKeyEncrypterDecrypter - ? module.project-factory.module.projects["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["key-0.gs-project-accounts"] - : condition: [] - crypto_key_id: projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce - role: roles/cloudkms.cryptoKeyEncrypterDecrypter module.project-factory.module.projects["dev-ta-app0-be"].google_project.project[0]: auto_create_network: false billing_account: 012345-67890A-BCDEF0 @@ -223,6 +238,7 @@ values: environment: test team: team-a name: test-pf-dev-ta-app0-be + org_id: null project_id: test-pf-dev-ta-app0-be tags: null terraform_labels: @@ -231,6 +247,10 @@ values: goog-terraform-provisioned: 'true' team: team-a timeouts: null + module.project-factory.module.projects["dev-ta-app0-be"].google_project_iam_member.service_agents["compute-system"]: + condition: [] + project: test-pf-dev-ta-app0-be + role: roles/compute.serviceAgent ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_iam_member.service_agents["container-engine-robot"] : condition: [] project: test-pf-dev-ta-app0-be @@ -239,6 +259,12 @@ values: condition: [] project: test-pf-dev-ta-app0-be role: roles/container.defaultNodeServiceAgent + module.project-factory.module.projects["dev-ta-app0-be"].google_project_service.project_services["compute.googleapis.com"]: + disable_dependent_services: false + disable_on_destroy: false + project: test-pf-dev-ta-app0-be + service: compute.googleapis.com + timeouts: null ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_service.project_services["container.googleapis.com"] : disable_dependent_services: false disable_on_destroy: false @@ -284,9 +310,6 @@ values: module.project-factory.module.projects["dev-tb-app0-0"].data.google_storage_project_service_account.gcs_sa[0]: project: test-pf-dev-tb-app0-0 user_project: null - module.project-factory.module.projects["dev-tb-app0-0"].google_compute_shared_vpc_host_project.shared_vpc_host[0]: - project: test-pf-dev-tb-app0-0 - timeouts: null module.project-factory.module.projects["dev-tb-app0-0"].google_essential_contacts_contact.contact["admin@example.org"]: email: admin@example.org language_tag: en @@ -304,6 +327,7 @@ values: labels: environment: test name: test-pf-dev-tb-app0-0 + org_id: null project_id: test-pf-dev-tb-app0-0 tags: null terraform_labels: @@ -360,6 +384,7 @@ values: environment: test team: team-b name: test-pf-dev-tb-app0-1 + org_id: null project_id: test-pf-dev-tb-app0-1 tags: null terraform_labels: @@ -415,7 +440,7 @@ values: effective_labels: environment: test goog-terraform-provisioned: 'true' - folder_id: '5678901234' + folder_id: null labels: environment: test name: test-pf-teams-iac-0 @@ -456,15 +481,15 @@ values: project: test-pf-teams-iac-0 service: container.googleapis.com timeouts: null - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-net-spoke-0-roles/compute.networkUser"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["$project_ids:dev-spoke-0-roles/compute.networkUser"] : condition: [] - project: test-pf-dev-net-spoke-0 + project: $project_ids:dev-spoke-0 role: roles/compute.networkUser - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/logging.logWriter"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["$project_ids:dev-ta-app0-be-roles/logging.logWriter"] : condition: [] project: test-pf-dev-ta-app0-be role: roles/logging.logWriter - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/monitoring.metricWriter"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["$project_ids:dev-ta-app0-be-roles/monitoring.metricWriter"] : condition: [] project: test-pf-dev-ta-app0-be role: roles/monitoring.metricWriter @@ -478,15 +503,15 @@ values: member: serviceAccount:app-0-be@test-pf-dev-ta-app0-be.iam.gserviceaccount.com project: test-pf-dev-ta-app0-be timeouts: null - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["test-pf-dev-net-spoke-0-roles/compute.networkUser"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["$project_ids:dev-spoke-0-roles/compute.networkUser"] : condition: [] - project: test-pf-dev-net-spoke-0 + project: $project_ids:dev-spoke-0 role: roles/compute.networkUser - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/logging.logWriter"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["$project_ids:dev-ta-app0-be-roles/logging.logWriter"] : condition: [] project: test-pf-dev-ta-app0-be role: roles/logging.logWriter - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/monitoring.metricWriter"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["$project_ids:dev-ta-app0-be-roles/monitoring.metricWriter"] : condition: [] project: test-pf-dev-ta-app0-be role: roles/monitoring.metricWriter @@ -500,13 +525,13 @@ values: member: serviceAccount:app-0-fe@test-pf-dev-ta-app0-be.iam.gserviceaccount.com project: test-pf-dev-ta-app0-be timeouts: null - ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-tb-app0-1-roles/logging.logWriter"] + ? module.project-factory.module.service-accounts["dev-tb-app0-0/vm-default"].google_project_iam_member.project-roles["$project_ids:dev-tb-app0-0-roles/logging.logWriter"] : condition: [] - project: test-pf-dev-tb-app0-1 + project: test-pf-dev-tb-app0-0 role: roles/logging.logWriter - ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-tb-app0-1-roles/monitoring.metricWriter"] + ? module.project-factory.module.service-accounts["dev-tb-app0-0/vm-default"].google_project_iam_member.project-roles["$project_ids:dev-tb-app0-0-roles/monitoring.metricWriter"] : condition: [] - project: test-pf-dev-tb-app0-1 + project: test-pf-dev-tb-app0-0 role: roles/monitoring.metricWriter module.project-factory.module.service-accounts["dev-tb-app0-0/vm-default"].google_service_account.service_account[0]: account_id: vm-default @@ -518,16 +543,11 @@ values: member: serviceAccount:vm-default@test-pf-dev-tb-app0-0.iam.gserviceaccount.com project: test-pf-dev-tb-app0-0 timeouts: null - ? module.project-factory.module.service-accounts["dev-tb-app0-0/vm-default"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"] - : condition: [] - members: - - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com - role: roles/iam.serviceAccountTokenCreator - ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-tb-app0-1-roles/logging.logWriter"] + ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["$project_ids:dev-tb-app0-1-roles/logging.logWriter"] : condition: [] project: test-pf-dev-tb-app0-1 role: roles/logging.logWriter - ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-tb-app0-1-roles/monitoring.metricWriter"] + ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["$project_ids:dev-tb-app0-1-roles/monitoring.metricWriter"] : condition: [] project: test-pf-dev-tb-app0-1 role: roles/monitoring.metricWriter @@ -541,6 +561,17 @@ values: member: serviceAccount:app-0-be@test-pf-dev-tb-app0-1.iam.gserviceaccount.com project: test-pf-dev-tb-app0-1 timeouts: null + ? module.project-factory.module.service_accounts-iam["dev-tb-app0-0/vm-default"].data.google_service_account.service_account[0] + : account_id: vm-default + ? module.project-factory.module.service_accounts-iam["dev-tb-app0-0/vm-default"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"] + : condition: [] + members: + - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com + role: roles/iam.serviceAccountTokenCreator + module.project-factory.terraform_data.defaults_preconditions: + input: null + output: null + triggers_replace: null counts: google_billing_budget: 1 @@ -556,7 +587,7 @@ counts: google_project_iam_member: 21 google_project_service: 13 google_project_service_identity: 4 - google_service_account: 6 + google_service_account: 7 google_service_account_iam_binding: 1 google_storage_bucket: 1 google_storage_bucket_iam_binding: 2 @@ -565,7 +596,6 @@ counts: google_tags_tag_key: 1 google_tags_tag_value: 2 google_tags_tag_value_iam_binding: 1 - modules: 21 - resources: 83 - -outputs: {} + modules: 23 + resources: 85 + terraform_data: 1 diff --git a/tests/modules/project_factory_experimental/bucket_iam.tfvars b/tests/modules/project_factory_experimental/bucket_iam.tfvars deleted file mode 100644 index c9163d0f8..000000000 --- a/tests/modules/project_factory_experimental/bucket_iam.tfvars +++ /dev/null @@ -1,54 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -data_defaults = { - billing_account = "1245-5678-9012" - storage_location = "EU" -} -# make sure the environment label and stackdriver service are always added -data_merges = { - labels = { - environment = "test" - } - services = [ - "stackdriver.googleapis.com" - ] -} -# always use this contacts and prefix, regardless of what is in the yaml file -data_overrides = { - contacts = { - "admin@example.org" = ["ALL"] - } - prefix = "test-pf" -} -# location where the yaml files are read from -factories_config = { - folders_data_path = "bucket_iam/hierarchy" - projects_data_path = "bucket_iam/projects" - context = { - folder_ids = { - default = "folders/5678901234" - teams = "folders/5678901234" - } - iam_principals = { - gcp-devops = "group:gcp-devops@example.org" - } - tag_values = { - "org-policies/drs-allow-all" = "tagValues/123456" - } - vpc_host_projects = { - dev-spoke-0 = "test-pf-dev-net-spoke-0" - } - } -} diff --git a/tests/modules/project_factory_experimental/bucket_iam.yaml b/tests/modules/project_factory_experimental/bucket_iam.yaml deleted file mode 100644 index d065543e8..000000000 --- a/tests/modules/project_factory_experimental/bucket_iam.yaml +++ /dev/null @@ -1,505 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -values: - module.buckets["project2/state"].google_storage_bucket.bucket[0]: - autoclass: [] - cors: [] - custom_placement_config: [] - default_event_based_hold: null - effective_labels: - goog-terraform-provisioned: 'true' - enable_object_retention: null - encryption: [] - force_destroy: false - hierarchical_namespace: [] - labels: null - lifecycle_rule: [] - location: EUROPE-WEST8 - logging: [] - name: test-pf-project2-state - project: test-pf-project2 - requester_pays: null - retention_policy: [] - storage_class: STANDARD - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - uniform_bucket_level_access: true - versioning: - - enabled: false - module.buckets["team-a/project1/state"].google_storage_bucket.bucket[0]: - autoclass: [] - cors: [] - custom_placement_config: [] - default_event_based_hold: null - effective_labels: - goog-terraform-provisioned: 'true' - enable_object_retention: null - encryption: [] - force_destroy: false - hierarchical_namespace: [] - labels: null - lifecycle_rule: [] - location: EUROPE-WEST8 - logging: [] - name: test-pf-project1-state - project: test-pf-project1 - requester_pays: null - retention_policy: [] - storage_class: STANDARD - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - uniform_bucket_level_access: true - versioning: - - enabled: false - module.buckets["team-a/project1/state"].google_storage_bucket_iam_binding.authoritative["roles/storage.admin"]: - bucket: test-pf-project1-state - condition: [] - members: - - serviceAccount:terraform-rw@test-pf-project1.iam.gserviceaccount.com - role: roles/storage.admin - module.hierarchy-folder-lvl-1["team-a"].google_folder.folder[0]: - deletion_protection: false - display_name: Team A - parent: folders/5678901234 - tags: null - timeouts: null - module.hierarchy-folder-lvl-1["team-a"].google_folder_iam_binding.authoritative["roles/viewer"]: - condition: [] - members: - - group:gcp-devops@example.org - - group:team-a-admins@example.org - role: roles/viewer - module.hierarchy-folder-lvl-1["team-b"].google_folder.folder[0]: - deletion_protection: false - display_name: Team B - parent: folders/5678901234 - tags: null - timeouts: null - module.projects["project2"].google_essential_contacts_contact.contact["admin@example.org"]: - email: admin@example.org - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-project2 - timeouts: null - module.projects["project2"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: DELETE - effective_labels: - app: app-0 - environment: test - goog-terraform-provisioned: 'true' - team: team-a - labels: - app: app-0 - environment: test - team: team-a - name: test-pf-project2 - project_id: test-pf-project2 - tags: null - terraform_labels: - app: app-0 - environment: test - goog-terraform-provisioned: 'true' - team: team-a - timeouts: null - module.projects["project2"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-project2 - service: stackdriver.googleapis.com - timeouts: null - module.projects["project3"].data.google_storage_project_service_account.gcs_sa[0]: - project: test-pf-top-project3 - user_project: null - module.projects["project3"].google_essential_contacts_contact.contact["admin@example.org"]: - email: admin@example.org - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-top-project3 - timeouts: null - module.projects["project3"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: DELETE - effective_labels: - environment: test - goog-terraform-provisioned: 'true' - labels: - environment: test - name: test-pf-top-project3 - project_id: test-pf-top-project3 - tags: null - terraform_labels: - environment: test - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["project3"].google_project_iam_member.service_agents["container-engine-robot"]: - condition: [] - project: test-pf-top-project3 - role: roles/container.serviceAgent - module.projects["project3"].google_project_iam_member.service_agents["gkenode"]: - condition: [] - project: test-pf-top-project3 - role: roles/container.defaultNodeServiceAgent - module.projects["project3"].google_project_service.project_services["container.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-top-project3 - service: container.googleapis.com - timeouts: null - module.projects["project3"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-top-project3 - service: stackdriver.googleapis.com - timeouts: null - module.projects["project3"].google_project_service.project_services["storage.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-top-project3 - service: storage.googleapis.com - timeouts: null - module.projects["project3"].google_project_service_identity.default["container.googleapis.com"]: - project: test-pf-top-project3 - service: container.googleapis.com - timeouts: null - module.projects["team-a/automation"].data.google_storage_project_service_account.gcs_sa[0]: - project: test-pf-auto-team-a - user_project: null - module.projects["team-a/automation"].google_essential_contacts_contact.contact["admin@example.org"]: - email: admin@example.org - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-auto-team-a - timeouts: null - module.projects["team-a/automation"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: DELETE - effective_labels: - environment: test - goog-terraform-provisioned: 'true' - labels: - environment: test - name: test-pf-auto-team-a - project_id: test-pf-auto-team-a - tags: null - terraform_labels: - environment: test - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["team-a/automation"].google_project_iam_member.service_agents["container-engine-robot"]: - condition: [] - project: test-pf-auto-team-a - role: roles/container.serviceAgent - module.projects["team-a/automation"].google_project_iam_member.service_agents["gkenode"]: - condition: [] - project: test-pf-auto-team-a - role: roles/container.defaultNodeServiceAgent - module.projects["team-a/automation"].google_project_service.project_services["container.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-auto-team-a - service: container.googleapis.com - timeouts: null - module.projects["team-a/automation"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-auto-team-a - service: stackdriver.googleapis.com - timeouts: null - module.projects["team-a/automation"].google_project_service.project_services["storage.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-auto-team-a - service: storage.googleapis.com - timeouts: null - module.projects["team-a/automation"].google_project_service_identity.default["container.googleapis.com"]: - project: test-pf-auto-team-a - service: container.googleapis.com - timeouts: null - module.projects["team-a/project1"].data.google_storage_project_service_account.gcs_sa[0]: - project: test-pf-project1 - user_project: null - module.projects["team-a/project1"].google_essential_contacts_contact.contact["admin@example.org"]: - email: admin@example.org - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-project1 - timeouts: null - module.projects["team-a/project1"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: DELETE - effective_labels: - environment: test - goog-terraform-provisioned: 'true' - labels: - environment: test - name: test-pf-project1 - project_id: test-pf-project1 - tags: null - terraform_labels: - environment: test - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["team-a/project1"].google_project_iam_member.service_agents["container-engine-robot"]: - condition: [] - project: test-pf-project1 - role: roles/container.serviceAgent - module.projects["team-a/project1"].google_project_iam_member.service_agents["gkenode"]: - condition: [] - project: test-pf-project1 - role: roles/container.defaultNodeServiceAgent - module.projects["team-a/project1"].google_project_service.project_services["container.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-project1 - service: container.googleapis.com - timeouts: null - module.projects["team-a/project1"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-project1 - service: stackdriver.googleapis.com - timeouts: null - module.projects["team-a/project1"].google_project_service.project_services["storage.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-project1 - service: storage.googleapis.com - timeouts: null - module.projects["team-a/project1"].google_project_service_identity.default["container.googleapis.com"]: - project: test-pf-project1 - service: container.googleapis.com - timeouts: null - module.projects["team-b/automation"].data.google_storage_project_service_account.gcs_sa[0]: - project: test-pf-auto-team-b - user_project: null - module.projects["team-b/automation"].google_essential_contacts_contact.contact["admin@example.org"]: - email: admin@example.org - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-auto-team-b - timeouts: null - module.projects["team-b/automation"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: DELETE - effective_labels: - environment: test - goog-terraform-provisioned: 'true' - labels: - environment: test - name: test-pf-auto-team-b - project_id: test-pf-auto-team-b - tags: null - terraform_labels: - environment: test - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["team-b/automation"].google_project_iam_member.service_agents["container-engine-robot"]: - condition: [] - project: test-pf-auto-team-b - role: roles/container.serviceAgent - module.projects["team-b/automation"].google_project_iam_member.service_agents["gkenode"]: - condition: [] - project: test-pf-auto-team-b - role: roles/container.defaultNodeServiceAgent - module.projects["team-b/automation"].google_project_service.project_services["container.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-auto-team-b - service: container.googleapis.com - timeouts: null - module.projects["team-b/automation"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-auto-team-b - service: stackdriver.googleapis.com - timeouts: null - module.projects["team-b/automation"].google_project_service.project_services["storage.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-auto-team-b - service: storage.googleapis.com - timeouts: null - module.projects["team-b/automation"].google_project_service_identity.default["container.googleapis.com"]: - project: test-pf-auto-team-b - service: container.googleapis.com - timeouts: null - module.projects["team-b/project3"].data.google_storage_project_service_account.gcs_sa[0]: - project: test-pf-project3 - user_project: null - module.projects["team-b/project3"].google_essential_contacts_contact.contact["admin@example.org"]: - email: admin@example.org - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-project3 - timeouts: null - module.projects["team-b/project3"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: DELETE - effective_labels: - environment: test - goog-terraform-provisioned: 'true' - labels: - environment: test - name: test-pf-project3 - project_id: test-pf-project3 - tags: null - terraform_labels: - environment: test - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["team-b/project3"].google_project_iam_member.service_agents["container-engine-robot"]: - condition: [] - project: test-pf-project3 - role: roles/container.serviceAgent - module.projects["team-b/project3"].google_project_iam_member.service_agents["gkenode"]: - condition: [] - project: test-pf-project3 - role: roles/container.defaultNodeServiceAgent - module.projects["team-b/project3"].google_project_service.project_services["container.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-project3 - service: container.googleapis.com - timeouts: null - module.projects["team-b/project3"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-project3 - service: stackdriver.googleapis.com - timeouts: null - module.projects["team-b/project3"].google_project_service.project_services["storage.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-project3 - service: storage.googleapis.com - timeouts: null - module.projects["team-b/project3"].google_project_service_identity.default["container.googleapis.com"]: - project: test-pf-project3 - service: container.googleapis.com - timeouts: null - module.service-accounts["project2/app-be-0"].google_service_account.service_account[0]: - account_id: app-be-0 - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: app-be-0@test-pf-project2.iam.gserviceaccount.com - member: serviceAccount:app-be-0@test-pf-project2.iam.gserviceaccount.com - project: test-pf-project2 - timeouts: null - ? module.service-accounts["project2/app-fe-1"].google_project_iam_member.project-roles["my-host-project-roles/compute.networkUser"] - : condition: [] - project: my-host-project - role: roles/compute.networkUser - ? module.service-accounts["project2/app-fe-1"].google_project_iam_member.project-roles["test-pf-project2-roles/storage.objectViewer"] - : condition: [] - project: test-pf-project2 - role: roles/storage.objectViewer - module.service-accounts["project2/app-fe-1"].google_service_account.service_account[0]: - account_id: app-fe-1 - create_ignore_already_exists: null - description: null - disabled: false - display_name: GCE frontend service account. - email: app-fe-1@test-pf-project2.iam.gserviceaccount.com - member: serviceAccount:app-fe-1@test-pf-project2.iam.gserviceaccount.com - project: test-pf-project2 - timeouts: null - module.service-accounts["project2/terraform-rw"].google_service_account.service_account[0]: - account_id: terraform-rw - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: terraform-rw@test-pf-project2.iam.gserviceaccount.com - member: serviceAccount:terraform-rw@test-pf-project2.iam.gserviceaccount.com - project: test-pf-project2 - timeouts: null - module.service-accounts["team-a/project1/app-be-0"].google_service_account.service_account[0]: - account_id: app-be-0 - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: app-be-0@test-pf-project1.iam.gserviceaccount.com - member: serviceAccount:app-be-0@test-pf-project1.iam.gserviceaccount.com - project: test-pf-project1 - timeouts: null - ? module.service-accounts["team-a/project1/app-fe-1"].google_project_iam_member.project-roles["my-host-project-roles/compute.networkUser"] - : condition: [] - project: my-host-project - role: roles/compute.networkUser - ? module.service-accounts["team-a/project1/app-fe-1"].google_project_iam_member.project-roles["test-pf-project1-roles/storage.objectViewer"] - : condition: [] - project: test-pf-project1 - role: roles/storage.objectViewer - module.service-accounts["team-a/project1/app-fe-1"].google_service_account.service_account[0]: - account_id: app-fe-1 - create_ignore_already_exists: null - description: null - disabled: false - display_name: GCE frontend service account. - email: app-fe-1@test-pf-project1.iam.gserviceaccount.com - member: serviceAccount:app-fe-1@test-pf-project1.iam.gserviceaccount.com - project: test-pf-project1 - timeouts: null - module.service-accounts["team-a/project1/terraform-rw"].google_service_account.service_account[0]: - account_id: terraform-rw - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: terraform-rw@test-pf-project1.iam.gserviceaccount.com - member: serviceAccount:terraform-rw@test-pf-project1.iam.gserviceaccount.com - project: test-pf-project1 - timeouts: null - -counts: - google_essential_contacts_contact: 6 - google_folder: 2 - google_folder_iam_binding: 1 - google_project: 6 - google_project_iam_member: 14 - google_project_service: 16 - google_project_service_identity: 5 - google_service_account: 6 - google_storage_bucket: 2 - google_storage_bucket_iam_binding: 1 - google_storage_project_service_account: 5 - modules: 16 - resources: 64 - -outputs: - buckets: - project2/state: test-pf-project2-state - team-a/project1/state: test-pf-project1-state - folders: __missing__ - projects: __missing__ - service_accounts: __missing__ diff --git a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-a/_config.yaml b/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-a/_config.yaml deleted file mode 100644 index 906fec0d8..000000000 --- a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-a/_config.yaml +++ /dev/null @@ -1,20 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -name: Team A -# implicit parent definition via 'default' key -iam: - roles/viewer: - - group:team-a-admins@example.org - - gcp-devops diff --git a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-a/automation.yaml b/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-a/automation.yaml deleted file mode 100644 index 0a744e97f..000000000 --- a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-a/automation.yaml +++ /dev/null @@ -1,20 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -billing_account: 012345-67890A-BCDEF0 -services: - - container.googleapis.com - - storage.googleapis.com - -name: auto-team-a diff --git a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-a/project1.yaml b/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-a/project1.yaml deleted file mode 100644 index 16965ce89..000000000 --- a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-a/project1.yaml +++ /dev/null @@ -1,35 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -billing_account: 012345-67890A-BCDEF0 -services: - - container.googleapis.com - - storage.googleapis.com - -service_accounts: - app-be-0: {} - app-fe-1: - display_name: GCE frontend service account. - iam_self_roles: - - roles/storage.objectViewer - iam_project_roles: - my-host-project: - - roles/compute.networkUser - terraform-rw: {} -buckets: - state: - location: europe-west8 - iam: - roles/storage.admin: - - terraform-rw diff --git a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-b/_config.yaml b/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-b/_config.yaml deleted file mode 100644 index fbdc4437e..000000000 --- a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-b/_config.yaml +++ /dev/null @@ -1,15 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -name: Team B diff --git a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-b/automation.yaml b/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-b/automation.yaml deleted file mode 100644 index 58a698cd7..000000000 --- a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-b/automation.yaml +++ /dev/null @@ -1,20 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -billing_account: 012345-67890A-BCDEF0 -services: - - container.googleapis.com - - storage.googleapis.com - -name: auto-team-b diff --git a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-b/project3.yaml b/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-b/project3.yaml deleted file mode 100644 index c953163bd..000000000 --- a/tests/modules/project_factory_experimental/data/bucket_iam/hierarchy/team-b/project3.yaml +++ /dev/null @@ -1,20 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -billing_account: 012345-67890A-BCDEF0 -services: - - container.googleapis.com - - storage.googleapis.com - -prefix: team-b diff --git a/tests/modules/project_factory_experimental/data/bucket_iam/projects/project2.yaml b/tests/modules/project_factory_experimental/data/bucket_iam/projects/project2.yaml deleted file mode 100644 index d169831d9..000000000 --- a/tests/modules/project_factory_experimental/data/bucket_iam/projects/project2.yaml +++ /dev/null @@ -1,36 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -billing_account: 012345-67890A-BCDEF0 -labels: - app: app-0 - team: team-a -parent: team-a -buckets: - state: - location: europe-west8 -# iam: -# roles/storage.admin: -# - terraform-rw - -service_accounts: - app-be-0: {} - app-fe-1: - display_name: GCE frontend service account. - iam_self_roles: - - roles/storage.objectViewer - iam_project_roles: - my-host-project: - - roles/compute.networkUser - terraform-rw: {} diff --git a/tests/modules/project_factory_experimental/data/bucket_iam/projects/project3.yaml b/tests/modules/project_factory_experimental/data/bucket_iam/projects/project3.yaml deleted file mode 100644 index e6b2fdd4c..000000000 --- a/tests/modules/project_factory_experimental/data/bucket_iam/projects/project3.yaml +++ /dev/null @@ -1,21 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -billing_account: 012345-67890A-BCDEF0 -services: - - container.googleapis.com - - storage.googleapis.com - -name: top-project3 -parent: team-b diff --git a/tests/modules/project_factory_experimental/data/data_overrides_defaults/projects/service1.yaml b/tests/modules/project_factory_experimental/data/data_overrides_defaults/projects/service1.yaml deleted file mode 100644 index e34bb85c9..000000000 --- a/tests/modules/project_factory_experimental/data/data_overrides_defaults/projects/service1.yaml +++ /dev/null @@ -1,25 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -billing_account: 012345-67890A-BCDEF0 - -contacts: # this should be overridden by value - admin-default@example.org: - - "ALL" - -tag_bindings: # this should be overridden with empty value - name1: project_id1 - -services: - - run.googleapis.com diff --git a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-a/_config.yaml b/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-a/_config.yaml deleted file mode 100644 index 410d9e86f..000000000 --- a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-a/_config.yaml +++ /dev/null @@ -1,21 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# yaml-language-server: $schema=../../../schemas/folder.schema.json - -name: Team A -parent: teams -# iam_by_principals: -# "group:team-a-admins@example.com": -# - roles/viewer diff --git a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-a/prod/_config.yaml b/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-a/prod/_config.yaml deleted file mode 100644 index a7079ab36..000000000 --- a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-a/prod/_config.yaml +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# yaml-language-server: $schema=../../../../schemas/folder.schema.json - -name: Production -tag_bindings: - environment: environment/production \ No newline at end of file diff --git a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-b/_config.yaml b/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-b/_config.yaml deleted file mode 100644 index 80d5faa67..000000000 --- a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-b/_config.yaml +++ /dev/null @@ -1,21 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# yaml-language-server: $schema=../../../schemas/folder.schema.json - -name: Team B -parent: teams -# iam_by_principals: -# "group:team-b-admins@example.com": -# - roles/viewer diff --git a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-b/prod/_config.yaml b/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-b/prod/_config.yaml deleted file mode 100644 index a7079ab36..000000000 --- a/tests/modules/project_factory_experimental/data/key_ignores_path/hierarchy/team-b/prod/_config.yaml +++ /dev/null @@ -1,19 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# yaml-language-server: $schema=../../../../schemas/folder.schema.json - -name: Production -tag_bindings: - environment: environment/production \ No newline at end of file diff --git a/tests/modules/project_factory_experimental/data/key_ignores_path/projects/dev-tb-0.yaml b/tests/modules/project_factory_experimental/data/key_ignores_path/projects/dev-tb-0.yaml deleted file mode 100644 index 655c55547..000000000 --- a/tests/modules/project_factory_experimental/data/key_ignores_path/projects/dev-tb-0.yaml +++ /dev/null @@ -1,21 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# yaml-language-server: $schema=../../../../../../modules/project-factory/schemas/project.schema.json - -parent: team-b/dev -shared_vpc_service_config: - host_project: dev-spoke-0 - network_users: - - gcp-devops \ No newline at end of file diff --git a/tests/modules/project_factory_experimental/data/key_ignores_path/projects/prod-tb-0.yaml b/tests/modules/project_factory_experimental/data/key_ignores_path/projects/prod-tb-0.yaml deleted file mode 100644 index dbb0ceb05..000000000 --- a/tests/modules/project_factory_experimental/data/key_ignores_path/projects/prod-tb-0.yaml +++ /dev/null @@ -1,21 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# yaml-language-server: $schema=../../../../../../modules/project-factory/schemas/project.schema.json - -parent: team-b/prod -shared_vpc_service_config: - host_project: prod-spoke-0 - network_users: - - gcp-devops \ No newline at end of file diff --git a/tests/modules/project_factory_experimental/data/key_ignores_path/projects/team-a/dev-ta-0.yaml b/tests/modules/project_factory_experimental/data/key_ignores_path/projects/team-a/dev-ta-0.yaml deleted file mode 100644 index d6367411a..000000000 --- a/tests/modules/project_factory_experimental/data/key_ignores_path/projects/team-a/dev-ta-0.yaml +++ /dev/null @@ -1,21 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# yaml-language-server: $schema=../../../../../../../modules/project-factory/schemas/project.schema.json - -parent: team-a/dev -shared_vpc_service_config: - host_project: dev-spoke-0 - network_users: - - gcp-devops diff --git a/tests/modules/project_factory_experimental/data/key_ignores_path/projects/team-a/prod-ta-0.yaml b/tests/modules/project_factory_experimental/data/key_ignores_path/projects/team-a/prod-ta-0.yaml deleted file mode 100644 index e8ac47ce9..000000000 --- a/tests/modules/project_factory_experimental/data/key_ignores_path/projects/team-a/prod-ta-0.yaml +++ /dev/null @@ -1,21 +0,0 @@ -# Copyright 2024 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -# yaml-language-server: $schema=../../../../../../../modules/project-factory/schemas/project.schema.json - -parent: team-a/prod -shared_vpc_service_config: - host_project: prod-spoke-0 - network_users: - - gcp-devops \ No newline at end of file diff --git a/tests/modules/project_factory_experimental/data/shared_vpc_network_user/projects/service1.yaml b/tests/modules/project_factory_experimental/data/shared_vpc_network_user/projects/service1.yaml deleted file mode 100644 index 5f38e4606..000000000 --- a/tests/modules/project_factory_experimental/data/shared_vpc_network_user/projects/service1.yaml +++ /dev/null @@ -1,35 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -billing_account: 012345-67890A-BCDEF0 - -service_accounts: - app-be-0: {} - terraform-rw: {} - -automation: - project: service-iac - service_accounts: - rw: - description: Service read/write automation sa. - ro: - description: Service read-only automation sa. - - -shared_vpc_service_config: - host_project: dev-spoke-0 - network_users: - - terraform-rw - - ro - - rw diff --git a/tests/modules/project_factory_experimental/data/shared_vpc_network_user/projects/service2.yaml b/tests/modules/project_factory_experimental/data/shared_vpc_network_user/projects/service2.yaml deleted file mode 100644 index 64adb5bb8..000000000 --- a/tests/modules/project_factory_experimental/data/shared_vpc_network_user/projects/service2.yaml +++ /dev/null @@ -1,22 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -billing_account: 012345-67890A-BCDEF0 -services: - - compute.googleapis.com - - storage.googleapis.com - -service_accounts: - app-be-0: {} - terraform-rw: {} diff --git a/tests/modules/project_factory_experimental/data_overrides_defaults.tfvars b/tests/modules/project_factory_experimental/data_overrides_defaults.tfvars deleted file mode 100644 index 168bd8f0b..000000000 --- a/tests/modules/project_factory_experimental/data_overrides_defaults.tfvars +++ /dev/null @@ -1,51 +0,0 @@ -data_defaults = { - billing_account = "1245-5678-9012" - parent = "folders/1234" - storage_location = "EU" - contacts = { - "admin-default@example.org" = ["ALL"] # should not surface, as overrides provide value - } - tag_bindings = { # should not surface, as overrides provide empty value - name1 = "default-id1" - name2 = "default-id2" - } - services = [ - "default-service.googleapis.com" - ] -} -# make sure the environment label and stackdriver service are always added -data_merges = { - labels = { - environment = "test" - } - services = [ - "stackdriver.googleapis.com" - ] -} -# always use this contacts and prefix, regardless of what is in the yaml file -data_overrides = { - contacts = { - "admin@example.org" = ["ALL"] - } - tag_bindings = {} # prevent setting any encryption keys - prefix = "test-pf" -} -# location where the yaml files are read from -factories_config = { - projects_data_path = "projects" - context = { - folder_ids = { - default = "folders/5678901234" - teams = "folders/5678901234" - } - iam_principals = { - gcp-devops = "group:gcp-devops@example.org" - } - tag_values = { - "org-policies/drs-allow-all" = "tagValues/123456" - } - vpc_host_projects = { - dev-spoke-0 = "test-pf-dev-net-spoke-0" - } - } -} diff --git a/tests/modules/project_factory_experimental/data_overrides_defaults.yaml b/tests/modules/project_factory_experimental/data_overrides_defaults.yaml deleted file mode 100644 index 9250d9189..000000000 --- a/tests/modules/project_factory_experimental/data_overrides_defaults.yaml +++ /dev/null @@ -1,63 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -values: - module.projects["service1"].google_essential_contacts_contact.contact["admin@example.org"]: - email: admin@example.org - parent: projects/test-pf-service1 - module.projects["service1"].google_project.project[0]: - billing_account: 012345-67890A-BCDEF0 - folder_id: '1234' - labels: - environment: test - name: test-pf-service1 - project_id: test-pf-service1 - module.projects["service1"].google_project_service.project_services["run.googleapis.com"]: - project: test-pf-service1 - service: run.googleapis.com - module.projects["service1"].google_project_service.project_services["stackdriver.googleapis.com"]: - project: test-pf-service1 - service: stackdriver.googleapis.com - module.projects["service2"].google_essential_contacts_contact.contact["admin@example.org"]: - email: admin@example.org - parent: projects/test-pf-service2 - module.projects["service2"].google_project.project[0]: - billing_account: 012345-67890A-BCDEF0 - folder_id: '1234' - labels: - environment: test - name: test-pf-service2 - project_id: test-pf-service2 - module.projects["service2"].google_project_service.project_services["default-service.googleapis.com"]: - project: test-pf-service2 - service: default-service.googleapis.com - module.projects["service2"].google_project_service.project_services["stackdriver.googleapis.com"]: - project: test-pf-service2 - service: stackdriver.googleapis.com - -counts: - google_essential_contacts_contact: 2 - google_project: 2 - google_project_iam_member: 1 - google_project_service: 4 - google_project_service_identity: 1 - google_tags_tag_binding: 0 # keep this, to ensure that tag_bindings are not created - modules: 2 - resources: 10 - -outputs: - buckets: {} - folders: {} - projects: __missing__ - service_accounts: {} diff --git a/tests/modules/project_factory_experimental/empty_vpc_defaults.tfvars b/tests/modules/project_factory_experimental/empty_vpc_defaults.tfvars deleted file mode 100644 index acf42163e..000000000 --- a/tests/modules/project_factory_experimental/empty_vpc_defaults.tfvars +++ /dev/null @@ -1,42 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -data_defaults = { - billing_account = "1245-5678-9012" - storage_location = "EU" - prefix = "my-prefix" - parent = "folders/1234" - shared_vpc_service_config = null -} -data_merges = { - services = [ - "stackdriver.googleapis.com" - ] -} -data_overrides = { - prefix = "myprefix" -} -# location where the yaml files are read from -factories_config = { - projects_data_path = "projects" - context = { - folder_ids = { - default = "folders/5678901234" - teams = "folders/4321056789" - } - vpc_host_projects = { - dev-spoke-0 = "test-pf-dev-net-spoke-0" - } - } -} diff --git a/tests/modules/project_factory_experimental/empty_vpc_defaults.yaml b/tests/modules/project_factory_experimental/empty_vpc_defaults.yaml deleted file mode 100644 index c45202289..000000000 --- a/tests/modules/project_factory_experimental/empty_vpc_defaults.yaml +++ /dev/null @@ -1,173 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -values: - module.automation-service-accounts["service1/ro"].google_service_account.service_account[0]: - account_id: myprefix-service1-ro - create_ignore_already_exists: null - description: Service read-only automation sa. - disabled: false - display_name: Service account ro for service1. - email: myprefix-service1-ro@service-iac.iam.gserviceaccount.com - member: serviceAccount:myprefix-service1-ro@service-iac.iam.gserviceaccount.com - project: service-iac - timeouts: null - module.automation-service-accounts["service1/rw"].google_service_account.service_account[0]: - account_id: myprefix-service1-rw - create_ignore_already_exists: null - description: Service read/write automation sa. - disabled: false - display_name: Service account rw for service1. - email: myprefix-service1-rw@service-iac.iam.gserviceaccount.com - member: serviceAccount:myprefix-service1-rw@service-iac.iam.gserviceaccount.com - project: service-iac - timeouts: null - module.projects-iam["service1"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: test-pf-dev-net-spoke-0 - service_project: myprefix-service1 - timeouts: null - ? module.projects-iam["service1"].google_project_iam_member.shared_vpc_host_iam["serviceAccount:myprefix-service1-ro@service-iac.iam.gserviceaccount.com"] - : condition: [] - member: serviceAccount:myprefix-service1-ro@service-iac.iam.gserviceaccount.com - project: test-pf-dev-net-spoke-0 - role: roles/compute.networkUser - ? module.projects-iam["service1"].google_project_iam_member.shared_vpc_host_iam["serviceAccount:myprefix-service1-rw@service-iac.iam.gserviceaccount.com"] - : condition: [] - member: serviceAccount:myprefix-service1-rw@service-iac.iam.gserviceaccount.com - project: test-pf-dev-net-spoke-0 - role: roles/compute.networkUser - ? module.projects-iam["service1"].google_project_iam_member.shared_vpc_host_iam["serviceAccount:terraform-rw@myprefix-service1.iam.gserviceaccount.com"] - : condition: [] - member: serviceAccount:terraform-rw@myprefix-service1.iam.gserviceaccount.com - project: test-pf-dev-net-spoke-0 - role: roles/compute.networkUser - module.projects["service1"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - folder_id: '1234' - labels: null - name: myprefix-service1 - org_id: null - project_id: myprefix-service1 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["service1"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: myprefix-service1 - service: stackdriver.googleapis.com - timeouts: null - module.projects["service2"].data.google_storage_project_service_account.gcs_sa[0]: - project: myprefix-service2 - user_project: null - module.projects["service2"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - folder_id: '1234' - labels: null - name: myprefix-service2 - org_id: null - project_id: myprefix-service2 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["service2"].google_project_iam_member.service_agents["compute-system"]: - condition: [] - project: myprefix-service2 - role: roles/compute.serviceAgent - module.projects["service2"].google_project_service.project_services["compute.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: myprefix-service2 - service: compute.googleapis.com - timeouts: null - module.projects["service2"].google_project_service.project_services["stackdriver.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: myprefix-service2 - service: stackdriver.googleapis.com - timeouts: null - module.projects["service2"].google_project_service.project_services["storage.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: myprefix-service2 - service: storage.googleapis.com - timeouts: null - module.service-accounts["service1/app-be-0"].google_service_account.service_account[0]: - account_id: app-be-0 - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: app-be-0@myprefix-service1.iam.gserviceaccount.com - member: serviceAccount:app-be-0@myprefix-service1.iam.gserviceaccount.com - project: myprefix-service1 - timeouts: null - module.service-accounts["service1/terraform-rw"].google_service_account.service_account[0]: - account_id: terraform-rw - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: terraform-rw@myprefix-service1.iam.gserviceaccount.com - member: serviceAccount:terraform-rw@myprefix-service1.iam.gserviceaccount.com - project: myprefix-service1 - timeouts: null - module.service-accounts["service2/app-be-0"].google_service_account.service_account[0]: - account_id: app-be-0 - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: app-be-0@myprefix-service2.iam.gserviceaccount.com - member: serviceAccount:app-be-0@myprefix-service2.iam.gserviceaccount.com - project: myprefix-service2 - timeouts: null - module.service-accounts["service2/terraform-rw"].google_service_account.service_account[0]: - account_id: terraform-rw - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: terraform-rw@myprefix-service2.iam.gserviceaccount.com - member: serviceAccount:terraform-rw@myprefix-service2.iam.gserviceaccount.com - project: myprefix-service2 - timeouts: null - -counts: - google_compute_shared_vpc_service_project: 1 - google_project: 2 - google_project_iam_member: 4 - google_project_service: 4 - google_service_account: 6 - google_storage_project_service_account: 1 - modules: 9 - resources: 18 - -outputs: - buckets: {} - folders: {} - foo: {} - projects: __missing__ - service_accounts: __missing__ diff --git a/tests/modules/project_factory_experimental/key_ignores_path.tfvars b/tests/modules/project_factory_experimental/key_ignores_path.tfvars deleted file mode 100644 index 80876e14b..000000000 --- a/tests/modules/project_factory_experimental/key_ignores_path.tfvars +++ /dev/null @@ -1,40 +0,0 @@ -data_defaults = { - billing_account = "1245-5678-9012" - parent = "folders/1234" - storage_location = "EU" - contacts = { - "admin-default@example.org" = ["ALL"] - } - tag_bindings = { - name1 = "default-id1" - name2 = "default-id2" - } - services = [ - "default-service.googleapis.com" - ] -} -data_overrides = { - prefix = "test-pf" -} -factories_config = { - folders_data_path = "key_ignores_path/hierarchy" - projects_data_path = "key_ignores_path/projects" - projects_config = { - key_ignores_path = true - } - context = { - folder_ids = { - default = "folders/5678901234" - teams = "folders/5678901234" - } - iam_principals = { - gcp-devops = "group:gcp-devops@example.org" - } - tag_values = { - "org-policies/drs-allow-all" = "tagValues/123456" - } - vpc_host_projects = { - dev-spoke-0 = "test-pf-dev-net-spoke-0" - } - } -} diff --git a/tests/modules/project_factory_experimental/key_ignores_path.yaml b/tests/modules/project_factory_experimental/key_ignores_path.yaml deleted file mode 100644 index 4c5cb0ae4..000000000 --- a/tests/modules/project_factory_experimental/key_ignores_path.yaml +++ /dev/null @@ -1,238 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -values: - module.hierarchy-folder-lvl-1["team-a"].google_folder.folder[0]: - deletion_protection: false - display_name: Team A - parent: folders/5678901234 - tags: null - timeouts: null - module.hierarchy-folder-lvl-1["team-b"].google_folder.folder[0]: - deletion_protection: false - display_name: Team B - parent: folders/5678901234 - tags: null - timeouts: null - module.hierarchy-folder-lvl-2["team-a/dev"].google_folder.folder[0]: - deletion_protection: false - display_name: Development - tags: null - timeouts: null - module.hierarchy-folder-lvl-2["team-a/dev"].google_tags_tag_binding.binding["environment"]: - tag_value: environment/development - timeouts: null - module.hierarchy-folder-lvl-2["team-a/prod"].google_folder.folder[0]: - deletion_protection: false - display_name: Production - tags: null - timeouts: null - module.hierarchy-folder-lvl-2["team-a/prod"].google_tags_tag_binding.binding["environment"]: - tag_value: environment/production - timeouts: null - module.hierarchy-folder-lvl-2["team-b/dev"].google_folder.folder[0]: - deletion_protection: false - display_name: Development - tags: null - timeouts: null - module.hierarchy-folder-lvl-2["team-b/dev"].google_tags_tag_binding.binding["environment"]: - tag_value: environment/development - timeouts: null - module.hierarchy-folder-lvl-2["team-b/prod"].google_folder.folder[0]: - deletion_protection: false - display_name: Production - tags: null - timeouts: null - module.hierarchy-folder-lvl-2["team-b/prod"].google_tags_tag_binding.binding["environment"]: - tag_value: environment/production - timeouts: null - module.projects-iam["dev-ta-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: test-pf-dev-net-spoke-0 - service_project: test-pf-dev-ta-0 - timeouts: null - module.projects-iam["dev-ta-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"]: - condition: [] - member: group:gcp-devops@example.org - project: test-pf-dev-net-spoke-0 - role: roles/compute.networkUser - module.projects-iam["dev-tb-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: test-pf-dev-net-spoke-0 - service_project: test-pf-dev-tb-0 - timeouts: null - module.projects-iam["dev-tb-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"]: - condition: [] - member: group:gcp-devops@example.org - project: test-pf-dev-net-spoke-0 - role: roles/compute.networkUser - module.projects-iam["prod-ta-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: prod-spoke-0 - service_project: test-pf-prod-ta-0 - timeouts: null - module.projects-iam["prod-ta-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"]: - condition: [] - member: group:gcp-devops@example.org - project: prod-spoke-0 - role: roles/compute.networkUser - module.projects-iam["prod-tb-0"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: prod-spoke-0 - service_project: test-pf-prod-tb-0 - timeouts: null - module.projects-iam["prod-tb-0"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"]: - condition: [] - member: group:gcp-devops@example.org - project: prod-spoke-0 - role: roles/compute.networkUser - module.projects["dev-ta-0"].google_essential_contacts_contact.contact["admin-default@example.org"]: - email: admin-default@example.org - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-dev-ta-0 - timeouts: null - module.projects["dev-ta-0"].google_project.project[0]: - auto_create_network: false - billing_account: 1245-5678-9012 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - labels: null - name: test-pf-dev-ta-0 - project_id: test-pf-dev-ta-0 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["dev-ta-0"].google_project_service.project_services["default-service.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-dev-ta-0 - service: default-service.googleapis.com - timeouts: null - module.projects["dev-ta-0"].google_tags_tag_binding.binding["name1"]: - tag_value: default-id1 - timeouts: null - module.projects["dev-ta-0"].google_tags_tag_binding.binding["name2"]: - tag_value: default-id2 - timeouts: null - module.projects["dev-tb-0"].google_essential_contacts_contact.contact["admin-default@example.org"]: - email: admin-default@example.org - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-dev-tb-0 - timeouts: null - module.projects["dev-tb-0"].google_project.project[0]: - auto_create_network: false - billing_account: 1245-5678-9012 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - labels: null - name: test-pf-dev-tb-0 - project_id: test-pf-dev-tb-0 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["dev-tb-0"].google_project_service.project_services["default-service.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-dev-tb-0 - service: default-service.googleapis.com - timeouts: null - module.projects["dev-tb-0"].google_tags_tag_binding.binding["name1"]: - tag_value: default-id1 - timeouts: null - module.projects["dev-tb-0"].google_tags_tag_binding.binding["name2"]: - tag_value: default-id2 - timeouts: null - module.projects["prod-ta-0"].google_essential_contacts_contact.contact["admin-default@example.org"]: - email: admin-default@example.org - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-prod-ta-0 - timeouts: null - module.projects["prod-ta-0"].google_project.project[0]: - auto_create_network: false - billing_account: 1245-5678-9012 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - labels: null - name: test-pf-prod-ta-0 - project_id: test-pf-prod-ta-0 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["prod-ta-0"].google_project_service.project_services["default-service.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-prod-ta-0 - service: default-service.googleapis.com - timeouts: null - module.projects["prod-ta-0"].google_tags_tag_binding.binding["name1"]: - tag_value: default-id1 - timeouts: null - module.projects["prod-ta-0"].google_tags_tag_binding.binding["name2"]: - tag_value: default-id2 - timeouts: null - module.projects["prod-tb-0"].google_essential_contacts_contact.contact["admin-default@example.org"]: - email: admin-default@example.org - language_tag: en - notification_category_subscriptions: - - ALL - parent: projects/test-pf-prod-tb-0 - timeouts: null - module.projects["prod-tb-0"].google_project.project[0]: - auto_create_network: false - billing_account: 1245-5678-9012 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - labels: null - name: test-pf-prod-tb-0 - project_id: test-pf-prod-tb-0 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["prod-tb-0"].google_project_service.project_services["default-service.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-prod-tb-0 - service: default-service.googleapis.com - timeouts: null - module.projects["prod-tb-0"].google_tags_tag_binding.binding["name1"]: - tag_value: default-id1 - timeouts: null - module.projects["prod-tb-0"].google_tags_tag_binding.binding["name2"]: - tag_value: default-id2 - timeouts: null - -counts: - google_compute_shared_vpc_service_project: 4 - google_essential_contacts_contact: 4 - google_folder: 6 - google_project: 4 - google_project_iam_member: 4 - google_project_service: 4 - google_tags_tag_binding: 12 - modules: 14 - resources: 38 diff --git a/tests/modules/project_factory_experimental/shared_vpc_network_user.tfvars b/tests/modules/project_factory_experimental/shared_vpc_network_user.tfvars deleted file mode 100644 index 6eff1d58e..000000000 --- a/tests/modules/project_factory_experimental/shared_vpc_network_user.tfvars +++ /dev/null @@ -1,39 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -data_defaults = { - billing_account = "1245-5678-9012" - storage_location = "EU" - prefix = "my-prefix" - parent = "folders/1234" -} -# location where the yaml files are read from -factories_config = { - projects_data_path = "projects" - context = { - folder_ids = { - default = "folders/5678901234" - teams = "folders/5678901234" - } - iam_principals = { - gcp-devops = "group:gcp-devops@example.org" - } - tag_values = { - "org-policies/drs-allow-all" = "tagValues/123456" - } - vpc_host_projects = { - dev-spoke-0 = "test-pf-dev-net-spoke-0" - } - } -} diff --git a/tests/modules/project_factory_experimental/shared_vpc_network_user.yaml b/tests/modules/project_factory_experimental/shared_vpc_network_user.yaml deleted file mode 100644 index 35418a119..000000000 --- a/tests/modules/project_factory_experimental/shared_vpc_network_user.yaml +++ /dev/null @@ -1,160 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -values: - module.automation-service-accounts["service1/automation/ro"].google_service_account.service_account[0]: - account_id: my-prefix-service1-ro - create_ignore_already_exists: null - description: Service read-only automation sa. - disabled: false - display_name: Service account ro for service1. - email: my-prefix-service1-ro@service-iac.iam.gserviceaccount.com - member: serviceAccount:my-prefix-service1-ro@service-iac.iam.gserviceaccount.com - project: service-iac - timeouts: null - module.automation-service-accounts["service1/automation/rw"].google_service_account.service_account[0]: - account_id: my-prefix-service1-rw - create_ignore_already_exists: null - description: Service read/write automation sa. - disabled: false - display_name: Service account rw for service1. - email: my-prefix-service1-rw@service-iac.iam.gserviceaccount.com - member: serviceAccount:my-prefix-service1-rw@service-iac.iam.gserviceaccount.com - project: service-iac - timeouts: null - module.projects-iam["service1"].google_compute_shared_vpc_service_project.shared_vpc_service[0]: - deletion_policy: null - host_project: test-pf-dev-net-spoke-0 - service_project: my-prefix-service1 - timeouts: null - ? module.projects-iam["service1"].google_project_iam_member.shared_vpc_host_iam["serviceAccount:my-prefix-service1-ro@service-iac.iam.gserviceaccount.com"] - : condition: [] - member: serviceAccount:my-prefix-service1-ro@service-iac.iam.gserviceaccount.com - project: test-pf-dev-net-spoke-0 - role: roles/compute.networkUser - ? module.projects-iam["service1"].google_project_iam_member.shared_vpc_host_iam["serviceAccount:my-prefix-service1-rw@service-iac.iam.gserviceaccount.com"] - : condition: [] - member: serviceAccount:my-prefix-service1-rw@service-iac.iam.gserviceaccount.com - project: test-pf-dev-net-spoke-0 - role: roles/compute.networkUser - ? module.projects-iam["service1"].google_project_iam_member.shared_vpc_host_iam["serviceAccount:terraform-rw@my-prefix-service1.iam.gserviceaccount.com"] - : condition: [] - member: serviceAccount:terraform-rw@my-prefix-service1.iam.gserviceaccount.com - project: test-pf-dev-net-spoke-0 - role: roles/compute.networkUser - module.projects["service1"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - folder_id: '1234' - labels: null - name: my-prefix-service1 - org_id: null - project_id: my-prefix-service1 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["service2"].data.google_storage_project_service_account.gcs_sa[0]: - project: my-prefix-service2 - user_project: null - module.projects["service2"].google_project.project[0]: - auto_create_network: false - billing_account: 012345-67890A-BCDEF0 - deletion_policy: DELETE - effective_labels: - goog-terraform-provisioned: 'true' - folder_id: '1234' - labels: null - name: my-prefix-service2 - org_id: null - project_id: my-prefix-service2 - tags: null - terraform_labels: - goog-terraform-provisioned: 'true' - timeouts: null - module.projects["service2"].google_project_iam_member.service_agents["compute-system"]: - condition: [] - project: my-prefix-service2 - role: roles/compute.serviceAgent - module.projects["service2"].google_project_service.project_services["compute.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: my-prefix-service2 - service: compute.googleapis.com - timeouts: null - module.projects["service2"].google_project_service.project_services["storage.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: my-prefix-service2 - service: storage.googleapis.com - timeouts: null - module.service-accounts["service1/app-be-0"].google_service_account.service_account[0]: - account_id: app-be-0 - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: app-be-0@my-prefix-service1.iam.gserviceaccount.com - member: serviceAccount:app-be-0@my-prefix-service1.iam.gserviceaccount.com - project: my-prefix-service1 - timeouts: null - module.service-accounts["service1/terraform-rw"].google_service_account.service_account[0]: - account_id: terraform-rw - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: terraform-rw@my-prefix-service1.iam.gserviceaccount.com - member: serviceAccount:terraform-rw@my-prefix-service1.iam.gserviceaccount.com - project: my-prefix-service1 - timeouts: null - module.service-accounts["service2/app-be-0"].google_service_account.service_account[0]: - account_id: app-be-0 - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: app-be-0@my-prefix-service2.iam.gserviceaccount.com - member: serviceAccount:app-be-0@my-prefix-service2.iam.gserviceaccount.com - project: my-prefix-service2 - timeouts: null - module.service-accounts["service2/terraform-rw"].google_service_account.service_account[0]: - account_id: terraform-rw - create_ignore_already_exists: null - description: null - disabled: false - display_name: Terraform-managed. - email: terraform-rw@my-prefix-service2.iam.gserviceaccount.com - member: serviceAccount:terraform-rw@my-prefix-service2.iam.gserviceaccount.com - project: my-prefix-service2 - timeouts: null - -counts: - google_compute_shared_vpc_service_project: 1 - google_project: 2 - google_project_iam_member: 4 - google_project_service: 2 - google_service_account: 6 - google_storage_project_service_account: 1 - modules: 9 - resources: 16 - -outputs: - buckets: {} - folders: {} - projects: __missing__ - service_accounts: __missing__ diff --git a/tests/modules/project_factory_experimental/tftest.yaml b/tests/modules/project_factory_experimental/tftest.yaml deleted file mode 100644 index d47232354..000000000 --- a/tests/modules/project_factory_experimental/tftest.yaml +++ /dev/null @@ -1,29 +0,0 @@ -# Copyright 2025 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -module: modules/project-factory - -tests: - bucket_iam: - extra_dirs: - - ../../tests/modules/project_factory/data/bucket_iam - shared_vpc_network_user: - extra_dirs: - - ../../tests/modules/project_factory/data/shared_vpc_network_user/projects - data_overrides_defaults: - extra_dirs: - - ../../tests/modules/project_factory/data/data_overrides_defaults/projects - key_ignores_path: - extra_dirs: - - ../../tests/modules/project_factory/data/key_ignores_path diff --git a/tests/modules/project_factory/bucket_iam.tfvars b/tests/modules/project_factory_legacy/bucket_iam.tfvars similarity index 100% rename from tests/modules/project_factory/bucket_iam.tfvars rename to tests/modules/project_factory_legacy/bucket_iam.tfvars diff --git a/tests/modules/project_factory/bucket_iam.yaml b/tests/modules/project_factory_legacy/bucket_iam.yaml similarity index 100% rename from tests/modules/project_factory/bucket_iam.yaml rename to tests/modules/project_factory_legacy/bucket_iam.yaml diff --git a/tests/modules/project_factory/data/bucket_iam/hierarchy/team-a/_config.yaml b/tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-a/_config.yaml similarity index 100% rename from tests/modules/project_factory/data/bucket_iam/hierarchy/team-a/_config.yaml rename to tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-a/_config.yaml diff --git a/tests/modules/project_factory/data/bucket_iam/hierarchy/team-a/automation.yaml b/tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-a/automation.yaml similarity index 100% rename from tests/modules/project_factory/data/bucket_iam/hierarchy/team-a/automation.yaml rename to tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-a/automation.yaml diff --git a/tests/modules/project_factory/data/bucket_iam/hierarchy/team-a/project1.yaml b/tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-a/project1.yaml similarity index 100% rename from tests/modules/project_factory/data/bucket_iam/hierarchy/team-a/project1.yaml rename to tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-a/project1.yaml diff --git a/tests/modules/project_factory/data/bucket_iam/hierarchy/team-b/_config.yaml b/tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-b/_config.yaml similarity index 100% rename from tests/modules/project_factory/data/bucket_iam/hierarchy/team-b/_config.yaml rename to tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-b/_config.yaml diff --git a/tests/modules/project_factory/data/bucket_iam/hierarchy/team-b/automation.yaml b/tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-b/automation.yaml similarity index 100% rename from tests/modules/project_factory/data/bucket_iam/hierarchy/team-b/automation.yaml rename to tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-b/automation.yaml diff --git a/tests/modules/project_factory/data/bucket_iam/hierarchy/team-b/project3.yaml b/tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-b/project3.yaml similarity index 100% rename from tests/modules/project_factory/data/bucket_iam/hierarchy/team-b/project3.yaml rename to tests/modules/project_factory_legacy/data/bucket_iam/hierarchy/team-b/project3.yaml diff --git a/tests/modules/project_factory/data/bucket_iam/projects/project2.yaml b/tests/modules/project_factory_legacy/data/bucket_iam/projects/project2.yaml similarity index 100% rename from tests/modules/project_factory/data/bucket_iam/projects/project2.yaml rename to tests/modules/project_factory_legacy/data/bucket_iam/projects/project2.yaml diff --git a/tests/modules/project_factory/data/bucket_iam/projects/project3.yaml b/tests/modules/project_factory_legacy/data/bucket_iam/projects/project3.yaml similarity index 100% rename from tests/modules/project_factory/data/bucket_iam/projects/project3.yaml rename to tests/modules/project_factory_legacy/data/bucket_iam/projects/project3.yaml diff --git a/tests/modules/project_factory/data/data_overrides_defaults/projects/service1.yaml b/tests/modules/project_factory_legacy/data/data_overrides_defaults/projects/service1.yaml similarity index 100% rename from tests/modules/project_factory/data/data_overrides_defaults/projects/service1.yaml rename to tests/modules/project_factory_legacy/data/data_overrides_defaults/projects/service1.yaml diff --git a/tests/modules/project_factory/data/data_overrides_defaults/projects/service2.yaml b/tests/modules/project_factory_legacy/data/data_overrides_defaults/projects/service2.yaml similarity index 100% rename from tests/modules/project_factory/data/data_overrides_defaults/projects/service2.yaml rename to tests/modules/project_factory_legacy/data/data_overrides_defaults/projects/service2.yaml diff --git a/tests/modules/project_factory/data/key_ignores_path/hierarchy/team-a/_config.yaml b/tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-a/_config.yaml similarity index 100% rename from tests/modules/project_factory/data/key_ignores_path/hierarchy/team-a/_config.yaml rename to tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-a/_config.yaml diff --git a/tests/modules/project_factory/data/key_ignores_path/hierarchy/team-a/dev/_config.yaml b/tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-a/dev/_config.yaml similarity index 100% rename from tests/modules/project_factory/data/key_ignores_path/hierarchy/team-a/dev/_config.yaml rename to tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-a/dev/_config.yaml diff --git a/tests/modules/project_factory/data/key_ignores_path/hierarchy/team-a/prod/_config.yaml b/tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-a/prod/_config.yaml similarity index 100% rename from tests/modules/project_factory/data/key_ignores_path/hierarchy/team-a/prod/_config.yaml rename to tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-a/prod/_config.yaml diff --git a/tests/modules/project_factory/data/key_ignores_path/hierarchy/team-b/_config.yaml b/tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-b/_config.yaml similarity index 100% rename from tests/modules/project_factory/data/key_ignores_path/hierarchy/team-b/_config.yaml rename to tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-b/_config.yaml diff --git a/tests/modules/project_factory/data/key_ignores_path/hierarchy/team-b/dev/_config.yaml b/tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-b/dev/_config.yaml similarity index 100% rename from tests/modules/project_factory/data/key_ignores_path/hierarchy/team-b/dev/_config.yaml rename to tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-b/dev/_config.yaml diff --git a/tests/modules/project_factory/data/key_ignores_path/hierarchy/team-b/prod/_config.yaml b/tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-b/prod/_config.yaml similarity index 100% rename from tests/modules/project_factory/data/key_ignores_path/hierarchy/team-b/prod/_config.yaml rename to tests/modules/project_factory_legacy/data/key_ignores_path/hierarchy/team-b/prod/_config.yaml diff --git a/tests/modules/project_factory/data/key_ignores_path/projects/dev-tb-0.yaml b/tests/modules/project_factory_legacy/data/key_ignores_path/projects/dev-tb-0.yaml similarity index 100% rename from tests/modules/project_factory/data/key_ignores_path/projects/dev-tb-0.yaml rename to tests/modules/project_factory_legacy/data/key_ignores_path/projects/dev-tb-0.yaml diff --git a/tests/modules/project_factory/data/key_ignores_path/projects/prod-tb-0.yaml b/tests/modules/project_factory_legacy/data/key_ignores_path/projects/prod-tb-0.yaml similarity index 100% rename from tests/modules/project_factory/data/key_ignores_path/projects/prod-tb-0.yaml rename to tests/modules/project_factory_legacy/data/key_ignores_path/projects/prod-tb-0.yaml diff --git a/tests/modules/project_factory/data/key_ignores_path/projects/team-a/dev-ta-0.yaml b/tests/modules/project_factory_legacy/data/key_ignores_path/projects/team-a/dev-ta-0.yaml similarity index 100% rename from tests/modules/project_factory/data/key_ignores_path/projects/team-a/dev-ta-0.yaml rename to tests/modules/project_factory_legacy/data/key_ignores_path/projects/team-a/dev-ta-0.yaml diff --git a/tests/modules/project_factory/data/key_ignores_path/projects/team-a/prod-ta-0.yaml b/tests/modules/project_factory_legacy/data/key_ignores_path/projects/team-a/prod-ta-0.yaml similarity index 100% rename from tests/modules/project_factory/data/key_ignores_path/projects/team-a/prod-ta-0.yaml rename to tests/modules/project_factory_legacy/data/key_ignores_path/projects/team-a/prod-ta-0.yaml diff --git a/tests/modules/project_factory/data/shared_vpc_network_user/projects/service1.yaml b/tests/modules/project_factory_legacy/data/shared_vpc_network_user/projects/service1.yaml similarity index 100% rename from tests/modules/project_factory/data/shared_vpc_network_user/projects/service1.yaml rename to tests/modules/project_factory_legacy/data/shared_vpc_network_user/projects/service1.yaml diff --git a/tests/modules/project_factory/data/shared_vpc_network_user/projects/service2.yaml b/tests/modules/project_factory_legacy/data/shared_vpc_network_user/projects/service2.yaml similarity index 100% rename from tests/modules/project_factory/data/shared_vpc_network_user/projects/service2.yaml rename to tests/modules/project_factory_legacy/data/shared_vpc_network_user/projects/service2.yaml diff --git a/tests/modules/project_factory/data_overrides_defaults.tfvars b/tests/modules/project_factory_legacy/data_overrides_defaults.tfvars similarity index 100% rename from tests/modules/project_factory/data_overrides_defaults.tfvars rename to tests/modules/project_factory_legacy/data_overrides_defaults.tfvars diff --git a/tests/modules/project_factory/data_overrides_defaults.yaml b/tests/modules/project_factory_legacy/data_overrides_defaults.yaml similarity index 100% rename from tests/modules/project_factory/data_overrides_defaults.yaml rename to tests/modules/project_factory_legacy/data_overrides_defaults.yaml diff --git a/tests/modules/project_factory/empty_vpc_defaults.tfvars b/tests/modules/project_factory_legacy/empty_vpc_defaults.tfvars similarity index 100% rename from tests/modules/project_factory/empty_vpc_defaults.tfvars rename to tests/modules/project_factory_legacy/empty_vpc_defaults.tfvars diff --git a/tests/modules/project_factory/empty_vpc_defaults.yaml b/tests/modules/project_factory_legacy/empty_vpc_defaults.yaml similarity index 100% rename from tests/modules/project_factory/empty_vpc_defaults.yaml rename to tests/modules/project_factory_legacy/empty_vpc_defaults.yaml diff --git a/tests/modules/project_factory_experimental/examples/example.yaml b/tests/modules/project_factory_legacy/examples/example.yaml similarity index 81% rename from tests/modules/project_factory_experimental/examples/example.yaml rename to tests/modules/project_factory_legacy/examples/example.yaml index 5828e9832..881813b7a 100644 --- a/tests/modules/project_factory_experimental/examples/example.yaml +++ b/tests/modules/project_factory_legacy/examples/example.yaml @@ -13,7 +13,7 @@ # limitations under the License. values: - module.project-factory.module.automation-bucket["dev-tb-app0-0/tf-state"].google_storage_bucket.bucket[0]: + module.project-factory.module.automation-bucket["dev-tb-app0-0"].google_storage_bucket.bucket[0]: autoclass: [] cors: [] custom_placement_config: [] @@ -24,7 +24,6 @@ values: encryption: [] force_destroy: false hierarchical_namespace: [] - ip_filter: [] labels: null lifecycle_rule: [] location: EU @@ -40,13 +39,13 @@ values: uniform_bucket_level_access: true versioning: - enabled: false - ? module.project-factory.module.automation-bucket["dev-tb-app0-0/tf-state"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectCreator"] + ? module.project-factory.module.automation-bucket["dev-tb-app0-0"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectCreator"] : bucket: test-pf-dev-tb-app0-0-tf-state condition: [] members: - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com role: roles/storage.objectCreator - ? module.project-factory.module.automation-bucket["dev-tb-app0-0/tf-state"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"] + ? module.project-factory.module.automation-bucket["dev-tb-app0-0"].google_storage_bucket_iam_binding.authoritative["roles/storage.objectViewer"] : bucket: test-pf-dev-tb-app0-0-tf-state condition: [] members: @@ -55,8 +54,8 @@ values: - serviceAccount:test-pf-dev-tb-app0-0-ro@test-pf-teams-iac-0.iam.gserviceaccount.com - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com role: roles/storage.objectViewer - module.project-factory.module.automation-service-accounts["dev-tb-app0-0/ro"].google_service_account.service_account[0]: - account_id: test-pf-dev-tb-app0-0-ro + ? module.project-factory.module.automation-service-accounts["dev-tb-app0-0/automation/ro"].google_service_account.service_account[0] + : account_id: test-pf-dev-tb-app0-0-ro create_ignore_already_exists: null description: Team B app 0 read-only automation sa. disabled: false @@ -65,8 +64,8 @@ values: member: serviceAccount:test-pf-dev-tb-app0-0-ro@test-pf-teams-iac-0.iam.gserviceaccount.com project: test-pf-teams-iac-0 timeouts: null - module.project-factory.module.automation-service-accounts["dev-tb-app0-0/rw"].google_service_account.service_account[0]: - account_id: test-pf-dev-tb-app0-0-rw + ? module.project-factory.module.automation-service-accounts["dev-tb-app0-0/automation/rw"].google_service_account.service_account[0] + : account_id: test-pf-dev-tb-app0-0-rw create_ignore_already_exists: null description: Team B app 0 read/write automation sa. disabled: false @@ -75,7 +74,7 @@ values: member: serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com project: test-pf-teams-iac-0 timeouts: null - module.project-factory.module.billing-budgets[0].google_billing_budget.default["test-100"]: + module.project-factory.module.billing-account[0].google_billing_budget.default["test-100"]: all_updates_rule: - disable_default_iam_recipients: true enable_project_level_recipients: false @@ -91,7 +90,6 @@ values: - calendar_period: null credit_types_treatment: INCLUDE_ALL_CREDITS custom_period: [] - projects: null resource_ancestors: - folders/1234567890 display_name: 100 dollars in current spend @@ -102,7 +100,7 @@ values: - spend_basis: CURRENT_SPEND threshold_percent: 0.75 timeouts: null - module.project-factory.module.billing-budgets[0].google_monitoring_notification_channel.default["billing-default"]: + module.project-factory.module.billing-account[0].google_monitoring_notification_channel.default["billing-default"]: description: null display_name: Budget email notification billing-default. enabled: true @@ -114,82 +112,61 @@ values: timeouts: null type: email user_labels: null - module.project-factory.module.folder-1-iam["team-a"].google_folder_iam_binding.authoritative["roles/viewer"]: - condition: [] - members: - - group:gcp-devops@example.org - - group:team-a-admins@example.org - role: roles/viewer - module.project-factory.module.folder-1["team-a"].google_folder.folder[0]: + module.project-factory.module.hierarchy-folder-lvl-1["team-a"].google_folder.folder[0]: deletion_protection: false display_name: Team A parent: folders/5678901234 tags: null timeouts: null - module.project-factory.module.folder-1["team-b"].google_folder.folder[0]: + module.project-factory.module.hierarchy-folder-lvl-1["team-a"].google_folder_iam_binding.authoritative["roles/viewer"]: + condition: [] + members: + - group:gcp-devops@example.org + - group:team-a-admins@example.org + role: roles/viewer + module.project-factory.module.hierarchy-folder-lvl-1["team-b"].google_folder.folder[0]: deletion_protection: false display_name: Team B parent: folders/5678901234 tags: null timeouts: null - module.project-factory.module.folder-1["team-c"].google_folder.folder[0]: + module.project-factory.module.hierarchy-folder-lvl-1["team-c"].google_folder.folder[0]: deletion_protection: false display_name: Team C parent: folders/5678901234 tags: null timeouts: null - module.project-factory.module.folder-2["team-a/app-0"].google_folder.folder[0]: + module.project-factory.module.hierarchy-folder-lvl-2["team-a/app-0"].google_folder.folder[0]: deletion_protection: false display_name: App 0 tags: null timeouts: null - module.project-factory.module.folder-2["team-b/app-0"].google_folder.folder[0]: + module.project-factory.module.hierarchy-folder-lvl-2["team-b/app-0"].google_folder.folder[0]: deletion_protection: false display_name: App 0 tags: null timeouts: null - module.project-factory.module.folder-2["team-b/app-0"].google_tags_tag_binding.binding["drs-allow-all"]: + module.project-factory.module.hierarchy-folder-lvl-2["team-b/app-0"].google_tags_tag_binding.binding["drs-allow-all"]: tag_value: tagValues/123456 timeouts: null ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_compute_shared_vpc_service_project.shared_vpc_service[0] : deletion_policy: null - host_project: $project_ids:dev-spoke-0 + host_project: test-pf-dev-net-spoke-0 service_project: test-pf-dev-ta-app0-be timeouts: null - ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["key-0.compute-system"] - : condition: [] - crypto_key_id: projects/kms-central-prj/locations/europe-west1/keyRings/my-keyring/cryptoKeys/ew1-compute - role: roles/cloudkms.cryptoKeyEncrypterDecrypter - ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["key-0.gs-project-accounts"] - : condition: [] - crypto_key_id: projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce - role: roles/cloudkms.cryptoKeyEncrypterDecrypter - ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_binding.authoritative["roles/cloudkms.cryptoKeyEncrypterDecrypter"] - : condition: [] - project: test-pf-dev-ta-app0-be - role: roles/cloudkms.cryptoKeyEncrypterDecrypter - ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_binding.authoritative["roles/storage.objectViewer"] - : condition: [] - members: - - serviceAccount:app-0-be@test-pf-dev-ta-app0-be.iam.gserviceaccount.com - project: test-pf-dev-ta-app0-be - role: roles/storage.objectViewer - ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_iam["$iam_principals:gcp-devops"] + ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_iam["group:gcp-devops@example.org"] : condition: [] member: group:gcp-devops@example.org - project: $project_ids:dev-spoke-0 + project: test-pf-dev-net-spoke-0 role: roles/compute.networkUser ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_robots["roles/compute.networkUser:container-engine"] : condition: [] - project: $project_ids:dev-spoke-0 + project: test-pf-dev-net-spoke-0 role: roles/compute.networkUser ? module.project-factory.module.projects-iam["dev-ta-app0-be"].google_project_iam_member.shared_vpc_host_robots["roles/container.hostServiceAgentUser:container-engine"] : condition: [] - project: $project_ids:dev-spoke-0 + project: test-pf-dev-net-spoke-0 role: roles/container.hostServiceAgentUser - module.project-factory.module.projects-iam["dev-tb-app0-0"].google_compute_shared_vpc_host_project.shared_vpc_host[0]: - project: test-pf-dev-tb-app0-0 - timeouts: null module.project-factory.module.projects-iam["dev-tb-app0-0"].google_project_iam_binding.authoritative["roles/owner"]: condition: [] members: @@ -224,6 +201,14 @@ values: - ALL parent: projects/test-pf-dev-ta-app0-be timeouts: null + ? module.project-factory.module.projects["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["key-0.compute-system"] + : condition: [] + crypto_key_id: projects/kms-central-prj/locations/europe-west1/keyRings/my-keyring/cryptoKeys/ew1-compute + role: roles/cloudkms.cryptoKeyEncrypterDecrypter + ? module.project-factory.module.projects["dev-ta-app0-be"].google_kms_crypto_key_iam_member.service_agent_cmek["key-0.gs-project-accounts"] + : condition: [] + crypto_key_id: projects/kms-central-prj/locations/europe-west3/keyRings/my-keyring/cryptoKeys/europe3-gce + role: roles/cloudkms.cryptoKeyEncrypterDecrypter module.project-factory.module.projects["dev-ta-app0-be"].google_project.project[0]: auto_create_network: false billing_account: 012345-67890A-BCDEF0 @@ -238,7 +223,6 @@ values: environment: test team: team-a name: test-pf-dev-ta-app0-be - org_id: null project_id: test-pf-dev-ta-app0-be tags: null terraform_labels: @@ -247,10 +231,6 @@ values: goog-terraform-provisioned: 'true' team: team-a timeouts: null - module.project-factory.module.projects["dev-ta-app0-be"].google_project_iam_member.service_agents["compute-system"]: - condition: [] - project: test-pf-dev-ta-app0-be - role: roles/compute.serviceAgent ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_iam_member.service_agents["container-engine-robot"] : condition: [] project: test-pf-dev-ta-app0-be @@ -259,12 +239,6 @@ values: condition: [] project: test-pf-dev-ta-app0-be role: roles/container.defaultNodeServiceAgent - module.project-factory.module.projects["dev-ta-app0-be"].google_project_service.project_services["compute.googleapis.com"]: - disable_dependent_services: false - disable_on_destroy: false - project: test-pf-dev-ta-app0-be - service: compute.googleapis.com - timeouts: null ? module.project-factory.module.projects["dev-ta-app0-be"].google_project_service.project_services["container.googleapis.com"] : disable_dependent_services: false disable_on_destroy: false @@ -310,6 +284,9 @@ values: module.project-factory.module.projects["dev-tb-app0-0"].data.google_storage_project_service_account.gcs_sa[0]: project: test-pf-dev-tb-app0-0 user_project: null + module.project-factory.module.projects["dev-tb-app0-0"].google_compute_shared_vpc_host_project.shared_vpc_host[0]: + project: test-pf-dev-tb-app0-0 + timeouts: null module.project-factory.module.projects["dev-tb-app0-0"].google_essential_contacts_contact.contact["admin@example.org"]: email: admin@example.org language_tag: en @@ -327,7 +304,6 @@ values: labels: environment: test name: test-pf-dev-tb-app0-0 - org_id: null project_id: test-pf-dev-tb-app0-0 tags: null terraform_labels: @@ -384,7 +360,6 @@ values: environment: test team: team-b name: test-pf-dev-tb-app0-1 - org_id: null project_id: test-pf-dev-tb-app0-1 tags: null terraform_labels: @@ -440,7 +415,7 @@ values: effective_labels: environment: test goog-terraform-provisioned: 'true' - folder_id: null + folder_id: '5678901234' labels: environment: test name: test-pf-teams-iac-0 @@ -481,15 +456,15 @@ values: project: test-pf-teams-iac-0 service: container.googleapis.com timeouts: null - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["$project_ids:dev-spoke-0-roles/compute.networkUser"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-net-spoke-0-roles/compute.networkUser"] : condition: [] - project: $project_ids:dev-spoke-0 + project: test-pf-dev-net-spoke-0 role: roles/compute.networkUser - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["$project_ids:dev-ta-app0-be-roles/logging.logWriter"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/logging.logWriter"] : condition: [] project: test-pf-dev-ta-app0-be role: roles/logging.logWriter - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["$project_ids:dev-ta-app0-be-roles/monitoring.metricWriter"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/monitoring.metricWriter"] : condition: [] project: test-pf-dev-ta-app0-be role: roles/monitoring.metricWriter @@ -503,15 +478,15 @@ values: member: serviceAccount:app-0-be@test-pf-dev-ta-app0-be.iam.gserviceaccount.com project: test-pf-dev-ta-app0-be timeouts: null - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["$project_ids:dev-spoke-0-roles/compute.networkUser"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["test-pf-dev-net-spoke-0-roles/compute.networkUser"] : condition: [] - project: $project_ids:dev-spoke-0 + project: test-pf-dev-net-spoke-0 role: roles/compute.networkUser - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["$project_ids:dev-ta-app0-be-roles/logging.logWriter"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/logging.logWriter"] : condition: [] project: test-pf-dev-ta-app0-be role: roles/logging.logWriter - ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["$project_ids:dev-ta-app0-be-roles/monitoring.metricWriter"] + ? module.project-factory.module.service-accounts["dev-ta-app0-be/app-0-fe"].google_project_iam_member.project-roles["test-pf-dev-ta-app0-be-roles/monitoring.metricWriter"] : condition: [] project: test-pf-dev-ta-app0-be role: roles/monitoring.metricWriter @@ -525,13 +500,13 @@ values: member: serviceAccount:app-0-fe@test-pf-dev-ta-app0-be.iam.gserviceaccount.com project: test-pf-dev-ta-app0-be timeouts: null - ? module.project-factory.module.service-accounts["dev-tb-app0-0/vm-default"].google_project_iam_member.project-roles["$project_ids:dev-tb-app0-0-roles/logging.logWriter"] + ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-tb-app0-1-roles/logging.logWriter"] : condition: [] - project: test-pf-dev-tb-app0-0 + project: test-pf-dev-tb-app0-1 role: roles/logging.logWriter - ? module.project-factory.module.service-accounts["dev-tb-app0-0/vm-default"].google_project_iam_member.project-roles["$project_ids:dev-tb-app0-0-roles/monitoring.metricWriter"] + ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-tb-app0-1-roles/monitoring.metricWriter"] : condition: [] - project: test-pf-dev-tb-app0-0 + project: test-pf-dev-tb-app0-1 role: roles/monitoring.metricWriter module.project-factory.module.service-accounts["dev-tb-app0-0/vm-default"].google_service_account.service_account[0]: account_id: vm-default @@ -543,11 +518,16 @@ values: member: serviceAccount:vm-default@test-pf-dev-tb-app0-0.iam.gserviceaccount.com project: test-pf-dev-tb-app0-0 timeouts: null - ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["$project_ids:dev-tb-app0-1-roles/logging.logWriter"] + ? module.project-factory.module.service-accounts["dev-tb-app0-0/vm-default"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"] + : condition: [] + members: + - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com + role: roles/iam.serviceAccountTokenCreator + ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-tb-app0-1-roles/logging.logWriter"] : condition: [] project: test-pf-dev-tb-app0-1 role: roles/logging.logWriter - ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["$project_ids:dev-tb-app0-1-roles/monitoring.metricWriter"] + ? module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-tb-app0-1-roles/monitoring.metricWriter"] : condition: [] project: test-pf-dev-tb-app0-1 role: roles/monitoring.metricWriter @@ -561,17 +541,6 @@ values: member: serviceAccount:app-0-be@test-pf-dev-tb-app0-1.iam.gserviceaccount.com project: test-pf-dev-tb-app0-1 timeouts: null - ? module.project-factory.module.service_accounts-iam["dev-tb-app0-0/vm-default"].data.google_service_account.service_account[0] - : account_id: vm-default - ? module.project-factory.module.service_accounts-iam["dev-tb-app0-0/vm-default"].google_service_account_iam_binding.authoritative["roles/iam.serviceAccountTokenCreator"] - : condition: [] - members: - - serviceAccount:test-pf-dev-tb-app0-0-rw@test-pf-teams-iac-0.iam.gserviceaccount.com - role: roles/iam.serviceAccountTokenCreator - module.project-factory.terraform_data.defaults_preconditions: - input: null - output: null - triggers_replace: null counts: google_billing_budget: 1 @@ -587,7 +556,7 @@ counts: google_project_iam_member: 21 google_project_service: 13 google_project_service_identity: 4 - google_service_account: 7 + google_service_account: 6 google_service_account_iam_binding: 1 google_storage_bucket: 1 google_storage_bucket_iam_binding: 2 @@ -596,6 +565,7 @@ counts: google_tags_tag_key: 1 google_tags_tag_value: 2 google_tags_tag_value_iam_binding: 1 - modules: 23 - resources: 85 - terraform_data: 1 + modules: 21 + resources: 83 + +outputs: {} diff --git a/tests/modules/project_factory/key_ignores_path.tfvars b/tests/modules/project_factory_legacy/key_ignores_path.tfvars similarity index 100% rename from tests/modules/project_factory/key_ignores_path.tfvars rename to tests/modules/project_factory_legacy/key_ignores_path.tfvars diff --git a/tests/modules/project_factory/key_ignores_path.yaml b/tests/modules/project_factory_legacy/key_ignores_path.yaml similarity index 100% rename from tests/modules/project_factory/key_ignores_path.yaml rename to tests/modules/project_factory_legacy/key_ignores_path.yaml diff --git a/tests/modules/project_factory/shared_vpc_network_user.tfvars b/tests/modules/project_factory_legacy/shared_vpc_network_user.tfvars similarity index 100% rename from tests/modules/project_factory/shared_vpc_network_user.tfvars rename to tests/modules/project_factory_legacy/shared_vpc_network_user.tfvars diff --git a/tests/modules/project_factory/shared_vpc_network_user.yaml b/tests/modules/project_factory_legacy/shared_vpc_network_user.yaml similarity index 100% rename from tests/modules/project_factory/shared_vpc_network_user.yaml rename to tests/modules/project_factory_legacy/shared_vpc_network_user.yaml diff --git a/tests/modules/project_factory/tftest.yaml b/tests/modules/project_factory_legacy/tftest.yaml similarity index 67% rename from tests/modules/project_factory/tftest.yaml rename to tests/modules/project_factory_legacy/tftest.yaml index d47232354..29342bf4d 100644 --- a/tests/modules/project_factory/tftest.yaml +++ b/tests/modules/project_factory_legacy/tftest.yaml @@ -12,18 +12,18 @@ # See the License for the specific language governing permissions and # limitations under the License. -module: modules/project-factory +module: modules/project-factory-legacy tests: bucket_iam: extra_dirs: - - ../../tests/modules/project_factory/data/bucket_iam + - ../../tests/modules/project_factory_legacy/data/bucket_iam shared_vpc_network_user: extra_dirs: - - ../../tests/modules/project_factory/data/shared_vpc_network_user/projects + - ../../tests/modules/project_factory_legacy/data/shared_vpc_network_user/projects data_overrides_defaults: extra_dirs: - - ../../tests/modules/project_factory/data/data_overrides_defaults/projects + - ../../tests/modules/project_factory_legacy/data/data_overrides_defaults/projects key_ignores_path: extra_dirs: - - ../../tests/modules/project_factory/data/key_ignores_path + - ../../tests/modules/project_factory_legacy/data/key_ignores_path diff --git a/tools/duplicate-diff.py b/tools/duplicate-diff.py index 9fb1c2cd4..9f7c4cab6 100755 --- a/tools/duplicate-diff.py +++ b/tools/duplicate-diff.py @@ -21,11 +21,11 @@ import sys duplicates = [ [ "modules/net-vpc-factory/factory-projects-object.tf", - "modules/project-factory/factory-projects-object.tf", + "modules/project-factory-legacy/factory-projects-object.tf", # data factory ], [ - "fast/stages/0-bootstrap/identity-providers-wfif-defs.tf", + "fast/stages/0-bootstrap-legacy/identity-providers-wfif-defs.tf", "fast/stages/2-secops/identity-providers-defs.tf", ], [