* data wip * wip data * update org schema, add note on expansion * all schemas, workload notes * Update WORKLOG.md * Update WORKLOG.md * Update WORKLOG.md * Update WORKLOG.md * wip * data wip * wip * wip * wip * wip * org module IAM context (using lookup) * new-style context expansion in project IAM * remove spurious file * project module contexts * finalize context replacement format for project module * revert org module changes * fix tag id interpolation in project * fix tag id interpolation in project * organization module context * organization context test * context expansion for folder tag bindings * test context expansion for tag bindings * service account module context * simplify context local * context for iam service account * nuke blueprints * remove links to blueprints * vpc sc context in project module * Add context to GCS module * Add inline deps to plan_summary script * Make context a top-level variable for folder, organization, sa * Add add context top-level to VPC-SC * move context out of factories_config variable * tfdoc * fix merge * fix merge * fix examples * net-vpc module context * add parent ids to folder context * rename folder parent context * fix folder parent check * new project factory stub * wip * wip * refactor defaults * project iam * bueckts and service accounts * start adding context replacements * better test data * automation resources for folders and projects * automation * add support for project id interpolation * first tested apply * improve IAM description in gcs module * add context to billing account module * add notification channels to billing account module context * add billing budgets to new pf * schemas and defaults * bootstrap wip * bootstrap wip * bootstrap wip * pf outputs * pf fixes * fix pf sample data * bootstrap lite fixes * add locations to organization module contexts * bootstrap lite fixes * org fixes, billing accounts * fix default project parent * bootstrap lite wip * add locations to gcs module context * add context support to logging bucket module * add context to pubsub module * split out iam variables in gcs module * fix logging bucket context test * bootstrap log sink destinations * streamline logging-bucket module variables * fix logging bucket context test * align logging bucket module interface in fast bootstrap * add support for project-level log buckets to project factory * support full context expansion in organization module log sinks * log buckets in fast-lite bootstrap * make og sink type optional in organization module * log sinks in fast-lite bootstrap * set tag values in factory context * bootstrap lite data * output files schema * billing account schema * output files * output providers * gcs output files * boilerplate * tflint * check documentation * check docs * fix project module parent variable validation * fix log bucket examples * allow null parent in project module * silence folder test errors * fix billing account sink example * fix project example * fix billing account module * fix folder tests * fix FAST * fix fast * tfvars outputs * wif * cicd service accounts * cicd * allow defaults in context, minimal org policies * support gcs managed folders in project factory and bootstrap lite * support prefix in provider output files * rename bootstrap stage * gitignore * gitignore * security folder, billing IAM * wip tfvars * fix typo * security IAM * control tag iam/context via variables in organization module * split tag creation from tag IAM to avoid circular refs * port organization module tag changes to project module * implement new-style context expansion in vpc-sc module * fix fast vpc-sc tests * boilerplate * vpc sc stage * schemas * fast-lite compatibility for vpc sc stage * make log project number optional in vpc-sc stage * networking * networking * networking * networking * rename and move new stage under fast * clone pf tests * use context replacement for internal notification channels in billing account module * support service agents in project module iam context replacements * support service agents in project module iam context replacements * add support for kms keys to project module context * experimental pf example test and fixes * fix schemas * fix tests * tfdoc * tfdoc * pf config * experimental pf * remove redundant dot from gcs managed folder IAM keys * bootstrap experimental test * project factory exp stage test * skip tflint for bootstrap experimental test * tflint * fix gcs test * documentation work * documentation work * Update README.md * tfdoc * tfdoc * readme * tfdoc * readme * readme * readme * readme * support universe in pf exp projects * missing universe service agents * org policies import, non-admin billing IAM * todo * fix test * custom constraints * fast classic dataset * fix test data * context replacements in billing module log sinks * fix typo * add support for billing log sinks * update docs * readme * cicd fix and test --------- Co-authored-by: Julio Castillo <jccb@google.com>
21 KiB
Google Cloud Folder Module
This module allows the creation and management of folders, including support for IAM bindings, organization policies, and hierarchical firewall rules.
- Basic example with IAM bindings
- IAM
- Assured Workload Folder
- Organization policies
- Hierarchical Firewall Policy Attachments
- Log Sinks
- Data Access Logs
- Tags
- Files
- Variables
- Outputs
Basic example with IAM bindings
module "folder" {
source = "./fabric/modules/folder"
parent = var.folder_id
name = "Folder name"
iam_by_principals = {
"group:${var.group_email}" = [
"roles/owner",
"roles/resourcemanager.folderAdmin",
"roles/resourcemanager.projectCreator"
]
}
iam = {
"roles/owner" = ["serviceAccount:${var.service_account.email}"]
}
iam_bindings_additive = {
am1-storage-admin = {
member = "serviceAccount:${var.service_account.email}"
role = "roles/storage.admin"
}
}
}
# tftest modules=1 resources=5 inventory=iam.yaml e2e
IAM
IAM is managed via several variables that implement different features and levels of control:
iamandiam_by_principalsconfigure authoritative bindings that manage individual roles exclusively, and are internally mergediam_bindingsconfigure authoritative bindings with optional support for conditions, and are not internally merged with the previous two variablesiam_bindings_additiveconfigure additive bindings via individual role/member pairs with optional support conditions
The authoritative and additive approaches can be used together, provided different roles are managed by each. Some care must also be taken with the iam_by_principals variable to ensure that variable keys are static values, so that Terraform is able to compute the dependency graph.
IAM also supports variable interpolation for both roles and principals, via the respective attributes in the var.context variable. Refer to the project module for examples of the IAM interface.
Assured Workload Folder
To create Assured Workload folder instead of regular folder.
Note that an existing folder can not be converted to an Assured Workload folder, hence assured_workload_config is mutually exclusive with folder_create=false.
module "folder" {
source = "./fabric/modules/folder"
parent = var.folder_id
name = "Folder name"
assured_workload_config = {
compliance_regime = "EU_REGIONS_AND_SUPPORT"
display_name = "workload-name"
location = "europe-west1"
organization = var.organization_id
enable_sovereign_controls = true
}
iam = {
"roles/owner" = ["serviceAccount:${var.service_account.email}"]
}
iam_bindings_additive = {
am1-storage-admin = {
member = "serviceAccount:${var.service_account.email}"
role = "roles/storage.admin"
}
}
}
# tftest modules=1 resources=3 inventory=assured-workload.yaml
Organization policies
To manage organization policies, the orgpolicy.googleapis.com service should be enabled in the quota project.
module "folder" {
source = "./fabric/modules/folder"
parent = var.folder_id
name = "Folder name"
org_policies = {
"compute.disableGuestAttributesAccess" = {
rules = [{ enforce = true }]
}
"compute.skipDefaultNetworkCreation" = {
rules = [{ enforce = true }]
}
"iam.disableServiceAccountKeyCreation" = {
rules = [{ enforce = true }]
}
"iam.disableServiceAccountKeyUpload" = {
rules = [
{
condition = {
expression = "resource.matchTagId('tagKeys/1234', 'tagValues/1234')"
title = "condition"
description = "test condition"
location = "somewhere"
}
enforce = true
},
{
enforce = false
}
]
}
"iam.allowedPolicyMemberDomains" = {
rules = [{
allow = {
values = ["C0xxxxxxx", "C0yyyyyyy", "C0zzzzzzz"]
}
}]
}
"compute.trustedImageProjects" = {
rules = [{
allow = {
values = ["projects/my-project"]
}
}]
}
"compute.vmExternalIpAccess" = {
rules = [{ deny = { all = true } }]
}
"essentialcontacts.managed.allowedContactDomains" = {
rules = [
{
enforce = true
parameters = jsonencode({
allowedDomains = ["@example.com", "@secondary.example.com"]
})
}
]
}
}
}
# tftest modules=1 resources=9 inventory=org-policies.yaml e2e
Organization Policy Factory
Organization policies can be loaded from a directory containing YAML files where each file defines one or more constraints. The structure of the YAML files is exactly the same as the org_policies variable.
Note that constraints defined via org_policies take precedence over those in org_policies_data_path. In other words, if you specify the same constraint in a YAML file and in the org_policies variable, the latter will take priority.
The example below deploys a few organization policies split between two YAML files.
module "folder" {
source = "./fabric/modules/folder"
parent = var.folder_id
name = "Folder name"
factories_config = {
org_policies = "configs/org-policies/"
}
context = {
org_policies = {
tags = {
my_conditional_tag = "tagKeys/1234"
}
domains = {
secondary = "@secondary.example.com"
}
customer_ids = {
extra = "C0zzzzzzz"
}
}
}
}
# tftest modules=1 resources=9 files=boolean,list inventory=org-policies.yaml e2e
compute.disableGuestAttributesAccess:
rules:
- enforce: true
compute.skipDefaultNetworkCreation:
rules:
- enforce: true
iam.disableServiceAccountKeyCreation:
rules:
- enforce: true
iam.disableServiceAccountKeyUpload:
rules:
- condition:
description: test condition
expression: resource.matchTagId('${tags.my_conditional_tag}', 'tagValues/1234')
location: somewhere
title: condition
enforce: true
- enforce: false
essentialcontacts.managed.allowedContactDomains:
rules:
- enforce: true
parameters: |
{"allowedDomains": ["@example.com", "${domains.secondary}"]}
# tftest-file id=boolean path=configs/org-policies/boolean.yaml schema=org-policies.schema.json
compute.trustedImageProjects:
rules:
- allow:
values:
- projects/my-project
compute.vmExternalIpAccess:
rules:
- deny:
all: true
iam.allowedPolicyMemberDomains:
rules:
- allow:
values:
- C0xxxxxxx
- C0yyyyyyy
- ${customer_ids.extra}
# tftest-file id=list path=configs/org-policies/list.yaml schema=org-policies.schema.json
Hierarchical Firewall Policy Attachments
Hierarchical firewall policies can be managed via the net-firewall-policy module, including support for factories. Once a policy is available, attaching it to the organization can be done either in the firewall policy module itself, or here:
module "firewall-policy" {
source = "./fabric/modules/net-firewall-policy"
name = "test-1"
parent_id = module.folder.id
# attachment via the firewall policy module
# attachments = {
# folder-1 = module.folder.id
# }
}
module "folder" {
source = "./fabric/modules/folder"
parent = var.folder_id
name = "Folder name"
# attachment via the organization module
firewall_policy = {
name = "test-1"
policy = module.firewall-policy.id
}
}
# tftest modules=2 resources=3 e2e serial
Log Sinks
module "gcs" {
source = "./fabric/modules/gcs"
project_id = var.project_id
prefix = var.prefix
name = "gcs_sink"
location = "EU"
force_destroy = true
}
module "dataset" {
source = "./fabric/modules/bigquery-dataset"
project_id = var.project_id
id = "bq_sink"
}
module "pubsub" {
source = "./fabric/modules/pubsub"
project_id = var.project_id
name = "pubsub_sink"
}
module "bucket" {
source = "./fabric/modules/logging-bucket"
parent = var.project_id
name = "${var.prefix}-bucket"
}
module "destination-project" {
source = "./fabric/modules/project"
name = "dest-prj"
billing_account = var.billing_account_id
parent = var.folder_id
prefix = var.prefix
services = [
"logging.googleapis.com"
]
}
module "folder-sink" {
source = "./fabric/modules/folder"
name = "Folder name"
parent = var.folder_id
logging_sinks = {
warnings = {
destination = module.gcs.id
filter = "severity=WARNING"
type = "storage"
}
info = {
destination = module.dataset.id
filter = "severity=INFO"
type = "bigquery"
}
notice = {
destination = module.pubsub.id
filter = "severity=NOTICE"
type = "pubsub"
}
debug = {
destination = module.bucket.id
filter = "severity=DEBUG"
exclusions = {
no-compute = "logName:compute"
}
type = "logging"
}
alert = {
destination = module.destination-project.id
filter = "severity=ALERT"
type = "project"
}
}
logging_exclusions = {
no-gce-instances = "resource.type=gce_instance"
}
}
# tftest modules=6 resources=18 inventory=logging.yaml e2e
Data Access Logs
Activation of data access logs can be controlled via the logging_data_access variable. If the iam_bindings_authoritative variable is used to set a resource-level IAM policy, the data access log configuration will also be authoritative as part of the policy.
This example shows how to set a non-authoritative access log configuration:
module "folder" {
source = "./fabric/modules/folder"
parent = var.folder_id
name = "Folder name"
logging_data_access = {
allServices = {
ADMIN_READ = {
exempted_members = ["group:${var.group_email}"]
}
}
"storage.googleapis.com" = {
DATA_READ = {}
DATA_WRITE = {}
}
}
}
# tftest modules=1 resources=3 inventory=logging-data-access.yaml e2e
Tags
Refer to the Creating and managing tags documentation for details on usage.
module "org" {
source = "./fabric/modules/organization"
organization_id = var.organization_id
tags = {
environment = {
description = "Environment specification."
values = {
dev = {}
prod = {}
}
}
}
}
module "folder" {
source = "./fabric/modules/folder"
name = "Folder name"
parent = var.folder_id
tag_bindings = {
env-prod = module.org.tag_values["environment/prod"].id
}
}
# tftest modules=2 resources=5 inventory=tags.yaml e2e serial
Files
| name | description | resources |
|---|---|---|
| iam.tf | IAM bindings. | google_folder_iam_binding · google_folder_iam_member |
| logging.tf | Log sinks and supporting resources. | google_bigquery_dataset_iam_member · google_folder_iam_audit_config · google_logging_folder_exclusion · google_logging_folder_settings · google_logging_folder_sink · google_project_iam_member · google_pubsub_topic_iam_member · google_storage_bucket_iam_member |
| main.tf | Module-level locals and resources. | google_assured_workloads_workload · google_compute_firewall_policy_association · google_essential_contacts_contact · google_folder |
| organization-policies.tf | Folder-level organization policies. | google_org_policy_policy |
| outputs.tf | Module outputs. | |
| tags.tf | None | google_tags_tag_binding |
| variables-iam.tf | None | |
| variables-logging.tf | None | |
| variables.tf | Module variables. | |
| versions.tf | Version pins. |
Variables
| name | description | type | required | default |
|---|---|---|---|---|
| assured_workload_config | Create AssuredWorkloads folder instead of regular folder when value is provided. Incompatible with folder_create=false. | object({…}) |
null |
|
| contacts | List of essential contacts for this resource. Must be in the form EMAIL -> [NOTIFICATION_TYPES]. Valid notification types are ALL, SUSPENSION, SECURITY, TECHNICAL, BILLING, LEGAL, PRODUCT_UPDATES. | map(list(string)) |
{} |
|
| context | Context-specific interpolations. | object({…}) |
{} |
|
| deletion_protection | Deletion protection setting for this folder. | bool |
false |
|
| factories_config | Paths to data files and folders that enable factory functionality. | object({…}) |
{} |
|
| firewall_policy | Hierarchical firewall policy to associate to this folder. | object({…}) |
null |
|
| folder_create | Create folder. When set to false, uses id to reference an existing folder. | bool |
true |
|
| iam | IAM bindings in {ROLE => [MEMBERS]} format. | map(list(string)) |
{} |
|
| iam_bindings | Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. | map(object({…})) |
{} |
|
| iam_bindings_additive | Individual additive IAM bindings. Keys are arbitrary. | map(object({…})) |
{} |
|
| iam_by_principals | Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid errors. Merged internally with the iam variable. |
map(list(string)) |
{} |
|
| iam_by_principals_additive | Additive IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid errors. Merged internally with the iam_bindings_additive variable. |
map(list(string)) |
{} |
|
| id | Folder ID in case you use folder_create=false. | string |
null |
|
| logging_data_access | Control activation of data access logs. The special 'allServices' key denotes configuration for all services. | map(object({…})) |
{} |
|
| logging_exclusions | Logging exclusions for this folder in the form {NAME -> FILTER}. | map(string) |
{} |
|
| logging_settings | Default settings for logging resources. | object({…}) |
null |
|
| logging_sinks | Logging sinks to create for the folder. | map(object({…})) |
{} |
|
| name | Folder name. | string |
null |
|
| org_policies | Organization policies applied to this folder keyed by policy name. | map(object({…})) |
{} |
|
| parent | Parent in folders/folder_id or organizations/org_id format. | string |
null |
|
| tag_bindings | Tag bindings for this folder, in key => tag value id format. | map(string) |
null |
Outputs
| name | description | sensitive |
|---|---|---|
| assured_workload | Assured Workloads workload resource. | |
| folder | Folder resource. | |
| id | Fully qualified folder id. | |
| name | Folder name. | |
| sink_writer_identities | Writer identities created for each sink. |