* data wip * wip data * update org schema, add note on expansion * all schemas, workload notes * Update WORKLOG.md * Update WORKLOG.md * Update WORKLOG.md * Update WORKLOG.md * wip * data wip * wip * wip * wip * wip * org module IAM context (using lookup) * new-style context expansion in project IAM * remove spurious file * project module contexts * finalize context replacement format for project module * revert org module changes * fix tag id interpolation in project * fix tag id interpolation in project * organization module context * organization context test * context expansion for folder tag bindings * test context expansion for tag bindings * service account module context * simplify context local * context for iam service account * nuke blueprints * remove links to blueprints * vpc sc context in project module * Add context to GCS module * Add inline deps to plan_summary script * Make context a top-level variable for folder, organization, sa * Add add context top-level to VPC-SC * move context out of factories_config variable * tfdoc * fix merge * fix merge * fix examples * net-vpc module context * add parent ids to folder context * rename folder parent context * fix folder parent check * new project factory stub * wip * wip * refactor defaults * project iam * bueckts and service accounts * start adding context replacements * better test data * automation resources for folders and projects * automation * add support for project id interpolation * first tested apply * improve IAM description in gcs module * add context to billing account module * add notification channels to billing account module context * add billing budgets to new pf * schemas and defaults * bootstrap wip * bootstrap wip * bootstrap wip * pf outputs * pf fixes * fix pf sample data * bootstrap lite fixes * add locations to organization module contexts * bootstrap lite fixes * org fixes, billing accounts * fix default project parent * bootstrap lite wip * add locations to gcs module context * add context support to logging bucket module * add context to pubsub module * split out iam variables in gcs module * fix logging bucket context test * bootstrap log sink destinations * streamline logging-bucket module variables * fix logging bucket context test * align logging bucket module interface in fast bootstrap * add support for project-level log buckets to project factory * support full context expansion in organization module log sinks * log buckets in fast-lite bootstrap * make og sink type optional in organization module * log sinks in fast-lite bootstrap * set tag values in factory context * bootstrap lite data * output files schema * billing account schema * output files * output providers * gcs output files * boilerplate * tflint * check documentation * check docs * fix project module parent variable validation * fix log bucket examples * allow null parent in project module * silence folder test errors * fix billing account sink example * fix project example * fix billing account module * fix folder tests * fix FAST * fix fast * tfvars outputs * wif * cicd service accounts * cicd * allow defaults in context, minimal org policies * support gcs managed folders in project factory and bootstrap lite * support prefix in provider output files * rename bootstrap stage * gitignore * gitignore * security folder, billing IAM * wip tfvars * fix typo * security IAM * control tag iam/context via variables in organization module * split tag creation from tag IAM to avoid circular refs * port organization module tag changes to project module * implement new-style context expansion in vpc-sc module * fix fast vpc-sc tests * boilerplate * vpc sc stage * schemas * fast-lite compatibility for vpc sc stage * make log project number optional in vpc-sc stage * networking * networking * networking * networking * rename and move new stage under fast * clone pf tests * use context replacement for internal notification channels in billing account module * support service agents in project module iam context replacements * support service agents in project module iam context replacements * add support for kms keys to project module context * experimental pf example test and fixes * fix schemas * fix tests * tfdoc * tfdoc * pf config * experimental pf * remove redundant dot from gcs managed folder IAM keys * bootstrap experimental test * project factory exp stage test * skip tflint for bootstrap experimental test * tflint * fix gcs test * documentation work * documentation work * Update README.md * tfdoc * tfdoc * readme * tfdoc * readme * readme * readme * readme * support universe in pf exp projects * missing universe service agents * org policies import, non-admin billing IAM * todo * fix test * custom constraints * fast classic dataset * fix test data * context replacements in billing module log sinks * fix typo * add support for billing log sinks * update docs * readme * cicd fix and test --------- Co-authored-by: Julio Castillo <jccb@google.com>
491 lines
19 KiB
HCL
491 lines
19 KiB
HCL
/**
|
|
* Copyright 2025 Google LLC
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
* you may not use this file except in compliance with the License.
|
|
* You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software
|
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
* See the License for the specific language governing permissions and
|
|
* limitations under the License.
|
|
*/
|
|
|
|
# tfdoc:file:description Projects and billing budgets factory resources.
|
|
|
|
locals {
|
|
_service_agent_emails = flatten([
|
|
for k, v in module.projects : [
|
|
for kk, vv in v.service_agents : {
|
|
key = "${k}/${kk}"
|
|
value = "serviceAccount:${vv.email}"
|
|
}
|
|
]
|
|
])
|
|
context = {
|
|
folder_ids = merge(
|
|
var.factories_config.context.folder_ids,
|
|
local.hierarchy
|
|
)
|
|
iam_principals = merge(
|
|
var.factories_config.context.iam_principals,
|
|
{
|
|
for k, v in module.automation-service-accounts :
|
|
k => v.iam_email
|
|
},
|
|
# module.service-accounts are excluded here, as adding them here results in dependency cycles
|
|
)
|
|
}
|
|
service_accounts_names = {
|
|
for k, v in module.service-accounts : k => v.name
|
|
}
|
|
service_agents_email = {
|
|
for v in local._service_agent_emails : v.key => v.value
|
|
}
|
|
}
|
|
|
|
module "projects" {
|
|
source = "../project"
|
|
for_each = local.projects
|
|
billing_account = each.value.billing_account
|
|
deletion_policy = each.value.deletion_policy
|
|
name = each.value.name
|
|
parent = lookup(
|
|
local.context.folder_ids, each.value.parent, each.value.parent
|
|
)
|
|
prefix = each.value.prefix
|
|
project_reuse = each.value.project_reuse
|
|
alerts = try(each.value.alerts, null)
|
|
auto_create_network = try(each.value.auto_create_network, false)
|
|
compute_metadata = try(each.value.compute_metadata, {})
|
|
# TODO: concat lists for each key
|
|
contacts = merge(
|
|
each.value.contacts, var.data_merges.contacts
|
|
)
|
|
default_service_account = try(each.value.default_service_account, "keep")
|
|
descriptive_name = try(each.value.descriptive_name, null)
|
|
factories_config = {
|
|
custom_roles = each.value.factories_config.custom_roles
|
|
observability = each.value.factories_config.observability
|
|
org_policies = each.value.factories_config.org_policies
|
|
quotas = each.value.factories_config.quotas
|
|
context = {
|
|
notification_channels = var.factories_config.context.notification_channels
|
|
}
|
|
}
|
|
labels = merge(
|
|
each.value.labels, var.data_merges.labels
|
|
)
|
|
lien_reason = try(each.value.lien_reason, null)
|
|
log_scopes = try(each.value.log_scopes, null)
|
|
logging_data_access = try(each.value.logging_data_access, {})
|
|
logging_exclusions = try(each.value.logging_exclusions, {})
|
|
logging_metrics = try(each.value.logging_metrics, null)
|
|
logging_sinks = try(each.value.logging_sinks, {})
|
|
metric_scopes = distinct(concat(
|
|
each.value.metric_scopes, var.data_merges.metric_scopes
|
|
))
|
|
notification_channels = try(each.value.notification_channels, null)
|
|
org_policies = each.value.org_policies
|
|
service_encryption_key_ids = {
|
|
for k, v in merge(
|
|
each.value.service_encryption_key_ids,
|
|
var.data_merges.service_encryption_key_ids
|
|
) : k => [
|
|
for key in v : lookup(var.factories_config.context.kms_keys, key, key)
|
|
]
|
|
}
|
|
services = distinct(concat(
|
|
each.value.services,
|
|
var.data_merges.services
|
|
))
|
|
shared_vpc_host_config = each.value.shared_vpc_host_config
|
|
tag_bindings = {
|
|
for k, v in merge(each.value.tag_bindings, var.data_merges.tag_bindings) :
|
|
k => lookup(var.factories_config.context.tag_values, v, v)
|
|
}
|
|
tags = each.value.tags
|
|
vpc_sc = each.value.vpc_sc == null ? null : {
|
|
perimeter_name = (
|
|
each.value.vpc_sc.perimeter_name == null
|
|
? null
|
|
: lookup(
|
|
var.factories_config.context.perimeters,
|
|
each.value.vpc_sc.perimeter_name,
|
|
each.value.vpc_sc.perimeter_name
|
|
)
|
|
)
|
|
is_dry_run = each.value.vpc_sc.is_dry_run
|
|
}
|
|
quotas = each.value.quotas
|
|
}
|
|
|
|
module "projects-iam" {
|
|
source = "../project"
|
|
for_each = local.projects
|
|
name = module.projects[each.key].project_id
|
|
project_reuse = {
|
|
use_data_source = false
|
|
attributes = {
|
|
name = module.projects[each.key].name
|
|
number = module.projects[each.key].number
|
|
services_enabled = module.projects[each.key].services
|
|
}
|
|
}
|
|
iam = {
|
|
for k, v in lookup(each.value, "iam", {}) :
|
|
lookup(var.factories_config.context.custom_roles, k, k) => [
|
|
for vv in v : try(
|
|
# project service accounts (sa)
|
|
module.service-accounts["${each.key}/${vv}"].iam_email,
|
|
# automation service account (rw)
|
|
local.context.iam_principals["${each.key}/automation/${vv}"],
|
|
# automation service account (automation/rw)
|
|
local.context.iam_principals["${each.key}/${vv}"],
|
|
# other projects service accounts (project/sa)
|
|
module.service-accounts[vv].iam_email,
|
|
# other automation service account (project/automation/rw)
|
|
local.context.iam_principals[vv],
|
|
# project's service identities
|
|
local.service_agents_email["${each.key}/${vv}"],
|
|
local.service_agents_email[vv],
|
|
# passthrough + error handling using tonumber until Terraform gets fail/raise function
|
|
(
|
|
strcontains(vv, ":")
|
|
? templatestring(
|
|
vv, { project_number = module.projects[each.key].number }
|
|
)
|
|
: tonumber("[Error] Invalid member: '${vv}' in project '${each.key}'")
|
|
)
|
|
)
|
|
]
|
|
}
|
|
iam_bindings = {
|
|
for k, v in lookup(each.value, "iam_bindings", {}) : k => merge(v, {
|
|
members = [
|
|
for vv in v.members : try(
|
|
# project service accounts (sa)
|
|
module.service-accounts["${each.key}/${vv}"].iam_email,
|
|
# automation service account (rw)
|
|
local.context.iam_principals["${each.key}/automation/${vv}"],
|
|
# automation service account (automation/rw)
|
|
local.context.iam_principals["${each.key}/${vv}"],
|
|
# other projects service accounts (project/sa)
|
|
module.service-accounts[vv].iam_email,
|
|
# other automation service account (project/automation/rw)
|
|
local.context.iam_principals[vv],
|
|
# project's service identities
|
|
local.service_agents_email["${each.key}/${vv}"],
|
|
local.service_agents_email[vv],
|
|
# passthrough + error handling using tonumber until Terraform gets fail/raise function
|
|
(
|
|
strcontains(vv, ":")
|
|
? templatestring(
|
|
vv, { project_number = module.projects[each.key].number }
|
|
)
|
|
: tonumber("[Error] Invalid member: '${vv}' in project '${each.key}'")
|
|
)
|
|
)
|
|
]
|
|
role = lookup(var.factories_config.context.custom_roles, v.role, v.role)
|
|
})
|
|
}
|
|
iam_bindings_additive = {
|
|
for k, v in lookup(each.value, "iam_bindings_additive", {}) : k => merge(v, {
|
|
member = try(
|
|
# project service accounts (sa)
|
|
module.service-accounts["${each.key}/${v.member}"].iam_email,
|
|
# automation service account (rw)
|
|
local.context.iam_principals["${each.key}/automation/${v.member}"],
|
|
# automation service account (automation/rw)
|
|
local.context.iam_principals["${each.key}/${v.member}"],
|
|
# other projects service accounts (project/sa)
|
|
module.service-accounts[v.member].iam_email,
|
|
# other automation service account (project/automation/rw)
|
|
local.context.iam_principals[v.member],
|
|
# project's service identities
|
|
local.service_agents_email["${each.key}/${v.member}"],
|
|
local.service_agents_email[v.member],
|
|
# passthrough + error handling using tonumber until Terraform gets fail/raise function
|
|
(
|
|
strcontains(v.member, ":")
|
|
? templatestring(
|
|
v.member, { project_number = module.projects[each.key].number }
|
|
)
|
|
: tonumber("[Error] Invalid member: '${v.member}' in project '${each.key}'")
|
|
)
|
|
)
|
|
role = lookup(var.factories_config.context.custom_roles, v.role, v.role)
|
|
})
|
|
}
|
|
# IAM by principals would trigger dynamic key errors so we don't interpolate
|
|
# iam_by_principals = try(each.value.iam_by_principals, {})
|
|
iam_by_principals = {
|
|
for k, v in try(each.value.iam_by_principals, {}) :
|
|
try(
|
|
# project service accounts (sa)
|
|
module.service-accounts["${each.key}/${k}"].iam_email,
|
|
# automation service account (rw)
|
|
local.context.iam_principals["${each.key}/automation/${k}"],
|
|
# automation service account (automation/rw)
|
|
local.context.iam_principals["${each.key}/${k}"],
|
|
# other projects service accounts (project/sa)
|
|
module.service-accounts[k].iam_email,
|
|
# other automation service account (project/automation/rw)
|
|
local.context.iam_principals[k],
|
|
# project's service identities
|
|
local.service_agents_email["${each.key}/${k}"],
|
|
local.service_agents_email[k],
|
|
# passthrough + error handling using tonumber until Terraform gets fail/raise function
|
|
(
|
|
strcontains(k, ":")
|
|
? templatestring(
|
|
k, { project_number = module.projects[each.key].number }
|
|
)
|
|
: tonumber("[Error] Invalid member: '${k}' in project '${each.key}'")
|
|
)
|
|
) => [
|
|
for vv in v : lookup(var.factories_config.context.custom_roles, vv, vv)
|
|
]
|
|
}
|
|
# Shared VPC configuration is done at stage 2, to avoid dependency cycle between project service accounts and
|
|
# IAM grants done for those service accounts
|
|
shared_vpc_service_config = (
|
|
try(each.value.shared_vpc_service_config.host_project, null) == null
|
|
? null
|
|
: merge(each.value.shared_vpc_service_config, {
|
|
host_project = try(
|
|
var.factories_config.context.vpc_host_projects[each.value.shared_vpc_service_config.host_project],
|
|
module.projects[each.value.shared_vpc_service_config.host_project].project_id,
|
|
each.value.shared_vpc_service_config.host_project
|
|
)
|
|
iam_bindings_additive = {
|
|
for k, v in try(each.value.shared_vpc_service_config.iam_bindings_additive, {}) : k => merge(v, {
|
|
member = try(
|
|
# project service accounts (sa)
|
|
module.service-accounts["${each.key}/${v.member}"].iam_email,
|
|
# automation service account (rw)
|
|
local.context.iam_principals["${each.key}/automation/${v.member}"],
|
|
# automation service account (automation/rw)
|
|
local.context.iam_principals["${each.key}/${v.member}"],
|
|
# other projects service accounts (project/sa)
|
|
module.service-accounts[v.member].iam_email,
|
|
# other automation service account (project/automation/rw)
|
|
local.context.iam_principals[v.member],
|
|
# project's service identities
|
|
local.service_agents_email["${each.key}/${v.member}"],
|
|
local.service_agents_email[v.member],
|
|
# passthrough + error handling using tonumber until Terraform gets fail/raise function
|
|
(
|
|
strcontains(v.member, ":")
|
|
? templatestring(
|
|
v.member, { project_number = module.projects[each.key].number }
|
|
)
|
|
: tonumber("[Error] Invalid member: '${v.member}' in project '${each.key}'")
|
|
)
|
|
)
|
|
role = lookup(var.factories_config.context.custom_roles, v.role, v.role)
|
|
})
|
|
}
|
|
network_users = [
|
|
for vv in try(each.value.shared_vpc_service_config.network_users, []) :
|
|
try(
|
|
# project service accounts (sa)
|
|
module.service-accounts["${each.key}/${vv}"].iam_email,
|
|
# automation service account (rw)
|
|
local.context.iam_principals["${each.key}/automation/${vv}"],
|
|
# automation service account (automation/rw)
|
|
local.context.iam_principals["${each.key}/${vv}"],
|
|
# other projects service accounts (project/sa)
|
|
module.service-accounts[vv].iam_email,
|
|
# other automation service account (project/automation/rw)
|
|
local.context.iam_principals[vv],
|
|
# passthrough + error handling using tonumber until Terraform gets fail/raise function
|
|
(
|
|
strcontains(vv, ":")
|
|
? templatestring(
|
|
vv, { project_number = module.projects[each.key].number }
|
|
)
|
|
: tonumber("[Error] Invalid member: '${vv}' in project '${each.key}'")
|
|
)
|
|
)
|
|
]
|
|
})
|
|
)
|
|
# add service agents config, so Service Agents can be referred in the IAM grants
|
|
service_agents_config = {
|
|
# default roles are granted in module.project
|
|
grant_default_roles = false
|
|
}
|
|
}
|
|
|
|
module "buckets" {
|
|
source = "../gcs"
|
|
for_each = {
|
|
for k in local.buckets : "${k.project_key}/${k.name}" => k
|
|
}
|
|
project_id = module.projects[each.value.project_key].project_id
|
|
prefix = each.value.prefix
|
|
name = "${each.value.project_name}-${each.value.name}"
|
|
encryption_key = each.value.encryption_key
|
|
force_destroy = each.value.force_destroy
|
|
iam = {
|
|
for k, v in each.value.iam : k => [
|
|
for vv in v : try(
|
|
# project service accounts (sa)
|
|
module.service-accounts["${each.value.project_key}/${vv}"].iam_email,
|
|
# automation service account (rw)
|
|
local.context.iam_principals["${each.value.project_key}/automation/${vv}"],
|
|
# automation service account (automation/rw)
|
|
local.context.iam_principals["${each.value.project_key}/${vv}"],
|
|
# other projects service accounts (project/sa)
|
|
module.service-accounts[vv].iam_email,
|
|
# other automation service account (project/automation/rw)
|
|
local.context.iam_principals[vv],
|
|
# project's service identities
|
|
local.service_agents_email["${each.value.project_key}/${vv}"],
|
|
local.service_agents_email[vv],
|
|
# passthrough + error handling using tonumber until Terraform gets fail/raise function
|
|
(
|
|
strcontains(vv, ":")
|
|
? templatestring(
|
|
vv, { project_number = module.projects[each.value.project_key].number }
|
|
)
|
|
: tonumber("[Error] Invalid member: '${vv}' in bucket '${each.key}'")
|
|
)
|
|
)
|
|
]
|
|
}
|
|
iam_bindings = {
|
|
for k, v in each.value.iam_bindings : k => merge(v, {
|
|
members = [
|
|
for vv in v.members : try(
|
|
# project service accounts (sa)
|
|
module.service-accounts["${each.value.project_key}/${vv}"].iam_email,
|
|
# automation service account (rw)
|
|
local.context.iam_principals["${each.value.project_key}/automation/${vv}"],
|
|
# automation service account (automation/rw)
|
|
local.context.iam_principals["${each.value.project_key}/${vv}"],
|
|
# other projects service accounts (project/sa)
|
|
module.service-accounts[vv].iam_email,
|
|
# other automation service account (project/automation/rw)
|
|
local.context.iam_principals[vv],
|
|
# project's service identities
|
|
local.service_agents_email["${each.value.project_key}/${vv}"],
|
|
local.service_agents_email[vv],
|
|
# passthrough + error handling using tonumber until Terraform gets fail/raise function
|
|
(
|
|
strcontains(vv, ":")
|
|
? templatestring(
|
|
vv, { project_number = module.projects[each.value.project_key].number }
|
|
)
|
|
: tonumber("[Error] Invalid member: '${vv}' in bucket '${each.key}'")
|
|
)
|
|
)
|
|
]
|
|
})
|
|
}
|
|
iam_bindings_additive = {
|
|
for k, v in each.value.iam_bindings_additive : k => merge(v, {
|
|
member = try(
|
|
# project service accounts (sa)
|
|
module.service-accounts["${each.value.project_key}/${v.member}"].iam_email,
|
|
# automation service account (rw)
|
|
local.context.iam_principals["${each.value.project_key}/automation/${v.member}"],
|
|
# automation service account (automation/rw)
|
|
local.context.iam_principals["${each.value.project_key}/${v.member}"],
|
|
# other projects service accounts (project/sa)
|
|
module.service-accounts[v.member].iam_email,
|
|
# other automation service account (project/automation/rw)
|
|
local.context.iam_principals[v.member],
|
|
# project's service identities
|
|
local.service_agents_email["${each.value.project_key}/${v.member}"],
|
|
local.service_agents_email[v.member],
|
|
# passthrough + error handling using tonumber until Terraform gets fail/raise function
|
|
(
|
|
strcontains(v.member, ":")
|
|
? templatestring(
|
|
v.member, { project_number = module.projects[each.value.project_key].number }
|
|
)
|
|
: tonumber("[Error] Invalid member: '${v.member}' in bucket '${each.key}'")
|
|
)
|
|
)
|
|
})
|
|
}
|
|
labels = each.value.labels
|
|
location = coalesce(
|
|
var.data_overrides.storage_location,
|
|
lookup(each.value, "location", null),
|
|
var.data_defaults.storage_location
|
|
)
|
|
storage_class = each.value.storage_class
|
|
uniform_bucket_level_access = each.value.uniform_bucket_level_access
|
|
versioning = each.value.versioning
|
|
}
|
|
|
|
module "service-accounts" {
|
|
source = "../iam-service-account"
|
|
for_each = {
|
|
for k in local.service_accounts : "${k.project_key}/${k.name}" => k
|
|
}
|
|
project_id = module.projects[each.value.project_key].project_id
|
|
name = each.value.name
|
|
display_name = each.value.display_name
|
|
iam = {
|
|
for k, v in lookup(each.value, "iam", {}) : k => [
|
|
for vv in v : try(
|
|
# automation service account (rw)
|
|
local.context.iam_principals["${each.value.project_key}/automation/${vv}"],
|
|
# automation service account (automation/rw)
|
|
local.context.iam_principals["${each.value.project_key}/${vv}"],
|
|
# other automation service account (project/automation/rw)
|
|
local.context.iam_principals[vv],
|
|
# passthrough + error handling using tonumber until Terraform gets fail/raise function
|
|
(
|
|
strcontains(vv, ":")
|
|
? vv
|
|
: tonumber("[Error] Invalid member: '${vv}' in project '${each.value.project_key}'")
|
|
)
|
|
)
|
|
]
|
|
}
|
|
iam_project_roles = merge(
|
|
{
|
|
for k, v in each.value.iam_project_roles :
|
|
lookup(var.factories_config.context.vpc_host_projects, k, k) => v
|
|
},
|
|
each.value.iam_self_roles == null ? {} : {
|
|
(module.projects[each.value.project_key].project_id) = each.value.iam_self_roles
|
|
}
|
|
)
|
|
}
|
|
|
|
module "service_accounts-iam" {
|
|
source = "../iam-service-account"
|
|
for_each = {
|
|
for k in local.service_accounts : "${k.project_key}/${k.name}" => k
|
|
if k.iam_sa_roles != {}
|
|
}
|
|
project_id = module.service-accounts[each.key].service_account.project
|
|
name = each.value.name
|
|
service_account_create = false
|
|
iam_sa_roles = {
|
|
for k, v in each.value.iam_sa_roles : lookup(
|
|
local.service_accounts_names, "${each.value.project_key}/${k}", k
|
|
) => v
|
|
}
|
|
}
|
|
|
|
module "billing-account" {
|
|
source = "../billing-account"
|
|
count = var.factories_config.budgets == null ? 0 : 1
|
|
id = var.factories_config.budgets.billing_account
|
|
budget_notification_channels = (
|
|
var.factories_config.budgets.notification_channels
|
|
)
|
|
budgets = local.budgets
|
|
}
|