Files
hunfabric/modules/project/service-agents.tf
Ludovico Magnocavallo 36648b6b63 FAST light implementation (#3255)
* data wip

* wip data

* update org schema, add note on expansion

* all schemas, workload notes

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* wip

* data wip

* wip

* wip

* wip

* wip

* org module IAM context (using lookup)

* new-style context expansion in project IAM

* remove spurious file

* project module contexts

* finalize context replacement format for project module

* revert org module changes

* fix tag id interpolation in project

* fix tag id interpolation in project

* organization module context

* organization context test

* context expansion for folder tag bindings

* test context expansion for tag bindings

* service account module context

* simplify context local

* context for iam service account

* nuke blueprints

* remove links to blueprints

* vpc sc context in project module

* Add context to GCS module

* Add inline deps to plan_summary script

* Make context a top-level variable for folder, organization, sa

* Add add context top-level to VPC-SC

* move context out of factories_config variable

* tfdoc

* fix merge

* fix merge

* fix examples

* net-vpc module context

* add parent ids to folder context

* rename folder parent context

* fix folder parent check

* new project factory stub

* wip

* wip

* refactor defaults

* project iam

* bueckts and service accounts

* start adding context replacements

* better test data

* automation resources for folders and projects

* automation

* add support for project id interpolation

* first tested apply

* improve IAM description in gcs module

* add context to billing account module

* add notification channels to billing account module context

* add billing budgets to new pf

* schemas and defaults

* bootstrap wip

* bootstrap wip

* bootstrap wip

* pf outputs

* pf fixes

* fix pf sample data

* bootstrap lite fixes

* add locations to organization module contexts

* bootstrap lite fixes

* org fixes, billing accounts

* fix default project parent

* bootstrap lite wip

* add locations to gcs module context

* add context support to logging bucket module

* add context to pubsub module

* split out iam variables in gcs module

* fix logging bucket context test

* bootstrap log sink destinations

* streamline logging-bucket module variables

* fix logging bucket context test

* align logging bucket module interface in fast bootstrap

* add support for project-level log buckets to project factory

* support full context expansion in organization module log sinks

* log buckets in fast-lite bootstrap

* make og sink type optional in organization module

* log sinks in fast-lite bootstrap

* set tag values in factory context

* bootstrap lite data

* output files schema

* billing account schema

* output files

* output providers

* gcs output files

* boilerplate

* tflint

* check documentation

* check docs

* fix project module parent variable validation

* fix log bucket examples

* allow null parent in project module

* silence folder test errors

* fix billing account sink example

* fix project example

* fix billing account module

* fix folder tests

* fix FAST

* fix fast

* tfvars outputs

* wif

* cicd service accounts

* cicd

* allow defaults in context, minimal org policies

* support gcs managed folders in project factory and bootstrap lite

* support prefix in provider output files

* rename bootstrap stage

* gitignore

* gitignore

* security folder, billing IAM

* wip tfvars

* fix typo

* security IAM

* control tag iam/context via variables in organization module

* split tag creation from tag IAM to avoid circular refs

* port organization module tag changes to project module

* implement new-style context expansion in vpc-sc module

* fix fast vpc-sc tests

* boilerplate

* vpc sc stage

* schemas

* fast-lite compatibility for vpc sc stage

* make log project number optional in vpc-sc stage

* networking

* networking

* networking

* networking

* rename and move new stage under fast

* clone pf tests

* use context replacement for internal notification channels in billing account module

* support service agents in project module iam context replacements

* support service agents in project module iam context replacements

* add support for kms keys to project module context

* experimental pf example test and fixes

* fix schemas

* fix tests

* tfdoc

* tfdoc

* pf config

* experimental pf

* remove redundant dot from gcs managed folder IAM keys

* bootstrap experimental test

* project factory exp stage test

* skip tflint for bootstrap experimental test

* tflint

* fix gcs test

* documentation work

* documentation work

* Update README.md

* tfdoc

* tfdoc

* readme

* tfdoc

* readme

* readme

* readme

* readme

* support universe in pf exp projects

* missing universe service agents

* org policies import, non-admin billing IAM

* todo

* fix test

* custom constraints

* fast classic dataset

* fix test data

* context replacements in billing module log sinks

* fix typo

* add support for billing log sinks

* update docs

* readme

* cicd fix and test

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2025-09-02 08:38:57 +02:00

173 lines
6.3 KiB
HCL

/**
* Copyright 2025 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
# tfdoc:file:description Service agents supporting resources.
locals {
services = [
for s in distinct(concat(
local.available_services, try(var.project_reuse.attributes.services_enabled, [])
)) : s if !contains(local._universe_unavailable_si, s)
]
_service_agents_data = yamldecode(file("${path.module}/service-agents.yaml"))
# map of api => list of agents
_service_agents_by_api = {
for agent in local._service_agents_data :
coalesce(agent.api, "cloudservices") => agent... # cloudservices api is null
}
_universe_domain = (
var.universe == null
? ""
: "${var.universe.prefix}-system."
)
_universe_unavailable_si = try(var.universe.unavailable_service_identities, [])
# map of service agent name => agent details for this project
_project_service_agents_0 = merge([
for api in concat(local.services, ["cloudservices"]) : {
for agent in lookup(local._service_agents_by_api, api, []) :
(agent.name) => merge(agent, {
email = (
# If universe variable is set, enfore the use of the service-PROJECT_NUMBER@gcp-sa-ekms.UNIVERSE-system.iam.gserviceaccount.com
# instead of service-PROJECT_NUMBER@gcp-sa-kms.UNIVERSE-system.iam.gserviceaccount.com
# as in the TPC universes, the partner KMS is enforced by design
var.universe != null && api == "cloudkms.googleapis.com"
? format("service-%s@gcp-sa-ekms.%siam.gserviceaccount.com", local.project.number, local._universe_domain)
: (
var.universe == null || api != "cloudservices"
? templatestring(agent.identity, { project_number = local.project.number, universe_domain = local._universe_domain })
: format("%s@cloudservices.%siam.gserviceaccount.com", local.project.number, local._universe_domain)
)
)
})
}
]...)
_project_service_agents = {
for k, v in local._project_service_agents_0 :
k => merge(v, {
iam_email = "serviceAccount:${v.email}"
create_jit = v.api == null ? false : contains(local.available_services, v.api)
}) if !contains(local._universe_unavailable_si, k)
}
# list of APIs with primary agents that should be created for the
# current project, if the user requested it
primary_service_agents = [
for agent in local._project_service_agents : agent.api if(
agent.is_primary &&
var.service_agents_config.create_primary_agents &&
agent.create_jit
)
]
# list of roles that should be granted to service agents for the
# current project, if the user requested it
service_agent_roles = {
for agent in local._project_service_agents :
(agent.name) => {
role = agent.role
iam_email = agent.iam_email
}
if alltrue([
var.service_agents_config.grant_default_roles,
agent.role != null,
# FIXME: granting roles to the non-primary agents listed below
# currently fails, possibly because the agents doesn't exist
# after API activation. As a workaround, automatic role
# assignment for these agents is disabled.
!contains([
"apigateway", "apigateway-mgmt", "bigqueryspark", "bigquerytardis",
"firebase", "krmapihosting", "krmapihosting-dataplane", "logging",
"networkactions", "prod-bigqueryomni", "scc-notification", "securitycenter",
], agent.name)
])
}
# map of name->agent including all known aliases
_aliased_service_agents = merge(
local._project_service_agents,
flatten([
for agent_name, agent in local._project_service_agents : [
for alias in agent.aliases :
{ (alias) = agent }
]
])...
)
# same as _aliased_service_agents with unneeded fields removed
aliased_service_agents = {
for k, v in local._aliased_service_agents :
k => {
api = v.api
display_name = v.display_name
email = v.email
iam_email = v.iam_email
is_primary = v.is_primary
role = v.role
}
}
}
data "google_storage_project_service_account" "gcs_sa" {
count = contains(var.services, "storage.googleapis.com") ? 1 : 0
project = local.project.project_id
depends_on = [google_project_service.project_services]
}
data "google_bigquery_default_service_account" "bq_sa" {
count = contains(var.services, "bigquery.googleapis.com") ? 1 : 0
project = local.project.project_id
depends_on = [google_project_service.project_services]
}
moved {
from = google_project_service_identity.jit_si
to = google_project_service_identity.default
}
moved {
from = google_project_service_identity.servicenetworking[0]
to = google_project_service_identity.default["servicenetworking.googleapis.com"]
}
resource "google_project_service_identity" "default" {
provider = google-beta
for_each = toset(local.primary_service_agents)
project = local.project.project_id
service = each.key
depends_on = [google_project_service.project_services]
}
moved {
from = google_project_iam_member.servicenetworking[0]
to = google_project_iam_member.service_agents["service-networking"]
}
resource "google_project_iam_member" "service_agents" {
for_each = local.service_agent_roles
project = local.project.project_id
role = each.value.role
member = each.value.iam_email
depends_on = [
google_project_service.project_services,
google_project_service_identity.default
]
}
resource "google_project_default_service_accounts" "default_service_accounts" {
count = upper(var.default_service_account) == "KEEP" ? 0 : 1
action = upper(var.default_service_account)
project = local.project.project_id
restore_policy = "REVERT_AND_IGNORE_FAILURE"
depends_on = [google_project_service.project_services]
}