kovagoadi/hunfabric
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add ability to refer to other project service accounts in Project Factory

Browse Source
This commit is contained in:
Wiktor Niesiobędzki
2025-02-16 16:19:58 +00:00
committed by Wiktor Niesiobędzki
parent 3ddfd59b88
commit 0a1b6c6e0f
6 changed files with 235 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files

69
modules/project-factory/README.md
View File

@@ -195,26 +195,34 @@ automation:
Interpolations leverage contexts from two separate sources: an internal set for resources managed by the project factory (folders, service accounts, etc.), and an external user-defined set passed in via the `factories_config.context` variable.
The following table lists the available context interpolations. External contexts are passed in via the `factories_config.contexts` variable. IAM principals are interpolated in all IAM attributes except `iam_by_principal`. First two columns show for which attribute of which resource context is interpolated. `external contexts` column show in which map passed as `var.factories_config.context` key will be looked up.
The following table lists the available context interpolations. External contexts are passed in via the `factories_config.contexts` variable. IAM principals are interpolated in all IAM attributes except `iam_by_principal`. First two columns show for which attribute of which resource context is interpolated. `external contexts` column show in which map passed as `var.factories_config.context` key will be looked up.
* Internally created folders creates keys under `${folder_name_1}[/${folder_name_2}/${folder_name_3}]`
* IAM principals are resolved within context of managed project
* IAM principals are resolved within context of managed project or use `${project}/${service_account}` to refer service account from other projects managed by the same project factory instance.
| resource | attribute | external contexts | internal contexts |
| ------------------- | --------------- | ------------------- | -------------------------- |
| folder | parent | `folder_ids` | internally created folders |
| folder | IAM principals | `iam_principals` | |
| folder | tag bindings | `tag_values` | |
| project | parent | `folder_ids` | internally created folders |
| project | Shared VPC host | `vpc_host_projects` | |
| project | Shared VPC IAM | `iam_principals` | |
| project | tag bindings | `tag_values` | |
| project | IAM principals | `iam_principals` | project service accounts |
| | | | IaC service accounts |
| bucket | IAM principals | `iam_principals` | project service accounts |
| service account | IAM projects | `vpc_host_projects` | |
| IaC bucket | IAM principals | `iam_principals` | IaC service accounts |
| IaC service account | IAM principals | `iam_principals` | |
| resource | attribute | external contexts | internal contexts |
|---------------------|-----------------|---------------------|------------------------------------|
| folder | parent | `folder_ids` | implicit through folder structure |
| folder | IAM principals | `iam_principals` | |
| folder | tag bindings | `tag_values` | |
| project | parent | `folder_ids` | internally created folders |
| project | Shared VPC host | `vpc_host_projects` | |
| project | Shared VPC IAM | `iam_principals` | project service accounts |
| | | | IaC service accounts |
| | | | other project service accounts |
| | | | other project IaC service accounts |
| project | tag bindings | `tag_values` | |
| project | IAM principals | `iam_principals` | project service accounts |
| | | | IaC service accounts |
| | | | other project service accounts |
| | | | other project IaC service accounts |
| bucket | IAM principals | `iam_principals` | project service accounts |
| | | | IaC service accounts |
| | | | other project service accounts |
| | | | other project IaC service accounts |
| service account | IAM projects | `vpc_host_projects` | |
| IaC bucket | IAM principals | `iam_principals` | IaC service accounts |
| IaC service account | IAM principals | `iam_principals` | |
## Example
@@ -278,7 +286,7 @@ module "project-factory" {
}
}
}
# tftest modules=16 resources=56 files=0,1,2,3,4,5,6,7,8 inventory=example.yaml
# tftest files=0,1,2,3,4,5,6,7,8,9 inventory=example.yaml
```
A simple hierarchy of folders:
@@ -433,6 +441,31 @@ update_rules:
# tftest-file id=8 path=data/budgets/test-100.yaml schema=budget.schema.json
```
Granting permissions to service accounts defined in other project through interpolation:
```yaml
billing_account: 012345-67890A-BCDEF0
labels:
app: app-0
team: team-b
parent: team-b/app-0
services:
- container.googleapis.com
- storage.googleapis.com
iam:
"roles/run.admin":
- dev-ta-app0-be/app-0-be # interpolate to app-0-be service account in project defined in file dev-ta-app0-be
"roles/run.developer":
- app-0-be # interpolate to app-0-be service account within the same project
service_accounts:
app-0-be:
display_name: "Backend instances."
iam_self_roles:
- roles/logging.logWriter
- roles/monitoring.metricWriter
# tftest-file id=9 path=data/projects/dev-tb-app0-1.yaml schema=project.schema.json
```
<!-- TFDOC OPTS files:1 -->
<!-- BEGIN TFDOC -->
## Files

99
modules/project-factory/main.tf
View File

@@ -27,7 +27,8 @@ locals {
{
for k, v in module.automation-service-accounts :
k => v.iam_email
}
},
# module.service-accounts are excluded here, as adding them here results in dependency cycles
)
}
}
@@ -82,22 +83,6 @@ module "projects" {
var.data_merges.services
))
shared_vpc_host_config = each.value.shared_vpc_host_config
shared_vpc_service_config = (
try(each.value.shared_vpc_service_config.host_project, null) == null
? null
: merge(each.value.shared_vpc_service_config, {
host_project = lookup(
var.factories_config.context.vpc_host_projects,
each.value.shared_vpc_service_config.host_project,
each.value.shared_vpc_service_config.host_project
)
network_users = [
for v in try(each.value.shared_vpc_service_config.network_users, []) :
lookup(local.context.iam_principals, v, v)
]
# TODO: network subnet users
})
)
tag_bindings = {
for k, v in merge(each.value.tag_bindings, var.data_merges.tag_bindings) :
k => lookup(var.factories_config.context.tag_values, v, v)
@@ -112,8 +97,9 @@ module "projects-iam" {
project_reuse = {
use_data_source = false
project_attributes = {
name = module.projects[each.key].name
number = module.projects[each.key].number
name = module.projects[each.key].name
number = module.projects[each.key].number
services_enabled = module.projects[each.key].services
}
}
iam = {
@@ -123,7 +109,9 @@ module "projects-iam" {
module.service-accounts["${each.key}/${vv}"].iam_email,
# automation service account
local.context.iam_principals["${each.key}/${vv}"],
# other context
# other projects service accounts
module.service-accounts[vv].iam_email,
# other automation service account
local.context.iam_principals[vv],
# passthrough
vv
@@ -138,7 +126,9 @@ module "projects-iam" {
module.service-accounts["${each.key}/${vv}"].iam_email,
# automation service account
local.context.iam_principals["${each.key}/${vv}"],
# other context
# other projects service accounts
module.service-accounts[vv].iam_email,
# other automation service account
local.context.iam_principals[vv],
# passthrough
vv
@@ -153,7 +143,9 @@ module "projects-iam" {
module.service-accounts["${each.key}/${v.member}"].iam_email,
# automation service account
local.context.iam_principals["${each.key}/${v.member}"],
# other context
# other projects service accounts
module.service-accounts[v.member].iam_email,
# other automation service account
local.context.iam_principals[v.member],
# passthrough
v.member
@@ -162,6 +154,42 @@ module "projects-iam" {
}
# IAM by principals would trigger dynamic key errors so we don't interpolate
iam_by_principals = try(each.value.iam_by_principals, {})
# Shared VPC configuration is done at stage 2, to avoid dependency cycle between project service accounts and
# IAM grants done for those service accounts
factories_config = {
custom_roles = each.value.factories_config.custom_roles
}
shared_vpc_service_config = (
try(each.value.shared_vpc_service_config.host_project, null) == null
? null
: merge(each.value.shared_vpc_service_config, {
host_project = lookup(
var.factories_config.context.vpc_host_projects,
each.value.shared_vpc_service_config.host_project,
each.value.shared_vpc_service_config.host_project
)
network_users = [
for v in try(each.value.shared_vpc_service_config.network_users, []) :
try(
# project service accounts
module.service-accounts["${each.key}/${v}"].iam_email,
# automation service account
local.context.iam_principals["${each.key}/${v}"],
# other projects service accounts
module.service-accounts[v].iam_email,
# other automation service account
local.context.iam_principals[v],
# passthrough
v
)
]
# TODO: network subnet users
})
)
# add service agents config, so Service Agents can be referred in the IAM grants
service_agents_config = {
grant_default_roles = false # Default roles are granted in module.project
}
}
module "buckets" {
@@ -176,8 +204,15 @@ module "buckets" {
iam = {
for k, v in each.value.iam : k => [
for vv in v : try(
# project service accounts
module.service-accounts["${each.value.project}/${vv}"].iam_email,
var.factories_config.context.iam_principals[vv],
# automation service account
local.context.iam_principals["${each.value.project}/${vv}"],
# other projects service accounts
module.service-accounts[vv].iam_email,
# other automation service account
local.context.iam_principals[vv],
# passthrough
vv
)
]
@@ -186,8 +221,15 @@ module "buckets" {
for k, v in each.value.iam_bindings : k => merge(v, {
members = [
for vv in v.members : try(
# project service accounts
module.service-accounts["${each.value.project}/${vv}"].iam_email,
var.factories_config.context.iam_principals[vv],
# automation service account
local.context.iam_principals["${each.value.project}/${vv}"],
# other projects service accounts
module.service-accounts[vv].iam_email,
# other automation service account
local.context.iam_principals[vv],
# passthrough
vv
)
]
@@ -196,8 +238,15 @@ module "buckets" {
iam_bindings_additive = {
for k, v in each.value.iam_bindings_additive : k => merge(v, {
member = try(
# project service accounts
module.service-accounts["${each.value.project}/${v.member}"].iam_email,
var.factories_config.context.iam_principals[v.member],
# automation service account
local.context.iam_principals["${each.value.project}/${v.member}"],
# other projects service accounts
module.service-accounts[v.member].iam_email,
# other automation service account
local.context.iam_principals[v.member],
# passthrough
v.member
)
})

4
modules/project/README.md
View File

@@ -1648,9 +1648,9 @@ alerts:
| [org_policies](variables.tf#L114) | Organization policies applied to this project keyed by policy name. | <code title="map&#40;object&#40;&#123;&#10; inherit_from_parent &#61; optional&#40;bool&#41; &#35; for list policies only.&#10; reset &#61; optional&#40;bool&#41;&#10; rules &#61; optional&#40;list&#40;object&#40;&#123;&#10; allow &#61; optional&#40;object&#40;&#123;&#10; all &#61; optional&#40;bool&#41;&#10; values &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#10; deny &#61; optional&#40;object&#40;&#123;&#10; all &#61; optional&#40;bool&#41;&#10; values &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#41;&#10; enforce &#61; optional&#40;bool&#41; &#35; for boolean policies only.&#10; condition &#61; optional&#40;object&#40;&#123;&#10; description &#61; optional&#40;string&#41;&#10; expression &#61; optional&#40;string&#41;&#10; location &#61; optional&#40;string&#41;&#10; title &#61; optional&#40;string&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10; parameters &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [parent](variables.tf#L142) | Parent folder or organization in 'folders/folder_id' or 'organizations/org_id' format. | <code>string</code> | | <code>null</code> |
| [prefix](variables.tf#L152) | Optional prefix used to generate project id and name. | <code>string</code> | | <code>null</code> |
| [project_reuse](variables.tf#L162) | Reuse existing project if not null. If name and number are not passed in, a data source is used. | <code title="object&#40;&#123;&#10; use_data_source &#61; optional&#40;bool, true&#41;&#10; project_attributes &#61; optional&#40;object&#40;&#123;&#10; name &#61; string&#10; number &#61; number&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [project_reuse](variables.tf#L162) | Reuse existing project if not null. If name and number are not passed in, a data source is used. | <code title="object&#40;&#123;&#10; use_data_source &#61; optional&#40;bool, true&#41;&#10; project_attributes &#61; optional&#40;object&#40;&#123;&#10; name &#61; string&#10; number &#61; number&#10; services_enabled &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [quotas](variables-quotas.tf#L17) | Service quota configuration. | <code title="map&#40;object&#40;&#123;&#10; service &#61; string&#10; quota_id &#61; string&#10; preferred_value &#61; number&#10; dimensions &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; justification &#61; optional&#40;string&#41;&#10; contact_email &#61; optional&#40;string&#41;&#10; annotations &#61; optional&#40;map&#40;string&#41;&#41;&#10; ignore_safety_checks &#61; optional&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [service_agents_config](variables.tf#L181) | Automatic service agent configuration options. | <code title="object&#40;&#123;&#10; create_primary_agents &#61; optional&#40;bool, true&#41;&#10; grant_default_roles &#61; optional&#40;bool, true&#41;&#10; services_enabled &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [service_agents_config](variables.tf#L182) | Automatic service agent configuration options. | <code title="object&#40;&#123;&#10; create_primary_agents &#61; optional&#40;bool, true&#41;&#10; grant_default_roles &#61; optional&#40;bool, true&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [service_config](variables.tf#L192) | Configure service API activation. | <code title="object&#40;&#123;&#10; disable_on_destroy &#61; bool&#10; disable_dependent_services &#61; bool&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code title="&#123;&#10; disable_on_destroy &#61; false&#10; disable_dependent_services &#61; false&#10;&#125;">&#123;&#8230;&#125;</code> |
| [service_encryption_key_ids](variables.tf#L204) | Service Agents to be granted encryption/decryption permissions over Cloud KMS encryption keys. Format {SERVICE_AGENT => [KEY_ID]}. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [services](variables.tf#L211) | Service APIs to enable. | <code>list&#40;string&#41;</code> | | <code>&#91;&#93;</code> |

6
modules/project/service-agents.tf
View File

@@ -18,7 +18,7 @@
locals {
services = distinct(concat(
local.available_services, var.service_agents_config.services_enabled
local.available_services, try(var.project_reuse.project_attributes.services_enabled, [])
))
_service_agents_data = yamldecode(file("${path.module}/service-agents.yaml"))
# map of api => list of agents
@@ -107,13 +107,13 @@ locals {
}
data "google_storage_project_service_account" "gcs_sa" {
count = contains(local.services, "storage.googleapis.com") ? 1 : 0
count = contains(var.services, "storage.googleapis.com") ? 1 : 0
project = local.project.project_id
depends_on = [google_project_service.project_services]
}
data "google_bigquery_default_service_account" "bq_sa" {
count = contains(local.services, "bigquery.googleapis.com") ? 1 : 0
count = contains(var.services, "bigquery.googleapis.com") ? 1 : 0
project = local.project.project_id
depends_on = [google_project_service.project_services]
}

6
modules/project/variables.tf
View File

@@ -164,8 +164,9 @@ variable "project_reuse" {
type = object({
use_data_source = optional(bool, true)
project_attributes = optional(object({
name = string
number = number
name = string
number = number
services_enabled = optional(list(string), [])
}))
})
default = null
@@ -183,7 +184,6 @@ variable "service_agents_config" {
type = object({
create_primary_agents = optional(bool, true)
grant_default_roles = optional(bool, true)
services_enabled = optional(list(string), [])
})
default = {}
nullable = false

113
tests/modules/project_factory/examples/example.yaml
View File

@@ -12,7 +12,6 @@
# See the License for the specific language governing permissions and
# limitations under the License.
values:
values:
module.project-factory.module.automation-buckets["dev-tb-app0-0/state"].google_storage_bucket.bucket[0]:
autoclass: []
@@ -162,6 +161,18 @@ values:
- serviceAccount:test-pf-dev-tb-app0-0-ro@test-pf-teams-iac-0.iam.gserviceaccount.com
project: test-pf-dev-tb-app0-0
role: roles/viewer
module.project-factory.module.projects-iam["dev-tb-app0-1"].google_project_iam_binding.authoritative["roles/run.admin"]:
condition: []
members:
- serviceAccount:app-0-be@test-pf-dev-ta-app0-be.iam.gserviceaccount.com
project: test-pf-dev-tb-app0-1
role: roles/run.admin
module.project-factory.module.projects-iam["dev-tb-app0-1"].google_project_iam_binding.authoritative["roles/run.developer"]:
condition: []
members:
- serviceAccount:app-0-be@test-pf-dev-tb-app0-1.iam.gserviceaccount.com
project: test-pf-dev-tb-app0-1
role: roles/run.developer
module.project-factory.module.projects["dev-ta-app0-be"].data.google_storage_project_service_account.gcs_sa[0]:
project: test-pf-dev-ta-app0-be
user_project: null
@@ -301,6 +312,68 @@ values:
project: test-pf-dev-tb-app0-0
service: run.googleapis.com
timeouts: null
module.project-factory.module.projects["dev-tb-app0-1"].data.google_storage_project_service_account.gcs_sa[0]:
project: test-pf-dev-tb-app0-1
user_project: null
module.project-factory.module.projects["dev-tb-app0-1"].google_essential_contacts_contact.contact["admin@example.org"]:
email: admin@example.org
language_tag: en
notification_category_subscriptions:
- ALL
parent: projects/test-pf-dev-tb-app0-1
timeouts: null
module.project-factory.module.projects["dev-tb-app0-1"].google_project.project[0]:
auto_create_network: false
billing_account: 012345-67890A-BCDEF0
deletion_policy: DELETE
effective_labels:
app: app-0
environment: test
goog-terraform-provisioned: 'true'
team: team-b
labels:
app: app-0
environment: test
team: team-b
name: test-pf-dev-tb-app0-1
project_id: test-pf-dev-tb-app0-1
tags: null
terraform_labels:
app: app-0
environment: test
goog-terraform-provisioned: 'true'
team: team-b
timeouts: null
module.project-factory.module.projects["dev-tb-app0-1"].google_project_iam_member.service_agents["container-engine-robot"]:
condition: []
project: test-pf-dev-tb-app0-1
role: roles/container.serviceAgent
module.project-factory.module.projects["dev-tb-app0-1"].google_project_iam_member.service_agents["gkenode"]:
condition: []
project: test-pf-dev-tb-app0-1
role: roles/container.defaultNodeServiceAgent
? module.project-factory.module.projects["dev-tb-app0-1"].google_project_service.project_services["container.googleapis.com"]
: disable_dependent_services: false
disable_on_destroy: false
project: test-pf-dev-tb-app0-1
service: container.googleapis.com
timeouts: null
? module.project-factory.module.projects["dev-tb-app0-1"].google_project_service.project_services["stackdriver.googleapis.com"]
: disable_dependent_services: false
disable_on_destroy: false
project: test-pf-dev-tb-app0-1
service: stackdriver.googleapis.com
timeouts: null
module.project-factory.module.projects["dev-tb-app0-1"].google_project_service.project_services["storage.googleapis.com"]:
disable_dependent_services: false
disable_on_destroy: false
project: test-pf-dev-tb-app0-1
service: storage.googleapis.com
timeouts: null
? module.project-factory.module.projects["dev-tb-app0-1"].google_project_service_identity.default["container.googleapis.com"]
: project: test-pf-dev-tb-app0-1
service: container.googleapis.com
timeouts: null
module.project-factory.module.projects["teams-iac-0"].data.google_storage_project_service_account.gcs_sa[0]:
project: test-pf-teams-iac-0
user_project: null
@@ -403,25 +476,43 @@ values:
member: serviceAccount:app-0-fe@test-pf-dev-ta-app0-be.iam.gserviceaccount.com
project: test-pf-dev-ta-app0-be
timeouts: null
module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-tb-app0-1-roles/logging.logWriter"]:
condition: []
project: test-pf-dev-tb-app0-1
role: roles/logging.logWriter
module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_project_iam_member.project-roles["test-pf-dev-tb-app0-1-roles/monitoring.metricWriter"]:
condition: []
project: test-pf-dev-tb-app0-1
role: roles/monitoring.metricWriter
module.project-factory.module.service-accounts["dev-tb-app0-1/app-0-be"].google_service_account.service_account[0]:
account_id: app-0-be
create_ignore_already_exists: null
description: null
disabled: false
display_name: Backend instances.
email: app-0-be@test-pf-dev-tb-app0-1.iam.gserviceaccount.com
member: serviceAccount:app-0-be@test-pf-dev-tb-app0-1.iam.gserviceaccount.com
project: test-pf-dev-tb-app0-1
timeouts: null
counts:
google_billing_budget: 1
google_compute_shared_vpc_host_project: 1
google_compute_shared_vpc_service_project: 1
google_essential_contacts_contact: 3
google_essential_contacts_contact: 4
google_folder: 5
google_folder_iam_binding: 1
google_kms_crypto_key_iam_member: 1
google_monitoring_notification_channel: 1
google_project: 3
google_project_iam_binding: 2
google_project_iam_member: 14
google_project_service: 9
google_project_service_identity: 3
google_service_account: 4
google_project: 4
google_project_iam_binding: 4
google_project_iam_member: 18
google_project_service: 12
google_project_service_identity: 4
google_service_account: 5
google_storage_bucket: 1
google_storage_bucket_iam_binding: 2
google_storage_project_service_account: 3
google_storage_project_service_account: 4
google_tags_tag_binding: 1
modules: 16
resources: 56
modules: 19
resources: 70