FAST light implementation (#3255)

* data wip

* wip data

* update org schema, add note on expansion

* all schemas, workload notes

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* wip

* data wip

* wip

* wip

* wip

* wip

* org module IAM context (using lookup)

* new-style context expansion in project IAM

* remove spurious file

* project module contexts

* finalize context replacement format for project module

* revert org module changes

* fix tag id interpolation in project

* fix tag id interpolation in project

* organization module context

* organization context test

* context expansion for folder tag bindings

* test context expansion for tag bindings

* service account module context

* simplify context local

* context for iam service account

* nuke blueprints

* remove links to blueprints

* vpc sc context in project module

* Add context to GCS module

* Add inline deps to plan_summary script

* Make context a top-level variable for folder, organization, sa

* Add add context top-level to VPC-SC

* move context out of factories_config variable

* tfdoc

* fix merge

* fix merge

* fix examples

* net-vpc module context

* add parent ids to folder context

* rename folder parent context

* fix folder parent check

* new project factory stub

* wip

* wip

* refactor defaults

* project iam

* bueckts and service accounts

* start adding context replacements

* better test data

* automation resources for folders and projects

* automation

* add support for project id interpolation

* first tested apply

* improve IAM description in gcs module

* add context to billing account module

* add notification channels to billing account module context

* add billing budgets to new pf

* schemas and defaults

* bootstrap wip

* bootstrap wip

* bootstrap wip

* pf outputs

* pf fixes

* fix pf sample data

* bootstrap lite fixes

* add locations to organization module contexts

* bootstrap lite fixes

* org fixes, billing accounts

* fix default project parent

* bootstrap lite wip

* add locations to gcs module context

* add context support to logging bucket module

* add context to pubsub module

* split out iam variables in gcs module

* fix logging bucket context test

* bootstrap log sink destinations

* streamline logging-bucket module variables

* fix logging bucket context test

* align logging bucket module interface in fast bootstrap

* add support for project-level log buckets to project factory

* support full context expansion in organization module log sinks

* log buckets in fast-lite bootstrap

* make og sink type optional in organization module

* log sinks in fast-lite bootstrap

* set tag values in factory context

* bootstrap lite data

* output files schema

* billing account schema

* output files

* output providers

* gcs output files

* boilerplate

* tflint

* check documentation

* check docs

* fix project module parent variable validation

* fix log bucket examples

* allow null parent in project module

* silence folder test errors

* fix billing account sink example

* fix project example

* fix billing account module

* fix folder tests

* fix FAST

* fix fast

* tfvars outputs

* wif

* cicd service accounts

* cicd

* allow defaults in context, minimal org policies

* support gcs managed folders in project factory and bootstrap lite

* support prefix in provider output files

* rename bootstrap stage

* gitignore

* gitignore

* security folder, billing IAM

* wip tfvars

* fix typo

* security IAM

* control tag iam/context via variables in organization module

* split tag creation from tag IAM to avoid circular refs

* port organization module tag changes to project module

* implement new-style context expansion in vpc-sc module

* fix fast vpc-sc tests

* boilerplate

* vpc sc stage

* schemas

* fast-lite compatibility for vpc sc stage

* make log project number optional in vpc-sc stage

* networking

* networking

* networking

* networking

* rename and move new stage under fast

* clone pf tests

* use context replacement for internal notification channels in billing account module

* support service agents in project module iam context replacements

* support service agents in project module iam context replacements

* add support for kms keys to project module context

* experimental pf example test and fixes

* fix schemas

* fix tests

* tfdoc

* tfdoc

* pf config

* experimental pf

* remove redundant dot from gcs managed folder IAM keys

* bootstrap experimental test

* project factory exp stage test

* skip tflint for bootstrap experimental test

* tflint

* fix gcs test

* documentation work

* documentation work

* Update README.md

* tfdoc

* tfdoc

* readme

* tfdoc

* readme

* readme

* readme

* readme

* support universe in pf exp projects

* missing universe service agents

* org policies import, non-admin billing IAM

* todo

* fix test

* custom constraints

* fast classic dataset

* fix test data

* context replacements in billing module log sinks

* fix typo

* add support for billing log sinks

* update docs

* readme

* cicd fix and test

---------

Co-authored-by: Julio Castillo <jccb@google.com>
This commit is contained in:
Ludovico Magnocavallo
2025-09-02 08:38:57 +02:00
committed by GitHub
parent dcd762370c
commit 36648b6b63
1180 changed files with 22259 additions and 87165 deletions

View File

@@ -207,7 +207,6 @@ The caller must have `cloudasset.assets.searchAllResources` permission to perfor
module "vpc-sc" {
source = "./fabric/modules/vpc-sc"
project_id_search_scope = var.org_id
access_policy = "12345678"
ingress_policies = {
i1 = {
@@ -226,7 +225,6 @@ module "vpc-sc" {
}
}
}
perimeters = {
p = {
spec = {
@@ -252,16 +250,16 @@ JSON Schema files for each factory object are available in the [`schemas`](./sch
module "test" {
source = "./fabric/modules/vpc-sc"
access_policy = "12345678"
context = {
resource_sets = {
foo_projects = ["projects/321", "projects/654"]
}
}
factories_config = {
access_levels = "data/access-levels"
egress_policies = "data/egress-policies"
ingress_policies = "data/ingress-policies"
perimeters = "data/perimeters"
context = {
resource_sets = {
foo_projects = ["projects/321", "projects/654"]
}
}
}
}
# tftest modules=1 resources=3 files=p1,a1,a2,e1,i1,i2 inventory=factory.yaml
@@ -373,7 +371,7 @@ to:
- service_name: "*"
resources:
- projects/1234567890
- foo_projects
- $resource_sets:foo_projects
# tftest-file id=i2 path=data/ingress-policies/sa-tf-test-geo.yaml schema=ingress-policy.schema.json
```
@@ -404,14 +402,15 @@ to:
| [access_policy](variables.tf#L68) | Access Policy name, set to null if creating one. | <code>string</code> | ✓ | |
| [access_levels](variables.tf#L17) | Access level definitions. | <code title="map&#40;object&#40;&#123;&#10; combining_function &#61; optional&#40;string&#41;&#10; conditions &#61; optional&#40;list&#40;object&#40;&#123;&#10; device_policy &#61; optional&#40;object&#40;&#123;&#10; allowed_device_management_levels &#61; optional&#40;list&#40;string&#41;&#41;&#10; allowed_encryption_statuses &#61; optional&#40;list&#40;string&#41;&#41;&#10; require_admin_approval &#61; bool&#10; require_corp_owned &#61; bool&#10; require_screen_lock &#61; optional&#40;bool&#41;&#10; os_constraints &#61; optional&#40;list&#40;object&#40;&#123;&#10; os_type &#61; string&#10; minimum_version &#61; optional&#40;string&#41;&#10; require_verified_chrome_os &#61; optional&#40;bool&#41;&#10; &#125;&#41;&#41;&#41;&#10; &#125;&#41;&#41;&#10; ip_subnetworks &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; members &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; negate &#61; optional&#40;bool&#41;&#10; regions &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; required_access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; vpc_subnets &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; description &#61; optional&#40;string&#41;&#10; title &#61; optional&#40;string&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [access_policy_create](variables.tf#L73) | Access Policy configuration, fill in to create. Parent is in 'organizations/123456' format, scopes are in 'folders/456789' or 'projects/project_id' format. | <code title="object&#40;&#123;&#10; parent &#61; string&#10; title &#61; string&#10; scopes &#61; optional&#40;list&#40;string&#41;, null&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [egress_policies](variables.tf#L83) | Egress policy definitions that can be referenced in perimeters. | <code title="map&#40;object&#40;&#123;&#10; title &#61; optional&#40;string&#41;&#10; from &#61; object&#40;&#123;&#10; access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; identity_type &#61; optional&#40;string&#41;&#10; identities &#61; optional&#40;list&#40;string&#41;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;&#10; to &#61; object&#40;&#123;&#10; external_resources &#61; optional&#40;list&#40;string&#41;&#41;&#10; operations &#61; optional&#40;list&#40;object&#40;&#123;&#10; method_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; permission_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_name &#61; string&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;&#41;&#10; roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [factories_config](variables.tf#L126) | Paths to folders that enable factory functionality. | <code title="object&#40;&#123;&#10; access_levels &#61; optional&#40;string&#41;&#10; egress_policies &#61; optional&#40;string&#41;&#10; ingress_policies &#61; optional&#40;string&#41;&#10; perimeters &#61; optional&#40;string&#41;&#10; context &#61; optional&#40;object&#40;&#123;&#10; resource_sets &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_sets &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; identity_sets &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; &#125;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam](variables.tf#L143) | IAM bindings in {ROLE => [MEMBERS]} format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam_bindings](variables.tf#L149) | Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. | <code title="map&#40;object&#40;&#123;&#10; members &#61; list&#40;string&#41;&#10; role &#61; string&#10; condition &#61; optional&#40;object&#40;&#123;&#10; expression &#61; string&#10; title &#61; string&#10; description &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam_bindings_additive](variables.tf#L164) | Individual additive IAM bindings. Keys are arbitrary. | <code title="map&#40;object&#40;&#123;&#10; member &#61; string&#10; role &#61; string&#10; condition &#61; optional&#40;object&#40;&#123;&#10; expression &#61; string&#10; title &#61; string&#10; description &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [ingress_policies](variables.tf#L179) | Ingress policy definitions that can be referenced in perimeters. | <code title="map&#40;object&#40;&#123;&#10; title &#61; optional&#40;string&#41;&#10; from &#61; object&#40;&#123;&#10; access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; identity_type &#61; optional&#40;string&#41;&#10; identities &#61; optional&#40;list&#40;string&#41;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;&#10; to &#61; object&#40;&#123;&#10; operations &#61; optional&#40;list&#40;object&#40;&#123;&#10; method_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; permission_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_name &#61; string&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;&#41;&#10; roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [perimeters](variables.tf#L221) | Regular service perimeters. | <code title="map&#40;object&#40;&#123;&#10; description &#61; optional&#40;string&#41;&#10; ignore_resource_changes &#61; optional&#40;bool, false&#41;&#10; title &#61; optional&#40;string&#41;&#10; use_explicit_dry_run_spec &#61; optional&#40;bool, false&#41;&#10; spec &#61; optional&#40;object&#40;&#123;&#10; access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; egress_policies &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; ingress_policies &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; restricted_services &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; vpc_accessible_services &#61; optional&#40;object&#40;&#123;&#10; allowed_services &#61; list&#40;string&#41;&#10; enable_restriction &#61; optional&#40;bool, true&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;&#10; status &#61; optional&#40;object&#40;&#123;&#10; access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; egress_policies &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; ingress_policies &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; restricted_services &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; vpc_accessible_services &#61; optional&#40;object&#40;&#123;&#10; allowed_services &#61; list&#40;string&#41;&#10; enable_restriction &#61; optional&#40;bool, true&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [project_id_search_scope](variables.tf#L255) | Set this to an organization or folder ID to use Cloud Asset Inventory to automatically translate project ids to numbers. | <code>string</code> | | <code>null</code> |
| [context](variables.tf#L83) | External context used in replacements. | <code title="object&#40;&#123;&#10; identity_sets &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; project_numbers &#61; optional&#40;map&#40;number&#41;, &#123;&#125;&#41;&#10; resource_sets &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; service_sets &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [egress_policies](variables.tf#L95) | Egress policy definitions that can be referenced in perimeters. | <code title="map&#40;object&#40;&#123;&#10; title &#61; optional&#40;string&#41;&#10; from &#61; object&#40;&#123;&#10; access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; identity_type &#61; optional&#40;string&#41;&#10; identities &#61; optional&#40;list&#40;string&#41;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;&#10; to &#61; object&#40;&#123;&#10; external_resources &#61; optional&#40;list&#40;string&#41;&#41;&#10; operations &#61; optional&#40;list&#40;object&#40;&#123;&#10; method_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; permission_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_name &#61; string&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;&#41;&#10; roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [factories_config](variables.tf#L138) | Paths to folders that enable factory functionality. | <code title="object&#40;&#123;&#10; access_levels &#61; optional&#40;string&#41;&#10; egress_policies &#61; optional&#40;string&#41;&#10; ingress_policies &#61; optional&#40;string&#41;&#10; perimeters &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam](variables.tf#L150) | IAM bindings in {ROLE => [MEMBERS]} format. | <code>map&#40;list&#40;string&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam_bindings](variables.tf#L156) | Authoritative IAM bindings in {KEY => {role = ROLE, members = [], condition = {}}}. Keys are arbitrary. | <code title="map&#40;object&#40;&#123;&#10; members &#61; list&#40;string&#41;&#10; role &#61; string&#10; condition &#61; optional&#40;object&#40;&#123;&#10; expression &#61; string&#10; title &#61; string&#10; description &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [iam_bindings_additive](variables.tf#L171) | Individual additive IAM bindings. Keys are arbitrary. | <code title="map&#40;object&#40;&#123;&#10; member &#61; string&#10; role &#61; string&#10; condition &#61; optional&#40;object&#40;&#123;&#10; expression &#61; string&#10; title &#61; string&#10; description &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [ingress_policies](variables.tf#L186) | Ingress policy definitions that can be referenced in perimeters. | <code title="map&#40;object&#40;&#123;&#10; title &#61; optional&#40;string&#41;&#10; from &#61; object&#40;&#123;&#10; access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; identity_type &#61; optional&#40;string&#41;&#10; identities &#61; optional&#40;list&#40;string&#41;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; &#125;&#41;&#10; to &#61; object&#40;&#123;&#10; operations &#61; optional&#40;list&#40;object&#40;&#123;&#10; method_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; permission_selectors &#61; optional&#40;list&#40;string&#41;&#41;&#10; service_name &#61; string&#10; &#125;&#41;&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;&#41;&#10; roles &#61; optional&#40;list&#40;string&#41;&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [perimeters](variables.tf#L228) | Regular service perimeters. | <code title="map&#40;object&#40;&#123;&#10; description &#61; optional&#40;string&#41;&#10; ignore_resource_changes &#61; optional&#40;bool, false&#41;&#10; title &#61; optional&#40;string&#41;&#10; use_explicit_dry_run_spec &#61; optional&#40;bool, false&#41;&#10; spec &#61; optional&#40;object&#40;&#123;&#10; access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; egress_policies &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; ingress_policies &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; restricted_services &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; vpc_accessible_services &#61; optional&#40;object&#40;&#123;&#10; allowed_services &#61; list&#40;string&#41;&#10; enable_restriction &#61; optional&#40;bool, true&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;&#10; status &#61; optional&#40;object&#40;&#123;&#10; access_levels &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; egress_policies &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; ingress_policies &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; resources &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; restricted_services &#61; optional&#40;list&#40;string&#41;, &#91;&#93;&#41;&#10; vpc_accessible_services &#61; optional&#40;object&#40;&#123;&#10; allowed_services &#61; list&#40;string&#41;&#10; enable_restriction &#61; optional&#40;bool, true&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [project_id_search_scope](variables.tf#L262) | Set this to an organization or folder ID to use Cloud Asset Inventory to automatically translate project ids to numbers. | <code>string</code> | | <code>null</code> |
## Outputs