Commit Graph

63 Commits

Author SHA1 Message Date
kovagoadam
1907c38e22 Add IAM deny policies support (#3970)
* Added IAM denial policies

* Moved default to empty, removed trys, added condition vars to expression

* remove redundant null checks

* reduce line length

* boilerplate and principal context expansion

* update readmes

* add explicit validation against null values

* add context tests

* Add missing license headers to examples

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2026-05-21 02:38:06 +00:00
Simon Roberts
26dbaa2d6e Enable terraform_naming_convention in tflint (#3930)
* Draft terraform_naming_convention

* Two fast/stages fixes for terraform_naming_convention

* Disable terraform_naming_convention for resources for now

* module fixes for terraform_naming_convention

* tfdoc

* Remove "moved" from recipe and needs-fixing

* Fix moved for spoke_ra

* fix tests

* Use default (snake_case) for resources

* factory.terraform_data.project-preconditions

* First-pass migration of resources + tests

* Fix tests/modules/organization

* Require snake_case for variables; Add annotations for _testing

* permit _fast_debug variable

* Fix net_vpc_factory and net_vpc_firewall tests

* tfdoc addons and recipe

* Fix more tests

* Fix some net-global -> net_global tests

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2026-05-06 06:06:26 +00:00
Julio Castillo
2eaa0d5e27 Add support for dynamic tags (#3897)
* Allow creation of dynamic tags

* Extend project factory and related modules to support dynamic values

* Extend folder and organization modules

* project and organization readme

* Simplify dynamic tag support and remove unnecessary restrictions

  • Schemas & Validations: Removed the restriction that forbade combining IAM fields with  allowed_values_regex  on tags. Updated validations in  project  and  organization  modules, and
  simplified all relevant JSON schemas.
  • Module Tag Bindings: Simplified the  tag_value  assignment in  folder ,  project ,  gcs ,  bigquery-dataset , and  kms  modules by removing the defensive  can(regex(...))  check and
  calling  templatestring  directly.
  • Outputs: Removed the  tags_dynamic  output from  project  and  organization  modules, as the same information is now available in  tag_keys .
  • Project Factory: Updated  tag_vars_projects  in  projects.tf  to use the native  namespaced_name  attribute and filtered manually for dynamic tags.

* fix(organization, project): fix linting and tests for dynamic tag support

- Align allowed_values_regex and description extraction in _tags_merged
  locals to use lookup() for consistency with other fields.
- Fix spacing in project context variable (alphabetical ordering).
- Update organization tags test to include the new cost_center tag key
  with allowed_values_regex.
- Update project tags test to include the new cost_center tag key and
  reflect the resolved allowed_values_regex on environment.

* refactor(gcs): refine tag bindings and fix context test

- Add _tag_bindings local to pre-resolve context references, enabling
  templatestring to receive a direct map reference (required by Terraform).
- Use var.context.tag_vars instead of the non-existent local.ctx.tag_vars.
- Fix HCL syntax in context.tfvars (escaped inner quotes).
- Update context test inventory to reflect 3 tag bindings including a
  dynamic value resolved via templatestring.

* refactor: align modules with tag binding context pattern

- Add _tag_bindings local + templatestring dance to cloud-run-v2,
  compute-vm, folder, kms modules (bigquery-dataset already had it)
- Exclude tag_vars from local.ctx in cloud-run-v2, compute-vm, folder,
  kms, project modules (bigquery-dataset already had it)
- Add tag_vars to context variable in cloud-run-v2, compute-vm modules
  (others already had it)
- Update all context tests with dynamic tag binding values using
  var.context.tag_vars

* docs: add module-level tftest.yaml test instructions to GEMINI.md

* docs: regenerate READMEs after tag-regex alignment

- Regenerate variable tables in 7 module READMEs to reflect
  line number shifts from prior tag-regex changes
- Add tag_vars exclusion to gcs ctx local
- Fix whitespace alignment in iam-service-account and
  project-factory tag_vars blocks
- Update tftest resource counts for organization and project
- Remove tags_dynamic from organization/project output tables

* fix(project-factory): update test inventory for tag_bindings module split

- Move tag binding address from folder-2 to folder-2-iam in test
  inventory (tag_bindings moved from creation to IAM modules)
- Update module instance count from 34 to 35
- Regenerate README tables after terraform fmt line shifts
- Apply terraform fmt to variables.tf

* refactor(project-factory): remove unnecessary depends_on from folder-iam modules

Folder IAM modules depend on their own folder creation modules, not
on module.projects. The explicit depends_on was leftover from an
earlier design.

* FAST stages

* Address review comments.

- FAST Stages:
  - Added tag_keys to output-files.tf in 0-org-setup to pass org tags via tfvars.
  - Sorted tag_keys and tag_values in output-files.tf.
  - Updated project-factory, networking, and security stages to use tag_keys.
  - Filtered tag_keys for dynamic tags only.
- Modules:
  - Excluded tag_vars from local.ctx in iam-service-account and organization.
  - Simplified tag_value in iam-service-account.
- Tests:
  - Updated test inventories for 0-org-setup and project-factory.

* Fix tf format

* Fix tfdoc

* docs: add ADR for templatestring vars convention and update status of base path ADR

* More tfdoc

* Update schemas

* Use endswith in context loop

* Address review

* Update FAST readmes

* Update last modules

* Terraform fmt

* Revert alloydb

* Fix whitespace

---------

Co-authored-by: Ludovico Magnocavallo <ludo@qix.it>
2026-04-24 20:45:45 +00:00
Luca Prete
f2d4e937d1 Fix pre-commit hook (#3882) 2026-04-18 10:07:14 +02:00
jnahelou
78e00682f8 Ignored condition attribute in IAM tag bindings within the organisation/project modules (#3762)
* fix(modules/organization): conditions ignored in tags

* fix(modules/project): conditions ignored in tags

* fix(modules/project): Tags:1 test skipped due to bad markdown block

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2026-03-18 17:12:46 +00:00
Vannick Trinquier
2af44b0651 Add support for security command center mute rules in module organization, folder and project (#3694) 2026-02-04 08:31:05 +07:00
Julio Castillo
d9e1b924a1 Add asset_feeds to resman modules (#3658)
* Add asset_feeds to resman modules

* Add examples and update readmes

* Extend pubsub_topic context to project and folder modules

* Use pubsub_topic context for pubsub_destination

* Update readmes and add project-factory asset_feed example

* Update context tests

* Update schemas
2026-01-20 14:37:35 +00:00
Julio Castillo
cff8a25c59 Introduce iam_by_principals_conditional (#3649)
* Introduce iam_by_principals_conditional

* Add iam_by_principals_conditional to project factory

* Update IAM ADR

* Update project factory readme

* Sync FAST schemas

* Update organization schema

* Add resman tests for iam_by_principals_conditional

* Update PF project-defaults.tf

* Update copyright
2026-01-14 11:16:07 +00:00
Eric Zhao
c1248d328a Allow any VPC for (secure) network_tags (#3634)
* feat: allow all for VPC networks

* feat: add examples

* feat: add header

* feat: module test

* fix: update network testing data to pass validation

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2026-01-12 09:34:18 +00:00
Vannick Trinquier
cc24046be8 Add CMEK support to FAST and controls for CMEK encryption (#3556) 2025-12-14 12:14:08 +07:00
Ludovico Magnocavallo
897c6ef8c3 Add support for Workload Identity to project module and project factory (#3531)
* module-level support

* fast stage 0

* fix inventory, add outputs/tfvars

* wip

* project factory

* pf outputs

* iam templates will be added where ci/cd configs are managed

* fix merge conflicts
2025-11-17 07:31:21 +00:00
Ludovico Magnocavallo
fc7aa71ada Add support for KMS key creation to project factory (#3518)
* initial implementation

* context

* tfdoc

* add support for autokey to projects

* fix typo
2025-11-11 07:23:50 +01:00
Wiktor Niesiobędzki
d5bc59a238 Fix E2E tests.
* Disable tests for VPC connector and Cloud Functions, CFs are not
  supporrted in the default region
* fix permissions to secrets for Cloud Run
* add permissions admin permissions to any SA within project to
  `var.bucket`
* add permissions to access the secret to any SA within project to
  secrets created by fixture
* disable custom roles in E2E tests, as `var.organization_id` is not the
  same org, within which projects are created in E2E
2025-11-04 10:23:55 +01:00
lcaggio
64632eb957 BigQuery reservation (#3441)
* First Commit.

* Improve

* Fix README

* Fix lint

* fix

* Fix Tests.

* Improve output

* fix yaml lint

* Fix

* Fic test

* Fix

* Fix output
2025-11-03 11:36:42 +01:00
Wiktor Niesiobędzki
f7c9a341b0 yamlint tests/ 2025-10-24 13:11:17 +02:00
Julio Castillo
772d064e1c Skip IAM grants for service agents that are not created on API activation (#3448)
* Skip IAM grants for service agents that are not created on API activation

* Fix tests
2025-10-21 14:31:32 +00:00
Julio Castillo
1566711c3a Add service agent outputs to folder and organization (#3436)
* Add service agent outputs to folder and organization

* Fix tests
2025-10-17 17:23:08 +02:00
Wiktor Niesiobędzki
9f2beb1ee6 Use fixture custom role for modules/project tests 2025-10-13 18:58:52 +02:00
Wiktor Niesiobędzki
849e856437 Use email from variables in tests 2025-10-13 18:58:52 +02:00
Vannick Trinquier
cfe2e21ce7 feat: add support for SCC Custom Security Health Analytics module in … (#3372)
* feat: add support for SCC Custom Security Health Analytics module in organization, folder and project modules

* fix: update description and docs

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2025-10-03 13:47:50 +02:00
Ludovico Magnocavallo
36648b6b63 FAST light implementation (#3255)
* data wip

* wip data

* update org schema, add note on expansion

* all schemas, workload notes

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* wip

* data wip

* wip

* wip

* wip

* wip

* org module IAM context (using lookup)

* new-style context expansion in project IAM

* remove spurious file

* project module contexts

* finalize context replacement format for project module

* revert org module changes

* fix tag id interpolation in project

* fix tag id interpolation in project

* organization module context

* organization context test

* context expansion for folder tag bindings

* test context expansion for tag bindings

* service account module context

* simplify context local

* context for iam service account

* nuke blueprints

* remove links to blueprints

* vpc sc context in project module

* Add context to GCS module

* Add inline deps to plan_summary script

* Make context a top-level variable for folder, organization, sa

* Add add context top-level to VPC-SC

* move context out of factories_config variable

* tfdoc

* fix merge

* fix merge

* fix examples

* net-vpc module context

* add parent ids to folder context

* rename folder parent context

* fix folder parent check

* new project factory stub

* wip

* wip

* refactor defaults

* project iam

* bueckts and service accounts

* start adding context replacements

* better test data

* automation resources for folders and projects

* automation

* add support for project id interpolation

* first tested apply

* improve IAM description in gcs module

* add context to billing account module

* add notification channels to billing account module context

* add billing budgets to new pf

* schemas and defaults

* bootstrap wip

* bootstrap wip

* bootstrap wip

* pf outputs

* pf fixes

* fix pf sample data

* bootstrap lite fixes

* add locations to organization module contexts

* bootstrap lite fixes

* org fixes, billing accounts

* fix default project parent

* bootstrap lite wip

* add locations to gcs module context

* add context support to logging bucket module

* add context to pubsub module

* split out iam variables in gcs module

* fix logging bucket context test

* bootstrap log sink destinations

* streamline logging-bucket module variables

* fix logging bucket context test

* align logging bucket module interface in fast bootstrap

* add support for project-level log buckets to project factory

* support full context expansion in organization module log sinks

* log buckets in fast-lite bootstrap

* make og sink type optional in organization module

* log sinks in fast-lite bootstrap

* set tag values in factory context

* bootstrap lite data

* output files schema

* billing account schema

* output files

* output providers

* gcs output files

* boilerplate

* tflint

* check documentation

* check docs

* fix project module parent variable validation

* fix log bucket examples

* allow null parent in project module

* silence folder test errors

* fix billing account sink example

* fix project example

* fix billing account module

* fix folder tests

* fix FAST

* fix fast

* tfvars outputs

* wif

* cicd service accounts

* cicd

* allow defaults in context, minimal org policies

* support gcs managed folders in project factory and bootstrap lite

* support prefix in provider output files

* rename bootstrap stage

* gitignore

* gitignore

* security folder, billing IAM

* wip tfvars

* fix typo

* security IAM

* control tag iam/context via variables in organization module

* split tag creation from tag IAM to avoid circular refs

* port organization module tag changes to project module

* implement new-style context expansion in vpc-sc module

* fix fast vpc-sc tests

* boilerplate

* vpc sc stage

* schemas

* fast-lite compatibility for vpc sc stage

* make log project number optional in vpc-sc stage

* networking

* networking

* networking

* networking

* rename and move new stage under fast

* clone pf tests

* use context replacement for internal notification channels in billing account module

* support service agents in project module iam context replacements

* support service agents in project module iam context replacements

* add support for kms keys to project module context

* experimental pf example test and fixes

* fix schemas

* fix tests

* tfdoc

* tfdoc

* pf config

* experimental pf

* remove redundant dot from gcs managed folder IAM keys

* bootstrap experimental test

* project factory exp stage test

* skip tflint for bootstrap experimental test

* tflint

* fix gcs test

* documentation work

* documentation work

* Update README.md

* tfdoc

* tfdoc

* readme

* tfdoc

* readme

* readme

* readme

* readme

* support universe in pf exp projects

* missing universe service agents

* org policies import, non-admin billing IAM

* todo

* fix test

* custom constraints

* fast classic dataset

* fix test data

* context replacements in billing module log sinks

* fix typo

* add support for billing log sinks

* update docs

* readme

* cicd fix and test

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2025-09-02 08:38:57 +02:00
Liam Nesteroff
1fbb2cb330 Added tag factory option for organization module (#3178)
* Added tag factory option for organization module

* added tags-factory tests

* added tag factory for project module

* missing header

* added lookup catch for org tag values and fixed incorrect link in doco

* fixed factory locals from copy/paste

* added full doco/tests for project tags factory. fixed failed test looking for ID in yamls

* added context option for factories_config to configure existing K/Vs

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2025-06-23 16:24:43 +10:00
Wiktor Niesiobędzki
6af479706d Fix failing E2E test 2025-02-28 18:45:14 +01:00
Ludovico Magnocavallo
1a4b298cc9 Project factory additions, project module reuse implementation (#2899)
* add support for buckets

* add project-level interpolation for own SAs

* docs

* project reuse changes

* fix example

* tfdoc

* update check documentation tool

* fast tests

* blueprints

* typo
2025-02-15 20:37:45 +01:00
Julio Castillo
d0c3f04c10 Make service agents work in different universes (#2894)
* Make service agents work in different universes

* Use templatestring and two passes for service agent emails

* Fix tests
2025-02-14 12:16:07 +00:00
Julio Castillo
3ffe838e06 Add context to organization policiy factories (#2876) 2025-02-10 22:24:01 +00:00
Wiktor Niesiobędzki
58f22ce647 Add prefix to KMS ring, to allow easy recreate 2025-02-10 00:56:19 +01:00
Julio Castillo
e08698a410 Fix tests 2025-02-07 10:55:05 +01:00
Julio Castillo
3fc7144c4f Update examples and tests 2025-02-07 10:55:05 +01:00
Julio Castillo
8a2320311d Add bucket_create to modules/gcs (#2827)
* Add `bucket_create` to `modules/gcs`

* Create local with bucket name

* Update variable description

* Fix bucket output

* Fix tests

* Fix tests

* Bump OpenTofu to 1.9.0 (needed for multi-var validations)
2025-01-21 23:48:36 +01:00
Julio Castillo
f1acc92864 Add support for log views and log scopes (#2776)
* Add views and tags to logging bucket

* Add logs scopes to project

* Add missing inventory
2024-12-18 18:29:44 +01:00
Wiktor Niesiobędzki
44a4268afc Fix E2E tests 2024-09-04 12:16:50 +02:00
Julio Castillo
f57635d044 Add managed folders suports to gcs module (#2530)
* Add RPO, make versioning dynamic

* Add manaed folders

* Change autoclass and cors defaults to null

* Update README

* Add iam_by_principals

* Add managed folders var description

* Remove need for managed folders to end in /

* Add inventory to example

* Update readme

* Fix FAST tests
2024-08-28 07:30:52 +00:00
Julio Castillo
0420dec32f Add deletion_policy to project module (#2502)
* Replace skip_delete with deletion_policy

* Fix blueprints

* Update apigee blueprint readme

* Remove skip_delete from test inventories
2024-08-16 18:33:39 +02:00
Wiktor Niesiobędzki
d395e9490d Fix quotas E2E tests 2024-08-06 11:49:29 +02:00
Julio Castillo
1bbff3cc3a Add support for dry-run org policies (#2454) 2024-07-30 13:12:57 +00:00
Julio Castillo
c0bf32e797 Refactor service agent management (#2423)
* Service agents script

* Service agents update

* WIP

* Update script and terraform

* Fix tests

* Fix linter

* Update docs

* Bring back pf example inventory

* Fix tests

* Fix more tests

* Fix tests

* Use dataclasses for build_service_agents.py

* Remove unneeded field() from build_service_agents

* Re-enable CMEK depends_on in project outputs

* Update tools/requirements.txt

* Enable storage in GCS example projects

* Fix tests

* Add CMEK Service Agents dependencies for services

* Fix typos and data platform cmek

* More typos
2024-07-23 22:05:38 +02:00
Ludovico Magnocavallo
41e583ffc9 add network tags outputs and examples to project module (#2350) 2024-06-09 07:52:15 +02:00
Ludovico Magnocavallo
309792c559 Refactor vpc-sc support in project module, add support for dry run (#2229) 2024-04-22 09:28:01 +02:00
Wiktor Niesiobędzki
bca5901691 Fix project outputs inventory 2024-04-11 11:51:19 +02:00
Wiktor Niesiobędzki
a236222a93 Add project quotas factory 2024-04-11 11:51:19 +02:00
Wiktor Niesiobędzki
9a95ac10ed Once again fix e2e tests 2024-02-23 19:21:39 +01:00
Wiktor Niesiobędzki
8fd8ee0541 Fix too long project names on e2e tests 2024-02-23 11:41:58 +01:00
Julio Castillo
5197d5ca8d Allow projects as destinations for log sinks (#2102)
* Add project log sink destination to project module

* Add project log sink destination to folder module

* Add project log sink destination to organization module

* Fix typos

* Add project log sink destination to billing-account module

* Make filter field optional

* Update READMEs

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2024-02-21 08:41:13 +01:00
Ludovico Magnocavallo
bf93b6fb4e fix typo in logging sinks interface (#2015) 2024-01-28 10:27:28 +01:00
Wiktor Niesiobędzki
0d486fb34e E2E tests fixes 2023-12-19 11:01:03 +01:00
simonebruzzechesse
c50b732c79 Allow granting network user role on host project from project module and factory (#1930)
* Update shared vpc config for project factory and project module for more granular Shared VPC configuration

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-12-15 14:39:21 +01:00
Ludovico Magnocavallo
f548b65b1c Add support for subnet-level service network user grants to project module, improve docs (#1907)
* improve project factory example

* light refactor of project modules shared vpc internals and docs

* add support for subnet-level grants on host project
2023-12-07 09:07:48 +00:00
Wiktor Niesiobędzki
4668b90e8a Fix inventory for project 2023-12-03 10:37:23 +00:00
Wiktor Niesiobędzki
fe485414e6 Add end-to-end tests for project module (#1823)
* Add end-to-end tests for project module
* Add inventory to data tests
* Add files to end-to-end test cases
* Review fixes - use named groups

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-11-03 18:04:19 +01:00