Commit Graph

32 Commits

Author SHA1 Message Date
Ludovico Magnocavallo
981e4581ee Add project-factory based data platform dataset to FAST project factory stage (#3957)
* dp rewrite stage 0, projects

* remove plan files

* generalize handling of basepath for projects in project-factory module

* central-0 ---> core-0

* add schemas, validate YAMLs, tags

* aspect types

* data catalog policy tag factory

* add support for data catalog taxonomy to project factory

* complete retrofit of old stage configuration, except networking

* shared vpc networking

* networking

* data platform as pf dataset

* docs

* test

* remove legacy dp stage, fix tests and links

* boilerplate

* tfdoc

* fix unrelated tfdoc

* schemas

* fix errors

* schema

* duplicate schemas

* yamllint

* Fix module naming convention for aspect-types

* Fix factories_config in vpcs.tf for net-vpc-factory compatibility

* Update schema documentation based on schema changes

* Fix false rename conflict in .config.yaml files

* Sync schemas and update documentation

* Fix path expansion for aspect-types and revert projects_input to master

* Restore path expansion for org_policies in projects-iam call

* Fix trailing newlines in schema duplicates to satisfy duplicate-diff

* Fix path expansion for data_catalog_taxonomy in taxonomies.tf

* Update inventory for data-platform test and clean up debug prints

* Add full values to data-platform inventory

* Align Stage 2 VPC Factory integration with Stage 0 and fix tests

TAG=agy

* Fix project factory context resolution and data platform datasets

- Update tag context keys in project factory to use file key without 'projects/' prefix.
- Fix tag reference in product-0.yaml.
- Fix shared_vpc_service_config in shared-0.yaml by moving service account to network_users.
- Set parent for domain-0 folder to data-platform.
- Mock net-dev-0 project ID in tests.
- Update inventories.

TAG=agy
CONV=4b37fa5b-bf59-4604-9e8f-b55353d967a0

* Fix project-level tag keys context resolution in project factory

* Fix commented out tag reference in domain-0 .config.yaml

* Fix merge() calls with empty arguments in project-factory and data-catalog-policy-tag

* Update Data Platform dataset README with prerequisites and customization guide

* Add Table of Contents to Data Platform dataset README

* docs: update Data Platform README with project templates tip

* Document data platform output files and linking sequence in README

* Update data platform README with VPC-SC and delegated IAM details

* Refactor data platform dataset and align stage defaults

* Update test inventory and variables for data platform with new prefix
2026-05-12 16:44:32 +02:00
Luca Prete
ba56d9afbc Add Agent Engine identity type (#3875) 2026-05-05 10:22:21 +02:00
Ludovico Magnocavallo
85b0871085 Add net-vpc-factory and starter-gcd dataset to 0-org-setup, fix context in net-vpc-factory (#3860)
* feat(fast/0-org-setup): integrate net-vpc-factory into stage 0

* initial starter dataset

* starter pack

* fix(fast/0-org-setup): correct factory vpcs paths definition and update test inventory

* feat(fast/0-org-setup): add VPC and subnet outputs and update stage tfvars

* test(fast/0-org-setup): update gcd starter defaults and regenerate inventory

* fix(modules/net-vpc-factory): remove internal defaults logic and pass context correctly

* fix typo in dataset fw rule, add missing boilerplate, set sane defaults for net vpc factory

* terraform fmt

* tfdoc, copyright year

* schema docs

* yamllint
2026-04-13 09:42:03 +02:00
Ludovico Magnocavallo
118b70d456 Remove the FAST data platform stage (#3851)
* chore: remove data platform stage and references

* fix checks
2026-04-10 10:19:20 +02:00
Julio Castillo
15c7951f97 Remove hcl2 python dependency (#3836)
* Migrate organization policy tests to standard tftest.yaml.

Remove python-hcl2 dependency and the custom python test file.
Consolidate the boolean, list, and custom constraint tests into a single `org_policies` test with a factory equivalent.
Restructure factory files into a unified `factory/` directory.

* Migrate project and folder org policy tests to standard tftest.yaml.

Replicate the organization module changes for project and folder modules:
- Remove python-hcl2 dependency usages and conftest.py.
- Remove custom python test files for org policies.
- Consolidate org policy tests into a single `org_policies` test with a factory equivalent.
- Unify factory files into a `factory/` directory.
- Remove redundant common.tfvars in folder module.

* Add factory policies directory to duplicate-diff checks.

Ensure the YAML factory files for org policies remain perfectly identical across the organization, folder, and project modules.

* Remove unused deepdiff dependency from requirements and pre-commit config.

* Add boilerplate

* fix broken link
2026-04-08 08:14:16 +02:00
Ludovico Magnocavallo
7a5664f475 Additional changes to folder module and project factory (#3782)
* allow null name, use basepath for factories in pf folders

* fix id in folder schema
2026-03-05 09:32:35 +00:00
Julio Castillo
ddab73d03f Add basic observability example to classic dataset (#3697)
* Add basic observability example to classic dataset

* Add boilerplate and observability schema

* Lint fix

* Add observability schema to project-factory module

* Extend duplicate-diff

* Fix tests

* Remove unused schemas
2026-02-10 14:35:13 +00:00
Ludovico Magnocavallo
30810146cf Merge remote-tracking branch 'origin/master' into fast-dev 2026-01-30 15:54:45 +00:00
Sven
e933340e02 feat: create new dataset classic-gcd (based on dataset classic) to enable deployment on GCD (#3679)
* feat: create classic dataset for GCD deployments

* Revert billing config to classic

* Revert GCD README

* Restore billing project in GCD

* Add basic alignment-test with classic dataset

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2026-01-27 16:59:18 +00:00
Vannick Trinquier
2ea4c27fe8 Add context support for constraints and additional controls for hardened datasets (IAM, GKE and others) (#3661) 2026-01-23 08:28:02 +07:00
Vannick Trinquier
8342558732 Implement various compliance configuration and principle of least privilege for hardened dataset (#3635) 2026-01-19 15:46:15 +07:00
Vannick Trinquier
cc24046be8 Add CMEK support to FAST and controls for CMEK encryption (#3556) 2025-12-14 12:14:08 +07:00
Ludovico Magnocavallo
897c6ef8c3 Add support for Workload Identity to project module and project factory (#3531)
* module-level support

* fast stage 0

* fix inventory, add outputs/tfvars

* wip

* project factory

* pf outputs

* iam templates will be added where ci/cd configs are managed

* fix merge conflicts
2025-11-17 07:31:21 +00:00
Ludovico Magnocavallo
0a2cc758ac Essential contacts in schemas, and email context substitutions (#3495)
* modules

* fast

* duplicate diff

* fix contacts in FAST stage 0 datasets, update contacts in YAML schemas
2025-11-03 08:53:29 +01:00
Ludovico Magnocavallo
717803e977 Merge remote-tracking branch 'origin/master' into fast-dev 2025-10-27 18:21:35 +00:00
Wiktor Niesiobędzki
4f2fbcab94 Align agent-engine with interface for Cloud Functions and Cloud Run 2025-10-27 16:14:37 +01:00
Simone Ruffilli
4a41a4237a Removes legacy FAST networking stages (#3479)
Removes the legacy networking stages, superseded by 2-networking.

**Breaking Changes**

```upgrade-note
`fast/stages/2-networking-legacy-a-simple`: The stage is being removed, and superseded by the `2-networking` FAST stage, introduced in #3435 
`fast/stages/2-networking-legacy-b-nva`: The stage is being removed, and superseded by the `2-networking` FAST stage, introduced in #3435 
`fast/stages/2-networking-legacy-c-separate-envs`: The stage is being removed, and superseded by the `2-networking` FAST stage, introduced in #3435 
```
2025-10-27 14:38:28 +00:00
Wiktor Niesiobędzki
2e42c1b548 service_account_config for Cloud Run v2
Additional changes:
* align vpc-connector interface to Cloud Functions
* split managed and unmanaged resources into separate files, this makes
  easier to introduce further changes
* add support for contexts
* move `vpc_connector` variable to variables.tf for Cloud Functions
* remove `create` from `vpc_connector` in Cloud Functions as it was
  sharing the meaning with `vpc_connector_create`
2025-10-27 08:03:53 +01:00
Ludovico Magnocavallo
4bf4047017 Merge remote-tracking branch 'origin/master' into fast-dev 2025-10-23 14:51:14 +00:00
Wiktor Niesiobędzki
d9029e47a0 VPC Connector alignment to Cloud Run v2 + contexts 2025-10-23 15:50:17 +02:00
Wiktor Niesiobędzki
36f2e65465 Cloud Function v2 - contexts and interface refactor 2025-10-23 15:50:17 +02:00
Simone Ruffilli
23f8326665 Factory based FAST Networking stage (#3435)
New factory based networking stage, shipping with a single dataset (peering) to keep the PR size somewhat manageable.
2025-10-23 14:17:44 +02:00
Simone Ruffilli
de8ebefe17 Start the deprecation process of the old networking stages (#3451)
This PR renames `2-networking-foo` stages to `2-networking-legacy-foo` - including references of stages in docs and tests.

**Breaking Changes**

```upgrade-note
`fast/stages/2-networking-a-simple`: The stage is being deprecated, and superseded the `2-networking` FAST stage, introduced in #3435 
`fast/stages/2-networking-b-nva`: The stage is being deprecated, and superseded the `2-networking` FAST stage, introduced in #3435 
`fast/stages/2-networking-c-separate-envs`: The stage is being deprecated, and superseded the `2-networking` FAST stage, introduced in #3435 
`modules/net-vpc-factory`:  The module has been deprecated, and superseded the `2-networking` FAST stage, introduced in #3435 
```
2025-10-22 16:31:14 +02:00
Ludovico Magnocavallo
a4cc7cd979 Align schemas (#3447)
* enforce schema uniqueness

* schemas

* fix tests
2025-10-21 14:03:40 +02:00
Vannick Trinquier
c60ae3652a Adding hardened datasets for preventive and detective Compliance Controls (#3410)
* Adding hardened datasets for preventive and detective Compliance Controls in stage 0 and stage 1 VPC-SC

* Move observability to factory file

* Update documentation

* Update local variable for use

* Update observability factory to use other module

* Add raw diagram file for hardened datasets

* Retrofit change

* Rename log_buckets context variable to be consistent across modules

* Update stage 0 documentation to mention hardened dataset

* Update customer ids list

* Update documentation, path to schema add ID to access level

* Comment organization policy gcp.resourceLocation by default

* Prevent duplicate key error by merging principal roles

* Adding ngfw roles files in hardened datasets

* Update script to validate files differences to support folder and datasets

* Format duplicate-diff python script

* Remove .config.yaml from duplicates

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2025-10-21 10:34:25 +00:00
Ludovico Magnocavallo
0103c64457 fix duplicate check 2025-09-20 12:12:01 +00:00
Ludovico Magnocavallo
bc6950e205 Rename FAST stages preparing for eventual deprecation (#3298)
* renames

* links

* readme

* docs

* update pf modules tests for renames

* condition_vars context in modules

* data platform dataset

* fix links in stage 3 docs

* schema changes

* schema docs

* tfdoc

* update duplicates check

* fast legacy tests

* legacy schema

* fix tests
2025-09-04 08:24:11 +02:00
Wiktor Niesiobędzki
a5af0703f6 Add notebooks, appengine and appspot to dns policy routing 2025-06-13 16:46:44 +02:00
simonebruzzechesse
1e2eea1296 2-secops stage (#3038)
* new 2-secops stage
* new 3-secops-dev stage

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2025-04-18 15:57:29 +02:00
Ludovico Magnocavallo
177a9ed8bf fix names in project factory automation resources (#3046) 2025-04-18 15:42:45 +02:00
Wiktor Niesiobędzki
f1a3cac8ca Use factory-projects-object to normalize inputs for project module 2025-04-11 10:53:08 +02:00
Wiktor Niesiobędzki
3e84236345 Project object c14n in separte file
Create separate file for canonicalization of project factory objects and
introduce duplicate-diff lint checker.
2025-03-30 10:39:08 +02:00