Commit Graph

334 Commits

Author SHA1 Message Date
Ludovico Magnocavallo
ad5de9b7ea Refactor FAST project factory and supporting documentation (#2505)
* untested

* teams pattern

* rework doc

* README

* boierplate

* tflint

* Fix tflint for project factory

* Correct path to pf

* resman changes

* fix factory variable default

* fix links

* project factory module substitutions

* tflint

* stage test

* tfdoc

* rename schema, address review comments

* README typos and wording

* tfdoc

* review comments

* remove test from yaml

* revert output workflow changes

* fix sa reference errors

* tfdoc

* pf tag roles

* schema validation

* pf tag roles

* avoid null values in pf context

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-08-20 16:45:42 +00:00
Julio Castillo
912cbb8281 Rename 1-vpc-sc stage to 1-vpcsc (#2471)
* Rename 1-vpc-sc stage to 1-vpcsc

* Fix tests
2024-08-06 11:21:55 +00:00
Julio Castillo
89333a5d43 Make policyReader binding additive in bootstrap (#2470) 2024-08-06 09:35:37 +00:00
Ludovico Magnocavallo
345716e576 VPC-SC as separate FAST stage 1 (#2460)
* initial commit

* README

* boilerplate

* tflint

* tfdoc

* fix security stage tests

* vpc-sc stage tests

* tflint

* fix resman stage test inventories

* security README

* stage-level README

* Update README.md

* flexible perimeter variable

* remove diagram

* change default to dry run

* default to dry run
2024-08-02 18:04:36 +02:00
Luca Prete
80f9ce6307 [FAST] Add basic NGFW enterprise stage (#2410) 2024-08-01 09:41:31 +00:00
Simone Ruffilli
27bb48df77 NCC in 2-net-a-simple (#2397)
* NCC in 2-net-a-simple
2024-07-25 18:03:09 +02:00
Julio Castillo
c0bf32e797 Refactor service agent management (#2423)
* Service agents script

* Service agents update

* WIP

* Update script and terraform

* Fix tests

* Fix linter

* Update docs

* Bring back pf example inventory

* Fix tests

* Fix more tests

* Fix tests

* Use dataclasses for build_service_agents.py

* Remove unneeded field() from build_service_agents

* Re-enable CMEK depends_on in project outputs

* Update tools/requirements.txt

* Enable storage in GCS example projects

* Fix tests

* Add CMEK Service Agents dependencies for services

* Fix typos and data platform cmek

* More typos
2024-07-23 22:05:38 +02:00
Ludovico Magnocavallo
5319184e71 FAST ng: stage 0 environments and VPC-SC IaC resources (#2440)
* FAST ng: stage 0 environments and VPC-SC IaC resources

* test inventories
2024-07-23 11:52:39 +02:00
Simone Ruffilli
3151b02eda FAST: IAM cleanups to reflect PF changes (#2430)
* FAST: IAM cleanups to reflect PF changes
2024-07-18 14:59:28 +02:00
Ludovico Magnocavallo
e3809e6735 Add main project factory service account (#2353)
* add main project factory service account

* add main project factory service account
2024-06-10 12:23:30 +02:00
Ludovico Magnocavallo
b13b6032d3 Remove support for source repositories from FAST CI/CD (#2352)
* stage 0

* stage 1

* stage 1 mt

* remove unused locals from resman

* remove unused locals from resman

* tfdoc
2024-06-10 09:02:55 +00:00
Julio Castillo
ef7083799c Update PGA domains (#2330)
* Update PGA domains

* Fix tests
2024-05-31 10:53:50 +00:00
Simone Ruffilli
4901b4aee8 FAST: Enable networkconnectivity when using NCC-RA in 2-b (#2329) 2024-05-31 08:22:24 +00:00
Luca Prete
eb5754e475 [FAST] Rename stage 2-networking-d-separate-envs to 2-netwroking-c-separate-envs (#2328)
Co-authored-by: Luca Prete <lucaprete@google.com>
2024-05-31 09:09:31 +03:00
Simone Ruffilli
532f1ecfc4 Merge FAST C and E network stages into a new B stage. (#2309)
Merge FAST C and E network stages into a new B stage.
2024-05-28 17:27:28 +02:00
Ludovico Magnocavallo
980011806c fix permadiff in cloud nat module (#2301) 2024-05-23 08:38:03 +02:00
Ludovico Magnocavallo
be9214f99a add support for tenant factory CI/CD (#2297) 2024-05-21 10:39:47 +02:00
simonebruzzechesse
79af34b69e Add wif permissions to bootstrap tf SA (#2290)
* add wif permissions to bootstrap tf SA
2024-05-20 18:15:23 +02:00
Simone Ruffilli
21f3b733ab FAST: Cleanup/harmonization of Simple and NVA net stages (#2287)
Cleanup/harmonization of Simple and NVA net stages
2024-05-16 16:49:15 +03:00
Simone Ruffilli
887c7e7926 Unify VPN and Peering FAST stages (#2284)
* Unify VPN and Peering FAST stages
2024-05-16 12:18:32 +03:00
Ludovico Magnocavallo
7a5dd4e6db FAST: add top-level folders and restructure teams/tenants in resman (#2254)
* remove teams and tenants from resman

* move fast features to stage 1, fix test inventories

* folders

* fix factory, add top level folder resources to outputs

* tfdoc

* stage 0 log sink defs

* tfdoc

* enable toc in resman readme

* simple tenants

* fast compatibility automation and logging

* testing fast-compatible tenants

* testing fast-compatible tenants

* tfdoc

* remove mt stages

* remove tests, fix links

* disable tflint

* fast tests

* make organization conditional in resman

* check names tool

* export real prefix to tfvars, prevent destroy errors

* prefix validation

* fix billing account export format

* tfdoc

* root node folder

* resman changes

* tenant resman roles

* first apply of tenant resman

* tenant log sinks in stage 1

* fix test vars

* tfdoc

* tenant vpc-sc access policy

* fix tests expected values

* tenant CI/CD

* identity providers

* wif

* tfdoc

* add comments to identity locals

* full-feature tenant resman apply

* tenant billing IAM

* stage test

* fix CI/CD comments

* tenant net stage verified

* tenant sec stage verified

* fix test

* README work

* tfdoc

* README

* README rewording

* README rewording

* tfdoc

* FAST excalidraw

* review comments

* diagram review changes

* add iam log sink for tenants

* remove redundant try from security stage

* Implement tflint-fast in Python driven by tftest.yaml files

* tflint

* test ci changes

* revert linting changes

* disable tflint for fast

* Create junit-style report for FAST tflint

* Remove junit-reporter

* YAPF tflint-fast.py

* Output tflint FAST to job summary

* Step summary

* Disable step_summary as output is not useful

* ignore tflint warning

* re-enable tflint on FAST

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-05-15 09:17:13 +00:00
Simone Ruffilli
9a26fe8635 Add support for reserved_internal_range in net-vpc (#2275)
Adds support for reserved_internal_range to net-vpc
2024-05-14 22:19:45 +03:00
Wiktor Niesiobędzki
af253c9702 Fix 0-bootstrap iam_by_principals not taking into account all principals (#2267)
* Fix 0-bootstrap iam_by_principals not taking into account all principals
* Add test-case for iam_by_principals for 0-bootstrap stage

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2024-05-12 21:02:04 +02:00
Julio Castillo
d838c4ac47 Make Simple NVA route IAP traffic through NIC 0 (#2262) 2024-05-09 18:29:25 +02:00
Julio Castillo
c58850c096 Add Hybrid NAT support (#2261)
* Updates to support hybid NAT

* Fix readme

* Fix variable order
2024-05-09 13:24:41 +00:00
Ludovico Magnocavallo
c9503d5ac5 Remove data source from folder module (#2260)
* remove data source from folder module

* fix fast tfdoc

* fix locals type error

* fix folder test

* fix fast test
2024-05-09 13:09:54 +00:00
Julio Castillo
94c32c1d71 Misc FAST fixes (#2253)
* Misc FAST fixes

* Fix readme

* Fix FAST nva bgp tests
2024-05-02 06:56:26 +00:00
Julio Castillo
99129d54a3 Update FAST logging (#2235)
* Update FAST logging

* Fix readme

* Fix tests
2024-04-25 08:31:51 +02:00
Wiktor Niesiobędzki
024d3255e6 Generalization of tflint call for FAST stages (#2225)
* Generalization of tflint call for FAST

* Fix tfvars path

* Fix tfvars path - depending where the file is

* Fix regex

* Reeanble linting

* Align test directory to stage name

* Align all fast stages to use tftest
2024-04-18 21:04:24 +02:00
Julio Castillo
3af7e257d2 Add tflint to pipelines (#2220)
* Fix terraform_deprecated_index

https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_index.md

* Fix terraform_deprecated_interpolation

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_interpolation.md

* Fix more indexing

* Remove unused variable

* Enable TFLint for modules

* Add tflint config file

* Fix chdir

* Lint modules

* TFLint fixes

* TFLint

* Fixes binauthz README

* Fixes DNS response policy tests. Restores MIG outputs.

* Fixes other DNS response policy tests.

* Update tests for fast 2-e

* Moar fixed tests

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
2024-04-17 10:23:48 +02:00
Ludovico Magnocavallo
198d90c6fc Remove data source from net-vpc module (#2216)
* remove data source from net-vpc module

* fix test inventories

* remove data source, fix fast inventories
2024-04-16 14:11:12 +03:00
Julio Castillo
a74a106f8b Add new org policies to FAST (#2215)
* Add new org policies to FAST

* Fix tests
2024-04-15 15:29:24 +02:00
Ludovico Magnocavallo
9cace5272f Add support for quotas to project module (#2210)
* add support for quotas to project module

* tfdoc

* better outputs

* Ensure keys in `quota_configs` are unique

* update fast tests

* Make quota E2E testable

* Remove quota project caveat

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-04-10 17:03:04 +00:00
Ludovico Magnocavallo
8511170412 FAST security stage refactor (#2203)
* first working change, missing docs and tests

* fixes

* tests

* boilerplate

* tfdoc
2024-04-07 20:14:39 -07:00
Julio Castillo
a9ac0f40cd Add variable to resman to control top-level folder IAM (#2196) 2024-04-04 10:26:35 +02:00
Ludovico Magnocavallo
33ffe2daa6 Allow r/o project factory SAs access to folder-level IAM (#2195)
* allow project factory SAs read access to folder-level IAM

* tfdoc
2024-04-03 21:51:46 +02:00
Elia
b80132a618 FAST GCVE stage (#2191)
* GCVE stage prerequisites

* fix gcve prereq

* gcve stage first deploy

* Updated readme filex

* docs updated

* some fixes after testing

* updated readme files

* elia fix

* gcve net admin custom role

* gcve net admin custom role

* elia fix

* ven peering deploy

* elia fix

* added blueprint and stage tests

* Edits to Readme files

* typo in outputs

* clean-up

* gcve stage tests fix

* readme fix

* fix sorting

* fix copyrights and readme file

* fix test

* fix copyright

* fixed gcve feature flag cond.

* removed validation

* fixed typo

* fixed typo

* fixed gcve tests

* fixed typo

* fixed typo

* fixed sorting

* fixed sorting

---------

Co-authored-by: Konrad Schieban <kschieban@google.com>
2024-04-03 17:25:12 +02:00
Julio Castillo
759e85d6af Logging updates (#2139)
* Add support for logging settings to the organization module

* Enable log analytics on FAST-managed logging buckets

* Fix README

* Fix tests
2024-03-08 10:07:12 +01:00
Ludovico Magnocavallo
3972eb6df4 Align resource names in FAST networking stages (#2115)
* stage c nva

* fix tests

* remove moved blocks from net c stage

* simplify subnet naming in stage 2 net e

* address most renames in stage 2 e

* address most renames in stage 2 e

* address most renames in stage 2 e

* complete renames in stage 2 e

* use non-regional names in subnets

* use non-regional names in subnets

* use non-regional names in subnets
2024-02-29 07:45:19 +01:00
Ludovico Magnocavallo
dbabfb9ae0 Add support for billing budgets to project factory (#2112)
* align factory variable name in project factory module

* tested

* align fast stage
2024-02-27 18:13:49 +00:00
Ludovico Magnocavallo
eb23bb62d2 Support domainless orgs in FAST (#2086)
* bootstrap

* align org policies to domainless enforced ones

* fix #2073

* fix tests

* fix team admin attribute in resman stage
2024-02-19 08:29:37 +00:00
simonebruzzechesse
946ae148f7 Add workforce_identity_federation in 0-bootstrap (#2077)
* add workforce_identity_federation in 0-bootstrap
* update tests
2024-02-15 00:10:24 +01:00
Ludovico Magnocavallo
71a64487d5 Extend FAST to support different principal types (#2064)
* add doc draft

* typos

* typo

* typo

* typos

* rewording

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* move iam variables to a separate file

* move billing-account module to iam_principals

* move data-catalog-policy-tag module to iam_principals

* move dataplex-datascan module to iam_principals

* move dataproc module to iam_principals

* move folder module to iam_principals

* copyright

* move organization module to iam_principals

* move project module to iam_principals

* move source-repository module to iam_principals

* update blueprints for iam_principals interface

* FAST bootstrap

* module READMEs fixes

* FAST bootstrap

* FAST networking stages

* FAST security stage

* FAST gke stage

* FAST multitenant bootstrap stage

* FAST multitenant resman stage

* tfdoc

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* fix module test

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Rename iam_principals to iam_by_principals

* Update IAM template to include iam_by_principals

* Update Resman README

* Fix ADR link format

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2024-02-12 14:35:30 +01:00
Wiktor Niesiobędzki
c58e61e98e Introduce variable to disable imports, fix tests 2024-02-07 17:25:11 +01:00
Julio Castillo
e219d92217 Enable additional recommended org policies (#2050)
* Enable additional recommended org policies

Fixes #2047
Fixes #2048
Fixes #2049

* Fix tests
2024-02-05 10:46:37 +01:00
Ludovico Magnocavallo
5448ab64c4 Leverage net-vpc module for DNS logging in FAST (#2041)
* revert #2023

* leverage net vpc module for dns logging in fast
2024-02-03 08:16:00 +01:00
Julio Castillo
13636ba07b Make Cloud NAT creation optional in FAST net stages. (#2038)
* Make Cloud NAT creation optional in FAST net stages.

Fixes #2021

* Update READMEs
2024-02-02 10:58:16 +01:00
Julio Castillo
4c68c016a9 Add DNS query logging to FAST net stages (#2033)
* Add DNS query logging to FAST net stages

Fixes #2020

* Update readmes

* Add variable to toggle DNS logging

* Extend DNS logging toggle to other net stages
2024-01-31 13:44:51 +01:00
Julio Castillo
da95434308 logging for default ingress rules in FAST (#2030)
* Add default ingress deny rule with logging to FAST net stages.

Fixes #2024

* Allow firewall factory to omit rules key

* Fix tests

* Fix fast tests

* fix fast tests
2024-01-30 16:53:01 +00:00
Ludovico Magnocavallo
99228363b2 enforce trusted image projects constraint in stage 0 (#2014) 2024-01-26 10:14:44 +00:00