Wiktor Niesiobędzki
303186e6d3
fix permadiff after apply
2025-05-04 16:46:39 +02:00
Simone Ruffilli
3c91acd181
Replace all instances of stackdriver.googleapis.com with log+mon ( #3022 )
...
Replace all instances of stackdriver.googleapis.com with logging.googleapis.com and monitoring.googleapis.com.
Fixes #2932
2025-04-11 12:04:50 +00:00
Ludovico Magnocavallo
c8e4179f2b
Add custom routes for directpath to net-vpc module ( #2966 )
...
* add custom routes for directpath to net-vpc module
* blueprint tests
* blueprint tests
* blueprint tests
* fast tests
* tfdoc
* module examples
2025-03-19 10:22:47 +00:00
javiergp
c832b61240
Update README.md
...
Fix image link
2025-03-03 10:01:41 +01:00
Ludovico Magnocavallo
1a4b298cc9
Project factory additions, project module reuse implementation ( #2899 )
...
* add support for buckets
* add project-level interpolation for own SAs
* docs
* project reuse changes
* fix example
* tfdoc
* update check documentation tool
* fast tests
* blueprints
* typo
2025-02-15 20:37:45 +01:00
Julio Castillo
d0c3f04c10
Make service agents work in different universes ( #2894 )
...
* Make service agents work in different universes
* Use templatestring and two passes for service agent emails
* Fix tests
2025-02-14 12:16:07 +00:00
Wiktor Niesiobędzki
5593192466
Deprecate composer-2 blueprint
2025-02-03 11:27:14 +01:00
Wiktor Niesiobędzki
cbaf6928d6
fix non-empty plan after apply for vertex mlops
2024-12-31 17:27:47 +01:00
Julio Castillo
c73f912d43
Fabric e2e fixes ( #2791 )
...
* Try to fix fabric e2e tests. Update service agents
* Fix typo
* Fix syntax
* Delay PSA creation until service agents have roles assigned
* Fix tests
2024-12-31 15:25:36 +01:00
Ludo
0d3cbfd912
Merge remote-tracking branch 'origin/master' into fast-dev
2024-11-05 12:37:34 +01:00
Julio Castillo
b7a4717fc3
Fix tests
2024-11-05 12:10:23 +01:00
Wiktor Niesiobędzki
d8a6f3a895
Fix tests
2024-11-05 11:22:43 +01:00
Wiktor Niesiobędzki
232a7cd853
Fix IAM drift in vertex-mlops
2024-11-05 11:22:43 +01:00
Ludo
7688072b0c
Merge remote-tracking branch 'origin/master' into fast-dev
2024-11-04 10:45:44 +01:00
Wiktor Niesiobędzki
4ab9c74e9f
Review fixes
2024-11-04 10:34:54 +01:00
Wiktor Niesiobędzki
81cb269e25
fix tests
2024-11-04 10:34:54 +01:00
Wiktor Niesiobędzki
59eade7c48
Workaround overlapping networks
2024-11-04 10:34:54 +01:00
Wiktor Niesiobędzki
28160055af
replace deprecated google_notebooks_instance with google_workbench_instance
2024-11-04 10:34:54 +01:00
Ludo
035fcd6cc5
Merge remote-tracking branch 'origin/master' into fast-dev
2024-11-04 10:18:46 +01:00
Wiktor Niesiobędzki
d6f0ee9447
fix Vertex-ML-Ops e2e tests
2024-11-04 10:13:32 +01:00
Ludovico Magnocavallo
50ac3a5013
Refactor of FAST resource management and subsequent stages ( #2648 )
...
* untested
* pllan testing
* fix stage 2s
* move providers to their own file
* single-environment stage 3
* fixes and moved blocks
* stage3 factory
* doc
* review comments
* review comments
* tfdoc
* fasts tage 1 tests
* netsec as stage 2
* fix backported roles
* fix backported roles
* tfdoc
* fixes
* fix tag value roles in stage 1
* remove checklist, fix stage 1 tests
* inventory
* Small bugfix
* refactor context tag values
* fix previous merge
* fix previous merge
* fix previous merge
* support short names for top level automation resources, change top level context variable
* fix new top level context
* roll back merge changes to stage 0 outputs
* roll back more merge changes
* linting errors
* tfdoc
* fix tests, roll back merge in tenants stage
* tfdoc
* fix inventory
* optional stage 2 env folders and tag bindings
* tflint
* damn tflint
* damn tflint
* tfdoc
* fix networking tests
* tflint
* fix test inventories
* tfdoc
* use coalesce for project parents
* fix billing role conditions
* fix billing role conditions
* security stage tested (ngw resources need fixing/porting)
* boilerplate
* fix inventory
* stage envs and stage linking script
* initial work on resman docs, update diagram, improve teams folder
* resman README
* fix stage 2 IAM delegation
* remove checklist from bootstrap
* stage 1 tests
* stage 0 1 and 2 tests
* tflint
* tflint
* tfdoc
* GCVE stage refactor (untested)
* GCVE stage refactor (untested)
* GCVE stage 3
* gcve tests
* tflint
* tfdoc
* fix links
* module tests
* stages README
* move network security to stage 2
* network security tests
* replace stage links in README files
* minimal netsec stage refactor
* use factory for iac org policies, add configurable drs org policy for iac
* test mt stage
* tfdoc
* fix cicd workflows
* fix cicd workflows
* gke-dev stage
* tflint
* remove data platform stage
* exclude provider files via tfdoc opts
* remove data platform tests and links
* fix merge
* fix resman inventory
* boilerplate
* inventory
---------
Co-authored-by: Simone Ruffilli <sruffilli@google.com >
2024-10-31 16:55:54 +01:00
Wiktor Niesiobędzki
8b7607af63
Codespell / whitespace fixes
2024-10-30 10:30:37 +01:00
Julio Castillo
f57635d044
Add managed folders suports to gcs module ( #2530 )
...
* Add RPO, make versioning dynamic
* Add manaed folders
* Change autoclass and cors defaults to null
* Update README
* Add iam_by_principals
* Add managed folders var description
* Remove need for managed folders to end in /
* Add inventory to example
* Update readme
* Fix FAST tests
2024-08-28 07:30:52 +00:00
Julio Castillo
2854ae6bd8
Remove "constraints/" from org policy names ( #2450 )
2024-07-29 15:15:04 +02:00
Julio Castillo
c0bf32e797
Refactor service agent management ( #2423 )
...
* Service agents script
* Service agents update
* WIP
* Update script and terraform
* Fix tests
* Fix linter
* Update docs
* Bring back pf example inventory
* Fix tests
* Fix more tests
* Fix tests
* Use dataclasses for build_service_agents.py
* Remove unneeded field() from build_service_agents
* Re-enable CMEK depends_on in project outputs
* Update tools/requirements.txt
* Enable storage in GCS example projects
* Fix tests
* Add CMEK Service Agents dependencies for services
* Fix typos and data platform cmek
* More typos
2024-07-23 22:05:38 +02:00
Julio Castillo
4e8adc9c43
Update modules/artifact-registry with newly-released features. ( #2396 )
...
Fixes #2377
2024-06-28 19:52:25 +02:00
Vince Gonzalez
2446b4dd7c
Update README.md ( #2239 )
2024-04-26 01:14:32 +02:00
Julio Castillo
3af7e257d2
Add tflint to pipelines ( #2220 )
...
* Fix terraform_deprecated_index
https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_index.md
* Fix terraform_deprecated_interpolation
Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_interpolation.md
* Fix more indexing
* Remove unused variable
* Enable TFLint for modules
* Add tflint config file
* Fix chdir
* Lint modules
* TFLint fixes
* TFLint
* Fixes binauthz README
* Fixes DNS response policy tests. Restores MIG outputs.
* Fixes other DNS response policy tests.
* Update tests for fast 2-e
* Moar fixed tests
---------
Co-authored-by: Simone Ruffilli <sruffilli@google.com >
2024-04-17 10:23:48 +02:00
Ludovico Magnocavallo
9414779cc2
Allow multiple PSA service providers in net-vpc module ( #2218 )
...
* allowing multiple PSA service providers in net-vpc module
* tfdoc
* tfdoc
* Add tfvars/yaml tests
* fix module and tests
* re-enable inventory
* merge fix
* Add multiple PSA test case
* fix cloudsql example
---------
Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com >
2024-04-16 15:02:36 +00:00
Wiktor Niesiobędzki
67382f53b7
Fix data-platform-minimal blueprint
2024-03-11 11:05:33 +01:00
Julio Castillo
993bef71aa
Introduce mandatory OWNERS file for blueprint maintainership ( #2131 )
...
* Delete deprecated/broken blueprints
* Adding OWNERS to all blueprints
* Fix links
* Update OWNERS
---------
Co-authored-by: javiergp <javiergp@users.noreply.github.com >
2024-03-08 09:40:46 +01:00
Ludovico Magnocavallo
6941313c7d
Factories refactor ( #1843 )
...
* factories refactor doc
* Adds file schema and filesystem organization
* Update 20231106-factories.md
* move factories out of blueprints and create new factories README
* align factory in billing-account module
* align factory in dataplex-datascan module
* align factory in billing-account module
* align factory in net-firewall-policy module
* align factory in dns-response-policy module
* align factory in net-vpc-firewall module
* align factory in net-vpc module
* align factory variable names in FAST
* remove decentralized firewall blueprint
* bump terraform version
* bump module versions
* update top-level READMEs
* move project factory to modules
* fix variable names and tests
* tfdoc
* remove changelog link
* add project factory to top-level README
* fix cludrun eventarc diff
* fix README
* fix cludrun eventarc diff
---------
Co-authored-by: Simone Ruffilli <sruffilli@google.com >
2024-02-26 10:16:52 +00:00
simonebruzzechesse
d11c380aec
Format python files in blueprints ( #2079 )
...
* format python files in blueprints
* update check on blueprints python code
* update python linter in CI workflow
2024-02-15 09:37:49 +01:00
Wiktor Niesiobędzki
b902b1dab9
Fix non-empty plan after apply
2024-02-13 07:40:31 +01:00
Ludovico Magnocavallo
71a64487d5
Extend FAST to support different principal types ( #2064 )
...
* add doc draft
* typos
* typo
* typo
* typos
* rewording
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* move iam variables to a separate file
* move billing-account module to iam_principals
* move data-catalog-policy-tag module to iam_principals
* move dataplex-datascan module to iam_principals
* move dataproc module to iam_principals
* move folder module to iam_principals
* copyright
* move organization module to iam_principals
* move project module to iam_principals
* move source-repository module to iam_principals
* update blueprints for iam_principals interface
* FAST bootstrap
* module READMEs fixes
* FAST bootstrap
* FAST networking stages
* FAST security stage
* FAST gke stage
* FAST multitenant bootstrap stage
* FAST multitenant resman stage
* tfdoc
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* fix module test
* Update 0-domainless-iam.md
* Update 0-domainless-iam.md
* Rename iam_principals to iam_by_principals
* Update IAM template to include iam_by_principals
* Update Resman README
* Fix ADR link format
---------
Co-authored-by: Julio Castillo <jccb@google.com >
2024-02-12 14:35:30 +01:00
Ludovico Magnocavallo
3b4b355800
Ensure data platform service accounts meet FAST requirements ( #2016 )
...
* use 9-chars prefix in data platform example test
* use 9-chars prefix in data platform example test
2024-01-28 14:00:32 +01:00
lcaggio
208902c8da
Fix Data platform foundation ( #1992 )
...
* FAST + Minimal DP
* Fix tests
---------
Co-authored-by: Julio Castillo <jccb@google.com >
2024-01-20 08:49:46 +01:00
Wiktor Niesiobędzki
6c258e6562
Fix description
2023-12-29 12:09:16 +01:00
Wiktor Niesiobędzki
1eea077460
Add service account email to outputs to manage its permissions
2023-12-29 12:09:16 +01:00
Wiktor Niesiobędzki
724dfda09c
Provider doesn't set defaults on config
...
Can't provide just one size (like `web_server` or `triggerrer`) because
of no defaults are taken:
module.composer.google_composer_environment.env: Modifying... [id=***]
╷
│ Error: googleapi: Error 400: Found 6 problems:
│ 1) You have to specify Scheduler CPUs not lower than 0.5.
│ 2) You have to specify number of schedulers larger than 0.
│ 3) You have to specify Web Server CPUs not lower than 0.5.
│ 4) You have to specify Worker CPUs not lower than 0.5.
│ 5) You have to specify minimum number of workers larger than 0.
│ 6) Triggerer memory must be between 1.00GB and 6.50GB for given vCpu
So provide the defaults as set workloads_config == null
2023-12-29 12:09:16 +01:00
Ludovico Magnocavallo
bba814c091
Custom role factories for organization and project modules ( #1912 )
...
* backport custom role factories
* backport from fast ci/cd branch
* indent
* tfdoc
* fix module tests
2023-12-11 14:16:39 +00:00
Wiktor Niesiobędzki
886734e1e9
Add trigger configuration for Composer ( #1916 )
2023-12-11 12:54:49 +01:00
luigi-bitonti
98accdb3ad
Added PSC support to CloudSQL Module ( #1874 )
...
* Added Feature
* Added PSC to CloudSQL module
* Added psc to read replica
* Changed variables
* Updated README
* Ran fmt
* Removed old variables
* Fix README
* Fixed blueprints
* Fix README
* Fixed output
* Added more outputs and bug fixes
* Changed variable structure
* Bug fix
* Added PSC example.
2023-11-24 15:47:45 +01:00
Wiktor Niesiobędzki
9ff67fdaf0
Add DLP Service Agent role
2023-11-20 15:34:27 +01:00
ddaluka
543ea6e7f3
Fix/dlpagent ( #1868 )
...
Create DLP Service Account on service activation.
2023-11-20 14:11:01 +01:00
Wiktor Niesiobędzki
63f1dfb6de
end-to-end tests for Vertex blueprint
2023-11-15 11:04:12 +00:00
Francesco Spinelli
1c2f1c7b0d
Sql user features ( #1856 )
...
* added user type feature
* fix readme
* fix comment
* fix blueprint cloudsql users value + minor fix
* readme fix
* variables fix
* local var fix
* fix for in local var
* fix on readme
* fix intentations var in readme
* fix blueprint user quote
---------
Co-authored-by: Francesco Spinelli <francesco.spinelli@nttdata.com >
2023-11-13 10:27:14 +01:00
luigi-bitonti
d07f8fd33d
Added CMEK for Secret auto managed ( #1739 )
...
Allow to specify custom KMS keys for Secret Manager secrets
2023-11-10 16:45:47 +01:00
lcaggio
a0ae43fc6f
[Minimal Data Platform] Fix Landing and curated IAM ( #1832 )
...
* Fix IAM on Minimal DP
* fix
2023-11-01 17:53:06 +01:00
Simone Ruffilli
4decc641bb
Stop wrapping yamldecode with try() ( #1812 )
2023-10-25 16:16:05 +02:00
