kovagoadi/hunfabric
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,792 Commits 1 Branch 1 Tag
647895a9283c97a46b236c554efd0e34fa2b7fc0
Commit Graph

283 Commits

Author SHA1 Message Date
Ludo
0d3cbfd912 Merge remote-tracking branch 'origin/master' into fast-dev 2024-11-05 12:37:34 +01:00
Julio Castillo
b7a4717fc3 Fix tests 2024-11-05 12:10:23 +01:00
Wiktor Niesiobędzki
d8a6f3a895 Fix tests 2024-11-05 11:22:43 +01:00
Wiktor Niesiobędzki
232a7cd853 Fix IAM drift in vertex-mlops 2024-11-05 11:22:43 +01:00
Ludo
7688072b0c Merge remote-tracking branch 'origin/master' into fast-dev 2024-11-04 10:45:44 +01:00
Wiktor Niesiobędzki
4ab9c74e9f Review fixes 2024-11-04 10:34:54 +01:00
Wiktor Niesiobędzki
81cb269e25 fix tests 2024-11-04 10:34:54 +01:00
Wiktor Niesiobędzki
59eade7c48 Workaround overlapping networks 2024-11-04 10:34:54 +01:00
Wiktor Niesiobędzki
28160055af replace deprecated google_notebooks_instance with google_workbench_instance 2024-11-04 10:34:54 +01:00
Ludo
035fcd6cc5 Merge remote-tracking branch 'origin/master' into fast-dev 2024-11-04 10:18:46 +01:00
Wiktor Niesiobędzki
d6f0ee9447 fix Vertex-ML-Ops e2e tests 2024-11-04 10:13:32 +01:00
Ludovico Magnocavallo
50ac3a5013 Refactor of FAST resource management and subsequent stages (#2648) 
* untested

* pllan testing

* fix stage 2s

* move providers to their own file

* single-environment stage 3

* fixes and moved blocks

* stage3 factory

* doc

* review comments

* review comments

* tfdoc

* fasts tage 1 tests

* netsec as stage 2

* fix backported roles

* fix backported roles

* tfdoc

* fixes

* fix tag value roles in stage 1

* remove checklist, fix stage 1 tests

* inventory

* Small bugfix

* refactor context tag values

* fix previous merge

* fix previous merge

* fix previous merge

* support short names for top level automation resources, change top level context variable

* fix new top level context

* roll back merge changes to stage 0 outputs

* roll back more merge changes

* linting errors

* tfdoc

* fix tests, roll back merge in tenants stage

* tfdoc

* fix inventory

* optional stage 2 env folders and tag bindings

* tflint

* damn tflint

* damn tflint

* tfdoc

* fix networking tests

* tflint

* fix test inventories

* tfdoc

* use coalesce for project parents

* fix billing role conditions

* fix billing role conditions

* security stage tested (ngw resources need fixing/porting)

* boilerplate

* fix inventory

* stage envs and stage linking script

* initial work on resman docs, update diagram, improve teams folder

* resman README

* fix stage 2 IAM delegation

* remove checklist from bootstrap

* stage 1 tests

* stage 0 1 and 2 tests

* tflint

* tflint

* tfdoc

* GCVE stage refactor (untested)

* GCVE stage refactor (untested)

* GCVE stage 3

* gcve tests

* tflint

* tfdoc

* fix links

* module tests

* stages README

* move network security to stage 2

* network security tests

* replace stage links in README files

* minimal netsec stage refactor

* use factory for iac org policies, add configurable drs org policy for iac

* test mt stage

* tfdoc

* fix cicd workflows

* fix cicd workflows

* gke-dev stage

* tflint

* remove data platform stage

* exclude provider files via tfdoc opts

* remove data platform tests and links

* fix merge

* fix resman inventory

* boilerplate

* inventory

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
 2024-10-31 16:55:54 +01:00
Wiktor Niesiobędzki
8b7607af63 Codespell / whitespace fixes 2024-10-30 10:30:37 +01:00
Julio Castillo
f57635d044 Add managed folders suports to gcs module (#2530) 
* Add RPO, make versioning dynamic

* Add manaed folders

* Change autoclass and cors defaults to null

* Update README

* Add iam_by_principals

* Add managed folders var description

* Remove need for managed folders to end in /

* Add inventory to example

* Update readme

* Fix FAST tests
 2024-08-28 07:30:52 +00:00
Julio Castillo
2854ae6bd8 Remove "constraints/" from org policy names (#2450) 2024-07-29 15:15:04 +02:00
Julio Castillo
c0bf32e797 Refactor service agent management (#2423) 
* Service agents script

* Service agents update

* WIP

* Update script and terraform

* Fix tests

* Fix linter

* Update docs

* Bring back pf example inventory

* Fix tests

* Fix more tests

* Fix tests

* Use dataclasses for build_service_agents.py

* Remove unneeded field() from build_service_agents

* Re-enable CMEK depends_on in project outputs

* Update tools/requirements.txt

* Enable storage in GCS example projects

* Fix tests

* Add CMEK Service Agents dependencies for services

* Fix typos and data platform cmek

* More typos
 2024-07-23 22:05:38 +02:00
Julio Castillo
4e8adc9c43 Update modules/artifact-registry with newly-released features. (#2396) 
Fixes #2377
 2024-06-28 19:52:25 +02:00
Vince Gonzalez
2446b4dd7c Update README.md (#2239) 2024-04-26 01:14:32 +02:00
Julio Castillo
3af7e257d2 Add tflint to pipelines (#2220) 
* Fix terraform_deprecated_index

https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_index.md

* Fix terraform_deprecated_interpolation

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_interpolation.md

* Fix more indexing

* Remove unused variable

* Enable TFLint for modules

* Add tflint config file

* Fix chdir

* Lint modules

* TFLint fixes

* TFLint

* Fixes binauthz README

* Fixes DNS response policy tests. Restores MIG outputs.

* Fixes other DNS response policy tests.

* Update tests for fast 2-e

* Moar fixed tests

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
 2024-04-17 10:23:48 +02:00
Ludovico Magnocavallo
9414779cc2 Allow multiple PSA service providers in net-vpc module (#2218) 
* allowing multiple PSA service providers in net-vpc module

* tfdoc

* tfdoc

* Add tfvars/yaml tests

* fix module and tests

* re-enable inventory

* merge fix

* Add multiple PSA test case

* fix cloudsql example

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
 2024-04-16 15:02:36 +00:00
Wiktor Niesiobędzki
67382f53b7 Fix data-platform-minimal blueprint 2024-03-11 11:05:33 +01:00
Julio Castillo
993bef71aa Introduce mandatory OWNERS file for blueprint maintainership (#2131) 
* Delete deprecated/broken blueprints

* Adding OWNERS to all blueprints

* Fix links

* Update OWNERS

---------

Co-authored-by: javiergp <javiergp@users.noreply.github.com>
 2024-03-08 09:40:46 +01:00
Ludovico Magnocavallo
6941313c7d Factories refactor (#1843) 
* factories refactor doc

* Adds file schema and filesystem organization

* Update 20231106-factories.md

* move factories out of blueprints and create new factories  README

* align factory in billing-account module

* align factory in dataplex-datascan module

* align factory in billing-account module

* align factory in net-firewall-policy module

* align factory in dns-response-policy module

* align factory in net-vpc-firewall module

* align factory in net-vpc module

* align factory variable names in FAST

* remove decentralized firewall blueprint

* bump terraform version

* bump module versions

* update top-level READMEs

* move project factory to modules

* fix variable names and tests

* tfdoc

* remove changelog link

* add project factory to top-level README

* fix cludrun eventarc diff

* fix README

* fix cludrun eventarc diff

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
 2024-02-26 10:16:52 +00:00
simonebruzzechesse
d11c380aec Format python files in blueprints (#2079) 
* format python files in blueprints
* update check on blueprints python code
* update python linter in CI workflow
 2024-02-15 09:37:49 +01:00
Wiktor Niesiobędzki
b902b1dab9 Fix non-empty plan after apply 2024-02-13 07:40:31 +01:00
Ludovico Magnocavallo
71a64487d5 Extend FAST to support different principal types (#2064) 
* add doc draft

* typos

* typo

* typo

* typos

* rewording

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* move iam variables to a separate file

* move billing-account module to iam_principals

* move data-catalog-policy-tag module to iam_principals

* move dataplex-datascan module to iam_principals

* move dataproc module to iam_principals

* move folder module to iam_principals

* copyright

* move organization module to iam_principals

* move project module to iam_principals

* move source-repository module to iam_principals

* update blueprints for iam_principals interface

* FAST bootstrap

* module READMEs fixes

* FAST bootstrap

* FAST networking stages

* FAST security stage

* FAST gke stage

* FAST multitenant bootstrap stage

* FAST multitenant resman stage

* tfdoc

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* fix module test

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Rename iam_principals to iam_by_principals

* Update IAM template to include iam_by_principals

* Update Resman README

* Fix ADR link format

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2024-02-12 14:35:30 +01:00
Ludovico Magnocavallo
3b4b355800 Ensure data platform service accounts meet FAST requirements (#2016) 
* use 9-chars prefix in data platform example test

* use 9-chars prefix in data platform example test
 2024-01-28 14:00:32 +01:00
lcaggio
208902c8da Fix Data platform foundation (#1992) 
* FAST + Minimal DP

* Fix tests

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2024-01-20 08:49:46 +01:00
Wiktor Niesiobędzki
6c258e6562 Fix description 2023-12-29 12:09:16 +01:00
Wiktor Niesiobędzki
1eea077460 Add service account email to outputs to manage its permissions 2023-12-29 12:09:16 +01:00
Wiktor Niesiobędzki
724dfda09c Provider doesn't set defaults on config 
Can't provide just one size (like `web_server` or `triggerrer`) because
of no defaults are taken:
module.composer.google_composer_environment.env: Modifying... [id=***]
╷
│ Error: googleapi: Error 400: Found 6 problems:
│       1) You have to specify Scheduler CPUs not lower than 0.5.
│       2) You have to specify number of schedulers larger than 0.
│       3) You have to specify Web Server CPUs not lower than 0.5.
│       4) You have to specify Worker CPUs not lower than 0.5.
│       5) You have to specify minimum number of workers larger than 0.
│       6) Triggerer memory must be between 1.00GB and 6.50GB for given vCpu

So provide the defaults as set workloads_config == null
 2023-12-29 12:09:16 +01:00
Ludovico Magnocavallo
bba814c091 Custom role factories for organization and project modules (#1912) 
* backport custom role factories

* backport from fast ci/cd branch

* indent

* tfdoc

* fix module tests
 2023-12-11 14:16:39 +00:00
Wiktor Niesiobędzki
886734e1e9 Add trigger configuration for Composer (#1916) 2023-12-11 12:54:49 +01:00
luigi-bitonti
98accdb3ad Added PSC support to CloudSQL Module (#1874) 
* Added Feature

* Added PSC to CloudSQL module

* Added psc to read replica

* Changed variables

* Updated README

* Ran fmt

* Removed old variables

* Fix README

* Fixed blueprints

* Fix README

* Fixed output

* Added more outputs and bug fixes

* Changed variable structure

* Bug fix

* Added PSC example.
 2023-11-24 15:47:45 +01:00
Wiktor Niesiobędzki
9ff67fdaf0 Add DLP Service Agent role 2023-11-20 15:34:27 +01:00
ddaluka
543ea6e7f3 Fix/dlpagent (#1868) 
Create DLP Service Account on service activation.
 2023-11-20 14:11:01 +01:00
Wiktor Niesiobędzki
63f1dfb6de end-to-end tests for Vertex blueprint 2023-11-15 11:04:12 +00:00
Francesco Spinelli
1c2f1c7b0d Sql user features (#1856) 
* added user type feature

* fix readme

* fix comment

* fix blueprint cloudsql users value + minor fix

* readme fix

* variables fix

* local var fix

* fix for in local var

* fix on readme

* fix intentations var in readme

* fix blueprint user quote

---------

Co-authored-by: Francesco Spinelli <francesco.spinelli@nttdata.com>
 2023-11-13 10:27:14 +01:00
luigi-bitonti
d07f8fd33d Added CMEK for Secret auto managed (#1739) 
Allow to specify custom KMS keys for Secret Manager secrets
 2023-11-10 16:45:47 +01:00
lcaggio
a0ae43fc6f [Minimal Data Platform] Fix Landing and curated IAM (#1832) 
* Fix IAM on Minimal DP

* fix
 2023-11-01 17:53:06 +01:00
Simone Ruffilli
4decc641bb Stop wrapping yamldecode with try() (#1812) 2023-10-25 16:16:05 +02:00
Simone Ruffilli
6d89b88149 versions.tf maintenance + copyright notice bump (#1782) 
* Bump copyright notice to 2023

* Delete versions.tf on blueprints

* Pin provider to major version 5

* Remove comment

* Fix lint

* fix bq-ml blueprint readme

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
 2023-10-20 18:17:47 +02:00
Luca Prete
6c48512f7e [#1764] net-lb-int: add support for dual stack and multiple forwarding rules 2023-10-17 09:30:34 +00:00
Julio Castillo
dfc5023e0b Make deletion protection consistent across all modules (#1735) 
* Expose deletion_protection in GKE modules

* Make deletion protection consistent across all modules

* Add deletion_protection option to blueprints

* Fix blueprints tests

* Fix types

* Update READMEs

* Fix dp readme

* Fix cmek blueprint default deletion_protection

* Fix blueprints tests
 2023-10-05 17:31:07 +02:00
lcaggio
de883957fe Minimal Data Platform - Fix (#1730) 
* Fix

* Fix README

* Fix data platform minimal tests

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2023-10-04 12:15:51 +02:00
lcaggio
6889f02954 Fix data platform roles (#1725) 
* Fix Data Platform roles

* Fix README

* Fix blueprint tests

* Update cleanup dp steps

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2023-10-04 07:31:40 +02:00
Ludovico Magnocavallo
789328ff5a Bump provider versions to v5.0.0 (#1724) 
* bump provider versions to 5.0.0

* fix cloud run, logging and vpc-sc

* Fix secret manager

* fix gke nodepool

* fix gke multitenant stage and blueprint

* Moving alloydb module to experimental.

* Add project to bare resources in examples

* tfdoc

* fix svpc blueprint test

* Revert "fix svpc blueprint test"

This reverts commit 14f02659098070136e64ead600580dd52c23c339.

* Fix GKE peering project

* Disable tests in alloydb module

* Bring back secret ids in secret manager tests

* Remove duplicate key

* last push

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2023-10-03 12:15:36 +00:00
Ludovico Magnocavallo
df5daab6cc Allow using no service account in compute-vm (#1692) 
* module and tests

* align blueprints

* tfdoc

* remove stale variables

* fix blueprint

* variable description
 2023-09-19 16:56:51 +00:00
Julio Castillo
ad14a7d415 Update READMEs 2023-09-17 00:21:36 +02:00
Julio Castillo
9c878dc9cf Fix tests for new KMS IAM interface 2023-09-17 00:21:36 +02:00