kovagoadi/hunfabric
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
5,645 Commits 1 Branch 1 Tag
2fcb81c05cc3ef7df5522e07ebee6513520ef68e
Commit Graph

271 Commits

Author SHA1 Message Date
Wiktor Niesiobędzki
8b7607af63 Codespell / whitespace fixes 2024-10-30 10:30:37 +01:00
Julio Castillo
f57635d044 Add managed folders suports to gcs module (#2530) 
* Add RPO, make versioning dynamic

* Add manaed folders

* Change autoclass and cors defaults to null

* Update README

* Add iam_by_principals

* Add managed folders var description

* Remove need for managed folders to end in /

* Add inventory to example

* Update readme

* Fix FAST tests
 2024-08-28 07:30:52 +00:00
Julio Castillo
2854ae6bd8 Remove "constraints/" from org policy names (#2450) 2024-07-29 15:15:04 +02:00
Julio Castillo
c0bf32e797 Refactor service agent management (#2423) 
* Service agents script

* Service agents update

* WIP

* Update script and terraform

* Fix tests

* Fix linter

* Update docs

* Bring back pf example inventory

* Fix tests

* Fix more tests

* Fix tests

* Use dataclasses for build_service_agents.py

* Remove unneeded field() from build_service_agents

* Re-enable CMEK depends_on in project outputs

* Update tools/requirements.txt

* Enable storage in GCS example projects

* Fix tests

* Add CMEK Service Agents dependencies for services

* Fix typos and data platform cmek

* More typos
 2024-07-23 22:05:38 +02:00
Julio Castillo
4e8adc9c43 Update modules/artifact-registry with newly-released features. (#2396) 
Fixes #2377
 2024-06-28 19:52:25 +02:00
Vince Gonzalez
2446b4dd7c Update README.md (#2239) 2024-04-26 01:14:32 +02:00
Julio Castillo
3af7e257d2 Add tflint to pipelines (#2220) 
* Fix terraform_deprecated_index

https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_index.md

* Fix terraform_deprecated_interpolation

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_interpolation.md

* Fix more indexing

* Remove unused variable

* Enable TFLint for modules

* Add tflint config file

* Fix chdir

* Lint modules

* TFLint fixes

* TFLint

* Fixes binauthz README

* Fixes DNS response policy tests. Restores MIG outputs.

* Fixes other DNS response policy tests.

* Update tests for fast 2-e

* Moar fixed tests

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
 2024-04-17 10:23:48 +02:00
Ludovico Magnocavallo
9414779cc2 Allow multiple PSA service providers in net-vpc module (#2218) 
* allowing multiple PSA service providers in net-vpc module

* tfdoc

* tfdoc

* Add tfvars/yaml tests

* fix module and tests

* re-enable inventory

* merge fix

* Add multiple PSA test case

* fix cloudsql example

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
 2024-04-16 15:02:36 +00:00
Wiktor Niesiobędzki
67382f53b7 Fix data-platform-minimal blueprint 2024-03-11 11:05:33 +01:00
Julio Castillo
993bef71aa Introduce mandatory OWNERS file for blueprint maintainership (#2131) 
* Delete deprecated/broken blueprints

* Adding OWNERS to all blueprints

* Fix links

* Update OWNERS

---------

Co-authored-by: javiergp <javiergp@users.noreply.github.com>
 2024-03-08 09:40:46 +01:00
Ludovico Magnocavallo
6941313c7d Factories refactor (#1843) 
* factories refactor doc

* Adds file schema and filesystem organization

* Update 20231106-factories.md

* move factories out of blueprints and create new factories  README

* align factory in billing-account module

* align factory in dataplex-datascan module

* align factory in billing-account module

* align factory in net-firewall-policy module

* align factory in dns-response-policy module

* align factory in net-vpc-firewall module

* align factory in net-vpc module

* align factory variable names in FAST

* remove decentralized firewall blueprint

* bump terraform version

* bump module versions

* update top-level READMEs

* move project factory to modules

* fix variable names and tests

* tfdoc

* remove changelog link

* add project factory to top-level README

* fix cludrun eventarc diff

* fix README

* fix cludrun eventarc diff

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
 2024-02-26 10:16:52 +00:00
simonebruzzechesse
d11c380aec Format python files in blueprints (#2079) 
* format python files in blueprints
* update check on blueprints python code
* update python linter in CI workflow
 2024-02-15 09:37:49 +01:00
Wiktor Niesiobędzki
b902b1dab9 Fix non-empty plan after apply 2024-02-13 07:40:31 +01:00
Ludovico Magnocavallo
71a64487d5 Extend FAST to support different principal types (#2064) 
* add doc draft

* typos

* typo

* typo

* typos

* rewording

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* move iam variables to a separate file

* move billing-account module to iam_principals

* move data-catalog-policy-tag module to iam_principals

* move dataplex-datascan module to iam_principals

* move dataproc module to iam_principals

* move folder module to iam_principals

* copyright

* move organization module to iam_principals

* move project module to iam_principals

* move source-repository module to iam_principals

* update blueprints for iam_principals interface

* FAST bootstrap

* module READMEs fixes

* FAST bootstrap

* FAST networking stages

* FAST security stage

* FAST gke stage

* FAST multitenant bootstrap stage

* FAST multitenant resman stage

* tfdoc

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* fix module test

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Rename iam_principals to iam_by_principals

* Update IAM template to include iam_by_principals

* Update Resman README

* Fix ADR link format

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2024-02-12 14:35:30 +01:00
Ludovico Magnocavallo
3b4b355800 Ensure data platform service accounts meet FAST requirements (#2016) 
* use 9-chars prefix in data platform example test

* use 9-chars prefix in data platform example test
 2024-01-28 14:00:32 +01:00
lcaggio
208902c8da Fix Data platform foundation (#1992) 
* FAST + Minimal DP

* Fix tests

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2024-01-20 08:49:46 +01:00
Wiktor Niesiobędzki
6c258e6562 Fix description 2023-12-29 12:09:16 +01:00
Wiktor Niesiobędzki
1eea077460 Add service account email to outputs to manage its permissions 2023-12-29 12:09:16 +01:00
Wiktor Niesiobędzki
724dfda09c Provider doesn't set defaults on config 
Can't provide just one size (like `web_server` or `triggerrer`) because
of no defaults are taken:
module.composer.google_composer_environment.env: Modifying... [id=***]
╷
│ Error: googleapi: Error 400: Found 6 problems:
│       1) You have to specify Scheduler CPUs not lower than 0.5.
│       2) You have to specify number of schedulers larger than 0.
│       3) You have to specify Web Server CPUs not lower than 0.5.
│       4) You have to specify Worker CPUs not lower than 0.5.
│       5) You have to specify minimum number of workers larger than 0.
│       6) Triggerer memory must be between 1.00GB and 6.50GB for given vCpu

So provide the defaults as set workloads_config == null
 2023-12-29 12:09:16 +01:00
Ludovico Magnocavallo
bba814c091 Custom role factories for organization and project modules (#1912) 
* backport custom role factories

* backport from fast ci/cd branch

* indent

* tfdoc

* fix module tests
 2023-12-11 14:16:39 +00:00
Wiktor Niesiobędzki
886734e1e9 Add trigger configuration for Composer (#1916) 2023-12-11 12:54:49 +01:00
luigi-bitonti
98accdb3ad Added PSC support to CloudSQL Module (#1874) 
* Added Feature

* Added PSC to CloudSQL module

* Added psc to read replica

* Changed variables

* Updated README

* Ran fmt

* Removed old variables

* Fix README

* Fixed blueprints

* Fix README

* Fixed output

* Added more outputs and bug fixes

* Changed variable structure

* Bug fix

* Added PSC example.
 2023-11-24 15:47:45 +01:00
Wiktor Niesiobędzki
9ff67fdaf0 Add DLP Service Agent role 2023-11-20 15:34:27 +01:00
ddaluka
543ea6e7f3 Fix/dlpagent (#1868) 
Create DLP Service Account on service activation.
 2023-11-20 14:11:01 +01:00
Wiktor Niesiobędzki
63f1dfb6de end-to-end tests for Vertex blueprint 2023-11-15 11:04:12 +00:00
Francesco Spinelli
1c2f1c7b0d Sql user features (#1856) 
* added user type feature

* fix readme

* fix comment

* fix blueprint cloudsql users value + minor fix

* readme fix

* variables fix

* local var fix

* fix for in local var

* fix on readme

* fix intentations var in readme

* fix blueprint user quote

---------

Co-authored-by: Francesco Spinelli <francesco.spinelli@nttdata.com>
 2023-11-13 10:27:14 +01:00
luigi-bitonti
d07f8fd33d Added CMEK for Secret auto managed (#1739) 
Allow to specify custom KMS keys for Secret Manager secrets
 2023-11-10 16:45:47 +01:00
lcaggio
a0ae43fc6f [Minimal Data Platform] Fix Landing and curated IAM (#1832) 
* Fix IAM on Minimal DP

* fix
 2023-11-01 17:53:06 +01:00
Simone Ruffilli
4decc641bb Stop wrapping yamldecode with try() (#1812) 2023-10-25 16:16:05 +02:00
Simone Ruffilli
6d89b88149 versions.tf maintenance + copyright notice bump (#1782) 
* Bump copyright notice to 2023

* Delete versions.tf on blueprints

* Pin provider to major version 5

* Remove comment

* Fix lint

* fix bq-ml blueprint readme

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
 2023-10-20 18:17:47 +02:00
Luca Prete
6c48512f7e [#1764] net-lb-int: add support for dual stack and multiple forwarding rules 2023-10-17 09:30:34 +00:00
Julio Castillo
dfc5023e0b Make deletion protection consistent across all modules (#1735) 
* Expose deletion_protection in GKE modules

* Make deletion protection consistent across all modules

* Add deletion_protection option to blueprints

* Fix blueprints tests

* Fix types

* Update READMEs

* Fix dp readme

* Fix cmek blueprint default deletion_protection

* Fix blueprints tests
 2023-10-05 17:31:07 +02:00
lcaggio
de883957fe Minimal Data Platform - Fix (#1730) 
* Fix

* Fix README

* Fix data platform minimal tests

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2023-10-04 12:15:51 +02:00
lcaggio
6889f02954 Fix data platform roles (#1725) 
* Fix Data Platform roles

* Fix README

* Fix blueprint tests

* Update cleanup dp steps

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2023-10-04 07:31:40 +02:00
Ludovico Magnocavallo
789328ff5a Bump provider versions to v5.0.0 (#1724) 
* bump provider versions to 5.0.0

* fix cloud run, logging and vpc-sc

* Fix secret manager

* fix gke nodepool

* fix gke multitenant stage and blueprint

* Moving alloydb module to experimental.

* Add project to bare resources in examples

* tfdoc

* fix svpc blueprint test

* Revert "fix svpc blueprint test"

This reverts commit 14f02659098070136e64ead600580dd52c23c339.

* Fix GKE peering project

* Disable tests in alloydb module

* Bring back secret ids in secret manager tests

* Remove duplicate key

* last push

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2023-10-03 12:15:36 +00:00
Ludovico Magnocavallo
df5daab6cc Allow using no service account in compute-vm (#1692) 
* module and tests

* align blueprints

* tfdoc

* remove stale variables

* fix blueprint

* variable description
 2023-09-19 16:56:51 +00:00
Julio Castillo
ad14a7d415 Update READMEs 2023-09-17 00:21:36 +02:00
Julio Castillo
9c878dc9cf Fix tests for new KMS IAM interface 2023-09-17 00:21:36 +02:00
Julio Castillo
da883bab8c Update kms module key-level IAM 2023-09-17 00:21:36 +02:00
Oliver Frolovs
6eb862a775 GKE cluster modules: add optional kube state metrics (#1682) 
* `gke-cluster-standard`: add optional kube state metrics
* `gke-cluster-autopilot`: add optional kube state metrics

* FAST: add kube state metrics support for GKE
* blueprints/gke: add kube state metrics support

* Bump up the provider version to `v4.82.0`
 2023-09-15 12:18:45 +01:00
lcaggio
79723f9ce1 Add lineage on Minimal Data Platform blueprint (#1679) 
Add lineage on Minimal Data Platform blueprint
 2023-09-14 17:52:19 +02:00
Pierre Formont
0e7cfc8b29 add AIRFLOW_VAR_ prefix to environment variables in data-platform blueprints (#1651) 
* add AIRFLOW_VAR_ prefix to env vars in minimal blueprint

* update DAGs for env vars in minimal blueprint

* add AIRFLOW_VAR_ prefix to env vars in foundation blueprint

* update DAGs for env vars in foundation blueprint

* apply tf linting

---------

Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2023-09-08 07:38:28 +00:00
simonebruzzechesse
77c1e69666 New phpIPAM serverless third parties solution in blueprints (#1642) 
* Added new phpIPAM serverless third parties solution in blueprints
* added jit to iap.googleapis.com service in project module
* updated tests
 2023-09-07 15:30:22 +02:00
Julio Castillo
3d4cc7164a Bump provider version to 4.80.0 2023-09-05 09:48:15 +02:00
Ludovico Magnocavallo
819894d2ba IAM interface refactor (#1595) 
* IAM modules refactor proposal

* policy

* subheading

* Update 20230816-iam-refactor.md

* log Julio's +1

* data-catalog-policy-tag

* dataproc

* dataproc

* folder

* folder

* folder

* folder

* project

* better filtering in test examples

* project

* folder

* folder

* organization

* fix variable descriptions

* kms

* net-vpc

* dataplex-datascan

* modules/iam-service-account

* modules/source-repository/

* blueprints/cloud-operations/vm-migration/

* blueprints/third-party-solutions/wordpress

* dataplex-datascan

* blueprints/cloud-operations/workload-identity-federation

* blueprints/data-solutions/cloudsql-multiregion/

* blueprints/data-solutions/composer-2

* Update 20230816-iam-refactor.md

* Update 20230816-iam-refactor.md

* capture discussion in architectural doc

* update variable names and refactor proposal

* project

* blueprints first round

* folder

* organization

* data-catalog-policy-tag

* re-enable folder inventory

* project module style fix

* dataproc

* source-repository

* source-repository tests

* dataplex-datascan

* dataplex-datascan tests

* net-vpc

* net-vpc test examples

* iam-service-account

* iam-service-account test examples

* kms

* boilerplate

* tfdoc

* fix module tests

* more blueprint fixes

* fix typo in data blueprints

* incomplete refactor of data platform foundations

* tfdoc

* data platform foundation

* refactor data platform foundation iam locals

* remove redundant example test

* shielded folder fix

* fix typo

* project factory

* project factory outputs

* tfdoc

* test workflow: less verbose tests, fix tf version

* re-enable -vv, shorter traceback, fix action version

* ignore github extension warning, re-enable action version

* fast bootstrap IAM, untested

* bootstrap stage IAM fixes

* stage 0 tests

* fast stage 1

* tenant stage 1

* minor changes to fast stage 0 and 1

* fast security stage

* fast mt stage 0

* fast mt stage 0

* fast pf
 2023-08-20 09:44:20 +02:00
Alejandro Leal
ea0de3adbb Fixing some typos 2023-08-18 05:51:00 +00:00
Ludovico Magnocavallo
79373721df Remove firewall policy management from resource management modules (#1581) 
* rename firewall policy module, fix outputs

* add TOC to firewall policy module

* don't depend policy on parent id

* remove firewall policy from resource management modules

* remove factory conditionals

* fast net a and b

* fast stages

* fast tfdoc

* fast tfdoc

* remove unused test

* fix shielded folder blueprint

* fix shielded folder blueprint
 2023-08-09 11:23:07 +00:00
Eunan Hardy
dd3a298892 Moved allow_net_admin to enable_features flag. Bumped provider version to 4.76 2023-08-07 14:27:20 +01:00
Wiktor Niesiobędzki
182ce27143 Add information about required groups 2023-08-06 20:27:59 +02:00
Ludovico Magnocavallo
9c75aa469c More module descriptions (#1572) 
* bigquery dataset

* data catalog policy tag

* net-address

* fix data catalog callers

* bigquery dataset views

* fix data catalog callers

* logging bucket

* net vpn ha
 2023-08-06 09:25:45 +00:00