kovagoadi/hunfabric
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,407 Commits 1 Branch 1 Tag
d3c80ce18b4e71e506ba079ac9b4521d3fe5f863
Commit Graph

221 Commits

Author SHA1 Message Date
Ludovico Magnocavallo
d3c80ce18b Rationalize location defaults across project factory module and FAST stages (#3392) 
* group default/overrides locations in a single object in pf module

* org setup

* project factory stage

* fix defaults

* pf defaults

* security stage defaults
 2025-10-08 07:12:15 +00:00
Ludovico Magnocavallo
7aa4e453d7 allow FAST stage 0 provider template to work with universe (#3379) 2025-10-05 11:17:31 +02:00
kovagoadam
505ee02fef Add support for billing export in 0-org-setup (#3347) 
* Add support for billing export in 0-org-setup

* Merge branch 'master' into add-billing-export-support

* Refactored billing export with adding support for bigquery_datasets in project_factory

* Renamed bigquery_dataset to datasets

* Fixed defaults.schema.md

* Fixed default.schema.md again

* Cleanup md's

* Fixed boilerplate

* Fixed JSON schema

* reword README, rename project file

* Moved dataset_id to the key of the map

---------

Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2025-10-01 10:12:45 +00:00
norbert-loderer
e205b11954 Add missing billing viewer role for org-ro service account in org-setup stage. (#3364) 2025-10-01 11:53:41 +02:00
kovagoadam
c334e3c60f Fixed small typo in project-factory module with log-buckets (#3357) 
* Fixed small typo in project-factory module

* Add log_analytics and retention to log_buckets
 2025-09-27 07:21:48 +00:00
norbert-loderer
2492494c68 Fixing typos and adding missing roles for Terraform and CI/CD service… (#3351) 
* Fixing typos and adding missing roles for Terraform and CI/CD service accounts in 0-org-setup stage.

* Updated organization schema to allow defining custom roles under iam_by_principals and updated organization.schema.md accordingly.
 2025-09-23 19:20:21 +00:00
Ludovico Magnocavallo
c0aa691052 v45.0.0 2025-09-20 12:07:40 +00:00
Ludovico Magnocavallo
177b70762f Merge remote-tracking branch 'origin/master' into fast-dev 2025-09-20 12:00:50 +00:00
norbert-loderer
2aae7b3ab6 Add conditional grants on security to pf service account in FAST stage 0 (#3338) 2025-09-19 08:15:05 +02:00
fenyvesi-levi
4102e53588 Added 2 IAM roles for pf SA and updated test (#3333) 
* Added 2 IAM roles for pf SA and updated test

* Updated role grant with condition
 2025-09-17 13:37:07 +00:00
Ludovico Magnocavallo
c93b628979 New security stage leveraging project factory and contexts (#3311) 
* wip

* wip

* fix contexts

* cas factory

* add support for context to cas module

* cas module contexts and tests

* cas factory

* rename legacy security stage, add test for new stage

* readmes

* doc

* tfdoc

* doc, outputs

* update inventory
 2025-09-08 18:59:16 +02:00
Ludovico Magnocavallo
68955ff809 Improve context support in vpc-sc module and stage / new FAST stages small fixes (#3305) 
* improve context support in vpc-sc module and stage

* fix stage env files

* fix stage env files

* fix FAST variables comments

* tfdoc

* fix locations order

* fix tests
 2025-09-07 14:59:46 +02:00
Ludovico Magnocavallo
756c02c21a Rename new botstrap stage to org-setup (#3299) 
* readme changes

* rename stage 0

* rename stage 0

* complete rename of stage 0

* tfdoc

* tfdoc

* tfdoc
 2025-09-04 13:34:16 +02:00
Ludovico Magnocavallo
bc6950e205 Rename FAST stages preparing for eventual deprecation (#3298) 
* renames

* links

* readme

* docs

* update pf modules tests for renames

* condition_vars context in modules

* data platform dataset

* fix links in stage 3 docs

* schema changes

* schema docs

* tfdoc

* update duplicates check

* fast legacy tests

* legacy schema

* fix tests
 2025-09-04 08:24:11 +02:00
Ludovico Magnocavallo
36648b6b63 FAST light implementation (#3255) 
* data wip

* wip data

* update org schema, add note on expansion

* all schemas, workload notes

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* wip

* data wip

* wip

* wip

* wip

* wip

* org module IAM context (using lookup)

* new-style context expansion in project IAM

* remove spurious file

* project module contexts

* finalize context replacement format for project module

* revert org module changes

* fix tag id interpolation in project

* fix tag id interpolation in project

* organization module context

* organization context test

* context expansion for folder tag bindings

* test context expansion for tag bindings

* service account module context

* simplify context local

* context for iam service account

* nuke blueprints

* remove links to blueprints

* vpc sc context in project module

* Add context to GCS module

* Add inline deps to plan_summary script

* Make context a top-level variable for folder, organization, sa

* Add add context top-level to VPC-SC

* move context out of factories_config variable

* tfdoc

* fix merge

* fix merge

* fix examples

* net-vpc module context

* add parent ids to folder context

* rename folder parent context

* fix folder parent check

* new project factory stub

* wip

* wip

* refactor defaults

* project iam

* bueckts and service accounts

* start adding context replacements

* better test data

* automation resources for folders and projects

* automation

* add support for project id interpolation

* first tested apply

* improve IAM description in gcs module

* add context to billing account module

* add notification channels to billing account module context

* add billing budgets to new pf

* schemas and defaults

* bootstrap wip

* bootstrap wip

* bootstrap wip

* pf outputs

* pf fixes

* fix pf sample data

* bootstrap lite fixes

* add locations to organization module contexts

* bootstrap lite fixes

* org fixes, billing accounts

* fix default project parent

* bootstrap lite wip

* add locations to gcs module context

* add context support to logging bucket module

* add context to pubsub module

* split out iam variables in gcs module

* fix logging bucket context test

* bootstrap log sink destinations

* streamline logging-bucket module variables

* fix logging bucket context test

* align logging bucket module interface in fast bootstrap

* add support for project-level log buckets to project factory

* support full context expansion in organization module log sinks

* log buckets in fast-lite bootstrap

* make og sink type optional in organization module

* log sinks in fast-lite bootstrap

* set tag values in factory context

* bootstrap lite data

* output files schema

* billing account schema

* output files

* output providers

* gcs output files

* boilerplate

* tflint

* check documentation

* check docs

* fix project module parent variable validation

* fix log bucket examples

* allow null parent in project module

* silence folder test errors

* fix billing account sink example

* fix project example

* fix billing account module

* fix folder tests

* fix FAST

* fix fast

* tfvars outputs

* wif

* cicd service accounts

* cicd

* allow defaults in context, minimal org policies

* support gcs managed folders in project factory and bootstrap lite

* support prefix in provider output files

* rename bootstrap stage

* gitignore

* gitignore

* security folder, billing IAM

* wip tfvars

* fix typo

* security IAM

* control tag iam/context via variables in organization module

* split tag creation from tag IAM to avoid circular refs

* port organization module tag changes to project module

* implement new-style context expansion in vpc-sc module

* fix fast vpc-sc tests

* boilerplate

* vpc sc stage

* schemas

* fast-lite compatibility for vpc sc stage

* make log project number optional in vpc-sc stage

* networking

* networking

* networking

* networking

* rename and move new stage under fast

* clone pf tests

* use context replacement for internal notification channels in billing account module

* support service agents in project module iam context replacements

* support service agents in project module iam context replacements

* add support for kms keys to project module context

* experimental pf example test and fixes

* fix schemas

* fix tests

* tfdoc

* tfdoc

* pf config

* experimental pf

* remove redundant dot from gcs managed folder IAM keys

* bootstrap experimental test

* project factory exp stage test

* skip tflint for bootstrap experimental test

* tflint

* fix gcs test

* documentation work

* documentation work

* Update README.md

* tfdoc

* tfdoc

* readme

* tfdoc

* readme

* readme

* readme

* readme

* support universe in pf exp projects

* missing universe service agents

* org policies import, non-admin billing IAM

* todo

* fix test

* custom constraints

* fast classic dataset

* fix test data

* context replacements in billing module log sinks

* fix typo

* add support for billing log sinks

* update docs

* readme

* cicd fix and test

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2025-09-02 08:38:57 +02:00
Ludovico Magnocavallo
0b974a3a00 Merge remote-tracking branch 'origin/master' into fast-dev 2025-09-01 15:09:29 +00:00
Zsolt Molnar
b3c7699b8c Enable ADMIN_READ audit log for sts.googleapis.com in automation (iac) project (#3290) 
* Enable ADMIN_READ audit log for sts.googleapis.com in the automation (iac) project for better workload identity debugging and auditing

* Fix FAST tests

* Test fix #2

* Test fix #3

* Final test fix

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2025-09-01 13:58:28 +00:00
Wiktor Niesiobędzki
71b7c68ca2 fix missing conditions in top-level-folders IAM 2025-09-01 12:52:52 +02:00
Matt
1b4dcd5b07 fix: Multi-tenant parameters from upstream bootstrap (#3265) 
* fix: Multi-tenant parameters from upstream bootstrap

* Delint

* fix tests, tfdoc

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2025-08-08 06:35:32 +00:00
Ludo
5459280294 Merge remote-tracking branch 'origin/master' into fast-dev 2025-07-25 18:16:09 +02:00
Ali-Aburub
ae6b3ee18d Conditionally creates billing sink if the billing account is of type resource (#3130) 
* updated billing.tf file to conditionally creates billing sink if the billing account resides out of org

* replace resources with modules

* replace individual iam resources with billing_iam local passed in the module

* update module name and path and move iam from local to module call

* update README.md

* Add log_bucket option to billing_account variable, and update billing account IAM assignments

* update 0-bootstrap README to reflect changes to billing account module

* Update current bootstrap tests to reflect the change to billing_account variable

* Create test for the case when billing account log bucket is created

* running fmt

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2025-07-21 15:09:36 +00:00
Jay Bana
88051c2d5b Add Data Product Reference Example to FAST Data Platform Stage (#3211) 
Add Data Product Reference Example to FAST Data Platform Stage

---------

Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
 2025-07-09 08:53:49 +00:00
Liam Nesteroff
1da9bbcfb2 Allowing multiple on-prem domains (#3219) 
* Allowing multiple on-prem domains

* added on_prem domain example for tests

* Made gcp domains (gcp.example.com) optional and not created if null
 2025-07-09 16:56:40 +10:00
Julio Castillo
7e20abc19d Update default FAST org policies (#3207) 2025-07-02 13:53:58 +00:00
Ludovico Magnocavallo
ef45aa704a Rename workflows config variable introduced in #3198 (#3199) 
* rename workflows_config.tf_var_files in resman stage

* fix test result

* fix cloud deploy module versions file
 2025-06-28 08:57:55 +00:00
Ludo
1dde5258fa Merge remote-tracking branch 'origin/master' into fast-dev 2025-06-21 17:37:53 +02:00
aumohr
71f969c21f Clean fast 2 security from vpcsc (#3187) 
* updated fast security readme

* removed not required data file

* change org policy condition to align with best practice documentation

* aligned test
 2025-06-20 13:22:22 +04:00
Ludovico Magnocavallo
fe0a8128dc Allow custom roles in context, add support for shared VPC IAM to project and project factory (#3163) 
* add support for custom roles and hos tproject iam to project modules

* align vpc factory
 2025-06-15 10:01:22 +02:00
Wiktor Niesiobędzki
a5af0703f6 Add notebooks, appengine and appspot to dns policy routing 2025-06-13 16:46:44 +02:00
Ludovico Magnocavallo
9e94a0836a Improve and document org policy tags use in FAST resman stage (#3162) 
* improve and document org policy tags use in resman

* tfdoc, tests
 2025-06-13 15:57:48 +02:00
Ludovico Magnocavallo
468c0ab49a fix IAM delegation for project factory on security KMS keys (#3147) 2025-06-10 12:56:29 +02:00
Ludovico Magnocavallo
743f771adb Add version tracking files to FAST (#3108) 
* Add version tracking files to FAST

* tfdoc

* Update test counts

* inventories

* tfdoc

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2025-05-21 16:14:06 +02:00
Jay Bana
0411074a73 Improves fast/data-platform-ng README for clarity (#3074) 
* Improves Data Platform README for clarity

* Quick fix to table of contents capitalization

* Fix broken link to stages README.md

* Fix broken link to stages README.md

* Add 2 diagrams.

* Update README.

* Update README.

* Fix section headings in README for consistency

* Clarify implementation details in README regarding Data Mesh principles and FAST stage integration

* Reposition folder structure diagram

* Remove Resource Hierarchy Overview section from README

* Remove diagram_resman.png from data platform development stage

* Update diagrams in README for Data Platform stage

* Refine descriptions of logical components in Data Platform architecture

* Refine README and configuration files for Data Platform stage, enhancing clarity and consistency in descriptions and structure.

* Add section for Provider and Terraform variables in README

* Remove demo files and configurations for Data Platform stage

* Fix file paths in IAM management references in README

* Add IAM principal for dp-platform and update resource counts in test files

* Update README to fix folder structure diagram order and alignment

* Fix capitalization and team naming inconsistencies in README

* Add note on IAM permissions review for production deployment in README

* Refactor IAM roles for dp-platform group in terraform.tfvars.sample

* Fix bucket name in product-0.yaml from 'exposed-ew8' to 'exposed'

---------

Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2025-05-21 07:30:25 +00:00
Julio Castillo
5ab29fc1c5 Disable creation of bridge perimeters (#3098) 
* Disable creation of bridge perimeters

* Simplify bootsstrap test manifests
 2025-05-20 09:13:27 +02:00
Ludovico Magnocavallo
bb5426128e relax wif org policy in IaC project (#3090) 2025-05-16 07:31:22 +00:00
Ludovico Magnocavallo
212e587f77 Fix permadiff in FAST bootstrap IAM (#3089) 
* Add ability to optionally update Cloud Run job containers outside Terraform.

Co-authored-by: Luca Prete <lucaprete@google.com>

* bug: mark policy_controller as optional (#3086)

* bug: mark policy_controller as optional

* Fix documentation

---------

Co-authored-by: Julio Castillo <jccb@google.com>

* fix bootstrap permadiff

* inventory

* inventory

---------

Co-authored-by: Luca Prete <preteluca@gmail.com>
Co-authored-by: Luca Prete <lucaprete@google.com>
Co-authored-by: Falcon Taylor-Carter <falconertc@gmail.com>
Co-authored-by: Julio Castillo <jccb@google.com>
 2025-05-16 09:10:39 +02:00
Julio Castillo
3bda69fd26 Map secops group to security by default (#3080) 
* Map secops group to security by default

* Update readmes

* Fix bootstrap inventories
 2025-05-12 11:16:45 +00:00
lcaggio
2d7b736657 Fix no VPC composer scenario and roles (#3075) 
Fix no VPC composer scenario and roles
 2025-05-09 10:49:45 +02:00
Ludovico Magnocavallo
cb7bed50e3 New FAST data platform (#3066) 
* copy from broken dp dev branch

* remove extra excalidraw file

* fix networking yaml

* tfdoc

* tfdoc

* nuke old data platform

* fix tests

* tests

* tflint

* high level diagram

* make location optional in composer schema

* add composer outputs

* docs

* remove schema docs

* tfdoc

* update service agent encryption composer def for composer 3

* encryption keys

* typo

* typo

* fix security IAM

* inventory

* tflint

* Fix roles and diagram.

* Fix tflint

* Fix test DP.

* Fix test

* Diagrams excalidraw gz

---------

Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
 2025-05-03 21:21:38 +00:00
simonebruzzechesse
1e2eea1296 2-secops stage (#3038) 
* new 2-secops stage
* new 3-secops-dev stage

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2025-04-18 15:57:29 +02:00
Simone Ruffilli
3c91acd181 Replace all instances of stackdriver.googleapis.com with log+mon (#3022) 
Replace all instances of stackdriver.googleapis.com with logging.googleapis.com  and monitoring.googleapis.com.
Fixes #2932
 2025-04-11 12:04:50 +00:00
Julio Castillo
b2e26e50ae Enable storage.restrictAuthTypes org policy (#3017) 2025-04-08 15:43:24 +02:00
Wiktor Niesiobędzki
9c93c6daa4 Add trusted images projects 2025-04-06 12:49:16 +02:00
Wiktor Niesiobędzki
bea36cb047 Add requireInvokerIam constraint to the polices to prevent public exposure of Cloud Run services 2025-03-31 20:46:48 +02:00
Ludo
0facab6724 Merge remote-tracking branch 'origin/master' into fast-dev 2025-03-28 09:38:22 +01:00
Ludovico Magnocavallo
c8e4179f2b Add custom routes for directpath to net-vpc module (#2966) 
* add custom routes for directpath to net-vpc module

* blueprint tests

* blueprint tests

* blueprint tests

* fast tests

* tfdoc

* module examples
 2025-03-19 10:22:47 +00:00
Julio Castillo
3ed801c464 Use VPC-SC perimeter factory in FAST 1-vpcsc stage (#2928) 
* Use VPC-SC perimeter factory in FAST 1-vpcsc stage

* Add boilerplate

* Fix linter and schema checks

* Fix tests

* Fix schemas
 2025-02-24 12:29:51 +00:00
Wiktor Niesiobędzki
58051e48e7 Fix tests 2025-02-19 16:47:15 +01:00
Julio Castillo
51bd19bc30 Update default FAST org policies (#2906) 
* Update org default org policies

* Update default FAST org policies
 2025-02-18 16:34:44 +01:00
Julio Castillo
348e4df081 Fix default compute.restrictProtocolForwardingCreationForTypes value (#2904) 2025-02-18 14:28:33 +01:00