kovagoadi/hunfabric
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
6,461 Commits 1 Branch 1 Tag
dc8a67b83adc1dfaa96111e5d31d771f007c9d83
Commit Graph

168 Commits

Author SHA1 Message Date
Julio Castillo
48f6b4cd49 Add PAM support (#3438) 
* PAM first pass

* Add factory and extend to organization

* Extend to project, add examples

* Add additionalProperties to all objects

* Fix boilerplate

* Expose pam_entitlements to project-factory

* Fix readme

* Move entitlements to second folder/project pass

* extend tests

* Fix readme

* Remove timeouts from inventories
 2025-10-20 12:50:37 +00:00
Wiktor Niesiobędzki
9f2beb1ee6 Use fixture custom role for modules/project tests 2025-10-13 18:58:52 +02:00
Wiktor Niesiobędzki
849e856437 Use email from variables in tests 2025-10-13 18:58:52 +02:00
Ludovico Magnocavallo
07d7a85da1 Lightly refactor service agents locals in project module (#3380) 
* lightly refactor project module service agent locals

* remove debug block

* tfdoc

* revert templatestring to format for universe kms

* reinstate moved block
 2025-10-05 10:25:04 +00:00
Ludovico Magnocavallo
4db29a48eb allow forcing jit service agents generation for universe (#3378) 2025-10-05 10:17:57 +02:00
Vannick Trinquier
cfe2e21ce7 feat: add support for SCC Custom Security Health Analytics module in … (#3372) 
* feat: add support for SCC Custom Security Health Analytics module in organization, folder and project modules

* fix: update description and docs

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2025-10-03 13:47:50 +02:00
Ludovico Magnocavallo
02b37c5958 Fix alert variable notification prompts type in project module (#3330) 2025-09-17 08:59:18 +02:00
Ludovico Magnocavallo
bc6950e205 Rename FAST stages preparing for eventual deprecation (#3298) 
* renames

* links

* readme

* docs

* update pf modules tests for renames

* condition_vars context in modules

* data platform dataset

* fix links in stage 3 docs

* schema changes

* schema docs

* tfdoc

* update duplicates check

* fast legacy tests

* legacy schema

* fix tests
 2025-09-04 08:24:11 +02:00
Ludovico Magnocavallo
36648b6b63 FAST light implementation (#3255) 
* data wip

* wip data

* update org schema, add note on expansion

* all schemas, workload notes

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* wip

* data wip

* wip

* wip

* wip

* wip

* org module IAM context (using lookup)

* new-style context expansion in project IAM

* remove spurious file

* project module contexts

* finalize context replacement format for project module

* revert org module changes

* fix tag id interpolation in project

* fix tag id interpolation in project

* organization module context

* organization context test

* context expansion for folder tag bindings

* test context expansion for tag bindings

* service account module context

* simplify context local

* context for iam service account

* nuke blueprints

* remove links to blueprints

* vpc sc context in project module

* Add context to GCS module

* Add inline deps to plan_summary script

* Make context a top-level variable for folder, organization, sa

* Add add context top-level to VPC-SC

* move context out of factories_config variable

* tfdoc

* fix merge

* fix merge

* fix examples

* net-vpc module context

* add parent ids to folder context

* rename folder parent context

* fix folder parent check

* new project factory stub

* wip

* wip

* refactor defaults

* project iam

* bueckts and service accounts

* start adding context replacements

* better test data

* automation resources for folders and projects

* automation

* add support for project id interpolation

* first tested apply

* improve IAM description in gcs module

* add context to billing account module

* add notification channels to billing account module context

* add billing budgets to new pf

* schemas and defaults

* bootstrap wip

* bootstrap wip

* bootstrap wip

* pf outputs

* pf fixes

* fix pf sample data

* bootstrap lite fixes

* add locations to organization module contexts

* bootstrap lite fixes

* org fixes, billing accounts

* fix default project parent

* bootstrap lite wip

* add locations to gcs module context

* add context support to logging bucket module

* add context to pubsub module

* split out iam variables in gcs module

* fix logging bucket context test

* bootstrap log sink destinations

* streamline logging-bucket module variables

* fix logging bucket context test

* align logging bucket module interface in fast bootstrap

* add support for project-level log buckets to project factory

* support full context expansion in organization module log sinks

* log buckets in fast-lite bootstrap

* make og sink type optional in organization module

* log sinks in fast-lite bootstrap

* set tag values in factory context

* bootstrap lite data

* output files schema

* billing account schema

* output files

* output providers

* gcs output files

* boilerplate

* tflint

* check documentation

* check docs

* fix project module parent variable validation

* fix log bucket examples

* allow null parent in project module

* silence folder test errors

* fix billing account sink example

* fix project example

* fix billing account module

* fix folder tests

* fix FAST

* fix fast

* tfvars outputs

* wif

* cicd service accounts

* cicd

* allow defaults in context, minimal org policies

* support gcs managed folders in project factory and bootstrap lite

* support prefix in provider output files

* rename bootstrap stage

* gitignore

* gitignore

* security folder, billing IAM

* wip tfvars

* fix typo

* security IAM

* control tag iam/context via variables in organization module

* split tag creation from tag IAM to avoid circular refs

* port organization module tag changes to project module

* implement new-style context expansion in vpc-sc module

* fix fast vpc-sc tests

* boilerplate

* vpc sc stage

* schemas

* fast-lite compatibility for vpc sc stage

* make log project number optional in vpc-sc stage

* networking

* networking

* networking

* networking

* rename and move new stage under fast

* clone pf tests

* use context replacement for internal notification channels in billing account module

* support service agents in project module iam context replacements

* support service agents in project module iam context replacements

* add support for kms keys to project module context

* experimental pf example test and fixes

* fix schemas

* fix tests

* tfdoc

* tfdoc

* pf config

* experimental pf

* remove redundant dot from gcs managed folder IAM keys

* bootstrap experimental test

* project factory exp stage test

* skip tflint for bootstrap experimental test

* tflint

* fix gcs test

* documentation work

* documentation work

* Update README.md

* tfdoc

* tfdoc

* readme

* tfdoc

* readme

* readme

* readme

* readme

* support universe in pf exp projects

* missing universe service agents

* org policies import, non-admin billing IAM

* todo

* fix test

* custom constraints

* fast classic dataset

* fix test data

* context replacements in billing module log sinks

* fix typo

* add support for billing log sinks

* update docs

* readme

* cicd fix and test

---------

Co-authored-by: Julio Castillo <jccb@google.com>
 2025-09-02 08:38:57 +02:00
Ludovico Magnocavallo
53f691320e Support IAM tag factory context expansion in organization / project modules and FAST resman stage (#3226) 
* tag factory context

* support tag factory contexts in resman

* schemas

* fix schema
 2025-07-09 09:04:17 +02:00
Liam Nesteroff
f07e4f64e9 Standardise reuse variable from project module and implement for net-vpc (#3205) 
* exposing the network_id from the net-vpc module for use with tag bindings

* convert vpc_create to vpc_reuse

* Changed the reuse vars to standardised attributes

* fixed readme tests and schemas

* modified apigee blueprint to use vpc_reuse with network_id passthrough

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2025-07-03 09:12:05 +10:00
Liam Nesteroff
1fbb2cb330 Added tag factory option for organization module (#3178) 
* Added tag factory option for organization module

* added tags-factory tests

* added tag factory for project module

* missing header

* added lookup catch for org tag values and fixed incorrect link in doco

* fixed factory locals from copy/paste

* added full doco/tests for project tags factory. fixed failed test looking for ID in yamls

* added context option for factories_config to configure existing K/Vs

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2025-06-23 16:24:43 +10:00
Ludovico Magnocavallo
fe0a8128dc Allow custom roles in context, add support for shared VPC IAM to project and project factory (#3163) 
* add support for custom roles and hos tproject iam to project modules

* align vpc factory
 2025-06-15 10:01:22 +02:00
jnahelou
d6004f0cd0 CMEK service agents mapping breaks Composer v2 backward compatibility (#3156) 
* allow backward compatibility for unmapped service agents

* docs(modules/project): provide a fallback mechanism in cmek for users to specify additional service agents when needed

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2025-06-12 20:27:41 +02:00
apichick
077d8719dd Added default compute network tier to project module (#3150) 2025-06-10 21:44:38 +00:00
Ludovico Magnocavallo
cb7bed50e3 New FAST data platform (#3066) 
* copy from broken dp dev branch

* remove extra excalidraw file

* fix networking yaml

* tfdoc

* tfdoc

* nuke old data platform

* fix tests

* tests

* tflint

* high level diagram

* make location optional in composer schema

* add composer outputs

* docs

* remove schema docs

* tfdoc

* update service agent encryption composer def for composer 3

* encryption keys

* typo

* typo

* fix security IAM

* inventory

* tflint

* Fix roles and diagram.

* Fix tflint

* Fix test DP.

* Fix test

* Diagrams excalidraw gz

---------

Co-authored-by: lcaggio <lorenzo.caggioni@gmail.com>
 2025-05-03 21:21:38 +00:00
Wiktor Niesiobędzki
6af479706d Fix failing E2E test 2025-02-28 18:45:14 +01:00
Wiktor Niesiobędzki
0a1b6c6e0f Add ability to refer to other project service accounts in Project Factory 2025-02-19 16:47:15 +01:00
Ludovico Magnocavallo
1a4b298cc9 Project factory additions, project module reuse implementation (#2899) 
* add support for buckets

* add project-level interpolation for own SAs

* docs

* project reuse changes

* fix example

* tfdoc

* update check documentation tool

* fast tests

* blueprints

* typo
 2025-02-15 20:37:45 +01:00
Julio Castillo
d0c3f04c10 Make service agents work in different universes (#2894) 
* Make service agents work in different universes

* Use templatestring and two passes for service agent emails

* Fix tests
 2025-02-14 12:16:07 +00:00
Julio Castillo
3ffe838e06 Add context to organization policiy factories (#2876) 2025-02-10 22:24:01 +00:00
Wiktor Niesiobędzki
58f22ce647 Add prefix to KMS ring, to allow easy recreate 2025-02-10 00:56:19 +01:00
Julio Castillo
e08698a410 Fix tests 2025-02-07 10:55:05 +01:00
Julio Castillo
3fc7144c4f Update examples and tests 2025-02-07 10:55:05 +01:00
Julio Castillo
68a5a701e7 Expose parameters module in org policy variables 2025-02-07 10:55:05 +01:00
Julio Castillo
c73035f4e5 Allow universe-bound projects to exclude services (#2852) 
* Allow universe-bound projects to exclude services

* Update README
 2025-01-30 08:48:58 +01:00
Julio Castillo
29e7669385 Support project creation in different universes (#2848) 
* Support project creation in different universes

* Fix typo

* Revert prefix validation

* Add test

* Call new test

* Do not override project name
 2025-01-29 12:40:41 +01:00
Julio Castillo
c59470a4fb Update logging_data_access type (#2816) 
* Update logging_data_access variables to use types

* Fix dependencies

* fix schema

* Add missing comma

* Fix try
 2025-01-14 16:00:35 +00:00
Julio Castillo
7eff7b19dc Add iam_by_principals_additive to project, organization and folder modules (#2814) 
* First attempt at iam_by_principals_additive

* Remove validation

* Update IAM ADR

* Apply to organization and project modules

* Update READMEs

* Add tests

* Remove "cycle errors"
 2025-01-14 12:32:19 +00:00
Joshua Wright
325a997d79 Add Alerts, Logging, Channels Factories (#2758) 
* WIP: Logging Alerts Factory

* Implement Logging Alerts on Remaining Modules

* Documentation & FMT

* Convert To Multiple Factories

* Correct Project

* Update Documentation

* Update modules/project/alerts-factory.tf

Co-authored-by: Julio Castillo <jccb@google.com>

* Update fast/stages/0-bootstrap/data/logging-metrics/compliance.yaml

Co-authored-by: Julio Castillo <jccb@google.com>

* Update Tests, Resources

* tests

* Fix Tests

* Fix formatting

* Reformat metric filters

* Formatting, reordering, and small fixes

* Bring back alerts and metrics documentation

* Revert change bootstrap outputs.tf

* Fix project notification channel vars and factories

* Fix vars and factory for logging alerts

* Complete alert variable and factory

* Reorder fields

* Update readme

* Reorder variables

* Add schemas, update README, and fix some types

* Remove default alerts email from project and project-factory

* Move observability factory to a single file

* Add outputs to project module

* Add factories_config to PF data_defaults and data_overrides

* Reorder PF field processing

* Revert fast/ to master.

We'll do observability stuff in a separate PR

* Remove observability from FAST

* Remove new FAST tests

* Remove unused local

* Fix tests

---------

Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
 2025-01-05 19:49:20 +00:00
Julio Castillo
f1acc92864 Add support for log views and log scopes (#2776) 
* Add views and tags to logging bucket

* Add logs scopes to project

* Add missing inventory
 2024-12-18 18:29:44 +01:00
Julio Castillo
5daba7fa03 Update service agents spec 2024-11-05 12:10:23 +01:00
Wiktor Niesiobędzki
8b7607af63 Codespell / whitespace fixes 2024-10-30 10:30:37 +01:00
Wiktor Niesiobędzki
44a4268afc Fix E2E tests 2024-09-04 12:16:50 +02:00
Julio Castillo
0420dec32f Add deletion_policy to project module (#2502) 
* Replace skip_delete with deletion_policy

* Fix blueprints

* Update apigee blueprint readme

* Remove skip_delete from test inventories
 2024-08-16 18:33:39 +02:00
Ludovico Magnocavallo
db044296e1 Additional module schemas (#2494) 
* resman modules

* billing account

* net-vpc subnets

* fast schemas and subnet validation
 2024-08-09 13:58:05 +00:00
Ludovico Magnocavallo
4a61dba841 organization module factory schemas (#2491) 2024-08-09 10:22:57 +00:00
Wiktor Niesiobędzki
d395e9490d Fix quotas E2E tests 2024-08-06 11:49:29 +02:00
Julio Castillo
1bbff3cc3a Add support for dry-run org policies (#2454) 2024-07-30 13:12:57 +00:00
Ludovico Magnocavallo
64b403860e Project service agents moved block and enabled services (#2443) 
* moved block for jit service agents

* allow passing in enabled services to service agent config
 2024-07-24 12:02:53 +00:00
Julio Castillo
c0bf32e797 Refactor service agent management (#2423) 
* Service agents script

* Service agents update

* WIP

* Update script and terraform

* Fix tests

* Fix linter

* Update docs

* Bring back pf example inventory

* Fix tests

* Fix more tests

* Fix tests

* Use dataclasses for build_service_agents.py

* Remove unneeded field() from build_service_agents

* Re-enable CMEK depends_on in project outputs

* Update tools/requirements.txt

* Enable storage in GCS example projects

* Fix tests

* Add CMEK Service Agents dependencies for services

* Fix typos and data platform cmek

* More typos
 2024-07-23 22:05:38 +02:00
Wiktor Niesiobędzki
e58c1b8892 Add info about roles for connectors service agent 2024-06-30 20:17:50 +02:00
Ludovico Magnocavallo
41e583ffc9 add network tags outputs and examples to project module (#2350) 2024-06-09 07:52:15 +02:00
Ludovico Magnocavallo
dc686c3a84 Remove default location from gcs module (#2303) 
* gcs module

* blueprints/apigee/bigquery-analytics

* tfdoc

* pubsub README md syntax
 2024-05-24 07:02:33 +00:00
Ludovico Magnocavallo
e4941c27f2 Implement the full IAM interface for tags (#2269) 
* IAM authoritative bindings in org module

* remove extra newline

* organization module

* project module

* tfdoc
 2024-05-13 20:18:51 +02:00
Ludovico Magnocavallo
309792c559 Refactor vpc-sc support in project module, add support for dry run (#2229) 2024-04-22 09:28:01 +02:00
Wiktor Niesiobędzki
825bfca66d Document how to get quota_ids 2024-04-11 11:51:19 +02:00
Wiktor Niesiobędzki
a236222a93 Add project quotas factory 2024-04-11 11:51:19 +02:00
Ludovico Magnocavallo
9cace5272f Add support for quotas to project module (#2210) 
* add support for quotas to project module

* tfdoc

* better outputs

* Ensure keys in `quota_configs` are unique

* update fast tests

* Make quota E2E testable

* Remove quota project caveat

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
 2024-04-10 17:03:04 +00:00
Ludovico Magnocavallo
e12cf83188 add links to factories doc (#2134) 2024-03-06 08:25:43 +01:00