200 Commits

Author SHA1 Message Date
Ludovico Magnocavallo
1a4b298cc9 Project factory additions, project module reuse implementation (#2899)
* add support for buckets

* add project-level interpolation for own SAs

* docs

* project reuse changes

* fix example

* tfdoc

* update check documentation tool

* fast tests

* blueprints

* typo
2025-02-15 20:37:45 +01:00
Julio Castillo
d0c3f04c10 Make service agents work in different universes (#2894)
* Make service agents work in different universes

* Use templatestring and two passes for service agent emails

* Fix tests
2025-02-14 12:16:07 +00:00
Julio Castillo
3ffe838e06 Add context to organization policiy factories (#2876) 2025-02-10 22:24:01 +00:00
Wiktor Niesiobędzki
58f22ce647 Add prefix to KMS ring, to allow easy recreate 2025-02-10 00:56:19 +01:00
Julio Castillo
e08698a410 Fix tests 2025-02-07 10:55:05 +01:00
Julio Castillo
3fc7144c4f Update examples and tests 2025-02-07 10:55:05 +01:00
Julio Castillo
68a5a701e7 Expose parameters module in org policy variables 2025-02-07 10:55:05 +01:00
Julio Castillo
c73035f4e5 Allow universe-bound projects to exclude services (#2852)
* Allow universe-bound projects to exclude services

* Update README
2025-01-30 08:48:58 +01:00
Julio Castillo
29e7669385 Support project creation in different universes (#2848)
* Support project creation in different universes

* Fix typo

* Revert prefix validation

* Add test

* Call new test

* Do not override project name
2025-01-29 12:40:41 +01:00
Julio Castillo
c59470a4fb Update logging_data_access type (#2816)
* Update logging_data_access variables to use types

* Fix dependencies

* fix schema

* Add missing comma

* Fix try
2025-01-14 16:00:35 +00:00
Julio Castillo
7eff7b19dc Add iam_by_principals_additive to project, organization and folder modules (#2814)
* First attempt at iam_by_principals_additive

* Remove validation

* Update IAM ADR

* Apply to organization and project modules

* Update READMEs

* Add tests

* Remove "cycle errors"
2025-01-14 12:32:19 +00:00
Joshua Wright
325a997d79 Add Alerts, Logging, Channels Factories (#2758)
* WIP: Logging Alerts Factory

* Implement Logging Alerts on Remaining Modules

* Documentation & FMT

* Convert To Multiple Factories

* Correct Project

* Update Documentation

* Update modules/project/alerts-factory.tf

Co-authored-by: Julio Castillo <jccb@google.com>

* Update fast/stages/0-bootstrap/data/logging-metrics/compliance.yaml

Co-authored-by: Julio Castillo <jccb@google.com>

* Update Tests, Resources

* tests

* Fix Tests

* Fix formatting

* Reformat metric filters

* Formatting, reordering, and small fixes

* Bring back alerts and metrics documentation

* Revert change bootstrap outputs.tf

* Fix project notification channel vars and factories

* Fix vars and factory for logging alerts

* Complete alert variable and factory

* Reorder fields

* Update readme

* Reorder variables

* Add schemas, update README, and fix some types

* Remove default alerts email from project and project-factory

* Move observability factory to a single file

* Add outputs to project module

* Add factories_config to PF data_defaults and data_overrides

* Reorder PF field processing

* Revert fast/ to master.

We'll do observability stuff in a separate PR

* Remove observability from FAST

* Remove new FAST tests

* Remove unused local

* Fix tests

---------

Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2025-01-05 19:49:20 +00:00
Julio Castillo
f1acc92864 Add support for log views and log scopes (#2776)
* Add views and tags to logging bucket

* Add logs scopes to project

* Add missing inventory
2024-12-18 18:29:44 +01:00
Julio Castillo
5daba7fa03 Update service agents spec 2024-11-05 12:10:23 +01:00
Wiktor Niesiobędzki
8b7607af63 Codespell / whitespace fixes 2024-10-30 10:30:37 +01:00
Wiktor Niesiobędzki
44a4268afc Fix E2E tests 2024-09-04 12:16:50 +02:00
Julio Castillo
0420dec32f Add deletion_policy to project module (#2502)
* Replace skip_delete with deletion_policy

* Fix blueprints

* Update apigee blueprint readme

* Remove skip_delete from test inventories
2024-08-16 18:33:39 +02:00
Ludovico Magnocavallo
db044296e1 Additional module schemas (#2494)
* resman modules

* billing account

* net-vpc subnets

* fast schemas and subnet validation
2024-08-09 13:58:05 +00:00
Ludovico Magnocavallo
4a61dba841 organization module factory schemas (#2491) 2024-08-09 10:22:57 +00:00
Wiktor Niesiobędzki
d395e9490d Fix quotas E2E tests 2024-08-06 11:49:29 +02:00
Julio Castillo
1bbff3cc3a Add support for dry-run org policies (#2454) 2024-07-30 13:12:57 +00:00
Ludovico Magnocavallo
64b403860e Project service agents moved block and enabled services (#2443)
* moved block for jit service agents

* allow passing in enabled services to service agent config
2024-07-24 12:02:53 +00:00
Julio Castillo
c0bf32e797 Refactor service agent management (#2423)
* Service agents script

* Service agents update

* WIP

* Update script and terraform

* Fix tests

* Fix linter

* Update docs

* Bring back pf example inventory

* Fix tests

* Fix more tests

* Fix tests

* Use dataclasses for build_service_agents.py

* Remove unneeded field() from build_service_agents

* Re-enable CMEK depends_on in project outputs

* Update tools/requirements.txt

* Enable storage in GCS example projects

* Fix tests

* Add CMEK Service Agents dependencies for services

* Fix typos and data platform cmek

* More typos
2024-07-23 22:05:38 +02:00
Wiktor Niesiobędzki
e58c1b8892 Add info about roles for connectors service agent 2024-06-30 20:17:50 +02:00
Ludovico Magnocavallo
41e583ffc9 add network tags outputs and examples to project module (#2350) 2024-06-09 07:52:15 +02:00
Ludovico Magnocavallo
dc686c3a84 Remove default location from gcs module (#2303)
* gcs module

* blueprints/apigee/bigquery-analytics

* tfdoc

* pubsub README md syntax
2024-05-24 07:02:33 +00:00
Ludovico Magnocavallo
e4941c27f2 Implement the full IAM interface for tags (#2269)
* IAM authoritative bindings in org module

* remove extra newline

* organization module

* project module

* tfdoc
2024-05-13 20:18:51 +02:00
Ludovico Magnocavallo
309792c559 Refactor vpc-sc support in project module, add support for dry run (#2229) 2024-04-22 09:28:01 +02:00
Wiktor Niesiobędzki
825bfca66d Document how to get quota_ids 2024-04-11 11:51:19 +02:00
Wiktor Niesiobędzki
a236222a93 Add project quotas factory 2024-04-11 11:51:19 +02:00
Ludovico Magnocavallo
9cace5272f Add support for quotas to project module (#2210)
* add support for quotas to project module

* tfdoc

* better outputs

* Ensure keys in `quota_configs` are unique

* update fast tests

* Make quota E2E testable

* Remove quota project caveat

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-04-10 17:03:04 +00:00
Ludovico Magnocavallo
e12cf83188 add links to factories doc (#2134) 2024-03-06 08:25:43 +01:00
Wiktor Niesiobędzki
ef19524b0b Update docs about role automatically granted to dataform SA 2024-03-04 07:47:26 +01:00
Wiktor Niesiobędzki
9a95ac10ed Once again fix e2e tests 2024-02-23 19:21:39 +01:00
Wiktor Niesiobędzki
8fd8ee0541 Fix too long project names on e2e tests 2024-02-23 11:41:58 +01:00
Julio Castillo
5197d5ca8d Allow projects as destinations for log sinks (#2102)
* Add project log sink destination to project module

* Add project log sink destination to folder module

* Add project log sink destination to organization module

* Fix typos

* Add project log sink destination to billing-account module

* Make filter field optional

* Update READMEs

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2024-02-21 08:41:13 +01:00
Ludovico Magnocavallo
71a64487d5 Extend FAST to support different principal types (#2064)
* add doc draft

* typos

* typo

* typo

* typos

* rewording

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* move iam variables to a separate file

* move billing-account module to iam_principals

* move data-catalog-policy-tag module to iam_principals

* move dataplex-datascan module to iam_principals

* move dataproc module to iam_principals

* move folder module to iam_principals

* copyright

* move organization module to iam_principals

* move project module to iam_principals

* move source-repository module to iam_principals

* update blueprints for iam_principals interface

* FAST bootstrap

* module READMEs fixes

* FAST bootstrap

* FAST networking stages

* FAST security stage

* FAST gke stage

* FAST multitenant bootstrap stage

* FAST multitenant resman stage

* tfdoc

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* fix module test

* Update 0-domainless-iam.md

* Update 0-domainless-iam.md

* Rename iam_principals to iam_by_principals

* Update IAM template to include iam_by_principals

* Update Resman README

* Fix ADR link format

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2024-02-12 14:35:30 +01:00
lcaggio
50c7d3c0e9 Add Tags in project output. (#2062) 2024-02-09 10:42:18 +01:00
Ludovico Magnocavallo
bf93b6fb4e fix typo in logging sinks interface (#2015) 2024-01-28 10:27:28 +01:00
Wiktor Niesiobędzki
0d486fb34e E2E tests fixes 2023-12-19 11:01:03 +01:00
Julio Castillo
01bd0b7b01 Add project-scoped secure tags (#1933) 2023-12-18 18:24:05 +01:00
Julio Castillo
b6e0557bbb Simplify organization tags.tf locals (#1932)
* Simplify organization tags.tf locals

* Fix boilerplate

* Override github provider version for tests
2023-12-18 16:09:22 +00:00
simonebruzzechesse
c50b732c79 Allow granting network user role on host project from project module and factory (#1930)
* Update shared vpc config for project factory and project module for more granular Shared VPC configuration

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-12-15 14:39:21 +01:00
Ludovico Magnocavallo
bba814c091 Custom role factories for organization and project modules (#1912)
* backport custom role factories

* backport from fast ci/cd branch

* indent

* tfdoc

* fix module tests
2023-12-11 14:16:39 +00:00
Wiktor Niesiobędzki
84a15c4d77 README fixes for #1907 2023-12-07 09:51:48 +00:00
Ludovico Magnocavallo
f548b65b1c Add support for subnet-level service network user grants to project module, improve docs (#1907)
* improve project factory example

* light refactor of project modules shared vpc internals and docs

* add support for subnet-level grants on host project
2023-12-07 09:07:48 +00:00
Wiktor Niesiobędzki
d9f9410ae5 Use unique names for logging buckets in examples
Logging bucket name can be reused only after 7 days (when it is actually
deleted). When different tests reuse the same name, the ones that are
executed as 2nd and later will fail with message:
```
Error updating Logging Bucket Config [...]: googleapi: Error 400: Buckets
must be in an ACTIVE state to be modified
```

As their actual state is:
```
lifecycleState: DELETE_REQUESTED
```
2023-12-03 10:03:22 +00:00
ddaluka
543ea6e7f3 Fix/dlpagent (#1868)
Create DLP Service Account on service activation.
2023-11-20 14:11:01 +01:00
Wiktor Niesiobędzki
fe485414e6 Add end-to-end tests for project module (#1823)
* Add end-to-end tests for project module
* Add inventory to data tests
* Add files to end-to-end test cases
* Review fixes - use named groups

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2023-11-03 18:04:19 +01:00
Simone Ruffilli
4decc641bb Stop wrapping yamldecode with try() (#1812) 2023-10-25 16:16:05 +02:00