Commit Graph

1639 Commits

Author SHA1 Message Date
Ludovico Magnocavallo
981e4581ee Add project-factory based data platform dataset to FAST project factory stage (#3957)
* dp rewrite stage 0, projects

* remove plan files

* generalize handling of basepath for projects in project-factory module

* central-0 ---> core-0

* add schemas, validate YAMLs, tags

* aspect types

* data catalog policy tag factory

* add support for data catalog taxonomy to project factory

* complete retrofit of old stage configuration, except networking

* shared vpc networking

* networking

* data platform as pf dataset

* docs

* test

* remove legacy dp stage, fix tests and links

* boilerplate

* tfdoc

* fix unrelated tfdoc

* schemas

* fix errors

* schema

* duplicate schemas

* yamllint

* Fix module naming convention for aspect-types

* Fix factories_config in vpcs.tf for net-vpc-factory compatibility

* Update schema documentation based on schema changes

* Fix false rename conflict in .config.yaml files

* Sync schemas and update documentation

* Fix path expansion for aspect-types and revert projects_input to master

* Restore path expansion for org_policies in projects-iam call

* Fix trailing newlines in schema duplicates to satisfy duplicate-diff

* Fix path expansion for data_catalog_taxonomy in taxonomies.tf

* Update inventory for data-platform test and clean up debug prints

* Add full values to data-platform inventory

* Align Stage 2 VPC Factory integration with Stage 0 and fix tests

TAG=agy

* Fix project factory context resolution and data platform datasets

- Update tag context keys in project factory to use file key without 'projects/' prefix.
- Fix tag reference in product-0.yaml.
- Fix shared_vpc_service_config in shared-0.yaml by moving service account to network_users.
- Set parent for domain-0 folder to data-platform.
- Mock net-dev-0 project ID in tests.
- Update inventories.

TAG=agy
CONV=4b37fa5b-bf59-4604-9e8f-b55353d967a0

* Fix project-level tag keys context resolution in project factory

* Fix commented out tag reference in domain-0 .config.yaml

* Fix merge() calls with empty arguments in project-factory and data-catalog-policy-tag

* Update Data Platform dataset README with prerequisites and customization guide

* Add Table of Contents to Data Platform dataset README

* docs: update Data Platform README with project templates tip

* Document data platform output files and linking sequence in README

* Update data platform README with VPC-SC and delegated IAM details

* Refactor data platform dataset and align stage defaults

* Update test inventory and variables for data platform with new prefix
2026-05-12 16:44:32 +02:00
Ludovico Magnocavallo
5d57658e36 Fix changelog categorization and enhance script (#3955)
* Fix categorization of PR #3949 in CHANGELOG.md

* Enhance changelog.py to error on uncategorized PRs

* Update skill to propose breaking changes to user
2026-05-10 17:15:16 +00:00
Ludovico Magnocavallo
34d99733ba Fix test race condition by ignoring pytest-* directories (#3954) 2026-05-10 17:33:30 +02:00
Ludovico Magnocavallo
a2493caf25 prep v55.4.0 2026-05-10 14:46:23 +00:00
Julio Castillo
78a5ffa198 Add support for SCIM to workforce identity (#3951)
* Add support for SCIM to workforce identity

* Update schemas and tests
2026-05-10 11:21:09 +00:00
Ludovico Magnocavallo
91fe329aca Fix E2E test failures for Cloud Run v2, KMS, Compute VM, and update GEMINI.md (#3953)
* Update Cloud Run v2 GPU examples in README

Remove launch_stage = "BETA" from examples as it now defaults to GA in the provider. This fixes E2E test failures where the API returns GA. Reference: https://github.com/hashicorp/terraform-provider-google/pull/17029

TAG=agy

* Fix KMS and Compute VM E2E test failures

Update README examples to avoid conflicts and unsupported modes, and update corresponding inventories.

TAG=agy

* Add instruction to run a single specific example test in GEMINI.md

TAG=agy
2026-05-10 11:53:30 +02:00
Ludovico Magnocavallo
48b62ea918 Fix cloud-function-v1 E2E test collision by adding prefix (#3952) 2026-05-09 20:55:10 +02:00
Ludovico Magnocavallo
1fd1c9ae98 Fix random test failures by ignoring .git in copytree (#3949) 2026-05-08 20:57:40 +02:00
Ludovico Magnocavallo
98554a9d20 feat(cloudsql): add support for enhanced query insights (#3948)
Adds support for enhanced query insights on cloud sql instances by adding enhanced_query_insights_enabled to the insights_config block. This allows enabling deeper visibility into query performance.

Closes #3890

TAG=agy
CONV=41331d43-c782-48a4-b0e7-bc8ad14866e9
2026-05-08 09:08:16 +02:00
Ludovico Magnocavallo
dff4378cdc add advanced_datapath_observability_config support (#3945)
Adds support for `advanced_datapath_observability_config` to the `gke-cluster-autopilot` module, matching the standard cluster module implementation.

Closes #3936

TAG=agy
CONV=9d4485ab-0fae-4f3d-a6e1-bbb6320d7c46
2026-05-07 16:02:38 +00:00
Ludovico Magnocavallo
48fdf03233 artifact-registry: support common_repository in maven, npm, and python remote formats (#3944)
Add `common_repository` support to `maven`, `npm`, and `python` remote repository configurations in the `artifact-registry` module. This replaces the deprecated `custom_repository` feature which is now discouraged by the provider.

Existing README example `registry-mirror` has been updated to use `common_repository`. A legacy test case `legacy_custom_repo` has been added to the bottom of `README.md` to ensure backward compatibility for `custom_repository` continues to work.

TAG=agy
CONV=ffe77e65-ccef-4701-95e6-4ba2d2446f1b
2026-05-07 12:00:40 +02:00
Ludovico Magnocavallo
bf9ccb7547 Allowing disabling logging and configuring optional fields in LB backend services (#3940)
* fix(modules): allow disabling logging and configuring optional fields in LB backend services

Replaced 'log_sample_rate' (number) with 'log_config' (object) in all Load Balancer Backend Service modules. This allows explicitly disabling logging ('enable = false') and configuring advanced options like 'optional_mode' and 'optional_fields', resolving infinite plan drift and the inability to disable logging.

Affected modules:
- net-lb-app-ext-regional
- net-lb-app-ext
- net-lb-app-int-cross-region
- net-lb-app-int
- net-lb-ext
- net-lb-int
- net-lb-proxy-int

Added test cases and updated documentation.

Fixes #3914

* style: format variables files with terraform fmt

* docs: add critical linting rule for AI agents to GEMINI.md
2026-05-07 07:07:58 +00:00
Ludovico Magnocavallo
30b9d4c74f feat(net-lb-app-int): support regional health checks (#3939)
Introduce support for regional health checks in the net-lb-app-int module while maintaining backward compatibility.

Added optional is_regional flag to health_check_configs (defaulting to false). When true, it creates google_compute_region_health_check instead of google_compute_health_check.

Updated backend services and outputs to merge both global and regional health check IDs. Added a new test case to verify regional health check functionality.

TAG=agy

CONV=6aff620c-e5a5-44eb-afe0-459cff820daa
2026-05-06 15:33:20 +00:00
lopezvit
6c9ed94602 Fix custom_roles not permeating when used in dns zone IAM (#3938)
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2026-05-06 16:23:42 +02:00
lopezvit
8eca61bb46 Add support for static IPs NAT to 2-networking (#3889)
* feat(2-networking): add support for static IPs NAT

* fix(linting): fix linting

* fix(linting): fix linting

* fix(2-networking): factory-cloudnat don't assume that the context values are present.

* fix(2-networking): factory-cloudnat pass region in a try to forward the problem to the module

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2026-05-06 13:21:58 +00:00
Luca Prete
04e64c4ae2 Move org policies lower in the project factory dependency chain to support extended context (#3937)
* module project-factory: include project in conditional_var context for org policies

* module project-factory: include project and folders in conditional_var context for org policies

- Move project org policies (explicit and factory) to projects-iam invocation.
- Move folder org policies (explicit and factory) to folder-X-iam invocations (levels 1-4).
- Inject folder_ids into projects-iam condition_vars and pass resolved folders.
- Update and regenerate test inventories (example.yaml, simple.yaml, hardened.yaml).

TAG=agy
CONV=e0f45850-ab01-4600-a2b6-4de62465c204

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2026-05-06 14:48:08 +02:00
Simon Roberts
26dbaa2d6e Enable terraform_naming_convention in tflint (#3930)
* Draft terraform_naming_convention

* Two fast/stages fixes for terraform_naming_convention

* Disable terraform_naming_convention for resources for now

* module fixes for terraform_naming_convention

* tfdoc

* Remove "moved" from recipe and needs-fixing

* Fix moved for spoke_ra

* fix tests

* Use default (snake_case) for resources

* factory.terraform_data.project-preconditions

* First-pass migration of resources + tests

* Fix tests/modules/organization

* Require snake_case for variables; Add annotations for _testing

* permit _fast_debug variable

* Fix net_vpc_factory and net_vpc_firewall tests

* tfdoc addons and recipe

* Fix more tests

* Fix some net-global -> net_global tests

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2026-05-06 06:06:26 +00:00
Luca Prete
ba56d9afbc Add Agent Engine identity type (#3875) 2026-05-05 10:22:21 +02:00
Luca Prete
4cf46f2dd9 Agent Engine: remove unnecesary permission after bug fix (#3926) 2026-05-04 09:05:48 +02:00
Samuele Perticarari
c0ee6842c6 feat(alloydb): Implement track_client_address field in AlloyDB module (#3916)
* Implement track_client_address field in AlloyDB

* Bump Google TF providers to 7.29
2026-04-28 07:40:25 +00:00
Julio Castillo
74d5cd633a Bump minimum OpenTofu version to 1.11.00 (#3918) 2026-04-27 19:12:01 +02:00
Sven Schliesing
be6eea5d6d Add example "Remote Docker registry with credentials" for artifact-registry (#3887)
* Add example "Remote Docker registry with credentials" for artifact-registry

* Add inventory

(cherry picked from commit 903c4c423c0264bf270f1da13245fa01e58163d9)

Add inventory

(cherry picked from commit fd439be6412c2ea281578ee49f61cb3399850521)

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2026-04-27 18:05:29 +02:00
Julio Castillo
99adea9d4f Add index when needed 2026-04-27 15:20:59 +02:00
Julio Castillo
c2283aa405 Add hints to pytest failures 2026-04-27 14:05:37 +02:00
Julio Castillo
ff529cc4f3 Fix e2e 2026-04-27 12:15:57 +02:00
Julio Castillo
99f53d37f0 New generate_plan_summary.py tool 2026-04-27 12:00:54 +02:00
Julio Castillo
8450edbf2b prep v55.3.0 2026-04-27 10:25:51 +02:00
Julio Castillo
78bf096052 Restore simple s0 inventory 2026-04-27 09:50:15 +02:00
Julio Castillo
30a76ae699 Fix FAST tests 2026-04-27 09:50:15 +02:00
Julio Castillo
ff5cd25869 Ignore md5_hash in plan_summary 2026-04-27 09:50:15 +02:00
Ludovico Magnocavallo
392ee6bda4 prep v55.2.0 2026-04-25 04:51:39 +00:00
Vijay Kumar Singh
a727a2ea02 feat(cloud-function-v2): add support for auto_update_policy (#3881)
Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2026-04-25 06:48:40 +02:00
Brewster Malevich
0a22ef3a6f Bump GH template action versions to avoid GHA Node 20 deprecation (#3906)
* Bump GH template action versions to avoid GHA Node 20 deprecation

Github Actions runners are deprecating Node 20 as Node 20 is EOL in April, 2026. More information: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/

This bumps all the GHAs in the FAST stage 0 templates to use newer versions that do not depend on Node 20. This avoids the EOL and clears the deprecation warning in GHA when users run any GHA workflows generated in stage 0.

I just ran stage 0 with these bumps and it seems to work fine.

* Fix failing fast tests from old GHA version in yaml inventory file

---------

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2026-04-25 04:48:00 +00:00
Julio Castillo
2eaa0d5e27 Add support for dynamic tags (#3897)
* Allow creation of dynamic tags

* Extend project factory and related modules to support dynamic values

* Extend folder and organization modules

* project and organization readme

* Simplify dynamic tag support and remove unnecessary restrictions

  • Schemas & Validations: Removed the restriction that forbade combining IAM fields with  allowed_values_regex  on tags. Updated validations in  project  and  organization  modules, and
  simplified all relevant JSON schemas.
  • Module Tag Bindings: Simplified the  tag_value  assignment in  folder ,  project ,  gcs ,  bigquery-dataset , and  kms  modules by removing the defensive  can(regex(...))  check and
  calling  templatestring  directly.
  • Outputs: Removed the  tags_dynamic  output from  project  and  organization  modules, as the same information is now available in  tag_keys .
  • Project Factory: Updated  tag_vars_projects  in  projects.tf  to use the native  namespaced_name  attribute and filtered manually for dynamic tags.

* fix(organization, project): fix linting and tests for dynamic tag support

- Align allowed_values_regex and description extraction in _tags_merged
  locals to use lookup() for consistency with other fields.
- Fix spacing in project context variable (alphabetical ordering).
- Update organization tags test to include the new cost_center tag key
  with allowed_values_regex.
- Update project tags test to include the new cost_center tag key and
  reflect the resolved allowed_values_regex on environment.

* refactor(gcs): refine tag bindings and fix context test

- Add _tag_bindings local to pre-resolve context references, enabling
  templatestring to receive a direct map reference (required by Terraform).
- Use var.context.tag_vars instead of the non-existent local.ctx.tag_vars.
- Fix HCL syntax in context.tfvars (escaped inner quotes).
- Update context test inventory to reflect 3 tag bindings including a
  dynamic value resolved via templatestring.

* refactor: align modules with tag binding context pattern

- Add _tag_bindings local + templatestring dance to cloud-run-v2,
  compute-vm, folder, kms modules (bigquery-dataset already had it)
- Exclude tag_vars from local.ctx in cloud-run-v2, compute-vm, folder,
  kms, project modules (bigquery-dataset already had it)
- Add tag_vars to context variable in cloud-run-v2, compute-vm modules
  (others already had it)
- Update all context tests with dynamic tag binding values using
  var.context.tag_vars

* docs: add module-level tftest.yaml test instructions to GEMINI.md

* docs: regenerate READMEs after tag-regex alignment

- Regenerate variable tables in 7 module READMEs to reflect
  line number shifts from prior tag-regex changes
- Add tag_vars exclusion to gcs ctx local
- Fix whitespace alignment in iam-service-account and
  project-factory tag_vars blocks
- Update tftest resource counts for organization and project
- Remove tags_dynamic from organization/project output tables

* fix(project-factory): update test inventory for tag_bindings module split

- Move tag binding address from folder-2 to folder-2-iam in test
  inventory (tag_bindings moved from creation to IAM modules)
- Update module instance count from 34 to 35
- Regenerate README tables after terraform fmt line shifts
- Apply terraform fmt to variables.tf

* refactor(project-factory): remove unnecessary depends_on from folder-iam modules

Folder IAM modules depend on their own folder creation modules, not
on module.projects. The explicit depends_on was leftover from an
earlier design.

* FAST stages

* Address review comments.

- FAST Stages:
  - Added tag_keys to output-files.tf in 0-org-setup to pass org tags via tfvars.
  - Sorted tag_keys and tag_values in output-files.tf.
  - Updated project-factory, networking, and security stages to use tag_keys.
  - Filtered tag_keys for dynamic tags only.
- Modules:
  - Excluded tag_vars from local.ctx in iam-service-account and organization.
  - Simplified tag_value in iam-service-account.
- Tests:
  - Updated test inventories for 0-org-setup and project-factory.

* Fix tf format

* Fix tfdoc

* docs: add ADR for templatestring vars convention and update status of base path ADR

* More tfdoc

* Update schemas

* Use endswith in context loop

* Address review

* Update FAST readmes

* Update last modules

* Terraform fmt

* Revert alloydb

* Fix whitespace

---------

Co-authored-by: Ludovico Magnocavallo <ludo@qix.it>
2026-04-24 20:45:45 +00:00
Ludovico Magnocavallo
fb33752d8d Support context interpolation for PAM email recipients (#3903) 2026-04-24 18:29:31 +02:00
Liam Nesteroff
f5ee82ab3a Add DNS Armor support (#3874)
* add dns armor module

* add dns armor to pf

* added missing/optional attributes

* Update project schemas

* Set version file copyright year to 2025

* replace module with single resource

* moved into it's own file

* Added tests and defaulting enabled to false

* Add optional name parameter and updated schemas

* make dns_threat_detector.enabled optional in project schemas

---------

Co-authored-by: Luca Prete <preteluca@gmail.com>
2026-04-24 08:58:48 +02:00
Luca Prete
427d9a3298 Add more context to dns-response-policy module (#3899) 2026-04-23 13:49:30 +02:00
Luca Prete
275dd6a9ea Add enable_deletion_protection variable to agent engine module (#3898) 2026-04-22 14:05:09 +00:00
Simone Ruffilli
9eb69ffaa3 Add attachment groups factory to 2-networking (#3871)
* feat(fast): add attachment groups factory to 2-networking

Adds support for `google_compute_interconnect_attachment_group` in the `2-networking` stage.

By implementing this at the factory level alongside `vlan-attachments`, users can now declaratively aggregate VLAN attachments across multiple VPCs and natively reference them using the `$attachment_groups:<key>` context identifier in their configuration YAMLs.

Includes:
- Factory implementation in `factory-vlan-attachments.tf`.
- New JSON schemas for `attachment-groups` and updates to `vlan-attachments` to support context linkage.
- Test coverage with new inventory generations.

Fixes #3791
2026-04-22 10:22:50 +00:00
Hemanand
eaa420534b Add agent engine BYOC support (#3885)
* feat(agent-engine): add support for container and custom image specs

- Add container_config to deployment_files.
- Add image_spec with build_args to source_config.
- Make agent_framework optional and document supported values.
- Implement dynamic specs for container and source deployments.
- Add examples and automated tests for new deployment types.

* chore: update Google provider version to 7.28.0 across modules

Mechanical update of versions.tf and versions.tofu files using tools/versions.py.

* feat(agent-engine): refactor for container deployments and API alignment

- Group deployment settings under 'deployment_config' (renamed from 'deployment_files').
- Support container-based deployments via 'container_config' and 'image_spec'.
- Refactor 'source_files_config' (renamed from 'source_config') to include mutually exclusive 'python_spec' and 'image_spec'.
- Support 'developer_connect_config' as a source code type.
- Group engine settings (framework, env, secrets) under 'agent_engine_config'.
- Add support for 'memory_bank_config' persistent memory.
- Overhaul reasoning engine resources with dynamic blocks to match provider schema.
- Update all documentation examples, add TOC, and refresh test inventories.

* Update dynamic python_spec block and related example yamls

* Ignore changes setting for developer_connect_source under lifecycle management

* fixing review comments for `try` and default path for `source_path`

---------

Co-authored-by: Hemanand <hemr@google.com>
Co-authored-by: Julio Castillo <jccb@google.com>
2026-04-21 17:46:20 +00:00
Simone Ruffilli
c8886b4d88 Support context interpolation for psc_allowed_consumer_projects in module cloudsql-instance (#3894) 2026-04-21 15:22:48 +00:00
Julio Castillo
c995ffba07 Skip IAM for the Agent Gateway Service Agent (#3893) 2026-04-21 10:33:14 +00:00
Ludovico Magnocavallo
25bd428d17 prep v55.1.0 2026-04-20 14:22:29 +00:00
Luca Prete
0abf021f04 Add more context and tests to agent-engine module (#3886) 2026-04-20 15:15:29 +02:00
Luca Prete
f2d4e937d1 Fix pre-commit hook (#3882) 2026-04-18 10:07:14 +02:00
Luca Prete
7a31a07df9 Add context to net-lb-app-int module (#3880) 2026-04-17 15:00:00 +02:00
Luca Prete
5ca68a3de0 Add context to net-lb-app-ext module (#3879) 2026-04-17 14:00:56 +02:00
Luca Prete
a851638f15 Add example for context on net-lb-int readme; updated copyright (#3878) 2026-04-17 09:43:53 +00:00
Luca Prete
cee2a48c87 Add context to net-address module (#3876) 2026-04-17 11:23:47 +02:00
Julio Castillo
ad912d795a Enable creation of organization- and folder-level service agents (#3877)
* Enable creation of organization- and folder-level service agents

* formatting

* Add folder test

* Add org tests

* linting

* more linting

* Fix tests
2026-04-16 17:35:17 +00:00