149 Commits

Author SHA1 Message Date
Julio Castillo
ee8b7396b7 Enable merge queue and remove fast-dev branch from CI (#4002) 2026-05-30 16:51:18 +02:00
Ludovico Magnocavallo
81f72e8068 Add FAST Prerequisites Skill and Gemini Skill Test Harness (#3979)
* initial version of a FAST pre-install skill

* first round of testing

* Update fast-0-org-setup-prereqs skill with improved UX and local path handling

- Add explicit lockout warning and stop condition if the user is not a member of the provided Admin Principal group.
- Streamline bootstrap project selection to only prompt for an override if the active gcloud project is rejected.
- Restrict dataset discovery strictly to the `fast/stages/0-org-setup/datasets/` directory.
- Improve location handling by referencing `defaults.schema.json` for Standard GCP and auto-configuring fixed regions for GCD.
- Add comprehensive `local_path` management: prompt for customization, create directories, move `defaults.yaml` to the local data folder, and symlink `0-org-setup.auto.tfvars` back to the stage directory.

* add testing scenarios, implement initial changes for scenario 2

* move skills

* move to a skills/fast subfolder

* Refactor fast-0-org-setup prereqs skill

* Add skill-turn-harness utility tool

* Use relative markdown links for skill references

* Use descriptive titles for markdown links in skill references

* Add descriptions to each phase in the prerequisites workflow map

* Use backslash for markdown line breaks in skill map

* Update README security warning to mention default .gitignore

* shebang

* Update fast prereqs skill rules to force sequential question flow and refine harness tool with proper ctrl+c handling and slugified log paths

* Move playbook-gcp-dev.yaml to fast/prerequisites/gcp-dev.yaml and update fast prerequisites

* docs(skill-turn-harness): detail autonomous pond testing approach

* docs(skill-turn-harness): add final_state_checks to pond architecture and update toc

* Refine fast prereqs SKILL and gcp-dev playbook to strictly align with one-question-at-a-time rule

* feat(skill-turn-harness): update playbook schema for autonomous persona mode

* feat(skill-turn-harness): implement autonomous persona testing mode and fallback logic

* docs(skill-turn-harness): document the three modes of testing and update ToC

* implement timeout, schema validation, configurable cli

* chore: remove accidentally committed log files

* chore: ignore logs directory

* feat(skill-harness): implement tool execution interception, configurable workspace, and modularized validation

* feat(skill-harness): add model configuration and update README

* fix(skill-harness): automatically inject -y flag to gemini commands

* docs(skill-harness): add TODO.md with analysis for skill environment dependencies

* feat(skill-harness): add working_dir support and clean up fixtures

- Implement working_dir in harness to run tests in specific directories.
- Rename test fixtures and playbooks to be more descriptive.
- Add E2E test for working_dir.
- Apply code quality improvements to harness.py (imports, linting).
- Update README with working directory considerations and usage notes.
- Update phase3-bootstrap-and-iam.md skill doc to add execution rule against creating temp scripts.

* fix: capture customer_id and respect relative paths

* Implement isolated temp workspace sandboxing with symlinks in test harness

* Configure GCD manual autonomous playbook and align Phase 3/4 steps order

* Fix linting and schema tests failures

- Add missing license headers to tools/skill-turn-harness files.

- Fix trailing spaces and newlines in playbooks.

- Ignore tools directory in schema tests workflow.

TAG=agy

CONV=1bb75453-c3e2-448b-bae9-8e332a068012

* Fix Python formatting with yapf

TAG=agy

CONV=1bb75453-c3e2-448b-bae9-8e332a068012

* Refactor skill-turn-harness to use Antigravity SDK

- Migrated harness from gemini-cli subprocesses to Antigravity SDK.
- Implemented real-time step streaming and console logging.
- Added color-coded terminal output (dark gray headers, blue inputs, pink outputs).
- Collapsed excessive newlines in streamed thoughts.
- Excluded harness codebase from workspace copy to prevent agent cheating.
- Enabled skills folder copy to resolve agent lookup loops.
- Added key validation and CLI --debug flag.

* Fix autonomous turn layout: print Turn ID before execution

- Moved the [Autonomous Turn X] header print to before running the agent turn.
- This groups the real-time thinking and tool calls under the correct Turn ID block, instead of displaying them before the label.

* Remove obsolete .log.md from prerequisites skill directory
2026-05-22 17:16:54 +00:00
Arpit Jain
748684dd9c ci: declare contents: read on linting and tests workflows (#3960)
linting runs prettier / yamllint / boilerplate checks.
tests runs terraform/tofu plan + unit tests with matrix providers.
Both are pure read - no commits, releases, or comments.

Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
2026-05-14 08:12:55 +02:00
Julio Castillo
ff6bacf297 Add PR title to automated review (#3917)
* Add PR title to automated review

* No emojis
2026-04-27 19:23:41 +00:00
Julio Castillo
74d5cd633a Bump minimum OpenTofu version to 1.11.00 (#3918) 2026-04-27 19:12:01 +02:00
Julio Castillo
13cd282d3b Add more context to automated reviews (#3891) 2026-04-20 18:55:08 +02:00
Ludovico Magnocavallo
11dba441f9 ci: remove automated-review label after PR review workflow (#3869) 2026-04-14 18:45:25 +02:00
Ludovico Magnocavallo
0b5e6cc2ac fix automated PR review workflow for forks (#3867) 2026-04-14 16:22:20 +02:00
Ludovico Magnocavallo
6847fae28d Automated PR Review (#3859)
* feat: add automated PR review workflow via Vertex AI Gemini

* fix: use workflow name instead of job_workflow_ref for WIF mapping

* fix: read review output from file to avoid JS syntax errors in github-script

* chore: remove redundant GITHUB_OUTPUT logic in PR review workflow

* chore: move configuration identifiers to GitHub Variables

* chore: upgrade model to 3.1-pro-preview and sanitize comment title

* fix: use full model path in vertex ai SDK for preview models

* fix: revert to gemini-2.5-pro due to Vertex API model resolution errors
2026-04-11 14:26:05 +02:00
Ludovico Magnocavallo
409c088c2c enforce schema docs in linting (#3843) 2026-04-08 10:20:50 +00:00
Wiktor Niesiobędzki
4f6e356e7f Fix issue in variable substitution 2026-02-18 18:49:30 +01:00
Ben Knutson
490d49bd3a Refactor Github Action per b/485167538 2026-02-18 18:49:30 +01:00
Ludovico Magnocavallo
fb21f6aaf8 Change factories_config type in FAST and project/vpc factory modules, add YAML schema validation (#3728)
* stage 0

* stage 1

* networking

* security

* pf stage

* tfdoc

* align schemas

* inventory

* fix observability

* pf module

* pf module budgets

* align fast stages

* align project subfactories

* tfdoc

* schema validation

* add missing schemas

* Fix observability types

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2026-02-11 15:29:49 +00:00
Wiktor Niesiobędzki
fca6b09728 Do not cancel terraform tests on tofu failure (#3538) 2025-11-18 14:10:22 +01:00
Wiktor Niesiobędzki
ada7df8855 Bump tflint version for performance 2025-10-28 07:33:15 +01:00
Wiktor Niesiobędzki
269e47a565 Improve linting and fix agent-engine module name 2025-10-26 11:56:41 +01:00
Wiktor Niesiobędzki
2c878aa90e .github linting 2025-10-24 13:11:17 +02:00
Wiktor Niesiobędzki
a61dc3cfdd Add yamlint step to linting workflow 2025-10-24 13:11:17 +02:00
Wiktor Niesiobędzki
0759cf5454 fix Terraform version linter 2025-10-20 13:52:11 +02:00
Wiktor Niesiobędzki
23dd44ce62 remove tf version from matrix, to keep workflow names stable across upgrades 2025-10-13 17:40:51 +02:00
Julio Castillo
9b9ad76ced Update gke-hub module to use new Policy Controller API (#3332)
* Update gke-hub to use new Policy Controller API

Fixes #3287

* Use same config format for servicemesh

* remove useless trys

* use ternaries as in the rest of the repo

* Update docs and fix tests

* Update variables

* Bump tofu version

* Bump terraform version 1.12
2025-10-13 09:47:39 +02:00
Ludovico Magnocavallo
36648b6b63 FAST light implementation (#3255)
* data wip

* wip data

* update org schema, add note on expansion

* all schemas, workload notes

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* wip

* data wip

* wip

* wip

* wip

* wip

* org module IAM context (using lookup)

* new-style context expansion in project IAM

* remove spurious file

* project module contexts

* finalize context replacement format for project module

* revert org module changes

* fix tag id interpolation in project

* fix tag id interpolation in project

* organization module context

* organization context test

* context expansion for folder tag bindings

* test context expansion for tag bindings

* service account module context

* simplify context local

* context for iam service account

* nuke blueprints

* remove links to blueprints

* vpc sc context in project module

* Add context to GCS module

* Add inline deps to plan_summary script

* Make context a top-level variable for folder, organization, sa

* Add add context top-level to VPC-SC

* move context out of factories_config variable

* tfdoc

* fix merge

* fix merge

* fix examples

* net-vpc module context

* add parent ids to folder context

* rename folder parent context

* fix folder parent check

* new project factory stub

* wip

* wip

* refactor defaults

* project iam

* bueckts and service accounts

* start adding context replacements

* better test data

* automation resources for folders and projects

* automation

* add support for project id interpolation

* first tested apply

* improve IAM description in gcs module

* add context to billing account module

* add notification channels to billing account module context

* add billing budgets to new pf

* schemas and defaults

* bootstrap wip

* bootstrap wip

* bootstrap wip

* pf outputs

* pf fixes

* fix pf sample data

* bootstrap lite fixes

* add locations to organization module contexts

* bootstrap lite fixes

* org fixes, billing accounts

* fix default project parent

* bootstrap lite wip

* add locations to gcs module context

* add context support to logging bucket module

* add context to pubsub module

* split out iam variables in gcs module

* fix logging bucket context test

* bootstrap log sink destinations

* streamline logging-bucket module variables

* fix logging bucket context test

* align logging bucket module interface in fast bootstrap

* add support for project-level log buckets to project factory

* support full context expansion in organization module log sinks

* log buckets in fast-lite bootstrap

* make og sink type optional in organization module

* log sinks in fast-lite bootstrap

* set tag values in factory context

* bootstrap lite data

* output files schema

* billing account schema

* output files

* output providers

* gcs output files

* boilerplate

* tflint

* check documentation

* check docs

* fix project module parent variable validation

* fix log bucket examples

* allow null parent in project module

* silence folder test errors

* fix billing account sink example

* fix project example

* fix billing account module

* fix folder tests

* fix FAST

* fix fast

* tfvars outputs

* wif

* cicd service accounts

* cicd

* allow defaults in context, minimal org policies

* support gcs managed folders in project factory and bootstrap lite

* support prefix in provider output files

* rename bootstrap stage

* gitignore

* gitignore

* security folder, billing IAM

* wip tfvars

* fix typo

* security IAM

* control tag iam/context via variables in organization module

* split tag creation from tag IAM to avoid circular refs

* port organization module tag changes to project module

* implement new-style context expansion in vpc-sc module

* fix fast vpc-sc tests

* boilerplate

* vpc sc stage

* schemas

* fast-lite compatibility for vpc sc stage

* make log project number optional in vpc-sc stage

* networking

* networking

* networking

* networking

* rename and move new stage under fast

* clone pf tests

* use context replacement for internal notification channels in billing account module

* support service agents in project module iam context replacements

* support service agents in project module iam context replacements

* add support for kms keys to project module context

* experimental pf example test and fixes

* fix schemas

* fix tests

* tfdoc

* tfdoc

* pf config

* experimental pf

* remove redundant dot from gcs managed folder IAM keys

* bootstrap experimental test

* project factory exp stage test

* skip tflint for bootstrap experimental test

* tflint

* fix gcs test

* documentation work

* documentation work

* Update README.md

* tfdoc

* tfdoc

* readme

* tfdoc

* readme

* readme

* readme

* readme

* support universe in pf exp projects

* missing universe service agents

* org policies import, non-admin billing IAM

* todo

* fix test

* custom constraints

* fast classic dataset

* fix test data

* context replacements in billing module log sinks

* fix typo

* add support for billing log sinks

* update docs

* readme

* cicd fix and test

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2025-09-02 08:38:57 +02:00
Julio Castillo
7e20abc19d Update default FAST org policies (#3207) 2025-07-02 13:53:58 +00:00
Julio Castillo
49bdf49909 Remove blueprint metadata validation (#3200) 2025-06-28 17:06:10 +00:00
Julio Castillo
271942f2ed Bump Terraform to 1.11 (#3120)
* Bump Terraform version to 1.11

* Update workflows
2025-05-29 11:11:39 +02:00
Ludovico Magnocavallo
4811f72a6c Add FAST to Python linting check (#3026)
* turn on Python linting for fast
* remove secops.py

---------

Co-authored-by: bruzzechesse <bruzzechesse@google.com>
2025-04-11 16:48:17 +02:00
Wiktor Niesiobędzki
3e84236345 Project object c14n in separte file
Create separate file for canonicalization of project factory objects and
introduce duplicate-diff lint checker.
2025-03-30 10:39:08 +02:00
simonebruzzechesse
b0021cc0f1 New SecOps anonymization pipeline (#2794)
* new secops anonymization pipeline

Co-authored-by: Ludovico Magnocavallo <ludomagno@google.com>
2025-02-17 19:23:19 +01:00
Julio Castillo
8a2320311d Add bucket_create to modules/gcs (#2827)
* Add `bucket_create` to `modules/gcs`

* Create local with bucket name

* Update variable description

* Fix bucket output

* Fix tests

* Fix tests

* Bump OpenTofu to 1.9.0 (needed for multi-var validations)
2025-01-21 23:48:36 +01:00
Julio Castillo
cf173701b6 New tool versions.py to manage versions.tf/tofu (#2803)
* Add provider_metadata to all modules

* Fix linting version check

* Another way

* Fix syntax

* Add fabric prefix to all provider meta

* Fix linting.yml
2025-01-09 09:57:48 +01:00
Wiktor Niesiobędzki
6d51c8da4d Use separate versions.tofu for OpenTofu constraints 2024-12-17 12:29:04 +01:00
Ludovico Magnocavallo
91da1c6482 Support customizable resource names to fast stage 1 (#2769)
* add support for resource names to fast stage 1

* tflint version
2024-12-16 18:07:28 +00:00
Ludovico Magnocavallo
0fa257e6b1 Support customizable resource names in FAST stage 0 (#2768)
* support customizable resource names in FAST stage 0

* tfdoc

* tflint

* remove comment

* use object type

* tfdoc

* bump tf version

* bump terraform version in versions files

* tf version in ci

* trigger workflow
2024-12-16 17:46:34 +01:00
Julio Castillo
71f72761af Run test and linting when pushing to master or fast-dev (#2730)
Maintainers occasionally force push to these branches. This will allow
tests to run without the need for a dummy PR
2024-11-22 19:21:38 +00:00
Ludovico Magnocavallo
abde265998 extend tests to fast-dev (#2646) 2024-10-30 10:39:36 +00:00
Wiktor Niesiobędzki
51ef390d04 Upload hidden files 2024-09-03 17:18:21 +02:00
Julio Castillo
bab85c88ba Bump provider to last release of version 5 (#2525)
* Bump provider to last release of version 5

* Remove default-versions copy in test env setup

* Use file instead of symlink in tools/lookfile

* Remove default-version reference

* Removed unused code
2024-08-27 16:50:59 +02:00
Julio Castillo
6c42ac00ab rename step 2024-08-09 15:43:11 +02:00
Julio Castillo
73fd175070 fix yaml again 2024-08-09 15:43:11 +02:00
Julio Castillo
9903c20ed9 Fix yaml 2024-08-09 15:43:11 +02:00
Julio Castillo
bb1dc40f77 Fix yaml syntax 2024-08-09 15:43:11 +02:00
Julio Castillo
714b97598f Add schema testing to PR workflow 2024-08-09 15:43:11 +02:00
Wiktor Niesiobędzki
afa6e7425c Test different versions of Terraform (#2399)
Add capability to run tests using terraform version 1.5.7 and OpenTofu. 

Automatic checks are run with  Terraform 1.7.4 and OpenTofu 1.7.2, it should be possible to override terraform version when Tests workflow is run manually.
2024-07-05 13:21:41 +02:00
Wiktor Niesiobędzki
cf31d83b15 Explicit YAPF style 2024-05-28 10:53:13 +02:00
Ludovico Magnocavallo
7a5dd4e6db FAST: add top-level folders and restructure teams/tenants in resman (#2254)
* remove teams and tenants from resman

* move fast features to stage 1, fix test inventories

* folders

* fix factory, add top level folder resources to outputs

* tfdoc

* stage 0 log sink defs

* tfdoc

* enable toc in resman readme

* simple tenants

* fast compatibility automation and logging

* testing fast-compatible tenants

* testing fast-compatible tenants

* tfdoc

* remove mt stages

* remove tests, fix links

* disable tflint

* fast tests

* make organization conditional in resman

* check names tool

* export real prefix to tfvars, prevent destroy errors

* prefix validation

* fix billing account export format

* tfdoc

* root node folder

* resman changes

* tenant resman roles

* first apply of tenant resman

* tenant log sinks in stage 1

* fix test vars

* tfdoc

* tenant vpc-sc access policy

* fix tests expected values

* tenant CI/CD

* identity providers

* wif

* tfdoc

* add comments to identity locals

* full-feature tenant resman apply

* tenant billing IAM

* stage test

* fix CI/CD comments

* tenant net stage verified

* tenant sec stage verified

* fix test

* README work

* tfdoc

* README

* README rewording

* README rewording

* tfdoc

* FAST excalidraw

* review comments

* diagram review changes

* add iam log sink for tenants

* remove redundant try from security stage

* Implement tflint-fast in Python driven by tftest.yaml files

* tflint

* test ci changes

* revert linting changes

* disable tflint for fast

* Create junit-style report for FAST tflint

* Remove junit-reporter

* YAPF tflint-fast.py

* Output tflint FAST to job summary

* Step summary

* Disable step_summary as output is not useful

* ignore tflint warning

* re-enable tflint on FAST

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
2024-05-15 09:17:13 +00:00
Wiktor Niesiobędzki
024d3255e6 Generalization of tflint call for FAST stages (#2225)
* Generalization of tflint call for FAST

* Fix tfvars path

* Fix tfvars path - depending where the file is

* Fix regex

* Reeanble linting

* Align test directory to stage name

* Align all fast stages to use tftest
2024-04-18 21:04:24 +02:00
Julio Castillo
f22837cd47 Enable TFLint in FAST stages (#2221) 2024-04-18 10:06:24 +02:00
Julio Castillo
3af7e257d2 Add tflint to pipelines (#2220)
* Fix terraform_deprecated_index

https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_index.md

* Fix terraform_deprecated_interpolation

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.5.0/docs/rules/terraform_deprecated_interpolation.md

* Fix more indexing

* Remove unused variable

* Enable TFLint for modules

* Add tflint config file

* Fix chdir

* Lint modules

* TFLint fixes

* TFLint

* Fixes binauthz README

* Fixes DNS response policy tests. Restores MIG outputs.

* Fixes other DNS response policy tests.

* Update tests for fast 2-e

* Moar fixed tests

---------

Co-authored-by: Simone Ruffilli <sruffilli@google.com>
2024-04-17 10:23:48 +02:00
Ludovico Magnocavallo
fc23c9c387 Update labeler version (#2192)
* update labeler version

* test on:documentation label

* test on:blueprints label

* remove on:documentation

* test

* test on:fast label

* test

* remove test event from workflow

* remove fast comment
2024-04-03 11:24:10 +02:00
Wiktor Niesiobędzki
7ec6c686a8 Add test reports to checks 2024-03-11 10:54:34 +01:00