diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 992fb39bf..2040288d9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -525,9 +525,9 @@ Similarly to our design principles above, we evolved a set of style conventions #### Group logical resources or modules in separate files -Over time and as our codebase got larger, we switched away from the canonical `main.tf`/`outputs.tf`/`variables.tf` triplet of file names and now tend to prefer descriptive file names that refer to the logical entities (resources or modules) they contain. +Over time and as our codebase got larger, we switched away from the canonical `main.tf` naming and now tend to prefer descriptive file names that refer to the logical entities (resources or modules) they contain. -We still use traditional names for variables and outputs, but tend to use main only for top-level locals or resources (e.g. the project resource in the `project` module), or for those resources that would end up in very small files. +We still use traditional names for variables and outputs, but tend to use main only for top-level locals or resources (e.g. the project resource in the `project` module), or for those resources that would end up in very small files. For smaller modules, a single `variables.tf` and `outputs.tf` is usually enough, however larger modules tend to group variables and outputs in multiple files, for example `variables-iam.tf` in `modules/project`. While some older modules and examples are still using three files, we are slowly bringing all code up to date and any new development should use descriptive file names. diff --git a/skills/README.md b/skills/README.md new file mode 100644 index 000000000..bef6c75c9 --- /dev/null +++ b/skills/README.md @@ -0,0 +1,7 @@ +# Fabric Skills + +This directory contains skills for AI agents to leverage Cloud Foundation Fabric (CFF). These skills provide context, guidelines, and tools for agents to perform specific tasks related to CFF. + +## Available Skills + +- **[fabric-builder](./fabric-builder)**: Generates idiomatic Cloud Foundation Fabric (CFF) Terraform code using CFF modules. Use when users ask to create GCP resources, use Fabric modules, or generate Terraform code for Google Cloud. diff --git a/skills/fabric-builder/SKILL.md b/skills/fabric-builder/SKILL.md new file mode 100644 index 000000000..8f89ce7ad --- /dev/null +++ b/skills/fabric-builder/SKILL.md @@ -0,0 +1,33 @@ +--- +name: fabric-builder +description: Generates idiomatic Cloud Foundation Fabric (CFF) Terraform code using CFF modules. Use when users ask to create GCP resources, use Fabric modules, or generate Terraform code for Google Cloud. +--- + +# Fabric Builder + +This skill generates idiomatic Terraform code using Cloud Foundation Fabric (CFF) modules, following established best practices and conventions. + +## Core Workflow + +1. **Understand Request:** Identify the GCP resources and relationships requested by the user. +2. **Fetch Module Info:** Identify the relevant CFF module(s) from the `modules/` folder on GitHub (`GoogleCloudPlatform/cloud-foundation-fabric`). +3. **Generate Terraform:** Output an idiomatic Terraform root module that consumes the CFF modules. + +## Important Guidance + +- **Conventions & Best Practices:** Consult [conventions.md](references/conventions.md) for guidelines on how to consume Fabric modules and write high-quality, idiomatic Terraform code. +- **Fetching Modules:** Do not invent module inputs or outputs. Use the `fabric.py` script to pull exact details (README, variables, examples) for a specific module from GitHub before using it. + - To list available modules: `python3 scripts/fabric.py modules` + - To fetch details for a module (README): `python3 scripts/fabric.py fetch readme ` + - To fetch variables files: `python3 scripts/fabric.py fetch variables ` + - To fetch outputs files: `python3 scripts/fabric.py fetch outputs ` + - To fetch schema files (useful for factories): `python3 scripts/fabric.py fetch schemas ` + - To fetch the latest release version: `python3 scripts/fabric.py release` + +## Guidelines for Output + +- **Root Module Output:** Your output must be a complete Terraform root module that calls the appropriate CFF modules to fulfill the user's requirements. +- **Use Fabric Modules:** Rely on CFF modules instead of raw `google_` resources whenever possible. +- **Example-based Learning:** Always refer to the module's README (fetched via `scripts/fabric.py`) for correct usage examples. +- **Module Source:** When generating module calls, use a GitHub source. It should look like this: `source = "github.com/GoogleCloudPlatform/cloud-foundation-fabric//modules/project?ref={VERSION}&depth=1"`. +- **Formatting & Validation:** Ensure the generated code is properly formatted. If possible, run `terraform fmt`, `terraform validate`, and `terraform plan` to check your work. diff --git a/skills/fabric-builder/references/conventions.md b/skills/fabric-builder/references/conventions.md new file mode 100644 index 000000000..ee4453442 --- /dev/null +++ b/skills/fabric-builder/references/conventions.md @@ -0,0 +1,64 @@ +# Cloud Foundation Fabric (CFF) Conventions for Module Consumers + +When generating Terraform code that consumes Cloud Foundation Fabric modules, you MUST adhere to the following conventions: + +## 1. Module Preference +- **Prefer Modules:** Always prefer CFF modules over raw `google_*` resources. +- **Flat Structure:** Avoid creating wrapper modules or nested module calls (modules calling other modules). Consume CFF modules directly in your root module. + +## 2. Naming Conventions +- **Use `prefix`:** For modules that support it (e.g., `project`, `gcs`), using a `prefix` variable is recommended but not mandatory. +- **Deterministic Naming:** Prefer using structured, deterministic tokens rather than random strings. + +## 3. Dependency Management +- **Output-to-Input:** Pass outputs from one module directly as inputs to another (e.g., `network = module.vpc.name`). +- **Ordering:** CFF modules encapsulate dependencies (like API activation) in their outputs. Rely on these outputs to ensure correct creation order. +- **Service Agents:** Use the `service_agents` output from the `project` module when granting IAM roles to Google-managed service accounts. +- **Implicit Dependencies:** Avoid explicit `depends_on` unless absolutely necessary. Rely on implicit dependencies (passing outputs to inputs) for better readability and maintainability. + +## 4. Factories +- **When to Use Factories:** Use factories when you need to manage a large number of similar resources (e.g., projects, VPCs, firewall rules) without duplicating module blocks. Factories separate configuration data (in YAML files) from Terraform logic. +- **Main Modules with Factory Support:** + - `project-factory`: For bulk creation of projects, folders, and budgets. + - `net-vpc-factory`: For bulk creation of VPCs and associated resources. + - `net-vpc`: Supports loading subnets and internal ranges from folders via `subnets_folder` and `internal_ranges_folder` keys in `factories_config`. + - `net-vpc-firewall`: Supports loading firewall rules from a folder via `rules_folder`. + - `organization` and `folder`: Support loading organization policies and custom roles. + - `vpc-sc`: Supports loading access levels and service perimeters. +- **Usage:** Pass the path to the directory containing YAML files to the `factories_config` variable of the respective module. + +## 5. Style for Root Modules +- **File Structure:** Make file structure dependent on size. For small configurations, use a single `main.tf`. For larger configurations, split into multiple files grouped by resource type (e.g., `main.tf` for general elements, `networking.tf`, `compute.tf`, etc.). +- **Variables & Defaults:** + - Define all variables in `variables.tf`, sorted alphabetically. + - Set defaults directly in the `default` attribute if a reasonable default exists or if provided by the user. + - Avoid creating a `terraform.tfvars` file unless explicitly requested by the user. +- **Value Handling & Providers:** + - **No Hardcoded Values:** Never use hardcoded values for project IDs, folder IDs, or other specific identifiers unless provided by the user. If a value is required, ask the user for it or create a variable. + - **Provider Parameters:** Do not set provider-level parameters like `project`, `zone`, or `region` at the resource level. Set them explicitly in the module calls. +- **Formatting:** Adhere to standard Terraform formatting and keep line lengths readable. Wrap complex ternaries in parentheses. +- **No Local Exec:** Never use `local-exec` or similar provisioners to run shell commands. Rely on native Terraform resources and data sources, preferably from official providers (i.e., no third-party providers). + +## 6. Impersonation and Backend Management +- **Impersonation:** If the user requires service account impersonation, add the `impersonate_service_account` attribute to the `provider "google"` and `provider "google-beta"` blocks in `providers.tf`. +- **Remote Backend:** If the user wants to use a remote backend, prefer `gcs`. Put the backend configuration in the `terraform` block inside `providers.tf`. If impersonation is used, also set it for the backend. + +**Template for `providers.tf`:** +```terraform +terraform { + backend "gcs" { + bucket = "${bucket}" + impersonate_service_account = "${service_account}" + } +} + +provider "google" { + impersonate_service_account = "${service_account}" +} + +provider "google-beta" { + impersonate_service_account = "${service_account}" +} +``` + + diff --git a/skills/fabric-builder/scripts/fabric.py b/skills/fabric-builder/scripts/fabric.py new file mode 100644 index 000000000..1a64f15b5 --- /dev/null +++ b/skills/fabric-builder/scripts/fabric.py @@ -0,0 +1,246 @@ +#!/usr/bin/env python3 +# Copyright 2026 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Download modules from Google Cloud Foundation Fabric via GitHub API.""" + +import argparse +import hashlib +import json +import pathlib +import shutil +import sys +import time +import urllib.request +import urllib.error +import logging +import re + +REPO = "GoogleCloudPlatform/cloud-foundation-fabric" +API = f"https://api.github.com/repos/{REPO}" +RAW_URL = f"https://raw.githubusercontent.com/{REPO}/master" +CACHE_DIR = pathlib.Path("/tmp/fabric_cache") +CACHE_TTL = 6 * 3600 +NO_CACHE = False + + +def cache_key(url): + return hashlib.sha1(url.encode()).hexdigest() + + +def cache_get(url): + if NO_CACHE: + return None + key = cache_key(url) + path = CACHE_DIR / key + if not path.is_file(): + logging.info(f"Cache miss for {url} (key: {key})") + return None + try: + entry = json.loads(path.read_text()) + if time.time() - entry["ts"] > CACHE_TTL: + logging.info(f"Cache expired for {url} (key: {key})") + path.unlink() + return None + logging.info(f"Cache hit for {url} (key: {key})") + return entry["data"] + except Exception: + return None + + +def cache_set(url, data): + if NO_CACHE: + return + key = cache_key(url) + try: + CACHE_DIR.mkdir(exist_ok=True) + (CACHE_DIR / key).write_text(json.dumps({"ts": time.time(), "data": data})) + logging.info(f"Cached data for {url} (key: {key})") + except Exception as e: + logging.warning(f"Could not write to cache: {e}") + + +def cache_clear(): + if CACHE_DIR.is_dir(): + shutil.rmtree(CACHE_DIR) + logging.info("Cache cleared") + else: + logging.info("No cache to clear") + + +def fetch(url, is_json=False, headers=None): + cached = cache_get(url) + if cached is not None: + return cached + + logging.info(f"Fetching: {url}") + req = urllib.request.Request(url, headers=headers or {}) + with urllib.request.urlopen(req, timeout=30) as r: + raw_data = r.read() + data = json.loads(raw_data) if is_json else raw_data.decode("utf-8") + + cache_set(url, data) + return data + + +def api_get(path): + url = f"{API}/{path}" + try: + return fetch(url, is_json=True, + headers={"Accept": "application/vnd.github.v3+json"}) + except urllib.error.HTTPError as e: + sys.exit(f"API Error ({e.code}): {e.reason}") + except Exception as e: + sys.exit(f"Error accessing API: {e}") + + +def fetch_url(url): + try: + return fetch(url) + except urllib.error.HTTPError as e: + if e.code == 404: + logging.warning(f"File not found: {url}") + return None + sys.exit(f"HTTP Error ({e.code}) fetching {url}") + except Exception as e: + sys.exit(f"Error fetching {url}: {e}") + + +def list_modules(): + entries = api_get("contents/modules") + if entries: + print("Available Fabric Modules:") + for e in entries: + if e["type"] == "dir": + print(f" - {e['name']}") + + +def latest_release(): + r = api_get("releases/latest") + if r: + print(r.get('tag_name')) + + +def fetch_module(module, fetch_type): + entries = api_get(f"contents/modules/{module}") + if not entries: + logging.error(f"Module '{module}' not found or empty.") + return + + files = [e["name"] for e in entries if e["type"] == "file"] + + if fetch_type == "readme": + want = [f for f in files if f == "README.md"] + elif fetch_type == "variables": + want = [f for f in files if f.startswith("variables") and f.endswith(".tf")] + elif fetch_type == "outputs": + want = [f for f in files if f.startswith("output") and f.endswith(".tf")] + elif fetch_type == "schemas": + # Check if schemas directory exists + has_schemas = any( + e["name"] == "schemas" and e["type"] == "dir" for e in entries) + if not has_schemas: + logging.warning(f"No schemas directory found in module '{module}'.") + return + + schema_entries = api_get(f"contents/modules/{module}/schemas") + if not schema_entries: + logging.error(f"Could not list schemas for module '{module}'.") + return + + schema_files = [ + e["name"] + for e in schema_entries + if e["type"] == "file" and e["name"].endswith(".schema.json") + ] + if not schema_files: + logging.warning(f"No schema files found in module '{module}/schemas'.") + return + + want = [f"schemas/{f}" for f in schema_files] + else: + want = [] + + if not want: + logging.warning( + f"No files found for type '{fetch_type}' in module '{module}'.") + return + + for f in want: + content = fetch_url(f"{RAW_URL}/modules/{module}/{f}") + if content: + # Strip copyright header + content = re.sub( + r"(?:/\*+\n)?^\s*(?:#|//|\*)\s*Copyright \d{4} Google LLC.*?limitations under the License\.\n(?:\s*\*/\n)?", + "", content, flags=re.DOTALL | re.MULTILINE) + # Strip leading newlines left after removing header + content = content.lstrip() + print(f"") + print(content) + print(f"") + + +def main(): + p = argparse.ArgumentParser( + description="Cloud Foundation Fabric helper for agents") + sp = p.add_subparsers(dest="cmd") + + sp.add_parser("modules", help="list available modules") + sp.add_parser("release", help="show latest release") + + p.add_argument("--clear-cache", action="store_true", + help="clear cache before running") + p.add_argument("--no-cache", action="store_true", + help="bypass cache and fetch from source") + p.add_argument("-v", "--verbose", action="store_true", + help="enable verbose logging") + + fm = sp.add_parser("fetch", help="fetch module files to stdout") + fsp = fm.add_subparsers(dest="fetch_cmd", required=True) + + f_readme = fsp.add_parser("readme", help="fetch README.md") + f_readme.add_argument("module", help="module name") + + f_vars = fsp.add_parser("variables", help="fetch variables*.tf") + f_vars.add_argument("module", help="module name") + + f_outputs = fsp.add_parser("outputs", help="fetch output*.tf") + f_outputs.add_argument("module", help="module name") + + f_schemas = fsp.add_parser("schemas", help="fetch JSON schemas") + f_schemas.add_argument("module", help="module name") + + args = p.parse_args() + + level = logging.INFO if args.verbose else logging.WARNING + logging.basicConfig(level=level, format='%(levelname)s: %(message)s', + stream=sys.stderr) + + global NO_CACHE + NO_CACHE = args.no_cache + + if args.clear_cache: + cache_clear() + + if args.cmd == "modules": + list_modules() + elif args.cmd == "release": + latest_release() + elif args.cmd == "fetch": + fetch_module(args.module, args.fetch_cmd) + else: + p.print_help() + + +if __name__ == "__main__": + main()