kovagoadi/hunfabric
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
271942f2ed06648770d8697c61ea931fd1f3cb76
hunfabric/modules/cloud-run-v2
History

README.md

Cloud Run Module

Cloud Run Services and Jobs, with support for IAM roles and Eventarc trigger creation. This module uses provider default value for deletion_protection, which means service is by default protected from removal (or reprovisioning).

IAM and environment variables

IAM bindings support the usual syntax. Container environment values can be declared as key-value strings or as references to Secret Manager secrets. Both can be combined as long as there is no duplication of keys:

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  name       = "hello"
  region     = var.region
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
      env = {
        VAR1 = "VALUE1"
        VAR2 = "VALUE2"
      }
      env_from_key = {
        SECRET1 = {
          secret  = module.secret-manager.secrets["credentials"].name
          version = module.secret-manager.version_versions["credentials:v1"]
        }
      }
    }
  }
  iam = {
    "roles/run.invoker" = ["allUsers"]
  }
  deletion_protection = false
}
# tftest modules=2 resources=5 fixtures=fixtures/secret-credentials.tf inventory=service-iam-env.yaml e2e

Mounting secrets as volumes

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  name       = "hello"
  region     = var.region
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
      volume_mounts = {
        "credentials" = "/credentials"
      }
    }
  }
  volumes = {
    credentials = {
      secret = {
        name    = module.secret-manager.secrets["credentials"].id
        path    = "my-secret"
        version = "latest" # TODO: should be optional, but results in API error
      }
    }
  }
  deletion_protection = false
}
# tftest modules=2 resources=4 fixtures=fixtures/secret-credentials.tf inventory=service-volume-secretes.yaml e2e

Mounting GCS buckets

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  name       = "hello"
  region     = var.region
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
      volume_mounts = {
        bucket = "/bucket"
      }
    }
  }
  revision = {
    gen2_execution_environment = true
  }
  volumes = {
    bucket = {
      gcs = {
        bucket       = var.bucket
        is_read_only = false
        mount_options = [ # Beta feature
          "metadata-cache-ttl-secs=120s",
          "type-cache-max-size-mb=4",
        ]
      }
    }
  }
  deletion_protection = false
}
# tftest inventory=gcs-mount.yaml e2e

Connecting to Cloud SQL database

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
      volume_mounts = {
        cloudsql = "/cloudsql"
      }
    }
  }
  volumes = {
    "cloudsql" = {
      cloud_sql_instances = [module.cloudsql-instance.connection_name]
    }
  }
  deletion_protection = false
}
# tftest fixtures=fixtures/cloudsql-instance.tf inventory=cloudsql.yaml e2e

Direct VPC Egress

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  name       = "hello"
  region     = var.region
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  revision = {
    gen2_execution_environment = true
    max_instance_count         = 20
    vpc_access = {
      egress = "ALL_TRAFFIC"
      subnet = var.subnet.name
      tags   = ["tag1", "tag2", "tag3"]
    }
  }
  deletion_protection = false
}
# E2E test disabled due to b/332419038
# tftest modules=1 resources=1 inventory=service-direct-vpc.yaml

VPC Access Connector

You can use an existing VPC Access Connector to connect to a VPC from Cloud Run.

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  revision = {
    vpc_access = {
      connector = google_vpc_access_connector.connector.id
      egress    = "ALL_TRAFFIC"
    }
  }
  deletion_protection = false
}
# tftest modules=1 resources=2 fixtures=fixtures/vpc-connector.tf inventory=service-vpc-access-connector.yaml e2e

If creation of the VPC Access Connector is required, use the vpc_connector_create variable which also supports optional attributes like number of instances, machine type, or throughput. The connector will be used automatically.

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  vpc_connector_create = {
    ip_cidr_range = "10.10.10.0/28"
    network       = var.vpc.self_link
    instances = {
      max = 10
      min = 3
    }
  }
  deletion_protection = false
}
# tftest modules=1 resources=2 inventory=service-vpc-access-connector-create.yaml e2e

Note that if you are using a Shared VPC for the connector, you need to specify a subnet and the host project if this is not where the Cloud Run service is deployed.

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = module.project-service.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  vpc_connector_create = {
    machine_type = "e2-standard-4"
    subnet = {
      name       = module.net-vpc-host.subnets["${var.region}/fixture-subnet-28"].name
      project_id = module.project-host.project_id
    }
    throughput = {
      max = 300
      min = 200
    }
  }
  deletion_protection = false
}
# tftest modules=4 resources=59 fixtures=fixtures/shared-vpc.tf inventory=service-vpc-access-connector-create-sharedvpc.yaml e2e

Using Customer-Managed Encryption Key

Deploy a Cloud Run service with environment variables encrypted using a Customer-Managed Encryption Key (CMEK). Ensure you specify the encryption_key with the full resource identifier of your Cloud KMS CryptoKey and that Cloud Run Service agent (service-<PROJECT_NUMBER>@serverless-robot-prod.iam.gserviceaccount.com) has permission to use the key, for example roles/cloudkms.cryptoKeyEncrypterDecrypter IAM role. This setup adds an extra layer of security by utilizing your own encryption keys.

module "project" {
  source          = "./fabric/modules/project"
  name            = "cloudrun"
  billing_account = var.billing_account_id
  prefix          = var.prefix
  parent          = var.folder_id
  services = [
    "cloudkms.googleapis.com",
    "run.googleapis.com",
  ]
}

module "kms" {
  source     = "./fabric/modules/kms"
  project_id = module.project.project_id
  keyring = {
    location = var.region
    name     = "${var.prefix}-keyring"
  }
  keys = {
    "key-regional" = {
    }
  }
  iam = {
    "roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
      module.project.service_agents.run.iam_email
    ]
  }
}

module "cloud_run" {
  source         = "./fabric/modules/cloud-run-v2"
  project_id     = module.project.project_id
  region         = var.region
  name           = "hello"
  encryption_key = module.kms.keys.key-regional.id
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  deletion_protection = false
}
# tftest modules=3 resources=11 e2e

Deploying OpenTelemetry Collector sidecar

# Reference: https://cloud.google.com/stackdriver/docs/instrumentation/opentelemetry-collector-cloud-run#gotc-provided-config

receivers:
  # Open two OTLP servers:
  # - On port 4317, open an OTLP GRPC server
  # - On port 4318, open an OTLP HTTP server
  #
  # Docs:
  # https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver/otlpreceiver
  otlp:
    protocols:
      grpc:
        endpoint: localhost:4317
      http:
        cors:
          # This effectively allows any origin
          # to make requests to the HTTP server.
          allowed_origins:
          - http://*
          - https://*
        endpoint: localhost:4318

  # Using the prometheus scraper, scrape the Collector's self metrics.
  #
  # Docs:
  # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/prometheusreceiver
  # https://opentelemetry.io/docs/collector/internal-telemetry/
  prometheus/self-metrics:
    config:
      scrape_configs:
      - job_name: otel-self-metrics
        scrape_interval: 1m
        static_configs:
        - targets:
          - localhost:8888

processors:
  # The batch processor is in place to regulate both the number of requests
  # being made and the size of those requests.
  #
  # Docs:
  # https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/batchprocessor
  batch:
    send_batch_max_size: 200
    send_batch_size: 200
    timeout: 5s

  # The memorylimiter will check the memory usage of the collector process.
  #
  # Docs:
  # https://github.com/open-telemetry/opentelemetry-collector/tree/main/processor/memorylimiterprocessor
  memory_limiter:
    check_interval: 1s
    limit_percentage: 65
    spike_limit_percentage: 20

  # The resourcedetection processor is configured to detect GCP resources.
  # Resource attributes that represent the GCP resource the collector is
  # running on will be attached to all telemetry that goes through this
  # processor.
  #
  # Docs:
  # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor
  # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/resourcedetectionprocessor#gcp-metadata
  resourcedetection:
    detectors: [gcp]
    timeout: 10s

  # The transform/collision processor ensures that any attributes that may
  # collide with the googlemanagedprometheus exporter's monitored resource
  # construction are moved to a similar name that is not reserved.
  transform/collision:
    metric_statements:
    - context: datapoint
      statements:
      - set(attributes["exported_location"], attributes["location"])
      - delete_key(attributes, "location")
      - set(attributes["exported_cluster"], attributes["cluster"])
      - delete_key(attributes, "cluster")
      - set(attributes["exported_namespace"], attributes["namespace"])
      - delete_key(attributes, "namespace")
      - set(attributes["exported_job"], attributes["job"])
      - delete_key(attributes, "job")
      - set(attributes["exported_instance"], attributes["instance"])
      - delete_key(attributes, "instance")
      - set(attributes["exported_project_id"], attributes["project_id"])
      - delete_key(attributes, "project_id")

exporters:
  # The googlecloud exporter will export telemetry to different
  # Google Cloud services:
  # Logs -> Cloud Logging
  # Metrics -> Cloud Monitoring
  # Traces -> Cloud Trace
  #
  # Docs:
  # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/googlecloudexporter
  googlecloud:
    log:
      default_log_name: opentelemetry-collector

  # The googlemanagedprometheus exporter will send metrics to
  # Google Managed Service for Prometheus.
  #
  # Docs:
  # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/googlemanagedprometheusexporter
  googlemanagedprometheus:

extensions:
  # Opens an endpoint on 13133 that can be used to check the
  # status of the collector. Since this does not configure the
  # `path` config value, the endpoint will default to `/`.
  #
  # When running on Cloud Run, this extension is required and not optional.
  # In other environments it is recommended but may not be required for operation
  # (i.e. in Container-Optimized OS or other GCE environments).
  #
  # Docs:
  # https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/extension/healthcheckextension
  health_check:
    endpoint: 0.0.0.0:13133

service:
  extensions:
  - health_check
  pipelines:
    logs:
      receivers:
      - otlp
      processors:
      - resourcedetection
      - memory_limiter
      - batch
      exporters:
      - googlecloud
    metrics/otlp:
      receivers:
      - otlp
      processors:
      - transform/collision
      - resourcedetection
      - memory_limiter
      - batch
      exporters:
      - googlemanagedprometheus
    metrics/self-metrics:
      receivers:
      - prometheus/self-metrics
      processors:
      - resourcedetection
      - memory_limiter
      - batch
      exporters:
      - googlemanagedprometheus
    traces:
      receivers:
      - otlp
      processors:
      - resourcedetection
      - memory_limiter
      - batch
      exporters:
      - googlecloud
  telemetry:
    metrics:
      address: localhost:8888

# tftest-file id=otel-config path=config/otel-config.yaml

module "secrets" {
  source     = "./fabric/modules/secret-manager"
  project_id = var.project_id
  secrets = {
    otel-config = {}
  }
  iam = {
    otel-config = {
      "roles/secretmanager.secretAccessor" = [
        "serviceAccount:${var.project_number}-compute@developer.gserviceaccount.com",
      ]
    }
  }
  versions = {
    otel-config = {
      v1 = { enabled = true, data = file("${path.module}/config/otel-config.yaml") }
    }
  }
}

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
      ports = {
        default = {
          container_port = 3000
        }
      }
      depends_on = ["collector"]
    }
    collector = {
      image = "us-docker.pkg.dev/cloud-ops-agents-artifacts/google-cloud-opentelemetry-collector/otelcol-google:0.122.1"
      startup_probe = {
        http_get = {
          path = "/"
          port = 13133
        }
        timeout_seconds = 30
        period_seconds  = 30
      }
      liveness_probe = {
        http_get = {
          path = "/"
          port = 13133
        }
        timeout_seconds = 30
        period_seconds  = 30
      }
      volume_mounts = {
        "otel-config" = "/etc/otelcol-google/"
      }
    }
  }
  volumes = {
    otel-config = {
      secret = {
        name    = "otel-config"
        version = "1"
        path    = "config.yaml"
      }
    }
  }
  deletion_protection = false
}
# tftest modules=2 resources=4 files=otel-config inventory=service-otel-sidecar.yaml e2e

Eventarc triggers

PubSub

This deploys a Cloud Run service that will be triggered when messages are published to Pub/Sub topics.

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  eventarc_triggers = {
    pubsub = {
      topic-1 = module.pubsub.topic.name
    }
  }
  deletion_protection = false
}
# tftest modules=2 resources=4 fixtures=fixtures/pubsub.tf inventory=service-eventarc-pubsub.yaml e2e

Audit logs

This deploys a Cloud Run service that will be triggered when specific log events are written to Google Cloud audit logs.

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  eventarc_triggers = {
    audit_log = {
      setiampolicy = {
        method  = "SetIamPolicy"
        service = "cloudresourcemanager.googleapis.com"
      }
    }
    service_account_create = true
  }
  deletion_protection = false
}
# tftest modules=1 resources=4 inventory=service-eventarc-auditlogs-sa-create.yaml

Using custom service accounts for triggers

By default Compute default service account is used to trigger Cloud Run. If you want to use custom Service Accounts you can either provide your own in eventarc_triggers.service_account_email or set eventarc_triggers.service_account_create to true and service account named tf-cr-trigger-${var.name} will be created with roles/run.invoker granted on this Cloud Run service.

Example using provided service account:

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  eventarc_triggers = {
    audit_log = {
      setiampolicy = {
        method  = "SetIamPolicy"
        service = "cloudresourcemanager.googleapis.com"
      }
    }
    service_account_email = "cloud-run-trigger@my-project.iam.gserviceaccount.com"
  }
}
# tftest modules=1 resources=2 inventory=service-eventarc-auditlogs-external-sa.yaml

Example using automatically created service account:

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  eventarc_triggers = {
    pubsub = {
      topic-1 = module.pubsub.topic.name
    }
    service_account_create = true
  }
  deletion_protection = false
}
# tftest modules=2 resources=6 fixtures=fixtures/pubsub.tf inventory=service-eventarc-pubsub-sa-create.yaml e2e

Cloud Run Invoker IAM Disable

To disables IAM permission check for run.routes.invoke for callers of this service set the invoker_iam_disabled variable of the module to true (default false). There should be no requirement to pass the roles/run.invoker to the IAM block to enable public access. This allows for the org policy domain restricted sharing org policy remain enabled.

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  invoker_iam_disabled = true
  deletion_protection  = false
}
# tftest modules=1 resources=1 inventory=service-invoker-iam-disable.yaml e2e

Cloud Run Service Account

To use a custom service account managed by the module, set service_account_create to true and leave service_account set to null (default).

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  service_account_create = true
  deletion_protection    = false
}
# tftest modules=1 resources=2 inventory=service-sa-create.yaml e2e

To use an externally managed service account, use its email in service_account and leave service_account_create to false (default).

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  region     = var.region
  name       = "hello"
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
    }
  }
  service_account     = module.iam-service-account.email
  deletion_protection = false
}
# tftest modules=2 resources=2 fixtures=fixtures/iam-service-account.tf inventory=service-external-sa.yaml e2e

Creating Cloud Run Jobs

To create a job instead of service set create_job to true. Jobs support all functions above apart from triggers.

Unsupported variables / attributes:

  • ingress
  • revision.gen2_execution_environment (they run by default in gen2)
  • revision.name
  • containers.liveness_probe
  • containers.startup_probe
  • containers.resources.cpu_idle
  • containers.resources.startup_cpu_boost
module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  name       = "hello"
  region     = var.region
  create_job = true
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
      env = {
        VAR1 = "VALUE1"
        VAR2 = "VALUE2"
      }
    }
  }
  iam = {
    "roles/run.invoker" = ["group:${var.group_email}"]
  }
  deletion_protection = false
}

# tftest modules=1 resources=2 inventory=job-iam-env.yaml e2e

Tag bindings

Tag bindings are not yet supported for jobs. Refer to the Creating and managing tags documentation for details on usage.

module "org" {
  source          = "./fabric/modules/organization"
  organization_id = var.organization_id
  tags = {
    environment = {
      description = "Environment specification."
      values = {
        dev     = {}
        prod    = {}
        sandbox = {}
      }
    }
  }
}

module "cloud_run" {
  source     = "./fabric/modules/cloud-run-v2"
  project_id = var.project_id
  name       = "hello"
  region     = var.region
  containers = {
    hello = {
      image = "us-docker.pkg.dev/cloudrun/container/hello"
      env = {
        VAR1 = "VALUE1"
        VAR2 = "VALUE2"
      }
    }
  }
  iam = {
    "roles/run.invoker" = ["allUsers"]
  }
  tag_bindings = {
    env-sandbox = module.org.tag_values["environment/sandbox"].id
  }
}
# tftest modules=2 resources=7

Variables

name description type required default
name Name used for Cloud Run service. string ✓
project_id Project id used for all resources. string ✓
region Region used for all resources. string ✓
containers Containers in name => attributes format. map(object({…})) {}
create_job Create Cloud Run Job instead of Service. bool false
custom_audiences Custom audiences for service. list(string) null
deletion_protection Deletion protection setting for this Cloud Run service. string null
encryption_key The full resource name of the Cloud KMS CryptoKey. string null
eventarc_triggers Event arc triggers for different sources. object({…}) {}
iam IAM bindings for Cloud Run service in {ROLE => [MEMBERS]} format. map(list(string)) {}
ingress Ingress settings. string null
invoker_iam_disabled Disables IAM permission check for run.routes.invoke for callers of this service. bool false
labels Resource labels. map(string) {}
launch_stage The launch stage as defined by Google Cloud Platform Launch Stages. string null
managed_revision Whether the Terraform module should control the deployment of revisions. bool true
prefix Optional prefix used for resource names. string null
revision Revision template configurations. object({…}) {}
service_account Service account email. Unused if service account is auto-created. string null
service_account_create Auto-create service account. bool false
tag_bindings Tag bindings for this service, in key => tag value id format. map(string) {}
volumes Named volumes in containers in name => attributes format. map(object({…})) {}
vpc_connector_create Populate this to create a Serverless VPC Access connector. object({…}) null

Outputs

name description sensitive
id Fully qualified job or service id.
invoke_command Command to invoke Cloud Run Service / submit job.
job Cloud Run Job.
service Cloud Run Service.
service_account Service account resource.
service_account_email Service account email.
service_account_iam_email Service account email.
service_name Cloud Run service name.
service_uri Main URI in which the service is serving traffic.
vpc_connector VPC connector resource if created.

Fixtures