hunfabric/modules/gke-cluster-autopilot

GKE Autopilot cluster module

This module offers a way to create and manage Google Kubernetes Engine (GKE) Autopilot clusters. With its sensible default settings based on best practices and authors' experience as Google Cloud practitioners, the module accommodates for many common use cases out-of-the-box, without having to rely on verbose configuration.

• GKE Autopilot cluster
• Cloud DNS
• Logging configuration
• Monitoring configuration
• Backup for GKE
  • Allowing access from Google Cloud services
  • Disable PSC endpoint creation
• Upgrade notifications
• Variables
• Outputs

For an explanation of cluster access configurations, please refer to the GKE cluster standard module.

GKE Autopilot cluster

This example shows how to create a GKE cluster in Autopilot mode.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = "myproject"
  name       = "cluster-1"
  location   = "europe-west1"
  access_config = {
    ip_access = {
      authorized_ranges = {
        internal-vms = "10.0.0.0/8"
      }
    }
  }
  vpc_config = {
    network    = var.vpc.self_link
    subnetwork = var.subnet.self_link
    secondary_range_names = {
      pods     = "pods"
      services = "services"
    }
  }
  labels = {
    environment = "dev"
  }
}
# tftest modules=1 resources=1 inventory=basic.yaml

Cloud DNS

Warning

Cloud DNS is the only DNS provider for Autopilot clusters running version 1.25.9-gke.400 and later, and version 1.26.4-gke.500 and later. It is pre-configured for those clusters. The following example only applies to Autopilot clusters running earlier versions.

This example shows how to use Cloud DNS as a Kubernetes DNS provider.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {} # use default names "pods" and "services"
  }
  enable_features = {
    dns = {
      provider = "CLOUD_DNS"
      scope    = "CLUSTER_SCOPE"
      domain   = "gke.local"
    }
  }
}
# tftest modules=1 resources=1 inventory=dns.yaml

Logging configuration

Note

System and workload logs collection is pre-configured for Autopilot clusters and cannot be disabled.

This example shows how to collect logs for the Kubernetes control plane components. The logs for these components are not collected by default.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {} # use default names "pods" and "services"
  }
  logging_config = {
    enable_api_server_logs         = true
    enable_scheduler_logs          = true
    enable_controller_manager_logs = true
  }
}
# tftest modules=1 resources=1 inventory=logging-config.yaml

Monitoring configuration

Note

System metrics collection is pre-configured for Autopilot clusters and cannot be disabled.

Warning

GKE workload metrics is deprecated and removed in GKE 1.24 and later. Workload metrics is replaced by Google Cloud Managed Service for Prometheus, which is Google's recommended way to monitor Kubernetes applications by using Cloud Monitoring.

This example shows how to configure collection of Kubernetes control plane metrics. These metrics are optional and are not collected by default.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {} # use default names "pods" and "services"
  }
  monitoring_config = {
    enable_api_server_metrics         = true
    enable_controller_manager_metrics = true
    enable_scheduler_metrics          = true
  }
}
# tftest modules=1 resources=1 inventory=monitoring-config-control-plane.yaml

The next example shows how to configure collection of kube state metrics. These metrics are optional and are not collected by default.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {} # use default names "pods" and "services"
  }
  monitoring_config = {
    enable_cadvisor_metrics    = true
    enable_daemonset_metrics   = true
    enable_deployment_metrics  = true
    enable_hpa_metrics         = true
    enable_pod_metrics         = true
    enable_statefulset_metrics = true
    enable_storage_metrics     = true
    # Kube state metrics collection requires Google Cloud Managed Service for Prometheus,
    # which is enabled by default.
    # enable_managed_prometheus = true
  }
}
# tftest modules=1 resources=1 inventory=monitoring-config-kube-state.yaml

The control plane metrics and kube state metrics collection can be configured in a single monitoring_config block.

Backup for GKE

Note

Although Backup for GKE can be enabled as an add-on when configuring your GKE clusters, it is a separate service from GKE.

Backup for GKE is a service for backing up and restoring workloads in GKE clusters. It has two components:

• A Google Cloud API that serves as the control plane for the service.
• A GKE add-on (the Backup for GKE agent) that must be enabled in each cluster for which you wish to perform backup and restore operations.

Backup for GKE is supported in GKE Autopilot clusters with some restrictions.

This example shows how to enable Backup for GKE on a new Autopilot cluster and plan a set of backups.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {}
  }
  backup_configs = {
    enable_backup_agent = true
    backup_plans = {
      "backup-1" = {
        region   = "europe-west-2"
        schedule = "0 9 * * 1"
      }
    }
  }
}
# tftest modules=1 resources=2 inventory=backup.yaml

Allowing access from Google Cloud services

To allow access to your cluster from Google Cloud services (like Cloud Shell, Cloud Build, etc.) without needing to manually specify all Google Cloud IP ranges, you can use the gcp_public_cidrs_access_enabled parameter:

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = "myproject"
  name       = "cluster-1"
  location   = "europe-west1"
  access_config = {
    ip_access = {
      gcp_public_cidrs_access_enabled = true
      authorized_ranges = {
        internal-vms = "10.0.0.0/8"
      }
    }
  }
  vpc_config = {
    network    = var.vpc.self_link
    subnetwork = var.subnet.self_link
    secondary_range_names = {
      pods     = "pods"
      services = "services"
    }
  }
  labels = {
    environment = "dev"
  }
}
# tftest modules=1 resources=1 inventory=access-google.yaml

Disable PSC endpoint creation

To disable IP access to the GKE control plane and prevent PSC endpoint creation, set var.access_config.ip_access to null or omit the variable.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = "myproject"
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network    = var.vpc.self_link
    subnetwork = var.subnet.self_link
    secondary_range_names = {
      pods     = "pods"
      services = "services"
    }
  }
  labels = {
    environment = "dev"
  }
}
# tftest modules=1 resources=1 inventory=no-ip-access.yaml

Upgrade notifications

Upgrade notifications are configured via the enable_features.upgrade_notifications. An existing PubSub topic can be defined via its topic attribute, or a new one can be created if the attribute is not set. The event_types attribute can be used to control which event types are sent. The kms_key_name attribute can be used to control which KMS key is used to encrypt the notification messages.

module "cluster-1" {
  source     = "./fabric/modules/gke-cluster-autopilot"
  project_id = var.project_id
  name       = "cluster-1"
  location   = "europe-west1"
  vpc_config = {
    network               = var.vpc.self_link
    subnetwork            = var.subnet.self_link
    secondary_range_names = {}
  }
  enable_features = {
    upgrade_notifications = {
      event_types  = ["SECURITY_BULLETIN_EVENT", "UPGRADE_EVENT"]
      kms_key_name = "projects/myproject/locations/global/keyRings/mykeyring/cryptoKeys/mykey"
    }
  }
}
# tftest modules=1 resources=2 inventory=notifications.yaml

Variables

name	description	type	required	default
location	Autopilot clusters are always regional.	string	✓
name	Cluster name.	string	✓
project_id	Cluster project ID.	string	✓
vpc_config	VPC-level configuration.	object({…})	✓
access_config	Control plane endpoint and nodes access configurations.	object({…})		{}
backup_configs	Configuration for Backup for GKE.	object({…})		{}
deletion_protection	Whether or not to allow Terraform to destroy the cluster. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the cluster will fail.	bool		true
description	Cluster description.	string		null
enable_addons	Addons enabled in the cluster (true means enabled).	object({…})		{}
enable_features	Enable cluster-level features. Certain features allow configuration.	object({…})		{}
fleet_project	The name of the fleet host project where this cluster will be registered.	string		null
issue_client_certificate	Enable issuing client certificate.	bool		false
labels	Cluster resource labels.	map(string)		null
logging_config	Logging configuration.	object({…})		{}
maintenance_config	Maintenance window configuration.	object({…})		{…}
min_master_version	Minimum version of the master, defaults to the version of the most recent official release.	string		null
monitoring_config	Monitoring configuration. System metrics collection cannot be disabled. Control plane metrics are optional. Kube state metrics are optional. Google Cloud Managed Service for Prometheus is enabled by default.	object({…})		{}
node_config	Configuration for nodes and nodepools.	object({…})		{}
node_locations	Zones in which the cluster's nodes are located.	list(string)		[]
release_channel	Release channel for GKE upgrades. Clusters created in the Autopilot mode must use a release channel. Choose between "RAPID", "REGULAR", and "STABLE".	string		"REGULAR"

Outputs

name	description	sensitive
ca_certificate	Public certificate of the cluster (base64-encoded).	✓
cluster	Cluster resource.	✓
dns_endpoint	Control plane DNS endpoint.
endpoint	Cluster endpoint.
id	Fully qualified cluster ID.
location	Cluster location.
master_version	Master version.
name	Cluster name.
notifications	GKE Pub/Sub notifications topic.
self_link	Cluster self link.	✓
workload_identity_pool	Workload identity pool.