Refactor GKE nodepool and blueprints (#875)
* first shot, untested * example tests working * module tests * work on gke blueprints * multitenant fleet doc examples * fix gke hub doc examples * blueprint tests * move master range to vpc config * fast stage 3 gke test * tfdoc * bump provider versions * and bump provider again to latest
This commit is contained in:
Ludovico Magnocavallo
committed by
GitHub
GitHub
parent
bb4aacd152
commit
e8056577ce
@@ -6,30 +6,31 @@ This module allows simplified creation and management of individual GKE nodepool
|
||||
|
||||
### Module defaults
|
||||
|
||||
If no specific node configuration is set via variables, the module uses the provider's defaults only setting OAuth scopes to a minimal working set (devstorage read-only, logging and monitoring write) and the node machine type to `n1-standard-1`. The service account set by the provider in this case is the GCE default service account.
|
||||
If no specific node configuration is set via variables, the module uses the provider's defaults only setting OAuth scopes to a minimal working set and the node machine type to `n1-standard-1`. The service account set by the provider in this case is the GCE default service account.
|
||||
|
||||
```hcl
|
||||
module "cluster-1-nodepool-1" {
|
||||
source = "./fabric/modules/gke-nodepool"
|
||||
project_id = "myproject"
|
||||
cluster_name = "cluster-1"
|
||||
location = "europe-west1-b"
|
||||
name = "nodepool-1"
|
||||
source = "./fabric/modules/gke-nodepool"
|
||||
project_id = "myproject"
|
||||
cluster_name = "cluster-1"
|
||||
location = "europe-west1-b"
|
||||
name = "nodepool-1"
|
||||
}
|
||||
# tftest modules=1 resources=1
|
||||
```
|
||||
|
||||
### Internally managed service account
|
||||
|
||||
To have the module auto-create a service account for the nodes, set the `node_service_account_create` variable to `true`. When a service account is created by the module, OAuth scopes are set to `cloud-platform` by default. The service account resource and email (in both plain and IAM formats) are then available in outputs to assign IAM roles from your own code.
|
||||
To have the module auto-create a service account for the nodes, define the `service_account` variable without setting its `email` attribute. You can then specify service account scopes, or use the default. The service account resource and email (in both plain and IAM formats) are then available in outputs to assign IAM roles from your own code.
|
||||
|
||||
```hcl
|
||||
module "cluster-1-nodepool-1" {
|
||||
source = "./fabric/modules/gke-nodepool"
|
||||
project_id = "myproject"
|
||||
cluster_name = "cluster-1"
|
||||
location = "europe-west1-b"
|
||||
name = "nodepool-1"
|
||||
node_service_account_create = true
|
||||
source = "./fabric/modules/gke-nodepool"
|
||||
project_id = "myproject"
|
||||
cluster_name = "cluster-1"
|
||||
location = "europe-west1-b"
|
||||
name = "nodepool-1"
|
||||
service_account = {}
|
||||
}
|
||||
# tftest modules=1 resources=2
|
||||
```
|
||||
@@ -39,48 +40,30 @@ module "cluster-1-nodepool-1" {
|
||||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---:|:---:|:---:|
|
||||
| [cluster_name](variables.tf#L26) | Cluster name. | <code>string</code> | ✓ | |
|
||||
| [location](variables.tf#L59) | Cluster location. | <code>string</code> | ✓ | |
|
||||
| [project_id](variables.tf#L216) | Cluster project id. | <code>string</code> | ✓ | |
|
||||
| [autoscaling_config](variables.tf#L17) | Optional autoscaling configuration. | <code title="object({ min_node_count = number max_node_count = number })">object({…})</code> | | <code>null</code> |
|
||||
| [gke_version](variables.tf#L31) | Kubernetes nodes version. Ignored if auto_upgrade is set in management_config. | <code>string</code> | | <code>null</code> |
|
||||
| [initial_node_count](variables.tf#L37) | Initial number of nodes for the pool. | <code>number</code> | | <code>1</code> |
|
||||
| [kubelet_config](variables.tf#L43) | Kubelet configuration. | <code title="object({ cpu_cfs_quota = string cpu_cfs_quota_period = string cpu_manager_policy = string })">object({…})</code> | | <code>null</code> |
|
||||
| [linux_node_config_sysctls](variables.tf#L53) | Linux node configuration. | <code>map(string)</code> | | <code>null</code> |
|
||||
| [management_config](variables.tf#L64) | Optional node management configuration. | <code title="object({ auto_repair = bool auto_upgrade = bool })">object({…})</code> | | <code>null</code> |
|
||||
| [max_pods_per_node](variables.tf#L73) | Maximum number of pods per node. | <code>number</code> | | <code>null</code> |
|
||||
| [name](variables.tf#L79) | Optional nodepool name. | <code>string</code> | | <code>null</code> |
|
||||
| [node_boot_disk_kms_key](variables.tf#L85) | Customer Managed Encryption Key used to encrypt the boot disk attached to each node. | <code>string</code> | | <code>null</code> |
|
||||
| [node_count](variables.tf#L91) | Number of nodes per instance group, can be updated after creation. Ignored when autoscaling is set. | <code>number</code> | | <code>null</code> |
|
||||
| [node_disk_size](variables.tf#L97) | Node disk size, defaults to 100GB. | <code>number</code> | | <code>100</code> |
|
||||
| [node_disk_type](variables.tf#L103) | Node disk type, defaults to pd-standard. | <code>string</code> | | <code>"pd-standard"</code> |
|
||||
| [node_guest_accelerator](variables.tf#L109) | Map of type and count of attached accelerator cards. | <code>map(number)</code> | | <code>{}</code> |
|
||||
| [node_image_type](variables.tf#L115) | Nodes image type. | <code>string</code> | | <code>null</code> |
|
||||
| [node_labels](variables.tf#L121) | Kubernetes labels attached to nodes. | <code>map(string)</code> | | <code>{}</code> |
|
||||
| [node_local_ssd_count](variables.tf#L127) | Number of local SSDs attached to nodes. | <code>number</code> | | <code>0</code> |
|
||||
| [node_locations](variables.tf#L132) | Optional list of zones in which nodes should be located. Uses cluster locations if unset. | <code>list(string)</code> | | <code>null</code> |
|
||||
| [node_machine_type](variables.tf#L138) | Nodes machine type. | <code>string</code> | | <code>"n1-standard-1"</code> |
|
||||
| [node_metadata](variables.tf#L144) | Metadata key/value pairs assigned to nodes. Set disable-legacy-endpoints to true when using this variable. | <code>map(string)</code> | | <code>null</code> |
|
||||
| [node_min_cpu_platform](variables.tf#L150) | Minimum CPU platform for nodes. | <code>string</code> | | <code>null</code> |
|
||||
| [node_preemptible](variables.tf#L156) | Use preemptible VMs for nodes. | <code>bool</code> | | <code>null</code> |
|
||||
| [node_sandbox_config](variables.tf#L162) | GKE Sandbox configuration. Needs image_type set to COS_CONTAINERD and node_version set to 1.12.7-gke.17 when using this variable. | <code>string</code> | | <code>null</code> |
|
||||
| [node_service_account](variables.tf#L168) | Service account email. Unused if service account is auto-created. | <code>string</code> | | <code>null</code> |
|
||||
| [node_service_account_create](variables.tf#L174) | Auto-create service account. | <code>bool</code> | | <code>false</code> |
|
||||
| [node_service_account_scopes](variables.tf#L182) | Scopes applied to service account. Default to: 'cloud-platform' when creating a service account; 'devstorage.read_only', 'logging.write', 'monitoring.write' otherwise. | <code>list(string)</code> | | <code>[]</code> |
|
||||
| [node_shielded_instance_config](variables.tf#L188) | Shielded instance options. | <code title="object({ enable_secure_boot = bool enable_integrity_monitoring = bool })">object({…})</code> | | <code>null</code> |
|
||||
| [node_spot](variables.tf#L197) | Use Spot VMs for nodes. | <code>bool</code> | | <code>null</code> |
|
||||
| [node_tags](variables.tf#L203) | Network tags applied to nodes. | <code>list(string)</code> | | <code>null</code> |
|
||||
| [node_taints](variables.tf#L209) | Kubernetes taints applied to nodes. E.g. type=blue:NoSchedule. | <code>list(string)</code> | | <code>[]</code> |
|
||||
| [upgrade_config](variables.tf#L221) | Optional node upgrade configuration. | <code title="object({ max_surge = number max_unavailable = number })">object({…})</code> | | <code>null</code> |
|
||||
| [workload_metadata_config](variables.tf#L230) | Metadata configuration to expose to workloads on the node pool. | <code>string</code> | | <code>"GKE_METADATA"</code> |
|
||||
| [cluster_name](variables.tf#L17) | Cluster name. | <code>string</code> | ✓ | |
|
||||
| [location](variables.tf#L35) | Cluster location. | <code>string</code> | ✓ | |
|
||||
| [project_id](variables.tf#L143) | Cluster project id. | <code>string</code> | ✓ | |
|
||||
| [gke_version](variables.tf#L22) | Kubernetes nodes version. Ignored if auto_upgrade is set in management_config. | <code>string</code> | | <code>null</code> |
|
||||
| [labels](variables.tf#L28) | Kubernetes labels applied to each node. | <code>map(string)</code> | | <code>{}</code> |
|
||||
| [max_pods_per_node](variables.tf#L40) | Maximum number of pods per node. | <code>number</code> | | <code>null</code> |
|
||||
| [name](variables.tf#L46) | Optional nodepool name. | <code>string</code> | | <code>null</code> |
|
||||
| [node_config](variables.tf#L52) | Node-level configuration. | <code title="object({ boot_disk_kms_key = optional(string) disk_size_gb = optional(number) disk_type = optional(string) ephemeral_ssd_count = optional(number) gcfs = optional(bool, false) guest_accelerator = optional(object({ count = number type = string gpu_partition_size = optional(string) })) gvnic = optional(bool, false) image_type = optional(string) kubelet_config = optional(object({ cpu_manager_policy = string cpu_cfs_quota = optional(bool) cpu_cfs_quota_period = optional(string) })) linux_node_config_sysctls = optional(map(string)) local_ssd_count = optional(number) machine_type = optional(string) metadata = optional(map(string)) min_cpu_platform = optional(string) preemptible = optional(bool) sandbox_config_gvisor = optional(bool) shielded_instance_config = optional(object({ enable_integrity_monitoring = optional(bool) enable_secure_boot = optional(bool) })) spot = optional(bool) workload_metadata_config_mode = optional(string) })">object({…})</code> | | <code title="{ disk_type = "pd-balanced" }">{…}</code> |
|
||||
| [node_count](variables.tf#L91) | Number of nodes per instance group. Initial value can only be changed by recreation, current is ignored when autoscaling is used. | <code title="object({ current = optional(number) initial = number })">object({…})</code> | | <code title="{ initial = 1 }">{…}</code> |
|
||||
| [node_locations](variables.tf#L103) | Node locations. | <code>list(string)</code> | | <code>null</code> |
|
||||
| [nodepool_config](variables.tf#L109) | Nodepool-level configuration. | <code title="object({ autoscaling = optional(object({ location_policy = optional(string) max_node_count = optional(number) min_node_count = optional(number) use_total_nodes = optional(bool, false) })) management = optional(object({ auto_repair = optional(bool) auto_upgrade = optional(bool) })) upgrade_settings = optional(object({ max_surge = number max_unavailable = number })) })">object({…})</code> | | <code>null</code> |
|
||||
| [pod_range](variables.tf#L131) | Pod secondary range configuration. | <code title="object({ secondary_pod_range = object({ cidr = optional(string) create = optional(bool) name = string }) })">object({…})</code> | | <code>null</code> |
|
||||
| [reservation_affinity](variables.tf#L148) | Configuration of the desired reservation which instances could take capacity from. | <code title="object({ consume_reservation_type = string key = optional(string) values = optional(list(string)) })">object({…})</code> | | <code>null</code> |
|
||||
| [service_account](variables.tf#L158) | Nodepool service account. If this variable is set to null, the default GCE service account will be used. If set and email is null, a service account will be created. If scopes are null a default will be used. | <code title="object({ email = optional(string) oauth_scopes = optional(list(string)) })">object({…})</code> | | <code>null</code> |
|
||||
| [sole_tenant_nodegroup](variables.tf#L167) | Sole tenant node group. | <code>string</code> | | <code>null</code> |
|
||||
| [tags](variables.tf#L173) | Network tags applied to nodes. | <code>list(string)</code> | | <code>null</code> |
|
||||
| [taints](variables.tf#L179) | Kubernetes taints applied to all nodes. | <code title="list(object({ key = string value = string effect = string }))">list(object({…}))</code> | | <code>null</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive |
|
||||
|---|---|:---:|
|
||||
| [name](outputs.tf#L17) | Nodepool name. | |
|
||||
| [service_account](outputs.tf#L22) | Service account resource. | |
|
||||
| [service_account_email](outputs.tf#L31) | Service account email. | |
|
||||
| [service_account_iam_email](outputs.tf#L36) | Service account email. | |
|
||||
| [service_account_email](outputs.tf#L22) | Service account email. | |
|
||||
| [service_account_iam_email](outputs.tf#L27) | Service account email. | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
||||
@@ -15,179 +15,209 @@
|
||||
*/
|
||||
|
||||
locals {
|
||||
service_account_email = (
|
||||
var.node_service_account_create
|
||||
? (
|
||||
length(google_service_account.service_account) > 0
|
||||
? google_service_account.service_account[0].email
|
||||
: null
|
||||
_image = coalesce(var.node_config.image_type, "-")
|
||||
image = {
|
||||
is_cos = length(regexall("COS", local._image)) > 0
|
||||
is_cos_containerd = (
|
||||
var.node_config.image_type == null
|
||||
||
|
||||
length(regexall("COS_CONTAINERD", local._image)) > 0
|
||||
)
|
||||
: var.node_service_account
|
||||
is_win = length(regexall("WIN", local._image)) > 0
|
||||
}
|
||||
node_metadata = var.node_config.metadata == null ? null : merge(
|
||||
var.node_config.metadata,
|
||||
{ disable-legacy-endpoints = "true" }
|
||||
)
|
||||
# if no attributes passed for service account, use the GCE default
|
||||
# if no email specified, create service account
|
||||
service_account_create = (
|
||||
var.service_account != null && try(var.service_account.email, null) == null
|
||||
)
|
||||
service_account_email = (
|
||||
local.service_account_create
|
||||
? google_service_account.service_account[0].email
|
||||
: try(var.service_account.email, null)
|
||||
)
|
||||
service_account_scopes = (
|
||||
length(var.node_service_account_scopes) > 0
|
||||
? var.node_service_account_scopes
|
||||
: (
|
||||
var.node_service_account_create
|
||||
? ["https://www.googleapis.com/auth/cloud-platform"]
|
||||
: [
|
||||
"https://www.googleapis.com/auth/devstorage.read_only",
|
||||
"https://www.googleapis.com/auth/logging.write",
|
||||
"https://www.googleapis.com/auth/monitoring",
|
||||
"https://www.googleapis.com/auth/monitoring.write"
|
||||
]
|
||||
)
|
||||
)
|
||||
node_taint_effect = {
|
||||
"NoExecute" = "NO_EXECUTE",
|
||||
"NoSchedule" = "NO_SCHEDULE"
|
||||
"PreferNoSchedule" = "PREFER_NO_SCHEDULE"
|
||||
}
|
||||
temp_node_pools_taints = [
|
||||
for taint in var.node_taints :
|
||||
{
|
||||
"key" = element(split("=", taint), 0),
|
||||
"value" = element(split(":", element(split("=", taint), 1)), 0),
|
||||
"effect" = lookup(local.node_taint_effect, element(split(":", taint), 1)),
|
||||
}
|
||||
]
|
||||
# The taint is added to match the one that
|
||||
# GKE implicitly adds when Windows node pools are created.
|
||||
win_node_pools_taint = (
|
||||
var.node_image_type == null
|
||||
? []
|
||||
: length(regexall("WINDOWS", var.node_image_type)) > 0
|
||||
? [
|
||||
{
|
||||
"key" = "node.kubernetes.io/os"
|
||||
"value" = "windows"
|
||||
"effect" = local.node_taint_effect.NoSchedule
|
||||
}
|
||||
try(var.service_account.scopes, null) != null
|
||||
? var.service_account.scopes
|
||||
: [
|
||||
"https://www.googleapis.com/auth/devstorage.read_only",
|
||||
"https://www.googleapis.com/auth/logging.write",
|
||||
"https://www.googleapis.com/auth/monitoring",
|
||||
"https://www.googleapis.com/auth/monitoring.write",
|
||||
"https://www.googleapis.com/auth/userinfo.email"
|
||||
]
|
||||
)
|
||||
taints_windows = (
|
||||
local.image.is_win
|
||||
? [{
|
||||
key = "node.kubernetes.io/os", value = "windows", effect = "NO_EXECUTE"
|
||||
}]
|
||||
: []
|
||||
)
|
||||
node_taints = concat(local.temp_node_pools_taints, local.win_node_pools_taint)
|
||||
}
|
||||
|
||||
resource "google_service_account" "service_account" {
|
||||
count = var.node_service_account_create ? 1 : 0
|
||||
count = local.service_account_create ? 1 : 0
|
||||
project = var.project_id
|
||||
account_id = "tf-gke-${var.name}"
|
||||
display_name = "Terraform GKE ${var.cluster_name} ${var.name}."
|
||||
}
|
||||
|
||||
resource "google_container_node_pool" "nodepool" {
|
||||
provider = google-beta
|
||||
|
||||
project = var.project_id
|
||||
cluster = var.cluster_name
|
||||
location = var.location
|
||||
name = var.name
|
||||
|
||||
initial_node_count = var.node_count == null ? var.initial_node_count : null // (dmarzi) TOFIX
|
||||
max_pods_per_node = var.max_pods_per_node
|
||||
node_count = var.autoscaling_config == null ? var.node_count : null
|
||||
node_locations = var.node_locations
|
||||
provider = google-beta
|
||||
project = var.project_id
|
||||
cluster = var.cluster_name
|
||||
location = var.location
|
||||
name = var.name
|
||||
version = var.gke_version
|
||||
|
||||
node_config {
|
||||
disk_size_gb = var.node_disk_size
|
||||
disk_type = var.node_disk_type
|
||||
image_type = var.node_image_type
|
||||
labels = var.node_labels
|
||||
taint = local.node_taints
|
||||
local_ssd_count = var.node_local_ssd_count
|
||||
machine_type = var.node_machine_type
|
||||
metadata = var.node_metadata
|
||||
min_cpu_platform = var.node_min_cpu_platform
|
||||
oauth_scopes = local.service_account_scopes
|
||||
preemptible = var.node_preemptible
|
||||
service_account = local.service_account_email
|
||||
tags = var.node_tags
|
||||
boot_disk_kms_key = var.node_boot_disk_kms_key
|
||||
spot = var.node_spot
|
||||
|
||||
dynamic "guest_accelerator" {
|
||||
for_each = var.node_guest_accelerator
|
||||
iterator = config
|
||||
content {
|
||||
type = config.key
|
||||
count = config.value
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "sandbox_config" {
|
||||
for_each = (
|
||||
var.node_sandbox_config != null
|
||||
? [var.node_sandbox_config]
|
||||
: []
|
||||
)
|
||||
iterator = config
|
||||
content {
|
||||
sandbox_type = config.value
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "shielded_instance_config" {
|
||||
for_each = (
|
||||
var.node_shielded_instance_config != null
|
||||
? [var.node_shielded_instance_config]
|
||||
: []
|
||||
)
|
||||
iterator = config
|
||||
content {
|
||||
enable_secure_boot = config.value.enable_secure_boot
|
||||
enable_integrity_monitoring = config.value.enable_integrity_monitoring
|
||||
}
|
||||
}
|
||||
|
||||
workload_metadata_config {
|
||||
mode = var.workload_metadata_config
|
||||
}
|
||||
|
||||
dynamic "kubelet_config" {
|
||||
for_each = var.kubelet_config != null ? [var.kubelet_config] : []
|
||||
iterator = config
|
||||
content {
|
||||
cpu_manager_policy = config.value.cpu_manager_policy
|
||||
cpu_cfs_quota = config.value.cpu_cfs_quota
|
||||
cpu_cfs_quota_period = config.value.cpu_cfs_quota_period
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "linux_node_config" {
|
||||
for_each = var.linux_node_config_sysctls != null ? [var.linux_node_config_sysctls] : []
|
||||
iterator = config
|
||||
content {
|
||||
sysctls = config.value
|
||||
}
|
||||
}
|
||||
}
|
||||
max_pods_per_node = var.max_pods_per_node
|
||||
initial_node_count = var.node_count.initial
|
||||
node_count = var.node_count.current
|
||||
node_locations = var.node_locations
|
||||
# placement_policy = var.nodepool_config.placement_policy
|
||||
|
||||
dynamic "autoscaling" {
|
||||
for_each = var.autoscaling_config != null ? [var.autoscaling_config] : []
|
||||
iterator = config
|
||||
for_each = (
|
||||
try(var.nodepool_config.autoscaling.use_total_nodes, false) ? [] : [""]
|
||||
)
|
||||
content {
|
||||
min_node_count = config.value.min_node_count
|
||||
max_node_count = config.value.max_node_count
|
||||
location_policy = try(var.nodepool_config.autoscaling.location_policy, null)
|
||||
max_node_count = try(var.nodepool_config.autoscaling.max_node_count, null)
|
||||
min_node_count = try(var.nodepool_config.autoscaling.min_node_count, null)
|
||||
}
|
||||
}
|
||||
dynamic "autoscaling" {
|
||||
for_each = (
|
||||
try(var.nodepool_config.autoscaling.use_total_nodes, false) ? [""] : []
|
||||
)
|
||||
content {
|
||||
location_policy = try(var.nodepool_config.autoscaling.location_policy, null)
|
||||
max_node_count = try(var.nodepool_config.autoscaling.max_node_count, null)
|
||||
min_node_count = try(var.nodepool_config.autoscaling.min_node_count, null)
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "management" {
|
||||
for_each = var.management_config != null ? [var.management_config] : []
|
||||
iterator = config
|
||||
for_each = try(var.nodepool_config.management, null) != null ? [""] : []
|
||||
content {
|
||||
auto_repair = config.value.auto_repair
|
||||
auto_upgrade = config.value.auto_upgrade
|
||||
auto_repair = try(var.nodepool_config.management.auto_repair, null)
|
||||
auto_upgrade = try(var.nodepool_config.management.auto_upgrade, null)
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "network_config" {
|
||||
for_each = var.pod_range != null ? [""] : []
|
||||
content {
|
||||
create_pod_range = var.pod_range.create
|
||||
pod_ipv4_cidr_block = var.pod_range.cidr
|
||||
pod_range = var.pod_range.name
|
||||
}
|
||||
}
|
||||
|
||||
dynamic "upgrade_settings" {
|
||||
for_each = var.upgrade_config != null ? [var.upgrade_config] : []
|
||||
iterator = config
|
||||
for_each = try(var.nodepool_config.upgrade_settings, null) != null ? [""] : []
|
||||
content {
|
||||
max_surge = config.value.max_surge
|
||||
max_unavailable = config.value.max_unavailable
|
||||
max_surge = try(var.nodepool_config.upgrade_settings.max_surge, null)
|
||||
max_unavailable = try(var.nodepool_config.upgrade_settings.max_unavailable, null)
|
||||
}
|
||||
}
|
||||
|
||||
node_config {
|
||||
boot_disk_kms_key = var.node_config.boot_disk_kms_key
|
||||
disk_size_gb = var.node_config.disk_size_gb
|
||||
disk_type = var.node_config.disk_type
|
||||
image_type = var.node_config.image_type
|
||||
labels = var.labels
|
||||
local_ssd_count = var.node_config.local_ssd_count
|
||||
machine_type = var.node_config.machine_type
|
||||
metadata = local.node_metadata
|
||||
min_cpu_platform = var.node_config.min_cpu_platform
|
||||
node_group = var.sole_tenant_nodegroup
|
||||
oauth_scopes = local.service_account_scopes
|
||||
preemptible = var.node_config.preemptible
|
||||
service_account = local.service_account_email
|
||||
spot = (
|
||||
var.node_config.spot == true && var.node_config.preemptible != true
|
||||
)
|
||||
tags = var.tags
|
||||
taint = (
|
||||
var.taints == null ? [] : concat(var.taints, local.taints_windows)
|
||||
)
|
||||
|
||||
dynamic "ephemeral_storage_config" {
|
||||
for_each = var.node_config.ephemeral_ssd_count != null ? [""] : []
|
||||
content {
|
||||
local_ssd_count = var.node_config.ephemeral_ssd_count
|
||||
}
|
||||
}
|
||||
dynamic "gcfs_config" {
|
||||
for_each = var.node_config.gcfs && local.image.is_cos_containerd ? [""] : []
|
||||
content {
|
||||
enabled = true
|
||||
}
|
||||
}
|
||||
dynamic "guest_accelerator" {
|
||||
for_each = var.node_config.guest_accelerator != null ? [""] : []
|
||||
content {
|
||||
count = var.node_config.guest_accelerator.count
|
||||
type = var.node_config.guest_accelerator.type
|
||||
gpu_partition_size = var.node_config.guest_accelerator.gpu_partition_size
|
||||
}
|
||||
}
|
||||
dynamic "gvnic" {
|
||||
for_each = var.node_config.gvnic && local.image.is_cos ? [""] : []
|
||||
content {
|
||||
enabled = true
|
||||
}
|
||||
}
|
||||
dynamic "kubelet_config" {
|
||||
for_each = var.node_config.kubelet_config != null ? [""] : []
|
||||
content {
|
||||
cpu_manager_policy = var.node_config.kubelet_config.cpu_manager_policy
|
||||
cpu_cfs_quota = var.node_config.kubelet_config.cpu_cfs_quota
|
||||
cpu_cfs_quota_period = var.node_config.kubelet_config.cpu_cfs_quota_period
|
||||
}
|
||||
}
|
||||
dynamic "linux_node_config" {
|
||||
for_each = var.node_config.linux_node_config_sysctls != null ? [""] : []
|
||||
content {
|
||||
sysctls = var.node_config.linux_node_config_sysctls
|
||||
}
|
||||
}
|
||||
dynamic "reservation_affinity" {
|
||||
for_each = var.reservation_affinity != null ? [""] : []
|
||||
content {
|
||||
consume_reservation_type = var.reservation_affinity.consume_reservation_type
|
||||
key = var.reservation_affinity.key
|
||||
values = var.reservation_affinity.values
|
||||
}
|
||||
}
|
||||
dynamic "sandbox_config" {
|
||||
for_each = (
|
||||
var.node_config.sandbox_config_gvisor == true &&
|
||||
local.image.is_cos_containerd != null
|
||||
? [""]
|
||||
: []
|
||||
)
|
||||
content {
|
||||
sandbox_type = "gvisor"
|
||||
}
|
||||
}
|
||||
dynamic "shielded_instance_config" {
|
||||
for_each = var.node_config.shielded_instance_config != null ? [""] : []
|
||||
content {
|
||||
enable_secure_boot = var.node_config.shielded_instance_config.enable_secure_boot
|
||||
enable_integrity_monitoring = var.node_config.shielded_instance_config.enable_integrity_monitoring
|
||||
}
|
||||
}
|
||||
dynamic "workload_metadata_config" {
|
||||
for_each = var.node_config.workload_metadata_config_mode != null ? [""] : []
|
||||
content {
|
||||
mode = var.node_config.workload_metadata_config_mode
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,15 +19,6 @@ output "name" {
|
||||
value = google_container_node_pool.nodepool.name
|
||||
}
|
||||
|
||||
output "service_account" {
|
||||
description = "Service account resource."
|
||||
value = (
|
||||
var.node_service_account_create
|
||||
? google_service_account.service_account[0]
|
||||
: null
|
||||
)
|
||||
}
|
||||
|
||||
output "service_account_email" {
|
||||
description = "Service account email."
|
||||
value = local.service_account_email
|
||||
@@ -35,8 +26,8 @@ output "service_account_email" {
|
||||
|
||||
output "service_account_iam_email" {
|
||||
description = "Service account email."
|
||||
value = join("", [
|
||||
"serviceAccount:",
|
||||
value = format(
|
||||
"serviceAccount:%s",
|
||||
local.service_account_email == null ? "" : local.service_account_email
|
||||
])
|
||||
)
|
||||
}
|
||||
|
||||
@@ -14,15 +14,6 @@
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
variable "autoscaling_config" {
|
||||
description = "Optional autoscaling configuration."
|
||||
type = object({
|
||||
min_node_count = number
|
||||
max_node_count = number
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "cluster_name" {
|
||||
description = "Cluster name."
|
||||
type = string
|
||||
@@ -34,26 +25,11 @@ variable "gke_version" {
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "initial_node_count" {
|
||||
description = "Initial number of nodes for the pool."
|
||||
type = number
|
||||
default = 1
|
||||
}
|
||||
|
||||
variable "kubelet_config" {
|
||||
description = "Kubelet configuration."
|
||||
type = object({
|
||||
cpu_cfs_quota = string
|
||||
cpu_cfs_quota_period = string
|
||||
cpu_manager_policy = string
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "linux_node_config_sysctls" {
|
||||
description = "Linux node configuration."
|
||||
variable "labels" {
|
||||
description = "Kubernetes labels applied to each node."
|
||||
type = map(string)
|
||||
default = null
|
||||
default = {}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "location" {
|
||||
@@ -61,15 +37,6 @@ variable "location" {
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "management_config" {
|
||||
description = "Optional node management configuration."
|
||||
type = object({
|
||||
auto_repair = bool
|
||||
auto_upgrade = bool
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "max_pods_per_node" {
|
||||
description = "Maximum number of pods per node."
|
||||
type = number
|
||||
@@ -82,153 +49,139 @@ variable "name" {
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_boot_disk_kms_key" {
|
||||
description = "Customer Managed Encryption Key used to encrypt the boot disk attached to each node."
|
||||
type = string
|
||||
default = null
|
||||
variable "node_config" {
|
||||
description = "Node-level configuration."
|
||||
type = object({
|
||||
boot_disk_kms_key = optional(string)
|
||||
disk_size_gb = optional(number)
|
||||
disk_type = optional(string)
|
||||
ephemeral_ssd_count = optional(number)
|
||||
gcfs = optional(bool, false)
|
||||
guest_accelerator = optional(object({
|
||||
count = number
|
||||
type = string
|
||||
gpu_partition_size = optional(string)
|
||||
}))
|
||||
gvnic = optional(bool, false)
|
||||
image_type = optional(string)
|
||||
kubelet_config = optional(object({
|
||||
cpu_manager_policy = string
|
||||
cpu_cfs_quota = optional(bool)
|
||||
cpu_cfs_quota_period = optional(string)
|
||||
}))
|
||||
linux_node_config_sysctls = optional(map(string))
|
||||
local_ssd_count = optional(number)
|
||||
machine_type = optional(string)
|
||||
metadata = optional(map(string))
|
||||
min_cpu_platform = optional(string)
|
||||
preemptible = optional(bool)
|
||||
sandbox_config_gvisor = optional(bool)
|
||||
shielded_instance_config = optional(object({
|
||||
enable_integrity_monitoring = optional(bool)
|
||||
enable_secure_boot = optional(bool)
|
||||
}))
|
||||
spot = optional(bool)
|
||||
workload_metadata_config_mode = optional(string)
|
||||
})
|
||||
default = {
|
||||
disk_type = "pd-balanced"
|
||||
}
|
||||
}
|
||||
|
||||
variable "node_count" {
|
||||
description = "Number of nodes per instance group, can be updated after creation. Ignored when autoscaling is set."
|
||||
type = number
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_disk_size" {
|
||||
description = "Node disk size, defaults to 100GB."
|
||||
type = number
|
||||
default = 100
|
||||
}
|
||||
|
||||
variable "node_disk_type" {
|
||||
description = "Node disk type, defaults to pd-standard."
|
||||
type = string
|
||||
default = "pd-standard"
|
||||
}
|
||||
|
||||
variable "node_guest_accelerator" {
|
||||
description = "Map of type and count of attached accelerator cards."
|
||||
type = map(number)
|
||||
default = {}
|
||||
}
|
||||
|
||||
variable "node_image_type" {
|
||||
description = "Nodes image type."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_labels" {
|
||||
description = "Kubernetes labels attached to nodes."
|
||||
type = map(string)
|
||||
default = {}
|
||||
}
|
||||
|
||||
variable "node_local_ssd_count" {
|
||||
description = "Number of local SSDs attached to nodes."
|
||||
type = number
|
||||
default = 0
|
||||
}
|
||||
variable "node_locations" {
|
||||
description = "Optional list of zones in which nodes should be located. Uses cluster locations if unset."
|
||||
type = list(string)
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_machine_type" {
|
||||
description = "Nodes machine type."
|
||||
type = string
|
||||
default = "n1-standard-1"
|
||||
}
|
||||
|
||||
variable "node_metadata" {
|
||||
description = "Metadata key/value pairs assigned to nodes. Set disable-legacy-endpoints to true when using this variable."
|
||||
type = map(string)
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_min_cpu_platform" {
|
||||
description = "Minimum CPU platform for nodes."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_preemptible" {
|
||||
description = "Use preemptible VMs for nodes."
|
||||
type = bool
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_sandbox_config" {
|
||||
description = "GKE Sandbox configuration. Needs image_type set to COS_CONTAINERD and node_version set to 1.12.7-gke.17 when using this variable."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_service_account" {
|
||||
description = "Service account email. Unused if service account is auto-created."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_service_account_create" {
|
||||
description = "Auto-create service account."
|
||||
type = bool
|
||||
default = false
|
||||
}
|
||||
|
||||
# scopes and scope aliases list
|
||||
# https://cloud.google.com/sdk/gcloud/reference/compute/instances/create#--scopes
|
||||
variable "node_service_account_scopes" {
|
||||
description = "Scopes applied to service account. Default to: 'cloud-platform' when creating a service account; 'devstorage.read_only', 'logging.write', 'monitoring.write' otherwise."
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
variable "node_shielded_instance_config" {
|
||||
description = "Shielded instance options."
|
||||
description = "Number of nodes per instance group. Initial value can only be changed by recreation, current is ignored when autoscaling is used."
|
||||
type = object({
|
||||
enable_secure_boot = bool
|
||||
enable_integrity_monitoring = bool
|
||||
current = optional(number)
|
||||
initial = number
|
||||
})
|
||||
default = {
|
||||
initial = 1
|
||||
}
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "node_locations" {
|
||||
description = "Node locations."
|
||||
type = list(string)
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "nodepool_config" {
|
||||
description = "Nodepool-level configuration."
|
||||
type = object({
|
||||
autoscaling = optional(object({
|
||||
location_policy = optional(string)
|
||||
max_node_count = optional(number)
|
||||
min_node_count = optional(number)
|
||||
use_total_nodes = optional(bool, false)
|
||||
}))
|
||||
management = optional(object({
|
||||
auto_repair = optional(bool)
|
||||
auto_upgrade = optional(bool)
|
||||
}))
|
||||
# placement_policy = optional(bool)
|
||||
upgrade_settings = optional(object({
|
||||
max_surge = number
|
||||
max_unavailable = number
|
||||
}))
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_spot" {
|
||||
description = "Use Spot VMs for nodes."
|
||||
type = bool
|
||||
default = null
|
||||
variable "pod_range" {
|
||||
description = "Pod secondary range configuration."
|
||||
type = object({
|
||||
secondary_pod_range = object({
|
||||
cidr = optional(string)
|
||||
create = optional(bool)
|
||||
name = string
|
||||
})
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_tags" {
|
||||
description = "Network tags applied to nodes."
|
||||
type = list(string)
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "node_taints" {
|
||||
description = "Kubernetes taints applied to nodes. E.g. type=blue:NoSchedule."
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
|
||||
|
||||
variable "project_id" {
|
||||
description = "Cluster project id."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "upgrade_config" {
|
||||
description = "Optional node upgrade configuration."
|
||||
variable "reservation_affinity" {
|
||||
description = "Configuration of the desired reservation which instances could take capacity from."
|
||||
type = object({
|
||||
max_surge = number
|
||||
max_unavailable = number
|
||||
consume_reservation_type = string
|
||||
key = optional(string)
|
||||
values = optional(list(string))
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "workload_metadata_config" {
|
||||
description = "Metadata configuration to expose to workloads on the node pool."
|
||||
type = string
|
||||
default = "GKE_METADATA"
|
||||
variable "service_account" {
|
||||
description = "Nodepool service account. If this variable is set to null, the default GCE service account will be used. If set and email is null, a service account will be created. If scopes are null a default will be used."
|
||||
type = object({
|
||||
email = optional(string)
|
||||
oauth_scopes = optional(list(string))
|
||||
})
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "sole_tenant_nodegroup" {
|
||||
description = "Sole tenant node group."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "tags" {
|
||||
description = "Network tags applied to nodes."
|
||||
type = list(string)
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "taints" {
|
||||
description = "Kubernetes taints applied to all nodes."
|
||||
type = list(object({
|
||||
key = string
|
||||
value = string
|
||||
effect = string
|
||||
}))
|
||||
default = null
|
||||
}
|
||||
|
||||
@@ -17,11 +17,11 @@ terraform {
|
||||
required_providers {
|
||||
google = {
|
||||
source = "hashicorp/google"
|
||||
version = ">= 4.32.0" # tftest
|
||||
version = ">= 4.36.0" # tftest
|
||||
}
|
||||
google-beta = {
|
||||
source = "hashicorp/google-beta"
|
||||
version = ">= 4.32.0" # tftest
|
||||
version = ">= 4.36.0" # tftest
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user