diff --git a/README.md b/README.md index 7087c9120..064e72fa7 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ Currently available modules: - **foundational** - [billing budget](./modules/billing-budget), [Cloud Identity group](./modules/cloud-identity-group/), [folder](./modules/folder), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [organization](./modules/organization), [project](./modules/project), [projects-data-source](./modules/projects-data-source) - **networking** - [DNS](./modules/dns), [DNS Response Policy](./modules/dns-response-policy/), [Cloud Endpoints](./modules/endpoints), [address reservation](./modules/net-address), [NAT](./modules/net-cloudnat), [Dedicated VLAN Attachment](./modules/net-dedicated-vlan-attachment/), [Global Load Balancer (classic)](./modules/net-glb/), [L4 ILB](./modules/net-ilb), [L7 ILB](./modules/net-ilb-l7), [IPSec over Interconnect](./modules/net-ipsec-over-interconnect), [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC firewall policy](./modules/net-vpc-firewall-policy), [VPC peering](./modules/net-vpc-peering), [VPN dynamic](./modules/net-vpn-dynamic), [HA VPN](./modules/net-vpn-ha), [VPN static](./modules/net-vpn-static), [Service Directory](./modules/service-directory) - **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid), [GKE cluster](./modules/gke-cluster-standard), [GKE hub](./modules/gke-hub), [GKE nodepool](./modules/gke-nodepool) -- **data** - [BigQuery dataset](./modules/bigquery-dataset), [Bigtable instance](./modules/bigtable-instance), [Cloud Dataplex](./modules/cloud-dataplex), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag), [Datafusion](./modules/datafusion), [Dataproc](./modules/dataproc), [GCS](./modules/gcs), [Pub/Sub](./modules/pubsub) +- **data** - [AlloyDB instance](./modules/alloydb-instance), [BigQuery dataset](./modules/bigquery-dataset), [Bigtable instance](./modules/bigtable-instance), [Cloud Dataplex](./modules/cloud-dataplex), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag), [Datafusion](./modules/datafusion), [Dataproc](./modules/dataproc), [GCS](./modules/gcs), [Pub/Sub](./modules/pubsub) - **development** - [API Gateway](./modules/api-gateway), [Apigee](./modules/apigee), [Artifact Registry](./modules/artifact-registry), [Container Registry](./modules/container-registry), [Cloud Source Repository](./modules/source-repository) - **security** - [Binauthz](./modules/binauthz/), [KMS](./modules/kms), [SecretManager](./modules/secret-manager), [VPC Service Control](./modules/vpc-sc) - **serverless** - [Cloud Function](./modules/cloud-function), [Cloud Run](./modules/cloud-run) diff --git a/modules/README.md b/modules/README.md index d896360ad..e5dd43478 100644 --- a/modules/README.md +++ b/modules/README.md @@ -70,6 +70,7 @@ These modules are used in the examples included in this repository. If you are u ## Data +- [AlloyDB instance](./alloydb-instance) - [BigQuery dataset](./bigquery-dataset) - [Bigtable instance](./bigtable-instance) - [Cloud Dataplex](./cloud-dataplex) diff --git a/modules/alloydb-instance/README.md b/modules/alloydb-instance/README.md new file mode 100644 index 000000000..1e2413115 --- /dev/null +++ b/modules/alloydb-instance/README.md @@ -0,0 +1,88 @@ +# AlloyDB cluster and instance with read replicas + +This module manages the creation of AlloyDB cluster and configuration with/without automated backup policy, Primary node instance and Read Node Pools. + + +## Simple example + +This example shows how to create Alloydb cluster and instance with multiple read pools in GCP project. + +```hcl +module "alloydb" { + source = "./fabric/modules/alloydb-instance" + project_id = "myproject" + cluster_id = "alloydb-cluster-all" + location = "europe-west2" + labels = {} + display_name = "" + initial_user = { + user = "alloydb-cluster-full", + password = "alloydb-cluster-password" + } + network_self_link = "projects/myproject/global/networks/default" + + automated_backup_policy = null + + primary_instance_config = { + instance_id = "primary-instance-1", + instance_type = "PRIMARY", + machine_cpu_count = 2, + database_flags = {}, + display_name = "alloydb-primary-instance" + } + read_pool_instance = [ + { + instance_id = "read-instance-1", + display_name = "read-instance-1", + instance_type = "READ_POOL", + node_count = 1, + database_flags = {}, + machine_cpu_count = 1 + }, + { + instance_id = "read-instance-2", + display_name = "read-instance-2", + instance_type = "READ_POOL", + node_count = 1, + database_flags = {}, + machine_cpu_count = 1 + } + ] + +} + +# tftest modules=1 resources=7 +``` +## TODO +- [ ] Add IAM support +- [ ] support password in output + + +## Variables + +| name | description | type | required | default | +|---|---|:---:|:---:|:---:| +| [cluster_id](variables.tf#L35) | The ID of the alloydb cluster. | string | ✓ | | +| [network_self_link](variables.tf#L83) | Network ID where the AlloyDb cluster will be deployed. | string | ✓ | | +| [primary_instance_config](variables.tf#L88) | Primary cluster configuration that supports read and write operations. | object({…}) | ✓ | | +| [project_id](variables.tf#L110) | The ID of the project in which to provision resources. | string | ✓ | | +| [automated_backup_policy](variables.tf#L17) | The automated backup policy for this cluster. | object({…}) | | null | +| [display_name](variables.tf#L44) | Human readable display name for the Alloy DB Cluster. | string | | null | +| [encryption_key_name](variables.tf#L50) | The fully-qualified resource name of the KMS key for cluster encryption. | string | | null | +| [initial_user](variables.tf#L56) | Alloy DB Cluster Initial User Credentials. | object({…}) | | null | +| [labels](variables.tf#L65) | User-defined labels for the alloydb cluster. | map(string) | | {} | +| [location](variables.tf#L71) | Location where AlloyDb cluster will be deployed. | string | | "europe-west2" | +| [network_name](variables.tf#L77) | The network name of the project in which to provision resources. | string | | "multiple-readpool" | +| [read_pool_instance](variables.tf#L115) | List of Read Pool Instances to be created. | list(object({…})) | | [] | + +## Outputs + +| name | description | sensitive | +|---|---|:---:| +| [cluster](outputs.tf#L17) | Cluster created. | ✓ | +| [cluster_id](outputs.tf#L23) | ID of the Alloy DB Cluster created. | | +| [primary_instance](outputs.tf#L28) | Primary instance created. | | +| [primary_instance_id](outputs.tf#L33) | ID of the primary instance created. | | +| [read_pool_instance_ids](outputs.tf#L38) | IDs of the read instances created. | | + + diff --git a/modules/alloydb-instance/main.tf b/modules/alloydb-instance/main.tf new file mode 100644 index 000000000..1a01aedff --- /dev/null +++ b/modules/alloydb-instance/main.tf @@ -0,0 +1,160 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +locals { + quantity_based_retention_count = ( + var.automated_backup_policy != null ? (var.automated_backup_policy.quantity_based_retention_count != null ? [var.automated_backup_policy.quantity_based_retention_count] : []) : [] + ) + read_pool_instance = ( + var.read_pool_instance != null ? + { for read_pool_instances in var.read_pool_instance : read_pool_instances.instance_id => read_pool_instances } : {} + ) + time_based_retention_count = ( + var.automated_backup_policy != null ? (var.automated_backup_policy.time_based_retention_count != null ? [var.automated_backup_policy.time_based_retention_count] : []) : [] + ) +} + +resource "google_alloydb_cluster" "default" { + cluster_id = var.cluster_id + location = var.location + network = var.network_self_link + display_name = var.display_name + project = var.project_id + labels = var.labels + + dynamic "automated_backup_policy" { + for_each = var.automated_backup_policy == null ? [] : [""] + content { + location = var.automated_backup_policy.location + backup_window = var.automated_backup_policy.backup_window + enabled = var.automated_backup_policy.enabled + labels = var.automated_backup_policy.labels + + + weekly_schedule { + days_of_week = automated_backup_policy.value.weekly_schedule.days_of_week + dynamic "start_times" { + for_each = { for i, time in automated_backup_policy.value.weekly_schedule.start_times : i => { + hours = tonumber(split(":", time)[0]) + minutes = tonumber(split(":", time)[1]) + seconds = tonumber(split(":", time)[2]) + nanos = tonumber(split(":", time)[3]) + } + } + content { + hours = start_times.value.hours + minutes = start_times.value.minutes + seconds = start_times.value.seconds + nanos = start_times.value.nanos + } + } + } + + dynamic "quantity_based_retention" { + for_each = local.quantity_based_retention_count + content { + count = quantity_based_retention.value + } + } + + dynamic "time_based_retention" { + for_each = local.time_based_retention_count + content { + retention_period = time_based_retention.value + } + } + + dynamic "encryption_config" { + for_each = automated_backup_policy.value.backup_encryption_key_name == null ? [] : ["encryption_config"] + content { + kms_key_name = automated_backup_policy.value.backup_encryption_key_name + } + } + + } + + } + + dynamic "initial_user" { + for_each = var.initial_user == null ? [] : ["initial_user"] + content { + user = var.initial_user.user + password = var.initial_user.password + } + } + + dynamic "encryption_config" { + for_each = var.encryption_key_name == null ? [] : ["encryption_config"] + content { + kms_key_name = var.encryption_key_name + } + } +} + +resource "google_alloydb_instance" "primary" { + cluster = google_alloydb_cluster.default.name + instance_id = var.primary_instance_config.instance_id + instance_type = "PRIMARY" + display_name = var.primary_instance_config.display_name + database_flags = var.primary_instance_config.database_flags + labels = var.primary_instance_config.labels + annotations = var.primary_instance_config.annotations + gce_zone = var.primary_instance_config.availability_type == "ZONAL" ? var.primary_instance_config.gce_zone : null + availability_type = var.primary_instance_config.availability_type + + machine_config { + cpu_count = var.primary_instance_config.machine_cpu_count + } + +} + +resource "google_alloydb_instance" "read_pool" { + for_each = local.read_pool_instance + cluster = google_alloydb_cluster.default.name + instance_id = each.key + instance_type = "READ_POOL" + availability_type = each.value.availability_type + gce_zone = each.value.availability_type == "ZONAL" ? each.value.availability_type.gce_zone : null + + read_pool_config { + node_count = each.value.node_count + } + + database_flags = each.value.database_flags + machine_config { + cpu_count = each.value.machine_cpu_count + } + + depends_on = [google_alloydb_instance.primary, google_compute_network.default, google_compute_global_address.private_ip_alloc, google_service_networking_connection.vpc_connection] +} + +resource "google_compute_network" "default" { + name = var.network_name +} + +resource "google_compute_global_address" "private_ip_alloc" { + name = "adb-all" + address_type = "INTERNAL" + purpose = "VPC_PEERING" + prefix_length = 16 + network = google_compute_network.default.id +} + +resource "google_service_networking_connection" "vpc_connection" { + network = google_compute_network.default.id + service = "servicenetworking.googleapis.com" + reserved_peering_ranges = [google_compute_global_address.private_ip_alloc.name] +} diff --git a/modules/alloydb-instance/outputs.tf b/modules/alloydb-instance/outputs.tf new file mode 100644 index 000000000..990278a94 --- /dev/null +++ b/modules/alloydb-instance/outputs.tf @@ -0,0 +1,43 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +output "cluster" { + description = "Cluster created." + value = resource.google_alloydb_cluster.default.display_name + sensitive = true +} + +output "cluster_id" { + description = "ID of the Alloy DB Cluster created." + value = google_alloydb_cluster.default.cluster_id +} + +output "primary_instance" { + description = "Primary instance created." + value = resource.google_alloydb_instance.primary.display_name +} + +output "primary_instance_id" { + description = "ID of the primary instance created." + value = google_alloydb_instance.primary.instance_id +} + +output "read_pool_instance_ids" { + description = "IDs of the read instances created." + value = [ + for rd, details in google_alloydb_instance.read_pool : details.instance_id + ] +} diff --git a/modules/alloydb-instance/variables.tf b/modules/alloydb-instance/variables.tf new file mode 100644 index 000000000..97ed36682 --- /dev/null +++ b/modules/alloydb-instance/variables.tf @@ -0,0 +1,127 @@ +/** + * Copyright 2023 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +variable "automated_backup_policy" { + description = "The automated backup policy for this cluster." + type = object({ + location = optional(string) + backup_window = optional(string) + enabled = optional(bool) + weekly_schedule = optional(object({ + days_of_week = optional(list(string)) + start_times = list(string) + })), + quantity_based_retention_count = optional(number) + time_based_retention_count = optional(string) + labels = optional(map(string)) + backup_encryption_key_name = optional(string) + }) + default = null +} + +variable "cluster_id" { + description = "The ID of the alloydb cluster." + type = string + validation { + condition = can(regex("^[a-z0-9-]+$", var.cluster_id)) + error_message = "ERROR: Cluster ID must contain only Letters(lowercase), number, and hyphen." + } +} + +variable "display_name" { + description = "Human readable display name for the Alloy DB Cluster." + type = string + default = null +} + +variable "encryption_key_name" { + description = "The fully-qualified resource name of the KMS key for cluster encryption." + type = string + default = null +} + +variable "initial_user" { + description = "Alloy DB Cluster Initial User Credentials." + type = object({ + user = optional(string), + password = string + }) + default = null +} + +variable "labels" { + description = "User-defined labels for the alloydb cluster." + type = map(string) + default = {} +} + +variable "location" { + description = "Location where AlloyDb cluster will be deployed." + type = string + default = "europe-west2" +} + +variable "network_name" { + description = "The network name of the project in which to provision resources." + type = string + default = "multiple-readpool" +} + +variable "network_self_link" { + description = "Network ID where the AlloyDb cluster will be deployed." + type = string +} + +variable "primary_instance_config" { + description = "Primary cluster configuration that supports read and write operations." + type = object({ + instance_id = string, + display_name = optional(string), + database_flags = optional(map(string)) + labels = optional(map(string)) + annotations = optional(map(string)) + gce_zone = optional(string) + availability_type = optional(string) + machine_cpu_count = optional(number, 2), + }) + validation { + condition = can(regex("^(2|4|8|16|32|64)$", var.primary_instance_config.machine_cpu_count)) + error_message = "cpu count must be one of [2 4 8 16 32 64]." + } + validation { + condition = can(regex("^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$", var.primary_instance_config.instance_id)) + error_message = "Primary Instance ID should satisfy the following pattern ^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$." + } +} + +variable "project_id" { + description = "The ID of the project in which to provision resources." + type = string +} + +variable "read_pool_instance" { + description = "List of Read Pool Instances to be created." + type = list(object({ + instance_id = string + display_name = string + node_count = optional(number, 1) + database_flags = optional(map(string)) + availability_type = optional(string) + gce_zone = optional(string) + machine_cpu_count = optional(number, 2) + })) + default = [] +} diff --git a/modules/alloydb-instance/versions.tf b/modules/alloydb-instance/versions.tf new file mode 100644 index 000000000..17d703629 --- /dev/null +++ b/modules/alloydb-instance/versions.tf @@ -0,0 +1,29 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +terraform { + required_version = ">= 1.0.0" + required_providers { + google = { + source = "hashicorp/google" + version = ">= 4.64.0" # tftest + } + google-beta = { + source = "hashicorp/google-beta" + version = ">= 4.64.0" # tftest + } + } +} + + diff --git a/tests/modules/alloydb_instance/examples/alloydb_instance.tfvars b/tests/modules/alloydb_instance/examples/alloydb_instance.tfvars new file mode 100644 index 000000000..03fda4bf8 --- /dev/null +++ b/tests/modules/alloydb_instance/examples/alloydb_instance.tfvars @@ -0,0 +1,40 @@ +project_id = "myproject" +cluster_id = "alloydb-cluster-all" +location = "europe-west2" +labels = {} +display_name = "alloydb-cluster-all" +initial_user = { + user = "alloydb-cluster-full", + password = "alloydb-cluster-password" +} +network_self_link = "projects/myproject/global/networks/default" + +automated_backup_policy = null + +primary_instance_config = { + instance_id = "primary-instance-1", + instance_type = "PRIMARY", + machine_cpu_count = 2, + database_flags = {}, + display_name = "alloydb-primary-instance" +} + + +read_pool_instance = [ + { + instance_id = "read-instance-1", + display_name = "read-instancename-1", + instance_type = "READ_POOL", + node_count = 1, + database_flags = {}, + machine_cpu_count = 1 + }, + { + instance_id = "read-instance-2", + display_name = "read-instancename-2", + instance_type = "READ_POOL", + node_count = 1, + database_flags = {}, + machine_cpu_count = 1 + } +] diff --git a/tests/modules/alloydb_instance/examples/alloydb_instance.yaml b/tests/modules/alloydb_instance/examples/alloydb_instance.yaml new file mode 100644 index 000000000..bf9a7c7ee --- /dev/null +++ b/tests/modules/alloydb_instance/examples/alloydb_instance.yaml @@ -0,0 +1,103 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + values: + google_alloydb_cluster.default: + cluster_id: alloydb-cluster-all + display_name: alloydb-cluster-all + encryption_config: [] + initial_user: + - password: alloydb-cluster-password + user: alloydb-cluster-full + labels: null + location: europe-west2 + network: projects/myproject/global/networks/default + project: myproject + timeouts: null + google_alloydb_instance.primary: + annotations: null + database_flags: null + display_name: alloydb-primary-instance + gce_zone: null + instance_id: primary-instance-1 + instance_type: PRIMARY + labels: null + machine_config: + - cpu_count: 2 + read_pool_config: [] + timeouts: null + google_alloydb_instance.read_pool["read-instance-1"]: + annotations: null + database_flags: null + display_name: null + gce_zone: null + instance_id: read-instance-1 + instance_type: READ_POOL + labels: null + machine_config: + - cpu_count: 1 + read_pool_config: + - node_count: 1 + timeouts: null + google_alloydb_instance.read_pool["read-instance-2"]: + annotations: null + database_flags: null + display_name: null + gce_zone: null + instance_id: read-instance-2 + instance_type: READ_POOL + labels: null + machine_config: + - cpu_count: 1 + read_pool_config: + - node_count: 1 + timeouts: null + google_compute_global_address.private_ip_alloc: + address_type: INTERNAL + description: null + ip_version: null + name: adb-all + prefix_length: 16 + purpose: VPC_PEERING + timeouts: null + google_compute_network.default: + auto_create_subnetworks: true + delete_default_routes_on_create: false + description: null + enable_ula_internal_ipv6: null + name: multiple-readpool + network_firewall_policy_enforcement_order: AFTER_CLASSIC_FIREWALL + timeouts: null + google_service_networking_connection.vpc_connection: + reserved_peering_ranges: + - adb-all + service: servicenetworking.googleapis.com + timeouts: null + +counts: + google_alloydb_cluster: 1 + google_alloydb_instance: 3 + google_compute_global_address: 1 + google_compute_network: 1 + google_service_networking_connection: 1 + modules: 0 + resources: 7 + +outputs: + cluster: alloydb-cluster-all + cluster_id: alloydb-cluster-all + primary_instance: alloydb-primary-instance + primary_instance_id: primary-instance-1 + read_pool_instance_ids: + - read-instance-1 + - read-instance-2