diff --git a/CHANGELOG.md b/CHANGELOG.md index 244af3025..dcb1d9317 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -183,7 +183,7 @@ All notable changes to this project will be documented in this file. - **incompatible change** removed `iam` key from logging sink configuration in the `project` and `organization` modules - remove GCS to BQ with Dataflow example, replace by GCS to BQ with least privileges - the `net-vpc` and `project` modules now use the beta provider for shared VPC-related resources -- new [iot-core](modules/iot-core) module +- new iot-core module - **incompatible change** the variables for host and service Shared VPCs have changed in the project module - **incompatible change** the variable for service identities IAM has changed in the project factory - add `data-catalog-policy-tag` module diff --git a/README.md b/README.md index b4711da73..bdb17b658 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ The current list of modules supports most of the core foundational and networkin Currently available modules: -- **foundational** - [folder](./modules/folder), [organization](./modules/organization), [project](./modules/project), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [billing budget](./modules/billing-budget), [naming convention](./modules/naming-convention), [projects-data-source](./modules/projects-data-source), [organization-policy](./modules/organization-policy) +- **foundational** - [folder](./modules/folder), [organization](./modules/organization), [project](./modules/project), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [billing budget](./modules/billing-budget), [projects-data-source](./modules/projects-data-source), [organization-policy](./modules/organization-policy) - **networking** - [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC peering](./modules/net-vpc-peering), [VPN static](./modules/net-vpn-static), [VPN dynamic](./modules/net-vpn-dynamic), [HA VPN](./modules/net-vpn-ha), [NAT](./modules/net-cloudnat), [address reservation](./modules/net-address), [DNS](./modules/dns), [L4 ILB](./modules/net-ilb), [L7 ILB](./modules/net-ilb-l7), [Service Directory](./modules/service-directory), [Cloud Endpoints](./modules/endpoints) - **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [GKE cluster](./modules/gke-cluster), [GKE nodepool](./modules/gke-nodepool), [GKE hub](./modules/gke-hub), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid) - **data** - [GCS](./modules/gcs), [BigQuery dataset](./modules/bigquery-dataset), [Pub/Sub](./modules/pubsub), [Datafusion](./modules/datafusion), [Bigtable instance](./modules/bigtable-instance), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag) diff --git a/modules/README.md b/modules/README.md index 3280a6828..281b2fda8 100644 --- a/modules/README.md +++ b/modules/README.md @@ -32,7 +32,6 @@ These modules are used in the examples included in this repository. If you are u - [billing budget](./billing-budget) - [folder](./folder) - [logging bucket](./logging-bucket) -- [naming convention](./naming-convention) - [organization](./organization) - [project](./project) - [projects-data-source](./projects-data-source) diff --git a/modules/folders-unit/README.md b/modules/folders-unit/README.md deleted file mode 100644 index e84a962b8..000000000 --- a/modules/folders-unit/README.md +++ /dev/null @@ -1,57 +0,0 @@ -# Google Cloud Unit Folders Module - -This module allows creation and management of an organizational hierarchy "unit" composed of a parent folder (usually mapped to a business unit or team), and a set of child folders (usually mapped to environments) each with a corresponding set of service accounts, IAM bindings and GCS buckets. - -## Example - -```hcl -module "folders-unit" { - source = "./fabric/modules/folders-unit" - name = "Business Intelligence" - short_name = "bi" - automation_project_id = "automation-project-394yr923811" - billing_account_id = "015617-16GHBC-AF02D9" - organization_id = "506128240800" - root_node = "folders/93469270123701" - prefix = "unique-prefix" - environments = { - dev = "Development", - test = "Testing", - prod = "Production" - } - service_account_keys = true -} -# tftest modules=1 resources=37 -``` - - -## Variables - -| name | description | type | required | default | -|---|---|:---:|:---:|:---:| -| [automation_project_id](variables.tf#L17) | Project id used for automation service accounts. | string | ✓ | | -| [billing_account_id](variables.tf#L22) | Country billing account account. | string | ✓ | | -| [name](variables.tf#L86) | Top folder name. | string | ✓ | | -| [organization_id](variables.tf#L91) | Organization id in organizations/nnnnnn format. | string | ✓ | | -| [root_node](variables.tf#L102) | Root node in folders/folder_id or organizations/org_id format. | string | ✓ | | -| [short_name](variables.tf#L113) | Short name used as GCS bucket and service account prefixes, do not use capital letters or spaces. | string | ✓ | | -| [environments](variables.tf#L27) | Unit environments short names. | map(string) | | {…} | -| [gcs_defaults](variables.tf#L36) | Defaults use for the state GCS buckets. | map(string) | | {…} | -| [iam](variables.tf#L45) | IAM bindings for the top-level folder in {ROLE => [MEMBERS]} format. | map(list(string)) | | {} | -| [iam_billing_config](variables.tf#L51) | Grant billing user role to service accounts, defaults to granting on the billing account. | object({…}) | | {…} | -| [iam_enviroment_roles](variables.tf#L63) | IAM roles granted to the environment service account on the environment sub-folder. | list(string) | | […] | -| [iam_xpn_config](variables.tf#L74) | Grant Shared VPC creation roles to service accounts, defaults to granting at folder level. | object({…}) | | {…} | -| [prefix](variables.tf#L96) | Optional prefix used for GCS bucket names to ensure uniqueness. | string | | null | -| [service_account_keys](variables.tf#L107) | Generate and store service account keys in the state file. | bool | | false | - -## Outputs - -| name | description | sensitive | -|---|---|:---:| -| [env_folders](outputs.tf#L17) | Unit environments folders. | | -| [env_gcs_buckets](outputs.tf#L28) | Unit environments tfstate gcs buckets. | | -| [env_sa_keys](outputs.tf#L36) | Unit environments service account keys. | ✓ | -| [env_service_accounts](outputs.tf#L45) | Unit environments service accounts. | | -| [unit_folder](outputs.tf#L53) | Unit top level folder. | | - - diff --git a/modules/folders-unit/locals.tf b/modules/folders-unit/locals.tf deleted file mode 100644 index bd5135092..000000000 --- a/modules/folders-unit/locals.tf +++ /dev/null @@ -1,58 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - folder_roles = concat(var.iam_enviroment_roles, local.sa_xpn_folder_roles) - iam = var.iam == null ? {} : var.iam - folder_iam_service_account_bindings = { - for pair in setproduct(keys(var.environments), local.folder_roles) : - "${pair.0}-${pair.1}" => { environment = pair.0, role = pair.1 } - } - org_iam_service_account_bindings = { - for pair in setproduct(keys(var.environments), concat( - local.sa_xpn_org_roles, - local.sa_billing_org_roles, - local.sa_billing_org_roles)) : - "${pair.0}-${pair.1}" => { environment = pair.0, role = pair.1 } - } - billing_iam_service_account_bindings = { - for pair in setproduct(keys(var.environments), local.sa_billing_account_roles) : - "${pair.0}-${pair.1}" => { environment = pair.0, role = pair.1 } - } - service_accounts = { - for key, sa in google_service_account.environment : - key => "serviceAccount:${sa.email}" - } - sa_billing_account_roles = ( - var.iam_billing_config.target_org ? [] : ["roles/billing.user"] - ) - sa_billing_org_roles = ( - !var.iam_billing_config.target_org ? [] : ["roles/billing.user"] - ) - sa_xpn_folder_roles = ( - local.sa_xpn_target_org ? [] : ["roles/compute.xpnAdmin"] - ) - sa_xpn_org_roles = ( - local.sa_xpn_target_org - ? ["roles/compute.xpnAdmin", "roles/resourcemanager.organizationViewer"] - : ["roles/resourcemanager.organizationViewer"] - ) - sa_xpn_target_org = ( - var.iam_xpn_config.target_org - || - substr(var.root_node, 0, 13) == "organizations" - ) -} diff --git a/modules/folders-unit/main.tf b/modules/folders-unit/main.tf deleted file mode 100644 index f609fffc1..000000000 --- a/modules/folders-unit/main.tf +++ /dev/null @@ -1,109 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - organization_id = element(split("/", var.organization_id), 1) -} - -############################################################################### -# Folders and folder IAM # -############################################################################### - -resource "google_folder" "unit" { - display_name = var.name - parent = var.root_node -} - -resource "google_folder" "environment" { - for_each = var.environments - display_name = each.value - parent = google_folder.unit.name -} - -resource "google_folder_iam_binding" "unit" { - for_each = var.iam - folder = google_folder.unit.name - role = each.key - members = each.value -} - -resource "google_folder_iam_binding" "environment" { - for_each = local.folder_iam_service_account_bindings - folder = google_folder.environment[each.value.environment].name - role = each.value.role - members = [local.service_accounts[each.value.environment]] -} - -############################################################################### -# Billing account and org IAM # -############################################################################### - -resource "google_organization_iam_member" "org_iam_member" { - for_each = local.org_iam_service_account_bindings - org_id = local.organization_id - role = each.value.role - member = local.service_accounts[each.value.environment] -} - -resource "google_billing_account_iam_member" "billing_iam_member" { - for_each = var.iam_billing_config.grant ? local.billing_iam_service_account_bindings : {} - billing_account_id = var.billing_account_id - role = each.value.role - member = local.service_accounts[each.value.environment] -} - -################################################################################ -# Service Accounts # -################################################################################ - -resource "google_service_account" "environment" { - for_each = var.environments - project = var.automation_project_id - account_id = "${var.short_name}-${each.key}" - display_name = "${var.short_name} ${each.key} (Terraform managed)." -} - -resource "google_service_account_key" "keys" { - for_each = var.service_account_keys ? var.environments : {} - service_account_id = google_service_account.environment[each.key].email -} - -################################################################################ -# GCS and GCS IAM # -################################################################################ - -resource "google_storage_bucket" "tfstate" { - for_each = var.environments - project = var.automation_project_id - name = join("", [ - var.prefix == null ? "" : "${var.prefix}-", - "${var.short_name}-${each.key}-tf" - ]) - location = var.gcs_defaults.location - storage_class = var.gcs_defaults.storage_class - force_destroy = false - uniform_bucket_level_access = true - versioning { - enabled = true - } -} - -resource "google_storage_bucket_iam_binding" "bindings" { - for_each = var.environments - bucket = google_storage_bucket.tfstate[each.key].name - role = "roles/storage.objectAdmin" - members = [local.service_accounts[each.key]] -} diff --git a/modules/folders-unit/outputs.tf b/modules/folders-unit/outputs.tf deleted file mode 100644 index 8e4d1066f..000000000 --- a/modules/folders-unit/outputs.tf +++ /dev/null @@ -1,59 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -output "env_folders" { - description = "Unit environments folders." - value = { - for key, folder in google_folder.environment - : key => { - id = folder.name, - name = folder.display_name - } - } -} - -output "env_gcs_buckets" { - description = "Unit environments tfstate gcs buckets." - value = { - for key, bucket in google_storage_bucket.tfstate - : key => bucket.name - } -} - -output "env_sa_keys" { - description = "Unit environments service account keys." - sensitive = true - value = { - for key, sa_key in google_service_account_key.keys : - key => sa_key.private_key - } -} - -output "env_service_accounts" { - description = "Unit environments service accounts." - value = { - for key, sa in google_service_account.environment - : key => sa.email - } -} - -output "unit_folder" { - description = "Unit top level folder." - value = { - id = google_folder.unit.name, - name = google_folder.unit.display_name - } -} diff --git a/modules/folders-unit/variables.tf b/modules/folders-unit/variables.tf deleted file mode 100644 index 10a167d47..000000000 --- a/modules/folders-unit/variables.tf +++ /dev/null @@ -1,116 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -variable "automation_project_id" { - description = "Project id used for automation service accounts." - type = string -} - -variable "billing_account_id" { - description = "Country billing account account." - type = string -} - -variable "environments" { - description = "Unit environments short names." - type = map(string) - default = { - non-prod = "Non production" - prod = "Production" - } -} - -variable "gcs_defaults" { - description = "Defaults use for the state GCS buckets." - type = map(string) - default = { - location = "EU" - storage_class = "MULTI_REGIONAL" - } -} - -variable "iam" { - description = "IAM bindings for the top-level folder in {ROLE => [MEMBERS]} format." - type = map(list(string)) - default = {} -} - -variable "iam_billing_config" { - description = "Grant billing user role to service accounts, defaults to granting on the billing account." - type = object({ - grant = bool - target_org = bool - }) - default = { - grant = true - target_org = false - } -} - -variable "iam_enviroment_roles" { - description = "IAM roles granted to the environment service account on the environment sub-folder." - type = list(string) - default = [ - "roles/compute.networkAdmin", - "roles/owner", - "roles/resourcemanager.folderAdmin", - "roles/resourcemanager.projectCreator", - ] -} - -variable "iam_xpn_config" { - description = "Grant Shared VPC creation roles to service accounts, defaults to granting at folder level." - type = object({ - grant = bool - target_org = bool - }) - default = { - grant = true - target_org = false - } -} - -variable "name" { - description = "Top folder name." - type = string -} - -variable "organization_id" { - description = "Organization id in organizations/nnnnnn format." - type = string -} - -variable "prefix" { - description = "Optional prefix used for GCS bucket names to ensure uniqueness." - type = string - default = null -} - -variable "root_node" { - description = "Root node in folders/folder_id or organizations/org_id format." - type = string -} - -variable "service_account_keys" { - description = "Generate and store service account keys in the state file." - type = bool - default = false -} - -variable "short_name" { - description = "Short name used as GCS bucket and service account prefixes, do not use capital letters or spaces." - type = string -} diff --git a/modules/folders-unit/versions.tf b/modules/folders-unit/versions.tf deleted file mode 100644 index 4278054b1..000000000 --- a/modules/folders-unit/versions.tf +++ /dev/null @@ -1,29 +0,0 @@ -# Copyright 2022 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -terraform { - required_version = ">= 1.1.0" - required_providers { - google = { - source = "hashicorp/google" - version = ">= 4.32.0" # tftest - } - google-beta = { - source = "hashicorp/google-beta" - version = ">= 4.32.0" # tftest - } - } -} - - diff --git a/modules/iot-core/README.md b/modules/iot-core/README.md deleted file mode 100644 index 18579d506..000000000 --- a/modules/iot-core/README.md +++ /dev/null @@ -1,145 +0,0 @@ -# Google Cloud IoT Core Module - -This module sets up Cloud IoT Core Registry, registers IoT Devices and configures Pub/Sub topics required in Cloud IoT Core. - -To use this module, ensure the following APIs are enabled: -* pubsub.googleapis.com -* cloudiot.googleapis.com - -## Simple Example - -Basic example showing how to create an IoT Platform (IoT Core), connected to a set of given Pub/Sub topics and provision IoT devices. - -Devices certificates must exist before calling this module. You can generate these certificates using the following command - -``` -openssl req -x509 -newkey rsa:2048 -keyout rsa_private.pem -nodes -out rsa_cert.pem -subj "/CN=unused" -``` - -And then provision public certificate path, together with the rest of device configuration in a devices yaml file following the following format -```yaml -device_id: # id of your IoT Device - is_blocked: # false to allow device connection with IoT Registry - is_gateway: # true to indicate the device connecting acts as a gateway for other IoT Devices - log_level: # device logs level - certificate_file: # public certificate path, generated as explained in the previous step - certificate_format: # Certificates format values are RSA_PEM, RSA_X509_PEM, ES256_PEM, and ES256_X509_PEM -``` - -Example Device config yaml configuration -```yaml -device_1: - is_blocked: false - is_gateway: false - log_level: INFO - certificate_file: device_certs/rsa_cert5.pem - certificate_format: RSA_X509_PEM -device_2: - is_blocked: true - is_gateway: false - log_level: INFO - certificate_file: device_certs/rsa_cert5.pem - certificate_format: RSA_X509_PEM -``` - -```hcl -module "iot-platform" { - source = "./fabric/modules/iot-core" - project_id = "my_project_id" - region = "europe-west1" - telemetry_pubsub_topic_id = "telemetry_topic_id" - status_pubsub_topic_id = "status_topic_id" - protocols = { - http = false, - mqtt = true - } - devices_config_directory = "./devices_config_folder" -} -# tftest:skip - -``` - -Now, we can test sending telemetry messages from devices to our IoT Platform, for example using the MQTT demo client at https://github.com/googleapis/nodejs-iot/tree/main/samples/mqtt_example - -## Example with specific PubSub topics for custom MQTT topics - -If you need to match specific MQTT topics (eg, /temperature) into specific PubSub topics, you can use extra_telemetry_pubsub_topic_ids for that, as in the following example: - -```hcl -module "iot-platform" { - source = "./fabric/modules/iot-core" - project_id = "my_project_id" - region = "europe-west1" - telemetry_pubsub_topic_id = "telemetry_topic_id" - status_pubsub_topic_id = "status_topic_id" - extra_telemetry_pubsub_topic_ids = { - "temperature" = "temp_topic_id", - "humidity" = "hum_topic_id" - } - protocols = { - http = false, - mqtt = true - } - devices_config_directory = "./devices_config_folder" -} -# tftest:skip - -``` - -## Example integrated with Data Foundation Platform -In this example, we will show how to extend the **[Data Foundations Platform](../../blueprints/data-solutions/data-platform-foundations/)** to include IoT Platform as a new source of data. - -![Target architecture](./diagram_iot.png) - -1. First, we will setup Environment following instructions in **[Environment Setup](../../blueprints/data-solutions/data-platform-foundations/)** to setup projects and SAs required. Get output variable project_ids.landing as will be used later - -1. Second, execute instructions in **[Environment Setup](../../blueprints/data-solutions/data-platform-foundations/)** to provision PubSub, DataFlow, BQ,... Get variable landing-pubsub as will be used later to create IoT Registry - -1. Now it is time to provision IoT Platform. Modify landing-project-id and landing_pubsub_topic_id with output variables obtained before. Create device certificates as shown in the Simple Example and register them in devices.yaml file together with deviceids. - -```hcl -module "iot-platform" { - source = "./fabric/modules/iot-core" - project_id = "landing-project-id" - region = "europe-west1" - telemetry_pubsub_topic_id = "landing_pubsub_topic_id" - status_pubsub_topic_id = "status_pubsub_topic_id" - protocols = { - http = false, - mqtt = true - } - devices_config_directory = "./devices_config_folder" -} -# tftest:skip -``` -1. After that, we can setup the pipeline "PubSub to BigQuery" shown at **[Pipeline Setup](../../blueprints/data-solutions/data-platform-foundations/)** - -1. Finally, instead of testing the pipeline by sending messages to PubSub, we can now test sending telemetry messages from simulated IoT devices to our IoT Platform, for example using the MQTT demo client at https://github.com/googleapis/nodejs-iot/tree/main/samples/mqtt_example . We shall edit the client script cloudiot_mqtt_example_nodejs.js to send messages following the pipeline message format, so they are processed by DataFlow job and inserted in the BigQuery table. -``` -const payload = '{"name": "device4", "surname": "NA", "timestamp":"'+Math.floor(Date.now()/1000)+'"}'; -``` - -Or even better, create a new BigQuery table with our IoT sensors data columns and modify the DataFlow job to push data to it. - - -## Variables - -| name | description | type | required | default | -|---|---|:---:|:---:|:---:| -| [devices_config_directory](variables.tf#L17) | Path to folder where devices configs are stored in yaml format. Folder may include subfolders with configuration files. Files suffix must be `.yaml`. | string | ✓ | | -| [project_id](variables.tf#L34) | Project were resources will be deployed | string | ✓ | | -| [region](variables.tf#L48) | Region were resources will be deployed | string | ✓ | | -| [status_pubsub_topic_id](variables.tf#L59) | pub sub topic for status messages (GCP-->Device) | string | ✓ | | -| [telemetry_pubsub_topic_id](variables.tf#L64) | pub sub topic for telemetry messages (Device-->GCP) | string | ✓ | | -| [extra_telemetry_pubsub_topic_ids](variables.tf#L22) | additional pubsub topics linked to adhoc MQTT topics (Device-->GCP) in the format MQTT_TOPIC: PUBSUB_TOPIC_ID | map(string) | | {} | -| [log_level](variables.tf#L28) | IoT Registry Log level | string | | "INFO" | -| [protocols](variables.tf#L39) | IoT protocols (HTTP / MQTT) activation | object({…}) | | { http = true, mqtt = true } | -| [registry_name](variables.tf#L53) | Name for the IoT Core Registry | string | | "cloudiot-registry" | - -## Outputs - -| name | description | sensitive | -|---|---|:---:| -| [iot_registry](outputs.tf#L17) | Cloud IoT Core Registry | | - - diff --git a/modules/iot-core/diagram.png b/modules/iot-core/diagram.png deleted file mode 100644 index 7a3393212..000000000 Binary files a/modules/iot-core/diagram.png and /dev/null differ diff --git a/modules/iot-core/diagram_iot.png b/modules/iot-core/diagram_iot.png deleted file mode 100644 index 4c82259f2..000000000 Binary files a/modules/iot-core/diagram_iot.png and /dev/null differ diff --git a/modules/iot-core/main.tf b/modules/iot-core/main.tf deleted file mode 100644 index 6a8c3db15..000000000 --- a/modules/iot-core/main.tf +++ /dev/null @@ -1,95 +0,0 @@ - -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - devices_config_files = [ - for config_file in fileset("${path.root}/${var.devices_config_directory}", "**/*.yaml") : - "${path.root}/${var.devices_config_directory}/${config_file}" - ] - - device_config = merge( - [ - for config_file in local.devices_config_files : - try(yamldecode(file(config_file)), {}) - ]... - ) -} - -#--------------------------------------------------------- -# Create IoT Core Registry -#--------------------------------------------------------- - -resource "google_cloudiot_registry" "registry" { - - name = var.registry_name - project = var.project_id - region = var.region - - dynamic "event_notification_configs" { - for_each = var.extra_telemetry_pubsub_topic_ids - content { - pubsub_topic_name = event_notification_configs.value - subfolder_matches = event_notification_configs.key - } - } - - event_notification_configs { - pubsub_topic_name = var.telemetry_pubsub_topic_id - subfolder_matches = "" - } - - state_notification_config = { - pubsub_topic_name = var.status_pubsub_topic_id - } - - mqtt_config = { - mqtt_enabled_state = var.protocols.mqtt ? "MQTT_ENABLED" : "MQTT_DISABLED" - } - - http_config = { - http_enabled_state = var.protocols.http ? "HTTP_ENABLED" : "HTTP_DISABLED" - } - - log_level = var.log_level - -} - -#--------------------------------------------------------- -# Create IoT Core Device -# certificate created using: openssl req -x509 -newkey rsa:2048 -keyout rsa_private.pem -nodes -out rsa_cert.pem -subj "/CN=unused" -#--------------------------------------------------------- - -resource "google_cloudiot_device" "device" { - for_each = local.device_config - name = each.key - registry = google_cloudiot_registry.registry.id - - credentials { - public_key { - format = try(each.value.certificate_format, null) - key = try(file(each.value.certificate_file), null) - } - } - - blocked = try(each.value.is_blocked, null) - - log_level = try(each.value.log_level, null) - - gateway_config { - gateway_type = try(each.value.is_gateway, null) ? "GATEWAY" : "NON_GATEWAY" - } -} \ No newline at end of file diff --git a/modules/iot-core/outputs.tf b/modules/iot-core/outputs.tf deleted file mode 100644 index fcce598ad..000000000 --- a/modules/iot-core/outputs.tf +++ /dev/null @@ -1,20 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -output "iot_registry" { - description = "Cloud IoT Core Registry" - value = google_cloudiot_registry.registry -} \ No newline at end of file diff --git a/modules/iot-core/variables.tf b/modules/iot-core/variables.tf deleted file mode 100644 index f19249826..000000000 --- a/modules/iot-core/variables.tf +++ /dev/null @@ -1,67 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -variable "devices_config_directory" { - description = "Path to folder where devices configs are stored in yaml format. Folder may include subfolders with configuration files. Files suffix must be `.yaml`." - type = string -} - -variable "extra_telemetry_pubsub_topic_ids" { - description = "additional pubsub topics linked to adhoc MQTT topics (Device-->GCP) in the format MQTT_TOPIC: PUBSUB_TOPIC_ID" - type = map(string) - default = {} -} - -variable "log_level" { - description = "IoT Registry Log level" - type = string - default = "INFO" -} - -variable "project_id" { - description = "Project were resources will be deployed" - type = string -} - -variable "protocols" { - description = "IoT protocols (HTTP / MQTT) activation" - type = object({ - http = bool, - mqtt = bool - }) - default = { http = true, mqtt = true } -} - -variable "region" { - description = "Region were resources will be deployed" - type = string -} - -variable "registry_name" { - description = "Name for the IoT Core Registry" - type = string - default = "cloudiot-registry" -} - -variable "status_pubsub_topic_id" { - description = "pub sub topic for status messages (GCP-->Device)" - type = string -} - -variable "telemetry_pubsub_topic_id" { - description = "pub sub topic for telemetry messages (Device-->GCP)" - type = string -} \ No newline at end of file diff --git a/modules/naming-convention/README.md b/modules/naming-convention/README.md deleted file mode 100644 index d3d0b85e0..000000000 --- a/modules/naming-convention/README.md +++ /dev/null @@ -1,89 +0,0 @@ -# Naming Convention Module - -This module allows defining a naming convention in a single place, and enforcing it by pre-creating resource names based on a set of tokens (environment, team name, etc.). - -It implements a fairly common naming convention with optional prefix or suffix, but is really meant to be forked, and modified to adapt to individual use cases: just replace the `environment` and `team` variables with whatever makes sense for you, and edit the few lines in `main.tf` marked by comments. - -The module also supports labels, generating sets of per-resource labels that combine the passed in tokens with optional resource-level labels. - -It's completely static, using no provider resources, so its outputs are safe to use where dynamic values are not supported, like in `for_each` statements. - -## Example - -In its default configuration, the module supports an option prefix and suffix, and two tokens: one for the environment, and one for the team name. - -```hcl -module "names-org" { - source = "./fabric/modules/naming-convention" - prefix = "myco" - environment = "dev" - team = "cloud" - resources = { - bucket = ["tf-org", "tf-sec", "tf-log"] - project = ["tf", "sec", "log"] - } - labels = { - project = { - tf = {scope = "global"} - } - } -} - -module "project-tf" { - source = "./fabric/modules/project" - # myco-cloud-dev-tf - name = module.names-org.names.project.tf - # { environment = "dev", scope = "global", team = "cloud" } - labels = module.names-org.labels.project.tf -} -``` - -You can also enable resource type naming, useful with some legacy CMDB setups. When doing this, resource type names become part of the final resource names and are usually shorted (e.g. `prj` instead of `project`): - -```hcl -module "names-org" { - source = "./fabric/modules/naming-convention" - prefix = "myco" - environment = "dev" - team = "cloud" - resources = { - bkt = ["tf-org", "tf-sec", "tf-log"] - prj = ["tf", "sec", "log"] - } - labels = { - prj = { - tf = {scope = "global"} - } - } - use_resource_prefixes = true -} - -module "project-tf" { - source = "./fabric/modules/project" - # prj-myco-cloud-dev-tf - name = module.names-org.names.prj.tf -} -``` - - -## Variables - -| name | description | type | required | default | -|---|---|:---:|:---:|:---:| -| [environment](variables.tf#L17) | Environment abbreviation used in names and labels. | string | ✓ | | -| [resources](variables.tf#L34) | Short resource names by type. | map(list(string)) | ✓ | | -| [team](variables.tf#L51) | Team name. | string | ✓ | | -| [labels](variables.tf#L22) | Per-resource labels. | map(map(map(string))) | | {} | -| [prefix](variables.tf#L28) | Optional name prefix. | string | | null | -| [separator_override](variables.tf#L39) | Optional separator override for specific resource types. | map(string) | | {} | -| [suffix](variables.tf#L45) | Optional name suffix. | string | | null | -| [use_resource_prefixes](variables.tf#L56) | Prefix names with the resource type. | bool | | false | - -## Outputs - -| name | description | sensitive | -|---|---|:---:| -| [labels](outputs.tf#L17) | Per resource labels. | | -| [names](outputs.tf#L22) | Per resource names. | | - - diff --git a/modules/naming-convention/main.tf b/modules/naming-convention/main.tf deleted file mode 100644 index aa7d406b0..000000000 --- a/modules/naming-convention/main.tf +++ /dev/null @@ -1,46 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -locals { - prefix = var.prefix == null ? "" : "${var.prefix}-" - suffix = var.suffix == null ? "" : "-${var.suffix}" - // merge common and per-resource labels - labels = { - for resource_type, resources in var.resources : resource_type => { - for name in resources : name => merge( - // change this line if you need different tokens - { environment = var.environment, team = var.team }, - try(var.labels[resource_type][name], {}) - ) - } - } - // create per-resource names by assembling tokens - names = { - for resource_type, resources in var.resources : resource_type => { - for name in resources : name => join( - try(var.separator_override[resource_type], "-"), - compact([ - var.use_resource_prefixes ? resource_type : "", - var.prefix, - // change these lines if you need different tokens - var.team, - var.environment, - name, - var.suffix - ])) - } - } -} diff --git a/modules/naming-convention/outputs.tf b/modules/naming-convention/outputs.tf deleted file mode 100644 index d96cfa985..000000000 --- a/modules/naming-convention/outputs.tf +++ /dev/null @@ -1,25 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -output "labels" { - description = "Per resource labels." - value = local.labels -} - -output "names" { - description = "Per resource names." - value = local.names -} diff --git a/modules/naming-convention/variables.tf b/modules/naming-convention/variables.tf deleted file mode 100644 index 3ba8cd2cf..000000000 --- a/modules/naming-convention/variables.tf +++ /dev/null @@ -1,60 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -variable "environment" { - description = "Environment abbreviation used in names and labels." - type = string -} - -variable "labels" { - description = "Per-resource labels." - type = map(map(map(string))) - default = {} -} - -variable "prefix" { - description = "Optional name prefix." - type = string - default = null -} - -variable "resources" { - description = "Short resource names by type." - type = map(list(string)) -} - -variable "separator_override" { - description = "Optional separator override for specific resource types." - type = map(string) - default = {} -} - -variable "suffix" { - description = "Optional name suffix." - type = string - default = null -} - -variable "team" { - description = "Team name." - type = string -} - -variable "use_resource_prefixes" { - description = "Prefix names with the resource type." - type = bool - default = false -} diff --git a/modules/naming-convention/versions.tf b/modules/naming-convention/versions.tf deleted file mode 100644 index 4278054b1..000000000 --- a/modules/naming-convention/versions.tf +++ /dev/null @@ -1,29 +0,0 @@ -# Copyright 2022 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# https://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -terraform { - required_version = ">= 1.1.0" - required_providers { - google = { - source = "hashicorp/google" - version = ">= 4.32.0" # tftest - } - google-beta = { - source = "hashicorp/google-beta" - version = ">= 4.32.0" # tftest - } - } -} - - diff --git a/tests/modules/naming_convention/__init__.py b/tests/modules/naming_convention/__init__.py deleted file mode 100644 index 6d6d1266c..000000000 --- a/tests/modules/naming_convention/__init__.py +++ /dev/null @@ -1,13 +0,0 @@ -# Copyright 2022 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. diff --git a/tests/modules/naming_convention/fixture/main.tf b/tests/modules/naming_convention/fixture/main.tf deleted file mode 100644 index d0387f95b..000000000 --- a/tests/modules/naming_convention/fixture/main.tf +++ /dev/null @@ -1,65 +0,0 @@ -/** - * Copyright 2022 Google LLC - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -module "test" { - source = "../../../../modules/naming-convention" - prefix = var.prefix - suffix = var.suffix - use_resource_prefixes = var.use_resource_prefixes - environment = "dev" - team = "cloud" - resources = { - bucket = ["tf-org", "tf-sec", "tf-log"] - dataset = ["foobar", "frobniz"] - project = ["tf", "sec", "log"] - } - labels = { - project = { - tf = { scope = "global" } - } - } - separator_override = { - dataset = "_" - } -} - -output "labels" { - value = module.test.labels -} - -output "names" { - value = module.test.names -} - -variable "prefix" { - type = string - default = null -} - -variable "separator_override" { - type = map(string) - default = {} -} - -variable "suffix" { - type = string - default = null -} - -variable "use_resource_prefixes" { - type = bool - default = false -} diff --git a/tests/modules/naming_convention/test_plan.py b/tests/modules/naming_convention/test_plan.py deleted file mode 100644 index a8f35b891..000000000 --- a/tests/modules/naming_convention/test_plan.py +++ /dev/null @@ -1,47 +0,0 @@ -# Copyright 2022 Google LLC -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -def test_no_prefix_suffix(apply_runner): - _, output = apply_runner() - assert output['names']['project']['tf'] == 'cloud-dev-tf' - assert output['names']['bucket']['tf-org'] == 'cloud-dev-tf-org' - assert output['labels']['project']['tf'] == { - 'environment': 'dev', 'scope': 'global', 'team': 'cloud'} - assert output['labels']['bucket']['tf-org'] == { - 'environment': 'dev', 'team': 'cloud'} - - -def test_prefix(apply_runner): - _, output = apply_runner(prefix='myco') - assert output['names']['project']['tf'] == 'myco-cloud-dev-tf' - assert output['names']['bucket']['tf-org'] == 'myco-cloud-dev-tf-org' - - -def test_suffix(apply_runner): - _, output = apply_runner(suffix='myco') - assert output['names']['project']['tf'] == 'cloud-dev-tf-myco' - assert output['names']['bucket']['tf-org'] == 'cloud-dev-tf-org-myco' - - -def test_resource_prefix(apply_runner): - _, output = apply_runner(prefix='myco', - use_resource_prefixes='true') - assert output['names']['project']['tf'] == 'project-myco-cloud-dev-tf' - assert output['names']['bucket']['tf-org'] == 'bucket-myco-cloud-dev-tf-org' - - -def test_separator(apply_runner): - _, output = apply_runner(separator_override='{ dataset = "_" }') - assert output['names']['dataset'] == { - 'foobar': 'cloud_dev_foobar', 'frobniz': 'cloud_dev_frobniz'}