diff --git a/CHANGELOG.md b/CHANGELOG.md index 939393558..1d1f83c36 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,8 @@ All notable changes to this project will be documented in this file. - new `iot-core` module - **incompatible change** the variables for host and service Shared VPCs have changed in the project module - **incompatible change** the variable for service identities IAM has changed in the project factory +- the `net-vpc` and `project` modules now use the beta provider for shared VPC-related resources +- add `data-catalog-policy-tag` module ## [13.0.0] - 2022-01-27 diff --git a/README.md b/README.md index 782fd5bce..609c58ffb 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Currently available modules: - **foundational** - [folder](./modules/folder), [organization](./modules/organization), [project](./modules/project), [service accounts](./modules/iam-service-account), [logging bucket](./modules/logging-bucket), [billing budget](./modules/billing-budget), [naming convention](./modules/naming-convention) - **networking** - [VPC](./modules/net-vpc), [VPC firewall](./modules/net-vpc-firewall), [VPC peering](./modules/net-vpc-peering), [VPN static](./modules/net-vpn-static), [VPN dynamic](./modules/net-vpn-dynamic), [VPN HA](./modules/net-vpn-ha), [NAT](./modules/net-cloudnat), [address reservation](./modules/net-address), [DNS](./modules/dns), [L4 ILB](./modules/net-ilb), [Service Directory](./modules/service-directory), [Cloud Endpoints](./modules/endpoints) - **compute** - [VM/VM group](./modules/compute-vm), [MIG](./modules/compute-mig), [GKE cluster](./modules/gke-cluster), [GKE nodepool](./modules/gke-nodepool), [COS container](./modules/cloud-config-container/cos-generic-metadata/) (coredns, mysql, onprem, squid) -- **data** - [GCS](./modules/gcs), [BigQuery dataset](./modules/bigquery-dataset), [Pub/Sub](./modules/pubsub), [Datafusion](./modules/datafusion), [Bigtable instance](./modules/bigtable-instance), [Cloud SQL instance](./modules/cloudsql-instance) +- **data** - [GCS](./modules/gcs), [BigQuery dataset](./modules/bigquery-dataset), [Pub/Sub](./modules/pubsub), [Datafusion](./modules/datafusion), [Bigtable instance](./modules/bigtable-instance), [Cloud SQL instance](./modules/cloudsql-instance), [Data Catalog Policy Tag](./modules/data-catalog-policy-tag) - **development** - [Cloud Source Repository](./modules/source-repository), [Container Registry](./modules/container-registry), [Artifact Registry](./modules/artifact-registry), [Apigee Organization](./modules/apigee-organization), [Apigee X Instance](./modules/apigee-x-instance) - **security** - [KMS](./modules/kms), [SecretManager](./modules/secret-manager), [VPC Service Control](./modules/vpc-sc) - **serverless** - [Cloud Function](./modules/cloud-function), [Cloud Run](./modules/cloud-run) diff --git a/modules/README.md b/modules/README.md index 9b1913c74..f22878bfc 100644 --- a/modules/README.md +++ b/modules/README.md @@ -69,6 +69,7 @@ These modules are used in the examples included in this repository. If you are u - [Pub/Sub](./pubsub) - [Bigtable instance](./bigtable-instance) - [Cloud SQL instance](./cloudsql-instance) +- [Data Catalog Policy Tag](./data-catalog-policy-tag) ## Development diff --git a/modules/data-catalog-policy-tag/README.md b/modules/data-catalog-policy-tag/README.md new file mode 100644 index 000000000..1ffd6a262 --- /dev/null +++ b/modules/data-catalog-policy-tag/README.md @@ -0,0 +1,62 @@ +# Data Catalog Module + +This module simplifies the creation of [Data Catalog](https://cloud.google.com/data-catalog) Policy Tags. Policy Tags can be used to configure [Bigquery column-level access](https://cloud.google.com/bigquery/docs/best-practices-policy-tags). + +Note: Data Catalog is still in beta, hence this module currently uses the beta provider. +## Examples + +### Simple Taxonomy with policy tags + +```hcl +module "cmn-dc" { + source = "./modules/data-catalog-policy-tag" + name = "my-datacatalog-policy-tags" + project_id = "my-project" + tags = ["low", "medium", "high"] +} +# tftest modules=1 resources=4 +``` + +### Simple Taxonomy with IAM binding + +```hcl +module "cmn-dc" { + source = "./modules/data-catalog-policy-tag" + name = "my-datacatalog-policy-tags" + project_id = "my-project" + tags = ["low", "medium", "high"] + iam = { + "roles/datacatalog.categoryAdmin" = ["group:GROUP_NAME@example.com"] + } +} +# tftest modules=1 resources=5 +``` + + +## Variables + +| name | description | type | required | default | +|---|---|:---:|:---:|:---:| +| [name](variables.tf#L59) | Name of this taxonomy. | string | ✓ | | +| [project_id](variables.tf#L70) | GCP project id. | | ✓ | | +| [activated_policy_types](variables.tf#L17) | A list of policy types that are activated for this taxonomy. | list(string) | | ["FINE_GRAINED_ACCESS_CONTROL"] | +| [description](variables.tf#L23) | Description of this taxonomy. | string | | "Taxonomy - Terraform managed" | +| [group_iam](variables.tf#L29) | Authoritative IAM binding for organization groups, in {GROUP_EMAIL => [ROLES]} format. Group emails need to be static. Can be used in combination with the `iam` variable. | map(list(string)) | | {} | +| [iam](variables.tf#L35) | IAM bindings in {ROLE => [MEMBERS]} format. | map(list(string)) | | {} | +| [iam_additive](variables.tf#L41) | IAM additive bindings in {ROLE => [MEMBERS]} format. | map(list(string)) | | {} | +| [iam_additive_members](variables.tf#L47) | IAM additive bindings in {MEMBERS => [ROLE]} format. This might break if members are dynamic values. | map(list(string)) | | {} | +| [location](variables.tf#L53) | Data Catalog Taxonomy location. | string | | "eu" | +| [prefix](variables.tf#L64) | Prefix used to generate project id and name. | string | | null | +| [tags](variables.tf#L74) | List of Data Catalog Policy tags to be created. | list(string) | | [] | + +## Outputs + +| name | description | sensitive | +|---|---|:---:| +| [tags](outputs.tf#L17) | Policy Tags. | | +| [taxonomy_id](outputs.tf#L22) | Taxonomy id. | | + + +## TODO +- Support IAM at tag level. +- Support Child policy tags \ No newline at end of file diff --git a/modules/data-catalog-policy-tag/iam.tf b/modules/data-catalog-policy-tag/iam.tf new file mode 100644 index 000000000..b7e50c0eb --- /dev/null +++ b/modules/data-catalog-policy-tag/iam.tf @@ -0,0 +1,67 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Data Catalog Taxonomy IAM definition. + +locals { + _group_iam = { + for r in local._group_iam_roles : r => [ + for k, v in var.group_iam : "group:${k}" if try(index(v, r), null) != null + ] + } + _group_iam_roles = distinct(flatten(values(var.group_iam))) + _iam_additive_member_pairs = flatten([ + for member, roles in var.iam_additive_members : [ + for role in roles : { role = role, member = member } + ] + ]) + _iam_additive_pairs = flatten([ + for role, members in var.iam_additive : [ + for member in members : { role = role, member = member } + ] + ]) + iam = { + for role in distinct(concat(keys(var.iam), keys(local._group_iam))) : + role => concat( + try(var.iam[role], []), + try(local._group_iam[role], []) + ) + } + iam_additive = { + for pair in concat(local._iam_additive_pairs, local._iam_additive_member_pairs) : + "${pair.role}-${pair.member}" => pair + } +} + +resource "google_data_catalog_taxonomy_iam_binding" "authoritative" { + provider = google-beta + for_each = local.iam + role = each.key + members = each.value + taxonomy = google_data_catalog_taxonomy.default.id +} + +resource "google_data_catalog_taxonomy_iam_member" "additive" { + provider = google-beta + for_each = ( + length(var.iam_additive) + length(var.iam_additive_members) > 0 + ? local.iam_additive + : {} + ) + role = each.value.role + member = each.value.member + taxonomy = google_data_catalog_taxonomy.default.id +} diff --git a/modules/data-catalog-policy-tag/main.tf b/modules/data-catalog-policy-tag/main.tf new file mode 100644 index 000000000..536b40b5b --- /dev/null +++ b/modules/data-catalog-policy-tag/main.tf @@ -0,0 +1,43 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Data Catalog Taxonomy definition + +locals { + name = ( + var.name != null ? var.name : "${local.prefix}taxonomy" + ) + prefix = var.prefix == null ? "" : "${var.prefix}-" +} + +resource "google_data_catalog_taxonomy" "default" { + provider = google-beta + project = var.project_id + region = var.location + display_name = local.name + description = var.description + activated_policy_types = var.activated_policy_types +} + +resource "google_data_catalog_policy_tag" "default" { + for_each = toset(var.tags) + provider = google-beta + taxonomy = google_data_catalog_taxonomy.default.id + display_name = each.key + description = "${each.key} - Terraform managed. " +} + +#TODO Add IAM at tag level diff --git a/modules/data-catalog-policy-tag/outputs.tf b/modules/data-catalog-policy-tag/outputs.tf new file mode 100644 index 000000000..fcd5cc292 --- /dev/null +++ b/modules/data-catalog-policy-tag/outputs.tf @@ -0,0 +1,25 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +output "tags" { + description = "Policy Tags." + value = { for k, v in google_data_catalog_policy_tag.default : v.id => v.name } +} + +output "taxonomy_id" { + description = "Taxonomy id." + value = google_data_catalog_taxonomy.default.id +} diff --git a/modules/data-catalog-policy-tag/variables.tf b/modules/data-catalog-policy-tag/variables.tf new file mode 100644 index 000000000..92cb6b383 --- /dev/null +++ b/modules/data-catalog-policy-tag/variables.tf @@ -0,0 +1,78 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +variable "activated_policy_types" { + description = "A list of policy types that are activated for this taxonomy." + type = list(string) + default = ["FINE_GRAINED_ACCESS_CONTROL"] +} + +variable "description" { + description = "Description of this taxonomy." + type = string + default = "Taxonomy - Terraform managed" +} + +variable "group_iam" { + description = "Authoritative IAM binding for organization groups, in {GROUP_EMAIL => [ROLES]} format. Group emails need to be static. Can be used in combination with the `iam` variable." + type = map(list(string)) + default = {} +} + +variable "iam" { + description = "IAM bindings in {ROLE => [MEMBERS]} format." + type = map(list(string)) + default = {} +} + +variable "iam_additive" { + description = "IAM additive bindings in {ROLE => [MEMBERS]} format." + type = map(list(string)) + default = {} +} + +variable "iam_additive_members" { + description = "IAM additive bindings in {MEMBERS => [ROLE]} format. This might break if members are dynamic values." + type = map(list(string)) + default = {} +} + +variable "location" { + description = "Data Catalog Taxonomy location." + type = string + default = "eu" +} + +variable "name" { + description = "Name of this taxonomy." + type = string +} + +variable "prefix" { + description = "Prefix used to generate project id and name." + type = string + default = null +} + +variable "project_id" { + description = "GCP project id." +} + +variable "tags" { + description = "List of Data Catalog Policy tags to be created." + type = list(string) + default = [] +} diff --git a/modules/data-catalog-policy-tag/versions.tf b/modules/data-catalog-policy-tag/versions.tf new file mode 100644 index 000000000..e72a78007 --- /dev/null +++ b/modules/data-catalog-policy-tag/versions.tf @@ -0,0 +1,29 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +terraform { + required_version = ">= 1.1.0" + required_providers { + google = { + source = "hashicorp/google" + version = ">= 4.0.0" + } + google-beta = { + source = "hashicorp/google-beta" + version = ">= 4.0.0" + } + } +} + +