kovagoadi/hunfabric
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

rename dataplex aspects module (#3053)

Browse Source
This commit is contained in:
Ludovico Magnocavallo
2025-04-22 15:06:40 +02:00
committed by GitHub
parent a39874413d
commit 9e6d1030d0
8 changed files with 4 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

167
modules/dataplex-aspect-types/README.md Normal file
View File

@@ -0,0 +1,167 @@
# Dataplex Aspect Types Module
This module allows managing [Dataplex Aspect Types](https://cloud.google.com/dataplex/docs/enrich-entries-metadata) and their associated IAM bindings via variables and YAML files defined via a resource factory.
The module manages Aspect Types for a single location in a single project. To manage them in different locations invoke the module multiple times, or use it with a `for_each` on locations/projects.
<!-- BEGIN TOC -->
- [Simple example](#simple-example)
- [Factory example](#factory-example)
- [Variables](#variables)
- [Outputs](#outputs)
<!-- END TOC -->
## Simple example
This example mirrors the one in the [`google_dataplex_aspect_type`](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dataplex_aspect_type) resource documentation, but also shows how to manage IAM on the single aspect type. More types can of course be defined by just adding them to the `aspect_types` map.
```hcl
module "aspect-types" {
source = "./fabric/modules/dataplex-aspect-types"
project_id = "test-project"
# var.location defaults to "global"
# location = "global"
aspect_types = {
tf-test-template = {
display_name = "Test template."
iam = {
"roles/dataplex.aspectTypeOwner" = ["group:data-owners@example.com"]
}
iam_bindings_additive = {
user = {
role = "roles/dataplex.aspectTypeUser"
member = "serviceAccount:sa-0@test-project.iam.gserviceaccount.com"
}
}
metadata_template = <<END
{
"name": "tf-test-template",
"type": "record",
"recordFields": [
{
"name": "type",
"type": "enum",
"annotations": {
"displayName": "Type",
"description": "Specifies the type of view represented by the entry."
},
"index": 1,
"constraints": {
"required": true
},
"enumValues": [
{
"name": "VIEW",
"index": 1
}
]
}
]
}
END
}
}
}
# tftest modules=1 resources=3
```
## Factory example
Aspect types can also be defined via a resource factory, where the file name will be used as the aspect type id. The resulting data is then internally combined with the `aspect_types` variable.
```hcl
module "aspect-types" {
source = "./fabric/modules/dataplex-aspect-types"
project_id = "test-project"
factories_config = {
aspect_types = "data/aspect-types"
}
}
# tftest modules=1 resources=4 files=aspect-0,aspect-1
```
```yaml
display_name: "Test template 0."
iam:
roles/dataplex.aspectTypeOwner:
- "group:data-owners@example.com"
metadata_template: |
{
"name": "tf-test-template-0",
"type": "record",
"recordFields": [
{
"name": "type",
"type": "enum",
"annotations": {
"displayName": "Type",
"description": "Specifies the type of view represented by the entry."
},
"index": 1,
"constraints": {
"required": true
},
"enumValues": [
{
"name": "VIEW",
"index": 1
}
]
}
]
}
# tftest-file id=aspect-0 path=data/aspect-types/aspect-0.yaml schema=aspect-type.schema.json
```
```yaml
display_name: "Test template 1."
iam_bindings_additive:
user:
role: "roles/dataplex.aspectTypeUser"
member: "serviceAccount:sa-0@test-project.iam.gserviceaccount.com"
metadata_template: |
{
"name": "tf-test-template-1",
"type": "record",
"recordFields": [
{
"name": "type",
"type": "enum",
"annotations": {
"displayName": "Type",
"description": "Specifies the type of view represented by the entry."
},
"index": 1,
"constraints": {
"required": true
},
"enumValues": [
{
"name": "VIEW",
"index": 1
}
]
}
]
}
# tftest-file id=aspect-1 path=data/aspect-types/aspect-1.yaml schema=aspect-type.schema.json
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [project_id](variables.tf#L64) | Project id where resources will be created. | <code>string</code> | ✓ | |
| [aspect_types](variables.tf#L17) | Aspect templates. Merged with those defined via the factory. | <code title="map&#40;object&#40;&#123;&#10; description &#61; optional&#40;string&#41;&#10; display_name &#61; optional&#40;string&#41;&#10; labels &#61; optional&#40;map&#40;string&#41;, &#123;&#125;&#41;&#10; metadata_template &#61; optional&#40;string&#41;&#10; iam &#61; optional&#40;map&#40;list&#40;string&#41;&#41;, &#123;&#125;&#41;&#10; iam_bindings &#61; optional&#40;map&#40;object&#40;&#123;&#10; members &#61; list&#40;string&#41;&#10; role &#61; string&#10; condition &#61; optional&#40;object&#40;&#123;&#10; expression &#61; string&#10; title &#61; string&#10; description &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10; iam_bindings_additive &#61; optional&#40;map&#40;object&#40;&#123;&#10; member &#61; string&#10; role &#61; string&#10; condition &#61; optional&#40;object&#40;&#123;&#10; expression &#61; string&#10; title &#61; string&#10; description &#61; optional&#40;string&#41;&#10; &#125;&#41;&#41;&#10; &#125;&#41;&#41;, &#123;&#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
| [factories_config](variables.tf#L48) | Paths to folders for the optional factories. | <code title="object&#40;&#123;&#10; aspect_types &#61; optional&#40;string&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [location](variables.tf#L57) | Location for aspect types. | <code>string</code> | | <code>&#34;global&#34;</code> |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| [ids](outputs.tf#L17) | Aspect type IDs. | |
| [names](outputs.tf#L29) | Aspect type names. | |
| [timestamps](outputs.tf#L41) | Aspect type create and update timestamps. | |
| [uids](outputs.tf#L56) | Aspect type gobally unique IDs. | |
<!-- END TFDOC -->

89
modules/dataplex-aspect-types/iam.tf Normal file
View File

@@ -0,0 +1,89 @@
/**
* Copyright 2023 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
iam = flatten([
for k, v in local.aspect_types : [
for role, members in v.iam : {
aspect_type_id = k
role = role
members = members
}
]
])
iam_bindings = merge([
for k, v in local.aspect_types : {
for binding_key, data in v.iam_bindings :
binding_key => {
aspect_type_id = k
role = data.role
members = data.members
condition = data.condition
}
}
]...)
iam_bindings_additive = merge([
for k, v in local.aspect_types : {
for binding_key, data in v.iam_bindings_additive :
binding_key => {
aspect_type_id = k
role = data.role
member = data.member
condition = data.condition
}
}
]...)
}
resource "google_dataplex_aspect_type_iam_binding" "authoritative" {
for_each = {
for binding in local.iam :
"${binding.aspect_type_id}.${binding.role}" => binding
}
role = each.value.role
aspect_type_id = google_dataplex_aspect_type.default[each.value.aspect_type_id].id
members = each.value.members
}
resource "google_dataplex_aspect_type_iam_binding" "bindings" {
for_each = local.iam_bindings
role = each.value.role
aspect_type_id = google_dataplex_aspect_type.default[each.value.aspect_type_id].id
members = each.value.members
dynamic "condition" {
for_each = each.value.condition == null ? [] : [""]
content {
expression = each.value.condition.expression
title = each.value.condition.title
description = each.value.condition.description
}
}
}
resource "google_dataplex_aspect_type_iam_member" "members" {
for_each = local.iam_bindings_additive
aspect_type_id = google_dataplex_aspect_type.default[each.value.aspect_type_id].id
role = each.value.role
member = each.value.member
dynamic "condition" {
for_each = each.value.condition == null ? [] : [""]
content {
expression = each.value.condition.expression
title = each.value.condition.title
description = each.value.condition.description
}
}
}

51
modules/dataplex-aspect-types/main.tf Normal file
View File

@@ -0,0 +1,51 @@
/**
* Copyright 2025 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
_factory_path = try(pathexpand(var.factories_config.aspect_types), null)
_factory_data_raw = {
for f in try(fileset(local._factory_path, "**/*.yaml"), []) :
trimsuffix(basename(f), ".yaml") => yamldecode(file("${local._factory_path}/${f}"))
}
aspect_types = merge(var.aspect_types, {
for k, v in local._factory_data_raw : k => {
description = lookup(v, "description", null)
display_name = lookup(v, "display_name", null)
iam = lookup(v, "iam", {})
iam_bindings = {
for ik, iv in lookup(v, "iam_bindings", {}) :
ik => merge({ condition = null }, iv)
}
iam_bindings_additive = {
for ik, iv in lookup(v, "iam_bindings_additive", {}) :
ik => merge({ condition = null }, iv)
}
labels = lookup(v, "labels", {})
metadata_template = lookup(v, "metadata_template", null)
}
})
}
resource "google_dataplex_aspect_type" "default" {
for_each = local.aspect_types
project = var.project_id
location = var.location
aspect_type_id = each.key
description = each.value.description
display_name = each.value.display_name
labels = each.value.labels
metadata_template = each.value.metadata_template
}

66
modules/dataplex-aspect-types/outputs.tf Normal file
View File

@@ -0,0 +1,66 @@
/**
* Copyright 2025 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "ids" {
description = "Aspect type IDs."
value = {
for k, v in google_dataplex_aspect_type.default : k => v.id
}
depends_on = [
google_dataplex_aspect_type_iam_binding.authoritative,
google_dataplex_aspect_type_iam_binding.bindings,
google_dataplex_aspect_type_iam_member.members
]
}
output "names" {
description = "Aspect type names."
value = {
for k, v in google_dataplex_aspect_type.default : k => v.name
}
depends_on = [
google_dataplex_aspect_type_iam_binding.authoritative,
google_dataplex_aspect_type_iam_binding.bindings,
google_dataplex_aspect_type_iam_member.members
]
}
output "timestamps" {
description = "Aspect type create and update timestamps."
value = {
for k, v in google_dataplex_aspect_type.default : k => {
create = v.create_time
update = v.update_time
}
}
depends_on = [
google_dataplex_aspect_type_iam_binding.authoritative,
google_dataplex_aspect_type_iam_binding.bindings,
google_dataplex_aspect_type_iam_member.members
]
}
output "uids" {
description = "Aspect type gobally unique IDs."
value = {
for k, v in google_dataplex_aspect_type.default : k => v.uid
}
depends_on = [
google_dataplex_aspect_type_iam_binding.authoritative,
google_dataplex_aspect_type_iam_binding.bindings,
google_dataplex_aspect_type_iam_member.members
]
}

125
modules/dataplex-aspect-types/schemas/aspect-type.schema.json Normal file
View File

@@ -0,0 +1,125 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Dataplex Aspect Type",
"type": "object",
"additionalProperties": false,
"properties": {
"description": {
"type": "string"
},
"display_name": {
"type": "string"
},
"labels": {
"type": "object"
},
"metadata_template": {
"type": "string"
},
"iam": {
"$ref": "#/$defs/iam"
},
"iam_bindings": {
"$ref": "#/$defs/iam_bindings"
},
"iam_bindings_additive": {
"$ref": "#/$defs/iam_bindings_additive"
}
},
"$defs": {
"iam": {
"type": "object",
"additionalProperties": false,
"patternProperties": {
"^roles/": {
"type": "array",
"items": {
"type": "string",
"pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])"
}
}
}
},
"iam_bindings": {
"type": "object",
"additionalProperties": false,
"patternProperties": {
"^[a-z0-9_-]+$": {
"type": "object",
"additionalProperties": false,
"properties": {
"members": {
"type": "array",
"items": {
"type": "string",
"pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])"
}
},
"role": {
"type": "string",
"pattern": "^roles/"
},
"condition": {
"type": "object",
"additionalProperties": false,
"required": [
"expression",
"title"
],
"properties": {
"expression": {
"type": "string"
},
"title": {
"type": "string"
},
"description": {
"type": "string"
}
}
}
}
}
}
},
"iam_bindings_additive": {
"type": "object",
"additionalProperties": false,
"patternProperties": {
"^[a-z0-9_-]+$": {
"type": "object",
"additionalProperties": false,
"properties": {
"member": {
"type": "string",
"pattern": "^(?:domain:|group:|serviceAccount:|user:|principal:|principalSet:|[a-z])"
},
"role": {
"type": "string",
"pattern": "^roles/"
},
"condition": {
"type": "object",
"additionalProperties": false,
"required": [
"expression",
"title"
],
"properties": {
"expression": {
"type": "string"
},
"title": {
"type": "string"
},
"description": {
"type": "string"
}
}
}
}
}
}
}
}
}

67
modules/dataplex-aspect-types/variables.tf Normal file
View File

@@ -0,0 +1,67 @@
/**
* Copyright 2025 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "aspect_types" {
description = "Aspect templates. Merged with those defined via the factory."
type = map(object({
description = optional(string)
display_name = optional(string)
labels = optional(map(string), {})
metadata_template = optional(string)
iam = optional(map(list(string)), {})
iam_bindings = optional(map(object({
members = list(string)
role = string
condition = optional(object({
expression = string
title = string
description = optional(string)
}))
})), {})
iam_bindings_additive = optional(map(object({
member = string
role = string
condition = optional(object({
expression = string
title = string
description = optional(string)
}))
})), {})
}))
nullable = false
default = {}
}
variable "factories_config" {
description = "Paths to folders for the optional factories."
type = object({
aspect_types = optional(string)
})
nullable = false
default = {}
}
variable "location" {
description = "Location for aspect types."
type = string
nullable = false
default = "global"
}
variable "project_id" {
description = "Project id where resources will be created."
type = string
}