Implement proper support for data access logs in resource manager modules (#1497)
* organization module * rename iam_bindings_authoritative to iam_policy, fix tests * add support for data access logs and iam policy to folder module * test inventories * add support for data access logs and iam policy to project module
This commit is contained in:
Ludovico Magnocavallo
committed by
GitHub
GitHub
parent
438a3134e2
commit
551dc581e8
@@ -11,6 +11,7 @@ This module allows the creation and management of folders, including support for
|
||||
- [Directly Defined](#directly-defined-firewall-policies)
|
||||
- [Factory](#firewall-policy-factory)
|
||||
- [Log Sinks](#log-sinks)
|
||||
- [Data Access Logs](#data-access-logs)
|
||||
- [Tags](#tags)
|
||||
|
||||
## Basic example with IAM bindings
|
||||
@@ -44,10 +45,13 @@ module "folder" {
|
||||
|
||||
## IAM
|
||||
|
||||
There are two mutually exclusive ways at the role level of managing IAM in this module
|
||||
There are three mutually exclusive ways at the role level of managing IAM in this module
|
||||
|
||||
- non-authoritative via the `iam_additive` and `iam_additive_members` variables, where bindings created outside this module will coexist with those managed here
|
||||
- authoritative via the `group_iam` and `iam` variables, where bindings created outside this module (eg in the console) will be removed at each `terraform apply` cycle if the same role is also managed here
|
||||
- authoritative policy via the `iam_policy` variable, where any binding created outside this module (eg in the console) will be removed at each `terraform apply` cycle regardless of the role
|
||||
|
||||
The authoritative and additive approaches can be used together, provided different roles are managed by each. The IAM policy is incompatible with the other approaches, and must be used with extreme care.
|
||||
|
||||
Some care must be taken with the `groups_iam` variable (and in some situations with the additive variables) to ensure that variable keys are static values, so that Terraform is able to compute the dependency graph.
|
||||
|
||||
@@ -301,6 +305,57 @@ module "folder-sink" {
|
||||
# tftest modules=5 resources=14 inventory=logging.yaml
|
||||
```
|
||||
|
||||
## Data Access Logs
|
||||
|
||||
Activation of data access logs can be controlled via the `logging_data_access` variable. If the `iam_bindings_authoritative` variable is used to set a resource-level IAM policy, the data access log configuration will also be authoritative as part of the policy.
|
||||
|
||||
This example shows how to set a non-authoritative access log configuration:
|
||||
|
||||
```hcl
|
||||
module "folder" {
|
||||
source = "./fabric/modules/folder"
|
||||
parent = "folders/657104291943"
|
||||
name = "my-folder"
|
||||
logging_data_access = {
|
||||
allServices = {
|
||||
# logs for principals listed here will be excluded
|
||||
ADMIN_READ = ["group:organization-admins@example.org"]
|
||||
}
|
||||
"storage.googleapis.com" = {
|
||||
DATA_READ = []
|
||||
DATA_WRITE = []
|
||||
}
|
||||
}
|
||||
}
|
||||
# tftest modules=1 resources=3 inventory=logging-data-access.yaml
|
||||
```
|
||||
|
||||
While this sets an authoritative policies that has exclusive control of both IAM bindings for all roles and data access log configuration, and should be used with extreme care:
|
||||
|
||||
```hcl
|
||||
module "folder" {
|
||||
source = "./fabric/modules/folder"
|
||||
parent = "folders/657104291943"
|
||||
name = "my-folder"
|
||||
iam_policy = {
|
||||
"roles/owner" = ["group:org-admins@example.com"]
|
||||
"roles/resourcemanager.folderAdmin" = ["group:org-admins@example.com"]
|
||||
"roles/resourcemanager.organizationAdmin" = ["group:org-admins@example.com"]
|
||||
"roles/resourcemanager.projectCreator" = ["group:org-admins@example.com"]
|
||||
}
|
||||
logging_data_access = {
|
||||
allServices = {
|
||||
ADMIN_READ = ["group:organization-admins@example.org"]
|
||||
}
|
||||
"storage.googleapis.com" = {
|
||||
DATA_READ = []
|
||||
DATA_WRITE = []
|
||||
}
|
||||
}
|
||||
}
|
||||
# tftest modules=1 resources=2 inventory=iam-policy.yaml
|
||||
```
|
||||
|
||||
## Tags
|
||||
|
||||
Refer to the [Creating and managing tags](https://cloud.google.com/resource-manager/docs/tags/tags-creating-and-managing) documentation for details on usage.
|
||||
@@ -341,8 +396,8 @@ module "folder" {
|
||||
| name | description | resources |
|
||||
|---|---|---|
|
||||
| [firewall-policies.tf](./firewall-policies.tf) | None | <code>google_compute_firewall_policy</code> · <code>google_compute_firewall_policy_association</code> · <code>google_compute_firewall_policy_rule</code> |
|
||||
| [iam.tf](./iam.tf) | IAM bindings, roles and audit logging resources. | <code>google_folder_iam_binding</code> · <code>google_folder_iam_member</code> |
|
||||
| [logging.tf](./logging.tf) | Log sinks and supporting resources. | <code>google_bigquery_dataset_iam_member</code> · <code>google_logging_folder_exclusion</code> · <code>google_logging_folder_sink</code> · <code>google_project_iam_member</code> · <code>google_pubsub_topic_iam_member</code> · <code>google_storage_bucket_iam_member</code> |
|
||||
| [iam.tf](./iam.tf) | IAM bindings, roles and audit logging resources. | <code>google_folder_iam_binding</code> · <code>google_folder_iam_member</code> · <code>google_folder_iam_policy</code> |
|
||||
| [logging.tf](./logging.tf) | Log sinks and supporting resources. | <code>google_bigquery_dataset_iam_member</code> · <code>google_folder_iam_audit_config</code> · <code>google_logging_folder_exclusion</code> · <code>google_logging_folder_sink</code> · <code>google_project_iam_member</code> · <code>google_pubsub_topic_iam_member</code> · <code>google_storage_bucket_iam_member</code> |
|
||||
| [main.tf](./main.tf) | Module-level locals and resources. | <code>google_essential_contacts_contact</code> · <code>google_folder</code> |
|
||||
| [organization-policies.tf](./organization-policies.tf) | Folder-level organization policies. | <code>google_org_policy_policy</code> |
|
||||
| [outputs.tf](./outputs.tf) | Module outputs. | |
|
||||
@@ -363,14 +418,16 @@ module "folder" {
|
||||
| [iam](variables.tf#L71) | IAM bindings in {ROLE => [MEMBERS]} format. | <code>map(list(string))</code> | | <code>{}</code> |
|
||||
| [iam_additive](variables.tf#L78) | Non authoritative IAM bindings, in {ROLE => [MEMBERS]} format. | <code>map(list(string))</code> | | <code>{}</code> |
|
||||
| [iam_additive_members](variables.tf#L85) | IAM additive bindings in {MEMBERS => [ROLE]} format. This might break if members are dynamic values. | <code>map(list(string))</code> | | <code>{}</code> |
|
||||
| [id](variables.tf#L92) | Folder ID in case you use folder_create=false. | <code>string</code> | | <code>null</code> |
|
||||
| [logging_exclusions](variables.tf#L98) | Logging exclusions for this folder in the form {NAME -> FILTER}. | <code>map(string)</code> | | <code>{}</code> |
|
||||
| [logging_sinks](variables.tf#L105) | Logging sinks to create for the organization. | <code title="map(object({ bq_partitioned_table = optional(bool) description = optional(string) destination = string disabled = optional(bool, false) exclusions = optional(map(string), {}) filter = string include_children = optional(bool, true) type = string }))">map(object({…}))</code> | | <code>{}</code> |
|
||||
| [name](variables.tf#L135) | Folder name. | <code>string</code> | | <code>null</code> |
|
||||
| [org_policies](variables.tf#L141) | Organization policies applied to this folder keyed by policy name. | <code title="map(object({ inherit_from_parent = optional(bool) # for list policies only. reset = optional(bool) rules = optional(list(object({ allow = optional(object({ all = optional(bool) values = optional(list(string)) })) deny = optional(object({ all = optional(bool) values = optional(list(string)) })) enforce = optional(bool) # for boolean policies only. condition = optional(object({ description = optional(string) expression = optional(string) location = optional(string) title = optional(string) }), {}) })), []) }))">map(object({…}))</code> | | <code>{}</code> |
|
||||
| [org_policies_data_path](variables.tf#L168) | Path containing org policies in YAML format. | <code>string</code> | | <code>null</code> |
|
||||
| [parent](variables.tf#L174) | Parent in folders/folder_id or organizations/org_id format. | <code>string</code> | | <code>null</code> |
|
||||
| [tag_bindings](variables.tf#L184) | Tag bindings for this folder, in key => tag value id format. | <code>map(string)</code> | | <code>null</code> |
|
||||
| [iam_policy](variables.tf#L92) | IAM authoritative policy in {ROLE => [MEMBERS]} format. Roles and members not explicitly listed will be cleared, use with extreme caution. | <code>map(list(string))</code> | | <code>null</code> |
|
||||
| [id](variables.tf#L98) | Folder ID in case you use folder_create=false. | <code>string</code> | | <code>null</code> |
|
||||
| [logging_data_access](variables.tf#L104) | Control activation of data access logs. Format is service => { log type => [exempted members]}. The special 'allServices' key denotes configuration for all services. | <code>map(map(list(string)))</code> | | <code>{}</code> |
|
||||
| [logging_exclusions](variables.tf#L119) | Logging exclusions for this folder in the form {NAME -> FILTER}. | <code>map(string)</code> | | <code>{}</code> |
|
||||
| [logging_sinks](variables.tf#L126) | Logging sinks to create for the organization. | <code title="map(object({ bq_partitioned_table = optional(bool) description = optional(string) destination = string disabled = optional(bool, false) exclusions = optional(map(string), {}) filter = string include_children = optional(bool, true) type = string }))">map(object({…}))</code> | | <code>{}</code> |
|
||||
| [name](variables.tf#L156) | Folder name. | <code>string</code> | | <code>null</code> |
|
||||
| [org_policies](variables.tf#L162) | Organization policies applied to this folder keyed by policy name. | <code title="map(object({ inherit_from_parent = optional(bool) # for list policies only. reset = optional(bool) rules = optional(list(object({ allow = optional(object({ all = optional(bool) values = optional(list(string)) })) deny = optional(object({ all = optional(bool) values = optional(list(string)) })) enforce = optional(bool) # for boolean policies only. condition = optional(object({ description = optional(string) expression = optional(string) location = optional(string) title = optional(string) }), {}) })), []) }))">map(object({…}))</code> | | <code>{}</code> |
|
||||
| [org_policies_data_path](variables.tf#L189) | Path containing org policies in YAML format. | <code>string</code> | | <code>null</code> |
|
||||
| [parent](variables.tf#L195) | Parent in folders/folder_id or organizations/org_id format. | <code>string</code> | | <code>null</code> |
|
||||
| [tag_bindings](variables.tf#L205) | Tag bindings for this folder, in key => tag value id format. | <code>map(string)</code> | | <code>null</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
@@ -380,7 +437,7 @@ module "folder" {
|
||||
| [firewall_policy_id](outputs.tf#L21) | Map of firewall policy ids created in this folder. | |
|
||||
| [folder](outputs.tf#L26) | Folder resource. | |
|
||||
| [id](outputs.tf#L31) | Fully qualified folder id. | |
|
||||
| [name](outputs.tf#L40) | Folder name. | |
|
||||
| [sink_writer_identities](outputs.tf#L45) | Writer identities created for each sink. | |
|
||||
| [name](outputs.tf#L41) | Folder name. | |
|
||||
| [sink_writer_identities](outputs.tf#L46) | Writer identities created for each sink. | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
||||
@@ -63,3 +63,34 @@ resource "google_folder_iam_member" "additive" {
|
||||
role = each.value.role
|
||||
member = each.value.member
|
||||
}
|
||||
|
||||
resource "google_folder_iam_policy" "authoritative" {
|
||||
count = var.iam_policy != null ? 1 : 0
|
||||
folder = local.folder.name
|
||||
policy_data = data.google_iam_policy.authoritative.0.policy_data
|
||||
}
|
||||
|
||||
data "google_iam_policy" "authoritative" {
|
||||
count = var.iam_policy != null ? 1 : 0
|
||||
dynamic "binding" {
|
||||
for_each = try(var.iam_policy, {})
|
||||
content {
|
||||
role = binding.key
|
||||
members = binding.value
|
||||
}
|
||||
}
|
||||
dynamic "audit_config" {
|
||||
for_each = var.logging_data_access
|
||||
content {
|
||||
service = audit_config.key
|
||||
dynamic "audit_log_configs" {
|
||||
for_each = audit_config.value
|
||||
iterator = config
|
||||
content {
|
||||
log_type = config.key
|
||||
exempted_members = config.value
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -27,6 +27,22 @@ locals {
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_folder_iam_audit_config" "default" {
|
||||
for_each = (
|
||||
var.iam_policy == null ? var.logging_data_access : {}
|
||||
)
|
||||
folder = local.folder.name
|
||||
service = each.key
|
||||
dynamic "audit_log_config" {
|
||||
for_each = each.value
|
||||
iterator = config
|
||||
content {
|
||||
log_type = config.key
|
||||
exempted_members = config.value
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "google_logging_folder_sink" "sink" {
|
||||
for_each = var.logging_sinks
|
||||
name = each.key
|
||||
|
||||
@@ -33,6 +33,7 @@ output "id" {
|
||||
value = local.folder.name
|
||||
depends_on = [
|
||||
google_folder_iam_binding.authoritative,
|
||||
google_folder_iam_policy.authoritative,
|
||||
google_org_policy_policy.default,
|
||||
]
|
||||
}
|
||||
|
||||
@@ -89,12 +89,33 @@ variable "iam_additive_members" {
|
||||
nullable = false
|
||||
}
|
||||
|
||||
variable "iam_policy" {
|
||||
description = "IAM authoritative policy in {ROLE => [MEMBERS]} format. Roles and members not explicitly listed will be cleared, use with extreme caution."
|
||||
type = map(list(string))
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "id" {
|
||||
description = "Folder ID in case you use folder_create=false."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "logging_data_access" {
|
||||
description = "Control activation of data access logs. Format is service => { log type => [exempted members]}. The special 'allServices' key denotes configuration for all services."
|
||||
type = map(map(list(string)))
|
||||
nullable = false
|
||||
default = {}
|
||||
validation {
|
||||
condition = alltrue(flatten([
|
||||
for k, v in var.logging_data_access : [
|
||||
for kk, vv in v : contains(["DATA_READ", "DATA_WRITE", "ADMIN_READ"], kk)
|
||||
]
|
||||
]))
|
||||
error_message = "Log type keys for each service can only be one of 'DATA_READ', 'DATA_WRITE', 'ADMIN_READ'."
|
||||
}
|
||||
}
|
||||
|
||||
variable "logging_exclusions" {
|
||||
description = "Logging exclusions for this folder in the form {NAME -> FILTER}."
|
||||
type = map(string)
|
||||
|
||||
Reference in New Issue
Block a user