Initial MVP for CI/CD (#608)

* preliminary support for wif in stage 0

* IAM wif role

* IAM wif role TODO

* add support for external SA IAM to SA module

* add name output to SA module

* separate cicd SA

* tfdoc

* GITLAB principal (untested)

* make GCS name output static

* outputs bucket

* fix stage 1 test

* tweak outputs

* tfdoc

* move wif_pool to automation variable

* add support for top-level and repository providers

* add missing boilerplate

* fix branchless principal

* initial workflow

* symlink provider template in stages

* remove service accounts from stage 0 cicd tfvars

* add cicd interface variable to resman stage

* fix cicd variable in resman stage

* better condition on outputs_location

* fix last change

* change outputs_location type

* revert outputs_location change

* split outputs in stage 0

* update ci/cd temporary notes

* rename additive IAM resource in SA module

* split outputs in stage 1

* remove unused locals

* fix stage 1 tests

* tfdoc

* Upload action files to outputs_bucket

* Fix tests and README

* rename template, streamline outputs

* local templates and gcs output for all stage 2

* add workflows to local output files

* Use lowercase WIF providers everywhere

* Bring back suffix for workflow files

* Remove unused files

* Update READMEs

* preliminary CI/CD implementation for stage 1

* fix stage 1

* stage 1 cicd

* tfdoc

* fix tests

* readme and links for cicd and wif

* refactor wif providers

* refactor cicd for stage 1

* fix stage 1

* wif org policies

* split identity provider configuration from cicd

* add type attribute to cicd repositories

* valid cicd repositories have a workflow template

* refactor stage 01

* fix stage 01 tests

* minimal CI/CD documentation

* better check_links error reporting

* fix links

* Added Gitlab specific configurations

Set the default issuer_uri for Gitlab. Added allowed audiences to OIDC configuration.

* Fixed TF formatting in identity providers.

* Changing identity provider audience to null

Changing identity provider audience to default to null.

* add instructions for renaming workflows

* address Julio's comments

Co-authored-by: Julio Castillo <jccb@google.com>
Co-authored-by: alexmeissner <alexmeissner@google.com>

modules/gcs/README.md

@@ -136,8 +136,8 @@ module "bucket-gcs-notification" {
 |---|---|:---:|
 | [bucket](outputs.tf#L17) | Bucket resource. |  |
 | [name](outputs.tf#L22) | Bucket name. |  |
-| [notification](outputs.tf#L26) | GCS Notification self link. |  |
-| [topic](outputs.tf#L30) | Topic ID used by GCS. |  |
-| [url](outputs.tf#L34) | Bucket URL. |  |
+| [notification](outputs.tf#L30) | GCS Notification self link. |  |
+| [topic](outputs.tf#L34) | Topic ID used by GCS. |  |
+| [url](outputs.tf#L38) | Bucket URL. |  |
 
 <!-- END TFDOC -->

modules/gcs/outputs.tf

@@ -21,7 +21,11 @@ output "bucket" {
 
 output "name" {
   description = "Bucket name."
-  value       = google_storage_bucket.bucket.name
+  value       = "${local.prefix}${lower(var.name)}"
+  depends_on = [
+    google_storage_bucket.bucket,
+    google_storage_bucket_iam_binding.bindings
+  ]
 }
 output "notification" {
   description = "GCS Notification self link."

modules/iam-service-account/README.md

@@ -31,7 +31,7 @@ module "myproject-default-service-accounts" {
 
 | name | description | resources |
 |---|---|---|
-| [iam.tf](./iam.tf) | IAM bindings. | <code>google_billing_account_iam_member</code> · <code>google_folder_iam_member</code> · <code>google_organization_iam_member</code> · <code>google_project_iam_member</code> · <code>google_service_account_iam_binding</code> · <code>google_storage_bucket_iam_member</code> |
+| [iam.tf](./iam.tf) | IAM bindings. | <code>google_billing_account_iam_member</code> · <code>google_folder_iam_member</code> · <code>google_organization_iam_member</code> · <code>google_project_iam_member</code> · <code>google_service_account_iam_binding</code> · <code>google_service_account_iam_member</code> · <code>google_storage_bucket_iam_member</code> |
 | [main.tf](./main.tf) | Module-level locals and resources. | <code>google_service_account</code> · <code>google_service_account_key</code> |
 | [outputs.tf](./outputs.tf) | Module outputs. |  |
 | [variables.tf](./variables.tf) | Module variables. |  |
@@ -41,20 +41,21 @@ module "myproject-default-service-accounts" {
 
 | name | description | type | required | default |
 |---|---|:---:|:---:|:---:|
-| [name](variables.tf#L77) | Name of the service account to create. | <code>string</code> | ✓ |  |
-| [project_id](variables.tf#L88) | Project id where service account will be created. | <code>string</code> | ✓ |  |
+| [name](variables.tf#L84) | Name of the service account to create. | <code>string</code> | ✓ |  |
+| [project_id](variables.tf#L95) | Project id where service account will be created. | <code>string</code> | ✓ |  |
 | [description](variables.tf#L17) | Optional description. | <code>string</code> |  | <code>null</code> |
 | [display_name](variables.tf#L23) | Display name of the service account to create. | <code>string</code> |  | <code>&#34;Terraform-managed.&#34;</code> |
 | [generate_key](variables.tf#L29) | Generate a key for service account. | <code>bool</code> |  | <code>false</code> |
 | [iam](variables.tf#L35) | IAM bindings on the service account in {ROLE => [MEMBERS]} format. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_billing_roles](variables.tf#L42) | Billing account roles granted to the service account, by billing account id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_folder_roles](variables.tf#L49) | Folder roles granted to the service account, by folder id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_organization_roles](variables.tf#L56) | Organization roles granted to the service account, by organization id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_project_roles](variables.tf#L63) | Project roles granted to the service account, by project id. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [iam_storage_roles](variables.tf#L70) | Storage roles granted to the service account, by bucket name. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
-| [prefix](variables.tf#L82) | Prefix applied to service account names. | <code>string</code> |  | <code>null</code> |
-| [public_keys_directory](variables.tf#L93) | Path to public keys data files to upload to the service account (should have `.pem` extension). | <code>string</code> |  | <code>&#34;&#34;</code> |
-| [service_account_create](variables.tf#L99) | Create service account. When set to false, uses a data source to reference an existing service account. | <code>bool</code> |  | <code>true</code> |
+| [iam_billing_roles](variables.tf#L42) | Billing account roles granted to this service account, by billing account id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_folder_roles](variables.tf#L49) | Folder roles granted to this service account, by folder id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_organization_roles](variables.tf#L56) | Organization roles granted to this service account, by organization id. Non-authoritative. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_project_roles](variables.tf#L63) | Project roles granted to this service account, by project id. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_sa_roles](variables.tf#L70) | Service account roles granted to this service account, by service account name. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [iam_storage_roles](variables.tf#L77) | Storage roles granted to this service account, by bucket name. | <code>map&#40;list&#40;string&#41;&#41;</code> |  | <code>&#123;&#125;</code> |
+| [prefix](variables.tf#L89) | Prefix applied to service account names. | <code>string</code> |  | <code>null</code> |
+| [public_keys_directory](variables.tf#L100) | Path to public keys data files to upload to the service account (should have `.pem` extension). | <code>string</code> |  | <code>&#34;&#34;</code> |
+| [service_account_create](variables.tf#L106) | Create service account. When set to false, uses a data source to reference an existing service account. | <code>bool</code> |  | <code>true</code> |
 
 ## Outputs
 
@@ -63,7 +64,8 @@ module "myproject-default-service-accounts" {
 | [email](outputs.tf#L17) | Service account email. |  |
 | [iam_email](outputs.tf#L25) | IAM-format service account email. |  |
 | [key](outputs.tf#L33) | Service account key. | ✓ |
-| [service_account](outputs.tf#L39) | Service account resource. |  |
-| [service_account_credentials](outputs.tf#L44) | Service account json credential templates for uploaded public keys data. |  |
+| [name](outputs.tf#L39) | Service account id. |  |
+| [service_account](outputs.tf#L44) | Service account resource. |  |
+| [service_account_credentials](outputs.tf#L49) | Service account json credential templates for uploaded public keys data. |  |
 
 <!-- END TFDOC -->

modules/iam-service-account/iam.tf

@@ -45,6 +45,13 @@ locals {
       ]
     ]
   ])
+  iam_sa_pairs = flatten([
+    for entity, roles in var.iam_sa_roles : [
+      for role in roles : [
+        { entity = entity, role = role }
+      ]
+    ]
+  ])
   iam_storage_pairs = flatten([
     for entity, roles in var.iam_storage_roles : [
       for role in roles : [
@@ -101,6 +108,16 @@ resource "google_project_iam_member" "project-roles" {
   member  = local.resource_iam_email
 }
 
+resource "google_service_account_iam_member" "additive" {
+  for_each = {
+    for pair in local.iam_sa_pairs :
+    "${pair.entity}-${pair.role}" => pair
+  }
+  service_account_id = each.value.entity
+  role               = each.value.role
+  member             = local.resource_iam_email
+}
+
 resource "google_storage_bucket_iam_member" "bucket-roles" {
   for_each = {
     for pair in local.iam_storage_pairs :

modules/iam-service-account/main.tf

@@ -21,10 +21,14 @@ locals {
     ? google_service_account_key.key["1"]
     : map("", null)
   , {})
-  prefix                    = var.prefix != null ? "${var.prefix}-" : ""
-  resource_email_static     = "${local.prefix}${var.name}@${var.project_id}.iam.gserviceaccount.com"
+  prefix                = var.prefix != null ? "${var.prefix}-" : ""
+  resource_email_static = "${local.prefix}${var.name}@${var.project_id}.iam.gserviceaccount.com"
+  resource_iam_email = (
+    local.service_account != null
+    ? "serviceAccount:${local.service_account.email}"
+    : local.resource_iam_email_static
+  )
   resource_iam_email_static = "serviceAccount:${local.resource_email_static}"
-  resource_iam_email        = local.service_account != null ? "serviceAccount:${local.service_account.email}" : local.resource_iam_email_static
   service_account = (
     var.service_account_create
     ? try(google_service_account.service_account.0, null)

modules/iam-service-account/outputs.tf

@@ -36,6 +36,11 @@ output "key" {
   value       = local.key
 }
 
+output "name" {
+  description = "Service account id."
+  value       = local.service_account.name
+}
+
 output "service_account" {
   description = "Service account resource."
   value       = local.service_account

modules/iam-service-account/variables.tf

@@ -40,35 +40,42 @@ variable "iam" {
 }
 
 variable "iam_billing_roles" {
-  description = "Billing account roles granted to the service account, by billing account id. Non-authoritative."
+  description = "Billing account roles granted to this service account, by billing account id. Non-authoritative."
   type        = map(list(string))
   default     = {}
   nullable    = false
 }
 
 variable "iam_folder_roles" {
-  description = "Folder roles granted to the service account, by folder id. Non-authoritative."
+  description = "Folder roles granted to this service account, by folder id. Non-authoritative."
   type        = map(list(string))
   default     = {}
   nullable    = false
 }
 
 variable "iam_organization_roles" {
-  description = "Organization roles granted to the service account, by organization id. Non-authoritative."
+  description = "Organization roles granted to this service account, by organization id. Non-authoritative."
   type        = map(list(string))
   default     = {}
   nullable    = false
 }
 
 variable "iam_project_roles" {
-  description = "Project roles granted to the service account, by project id."
+  description = "Project roles granted to this service account, by project id."
+  type        = map(list(string))
+  default     = {}
+  nullable    = false
+}
+
+variable "iam_sa_roles" {
+  description = "Service account roles granted to this service account, by service account name."
   type        = map(list(string))
   default     = {}
   nullable    = false
 }
 
 variable "iam_storage_roles" {
-  description = "Storage roles granted to the service account, by bucket name."
+  description = "Storage roles granted to this service account, by bucket name."
   type        = map(list(string))
   default     = {}
   nullable    = false