Add support to attach tags to service accounts (#3008)
* Remove service account key upload. Add create_ignore_already_exists * Add tag bindings to service accounts * Add description to create_ignore_already_exists * Remove broken links
This commit is contained in:
Julio Castillo
@@ -5,7 +5,7 @@ This section provides **[networking blueprints](./networking/)** that implement
|
||||
Currently available blueprints:
|
||||
|
||||
- **apigee** - [Apigee X foundations](./apigee/apigee-x-foundations/). [Apigee Hybrid on GKE](./apigee/hybrid-gke/), [Apigee X analytics in BigQuery](./apigee/bigquery-analytics), [Apigee network patterns](./apigee/network-patterns/)
|
||||
- **cloud operations** - [Active Directory Federation Services](./cloud-operations/adfs), [Cloud Asset Inventory feeds for resource change tracking and remediation](./cloud-operations/asset-inventory-feed-remediation), [Fine-grained Cloud DNS IAM via Service Directory](./cloud-operations/dns-fine-grained-iam), [Cloud DNS & Shared VPC design](./cloud-operations/dns-shared-vpc), [Delegated Role Grants](./cloud-operations/iam-delegated-role-grants), [Network Quota Monitoring](./cloud-operations/network-quota-monitoring), [Managing on-prem service account keys by uploading public keys](./cloud-operations/onprem-sa-key-management), [Compute Image builder with Hashicorp Packer](./cloud-operations/packer-image-builder), [Packer example](./cloud-operations/packer-image-builder/packer), [Compute Engine quota monitoring](./cloud-operations/compute-quota-monitoring), [Scheduled Cloud Asset Inventory Export to Bigquery](./cloud-operations/scheduled-asset-inventory-export-bq), [Configuring workload identity federation with Terraform Cloud/Enterprise workflows](./cloud-operations/terraform-cloud-dynamic-credentials), [TCP healthcheck and restart for unmanaged GCE instances](./cloud-operations/unmanaged-instances-healthcheck), [Migrate for Compute Engine (v5) blueprints](./cloud-operations/vm-migration), [Configuring workload identity federation to access Google Cloud resources from apps running on Azure](./cloud-operations/workload-identity-federation)
|
||||
- **cloud operations** - [Active Directory Federation Services](./cloud-operations/adfs), [Cloud Asset Inventory feeds for resource change tracking and remediation](./cloud-operations/asset-inventory-feed-remediation), [Fine-grained Cloud DNS IAM via Service Directory](./cloud-operations/dns-fine-grained-iam), [Cloud DNS & Shared VPC design](./cloud-operations/dns-shared-vpc), [Delegated Role Grants](./cloud-operations/iam-delegated-role-grants), [Network Quota Monitoring](./cloud-operations/network-quota-monitoring), [Compute Image builder with Hashicorp Packer](./cloud-operations/packer-image-builder), [Packer example](./cloud-operations/packer-image-builder/packer), [Compute Engine quota monitoring](./cloud-operations/compute-quota-monitoring), [Scheduled Cloud Asset Inventory Export to Bigquery](./cloud-operations/scheduled-asset-inventory-export-bq), [Configuring workload identity federation with Terraform Cloud/Enterprise workflows](./cloud-operations/terraform-cloud-dynamic-credentials), [TCP healthcheck and restart for unmanaged GCE instances](./cloud-operations/unmanaged-instances-healthcheck), [Migrate for Compute Engine (v5) blueprints](./cloud-operations/vm-migration), [Configuring workload identity federation to access Google Cloud resources from apps running on Azure](./cloud-operations/workload-identity-federation)
|
||||
- **data solutions** - [GCE and GCS CMEK via centralized Cloud KMS](./data-solutions/cmek-via-centralized-kms), [Cloud SQL instance with multi-region read replicas](./data-solutions/cloudsql-multiregion), [Data Platform](./data-solutions/data-platform-foundations), [Minimal Data Platform](./data-solutions/data-platform-minimal), [Spinning up a foundation data pipeline on Google Cloud using Cloud Storage, Dataflow and BigQuery](./data-solutions/gcs-to-bq-with-least-privileges), [#SQL Server Always On Groups blueprint](./data-solutions/sqlserver-alwayson), [Data Playground](./data-solutions/data-playground), [MLOps with Vertex AI](./data-solutions/vertex-mlops), [Shielded Folder](./data-solutions/shielded-folder), [BigQuery ML and Vertex AI Pipeline](./data-solutions/bq-ml)
|
||||
- **factories** - [Fabric resource factories](./factories)
|
||||
- **GKE** - [Binary Authorization Pipeline Blueprint](./gke/binauthz), [Storage API](./gke/binauthz/image), [Multi-cluster mesh on GKE (fleet API)](./gke/multi-cluster-mesh-gke-fleet-api), [GKE Multitenant](../fast/stages/3-gke-dev), [Shared VPC with GKE support](./networking/shared-vpc-gke/), [GKE Autopilot](./gke/autopilot)
|
||||
|
||||
@@ -46,12 +46,6 @@ The blueprint's feed tracks changes to Google Compute instances, and the Cloud F
|
||||
|
||||
<br clear="left">
|
||||
|
||||
## On-prem Service Account key management
|
||||
|
||||
This [blueprint](./onprem-sa-key-management) shows how to manage IAM Service Account Keys by manually generating a key pair and uploading the public part of the key to GCP.
|
||||
|
||||
<br clear="left">
|
||||
|
||||
## Packer image builder
|
||||
|
||||
<a href="./packer-image-builder" title="Packer image builder"><img src="./packer-image-builder/diagram.png" align="left" width="280px"></a> This [blueprint](./packer-image-builder) shows how to deploy infrastructure for a Compute Engine image builder based on [Hashicorp's Packer tool](https://www.packer.io).
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
averbuks
|
||||
@@ -1,91 +0,0 @@
|
||||
# Managing on-prem service account keys by uploading public keys
|
||||
|
||||
When managing GCP Service Accounts with terraform, it's often a question on **how to avoid Service Account Key in the terraform state?**
|
||||
|
||||
This blueprint shows how to manage IAM Service Account Keys by manually generating a key pair and uploading the public part of the key to GCP. It has the following benefits:
|
||||
|
||||
- no [passing keys between users](https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys#pass-between-users) or systems
|
||||
- no private keys stored in the terraform state (only public part of the key is in the state)
|
||||
- let keys [expire automatically](https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys#key-expiryhaving)
|
||||
|
||||
|
||||
## Running the blueprint
|
||||
|
||||
Clone this repository or [open it in cloud shell](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2Fterraform-google-modules%2Fcloud-foundation-fabric&cloudshell_print=cloud-shell-readme.txt&cloudshell_working_dir=blueprints%2Fcloud-operations%2Fonprem-sa-key-management&cloudshell_open_in_editor=cloudshell_open%2Fcloud-foundation-fabric%2Fblueprints%2Fcloud-operations%2Fonprem-sa-key-management%2Fvariables.tf), then go through the following steps to create resources:
|
||||
|
||||
Cleaning up blueprint keys
|
||||
```bash
|
||||
rm -f /public-keys/data-uploader/
|
||||
rm -f /public-keys/prisma-security/
|
||||
```
|
||||
|
||||
Generate keys for service accounts
|
||||
```bash
|
||||
mkdir keys && cd keys
|
||||
openssl req -x509 -nodes -newkey rsa:2048 -days 30 \
|
||||
-keyout data_uploader_private_key.pem \
|
||||
-out ../public-keys/data-uploader/public_key.pem \
|
||||
-subj "/CN=unused"
|
||||
openssl req -x509 -nodes -newkey rsa:2048 -days 30 \
|
||||
-keyout prisma_security_private_key.pem \
|
||||
-out ../public-keys/prisma-security/public_key.pem \
|
||||
-subj "/CN=unused"
|
||||
```
|
||||
|
||||
Deploy service accounts and keys
|
||||
```bash
|
||||
cd ..
|
||||
terraform init
|
||||
terraform apply -var project_id=$GOOGLE_CLOUD_PROJECT
|
||||
|
||||
```
|
||||
|
||||
Extract JSON credentials templates from terraform output and put the private part of the keys into templates
|
||||
```bash
|
||||
terraform show -json | jq '.values.outputs."sa-credentials".value."data-uploader"."public_key.pem" | fromjson' > data-uploader.json
|
||||
terraform show -json | jq '.values.outputs."sa-credentials".value."prisma-security"."public_key.pem" | fromjson' > prisma-security.json
|
||||
|
||||
contents=$(jq --arg key "$(cat keys/data_uploader_private_key.pem)" '.private_key=$key' data-uploader.json) && echo "$contents" > data-uploader.json
|
||||
contents=$(jq --arg key "$(cat keys/prisma_security_private_key.pem)" '.private_key=$key' prisma-security.json) && echo "$contents" > prisma-security.json
|
||||
```
|
||||
|
||||
## Testing the blueprint
|
||||
Validate that service accounts json credentials are valid
|
||||
```bash
|
||||
gcloud auth activate-service-account --key-file prisma-security.json
|
||||
gcloud auth activate-service-account --key-file data-uploader.json
|
||||
```
|
||||
|
||||
## Cleaning up
|
||||
```bash
|
||||
terraform destroy -var project_id=$GOOGLE_CLOUD_PROJECT
|
||||
```
|
||||
<!-- BEGIN TFDOC -->
|
||||
|
||||
## Variables
|
||||
|
||||
| name | description | type | required | default |
|
||||
|---|---|:---:|:---:|:---:|
|
||||
| [project_id](variables.tf#L23) | Project id. | <code>string</code> | ✓ | |
|
||||
| [project_create](variables.tf#L17) | Create project instead of using an existing one. | <code>bool</code> | | <code>false</code> |
|
||||
| [service_accounts](variables.tf#L28) | List of service accounts. | <code title="list(object({ name = string iam_project_roles = list(string) public_keys_path = string }))">list(object({…}))</code> | | <code title="[ { name = "data-uploader" iam_project_roles = [ "roles/bigquery.dataOwner", "roles/bigquery.jobUser", "roles/storage.objectAdmin" ] public_keys_path = "public-keys/data-uploader/" }, { name = "prisma-security" iam_project_roles = [ "roles/iam.securityReviewer" ] public_keys_path = "public-keys/prisma-security/" }, ]">[…]</code> |
|
||||
| [services](variables.tf#L56) | Service APIs to enable. | <code>list(string)</code> | | <code>[]</code> |
|
||||
|
||||
## Outputs
|
||||
|
||||
| name | description | sensitive |
|
||||
|---|---|:---:|
|
||||
| [sa-credentials](outputs.tf#L17) | SA json key templates. | |
|
||||
|
||||
<!-- END TFDOC -->
|
||||
|
||||
## Test
|
||||
|
||||
```hcl
|
||||
module "test" {
|
||||
source = "./fabric/blueprints/cloud-operations/onprem-sa-key-management"
|
||||
project_create = true
|
||||
project_id = "test"
|
||||
}
|
||||
# tftest modules=4 resources=7
|
||||
```
|
||||
@@ -1,23 +0,0 @@
|
||||
# Copyright 2023 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
# set a valid bucket below and rename this file to backend.tf
|
||||
|
||||
terraform {
|
||||
backend "gcs" {
|
||||
bucket = ""
|
||||
prefix = "fabric/operations/onprem-sa-key-management"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
|
||||
|
||||
################################# Quickstart #################################
|
||||
|
||||
# cleaning up example keys
|
||||
|
||||
- rm -f /public-keys/data-uploader/
|
||||
- rm -f /public-keys/prisma-security/
|
||||
|
||||
# generate keys for service accounts
|
||||
|
||||
- mkdir keys && cd keys
|
||||
- openssl req -x509 -nodes -newkey rsa:2048 -days 30 \
|
||||
-keyout data_uploader_private_key.pem \
|
||||
-out ../public-keys/data-uploader/public_key.pem \
|
||||
-subj "/CN=unused"
|
||||
- openssl req -x509 -nodes -newkey rsa:2048 -days 30 \
|
||||
-keyout prisma_security_private_key.pem \
|
||||
-out ../public-keys/prisma-security/public_key.pem \
|
||||
-subj "/CN=unused"
|
||||
|
||||
# deploy service accounts and keys
|
||||
|
||||
- cd ..
|
||||
- terraform init
|
||||
- terraform apply -var project_id=$GOOGLE_CLOUD_PROJECT
|
||||
|
||||
|
||||
# extract JSON credentials templates from terraform output and put the private part of the keys into templates
|
||||
|
||||
- terraform show -json | jq '.values.outputs."sa-credentials".value."data-uploader"."public_key.pem" | fromjson' > data-uploader.json
|
||||
- terraform show -json | jq '.values.outputs."sa-credentials".value."prisma-security"."public_key.pem" | fromjson' > prisma-security.json
|
||||
|
||||
- contents=$(jq --arg key "$(cat keys/data_uploader_private_key.pem)" '.private_key=$key' data-uploader.json) && echo "$contents" > data-uploader.json
|
||||
- contents=$(jq --arg key "$(cat keys/prisma_security_private_key.pem)" '.private_key=$key' prisma-security.json) && echo "$contents" > prisma-security.json
|
||||
|
||||
|
||||
|
||||
# validate that service accounts json credentials are valid
|
||||
|
||||
- gcloud auth activate-service-account --key-file prisma-security.json
|
||||
- gcloud auth activate-service-account --key-file data-uploader.json
|
||||
|
||||
|
||||
# cleaning up
|
||||
- terraform destroy -var project_id=$GOOGLE_CLOUD_PROJECT
|
||||
@@ -1,37 +0,0 @@
|
||||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
locals {
|
||||
service_accounts = { for sa in var.service_accounts : sa.name => sa }
|
||||
}
|
||||
|
||||
module "project" {
|
||||
source = "../../../modules/project"
|
||||
name = var.project_id
|
||||
project_reuse = var.project_create != true ? {} : null
|
||||
services = var.services
|
||||
}
|
||||
|
||||
module "integration-sa" {
|
||||
source = "../../../modules/iam-service-account"
|
||||
for_each = local.service_accounts
|
||||
project_id = module.project.project_id
|
||||
name = each.value.name
|
||||
iam_project_roles = {
|
||||
(module.project.project_id) = each.value.iam_project_roles
|
||||
}
|
||||
public_keys_directory = each.value.public_keys_path
|
||||
}
|
||||
@@ -1,20 +0,0 @@
|
||||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
output "sa-credentials" {
|
||||
description = "SA json key templates."
|
||||
value = { for key, value in module.integration-sa : key => value.service_account_credentials }
|
||||
}
|
||||
@@ -1,17 +0,0 @@
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIICnjCCAYYCCQDhgw8htVCGmTANBgkqhkiG9w0BAQsFADARMQ8wDQYDVQQDDAZ1
|
||||
bnVzZWQwHhcNMjExMjA2MDgwNTAyWhcNMzExMjA0MDgwNTAyWjARMQ8wDQYDVQQD
|
||||
DAZ1bnVzZWQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC0xlwdjkBS
|
||||
1ovANJ1RXKpFdbPQWYlqKUUo+/KLClNYC9KxRqrc+u5FtPIdCPv5WRH5sz+z8gcf
|
||||
3zJMht0dO7fOwJ9wSDKzvHkMUdXTGBPbm2i9PNA6f+YEwJQjWJlAHFH4Lp3x6ddT
|
||||
4KO4FRQEkN/5V1+sfmyGGFaSXaoi+PcDcQHvfUUlp5iyX4I+8tqwh1kdg1M5orkE
|
||||
7iBG0wHWzfOSmZq5in6t9+lWzOZeYapi8bVBm7Vz+dmHZPKS6EGmAXS1wpLCSKHB
|
||||
uv23KXY4gAXOPHiDI70JpeNiSJBE9WgXs+nL78vNjLTvDhpC10b9nOxLjRc6wA5b
|
||||
3q2Am0dW1DPRAgMBAAEwDQYJKoZIhvcNAQELBQADggEBAJqyTIibNZM/q30Fn+vR
|
||||
V9q++19CIervZig1uCarH1M86cpPYRfKcYHOi6tnoCTL9VG8Ky8pbmkZNkET7vnN
|
||||
OQirpsPmqu3d+FBoqXUt8w1mT1JVr0YiTo3i07zTH8rvQKHjEfPxR73IAyYNvJ3D
|
||||
k3SdUvU3xXOa+otOQcBKIxX6mJPLhzXgZd144KCfD95qOvpoQOsNW4UWXZ3sPC0k
|
||||
VcMlN5O8/+D65y63nNtyECXvLicLdn/cdpA2H7Tqhz2ZZR+6tLcDW1kSsA8b6+rQ
|
||||
1IaKpF+TYo0jMD+WLatRrOHXOWije8871zooAXq9MLVJrT889TdsmEIYT7YPWIeJ
|
||||
Jcg=
|
||||
-----END CERTIFICATE-----
|
||||
@@ -1,17 +0,0 @@
|
||||
-----BEGIN CERTIFICATE-----
|
||||
MIICnjCCAYYCCQDXMv59IiZqfTANBgkqhkiG9w0BAQsFADARMQ8wDQYDVQQDDAZ1
|
||||
bnVzZWQwHhcNMjExMjA2MDgwNjE3WhcNMzExMjA0MDgwNjE3WjARMQ8wDQYDVQQD
|
||||
DAZ1bnVzZWQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDOK6XwgTzL
|
||||
icSITBrQBmhnYNOuggDhQr40j8/pIuTOiFZbd+ne3MhcFxpE58T9cOXgR0i/S4ok
|
||||
+kcGE74H2U7RsRpNi7fJhi62T9e2CXpibURQNJD6y0lXBQkfx6kCrhyvXqHbTxm5
|
||||
J0f5mpLlze+w7ATikmYI0mrU9XjtnRJOdxtGfiIaQ/suGTaZ0z4tZgAXy9RnwUAb
|
||||
LPXn0BD1+GYpCs82+1q7HpMIf343VRH0AdsQJteQSj5LKfaZZTNUF9NIgKtMylck
|
||||
z0Pt8TmBU0GtJX/XkSWCwMUdqdedXkvhY1XoAZPjaEBSSwq6P15PmdpDq9q2TYjD
|
||||
8U3kuCX0AlWjAgMBAAEwDQYJKoZIhvcNAQELBQADggEBADAmjI1sg150MK97DCSl
|
||||
d5OpEShCypaEZSLb/mFONW6mTX2OSdF9ipd9B07BQ2DrL8Xou2/V1aDtQZOWPIGu
|
||||
Hlm1LKw8sZY2rWX0Rq/v/NxY5iGRlwPMh7Rn9fnpHgaC1PktoDJEcvNMpzBjtfKn
|
||||
beKP9MNSChAFTTbJVWO5xT/ljE/yoPL3jyJKzKHH7y7AfbonrbQjAENbX/WCRYh3
|
||||
zOEWZG/fusRcKkZ/cO7wFFP1gzJFE9wFRu7LOA/FntCixtVSnclsOnunQfqQEVmp
|
||||
Y0IjfceIerJysCTo0I5HfRw0DOFfZimallOa4Mv5BDmzMWWyX9TvppHCnmqvM2El
|
||||
ISY=
|
||||
-----END CERTIFICATE-----
|
||||
@@ -1,60 +0,0 @@
|
||||
/**
|
||||
* Copyright 2022 Google LLC
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
variable "project_create" {
|
||||
description = "Create project instead of using an existing one."
|
||||
type = bool
|
||||
default = false
|
||||
}
|
||||
|
||||
variable "project_id" {
|
||||
description = "Project id."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "service_accounts" {
|
||||
description = "List of service accounts."
|
||||
type = list(object({
|
||||
name = string
|
||||
iam_project_roles = list(string)
|
||||
public_keys_path = string
|
||||
}))
|
||||
default = [
|
||||
{
|
||||
name = "data-uploader"
|
||||
iam_project_roles = [
|
||||
"roles/bigquery.dataOwner",
|
||||
"roles/bigquery.jobUser",
|
||||
"roles/storage.objectAdmin"
|
||||
]
|
||||
public_keys_path = "public-keys/data-uploader/"
|
||||
},
|
||||
{
|
||||
name = "prisma-security"
|
||||
iam_project_roles = [
|
||||
"roles/iam.securityReviewer"
|
||||
]
|
||||
public_keys_path = "public-keys/prisma-security/"
|
||||
},
|
||||
]
|
||||
|
||||
}
|
||||
|
||||
variable "services" {
|
||||
description = "Service APIs to enable."
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
Reference in New Issue
Block a user