Add e2e example to use Cloud KMS with GCE and GCS.

KMS deployed in a separate project to reflect common architecture pattern where companies
have a centralized KMS and service projects using crypto keys.

README.md

@@ -19,8 +19,9 @@ Currently available examples:
 
 - **foundations** - [single level hierarchy](./foundations/environments/) (environments), [multiple level hierarchy](./foundations/business-units/) (business units + environments)
 - **infrastructure** - [hub and spoke via peering](./infrastructure/hub-and-spoke-peering/), [hub and spoke via VPN](./infrastructure/hub-and-spoke-vpn/), [DNS and Google Private Access for on-premises](./infrastructure/onprem-google-access-dns/), [Shared VPC with GKE support](./infrastructure/shared-vpc-gke/)
+- **data solutions** - [Cloud KMS with Cloud Storage and Compute Engine](./data-solutions/cloud-kms/)
 
-For more information see the README files in the [foundations](./foundations/) and [infrastructure](./infrastructure/) folders.
+For more information see the README files in the [foundations](./foundations/), [infrastructure](./infrastructure/) and [data solutions](./data-solutions/) folders.
 
 ## Modules
 

data-solutions/README.md

@@ -0,0 +1,14 @@
+# GCP Data Services examples
+
+The examples in this folder implement **typical data servicies topologies**, or **end-to-end scenarios** that allow testing specific features like Cloud KMS to encrypt your data or VPC-SC to mitigate data exfiltration.
+
+They are meant to be used as minimal but complete starting points to create actual infrastructure, and as playgrounds to experiment with specific Google Cloud features.
+
+## Examples
+
+### Cloud KMS with Cloud Storage and Compute Engine
+
+<a href="./cloud-kms/" title="Cloud KMS with Cloud Storage and Compute Engine"><img src="./cloud-kms/diagram.png" align="left" width="280px"></a> This [example](./cloud-kms/) implements a project to host a centralized Cloud KMS instance and a service project that will use Crypto keys to encrypt resources at rest.
+
+The sample highlights roles and permissions needed to implement the architecture described.
+<br clear="left">

data-solutions/cloud-kms/README.md

@@ -0,0 +1,65 @@
+# Cloud KMS with Cloud Storage and Compute Engine
+
+This sample creates a sample [Cloud KMS](https://cloud.google.com/kms?hl=it) configuration to be used with [Cloud Storage](https://cloud.google.com/storage/docs/encryption/using-customer-managed-keys) and [Copute Engine](https://cloud.google.com/compute/docs/disks/customer-managed-encryption).
+Cloud KMS is deployed in a separate project to highlight the IAM binding needed and to mock a more real scenario where you usually have a project to manage keys across all your projects in one single place.
+
+The sample has been purposefully kept simple so that it can be used as a basis for different and more complex configuration. This is the high level diagram:
+
+![High-level diagram](diagram.png "High-level diagram")
+
+## Managed resources and services
+
+This sample creates several distinct groups of resources:
+
+- projects
+  - Cloud KMS project
+  - Service Project configured for GCE instances and GCS buckets
+- networking
+  - VPC network
+  - One subnet
+  - Firewall rules for [SSH access via IAP](https://cloud.google.com/iap/docs/using-tcp-forwarding) and open communication within the VPC
+- IAM
+  - One service account for the GGE instance
+- KMS
+  - One key ring
+  - One crypto key (Procection level: softwere) for Cloud Engine
+  - One crypto key (Protection level: softwere) for Cloud Storage
+- GCE
+  - One instance encrypted with a CMEK Cryptokey hosted in Cloud KMS
+- GCS
+  - One bucket encrypted with a CMEK Cryptokey hosted in Cloud KMS
+
+## Accessing the bastion instance and GKE cluster
+
+The bastion VM has no public address so access is mediated via [IAP](https://cloud.google.com/iap/docs), which is supported transparently in the `gcloud compute ssh` command. Authentication is via OS Login set as a project default.
+
+Cluster access from the bastion can leverage the instance service account's `container.developer` role: the only configuration needed is to fetch cluster credentials via `gcloud container clusters get-credentials` passing the correct cluster name, location and project via command options.
+
+## Destroying
+
+There's a minor glitch that can surface running `terraform destroy`, where the service project attachments to the Shared VPC will not get destroyed even with the relevant API call succeeding. We are investigating the issue, in the meantime just manually remove the attachment in the Cloud console or via the `gcloud beta compute shared-vpc associated-projects remove` command when `terraform destroy` fails, and then relaunch the command.
+
+<!-- BEGIN TFDOC -->
+## Variables
+
+| name | description | type | required | default |
+|---|---|:---: |:---:|:---:|
+| billing_account | Billing account id used as default for new projects. | <code title="">string</code> | ✓ |  |
+| projects_parent | The resource name of the parent Folder or Organization. Must be of the form folders/folder_id or organizations/org_id. | <code title="">string</code> | ✓ |  |
+| *project_kms_name* | Name for the new KMS Project. | <code title="">string</code> |  | <code title="">my-project-kms-001</code> |
+| *project_service_name* | Name for the new Service Project. | <code title="">string</code> |  | <code title="">my-project-service-001</code> |
+| *resource_location* | The location where resources will be deployed. | <code title="">string</code> |  | <code title="">europe</code> |
+| *resource_region* | The region where resources will be deployed. | <code title="">string</code> |  | <code title="">europe-west1</code> |
+| *resource_zone* | The zone where resources will be deployed. | <code title="">string</code> |  | <code title="">europe-west1-b</code> |
+| *vpc_ip_cidr_range* | Ip range used in the subnet deployef in the Service Project. | <code title="">string</code> |  | <code title="">10.0.0.0/20</code> |
+| *vpc_name* | Name of the VPC created in the Service Project. | <code title="">string</code> |  | <code title="">local</code> |
+| *vpc_subnet_name* | Name of the subnet created in the Service Project. | <code title="">string</code> |  | <code title="">subnet</code> |
+
+## Outputs
+
+| name | description | sensitive |
+|---|---|:---:|
+| buckets_keys | GCS Buckets Cloud KMS crypto keys. |  |
+| projects | Project ids. |  |
+| vms_keys | GCE VMs Cloud KMS crypto keys. |  |
+<!-- END TFDOC -->

data-solutions/cloud-kms/backend.tf.sample

@@ -0,0 +1,20 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+terraform {
+  backend "gcs" {
+    bucket = ""
+  }
+}

data-solutions/cloud-kms/main.tf

@@ -0,0 +1,156 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+###############################################################################
+#                                   Projects                                  #
+###############################################################################
+
+module "project-service" {
+  source          = "../../modules/project"
+  name            = var.project_service_name
+  parent          = var.projects_parent
+  billing_account = var.billing_account
+  services = [
+    "compute.googleapis.com",
+    "servicenetworking.googleapis.com",
+    "storage-component.googleapis.com"
+  ]
+  oslogin = true
+}
+
+module "project-kms" {
+  source          = "../../modules/project"
+  name            = var.project_kms_name
+  parent          = var.projects_parent
+  billing_account = var.billing_account
+  services = [
+    "cloudkms.googleapis.com",
+    "servicenetworking.googleapis.com"
+  ]
+  oslogin = true
+}
+
+###############################################################################
+#                                   Networking                                #
+###############################################################################
+
+module "vpc" {
+  source     = "../../modules/net-vpc"
+  project_id = module.project-service.project_id
+  name       = var.vpc_name
+  subnets = [
+    {
+      ip_cidr_range      = var.vpc_ip_cidr_range
+      name               = var.vpc_subnet_name
+      region             = var.resource_region
+      secondary_ip_range = {}
+    }
+  ]
+}
+
+module "vpc-firewall" {
+  source               = "../../modules/net-vpc-firewall"
+  project_id           = module.project-service.project_id
+  network              = module.vpc.name
+  admin_ranges_enabled = true
+  admin_ranges         = [var.vpc_ip_cidr_range]
+  ssh_source_ranges    = ["0.0.0.0/0"]
+}
+
+###############################################################################
+#                                   KMS                                       #
+###############################################################################
+
+module "kms" {
+  source     = "../../modules/kms"
+  project_id = module.project-kms.project_id
+  keyring = {
+    name     = "my-keyring",
+    location = var.resource_location
+  }
+  keys = { key-gce = null, key-gcs = null }
+  key_iam_roles = {
+    key-gce = ["roles/cloudkms.cryptoKeyEncrypterDecrypter"]
+  }
+  key_iam_members = {
+    key-gce = {
+      "roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
+        "serviceAccount:${module.project-service.service_accounts.robots.compute}",
+      ]
+    },
+    key-gcs = {
+      "roles/cloudkms.cryptoKeyEncrypterDecrypter" = [
+        "serviceAccount:${module.project-service.service_accounts.robots.storage}",
+      ]
+    }
+  }
+}
+
+###############################################################################
+#                                   GCE                                       #
+###############################################################################
+
+module "kms_vm_example" {
+  source     = "../../modules/compute-vm"
+  project_id = module.project-service.project_id
+  region     = var.resource_region
+  zone       = var.resource_zone
+  name       = "kms-vm"
+  network_interfaces = [{
+    network    = module.vpc.self_link,
+    subnetwork = module.vpc.subnet_self_links["${var.resource_region}/subnet"],
+    nat        = false,
+    addresses  = null
+  }]
+  attached_disks = [
+    {
+      name  = "attacheddisk"
+      size  = 10
+      image = null
+      options = {
+        auto_delete = true
+        mode        = null
+        source      = null
+        type        = null
+      }
+    }
+  ]
+  instance_count = 1
+  boot_disk = {
+    image        = "projects/debian-cloud/global/images/family/debian-10"
+    type         = "pd-ssd"
+    size         = 10
+    encrypt_disk = true
+  }
+  tags = ["ssh"]
+  encryption = {
+    encrypt_boot            = true
+    disk_encryption_key_raw = null
+    kms_key_self_link       = module.kms.key_self_links.key-gce
+  }
+}
+
+###############################################################################
+#                                   GCS                                       #
+###############################################################################
+
+module "kms-gcs" {
+  source     = "../../modules/gcs"
+  project_id = module.project-service.project_id
+  prefix     = "my-bucket-001"
+  names      = ["kms-gcs"]
+  encryption_key = {
+    kms-gcs = module.kms.keys.key-gce.self_link,
+  }
+}

data-solutions/cloud-kms/outputs.tf

@@ -0,0 +1,37 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+output "projects" {
+  description = "Project ids."
+  value = {
+    service-project = module.project-service.project_id
+    kms-project     = module.project-kms.project_id
+  }
+}
+
+output "vms_keys" {
+  description = "GCE VMs Cloud KMS crypto keys."
+  value = {
+    for instance in module.kms_vm_example.instances :
+    instance.name => instance.boot_disk.0.kms_key_self_link
+  }
+}
+
+output "buckets_keys" {
+  description = "GCS Buckets Cloud KMS crypto keys."
+  value = {
+    for bucket in module.kms-gcs.buckets :
+    bucket.name => bucket.encryption
+  }
+}

data-solutions/cloud-kms/variables.tf

@@ -0,0 +1,72 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+variable "billing_account" {
+  description = "Billing account id used as default for new projects."
+  type        = string
+}
+
+variable "project_service_name" {
+  description = "Name for the new Service Project."
+  type        = string
+  default     = "my-project-service-001"
+}
+
+variable "project_kms_name" {
+  description = "Name for the new KMS Project."
+  type        = string
+  default     = "my-project-kms-001"
+}
+
+variable "projects_parent" {
+  description = "The resource name of the parent Folder or Organization. Must be of the form folders/folder_id or organizations/org_id."
+  type        = string
+}
+
+variable "resource_location" {
+  description = "The location where resources will be deployed."
+  type        = string
+  default     = "europe"
+}
+
+variable "resource_region" {
+  description = "The region where resources will be deployed."
+  type        = string
+  default     = "europe-west1"
+}
+
+variable "resource_zone" {
+  description = "The zone where resources will be deployed."
+  type        = string
+  default     = "europe-west1-b"
+}
+
+variable "vpc_name" {
+  description = "Name of the VPC created in the Service Project."
+  type        = string
+  default     = "local"
+}
+
+variable "vpc_subnet_name" {
+  description = "Name of the subnet created in the Service Project."
+  type        = string
+  default     = "subnet"
+}
+
+variable "vpc_ip_cidr_range" {
+  description = "Ip range used in the subnet deployef in the Service Project."
+  type        = string
+  default     = "10.0.0.0/20"
+}

data-solutions/cloud-kms/versions.tf

@@ -0,0 +1,17 @@
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+terraform {
+  required_version = ">= 0.12"
+}