* untested * pllan testing * fix stage 2s * move providers to their own file * single-environment stage 3 * fixes and moved blocks * stage3 factory * doc * review comments * review comments * tfdoc * fasts tage 1 tests * netsec as stage 2 * fix backported roles * fix backported roles * tfdoc * fixes * fix tag value roles in stage 1 * remove checklist, fix stage 1 tests * inventory * Small bugfix * refactor context tag values * fix previous merge * fix previous merge * fix previous merge * support short names for top level automation resources, change top level context variable * fix new top level context * roll back merge changes to stage 0 outputs * roll back more merge changes * linting errors * tfdoc * fix tests, roll back merge in tenants stage * tfdoc * fix inventory * optional stage 2 env folders and tag bindings * tflint * damn tflint * damn tflint * tfdoc * fix networking tests * tflint * fix test inventories * tfdoc * use coalesce for project parents * fix billing role conditions * fix billing role conditions * security stage tested (ngw resources need fixing/porting) * boilerplate * fix inventory * stage envs and stage linking script * initial work on resman docs, update diagram, improve teams folder * resman README * fix stage 2 IAM delegation * remove checklist from bootstrap * stage 1 tests * stage 0 1 and 2 tests * tflint * tflint * tfdoc * GCVE stage refactor (untested) * GCVE stage refactor (untested) * GCVE stage 3 * gcve tests * tflint * tfdoc * fix links * module tests * stages README * move network security to stage 2 * network security tests * replace stage links in README files * minimal netsec stage refactor * use factory for iac org policies, add configurable drs org policy for iac * test mt stage * tfdoc * fix cicd workflows * fix cicd workflows * gke-dev stage * tflint * remove data platform stage * exclude provider files via tfdoc opts * remove data platform tests and links * fix merge * fix resman inventory * boilerplate * inventory --------- Co-authored-by: Simone Ruffilli <sruffilli@google.com>
GCVE Private Cloud Minimal
This stage implements a simple architecture that integrates Google VMware Engine in a FAST organization.
The setup configured here is for a single environment in a single region, and is provided as a starting point for the more complex patterns described below in this document which can be easily implemented by extending this stage, and/or duplicating it across environments. Some configuration examples are provided in the GCVE module.
Stage configuration
Project-level IAM
Project-level IAM is controlled via the iam and iam_by_principals variables, which allow controlling authoritative bindings on the project.
To manage GCVE assign the roles/vmwareengine.vmwareengineAdmin and roles/vmwareengine.vmwareengineViewer roles to suitable groups via either of the above variables.
Networking
Any of the FAST networking stages can be used to provide prerequisites for this stage. The development spoke VPC is used by default to attach the GCVE Private Cloud. To adapt this stage to production (or to a custom VPC) simply change the configuration of the GCVE module in the main.tf file.
Peerings can be configured to additional VPCs via the network_peerings variable, provided the service account running this stage has suitable permissions on the VPCs. When running FAST, network projects matching this stage's environment already have the suitable IAM binding via the custom gcveNetworkAdmin role defined in the bootstrap stage. For custom setups outside of FAST, the VMware Engine Admin role can be used.
Architectural patterns
The patterns shown here can be achieved by combining this stage with the relevant networking stage, and configuring network peerings to achieve the desired connectivity layout. Different patterns can of course be implemented by modifying the default configuration.
Single-region shared GCVE deployment
This approach creates one GCVE deployment in a single region connected to every environment. When using a networking stage with a dedicated landing VPC as in the first two diagrams, an additional peering is created there to allow connections to the Private Cloud from on premises.
With hub and spoke networking stage.
With separate environments networking stage.
Single-region per-environment GCVE deployment
This approach creates one GCVE deployment per environment in a single region. As in the approach above, when using a networking stage with a dedicated landing VPC as in the first two diagrams, additional peerings are created there to allow connections to the Private Cloud from on premises.
With hub and spoke networking stage.
With separate environments networking stage.
Multi-regional deployments
A design for a multi-regional deployment with the NVA FAST networking stage is shown below.
How to run this stage
This stage is meant to be executed after the FAST "foundational" stages: bootstrap, resource management, security and networking stages.
It is also possible to run this stage in isolation. Refer to the Running in isolation section below for details.
Before running this stage, you need to make sure you have the correct credentials and permissions, and localize variables by assigning values that match your configuration.
Provider and Terraform variables
As all other FAST stages, the mechanism used to pass variable values and pre-built provider files from one stage to the next is also leveraged here.
The commands to link or copy the provider and terraform variable files can be easily derived from the fast-links.sh script in the FAST stages folder, passing it a single argument with the local output files folder (if configured) or the GCS output bucket in the automation project (derived from stage 0 outputs). The following examples demonstrate both cases, and the resulting commands that then need to be copy/pasted and run.
../fast-links.sh ~/fast-config
# File linking commands for GCVE (dev) stage
# provider file
ln -s ~/fast-config/fast-test-00/providers/3-gcve-dev-providers.tf ./
# input files from other stages
ln -s ~/fast-config/fast-test-00/tfvars/0-globals.auto.tfvars.json ./
ln -s ~/fast-config/fast-test-00/tfvars/0-bootstrap.auto.tfvars.json ./
ln -s ~/fast-config/fast-test-00/tfvars/1-resman.auto.tfvars.json ./
ln -s ~/fast-config/fast-test-00/tfvars/2-networking.auto.tfvars.json ./
# conventional place for stage tfvars (manually created)
ln -s ~/fast-config/fast-test-00/3-gcve-dev.auto.tfvars ./
../fast-links.sh gs://xxx-prod-iac-core-outputs-0
# File linking commands for GCVE (dev) stage
# provider file
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/3-gcve-dev-providers.tf ./
# input files from other stages
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json ./
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/1-resman.auto.tfvars.json ./
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/2-networking.auto.tfvars.json ./
# conventional place for stage tfvars (manually created)
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/3-gcve-dev.auto.tfvars ./
Impersonating the automation service account
The preconfigured provider file uses impersonation to run with this stage's automation service account's credentials. The gcp-devops and organization-admins groups have the necessary IAM bindings in place to do that, so make sure the current user is a member of one of those groups.
Variable configuration
Variables in this stage -- like most other FAST stages -- are broadly divided into three separate sets:
- variables which refer to global values for the whole organization (org id, billing account id, prefix, etc.), which are pre-populated via the
0-globals.auto.tfvars.jsonfile linked or copied above - variables which refer to resources managed by previous stage, which are prepopulated here via the
*.auto.tfvars.jsonfiles linked or copied above - and finally variables that optionally control this stage's behaviour and customizations, and can to be set in a custom
terraform.tfvarsfile
The full list can be found in the Variables table at the bottom of this document.
Running the stage
Once provider and variable values are in place and the correct user is configured, the stage can be run:
terraform init
terraform apply
Files
| name | description | modules | resources |
|---|---|---|---|
| gcve-pc.tf | GCVE private cloud resources. | gcve-private-cloud |
google_vmwareengine_network_peering |
| main.tf | Locals and project-level resources. | project |
|
| outputs.tf | Output variables. | ||
| variables-fast.tf | FAST stage interface. | ||
| variables.tf | Module variables. |
Variables
| name | description | type | required | default | producer |
|---|---|---|---|---|---|
| billing_account | Billing account id. If billing account is not part of the same org set is_org_level to false. |
object({…}) |
✓ | 0-bootstrap |
|
| environments | Long environment names. | object({…}) |
✓ | 1-resman |
|
| prefix | Prefix used for resources that need unique names. Use a maximum of 9 chars for organizations, and 11 chars for tenants. | string |
✓ | 0-bootstrap |
|
| folder_ids | Folders used by FAST stages in folders/nnnnnnnnnnn format. | map(string) |
{} |
1-resman |
|
| iam | Project-level authoritative IAM bindings for users and service accounts in {ROLE => [MEMBERS]} format. | map(list(string)) |
{} |
||
| iam_by_principals | Authoritative IAM binding in {PRINCIPAL => [ROLES]} format. Principals need to be statically defined to avoid cycle errors. Merged internally with the iam variable. |
map(list(string)) |
{} |
||
| network_peerings | The network peerings between users' VPCs and the VMware Engine networks. Key is used for the peering name suffix. Network is expanded for FAST defined networks. | map(object({…})) |
{…} |
||
| private_cloud_configs | The VMware private cloud configurations. Key is used for the private cloud name suffix. | map(object({…})) |
{} |
||
| stage_config | FAST stage configuration used to find resource ids. Must match name defined for the stage in resource management. | object({…}) |
{…} |
||
| vpc_self_links | FAST host VPC self links. | map(string) |
{} |
2-networking |
Outputs
| name | description | sensitive | consumers |
|---|---|---|---|
| network | VMware engine network. | ||
| network_peerings | The peerings created towards the user VPC or other VMware engine networks. | ||
| private_clouds | VMware engine private cloud resources. | ||
| project_id | GCVE project id. |
