hunfabric/fast/stages/1-vpcsc

VPC Service Controls

This stage sets up VPC Service Controls (VPC-SC) for the whole organization and is a thing FAST-compliant wrapper on the VPC-SC module, with some minimal defaults.

• Design overview and choices
  • Single perimeter with built-in extensibility
  • Factories for VPC-SC configuration
  • Default geo-based access level
  • Ingress policy for organization-level log sinks
  • Asset Inventory for perimeter membership
• How to run this stage
  • Provider and Terraform variables
  • Impersonating the automation service account
  • Variable configuration
  • Running the stage
• Customizations
  • Access policy
  • Default perimeter
  • Access levels
  • Ingress/egress policies
• Notes
• Files
• Variables
• Outputs

Design overview and choices

The approach to VPC-SC design implemented in this stage aims at providing the simplest possible configuration that can be set to enforced mode, so as to provide immediate protection while minimizing operational complexity (which can be very high for VPC-SC).

Single perimeter with built-in extensibility

This stage uses a single VPC-SC perimeter by default, which is enough to provide protection against data exfiltration and use of credentials from outside of established boundaries, while minimizing operational toil.

The perimeter is set to dry-run mode by default, but the suggestion is to switch to enforced mode immediately after defining the initial set of access level and ingress/egress policies. This prevents the common situation where a complex design is deployed in dry-run mode, and then never enforced as the burden of addressing all violations is too high. A simpler design like the one presented here that employs very coarse access levels can be enforced quickly, and then refined iteratively as operations are streamlined and familiarity with VPC-SC quirks increases.

The stage is designed to allow defining additional perimeters via the perimeters variable, with a few caveats:

• there's no support for perimeter bridges, if those are needed they need to be integrated via code (which is easy enough to do anyway)
• resource discovery is only supported for the default perimeter, using the default key in the perimeters variable (again, that is reasonably easy to change via code if needed)
• the factory files for access levels and ingress/egress policies use the same folder regardless of perimeter, but their inclusion in a perimeter is controlled by the individual perimeters access_levels, ingress_policies, and egress_policies attributes.

Factories for VPC-SC configuration

Restricted services, access levels, ingress and egress policies can all be configured via YAML-based files, which allow intuitive editing and minimize the complexity of running operations.

The default setup only contains a single access level and an initial list of restricted services in the data/access-levels folder and the data/restricted-services.yaml file.

To configure ingress and egress policies simply add ingress_policies and/or egress_policies folders under data, or point the factories to your own folders by changing the factories_config variable values.

Examples on how to write access level and policy YAML files are provided further down in this document.

Default geo-based access level

The way we set up the perimeter for broad access is via a single geo-based access level, which is configured to allow access from one or more countries and deny all other traffic coming from outside the perimeter.

The data/access-levels/geo.yaml file serves as an example and should be edited to contain the countries you need (or replaced/removed for more granular configuration).

conditions:
  - regions:
      # replace the following country codes with those you need
      - ES
      - IT

More access levels can be of course added to better tailor the configuration to specific needs. Some use cases are addressed further down in this document.

Ingress policy for organization-level log sinks

An ingress policy that allows ingress to the perimeter for identities used in organization-level sinks is automatically created, but needs to be explicitly referenced in the perimeter via the fast-org-log-sinks key.

This only supports sinks defined in the bootstrap stage, but it can easily be used as a reference for different, specific needs (or replaced with a policy leveraging Asset Inventory for automatic inclusion of folder-level sinks).

Asset Inventory for perimeter membership

One more feature this setup provides out of the box to reduce toil, is semi-automatic resource discovery and management of perimeter membership via Cloud Asset Inventory.

This is only supported for the default perimeters, and requires this stage to be run every time new projects are created. It is mainly meant for simple installations where project churn is low and the organization is fairly stable. For large installations, direct perimeter inclusion of projects at creation time via the project factory is probably a better choice.

Resource discovery can be configured (or turned off if needed) via the resource_discovery variable.

How to run this stage

This stage is meant to be executed after the bootstrap stage has run, as it leverages the automation service account and bucket created there. It does not depend from any other stage and no other stage requires it, so it can be run in any order or even skipped entirely.

It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the previous stage for the environment requirements.

Provider and Terraform variables

As all other FAST stages, the mechanism used to pass variable values and pre-built provider files from one stage to the next is also leveraged here.

The commands to link or copy the provider and terraform variable files can be get from the fast-links.sh script in the FAST stages folder, passing it a single argument with the local output files folder or GCS output bucket. The following examples demonstrate both cases, and the resulting commands that then need to be copy/pasted and run.

../fast-links.sh ~/fast-config

# File linking commands for vpc service controls stage

# provider file
ln -s ~/fast-config/fast-test-00/providers/1-vpcsc-providers.tf ./

# input files from other stages
ln -s ~/fast-config/fast-test-00/tfvars/0-globals.auto.tfvars.json ./
ln -s ~/fast-config/fast-test-00/tfvars/0-org-setup.auto.tfvars.json ./

# conventional place for stage tfvars (manually created)
ln -s ~/fast-config/fast-test-00/1-vpcsc.auto.tfvars ./

# the outputs bucket name is in the stage 0 outputs and tfvars file
../fast-links.sh gs://xxx-prod-iac-core-outputs-0

# File linking commands for vpc service controls stage

# provider file
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/1-vpcsc-providers.tf ./

# input files from other stages
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json ./
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-org-setup.auto.tfvars.json ./

# conventional place for stage tfvars (manually created)
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/1-vpcsc.auto.tfvars ./

Impersonating the automation service account

The preconfigured provider file uses impersonation to run with this stage's automation service account's credentials. The gcp-devops and organization-admins groups have the necessary IAM bindings in place to do that, so make sure the current user is a member of one of those groups.

Variable configuration

Variables in this stage -- like most other FAST stages -- are broadly divided into three separate sets:

• variables which refer to global values for the whole organization (org id, billing account id, prefix, etc.), which are pre-populated via the 0-globals.auto.tfvars.json file linked or copied above
• variables which refer to resources managed by previous stages, which are prepopulated here via the 0-org-setup.auto.tfvars.json file linked or copied above
• and finally variables that optionally control this stage's behaviour and customizations, that you are expected to configure in a custom terraform.tfvars file

The latter set is explained in the Customization sections below, and the full list can be found in the Variables table at the bottom of this document.

Note that the outputs_location variable is disabled by default, you need to explicitly set it in your terraform.tfvars file if you want output files to be generated by this stage. This is an example of doing it:

outputs_location = "~/fast-config"

Running the stage

Once provider and variable values are in place and the correct user is configured, the stage can be run:

terraform init
terraform apply

Customizations

The stage is a thin wrapper that implements a single-perimeter design via the vpc-sc module, and most of the customizations available in the module are also available here. A few examples are provided below for ease of reference.

Access policy

The stage creates the org-level access policy by default. A pre-existing policy can instead be used by populating the access_policy variable with the policy id. In tenant-level mode this is done automatically by the FAST input/output files mechanism.

Default perimeter

The default perimeter is exposed via the perimeters.default variable which allows customizing most of its features.

The only exception is the list of restricted services, which is configured via a YAML file with a list of services. To configure restricted services edit the list in data/restricted-services.yaml, or set the list of services in the restricted_services perimeter attribute.

Note that it's not enough to define access levels and ingress/egress policies via their variables or via factory files: in order for them to be deployed they also need to be referenced by name in the perimeter via the attributes shown in this example.

perimeters = {
  default = {
    # enable access levels defined in YAML and/or variables
    access_levels    = ["geo"]
    # switch to enforced mode (defaults to true)
    dry_run          = false
    # enable egress policies defined in YAML and/or variables
    egress_policies  = []
    # enable ingress policies defined in YAML and/or variables
    # and/or the built-in ingress policy for org-level log sinks
    ingress_policies = ["fast-org-log-sinks"]
    # list resources part of this perimeter
    resources        = []
    # turn on VPC accessible services
    vpc_accessible_services = {
      allowed_services   = []
      enable_restriction = false
    }
  }
}

Access levels

Access levels can be defined via tfvars or factory files using the same mechanism in the underlying vpc-sc module.

These are example access levels defined via tfvars:

access_levels = {
  a1 = {
    conditions = [
      { members = ["user:user1@example.com"] }
    ]
  }
  a2 = {
    combining_function = "OR"
    conditions = [
      { regions = ["IT", "FR"] },
      { ip_subnetworks = ["101.101.101.0/24"] }
    ]
  }
}

And the same defined instead via factory files.

# data/access-levels/a1.yaml
conditions:
  - members:
    - "user:user1@example.com"

# data/access-levels/a2.yaml
combining_function: OR
conditions:
  - regions:
    - ES
    - IT
  - ip_subnetworks:
    - 8.8.0.0/16

Remember that in order for them to deployed, access levels need to be referenced by name in the perimeter definition.

Ingress/egress policies

Like access levels, ingress and egress policies can be defined via tfvars or factory files using the same mechanism in the underlying vpc-sc module.

This is an example ingress policy defined in yaml:

# data/ingress-policies/sa-tf-test.yaml
from:
  access_levels:
    - "*"
  identities:
    - serviceAccount:test-tf@myproject.iam.gserviceaccount.com
to:
  operations:
    - service_name: compute.googleapis.com
      method_selectors:
        - ProjectsService.Get
        - RegionsService.Get
  resources:
    - "*"

And this is an example egress policy:

# data/egress-policies/gcs-sa-foo.yaml
from:
  identities:
    - serviceAccount:foo@myproject.iam.gserviceaccount.com
to:
  operations:
    - method_selectors:
        - "*"
      service_name: storage.googleapis.com
  resources:
    - projects/123456789

Remember that in order for them to deployed, ingress/egress policies need to be referenced by name in the perimeter definition.

Notes

Some references that might be useful in setting up this stage:

• VPC SC CSCC requirements.

Files

name	description	modules	resources
main.tf	Module-level locals and resources.	projects-data-source · vpc-sc
outputs.tf	Module outputs.		google_storage_bucket_object · local_file
variables-fast.tf	None
variables.tf	Module variables.

Variables

name	description	type	required	default	producer
automation	Automation resources created by the bootstrap stage.	object({…})	✓		0-org-setup
organization	Organization details.	object({…})	✓		0-org-setup
access_levels	Access level definitions.	map(object({…}))		{}
access_policy	Access policy id (used for tenant-level VPC-SC configurations).	number		null
context	External context used in replacements.	object({…})		{}
egress_policies	Egress policy definitions that can be referenced in perimeters.	map(object({…}))		{}
factories_config	Paths to folders that enable factory functionality.	object({…})		{}
iam_principals	Org-level IAM principals.	map(string)		{}	0-org-setup
ingress_policies	Ingress policy definitions that can be referenced in perimeters.	map(object({…}))		{}
logging	Log writer identities for organization / folders.	object({…})		null	0-org-setup
outputs_location	Path where providers, tfvars files, and lists for the following stages are written. Leave empty to disable.	string		null
perimeters	Perimeter definitions.	map(object({…}))		{}
project_numbers	Project numbers.	map(number)		{}	0-org-setup
resource_discovery	Automatic discovery of perimeter projects.	object({…})		{}
root_node	Root node for the hierarchy, if running in tenant mode.	string		null	0-org-setup
service_accounts	Org-level service accounts.	map(string)		{}	0-org-setup

Outputs

name	description	sensitive	consumers
tfvars	Terraform variable files for the following stages.	✓
vpc_sc_perimeter_default	Raw default perimeter resource.	✓