diff --git a/fast/stages/02-networking-nva/README.md b/fast/stages/02-networking-nva/README.md new file mode 100644 index 000000000..fb6286e49 --- /dev/null +++ b/fast/stages/02-networking-nva/README.md @@ -0,0 +1,350 @@ +# Networking with Network Virtual Appliance + +This stage sets up the shared network infrastructure for the whole organization. +It is an alternative to the [02-networking stage](../02-networking/README.md). + +It is designed for those who would like to leverage Network Virtual Appliances (NVAs) between trusted and untrusted areas of the network, for example for Intrusion Prevention System (IPS) purposes. + +It adopts the common “hub and spoke” reference design, which is well suited for multiple scenarios, and it offers several advantages versus other designs: + +- the "trusted hub" VPC centralizes the external connectivity towards trusted network resources (e.g. on-prem, other cloud environments and the spokes), and it is ready to host cross-environment services like CI/CD, code repositories, and monitoring probes +- the "spoke" VPCs allow partitioning workloads (e.g. by environment like in this setup), while still retaining controlled access to central connectivity and services +- Shared VPCs -both in hub and spokes- split the management of the network resources into specific (host) projects, while still allowing them to be consumed from the workload (service) projects +- the design facilitates DNS centralization + +Connectivity between the hub and the spokes is established via [VPC network peerings](https://cloud.google.com/vpc/docs/vpc-peering), which offer uncapped bandwidth, lower latencies, at no additional costs and with a very low management overhead. Different ways of implementing connectivity, and related some pros and cons, are discussed below. + +The diagram shows the high-level design and it should be used as a reference throughout the following sections. + +The final number of subnets, and their IP addressing will depend on the user-specific requirements. It can be easily changed via variables or external data files, without any need to edit the code. + +

+ Networking diagram +

+ +## Design overview and choices + +### Multi-regional deployment + +The stage deploys the the infrastructure in two regions. By default, europe-west1 and europe-west4. Regional resources include NVAs (templates, MIGs, ILBs) and test VMs. +This provides enough redundancy to be resilient to regional failures. + +### VPC design + +The "landing zone" is divided into two VPC networks: + +- the trusted VPC: the connectivity hub towards other trusted networks +- the untrusted VPC: the connectivity hub towards any other untrusted network + +The VPCs are connected with two sets of sample NVA machines, grouped in regional (multi-zone) [Managed Instance Groups (MIGs)](https://cloud.google.com/compute/docs/instance-groups). The appliances are plain Linux machines, performing simple routing/natting, leveraging some standard Linux features, such as *ip route* or *iptables*. The appliances are suited for demo purposes only and they should be replaced with enterprise-grade solutions before moving to production. +The traffic destined to the VMs in each MIG is mediated through regional internal load balancers, both in the trusted and in the untrusted networks. + +By default, the design assumes the following: + +- on-premise networks (and related resources) are considered trusted. As such, the VPNs connecting with on-premises are terminated in GCP, in the trusted VPC +- the public Internet is considered untrusted. As such [Cloud NAT](https://cloud.google.com/nat/docs/overview) has been deployed in the untrusted landing VPC only +- cross-environment traffic and traffic from any untrusted network to any trusted network (and vice versa) pass through the NVAs. For demo purposes, the current NVA performs simple routing/natting only +- any traffic from a trusted network to an untrusted network (e.g. Internet) is natted by the NVAs. Users can configure further exclusions + +The trusted landing VPC acts as a hub: it bridges internal resources with the outside world and it hosts the shared services consumed by the spoke VPCs, connected to the hub thorugh VPC network peerings. Spokes are used to partition the environments. By default: + +- one spoke VPC hosts the development environment resources +- one spoke VPC hosts the production environment resources + +Each virtual network is a [shared VPC](https://cloud.google.com/vpc/docs/shared-vpc): shared VPCs are managed in dedicated *host projects* and shared with other *service projects* that consume the network resources. +Shared VPC lets organization administrators delegate administrative responsibilities, such as creating and managing instances, to Service Project Admins while maintaining centralized control over network resources like subnets, routes, and firewalls. + +Users can easily extend the design to host additional environments, or adopt different logical mappings for the spokes (for example, in order to create a new spoke for each company entity). Adding spokes is trivial and it does not increase the design complexity. The steps to add more spokes are provided in the following sections. + +In multi-organization scenarios, where production and non-production resources use different Cloud Identity and GCP organizations, the hub/landing VPC is usually part of the production organization. It establishes connections with the production spokes within the same organization, and with non-production spokes in a different organization. + +### External connectivity + +External connectivity to on-prem is implemented leveraging [Cloud VPN HA](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies) (two tunnels per region). This is what users normally deploy as a final solution, or to validate routing and to transfer data, while waiting for [interconnects](https://cloud.google.com/network-connectivity/docs/interconnect) to be provisioned. + +Connectivity to additional on-prem sites or to other cloud providers should be implemented in a similar fashion, via VPN tunnels or interconnects, in the landing VPC (either trusted or untrusted, depending by the nature of the peers), sharing the same regional routers. + +### Internal connectivity + +Internal connectivity (e.g. between the trusted landing VPC and the spokes) is realized with VPC network peerings. As mentioned, there are other ways to implement connectivity. These can be easily retrofitted with minimal code changes, although they introduce additional considerations on service interoperability, quotas and management. + +This is an options summary: + +- [VPC Peering](https://cloud.google.com/vpc/docs/vpc-peering) (implemented here to connect the trusted landing VPC with the spokes) + - Pros: no additional costs, full bandwidth with no configurations, no extra latency + - Cons: no transitivity (e.g. to GKE masters, Cloud SQL, etc.), no selective exchange of routes, several quotas and limits shared between VPCs in a peering group +- [Multi-NIC appliances](https://cloud.google.com/architecture/best-practices-vpc-design#multi-nic) (implemented here to connect the trusted landing and untrusted VPCs) + - Pros: provides additional security features (e.g. IPS), potentially better integration with on-prem systems by using the same vendor + - Cons: complex HA/failover setup, limited by VM bandwidth and scale, additional costs for VMs and licenses, out of band management of a critical cloud component +- [VPN HA](https://cloud.google.com/network-connectivity/docs/vpn/concepts/topologies) + - Pros: simple compatibility with GCP services that leverage peering internally, better control on routes, avoids peering groups shared quotas and limits + - Cons: additional costs, marginal increase in latency, requires multiple tunnels for full bandwidth + +### IP ranges, subnetting, routing + +Minimizing the number of routes (and subnets) in the cloud environment is important, as it simplifies management and it avoids hitting [Cloud Router](https://cloud.google.com/network-connectivity/docs/router/quotas) and [VPC](https://cloud.google.com/vpc/docs/quota) quotas and limits. For this reason, we recommend to carefully plan the IP space used in your cloud environment. This allows the use of larger IP CIDR blocks in routes, whenever possible. + +This stage uses a dedicated /16 block, which should be sized to the own needs. The subnets created in each VPC derive from this range. + +Spoke VPCs also define and reserve two "special" CIDR ranges dedicated to [PSA (Private Service Access)](https://cloud.google.com/vpc/docs/private-services-access) and [Internal HTTPs Load Balancers (L7ILB)](https://cloud.google.com/load-balancing/docs/l7-internal). + +Routes in GCP are either automatically created (for example, when a subnet is added to a VPC), manually created via static routes, dynamically exchanged through VPC peerings, or dynamically programmed by [Cloud Routers](https://cloud.google.com/network-connectivity/docs/router#docs) when a BGP session is established. BGP sessions can be configured to advertise VPC ranges, and/or custom ranges via custom advertisements. + +In this setup: + +- routes between multiple subnets within the same VPC are automatically exchanged by GCP +- the spokes and the trusted landing VPC exchange routes through VPC peerings +- on-premises is connected to the trusted landing VPC and it dynamically exchanges BGP routes with GCP (with the trusted VPC) using VPN HA +- for cross-environment (spokes) communications, and for connections to on-premises and to the Internet, the spokes leverage some default tagged routes that send the traffic of each region (whose machines are identified by a dedicated network tag, e.g. *ew1*) to a corresponding regional NVA in the trusted VPC, through an ILB (whose VIP is set as the route next-hop) +- the spokes are configured with backup default routes, so if the NVAs in the same region become unavailable, more routes to the NVAs in the other region are already available. Current routes are not able to understand if the next-hop ILBs become unhealthy. As such, in case of a regional failure, users will need to manually withdraw the primary default routes, so the secondaries will take over +- the NVAs are configured with static routes that allow the communication with on-premises and between the GCP resources (including the cross-environment communication) + +The Cloud Routers (connected to the VPN gateways in the trusted VPC) are configured to exclude the default advertisement of VPC ranges and they only advertise their respective aggregate ranges, via custom advertisements. This greatly simplifies the routing configuration and avoids quota or limit issues, by keeping the number of routes small, instead of making it proportional to the subnets and to the secondary ranges in the VPCs. + +### Internet egress + +In this setup, Internet egress is realized through [Cloud NAT](https://cloud.google.com/nat/docs/overview), deployed in the untrusted landing VPC. This allows instances in all other VPCs to reach the Internet, passing through the NVAs (being the public Internet considered untrusted). + +Several other scenarios are possible, with various degrees of complexity: + +- deploy Cloud NAT in every VPC +- add forwarding proxies, with optional URL filters +- send Internet traffic to on-premises, so the existing egress infrastructure can be leveraged + +Future pluggable modules will allow users to easily experiment with the above scenarios. + +### VPC and Hierarchical Firewall + +The GCP Firewall is a stateful, distributed feature that allows the creation of L4 policies, either via VPC-level rules or -more recently- via hierarchical policies, applied on the resource hierarchy (organization, folders). + +The current setup adopts both firewall types. Hierarchical firewall rules are applied in the networking folder for common ingress rules (egress is open by default): for example, it allows the health checks and the IAP forwarders traffic to reach the VMs. + +Rules and policies are defined in simple YAML files, described below. + +### DNS + +DNS often goes hand in hand with networking, especially on GCP where Cloud DNS zones and policies are associated at VPC level. This setup implements two DNS flows: + +- on-premises to cloud, using private DNS zones pointing cloud-managed domains, and an [inbound policy](https://cloud.google.com/dns/docs/server-policies-overview#dns-server-policy-in) used as the forwarding target +- cloud to on-premises, leveraging Cloud DNS forwarding zones, pointing to the on-premise managed domains + +The DNS configuration is centralized by leveraging peering zones, so that + +- the landing project hosts the Cloud DNS configurations for the on-premise forwarding and Google API domains. Both the trusted and the untrusted VPCs are given visibility to these zones and the spokes consume them through their DNS peering zones +- Cloud DNS peering zones in the spokes host the environment-specific domains configurations, with the trusted and the untrusted VPCs acting as the consumers (leveraging the DNS peering zones configured in the landing project) + +To complete the configuration, the 35.199.192.0/19 range should be routed to the VPN tunnels from on-premises, and the following names should be configured for DNS forwarding to cloud: + +- `private.googleapis.com` +- `restricted.googleapis.com` +- `gcp.example.com` (used as a placeholder) + +In GCP, a forwarding zone in the landing project is configured to forward queries to the placeholder domain `onprem.example.com` to on-premises. + +This configuration is battle-tested, and flexible enough to lend itself to simple modifications without subverting its design. + +## How to run this stage + +This stage is meant to be executed after the [resman](../01-resman) stage has run. It leverages the automation service account and the storage bucket created there, and additional resources configured in the [bootstrap](../00-boostrap) stage. + +It's possible to run this stage in isolation, but that's outside of the scope of this document. Please, refer to the previous stages for the environment requirements. + +Before running this stage, you need to make sure you have the correct credentials and permissions. You'll also need identify the module variables and make sure you assign them the values that match your configuration. + +### Providers configuration + +The default way of making sure you have the right permissions, is to use the identity of the service account pre-created for this stage, during the [resource management](./01-resman) stage, and that you are a member of the group that can impersonate it via provider-level configuration (`gcp-devops` or `organization-admins`). + +To simplify the setup, the previous stage pre-configures a valid providers file in its output and optionally writes it to a local file if the `outputs_location` variable is set to a valid path. + +If you have set a valid value for `outputs_location` in the bootstrap stage, simply link the relevant `providers.tf` file from this stage folder in the path you selected: + +```bash +# `outputs_location` is set to `../../configs/example` +ln -s ../../configs/example/02-networking-nva/providers.tf +``` + +If you have not configured `outputs_location` in bootstrap, you can derive the providers file from that stage outputs: + +```bash +cd ../00-bootstrap +terraform output -json providers | jq -r '.["02-networking-nva"]' \ + > ../02-networking-nva-nva/providers.tf +``` + +### Variable configuration + +There are two broad sets of variables you will need to fill in: + +- variables shared by other stages (org id, billing account id, etc.), or derived from a resource managed by a different stage (folder id, automation project id, etc.) +- variables specific to resources managed by this stage + +To avoid the tedious job of filling in the first group of variables with values derived from other stages outputs, the same mechanism used above for the provider configuration can be used to leverage pre-configured `.tfvars` files. + +If you have set a valid value for `outputs_location` in the bootstrap and in the resman stage, simply link the relevant `terraform-*.auto.tfvars.json` files from this stage's folder in the path you specified, where the `*` above is set to the name of the stage that produced it. For this stage, a single `.tfvars` file is available: + +```bash +# `outputs_location` is set to `../../configs/example` +ln -s ../../configs/example/02-networking-nva/terraform-bootstrap.auto.tfvars.json +ln -s ../../configs/example/02-networking-nva/terraform-resman.auto.tfvars.json +``` + +Please, refer to the [variables](#variables) table below for a map of the variable origins, and use the sections below to understand how to adapt this stage to your networking configuration. + +### VPCs + +VPCs are defined in separate files, one for `untrusted landing`, one for `trusted landing`, one for `prod` and one for `dev`. + +These files contain different resources: + +- **project** ([`projects`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/project)): the "[host projects](https://cloud.google.com/vpc/docs/shared-vpc)" containing the VPCs and enabling the required APIs. +- **VPCs** ([`net-vpc`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpc)): manage the subnets, the explicit routes for `{private,restricted}.googleapis.com` and the DNS inbound policy (for the trusted landing VPC). Subnets are created leveraging "resource factories": the configuration is separated from the module that implements it, and stored in a well-structured file. To add a new subnet, simply create a new file in the `data_folder` directory defined in the module, following the examples found in the [Fabric `net-vpc` documentation](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpc#subnet-factory). Sample subnets are shipped in [data/subnets](./data/subnets) and can be easily customized to fit users' needs. + +Subnets for [L7 ILBs](https://cloud.google.com/load-balancing/docs/l7-internal/proxy-only-subnets) are handled differently, and defined in variable `l7ilb_subnets`, while ranges for [PSA](https://cloud.google.com/vpc/docs/configure-private-services-access#allocating-range) are configured by variable `psa_ranges` - such variables are consumed by spoke VPCs. + +- **Cloud NAT** ([`net-cloudnat`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-cloudnat)) (in the untrusted landing VPC only): it manages the networking infrastructure required to enable the Internet egress. + +### VPNs + +The connectivity between on-premises and GCP (the trusted landing VPC) is implemented with Cloud VPN HA ([`net-vpn`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpn)) and defined in [`vpn-onprem.tf`](./vpn-onprem.tf). The file implements a single logical connection between on-premises and the trusted landing VPC, both in `europe-west1` and `europe-west4`. The relevant parameters for its configuration are found in the variable `vpn_onprem_configs`. + +### Routing and BGP + +Each VPC network ([`net-vpc`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpc)) manages a separate routing table, which can define static routes (e.g. to private.googleapis.com) and receives dynamic routes through VPC peering and BGP sessions established with the neighbor networks (e.g. the trusted landing VPC receives routes from on-premises, and the spokes receive RFC1918 from the trusted landing VPC). + +Static routes are defined in `vpc-*.tf` files in the `routes` section of each `net-vpc` module. + +BGP sessions for trusted landing to on-premises are configured through the variable `vpn_onprem_configs`. + +### Firewall + +**VPC firewall rules** ([`net-vpc-firewall`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpc-firewall)) are defined per-vpc on each `vpc-*.tf` file and leverage a resource factory to massively create rules. +To add a new firewall rule, create a new file or edit an existing one in the `data_folder` directory defined in the module `net-vpc-firewall`, following the examples of the "[Rules factory](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/net-vpc-firewall#rules-factory)" section of the module documentation. Sample firewall rules are shipped in [data/firewall-rules/landing-untrusted](./data/firewall-rules/landing-untrusted) and in [data/firewall-rules/landing-trusted](./data/firewall-rules/landing-trusted), and can be easily customized. + +**Hierarchical firewall policies** ([`folder`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/folder)) are defined in `main.tf`, and managed through a policy factory implemented by the `folder` module, which applies the defined hierarchical to the `Networking` folder, which contains all the core networking infrastructure. Policies are defined in the `rules_file` file - to define a new one simply use the instructions found on "[Firewall policy factory](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/organization#firewall-policy-factory)". Sample hierarchical firewall policies are shipped in [data/hierarchical-policy-rules.yaml](./data/hierarchical-policy-rules.yaml) and can be easily customized. + +### DNS architecture + +The DNS ([`dns`](https://github.com/terraform-google-modules/cloud-foundation-fabric/tree/master/modules/dns)) infrastructure is defined in [`dns-*.tf`] files. + +Cloud DNS manages on-premises forwarding, the main GCP zone (in this example `gcp.example.com`) and is peered to environment-specific zones (i.e. `dev.gcp.example.com` and `prod.gcp.example.com`). + +#### Cloud environment + +The root DNS zone defined in the landing project acts as the source of truth for DNS within the Cloud environment. The resources defined in the spoke VPCs consume the landing DNS infrastructure through DNS peering (e.g. `prod-landing-root-dns-peering`). +The spokes can optionally define private zones (e.g. `prod-dns-private-zone`). Granting visibility both to the trusted and untrusted landing VPCs ensures that the whole cloud environment can query such zones. + +#### Cloud to on-premises + +Leveraging the forwarding zone defined in the landing project (e.g. `onprem-example-dns-forwarding` and `reverse-10-dns-forwarding`), the cloud environment can resolve `in-addr.arpa.` and `onprem.example.com.` using the on-premise DNS infrastructure. On-premise resolver IPs are set in the variable `dns.onprem`. + +DNS queries sent to the on-premise infrastructure come from the `35.199.192.0/19` source range. + +#### On-premises to cloud + +The [Inbound DNS Policy](https://cloud.google.com/dns/docs/server-policies-overview#dns-server-policy-in) defined in the *trusted landing VPC module* ([`vpc-landing.tf`](./vpc-landing.tf)) automatically reserves the first available IP address on each subnet (typically the third one in a CIDR) to expose the Cloud DNS service, so that it can be consumed from outside of GCP. + +### Private Google Access + +[Private Google Access](https://cloud.google.com/vpc/docs/private-google-access) (or PGA) is configured in this environment. It enables VMs and on-premise systems to consume Google APIs from within the Google network. + +For PGA to work: + +- Private Google Access should be enabled on the subnet. \ +Subnets created using the `net-vpc` module are PGA-enabled by default. + +- 199.36.153.4/30 (`restricted.googleapis.com`) and 199.36.153.8/30 (`private.googleapis.com`) should be routed from on-premises to the trusted landing VPC, and from there to the `default-internet-gateway`. \ +The `vpn_onprem_configs` variable contains the ranges advertised from GCP to on-premises. Furthermore, the trusted landing VPC (e.g. see `landing-trusted-vpc` in [`vpc-landing.tf`](./vpc-landing.tf)) has explicit routes to send traffic destined to restricted and private - googleapis.com to the Internet gateway (which works for Google APIs only, and not for the whole Internet, since Cloud NAT is not configured in the trusted landing VPC). + +- On-premises, a private DNS zone for `googleapis.com` should be created and configured per [this article](https://cloud.google.com/vpc/docs/configure-private-google-access-hybrid#config-domain). Its configuration can be copied from the module `googleapis-private-zone` in [`dns.tf`](./dns.tf) + +### Preliminar activities + +Before running `terraform apply`, make sure to adapt `variables.tf` to your needs, to update the variable values using a new `terraform.tfvars` file, and to update the references to the regions in the whole directory, in order to match your preferences (e.g. `europe-west1` or `ew1`). + +If you're not using other FAST stages, you'll also need to create a `providers.tf` file to configure the GCS backend and the service account to use to run the deployment. + +You're now ready to run `terraform init` and `terraform apply`. + +### Post-deployment activities + +- On-premise routers should be configured to advertise all relevant CIDRs to the GCP environments. To avoid hitting GCP quotas, we recommend aggregating routes as much as possible +- On-premise routers should accept BGP sessions from their cloud peers +- On-premise DNS servers should have forward zones configured, in order to resolve GCP-managed domains + +## Customizations + +### Adding an environment + +To create a new environment (e.g. `staging`), a few changes are required: + +Create a `vpc-spoke-staging.tf` file by copying `vpc-spoke-prod.tf` file. +Adapt the new file by replacing the value "prod" with the value "staging". +Running `diff vpc-spoke-dev.tf vpc-spoke-prod.tf` can help to see how environment files differ. + +The new VPC requires a set of dedicated CIDRs, one per region, added to variable `custom_adv` (for example as `spoke_staging_ew1` and `spoke_staging_ew4`). +>`custom_adv` is a map that "resolves" CIDR names to the actual addresses, and will be used later to configure routing. +> +Variables managing L7 Internal Load Balancers (`l7ilb_subnets`) and Private Service Access (`psa_ranges`) should also be adapted, and subnets and firewall rules for the new spoke should be added, as described above. + +VPC network peering connectivity to the `trusted landing VPC` is managed by the `vpc-peering-*.tf` files. +Copy `vpc-peering-prod.tf` to `vpc-peering-staging.tf` and replace "prod" with "staging", where relevant. + +Configure the NVAs deployed or update the sample NVA config files ([ew1](data/nva-startup-script-ew1.tftpl) and [ew4](data/nva-startup-script-ew1.tftpl)), thus making sure they support the new subnets. + +DNS configurations are managed in the `dns-*.tf` files. +Copy the `dns-prod.tf` to `dns-staging.tf` and replace within the files "prod" with "staging", where relevant. +Don't forget to add a peering zone in the landing project and point it to the newly created environment private zone. + + + + +## Files + +| name | description | modules | resources | +|---|---|---|---| +| [dns-dev.tf](./dns-dev.tf) | Development spoke DNS zones and peerings setup. | dns | | +| [dns-landing.tf](./dns-landing.tf) | Landing DNS zones and peerings setup. | dns | | +| [dns-prod.tf](./dns-prod.tf) | Production spoke DNS zones and peerings setup. | dns | | +| [main.tf](./main.tf) | Networking folder and hierarchical policy. | folder | | +| [monitoring.tf](./monitoring.tf) | Network monitoring dashboards. | | google_monitoring_dashboard | +| [nva.tf](./nva.tf) | None | compute-mig · compute-vm · net-ilb | | +| [outputs.tf](./outputs.tf) | Module outputs. | | local_file | +| [test-resources.tf](./test-resources.tf) | temporary instances for testing | compute-vm | | +| [variables.tf](./variables.tf) | Module variables. | | | +| [vpc-landing.tf](./vpc-landing.tf) | Landing VPC and related resources. | net-cloudnat · net-vpc · net-vpc-firewall · project | | +| [vpc-spoke-dev.tf](./vpc-spoke-dev.tf) | Dev spoke VPC and related resources. | net-address · net-vpc · net-vpc-firewall · net-vpc-peering · project | | +| [vpc-spoke-prod.tf](./vpc-spoke-prod.tf) | Production spoke VPC and related resources. | net-address · net-vpc · net-vpc-firewall · net-vpc-peering · project | | +| [vpn-onprem.tf](./vpn-onprem.tf) | VPN between landing and onprem. | net-vpn-ha | | + +## Variables + +| name | description | type | required | default | producer | +|---|---|:---:|:---:|:---:|:---:| +| [billing_account_id](variables.tf#L17) | Billing account id. | string | ✓ | | 00-bootstrap | +| [organization](variables.tf#L99) | Organization details. | object({…}) | ✓ | | 00-bootstrap | +| [prefix](variables.tf#L115) | Prefix used for resources that need unique names. | string | ✓ | | 00-bootstrap | +| [custom_adv](variables.tf#L23) | Custom advertisement definitions in name => range format. | map(string) | | {…} | | +| [data_dir](variables.tf#L45) | Relative path for the folder storing configuration data for network resources. | string | | "data" | | +| [dns](variables.tf#L51) | Onprem DNS resolvers | map(list(string)) | | {…} | | +| [folder_id](variables.tf#L59) | Folder to be used for the networking resources in folders/nnnnnnnnnnn format. If null, folder will be created. | string | | null | 01-resman | +| [l7ilb_subnets](variables.tf#L73) | Subnets used for L7 ILBs. | map(list(object({…}))) | | {…} | | +| [onprem_cidr](variables.tf#L91) | Onprem addresses in name => range format. | map(string) | | {…} | | +| [outputs_location](variables.tf#L109) | Path where providers and tfvars files for the following stages are written. Leave empty to disable. | string | | null | | +| [project_factory_sa](variables.tf#L121) | IAM emails for project factory service accounts | map(string) | | {} | 01-resman | +| [psa_ranges](variables.tf#L128) | IP ranges used for Private Service Access (e.g. CloudSQL). | map(map(string)) | | {…} | | +| [router_configs](variables.tf#L143) | Configurations for CRs and onprem routers. | map(object({…})) | | {…} | | +| [vpn_onprem_configs](variables.tf#L166) | VPN gateway configuration for onprem interconnection. | map(object({…})) | | {…} | | + +## Outputs + +| name | description | sensitive | consumers | +|---|---|:---:|---| +| [project_ids](outputs.tf#L42) | Network project ids. | | | +| [project_numbers](outputs.tf#L51) | Network project numbers. | | | +| [shared_vpc_host_projects](outputs.tf#L60) | Shared VPC host projects. | | | +| [shared_vpc_self_links](outputs.tf#L69) | Shared VPC host projects. | | | +| [tfvars](outputs.tf#L93) | Network-related variables used in other stages. | ✓ | | +| [vpn_gateway_endpoints](outputs.tf#L79) | External IP Addresses for the GCP VPN gateways. | | | + + diff --git a/fast/stages/02-networking-nva/data/cidrs.yaml b/fast/stages/02-networking-nva/data/cidrs.yaml new file mode 100644 index 000000000..b6c25e21a --- /dev/null +++ b/fast/stages/02-networking-nva/data/cidrs.yaml @@ -0,0 +1,15 @@ +# skip boilerplate check + +healthchecks: + - 35.191.0.0/16 + - 130.211.0.0/22 + - 209.85.152.0/22 + - 209.85.204.0/22 + +rfc1918: + - 10.0.0.0/8 + - 172.16.0.0/12 + - 192.168.0.0/16 + +onprem_probes: + - 10.255.255.254/32 diff --git a/fast/stages/02-networking-nva/data/dashboards/firewall_insights.json b/fast/stages/02-networking-nva/data/dashboards/firewall_insights.json new file mode 100644 index 000000000..4c0ebe287 --- /dev/null +++ b/fast/stages/02-networking-nva/data/dashboards/firewall_insights.json @@ -0,0 +1,68 @@ +{ + "displayName": "Firewall Insights Monitoring", + "gridLayout": { + "columns": "2", + "widgets": [ + { + "title": "Subnet Firewall Hit Counts", + "xyChart": { + "chartOptions": { + "mode": "COLOR" + }, + "dataSets": [ + { + "minAlignmentPeriod": "60s", + "plotType": "LINE", + "targetAxis": "Y1", + "timeSeriesQuery": { + "timeSeriesFilter": { + "aggregation": { + "perSeriesAligner": "ALIGN_RATE" + }, + "filter": "metric.type=\"firewallinsights.googleapis.com/subnet/firewall_hit_count\" resource.type=\"gce_subnetwork\"", + "secondaryAggregation": {} + }, + "unitOverride": "1" + } + } + ], + "timeshiftDuration": "0s", + "yAxis": { + "label": "y1Axis", + "scale": "LINEAR" + } + } + }, + { + "title": "VM Firewall Hit Counts", + "xyChart": { + "chartOptions": { + "mode": "COLOR" + }, + "dataSets": [ + { + "minAlignmentPeriod": "60s", + "plotType": "LINE", + "targetAxis": "Y1", + "timeSeriesQuery": { + "timeSeriesFilter": { + "aggregation": { + "perSeriesAligner": "ALIGN_RATE" + }, + "filter": "metric.type=\"firewallinsights.googleapis.com/vm/firewall_hit_count\" resource.type=\"gce_instance\"", + "secondaryAggregation": {} + }, + "unitOverride": "1" + } + } + ], + "timeshiftDuration": "0s", + "yAxis": { + "label": "y1Axis", + "scale": "LINEAR" + } + } + } + ] + } +} diff --git a/fast/stages/02-networking-nva/data/dashboards/vpn.json b/fast/stages/02-networking-nva/data/dashboards/vpn.json new file mode 100644 index 000000000..1aec3e45b --- /dev/null +++ b/fast/stages/02-networking-nva/data/dashboards/vpn.json @@ -0,0 +1,248 @@ +{ + "displayName": "VPN Monitoring", + "gridLayout": { + "columns": "2", + "widgets": [ + { + "title": "Number of connections", + "xyChart": { + "chartOptions": { + "mode": "COLOR" + }, + "dataSets": [ + { + "minAlignmentPeriod": "60s", + "plotType": "LINE", + "targetAxis": "Y1", + "timeSeriesQuery": { + "timeSeriesFilter": { + "aggregation": { + "perSeriesAligner": "ALIGN_MEAN" + }, + "filter": "metric.type=\"vpn.googleapis.com/gateway/connections\" resource.type=\"vpn_gateway\"", + "secondaryAggregation": {} + }, + "unitOverride": "1" + } + } + ], + "timeshiftDuration": "0s", + "yAxis": { + "label": "y1Axis", + "scale": "LINEAR" + } + } + }, + { + "title": "Tunnel established", + "xyChart": { + "chartOptions": { + "mode": "COLOR" + }, + "dataSets": [ + { + "minAlignmentPeriod": "60s", + "plotType": "LINE", + "targetAxis": "Y1", + "timeSeriesQuery": { + "timeSeriesFilter": { + "aggregation": { + "perSeriesAligner": "ALIGN_MEAN" + }, + "filter": "metric.type=\"vpn.googleapis.com/tunnel_established\" resource.type=\"vpn_gateway\"", + "secondaryAggregation": {} + }, + "unitOverride": "1" + } + } + ], + "timeshiftDuration": "0s", + "yAxis": { + "label": "y1Axis", + "scale": "LINEAR" + } + } + }, + { + "title": "Cloud VPN Gateway - Received bytes", + "xyChart": { + "chartOptions": { + "mode": "COLOR" + }, + "dataSets": [ + { + "minAlignmentPeriod": "60s", + "plotType": "LINE", + "targetAxis": "Y1", + "timeSeriesQuery": { + "timeSeriesFilter": { + "aggregation": { + "perSeriesAligner": "ALIGN_RATE" + }, + "filter": "metric.type=\"vpn.googleapis.com/network/received_bytes_count\" resource.type=\"vpn_gateway\"", + "secondaryAggregation": {} + }, + "unitOverride": "By" + } + } + ], + "timeshiftDuration": "0s", + "yAxis": { + "label": "y1Axis", + "scale": "LINEAR" + } + } + }, + { + "title": "Cloud VPN Gateway - Sent bytes", + "xyChart": { + "chartOptions": { + "mode": "COLOR" + }, + "dataSets": [ + { + "minAlignmentPeriod": "60s", + "plotType": "LINE", + "targetAxis": "Y1", + "timeSeriesQuery": { + "timeSeriesFilter": { + "aggregation": { + "perSeriesAligner": "ALIGN_RATE" + }, + "filter": "metric.type=\"vpn.googleapis.com/network/sent_bytes_count\" resource.type=\"vpn_gateway\"", + "secondaryAggregation": {} + }, + "unitOverride": "By" + } + } + ], + "timeshiftDuration": "0s", + "yAxis": { + "label": "y1Axis", + "scale": "LINEAR" + } + } + }, + { + "title": "Cloud VPN Gateway - Received packets", + "xyChart": { + "chartOptions": { + "mode": "COLOR" + }, + "dataSets": [ + { + "minAlignmentPeriod": "60s", + "plotType": "LINE", + "targetAxis": "Y1", + "timeSeriesQuery": { + "timeSeriesFilter": { + "aggregation": { + "perSeriesAligner": "ALIGN_RATE" + }, + "filter": "metric.type=\"vpn.googleapis.com/network/received_packets_count\" resource.type=\"vpn_gateway\"", + "secondaryAggregation": {} + }, + "unitOverride": "{packets}" + } + } + ], + "timeshiftDuration": "0s", + "yAxis": { + "label": "y1Axis", + "scale": "LINEAR" + } + } + }, + { + "title": "Cloud VPN Gateway - Sent packets", + "xyChart": { + "chartOptions": { + "mode": "COLOR" + }, + "dataSets": [ + { + "minAlignmentPeriod": "60s", + "plotType": "LINE", + "targetAxis": "Y1", + "timeSeriesQuery": { + "timeSeriesFilter": { + "aggregation": { + "perSeriesAligner": "ALIGN_RATE" + }, + "filter": "metric.type=\"vpn.googleapis.com/network/sent_packets_count\" resource.type=\"vpn_gateway\"", + "secondaryAggregation": {} + }, + "unitOverride": "{packets}" + } + } + ], + "timeshiftDuration": "0s", + "yAxis": { + "label": "y1Axis", + "scale": "LINEAR" + } + } + }, + { + "title": "Incoming packets dropped", + "xyChart": { + "chartOptions": { + "mode": "COLOR" + }, + "dataSets": [ + { + "minAlignmentPeriod": "60s", + "plotType": "LINE", + "targetAxis": "Y1", + "timeSeriesQuery": { + "timeSeriesFilter": { + "aggregation": { + "perSeriesAligner": "ALIGN_RATE" + }, + "filter": "metric.type=\"vpn.googleapis.com/network/dropped_received_packets_count\" resource.type=\"vpn_gateway\"", + "secondaryAggregation": {} + }, + "unitOverride": "1" + } + } + ], + "timeshiftDuration": "0s", + "yAxis": { + "label": "y1Axis", + "scale": "LINEAR" + } + } + }, + { + "title": "Outgoing packets dropped", + "xyChart": { + "chartOptions": { + "mode": "COLOR" + }, + "dataSets": [ + { + "minAlignmentPeriod": "60s", + "plotType": "LINE", + "targetAxis": "Y1", + "timeSeriesQuery": { + "timeSeriesFilter": { + "aggregation": { + "perSeriesAligner": "ALIGN_RATE" + }, + "filter": "metric.type=\"vpn.googleapis.com/network/dropped_sent_packets_count\" resource.type=\"vpn_gateway\"", + "secondaryAggregation": {} + }, + "unitOverride": "1" + } + } + ], + "timeshiftDuration": "0s", + "yAxis": { + "label": "y1Axis", + "scale": "LINEAR" + } + } + } + ] + } +} diff --git a/fast/stages/02-networking-nva/data/firewall-rules/landing-trusted/rules.yaml b/fast/stages/02-networking-nva/data/firewall-rules/landing-trusted/rules.yaml new file mode 100644 index 000000000..672af07f7 --- /dev/null +++ b/fast/stages/02-networking-nva/data/firewall-rules/landing-trusted/rules.yaml @@ -0,0 +1,29 @@ +# skip boilerplate check + +allow-hc-nva-ssh-trusted: + description: "Allow traffic from Google healthchecks to NVA appliances" + direction: INGRESS + action: allow + sources: [] + ranges: + - $healthchecks + targets: [] + use_service_accounts: false + rules: + - protocol: tcp + ports: + - 22 + +allow-onprem-probes-trusted-example: + description: "Allow traffic from onprem probes" + direction: INGRESS + action: allow + sources: [] + ranges: + - $onprem_probes + targets: [] + use_service_accounts: false + rules: + - protocol: tcp + ports: + - 12345 diff --git a/fast/stages/02-networking-nva/data/firewall-rules/landing-untrusted/rules.yaml b/fast/stages/02-networking-nva/data/firewall-rules/landing-untrusted/rules.yaml new file mode 100644 index 000000000..15db503ba --- /dev/null +++ b/fast/stages/02-networking-nva/data/firewall-rules/landing-untrusted/rules.yaml @@ -0,0 +1,15 @@ +# skip boilerplate check + +allow-hc-nva-ssh-untrusted: + description: "Allow traffic from Google healthchecks to NVA appliances" + direction: INGRESS + action: allow + sources: [] + ranges: + - $healthchecks + targets: [] + use_service_accounts: false + rules: + - protocol: tcp + ports: + - 22 diff --git a/fast/stages/02-networking-nva/data/hierarchical-policy-rules.yaml b/fast/stages/02-networking-nva/data/hierarchical-policy-rules.yaml new file mode 100644 index 000000000..0172a3091 --- /dev/null +++ b/fast/stages/02-networking-nva/data/hierarchical-policy-rules.yaml @@ -0,0 +1,49 @@ +# skip boilerplate check + +allow-admins: + description: Access from the admin subnet to all subnets + direction: INGRESS + action: allow + priority: 1000 + ranges: + - $rfc1918 + ports: + all: [] + target_resources: null + enable_logging: false + +allow-healthchecks: + description: Enable HTTP and HTTPS healthchecks + direction: INGRESS + action: allow + priority: 1001 + ranges: + - $healthchecks + ports: + tcp: ["80", "443"] + target_resources: null + enable_logging: false + +allow-ssh-from-iap: + description: Enable SSH from IAP + direction: INGRESS + action: allow + priority: 1002 + ranges: + - 35.235.240.0/20 + ports: + tcp: ["22"] + target_resources: null + enable_logging: false + +allow-icmp: + description: Enable ICMP + direction: INGRESS + action: allow + priority: 1003 + ranges: + - 0.0.0.0/0 + ports: + icmp: [] + target_resources: null + enable_logging: false diff --git a/fast/stages/02-networking-nva/data/nva-startup-script.tftpl b/fast/stages/02-networking-nva/data/nva-startup-script.tftpl new file mode 100644 index 000000000..353c6fa19 --- /dev/null +++ b/fast/stages/02-networking-nva/data/nva-startup-script.tftpl @@ -0,0 +1,30 @@ +#!/bin/bash + +echo 'Enabling IP forwarding' +sed '/net.ipv4.ip_forward=1/s/^#//g' -i /etc/sysctl.conf && +sysctl -p /etc/sysctl.conf && +/etc/init.d/procps restart + +echo 'Setting Routes' +ip route add ${landing-untrusted-other-region} via ${gateway-untrusted} dev ens4 +ip route add ${landing-trusted-other-region} via ${gateway-trusted} dev ens5 +ip route add ${dev-default-ew1-cidr} via ${gateway-trusted} dev ens5 +ip route add ${dev-default-ew4-cidr} via ${gateway-trusted} dev ens5 +ip route add ${prod-default-ew1-cidr} via ${gateway-trusted} dev ens5 +ip route add ${prod-default-ew4-cidr} via ${gateway-trusted} dev ens5 +ip route add ${onprem-main-cidr} via ${gateway-trusted} dev ens5 + +echo 'Adding PBR rules to answer HCs also from the secondary nic' +grep -qxF '200 hc' /etc/iproute2/rt_tables || echo '200 hc' >> /etc/iproute2/rt_tables +ip_addr_ens5=$(ip route ls table local | awk '/ens5 proto 66 scope host/ {print $2}') +while [ -z $ip_addr_ens5 ]; do + echo 'Waiting for networking stack to be ready' + sleep 2 + ip_addr_ens5=$(ip route ls table local | awk '/ens5 proto 66 scope host/ {print $2}') +done +ip rule add from $ip_addr_ens5 lookup hc +ip route add default via ${gateway-trusted} dev ens5 table hc + +echo 'Setting NAT masquerade (for Internet connectivity)' +iptables --append FORWARD --in-interface ens5 -j ACCEPT +iptables --table nat --append POSTROUTING --out-interface ens4 -j MASQUERADE diff --git a/fast/stages/02-networking-nva/data/subnets/dev/dev-default-ew1.yaml b/fast/stages/02-networking-nva/data/subnets/dev/dev-default-ew1.yaml new file mode 100644 index 000000000..3baaf1482 --- /dev/null +++ b/fast/stages/02-networking-nva/data/subnets/dev/dev-default-ew1.yaml @@ -0,0 +1,5 @@ +# skip boilerplate check + +region: europe-west1 +ip_cidr_range: 10.128.128.0/19 +description: Default europe-west1 subnet for dev diff --git a/fast/stages/02-networking-nva/data/subnets/dev/dev-default-ew4.yaml b/fast/stages/02-networking-nva/data/subnets/dev/dev-default-ew4.yaml new file mode 100644 index 000000000..387694555 --- /dev/null +++ b/fast/stages/02-networking-nva/data/subnets/dev/dev-default-ew4.yaml @@ -0,0 +1,5 @@ +# skip boilerplate check + +region: europe-west4 +ip_cidr_range: 10.128.160.0/19 +description: Default europe-west4 subnet for dev diff --git a/fast/stages/02-networking-nva/data/subnets/landing-trusted/landing-trusted-default-ew1.yaml b/fast/stages/02-networking-nva/data/subnets/landing-trusted/landing-trusted-default-ew1.yaml new file mode 100644 index 000000000..474045234 --- /dev/null +++ b/fast/stages/02-networking-nva/data/subnets/landing-trusted/landing-trusted-default-ew1.yaml @@ -0,0 +1,5 @@ +# skip boilerplate check + +region: europe-west1 +ip_cidr_range: 10.128.64.0/19 +description: Default europe-west1 subnet for landing trusted diff --git a/fast/stages/02-networking-nva/data/subnets/landing-trusted/landing-trusted-default-ew4.yaml b/fast/stages/02-networking-nva/data/subnets/landing-trusted/landing-trusted-default-ew4.yaml new file mode 100644 index 000000000..463066fb4 --- /dev/null +++ b/fast/stages/02-networking-nva/data/subnets/landing-trusted/landing-trusted-default-ew4.yaml @@ -0,0 +1,5 @@ +# skip boilerplate check + +region: europe-west4 +ip_cidr_range: 10.128.96.0/19 +description: Default europe-west4 subnet for landing trusted diff --git a/fast/stages/02-networking-nva/data/subnets/landing-untrusted/landing-untrusted-default-ew1.yaml b/fast/stages/02-networking-nva/data/subnets/landing-untrusted/landing-untrusted-default-ew1.yaml new file mode 100644 index 000000000..2758da5f9 --- /dev/null +++ b/fast/stages/02-networking-nva/data/subnets/landing-untrusted/landing-untrusted-default-ew1.yaml @@ -0,0 +1,5 @@ +# skip boilerplate check + +region: europe-west1 +ip_cidr_range: 10.128.0.0/19 +description: Default europe-west1 subnet for landing untrusted diff --git a/fast/stages/02-networking-nva/data/subnets/landing-untrusted/landing-untrusted-default-ew4.yaml b/fast/stages/02-networking-nva/data/subnets/landing-untrusted/landing-untrusted-default-ew4.yaml new file mode 100644 index 000000000..25bad9db6 --- /dev/null +++ b/fast/stages/02-networking-nva/data/subnets/landing-untrusted/landing-untrusted-default-ew4.yaml @@ -0,0 +1,5 @@ +# skip boilerplate check + +region: europe-west4 +ip_cidr_range: 10.128.32.0/19 +description: Default europe-west4 subnet for landing untrusted diff --git a/fast/stages/02-networking-nva/data/subnets/prod/prod-default-ew1.yaml b/fast/stages/02-networking-nva/data/subnets/prod/prod-default-ew1.yaml new file mode 100644 index 000000000..b829cb94d --- /dev/null +++ b/fast/stages/02-networking-nva/data/subnets/prod/prod-default-ew1.yaml @@ -0,0 +1,5 @@ +# skip boilerplate check + +region: europe-west1 +ip_cidr_range: 10.128.192.0/19 +description: Default europe-west1 subnet for prod diff --git a/fast/stages/02-networking-nva/data/subnets/prod/prod-default-ew4.yaml b/fast/stages/02-networking-nva/data/subnets/prod/prod-default-ew4.yaml new file mode 100644 index 000000000..dbd716cdb --- /dev/null +++ b/fast/stages/02-networking-nva/data/subnets/prod/prod-default-ew4.yaml @@ -0,0 +1,5 @@ +# skip boilerplate check + +region: europe-west4 +ip_cidr_range: 10.128.224.0/19 +description: Default europe-west4 subnet for prod diff --git a/fast/stages/02-networking-nva/diagram.png b/fast/stages/02-networking-nva/diagram.png new file mode 100644 index 000000000..5e789ed79 Binary files /dev/null and b/fast/stages/02-networking-nva/diagram.png differ diff --git a/fast/stages/02-networking-nva/diagram.svg b/fast/stages/02-networking-nva/diagram.svg new file mode 100644 index 000000000..b5ccea7e0 --- /dev/null +++ b/fast/stages/02-networking-nva/diagram.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/fast/stages/02-networking-nva/dns-dev.tf b/fast/stages/02-networking-nva/dns-dev.tf new file mode 100644 index 000000000..08de34cce --- /dev/null +++ b/fast/stages/02-networking-nva/dns-dev.tf @@ -0,0 +1,53 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Development spoke DNS zones and peerings setup. + +# GCP-specific environment zone + +module "dev-dns-private-zone" { + source = "../../../modules/dns" + project_id = module.landing-project.project_id + type = "private" + name = "dev-gcp-example-com" + domain = "dev.gcp.example.com." + client_networks = [module.dev-spoke-vpc.self_link] + recordsets = { + "A localhost" = { type = "A", ttl = 300, records = ["127.0.0.1"] } + } +} + +# root zone peering to landing to centralize configuration; remove if unneeded + +module "dev-landing-root-dns-peering" { + source = "../../../modules/dns" + project_id = module.dev-spoke-project.project_id + type = "peering" + name = "dev-root-dns-peering" + domain = "." + client_networks = [module.dev-spoke-vpc.self_link] + peer_network = module.landing-trusted-vpc.self_link +} + +module "dev-reverse-10-dns-peering" { + source = "../../../modules/dns" + project_id = module.dev-spoke-project.project_id + type = "peering" + name = "dev-reverse-10-dns-peering" + domain = "10.in-addr.arpa." + client_networks = [module.dev-spoke-vpc.self_link] + peer_network = module.landing-trusted-vpc.self_link +} diff --git a/fast/stages/02-networking-nva/dns-landing.tf b/fast/stages/02-networking-nva/dns-landing.tf new file mode 100644 index 000000000..f13ef9960 --- /dev/null +++ b/fast/stages/02-networking-nva/dns-landing.tf @@ -0,0 +1,111 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Landing DNS zones and peerings setup. + +# forwarding to on-prem DNS resolvers + +module "onprem-example-dns-forwarding" { + source = "../../../modules/dns" + project_id = module.landing-project.project_id + type = "forwarding" + name = "example-com" + domain = "onprem.example.com." + client_networks = [ + module.landing-untrusted-vpc.self_link, + module.landing-trusted-vpc.self_link + ] + forwarders = { for ip in var.dns.onprem : ip => null } +} + +module "reverse-10-dns-forwarding" { + source = "../../../modules/dns" + project_id = module.landing-project.project_id + type = "forwarding" + name = "root-reverse-10" + domain = "10.in-addr.arpa." + client_networks = [ + module.landing-untrusted-vpc.self_link, + module.landing-trusted-vpc.self_link + ] + forwarders = { for ip in var.dns.onprem : ip => null } +} + +module "gcp-example-dns-private-zone" { + source = "../../../modules/dns" + project_id = module.landing-project.project_id + type = "private" + name = "gcp-example-com" + domain = "gcp.example.com." + client_networks = [ + module.landing-untrusted-vpc.self_link, + module.landing-trusted-vpc.self_link + ] + recordsets = { + "A localhost" = { type = "A", ttl = 300, records = ["127.0.0.1"] } + } +} + +# GCP-specific DNS zones peered to the environment spoke that holds the config + +module "prod-gcp-example-dns-peering" { + source = "../../../modules/dns" + project_id = module.landing-project.project_id + type = "peering" + name = "prod-root-dns-peering" + domain = "prod.gcp.example.com." + client_networks = [ + module.landing-untrusted-vpc.self_link, + module.landing-trusted-vpc.self_link + ] + peer_network = module.prod-spoke-vpc.self_link +} + +module "dev-gcp-example-dns-peering" { + source = "../../../modules/dns" + project_id = module.landing-project.project_id + type = "peering" + name = "dev-root-dns-peering" + domain = "dev.gcp.example.com." + client_networks = [ + module.landing-untrusted-vpc.self_link, + module.landing-trusted-vpc.self_link + ] + peer_network = module.dev-spoke-vpc.self_link +} + +# Google API zone to trigger Private Access + +module "googleapis-private-zone" { + source = "../../../modules/dns" + project_id = module.landing-project.project_id + type = "private" + name = "googleapis-com" + domain = "googleapis.com." + client_networks = [ + module.landing-untrusted-vpc.self_link, + module.landing-trusted-vpc.self_link + ] + recordsets = { + "A private" = { type = "A", ttl = 300, records = [ + "199.36.153.8", "199.36.153.9", "199.36.153.10", "199.36.153.11" + ] } + "A restricted" = { type = "A", ttl = 300, records = [ + "199.36.153.4", "199.36.153.5", "199.36.153.6", "199.36.153.7" + ] } + "CNAME *" = { type = "CNAME", ttl = 300, records = ["private.googleapis.com."] } + } +} diff --git a/fast/stages/02-networking-nva/dns-prod.tf b/fast/stages/02-networking-nva/dns-prod.tf new file mode 100644 index 000000000..d92157e82 --- /dev/null +++ b/fast/stages/02-networking-nva/dns-prod.tf @@ -0,0 +1,53 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Production spoke DNS zones and peerings setup. + +# GCP-specific environment zone + +module "prod-dns-private-zone" { + source = "../../../modules/dns" + project_id = module.landing-project.project_id + type = "private" + name = "prod-gcp-example-com" + domain = "prod.gcp.example.com." + client_networks = [module.prod-spoke-vpc.self_link] + recordsets = { + "A localhost" = { type = "A", ttl = 300, records = ["127.0.0.1"] } + } +} + +# root zone peering to landing to centralize configuration; remove if unneeded + +module "prod-landing-root-dns-peering" { + source = "../../../modules/dns" + project_id = module.prod-spoke-project.project_id + type = "peering" + name = "prod-root-dns-peering" + domain = "." + client_networks = [module.prod-spoke-vpc.self_link] + peer_network = module.landing-trusted-vpc.self_link +} + +module "prod-reverse-10-dns-peering" { + source = "../../../modules/dns" + project_id = module.prod-spoke-project.project_id + type = "peering" + name = "prod-reverse-10-dns-peering" + domain = "10.in-addr.arpa." + client_networks = [module.prod-spoke-vpc.self_link] + peer_network = module.landing-trusted-vpc.self_link +} diff --git a/fast/stages/02-networking-nva/main.tf b/fast/stages/02-networking-nva/main.tf new file mode 100644 index 000000000..db03c69a1 --- /dev/null +++ b/fast/stages/02-networking-nva/main.tf @@ -0,0 +1,42 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Networking folder and hierarchical policy. + +locals { + l7ilb_subnets = { for env, v in var.l7ilb_subnets : env => [ + for s in v : merge(s, { + active = true + name = "${env}-l7ilb-${s.region}" + })] + } +} + +module "folder" { + source = "../../../modules/folder" + parent = "organizations/${var.organization.id}" + name = "Networking" + folder_create = var.folder_id == null + id = var.folder_id + firewall_policy_factory = { + cidr_file = "${var.data_dir}/cidrs.yaml" + policy_name = null + rules_file = "${var.data_dir}/hierarchical-policy-rules.yaml" + } + firewall_policy_association = { + factory-policy = "factory" + } +} diff --git a/fast/stages/02-networking-nva/monitoring.tf b/fast/stages/02-networking-nva/monitoring.tf new file mode 100644 index 000000000..7b8b70c51 --- /dev/null +++ b/fast/stages/02-networking-nva/monitoring.tf @@ -0,0 +1,32 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Network monitoring dashboards. + +locals { + dashboard_path = "${var.data_dir}/dashboards" + dashboard_files = fileset(local.dashboard_path, "*.json") + dashboards = { + for filename in local.dashboard_files : + filename => "${local.dashboard_path}/${filename}" + } +} + +resource "google_monitoring_dashboard" "dashboard" { + for_each = local.dashboards + project = module.landing-project.project_id + dashboard_json = file(each.value) +} diff --git a/fast/stages/02-networking-nva/nva.tf b/fast/stages/02-networking-nva/nva.tf new file mode 100644 index 000000000..aaf4c6de8 --- /dev/null +++ b/fast/stages/02-networking-nva/nva.tf @@ -0,0 +1,222 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +locals { + _subnets = var.data_dir == null ? tomap({}) : { + for f in fileset("${var.data_dir}/subnets", "**/*.yaml") : + trimsuffix(basename(f), ".yaml") => yamldecode(file("${var.data_dir}/subnets/${f}")) + } + subnets = merge( + { for k, v in local._subnets : "${k}-cidr" => v.ip_cidr_range }, + { for k, v in local._subnets : "${k}-gw" => cidrhost(v.ip_cidr_range, 1) } + ) +} + +# europe-west1 + +module "nva-template-ew1" { + source = "../../../modules/compute-vm" + project_id = module.landing-project.project_id + name = "nva-template" + zone = "europe-west1-b" + tags = ["nva"] + can_ip_forward = true + network_interfaces = [ + { + network = module.landing-untrusted-vpc.self_link + subnetwork = module.landing-untrusted-vpc.subnet_self_links["europe-west1/landing-untrusted-default-ew1"] + nat = false + addresses = null + }, + { + network = module.landing-trusted-vpc.self_link + subnetwork = module.landing-trusted-vpc.subnet_self_links["europe-west1/landing-trusted-default-ew1"] + nat = false + addresses = null + } + ] + boot_disk = { + image = "projects/debian-cloud/global/images/family/debian-10" + type = "pd-balanced" + size = 10 + } + create_template = true + metadata = { + startup-script = templatefile( + "${path.module}/data/nva-startup-script.tftpl", + { + dev-default-ew1-cidr = local.subnets.dev-default-ew1-cidr + dev-default-ew4-cidr = local.subnets.dev-default-ew4-cidr + gateway-trusted = local.subnets.landing-trusted-default-ew1-gw + gateway-untrusted = local.subnets.landing-untrusted-default-ew1-gw + landing-trusted-other-region = local.subnets.landing-trusted-default-ew4-cidr + landing-untrusted-other-region = local.subnets.landing-untrusted-default-ew4-cidr + onprem-main-cidr = var.onprem_cidr.main + prod-default-ew1-cidr = local.subnets.prod-default-ew1-cidr + prod-default-ew4-cidr = local.subnets.prod-default-ew4-cidr + } + ) + } +} + +module "nva-mig-ew1" { + source = "../../../modules/compute-mig" + project_id = module.landing-project.project_id + regional = true + location = "europe-west1" + name = "nva" + target_size = 2 + default_version = { + instance_template = module.nva-template-ew1.template.self_link + name = "default" + } +} + +module "ilb-nva-untrusted-ew1" { + source = "../../../modules/net-ilb" + project_id = module.landing-project.project_id + region = "europe-west1" + name = "ilb-nva-untrusted-ew1" + service_label = var.prefix + global_access = true + network = module.landing-untrusted-vpc.self_link + subnetwork = module.landing-untrusted-vpc.subnet_self_links["europe-west1/landing-untrusted-default-ew1"] + backends = [{ + failover = false + group = module.nva-mig-ew1.group_manager.instance_group + balancing_mode = "CONNECTION" + }] + health_check_config = { + type = "tcp", check = { port = 22 }, config = {}, logging = false + } +} + +module "ilb-nva-trusted-ew1" { + source = "../../../modules/net-ilb" + project_id = module.landing-project.project_id + region = "europe-west1" + name = "ilb-nva-trusted-ew1" + service_label = var.prefix + global_access = true + network = module.landing-trusted-vpc.self_link + subnetwork = module.landing-trusted-vpc.subnet_self_links["europe-west1/landing-trusted-default-ew1"] + backends = [{ + failover = false + group = module.nva-mig-ew1.group_manager.instance_group + balancing_mode = "CONNECTION" + }] + health_check_config = { + type = "tcp", check = { port = 22 }, config = {}, logging = false + } +} + +# europe-west4 + +module "nva-template-ew4" { + source = "../../../modules/compute-vm" + project_id = module.landing-project.project_id + name = "nva-template" + zone = "europe-west4-a" + tags = ["nva"] + can_ip_forward = true + network_interfaces = [ + { + network = module.landing-untrusted-vpc.self_link + subnetwork = module.landing-untrusted-vpc.subnet_self_links["europe-west4/landing-untrusted-default-ew4"] + nat = false + addresses = null + }, + { + network = module.landing-trusted-vpc.self_link + subnetwork = module.landing-trusted-vpc.subnet_self_links["europe-west4/landing-trusted-default-ew4"] + nat = false + addresses = null + } + ] + boot_disk = { + image = "projects/debian-cloud/global/images/family/debian-10" + type = "pd-balanced" + size = 10 + } + create_template = true + metadata = { + startup-script = templatefile( + "${path.module}/data/nva-startup-script.tftpl", + { + dev-default-ew1-cidr = local.subnets.dev-default-ew1-cidr + dev-default-ew4-cidr = local.subnets.dev-default-ew4-cidr + gateway-trusted = local.subnets.landing-trusted-default-ew4-gw + gateway-untrusted = local.subnets.landing-untrusted-default-ew4-gw + landing-trusted-other-region = local.subnets.landing-trusted-default-ew1-cidr + landing-untrusted-other-region = local.subnets.landing-untrusted-default-ew1-cidr + onprem-main-cidr = var.onprem_cidr.main + prod-default-ew1-cidr = local.subnets.prod-default-ew1-cidr + prod-default-ew4-cidr = local.subnets.prod-default-ew4-cidr + } + ) + } +} + +module "nva-mig-ew4" { + source = "../../../modules/compute-mig" + project_id = module.landing-project.project_id + regional = true + location = "europe-west4" + name = "nva" + target_size = 2 + default_version = { + instance_template = module.nva-template-ew4.template.self_link + name = "default" + } +} + +module "ilb-nva-untrusted-ew4" { + source = "../../../modules/net-ilb" + project_id = module.landing-project.project_id + region = "europe-west4" + name = "ilb-nva-untrusted-ew4" + service_label = var.prefix + global_access = true + network = module.landing-untrusted-vpc.self_link + subnetwork = module.landing-untrusted-vpc.subnet_self_links["europe-west4/landing-untrusted-default-ew4"] + backends = [{ + failover = false + group = module.nva-mig-ew4.group_manager.instance_group + balancing_mode = "CONNECTION" + }] + health_check_config = { + type = "tcp", check = { port = 22 }, config = {}, logging = false + } +} + +module "ilb-nva-trusted-ew4" { + source = "../../../modules/net-ilb" + project_id = module.landing-project.project_id + region = "europe-west4" + name = "ilb-nva-trusted-ew4" + service_label = var.prefix + global_access = true + network = module.landing-trusted-vpc.self_link + subnetwork = module.landing-trusted-vpc.subnet_self_links["europe-west4/landing-trusted-default-ew4"] + backends = [{ + failover = false + group = module.nva-mig-ew4.group_manager.instance_group + balancing_mode = "CONNECTION" + }] + health_check_config = { + type = "tcp", check = { port = 22 }, config = {}, logging = false + } +} diff --git a/fast/stages/02-networking-nva/outputs.tf b/fast/stages/02-networking-nva/outputs.tf new file mode 100644 index 000000000..39c5d2ef3 --- /dev/null +++ b/fast/stages/02-networking-nva/outputs.tf @@ -0,0 +1,97 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# Optionally, generate providers and tfvars files for subsequent stages + +locals { + tfvars = { + "03-project-factory-dev" = jsonencode({ + environment_dns_zone = module.dev-dns-private-zone.domain + shared_vpc_self_link = module.dev-spoke-vpc.self_link + vpc_host_project = module.dev-spoke-project.project_id + }) + "03-project-factory-prod" = jsonencode({ + environment_dns_zone = module.prod-dns-private-zone.domain + shared_vpc_self_link = module.prod-spoke-vpc.self_link + vpc_host_project = module.prod-spoke-project.project_id + }) + } +} + +resource "local_file" "tfvars" { + for_each = var.outputs_location == null ? {} : local.tfvars + filename = "${var.outputs_location}/${each.key}/terraform-networking.auto.tfvars.json" + content = each.value +} + +# Outputs + +output "project_ids" { + description = "Network project ids." + value = { + dev = module.dev-spoke-project.project_id + landing = module.landing-project.project_id + prod = module.prod-spoke-project.project_id + } +} + +output "project_numbers" { + description = "Network project numbers." + value = { + dev = "projects/${module.dev-spoke-project.number}" + landing = "projects/${module.landing-project.number}" + prod = "projects/${module.prod-spoke-project.number}" + } +} + +output "shared_vpc_host_projects" { + description = "Shared VPC host projects." + value = { + dev = module.dev-spoke-project.project_id + landing = module.landing-project.project_id + prod = module.prod-spoke-project.project_id + } +} + +output "shared_vpc_self_links" { + description = "Shared VPC host projects." + value = { + dev = module.dev-spoke-vpc.self_link + landing-trusted = module.landing-trusted-vpc.self_link + landing-untrusted = module.landing-untrusted-vpc.self_link + prod = module.prod-spoke-vpc.self_link + } +} + +output "vpn_gateway_endpoints" { + description = "External IP Addresses for the GCP VPN gateways." + value = { + onprem-ew1 = { + for v in module.landing-to-onprem-ew1-vpn.gateway.vpn_interfaces : + v.id => v.ip_address + } + onprem-ew4 = { + for v in module.landing-to-onprem-ew4-vpn.gateway.vpn_interfaces : + v.id => v.ip_address + } + } +} + +output "tfvars" { + description = "Network-related variables used in other stages." + sensitive = true + value = local.tfvars +} diff --git a/fast/stages/02-networking-nva/test-resources.tf b/fast/stages/02-networking-nva/test-resources.tf new file mode 100644 index 000000000..cd9ae9b0a --- /dev/null +++ b/fast/stages/02-networking-nva/test-resources.tf @@ -0,0 +1,245 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description temporary instances for testing + +# Untrusted (Landing) + +module "test-vm-landing-untrusted-ew1-0" { + source = "../../../modules/compute-vm" + project_id = module.landing-project.project_id + zone = "europe-west1-b" + name = "test-vm-lnd-unt-ew1-0" + network_interfaces = [{ + network = module.landing-untrusted-vpc.self_link + subnetwork = module.landing-untrusted-vpc.subnet_self_links["europe-west1/landing-untrusted-default-ew1"] + alias_ips = {} + nat = false + addresses = null + }] + tags = ["ew1", "ssh"] + service_account_create = true + boot_disk = { + image = "projects/debian-cloud/global/images/family/debian-10" + type = "pd-balanced" + size = 10 + } + metadata = { + startup-script = < { + address = cidrhost(v, 0) + network = module.dev-spoke-vpc.self_link + prefix_length = split("/", v)[1] + } + } +} + +module "peering-dev" { + source = "../../../modules/net-vpc-peering" + prefix = "dev-peering-0" + local_network = module.dev-spoke-vpc.self_link + peer_network = module.landing-trusted-vpc.self_link +} diff --git a/fast/stages/02-networking-nva/vpc-spoke-prod.tf b/fast/stages/02-networking-nva/vpc-spoke-prod.tf new file mode 100644 index 000000000..320175dc0 --- /dev/null +++ b/fast/stages/02-networking-nva/vpc-spoke-prod.tf @@ -0,0 +1,129 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description Production spoke VPC and related resources. + +module "prod-spoke-project" { + source = "../../../modules/project" + billing_account = var.billing_account_id + name = "prod-net-spoke-0" + parent = var.folder_id + prefix = var.prefix + service_config = { + disable_on_destroy = false + disable_dependent_services = false + } + services = [ + "compute.googleapis.com", + "dns.googleapis.com", + "iap.googleapis.com", + "networkmanagement.googleapis.com", + "servicenetworking.googleapis.com", + ] + shared_vpc_host_config = { + enabled = true + service_projects = [] + } + metric_scopes = [module.landing-project.project_id] + iam = { + "roles/dns.admin" = [var.project_factory_sa.prod] + } +} + +module "prod-spoke-vpc" { + source = "../../../modules/net-vpc" + project_id = module.prod-spoke-project.project_id + name = "prod-spoke-0" + mtu = 1500 + data_folder = "${var.data_dir}/subnets/prod" + delete_default_routes_on_create = true + subnets_l7ilb = local.l7ilb_subnets.prod + # Set explicit routes for googleapis; send everything else to NVAs + routes = { + private-googleapis = { + dest_range = "199.36.153.8/30" + priority = 999 + tags = [] + next_hop_type = "gateway" + next_hop = "default-internet-gateway" + } + restricted-googleapis = { + dest_range = "199.36.153.4/30" + priority = 999 + tags = [] + next_hop_type = "gateway" + next_hop = "default-internet-gateway" + } + nva-ew1-to-ew1 = { + dest_range = "0.0.0.0/0" + priority = 1000 + tags = ["ew1"] + next_hop_type = "ilb" + next_hop = module.ilb-nva-trusted-ew1.forwarding_rule_address + } + nva-ew4-to-ew4 = { + dest_range = "0.0.0.0/0" + priority = 1000 + tags = ["ew4"] + next_hop_type = "ilb" + next_hop = module.ilb-nva-trusted-ew4.forwarding_rule_address + } + nva-ew1-to-ew4 = { + dest_range = "0.0.0.0/0" + priority = 1001 + tags = ["ew1"] + next_hop_type = "ilb" + next_hop = module.ilb-nva-trusted-ew4.forwarding_rule_address + } + nva-ew4-to-ew1 = { + dest_range = "0.0.0.0/0" + priority = 1001 + tags = ["ew4"] + next_hop_type = "ilb" + next_hop = module.ilb-nva-trusted-ew1.forwarding_rule_address + } + } +} + +module "prod-spoke-firewall" { + source = "../../../modules/net-vpc-firewall" + project_id = module.prod-spoke-project.project_id + network = module.prod-spoke-vpc.name + admin_ranges = [] + http_source_ranges = [] + https_source_ranges = [] + ssh_source_ranges = [] + data_folder = "${var.data_dir}/firewall-rules/prod" + cidr_template_file = "${var.data_dir}/cidrs.yaml" +} + +module "prod-spoke-psa-addresses" { + source = "../../../modules/net-address" + project_id = module.prod-spoke-project.project_id + psa_addresses = { for r, v in var.psa_ranges.prod : r => { + address = cidrhost(v, 0) + network = module.prod-spoke-vpc.self_link + prefix_length = split("/", v)[1] + } + } +} + +module "peering-prod" { + source = "../../../modules/net-vpc-peering" + prefix = "prod-peering-0" + local_network = module.prod-spoke-vpc.self_link + peer_network = module.landing-trusted-vpc.self_link +} diff --git a/fast/stages/02-networking-nva/vpn-onprem.tf b/fast/stages/02-networking-nva/vpn-onprem.tf new file mode 100644 index 000000000..d28a66ee4 --- /dev/null +++ b/fast/stages/02-networking-nva/vpn-onprem.tf @@ -0,0 +1,88 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +# tfdoc:file:description VPN between landing and onprem. + +locals { + bgp_peer_options_onprem = { + for k, v in var.vpn_onprem_configs : + k => v.adv == null ? null : { + advertise_groups = [] + advertise_ip_ranges = { + for adv in(v.adv == null ? [] : v.adv.custom) : + var.custom_adv[adv] => adv + } + advertise_mode = try(v.adv.default, false) ? "DEFAULT" : "CUSTOM" + route_priority = null + } + } +} + +module "landing-to-onprem-ew1-vpn" { + source = "../../../modules/net-vpn-ha" + project_id = module.landing-project.project_id + network = module.landing-trusted-vpc.self_link + region = "europe-west1" + name = "vpn-to-onprem-ew1" + router_create = true + router_name = "landing-onprem-vpn-ew1" + router_asn = var.router_configs.landing-trusted-ew1.asn + peer_external_gateway = var.vpn_onprem_configs.landing-trusted-ew1.peer_external_gateway + tunnels = { + for t in var.vpn_onprem_configs.landing-trusted-ew1.tunnels : + "remote-${t.vpn_gateway_interface}-${t.peer_external_gateway_interface}" => { + bgp_peer = { + address = cidrhost(t.session_range, 1) + asn = t.peer_asn + } + bgp_peer_options = local.bgp_peer_options_onprem.landing-trusted-ew1 + bgp_session_range = "${cidrhost(t.session_range, 2)}/30" + ike_version = 2 + peer_external_gateway_interface = t.peer_external_gateway_interface + router = null + shared_secret = t.secret + vpn_gateway_interface = t.vpn_gateway_interface + } + } +} + +module "landing-to-onprem-ew4-vpn" { + source = "../../../modules/net-vpn-ha" + project_id = module.landing-project.project_id + network = module.landing-trusted-vpc.self_link + region = "europe-west4" + name = "vpn-to-onprem-ew4" + router_create = true + router_name = "landing-onprem-vpn-ew4" + router_asn = var.router_configs.landing-trusted-ew4.asn + peer_external_gateway = var.vpn_onprem_configs.landing-trusted-ew4.peer_external_gateway + tunnels = { + for t in var.vpn_onprem_configs.landing-trusted-ew4.tunnels : + "remote-${t.vpn_gateway_interface}-${t.peer_external_gateway_interface}" => { + bgp_peer = { + address = cidrhost(t.session_range, 1) + asn = t.peer_asn + } + bgp_peer_options = local.bgp_peer_options_onprem.landing-trusted-ew4 + bgp_session_range = "${cidrhost(t.session_range, 2)}/30" + ike_version = 2 + peer_external_gateway_interface = t.peer_external_gateway_interface + router = null + shared_secret = t.secret + vpn_gateway_interface = t.vpn_gateway_interface + } + } +} diff --git a/tests/fast/stages/s02_networking_nva/__init__.py b/tests/fast/stages/s02_networking_nva/__init__.py new file mode 100644 index 000000000..6d6d1266c --- /dev/null +++ b/tests/fast/stages/s02_networking_nva/__init__.py @@ -0,0 +1,13 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. diff --git a/tests/fast/stages/s02_networking_nva/fixture/main.tf b/tests/fast/stages/s02_networking_nva/fixture/main.tf new file mode 100644 index 000000000..d36382505 --- /dev/null +++ b/tests/fast/stages/s02_networking_nva/fixture/main.tf @@ -0,0 +1,31 @@ +/** + * Copyright 2022 Google LLC + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +module "stage" { + source = "../../../../../fast/stages/02-networking-nva" + billing_account_id = "000000-111111-222222" + organization = { + domain = "gcp-pso-italy.net" + id = 856933387836 + customer_id = "C01lmug8b" + } + prefix = "fast" + project_factory_sa = { + dev = "foo@iam" + prod = "bar@iam" + } + data_dir = "../../../../../fast/stages/02-networking-nva/data/" +} diff --git a/tests/fast/stages/s02_networking_nva/test_plan.py b/tests/fast/stages/s02_networking_nva/test_plan.py new file mode 100644 index 000000000..6189f62e3 --- /dev/null +++ b/tests/fast/stages/s02_networking_nva/test_plan.py @@ -0,0 +1,20 @@ +# Copyright 2022 Google LLC +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +def test_counts(fast_e2e_plan_runner): + "Test stage." + num_modules, num_resources, _ = fast_e2e_plan_runner() + # TODO: to re-enable per-module resource count check print _, then test + assert num_modules > 0 and num_resources > 0