kovagoadi/hunfabric
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

New SecOps blueprints section and SecOps GKE Forwarder (#2514)

Browse Source 
* new secops gke forwarder
This commit is contained in:
simonebruzzechesse
2024-11-05 14:41:37 +01:00
committed by GitHub
parent b7a4717fc3
commit 08e4d338cd
17 changed files with 1138 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
blueprints/README.md
View File

@@ -1,6 +1,6 @@
# Terraform end-to-end blueprints for Google Cloud
This section provides **[networking blueprints](./networking/)** that implement core patterns or features, **[data solutions blueprints](./data-solutions/)** that demonstrate how to integrate data services in complete scenarios, **[cloud operations blueprints](./cloud-operations/)** that leverage specific products to meet specific operational needs, **[GKE](./gke/)** and **[Serverless](./serverless/)** blueprints, and **[factories](./factories/)** that implement resource factories for the repetitive creation of specific resources.
This section provides **[networking blueprints](./networking/)** that implement core patterns or features, **[data solutions blueprints](./data-solutions/)** that demonstrate how to integrate data services in complete scenarios, **[cloud operations blueprints](./cloud-operations/)** that leverage specific products to meet specific operational needs, **[GKE](./gke/)**, **[SecOps](./secops/)** security blueprints, **[Serverless](./serverless/)** blueprints, and **[factories](./factories/)** that implement resource factories for the repetitive creation of specific resources.
Currently available blueprints:
@@ -10,6 +10,7 @@ Currently available blueprints:
- **factories** - [Fabric resource factories](./factories)
- **GKE** - [Binary Authorization Pipeline Blueprint](./gke/binauthz), [Storage API](./gke/binauthz/image), [Multi-cluster mesh on GKE (fleet API)](./gke/multi-cluster-mesh-gke-fleet-api), [GKE Multitenant Blueprint](./gke/multitenant-fleet), [Shared VPC with GKE support](./networking/shared-vpc-gke/), [GKE Autopilot](./gke/autopilot)
- **networking** - [Calling a private Cloud Function from On-premises](./networking/private-cloud-function-from-onprem), [HA VPN over Interconnect](./networking/ha-vpn-over-interconnect/), [GLB and multi-regional daisy-chaining through hybrid NEGs](./networking/glb-hybrid-neg-internal), [Hybrid connectivity to on-premise services through PSC](./networking/psc-hybrid), [HTTP Load Balancer with Cloud Armor](./networking/glb-and-armor), [Internal Load Balancer as Next Hop](./networking/ilb-next-hop), On-prem DNS and Google Private Access, [PSC Producer](./networking/psc-hybrid/psc-producer), [PSC Consumer](./networking/psc-hybrid/psc-consumer), [Shared VPC with optional GKE cluster](./networking/shared-vpc-gke), [VPC Connectivity Lab](./networking/vpc-connectivity-lab/)
- **SecOps** - [SecOps GKE Forwarder](./secops/secops-gke-forwarder)
- **serverless** - [Cloud Run series](./serverless/cloud-run-explore)
- **third party solutions** - [OpenShift on GCP user-provisioned infrastructure](./third-party-solutions/openshift), [Wordpress deployment on Cloud Run](./third-party-solutions/wordpress/cloudrun)

9
blueprints/secops/README.md Normal file
View File

@@ -0,0 +1,9 @@
# Operations blueprints
This repository provides a collection of Terraform blueprints designed to automate the implementation of custom integrations, agents and configurations for Google Cloud Security and Operations SecOps (aka Chronicle).
## SecOps GKE Forwarder
<a href="./secops-gke-forwarder/" title="SecOps GKE Forwarder"><img src="./secops-gke-forwarder/images/diagram.png" align="left" width="280px"></a> This [blueprint](./secops-gke-forwarder/) is a modular and scalable solution for setting up a SecOps forwarder on Google Kubernetes Engine (GKE). This forwarder is designed to handle multi-tenant data ingestion, ensuring secure and efficient log forwarding to your SecOps SIEM instances.
<br clear="left">

1
blueprints/secops/secops-gke-forwarder/OWNERS Normal file
View File

@@ -0,0 +1 @@
simonebruzzechesse

201
blueprints/secops/secops-gke-forwarder/README.md Normal file
View File

@@ -0,0 +1,201 @@
# SecOps GKE Forwarder
This Terraform repository provides a modular and scalable solution for setting up a SecOps forwarder on Google Kubernetes Engine (GKE). This forwarder is designed to handle multi-tenant data ingestion, ensuring secure and efficient log forwarding to your Chronicle instance.
### High level architecture
The following diagram illustrates the high-level design of created resources, which can be adapted to specific requirements via variables:
![Chronicle Forwarder](./images/diagram.png)
### Key Features
- **Automated GKE Cluster Creation**: Streamlines the provisioning of a dedicated Kubernetes cluster for the forwarder.
- **Scalable Forwarder Deployment**: Deploys the Chronicle forwarder as a Kubernetes Deployment, allowing for easy scaling to accommodate varying log volumes.
- **Multi-tenant Support**: Enables the forwarder to ingest log data from multiple sources or tenants, maintaining clear separation within Chronicle.
- **Modular Configuration**: Provides flexible Terraform modules to customize network settings, resource allocation, and tenant-specific configurations.
### Deployment
#### Step 0: Cloning the repository
If you want to deploy from your Cloud Shell, click on the image below, sign in
if required and when the prompt appears, click on “confirm”.
[![Open Cloudshell](./images/cloud-shell-button.png)](https://shell.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fcloud-foundation-fabric&cloudshell_workspace=blueprints%2Fthird-party-solutions%2Fwordpress%2Fcloudrun)
Otherwise, in your console of choice:
```bash
git clone https://github.com/GoogleCloudPlatform/cloud-foundation-fabric.git
```
Before you deploy the architecture, you will need at least the following
information/configurations in place (for more precise configuration see the Variables section):
* The project ID
* The VPC host project
* VPC and subnets should already exist
#### Step 2: Prepare the variables
Once you have the required information, head back to your cloned repository.
Make sure you’re in the directory of this tutorial (where this README is in).
Configure the Terraform variables in your `terraform.tfvars` file.
See the example test at the end of this README.md as starting point - just
copy it to `terraform.tfvars` and edit the latter. See the variables
documentation below.
#### Step 3: Prepare the providers in the root module
Setup terraform providers in the root module to deal with kubernetes resources as follows:
```terraform
data "google_client_config" "identity" {
count = module.chronicle-forwarder.fleet_host != null ? 1 : 0
}
provider "kubernetes" {
host = module.chronicle-forwarder.fleet_host
token = try(data.google_client_config.identity.0.access_token, null)
}
provider "kubectl" {
host = module.chronicle-forwarder.fleet_host
token = try(data.google_client_config.identity.0.access_token, null)
}
```
#### Step 4: Deploy resources
Initialize your Terraform environment and deploy the resources:
```shell
terraform init
terraform apply
```
Get kubeconfig to connect to the cluster using the command below:
```shell
gcloud container fleet memberships get-credentials CLUSTER_NAME --project PROJECT
```
Then running the command `kubectl get pods` you should receive the following message:
```
"No resources found in default namespace."
```
<!-- BEGIN TFDOC -->
## Variables
| name | description | type | required | default |
|---|---|:---:|:---:|:---:|
| [network_config](variables.tf#L28) | Shared VPC network configurations to use for GKE cluster. | <code title="object&#40;&#123;&#10; host_project &#61; optional&#40;string&#41;&#10; network_self_link &#61; string&#10; subnet_self_link &#61; string&#10; ip_range_gke_master &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | ✓ | |
| [prefix](variables.tf#L38) | Prefix used for resource names. | <code>string</code> | ✓ | |
| [project_id](variables.tf#L57) | Project id, references existing project if `project_create` is null. | <code>string</code> | ✓ | |
| [region](variables.tf#L62) | GCP region. | <code>string</code> | ✓ | |
| [chronicle_forwarder](variables.tf#L17) | Chronicle GKE forwarder configuration. | <code title="object&#40;&#123;&#10; cluster_name &#61; optional&#40;string, &#34;chronicle-log-ingestion&#34;&#41;&#10; master_authorized_ranges &#61; optional&#40;map&#40;string&#41;, &#123;&#10; rfc-1918-10-8 &#61; &#34;10.0.0.0&#47;8&#34;&#10; &#125;&#41;&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>&#123;&#125;</code> |
| [project_create](variables.tf#L48) | Provide values if project creation is needed, uses existing project if null. Parent is in 'folders/nnn' or 'organizations/nnn' format. | <code title="object&#40;&#123;&#10; billing_account_id &#61; string&#10; parent &#61; string&#10;&#125;&#41;">object&#40;&#123;&#8230;&#125;&#41;</code> | | <code>null</code> |
| [tenants](variables.tf#L67) | Chronicle forwarders tenants config. | <code title="map&#40;object&#40;&#123;&#10; chronicle_forwarder_image &#61; optional&#40;string, &#34;cf_production_stable&#34;&#41;&#10; chronicle_region &#61; string&#10; tenant_id &#61; string&#10; namespace &#61; string&#10; forwarder_config &#61; object&#40;&#123;&#10; config_file_content &#61; optional&#40;string&#41;&#10; customer_id &#61; optional&#40;string&#41;&#10; collector_id &#61; optional&#40;string&#41;&#10; secret_key &#61; optional&#40;string&#41;&#10; tls_config &#61; optional&#40;object&#40;&#123;&#10; required &#61; optional&#40;bool, false&#41;&#10; cert_pub &#61; optional&#40;string&#41;&#10; cert_key &#61; optional&#40;string&#41;&#10; &#125;&#41;, &#123; required &#61; false &#125;&#41;&#10; &#125;&#41;&#10;&#125;&#41;&#41;">map&#40;object&#40;&#123;&#8230;&#125;&#41;&#41;</code> | | <code>&#123;&#125;</code> |
## Outputs
| name | description | sensitive |
|---|---|:---:|
| [fleet_host](outputs.tf#L17) | GKE Fleet host. | |
<!-- END TFDOC -->
## Test
```hcl
module "test" {
source = "./fabric/blueprints/secops/secops-gke-forwarder"
project_id = "test"
project_create = {
billing_account_id = "12345-ABCDEF-12345"
parent = "folders/2345678901"
}
region = "europe-west8"
network_config = {
host_project = "prod-net-landing-0"
network_self_link = "https://www.googleapis.com/compute/v1/projects/prod-net-landing-0/global/networks/prod-landing-0"
subnet_self_link = "https://www.googleapis.com/compute/v1/projects/prod-net-landing-0/regions/europe-west1/subnetworks/gke"
ip_range_gke_master = "192.168.0.0/28"
}
prefix = "tmp"
tenants = {
tenant-1 = {
chronicle_forwarder_image = "cf_production_stable"
chronicle_region = "europe"
tenant_id = "tenant-1"
namespace = "ten-1"
forwarder_config = {
config_file_content = file("data/config.yaml")
}
}
tenant-2 = {
chronicle_forwarder_image = "cf_production_stable"
chronicle_region = "europe"
tenant_id = "tenant-2"
namespace = "tenant-2"
forwarder_config = {
secret_key = file("data/secret_key.json")
customer_id = "XXXXXXX-XXXX-XXXX-XXXX-XXXXXX"
collector_id = "XXXXXXX-XXXX-XXXX-XXXX-XXXXXX"
}
}
}
}
# tftest modules=5 resources=34 files=credentials,config
```
```
# tftest-file id=credentials path=data/secret_key.json
{
"type": "service_account",
"project_id": "xxxx",
"private_key_id": "xxxxxxxxxxxxxx",
"private_key": "-----BEGIN PRIVATE KEY-----\nsdcCDSCsLxhfQIOwdvzCn5wcwJ7xVA=\n-----END PRIVATE KEY-----\n",
"client_email": "sample@sample.iam.gserviceaccount.com",
"client_id": "ASDCVSACSA",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/sample.iam.gserviceaccount.com",
"universe_domain": "googleapis.com"
}
```
```
# tftest-file id=config path=data/config.yaml
output:
url: malachiteingestion-pa.googleapis.com:443
identity:
identity:
collector_id: COLLECTOR_ID \
customer_id: CUSTOMER_ID \
collectors:
- syslog:
common:
enabled: true
data_type: "WINDOWS_DHCP"
data_hint:
batch_n_seconds: 10
batch_n_bytes: 1048576
tcp_address: 0.0.0.0:10514
udp_address: 0.0.0.0:10514
connection_timeout_sec: 60
tcp_buffer_size: 524288
- syslog:
common:
enabled: true
data_type: "WINDOWS_DNS"
data_hint:
batch_n_seconds: 10
batch_n_bytes: 1048576
tcp_address: 0.0.0.0:10515
connection_timeout_sec: 60
tcp_buffer_size: 524288
```

BIN
blueprints/secops/secops-gke-forwarder/images/cloud-shell-button.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 10 KiB

BIN
blueprints/secops/secops-gke-forwarder/images/diagram.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 72 KiB

103
blueprints/secops/secops-gke-forwarder/main.tf Normal file
View File

@@ -0,0 +1,103 @@
/**
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
fleet_host = join("", [
"https://connectgateway.googleapis.com/v1/",
"projects/${module.project.number}/",
"locations/global/gkeMemberships/${var.chronicle_forwarder.cluster_name}"
])
}
module "project" {
source = "../../../modules/project"
billing_account = (var.project_create != null
? var.project_create.billing_account_id
: null
)
parent = (var.project_create != null
? var.project_create.parent
: null
)
prefix = var.prefix
project_create = var.project_create != null
name = var.project_id
services = concat([
"compute.googleapis.com",
"iap.googleapis.com",
"stackdriver.googleapis.com",
"chronicle.googleapis.com",
"container.googleapis.com",
"gkehub.googleapis.com",
"connectgateway.googleapis.com",
"gkeconnect.googleapis.com"
])
}
module "fleet" {
source = "../../../modules/gke-hub"
project_id = var.project_id
clusters = {
"chronicle-log-ingestion" = module.chronicle-forwarder.id
}
}
module "chronicle-forwarder" {
source = "../../../modules/gke-cluster-autopilot"
project_id = var.project_id
name = var.chronicle_forwarder.cluster_name
location = var.region
deletion_protection = false
vpc_config = {
network = var.network_config.network_self_link
subnetwork = var.network_config.subnet_self_link
secondary_range_names = {
pods = "pods"
services = "services"
}
master_ipv4_cidr_block = var.network_config.ip_range_gke_master
master_authorized_ranges = var.chronicle_forwarder.master_authorized_ranges
}
private_cluster_config = {
enable_private_endpoint = true
master_global_access = true
}
enable_features = {
gateway_api = true
}
logging_config = {
enable_api_server_logs = true
enable_scheduler_logs = true
enable_controller_manager_logs = true
}
monitoring_config = {
enable_daemonset_metrics = true
enable_deployment_metrics = true
enable_hpa_metrics = true
enable_pod_metrics = true
enable_statefulset_metrics = true
enable_storage_metrics = true
enable_api_server_metrics = true
enable_controller_manager_metrics = true
enable_scheduler_metrics = true
}
}
module "chronicle-forwarder-deployment" {
source = "./secops-forwarder-deployment"
depends_on = [module.chronicle-forwarder]
tenants = var.tenants
}

20
blueprints/secops/secops-gke-forwarder/outputs.tf Normal file
View File

@@ -0,0 +1,20 @@
/**
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "fleet_host" {
description = "GKE Fleet host."
value = local.fleet_host
}

174
blueprints/secops/secops-gke-forwarder/secops-forwarder-deployment/data/default-config.yaml.tpl Normal file
View File

@@ -0,0 +1,174 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
output:
compression: true
url: ${chronicle_url}
identity:
secret_key: |
${indent(6, secret_key)}
collector_id: ${collector_id}
customer_id: ${customer_id}
server:
graceful_timeout: 15s
drain_timeout: 10s
http:
port: 8080
host: 0.0.0.0
read_timeout: 3s
read_header_timeout: 3s
write_timeout: 3s
idle_timeout: 3s
routes:
- meta:
available_status: 204
ready_status: 204
unready_status: 503
collectors:
- syslog:
common:
enabled: true
data_type: PAN_FIREWALL
data_hint:
batch_n_seconds: 10
batch_n_bytes: 1048576
tcp_address: 0.0.0.0:2001
%{ if ! tls_required ~}
udp_address: 0.0.0.0:2001
%{ endif ~}
connection_timeout_sec: 60
%{ if tls_required ~}
certificate: "/opt/chronicle/external/certs/tls.crt"
certificate_key: "/opt/chronicle/external/certs/tls.key"
minimum_tls_version: "TLSv1_3"
%{ endif ~}
- syslog:
common:
enabled: true
data_type: F5_BIGIP_LTM
data_hint:
batch_n_seconds: 10
batch_n_bytes: 1048576
tcp_address: 0.0.0.0:2011
%{ if ! tls_required ~}
udp_address: 0.0.0.0:2011
%{ endif ~}
connection_timeout_sec: 60
%{ if tls_required ~}
certificate: "/opt/chronicle/external/certs/tls.crt"
certificate_key: "/opt/chronicle/external/certs/tls.key"
minimum_tls_version: "TLSv1_3"
%{ endif ~}
- syslog:
common:
enabled: true
data_type: NIX_SYSTEM
data_hint:
batch_n_seconds: 10
batch_n_bytes: 1048576
tcp_address: 0.0.0.0:2021
%{ if ! tls_required ~}
udp_address: 0.0.0.0:2021
%{ endif ~}
connection_timeout_sec: 60
%{ if tls_required ~}
certificate: "/opt/chronicle/external/certs/tls.crt"
certificate_key: "/opt/chronicle/external/certs/tls.key"
minimum_tls_version: "TLSv1_3"
%{ endif ~}
- syslog:
common:
enabled: true
data_type: AUDITD
data_hint:
batch_n_seconds: 10
batch_n_bytes: 1048576
tcp_address: 0.0.0.0:2031
%{ if ! tls_required ~}
udp_address: 0.0.0.0:2031
%{ endif ~}
connection_timeout_sec: 60
%{ if tls_required ~}
certificate: "/opt/chronicle/external/certs/tls.crt"
certificate_key: "/opt/chronicle/external/certs/tls.key"
minimum_tls_version: "TLSv1_3"
%{ endif ~}
- syslog:
common:
enabled: true
data_type: WINEVTLOG
data_hint:
batch_n_seconds: 10
batch_n_bytes: 1048576
tcp_address: 0.0.0.0:2041
%{ if ! tls_required ~}
udp_address: 0.0.0.0:2041
%{ endif ~}
connection_timeout_sec: 60
%{ if tls_required ~}
certificate: "/opt/chronicle/external/certs/tls.crt"
certificate_key: "/opt/chronicle/external/certs/tls.key"
minimum_tls_version: "TLSv1_3"
%{ endif ~}
- syslog:
common:
enabled: true
data_type: WINDOWS_DEFENDER_AV
data_hint:
batch_n_seconds: 10
batch_n_bytes: 1048576
tcp_address: 0.0.0.0:2051
%{ if ! tls_required ~}
udp_address: 0.0.0.0:2051
%{ endif ~}
connection_timeout_sec: 60
%{ if tls_required ~}
certificate: "/opt/chronicle/external/certs/tls.crt"
certificate_key: "/opt/chronicle/external/certs/tls.key"
minimum_tls_version: "TLSv1_3"
%{ endif ~}
- syslog:
common:
enabled: true
data_type: POWERSHELL
data_hint:
batch_n_seconds: 10
batch_n_bytes: 1048576
tcp_address: 0.0.0.0:2061
%{ if ! tls_required ~}
udp_address: 0.0.0.0:2061
%{ endif ~}
connection_timeout_sec: 60
%{ if tls_required ~}
certificate: "/opt/chronicle/external/certs/tls.crt"
certificate_key: "/opt/chronicle/external/certs/tls.key"
minimum_tls_version: "TLSv1_3"
%{ endif ~}
- syslog:
common:
enabled: true
data_type: WINDOWS_FIREWALL
data_hint:
batch_n_seconds: 10
batch_n_bytes: 1048576
tcp_address: 0.0.0.0:2071
%{ if ! tls_required ~}
udp_address: 0.0.0.0:2071
%{ endif ~}
connection_timeout_sec: 60
%{ if tls_required ~}
certificate: "/opt/chronicle/external/certs/tls.crt"
certificate_key: "/opt/chronicle/external/certs/tls.key"
minimum_tls_version: "TLSv1_3"
%{ endif ~}

283
blueprints/secops/secops-gke-forwarder/secops-forwarder-deployment/main.tf Normal file
View File

@@ -0,0 +1,283 @@
/**
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
_chronicle_forwarder_default_lb_ports = [
"2001", "2011", "2021", "2031", "2041", "2051", "2061", "2071"
]
chronicle_forwarder_config = {
for key, value in var.tenants : key =>
coalesce(value.forwarder_config.config_file_content, try(templatefile("${path.module}/data/default-config.yaml.tpl", {
chronicle_url = "${value.chronicle_region}-malachiteingestion-pa.googleapis.com:443",
customer_id = value.forwarder_config.customer_id
collector_id = value.forwarder_config.collector_id
secret_key = value.forwarder_config.secret_key
tls_required = value.forwarder_config.tls_config.required
}), null))
}
deployment_names = {
for key, value in var.tenants : key => "cfps-${value.tenant_id}"
}
tenants_exposing_service_attachments = {
for key, value in var.tenants : key => value if value.network_config.expose_service_attachment
}
tenants_exposing_udp_service_attachments = {
for key, value in var.tenants : key => value if value.network_config.expose_service_attachment && value.forwarder_config.tls_config == null
}
tenants_chronicle_forwarder_lb_ports = {
for k, v in var.tenants : k => coalesce(v.network_config.load_balancer_ports, local._chronicle_forwarder_default_lb_ports)
}
tenants_secret_tls = {
for k, v in var.tenants : k => {
cer = try(tls_locally_signed_cert.forwarder_server_singed_cert[k].cert_pem, v.forwarder_config.tls_config.cert_pub)
key = try(tls_private_key.forwarder_server_key[k].private_key_pem, v.forwarder_config.tls_config.cert_key)
} if v.forwarder_config.tls_config.required
}
}
resource "kubernetes_namespace" "namespace" {
for_each = var.tenants
metadata {
name = each.value.namespace
}
}
resource "kubernetes_service" "service_tcp" {
for_each = var.tenants
metadata {
name = "cfps-${each.value.tenant_id}-tcp"
namespace = kubernetes_namespace.namespace[each.key].metadata[0].name
annotations = {
"networking.gke.io/load-balancer-type" = "Internal"
}
}
spec {
selector = {
app = local.deployment_names[each.key]
}
type = "LoadBalancer"
port {
name = "hc"
protocol = "TCP"
port = 8080
target_port = 8080
}
dynamic "port" {
for_each = local.tenants_chronicle_forwarder_lb_ports[each.key]
content {
name = "tcp-${port.value}"
protocol = "TCP"
port = port.value
target_port = port.value
}
}
}
timeouts {
create = "5m"
}
}
resource "kubectl_manifest" "service_attachment_tcp" {
for_each = local.tenants_exposing_service_attachments
yaml_body = templatefile("${path.module}/manifests/service-attachment.yaml", {
tenant = each.value.tenant_id
namespace = kubernetes_namespace.namespace[each.key].metadata[0].name
service_name = kubernetes_service.service_tcp[each.key].metadata[0].name
})
timeouts {
create = "5m"
}
}
resource "kubernetes_service" "service_udp" {
for_each = { for k, v in var.tenants : k => v if v.forwarder_config.tls_config != null }
metadata {
name = "cfps-${each.value.tenant_id}-udp"
namespace = kubernetes_namespace.namespace[each.key].metadata[0].name
annotations = {
"networking.gke.io/load-balancer-type" = "Internal"
}
}
spec {
selector = {
app = local.deployment_names[each.key]
}
type = "LoadBalancer"
dynamic "port" {
for_each = local.tenants_chronicle_forwarder_lb_ports[each.key]
content {
name = "udp-${port.value}"
protocol = "UDP"
port = port.value
target_port = port.value
}
}
}
}
resource "kubectl_manifest" "service_attachment_udp" {
for_each = local.tenants_exposing_udp_service_attachments
yaml_body = templatefile("${path.module}/manifests/service-attachment.yaml", {
tenant = each.value.tenant_id
namespace = kubernetes_namespace.namespace[each.key].metadata[0].name
service_name = kubernetes_service.service_udp[each.key].metadata[0].name
})
timeouts {
create = "5m"
}
}
resource "kubernetes_deployment" "cfps" {
for_each = var.tenants
metadata {
name = local.deployment_names[each.key]
namespace = kubernetes_namespace.namespace[each.key].metadata[0].name
labels = {
app = local.deployment_names[each.key]
}
}
spec {
replicas = 2 # default number of pods
selector {
match_labels = {
app = local.deployment_names[each.key]
}
}
template {
metadata {
labels = {
app = local.deployment_names[each.key]
}
}
spec {
container {
name = "cfps"
image = "gcr.io/chronicle-container/${var.tenants[each.key].chronicle_forwarder_image}"
image_pull_policy = "IfNotPresent"
resources {
requests = {
cpu = "2" # 2 CPUs requested
memory = "2Gi" # 2 GB of RAM requested
}
limits = {
cpu = "4" # 4 CPUs limit
memory = "6Gi" # 4 GB of RAM limit
}
}
volume_mount {
mount_path = "/opt/chronicle/external/"
name = kubernetes_secret.secret_config[each.key].metadata[0].name
read_only = true
}
dynamic "volume_mount" {
for_each = try(kubernetes_secret.tls[each.key], null) != null ? [""] : []
content {
mount_path = "/opt/chronicle/external/certs"
name = kubernetes_secret.tls[each.key].metadata[0].name
read_only = true
}
}
liveness_probe {
http_get {
path = "/meta/available"
port = 8080
}
failure_threshold = 3
period_seconds = 30
}
readiness_probe {
http_get {
path = "/meta/ready"
port = 8080
}
failure_threshold = 1
period_seconds = 30
}
}
volume {
name = kubernetes_secret.secret_config[each.key].metadata[0].name
secret {
secret_name = kubernetes_secret.secret_config[each.key].metadata[0].name
}
}
dynamic "volume" {
for_each = try(kubernetes_secret.tls[each.key], null) != null ? [""] : []
content {
name = kubernetes_secret.tls[each.key].metadata[0].name
secret {
secret_name = kubernetes_secret.tls[each.key].metadata[0].name
}
}
}
}
}
}
}
resource "kubernetes_horizontal_pod_autoscaler_v2" "cfps" {
for_each = var.tenants
metadata {
name = "${local.deployment_names[each.key]}-hpa"
namespace = kubernetes_namespace.namespace[each.key].metadata[0].name
}
spec {
scale_target_ref {
kind = "Deployment"
name = kubernetes_deployment.cfps[each.key].metadata[0].name
}
min_replicas = 2 # Minimum number of pods
max_replicas = 5 # Maximum allowed replicas
metric {
type = "Resource"
resource {
name = "cpu"
target {
type = "Utilization"
average_utilization = "80"
}
}
}
}
}
resource "kubernetes_secret" "secret_config" {
for_each = var.tenants
metadata {
name = "cfps-secret-config-${each.key}"
namespace = kubernetes_namespace.namespace[each.key].metadata[0].name
}
data = {
"config.conf" = local.chronicle_forwarder_config[each.key]
}
type = "Opaque"
}
resource "kubernetes_secret" "tls" {
for_each = local.tenants_secret_tls
metadata {
name = "cfps-secret-tls-${each.key}"
namespace = kubernetes_namespace.namespace[each.key].metadata[0].name
}
data = {
"tls.crt" = each.value.cer
"tls.key" = each.value.key
}
type = "kubernetes.io/tls"
}

27
blueprints/secops/secops-gke-forwarder/secops-forwarder-deployment/manifests/service-attachment.yaml Normal file
View File

@@ -0,0 +1,27 @@
# Copyright 2023 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: networking.gke.io/v1
kind: ServiceAttachment
metadata:
name: chronicle-sa-${tenant}
namespace: ${namespace}
spec:
connectionPreference: ACCEPT_MANUAL
natSubnets:
- psc
proxyProtocol: false
resourceRef:
kind: Service
name: ${service_name}

20
blueprints/secops/secops-gke-forwarder/secops-forwarder-deployment/outputs.tf Normal file
View File

@@ -0,0 +1,20 @@
/**
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
output "tenants_ssl_cert_ca" {
description = "Tenants Self Signed CA certificates."
value = tls_self_signed_cert.forwarder_ca_cert
}

109
blueprints/secops/secops-gke-forwarder/secops-forwarder-deployment/ssl.tf Normal file
View File

@@ -0,0 +1,109 @@
/**
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
locals {
self_signed_certs_for_tenant = {
for k, v in var.tenants : k => v if v.forwarder_config.tls_config.required && v.forwarder_config.tls_config.cert_key == null
}
cert_subjects = [
{
country = "IT"
province = "Lombardy"
locality = "Milan"
organization = "Example"
organizational_unit = "Example"
}
]
}
#######################################################################
# FORWARDER CA PRIVATE KEY #
#######################################################################
resource "tls_private_key" "forwarder_ca_private_key" {
for_each = local.self_signed_certs_for_tenant
algorithm = "RSA"
}
#######################################################################
# FORWARDER CA CERT #
#######################################################################
resource "tls_self_signed_cert" "forwarder_ca_cert" {
for_each = local.self_signed_certs_for_tenant
private_key_pem = tls_private_key.forwarder_ca_private_key[each.key].private_key_pem
is_ca_certificate = true
dynamic "subject" {
for_each = toset(local.cert_subjects)
content {
country = subject.value.country
province = subject.value.province
locality = subject.value.locality
common_name = "Chronicle Forwarder"
organization = subject.value.organization
organizational_unit = each.value.tenant_id
}
}
validity_period_hours = 87600 // 3650 days or 10 years
allowed_uses = [
"digital_signature",
"cert_signing",
"crl_signing",
]
}
#######################################################################
# SERVER CERT SIGNED BY CA #
#######################################################################
resource "tls_private_key" "forwarder_server_key" {
for_each = local.self_signed_certs_for_tenant
algorithm = "RSA"
}
resource "tls_cert_request" "forwarder_server_csr" {
for_each = local.self_signed_certs_for_tenant
private_key_pem = tls_private_key.forwarder_server_key[each.key].private_key_pem
dns_names = ["${each.value.tenant_id}.chronicle-forwarder.gke"]
dynamic "subject" {
for_each = toset(local.cert_subjects)
content {
country = subject.value.country
province = subject.value.province
locality = subject.value.locality
common_name = "Gitlab"
organization = subject.value.organization
organizational_unit = each.value.tenant_id
}
}
}
resource "tls_locally_signed_cert" "forwarder_server_singed_cert" {
for_each = local.self_signed_certs_for_tenant
cert_request_pem = tls_cert_request.forwarder_server_csr[each.key].cert_request_pem
ca_private_key_pem = tls_private_key.forwarder_ca_private_key[each.key].private_key_pem
ca_cert_pem = tls_self_signed_cert.forwarder_ca_cert[each.key].cert_pem
validity_period_hours = 87600 // 3650 days or 10 years
allowed_uses = [
"digital_signature",
"key_encipherment",
"server_auth",
"client_auth",
]
}

48
blueprints/secops/secops-gke-forwarder/secops-forwarder-deployment/variables.tf Normal file
View File

@@ -0,0 +1,48 @@
/**
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "image" {
description = "Container image to use."
type = string
nullable = false
default = "redis:6.2"
}
variable "tenants" {
description = "Chronicle forwarders tenants config."
type = map(object({
chronicle_forwarder_image = optional(string, "cf_production_stable")
chronicle_region = string
tenant_id = string
namespace = string
network_config = optional(object({
expose_service_attachment = optional(bool, false)
load_balancer_ports = optional(list(string), null)
}), {})
forwarder_config = object({
config_file_content = optional(string)
customer_id = optional(string)
collector_id = optional(string)
secret_key = optional(string)
tls_config = optional(object({
required = optional(bool, false)
cert_pub = optional(string)
cert_key = optional(string)
}), { required = false })
})
}))
default = {}
}

25
blueprints/secops/secops-gke-forwarder/secops-forwarder-deployment/versions_override.tf Normal file
View File

@@ -0,0 +1,25 @@
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
terraform {
required_providers {
restful = {
source = "magodo/restful"
}
kubectl = {
source = "gavinbunney/kubectl"
version = ">= 1.7.0"
}
}
}

87
blueprints/secops/secops-gke-forwarder/variables.tf Normal file
View File

@@ -0,0 +1,87 @@
/**
* Copyright 2024 Google LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
variable "chronicle_forwarder" {
description = "Chronicle GKE forwarder configuration."
type = object({
cluster_name = optional(string, "chronicle-log-ingestion")
master_authorized_ranges = optional(map(string), {
rfc-1918-10-8 = "10.0.0.0/8"
})
})
default = {}
}
variable "network_config" {
description = "Shared VPC network configurations to use for GKE cluster."
type = object({
host_project = optional(string)
network_self_link = string
subnet_self_link = string
ip_range_gke_master = string
})
}
variable "prefix" {
description = "Prefix used for resource names."
type = string
nullable = false
validation {
condition = var.prefix != ""
error_message = "Prefix cannot be empty."
}
}
variable "project_create" {
description = "Provide values if project creation is needed, uses existing project if null. Parent is in 'folders/nnn' or 'organizations/nnn' format."
type = object({
billing_account_id = string
parent = string
})
default = null
}
variable "project_id" {
description = "Project id, references existing project if `project_create` is null."
type = string
}
variable "region" {
description = "GCP region."
type = string
}
variable "tenants" {
description = "Chronicle forwarders tenants config."
type = map(object({
chronicle_forwarder_image = optional(string, "cf_production_stable")
chronicle_region = string
tenant_id = string
namespace = string
forwarder_config = object({
config_file_content = optional(string)
customer_id = optional(string)
collector_id = optional(string)
secret_key = optional(string)
tls_config = optional(object({
required = optional(bool, false)
cert_pub = optional(string)
cert_key = optional(string)
}), { required = false })
})
}))
default = {}
}

29
blueprints/secops/secops-gke-forwarder/versions.tf Normal file
View File

@@ -0,0 +1,29 @@
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Fabric release: v35.0.0
terraform {
required_version = ">= 1.7.4"
required_providers {
google = {
source = "hashicorp/google"
version = ">= 6.1.0, < 7.0.0" # tftest
}
google-beta = {
source = "hashicorp/google-beta"
version = ">= 6.1.0, < 7.0.0" # tftest
}
}
}