Files
hunfabric/modules/secops-rules
Ludovico Magnocavallo 36648b6b63 FAST light implementation (#3255)
* data wip

* wip data

* update org schema, add note on expansion

* all schemas, workload notes

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* Update WORKLOG.md

* wip

* data wip

* wip

* wip

* wip

* wip

* org module IAM context (using lookup)

* new-style context expansion in project IAM

* remove spurious file

* project module contexts

* finalize context replacement format for project module

* revert org module changes

* fix tag id interpolation in project

* fix tag id interpolation in project

* organization module context

* organization context test

* context expansion for folder tag bindings

* test context expansion for tag bindings

* service account module context

* simplify context local

* context for iam service account

* nuke blueprints

* remove links to blueprints

* vpc sc context in project module

* Add context to GCS module

* Add inline deps to plan_summary script

* Make context a top-level variable for folder, organization, sa

* Add add context top-level to VPC-SC

* move context out of factories_config variable

* tfdoc

* fix merge

* fix merge

* fix examples

* net-vpc module context

* add parent ids to folder context

* rename folder parent context

* fix folder parent check

* new project factory stub

* wip

* wip

* refactor defaults

* project iam

* bueckts and service accounts

* start adding context replacements

* better test data

* automation resources for folders and projects

* automation

* add support for project id interpolation

* first tested apply

* improve IAM description in gcs module

* add context to billing account module

* add notification channels to billing account module context

* add billing budgets to new pf

* schemas and defaults

* bootstrap wip

* bootstrap wip

* bootstrap wip

* pf outputs

* pf fixes

* fix pf sample data

* bootstrap lite fixes

* add locations to organization module contexts

* bootstrap lite fixes

* org fixes, billing accounts

* fix default project parent

* bootstrap lite wip

* add locations to gcs module context

* add context support to logging bucket module

* add context to pubsub module

* split out iam variables in gcs module

* fix logging bucket context test

* bootstrap log sink destinations

* streamline logging-bucket module variables

* fix logging bucket context test

* align logging bucket module interface in fast bootstrap

* add support for project-level log buckets to project factory

* support full context expansion in organization module log sinks

* log buckets in fast-lite bootstrap

* make og sink type optional in organization module

* log sinks in fast-lite bootstrap

* set tag values in factory context

* bootstrap lite data

* output files schema

* billing account schema

* output files

* output providers

* gcs output files

* boilerplate

* tflint

* check documentation

* check docs

* fix project module parent variable validation

* fix log bucket examples

* allow null parent in project module

* silence folder test errors

* fix billing account sink example

* fix project example

* fix billing account module

* fix folder tests

* fix FAST

* fix fast

* tfvars outputs

* wif

* cicd service accounts

* cicd

* allow defaults in context, minimal org policies

* support gcs managed folders in project factory and bootstrap lite

* support prefix in provider output files

* rename bootstrap stage

* gitignore

* gitignore

* security folder, billing IAM

* wip tfvars

* fix typo

* security IAM

* control tag iam/context via variables in organization module

* split tag creation from tag IAM to avoid circular refs

* port organization module tag changes to project module

* implement new-style context expansion in vpc-sc module

* fix fast vpc-sc tests

* boilerplate

* vpc sc stage

* schemas

* fast-lite compatibility for vpc sc stage

* make log project number optional in vpc-sc stage

* networking

* networking

* networking

* networking

* rename and move new stage under fast

* clone pf tests

* use context replacement for internal notification channels in billing account module

* support service agents in project module iam context replacements

* support service agents in project module iam context replacements

* add support for kms keys to project module context

* experimental pf example test and fixes

* fix schemas

* fix tests

* tfdoc

* tfdoc

* pf config

* experimental pf

* remove redundant dot from gcs managed folder IAM keys

* bootstrap experimental test

* project factory exp stage test

* skip tflint for bootstrap experimental test

* tflint

* fix gcs test

* documentation work

* documentation work

* Update README.md

* tfdoc

* tfdoc

* readme

* tfdoc

* readme

* readme

* readme

* readme

* support universe in pf exp projects

* missing universe service agents

* org policies import, non-admin billing IAM

* todo

* fix test

* custom constraints

* fast classic dataset

* fix test data

* context replacements in billing module log sinks

* fix typo

* add support for billing log sinks

* update docs

* readme

* cicd fix and test

---------

Co-authored-by: Julio Castillo <jccb@google.com>
2025-09-02 08:38:57 +02:00
..
2025-09-02 08:38:57 +02:00
2025-04-11 15:44:31 +02:00

SecOps Rules

This module allows creation and management of custom rules as well as reference lists in Google SecOps.

  • rule definition (yaral code) and reference list entries are managed as files in data folder as per the factories_config variable and sample code
  • rule and reference list deployments can leverage both rules_config and reference_lists_config variables or YAML file still specified in the factories_config variable.

Examples

Sample SecOps Rules and reference list deployment

This is a sample usage of the secops-rules module for deploying a rule (network_traffic_to_specific_country) and a reference list (private_ip_ranges), definition of the rule in yaral is available in the corresponding file in the data/rules folder and the reference list in the data/reference_lists folder. Deployment configuration for both is passed as an input to the module using the rules_config and reference_lists_config variables.

module "secops" {
  source        = "./fabric/modules/secops-rules"
  project_id    = var.project_id
  tenant_config = var.secops_tenant_config
  reference_lists_config = {
    "private_ip_ranges" = {
      description = "Private CIDR ranges"
      type        = "CIDR"
    }
  }
  rules_config = {
    "network_traffic_to_specific_country" = {
      enabled       = true
      alerting      = true
      archived      = false
      run_frequency = "LIVE"
    }
  }
  factories_config = {
    rules_defs           = "./data/rules"
    reference_lists_defs = "./data/reference_lists"
  }
}
# tftest modules=1 resources=3 files=reference,rule inventory=basic.yaml
rule network_traffic_to_specific_country {

meta:
  author = "Google Cloud Security"
  description = "Identify network traffic based on target country"
  type = "alert"
  tags = "geoip enrichment"
  data_source = "microsoft windows events"
  severity = "Low"
  priority = "Low"

events:
  $network.metadata.event_type = "NETWORK_CONNECTION"
  //Specify a country of interest to monitor or add additional countries using an or statement
  $network.target.ip_geo_artifact.location.country_or_region = "France" nocase
  $network.target.ip = $ip

match:
  $ip over 30m

outcome:
  $risk_score = max(35)
  $event_count = count_distinct($network.metadata.id)

  // added to populate alert graph with additional context
  $principal_ip = array_distinct($network.principal.ip)

  // Commented out target.ip because it is already represented in graph as match variable. If match changes, can uncomment to add to results
  //$target_ip = array_distinct($network.target.ip)
  $principal_process_pid = array_distinct($network.principal.process.pid)
  $principal_process_command_line = array_distinct($network.principal.process.command_line)
  $principal_process_file_sha256 = array_distinct($network.principal.process.file.sha256)
  $principal_process_file_full_path = array_distinct($network.principal.process.file.full_path)
  $principal_process_product_specfic_process_id = array_distinct($network.principal.process.product_specific_process_id)
  $principal_process_parent_process_product_specfic_process_id = array_distinct($network.principal.process.parent_process.product_specific_process_id)
  $target_process_pid = array_distinct($network.target.process.pid)
  $target_process_command_line = array_distinct($network.target.process.command_line)
  $target_process_file_sha256 = array_distinct($network.target.process.file.sha256)
  $target_process_file_full_path = array_distinct($network.target.process.file.full_path)
  $target_process_product_specfic_process_id = array_distinct($network.target.process.product_specific_process_id)
  $target_process_parent_process_product_specfic_process_id = array_distinct($network.target.process.parent_process.product_specific_process_id)
  $principal_user_userid = array_distinct($network.principal.user.userid)
  $target_user_userid = array_distinct($network.target.user.userid)

condition:
  $network
}
# tftest-file id=rule path=data/rules/network_traffic_to_specific_country.yaral
10.0.0.0/8
172.16.0.0/12
192.168.0.0/16
127.0.0.1/32
::1/128
fc00::/7
fe80::/10
# tftest-file id=reference path=data/reference_lists/private_ip_ranges.txt

SecOps Rules Factory

The module includes a secops rules and reference list factory for the configuration of rules and reference lists leveraging YAML configuration files. Each configuration file for rules and reference lists contains more than one rule with a structure that reflects the rules_config and reference_lists_config variables. Again rules and reference list definition is available in the corresponding yaral and txt files in the data folder.

module "secops" {
  source        = "./fabric/modules/secops-rules"
  project_id    = var.project_id
  tenant_config = var.secops_tenant_config
  factories_config = {
    rules                = "./secops_rules.yaml"
    rules_defs           = "./data/rules"
    reference_lists      = "./secops_reference_lists.yaml"
    reference_lists_defs = "./data/reference_lists"
  }
}
# tftest modules=1 resources=3 files=1,2,reference,rule inventory=factory.yaml

network_traffic_to_specific_country:
  enabled: true
  alerting: true
  archived: false
  run_frequency: "DAILY"
# tftest-file id=1 path=secops_rules.yaml
private_ip_ranges:
  description: "Private CIDR ranges"
  type: CIDR # either CIDR, STRING, REGEX
# tftest-file id=2 path=secops_reference_lists.yaml
rule network_traffic_to_specific_country {

meta:
  author = "Google Cloud Security"
  description = "Identify network traffic based on target country"
  type = "alert"
  tags = "geoip enrichment"
  data_source = "microsoft windows events"
  severity = "Low"
  priority = "Low"

events:
  $network.metadata.event_type = "NETWORK_CONNECTION"
  //Specify a country of interest to monitor or add additional countries using an or statement
  $network.target.ip_geo_artifact.location.country_or_region = "France" nocase
  $network.target.ip = $ip

match:
  $ip over 30m

outcome:
  $risk_score = max(35)
  $event_count = count_distinct($network.metadata.id)

  // added to populate alert graph with additional context
  $principal_ip = array_distinct($network.principal.ip)

  // Commented out target.ip because it is already represented in graph as match variable. If match changes, can uncomment to add to results
  //$target_ip = array_distinct($network.target.ip)
  $principal_process_pid = array_distinct($network.principal.process.pid)
  $principal_process_command_line = array_distinct($network.principal.process.command_line)
  $principal_process_file_sha256 = array_distinct($network.principal.process.file.sha256)
  $principal_process_file_full_path = array_distinct($network.principal.process.file.full_path)
  $principal_process_product_specfic_process_id = array_distinct($network.principal.process.product_specific_process_id)
  $principal_process_parent_process_product_specfic_process_id = array_distinct($network.principal.process.parent_process.product_specific_process_id)
  $target_process_pid = array_distinct($network.target.process.pid)
  $target_process_command_line = array_distinct($network.target.process.command_line)
  $target_process_file_sha256 = array_distinct($network.target.process.file.sha256)
  $target_process_file_full_path = array_distinct($network.target.process.file.full_path)
  $target_process_product_specfic_process_id = array_distinct($network.target.process.product_specific_process_id)
  $target_process_parent_process_product_specfic_process_id = array_distinct($network.target.process.parent_process.product_specific_process_id)
  $principal_user_userid = array_distinct($network.principal.user.userid)
  $target_user_userid = array_distinct($network.target.user.userid)

condition:
  $network
}
# tftest-file id=rule path=data/rules/network_traffic_to_specific_country.yaral
10.0.0.0/8
172.16.0.0/12
192.168.0.0/16
127.0.0.1/32
::1/128
fc00::/7
fe80::/10
# tftest-file id=reference path=data/reference_lists/private_ip_ranges.txt

Variables

name description type required default
project_id Project used for resources. string
tenant_config SecOps Tenant configuration. object({…})
factories_config Paths to YAML config expected in 'rules' and 'reference_lists'. Path to folders containing rules definitions (yaral files) and reference lists content (txt files) for the corresponding _defs keys. object({…}) {}
reference_lists_config SecOps Reference lists configuration. map(object({…})) {}
rules_config SecOps Detection rules configuration. map(object({…})) {}