20579394b8
This adds the possiblity to define several clusters in a Bigtable instance, which enables replication in Bigtable. Some of the configurations options are moved inside a `map` that defines the properties of each cluster. These changes alter the interface of the module, so any previous code using this module will have to adapt to the new options interface.
254 lines
8.2 KiB
Markdown
254 lines
8.2 KiB
Markdown
# Google Cloud BigTable Module
|
|
|
|
This module allows managing a single BigTable instance, including access configuration and tables.
|
|
|
|
## TODO
|
|
|
|
- [ ] support bigtable_app_profile
|
|
- [ ] support IAM for tables
|
|
|
|
## Examples
|
|
|
|
### Instance with access configuration
|
|
|
|
```hcl
|
|
|
|
module "bigtable-instance" {
|
|
source = "./fabric/modules/bigtable-instance"
|
|
project_id = "my-project"
|
|
name = "instance"
|
|
clusters = {
|
|
my-cluster = {
|
|
zone = "europe-west1-b"
|
|
}
|
|
}
|
|
tables = {
|
|
test1 = {},
|
|
test2 = {
|
|
split_keys = ["a", "b", "c"]
|
|
}
|
|
}
|
|
iam = {
|
|
"roles/bigtable.user" = ["user:viewer@testdomain.com"]
|
|
}
|
|
}
|
|
# tftest modules=1 resources=4
|
|
```
|
|
|
|
### Instance with tables and column families
|
|
|
|
```hcl
|
|
|
|
module "bigtable-instance" {
|
|
source = "./fabric/modules/bigtable-instance"
|
|
project_id = "my-project"
|
|
name = "instance"
|
|
clusters = {
|
|
my-cluster = {
|
|
zone = "europe-west1-b"
|
|
}
|
|
}
|
|
tables = {
|
|
test1 = {},
|
|
test2 = {
|
|
split_keys = ["a", "b", "c"]
|
|
column_families = {
|
|
cf1 = {}
|
|
cf2 = {}
|
|
cf3 = {}
|
|
}
|
|
}
|
|
test3 = {
|
|
column_families = {
|
|
cf1 = {}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
# tftest modules=1 resources=4
|
|
```
|
|
|
|
### Instance with replication enabled
|
|
|
|
```hcl
|
|
|
|
module "bigtable-instance" {
|
|
source = "./fabric/modules/bigtable-instance"
|
|
project_id = "my-project"
|
|
name = "instance"
|
|
clusters = {
|
|
first-cluster = {
|
|
zone = "europe-west1-b"
|
|
}
|
|
second-cluster = {
|
|
zone = "europe-southwest1-a"
|
|
}
|
|
third-cluster = {
|
|
zone = "us-central1-b"
|
|
}
|
|
}
|
|
}
|
|
# tftest modules=1 resources=1
|
|
```
|
|
|
|
### Instance with garbage collection policy
|
|
|
|
```hcl
|
|
|
|
module "bigtable-instance" {
|
|
source = "./fabric/modules/bigtable-instance"
|
|
project_id = "my-project"
|
|
name = "instance"
|
|
clusters = {
|
|
my-cluster = {
|
|
zone = "europe-west1-b"
|
|
}
|
|
}
|
|
tables = {
|
|
test1 = {
|
|
column_families = {
|
|
cf1 = {
|
|
gc_policy = {
|
|
deletion_policy = "ABANDON"
|
|
max_age = "18h"
|
|
}
|
|
}
|
|
cf2 = {}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
# tftest modules=1 resources=3
|
|
```
|
|
|
|
### Instance with default garbage collection policy
|
|
|
|
The default garbage collection policy is applied to any column family that does
|
|
not specify a `gc_policy`. If a column family specifies a `gc_policy`, the
|
|
default garbage collection policy is ignored for that column family.
|
|
|
|
```hcl
|
|
|
|
module "bigtable-instance" {
|
|
source = "./fabric/modules/bigtable-instance"
|
|
project_id = "my-project"
|
|
name = "instance"
|
|
clusters = {
|
|
my-cluster = {
|
|
zone = "europe-west1-b"
|
|
}
|
|
}
|
|
default_gc_policy = {
|
|
deletion_policy = "ABANDON"
|
|
max_age = "18h"
|
|
max_version = 7
|
|
}
|
|
tables = {
|
|
test1 = {
|
|
column_families = {
|
|
cf1 = {}
|
|
cf2 = {}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
# tftest modules=1 resources=4
|
|
```
|
|
|
|
### Instance with static number of nodes
|
|
|
|
If you are not using autoscaling settings, you must set a specific number of nodes with the variable `num_nodes`.
|
|
|
|
```hcl
|
|
|
|
module "bigtable-instance" {
|
|
source = "./fabric/modules/bigtable-instance"
|
|
project_id = "my-project"
|
|
name = "instance"
|
|
clusters = {
|
|
my-cluster = {
|
|
zone = "europe-west1-b"
|
|
num_nodes = 5
|
|
}
|
|
}
|
|
}
|
|
# tftest modules=1 resources=1
|
|
```
|
|
|
|
### Instance with autoscaling (based on CPU only)
|
|
|
|
If you use autoscaling, you should not set the variable `num_nodes`.
|
|
|
|
```hcl
|
|
|
|
module "bigtable-instance" {
|
|
source = "./fabric/modules/bigtable-instance"
|
|
project_id = "my-project"
|
|
name = "instance"
|
|
clusters = {
|
|
my-cluster = {
|
|
zone = "europe-southwest1-b"
|
|
autoscaling = {
|
|
min_nodes = 3
|
|
max_nodes = 7
|
|
cpu_target = 70
|
|
}
|
|
}
|
|
}
|
|
|
|
|
|
}
|
|
# tftest modules=1 resources=1
|
|
```
|
|
|
|
### Instance with autoscaling (based on CPU and/or storage)
|
|
|
|
```hcl
|
|
|
|
module "bigtable-instance" {
|
|
source = "./fabric/modules/bigtable-instance"
|
|
project_id = "my-project"
|
|
name = "instance"
|
|
clusters = {
|
|
my-cluster = {
|
|
zone = "europe-southwest1-a"
|
|
storage_type = "SSD"
|
|
autoscaling = {
|
|
min_nodes = 3
|
|
max_nodes = 7
|
|
cpu_target = 70
|
|
storage_target = 4096
|
|
}
|
|
}
|
|
}
|
|
}
|
|
# tftest modules=1 resources=1
|
|
```
|
|
<!-- BEGIN TFDOC -->
|
|
|
|
## Variables
|
|
|
|
| name | description | type | required | default |
|
|
|---|---|:---:|:---:|:---:|
|
|
| [clusters](variables.tf#L17) | Clusters to be created in the BigTable instance. Set more than one cluster to enable replication. If you set autoscaling, num_nodes will be ignored. | <code title="map(object({ zone = optional(string) storage_type = optional(string) num_nodes = optional(number) autoscaling = optional(object({ min_nodes = number max_nodes = number cpu_target = number storage_target = optional(number) })) }))">map(object({…}))</code> | ✓ | |
|
|
| [name](variables.tf#L78) | The name of the Cloud Bigtable instance. | <code>string</code> | ✓ | |
|
|
| [project_id](variables.tf#L83) | Id of the project where datasets will be created. | <code>string</code> | ✓ | |
|
|
| [default_autoscaling](variables.tf#L33) | Default settings for autoscaling of clusters. This will be the default autoscaling for any cluster not specifying any autoscaling details. | <code title="object({ min_nodes = number max_nodes = number cpu_target = number storage_target = optional(number) })">object({…})</code> | | <code>null</code> |
|
|
| [default_gc_policy](variables.tf#L44) | Default garbage collection policy, to be applied to all column families and all tables. Can be override in the tables variable for specific column families. | <code title="object({ deletion_policy = optional(string) gc_rules = optional(string) mode = optional(string) max_age = optional(string) max_version = optional(string) })">object({…})</code> | | <code>null</code> |
|
|
| [deletion_protection](variables.tf#L56) | Whether or not to allow Terraform to destroy the instance. Unless this field is set to false in Terraform state, a terraform destroy or terraform apply that would delete the instance will fail. | <code></code> | | <code>true</code> |
|
|
| [display_name](variables.tf#L61) | The human-readable display name of the Bigtable instance. | <code></code> | | <code>null</code> |
|
|
| [iam](variables.tf#L66) | IAM bindings for topic in {ROLE => [MEMBERS]} format. | <code>map(list(string))</code> | | <code>{}</code> |
|
|
| [instance_type](variables.tf#L72) | (deprecated) The instance type to create. One of 'DEVELOPMENT' or 'PRODUCTION'. | <code>string</code> | | <code>null</code> |
|
|
| [tables](variables.tf#L88) | Tables to be created in the BigTable instance. | <code title="map(object({ split_keys = optional(list(string), []) column_families = optional(map(object( { gc_policy = optional(object({ deletion_policy = optional(string) gc_rules = optional(string) mode = optional(string) max_age = optional(string) max_version = optional(string) }), null) })), {}) }))">map(object({…}))</code> | | <code>{}</code> |
|
|
|
|
## Outputs
|
|
|
|
| name | description | sensitive |
|
|
|---|---|:---:|
|
|
| [id](outputs.tf#L17) | An identifier for the resource with format projects/{{project}}/instances/{{name}}. | |
|
|
| [instance](outputs.tf#L26) | BigTable intance. | |
|
|
| [table_ids](outputs.tf#L35) | Map of fully qualified table ids keyed by table name. | |
|
|
| [tables](outputs.tf#L40) | Table resources. | |
|
|
|
|
<!-- END TFDOC -->
|