kovagoadi/hunfabric
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Okta as Workload identity provider (#3494)

Browse Source 
* Explain cursed Create Before Destroy

* okta for workload idenitty provider

* changed readme with okta wif

* fix readme for okta wif

* 0 org setup readme passing check_documentation.py

* fix error in readme for 0-org-setup

---------

Co-authored-by: Wiktor Niesiobędzki <wiktorn@google.com>
Co-authored-by: Leonardo Hoet <leohoet98@gmail.com>
Co-authored-by: mwillig <mwillig@google.com>
This commit is contained in:
lhoet-google
2025-11-01 05:00:06 -03:00
committed by GitHub
parent f9f015a692
commit 153046307f
6 changed files with 175 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

43
CURSED_KNOWLEDGE.md
View File

@@ -2,14 +2,16 @@
<!-- new entries go at the top -->
* 2025-10-23 Some [service agents](https://cloud.google.com/iam/docs/service-agents) are not created upon API activation nor calling `google_project_service_identity`. Since we have no way of knowing if they exist, we avoid automatically granting their respective roles in the project module. The list of agents for which we do not perform automatic grants can be found in the [tools/build_service_agents.py](./tools/build_service_agents.py) script.
* 2025-10-23 Use `terraform plan` after `terraform apply` to confirm that the plan is empty after applying the changes. Non-empty plan is a sign of potential bug in either Terraform code or provider and suggests, that configuration might not have been applied as expected or potential problems when implementing future changes
* 2025-10-23 Do not use `data` resource. Even if you must, then still it might be [a bad idea](#avoid-data-resources)
* 2025-10-23 when referring other resource prefer using `.id` attribute over names. `.id` is computed field, and will force update when referred resource is replaced. Sometimes this requires explicit `depends_on` - for example for Cloud Run IAM, so it is recreated when parent resource is replaced
* 2025-10-23 Maps are the best drivers for `for_each` on the resource level. When using lists, and adding something in the middle of list means that all resources following insertion needs to be replaced
* 2025-10-21 Type checking in ternaries requires both sides to have identical types. For objects, it means that they need to define the same fields. And sometimes `null` and `tonumber(null)` don't converge to a common type (citation needed)
* 2025-10-21 Terraform dependency graph considers a variable or a local as one node in the graph [adrs/20251013-context-locals.md], you may resolve your dependency cycles by just rearranging your variables / locals. But for resources - the dependency is tracked on attribute level and plan may differ depending on which attribute you depend
* 2025-10-21 `create_before_destory` meta-argument is [contagious](https://github.com/hashicorp/terraform/blob/main/docs/destroying.md#forced-create-before-destroy), which means - any resource that any resource depending on CBD resource will also be marked as CBD. This hits hard, when affected resource is silently accepting creation with the same name, even if the object exists (`google_storage_bucket_object`, I'm looking at you)
| date | item |
|------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 2025-10-23 | Some [service agents](https://cloud.google.com/iam/docs/service-agents) are not created upon API activation nor calling `google_project_service_identity`. Since we have no way of knowing if they exist, we avoid automatically granting their respective roles in the project module. The list of agents for which we do not perform automatic grants can be found in the [tools/build_service_agents.py](./tools/build_service_agents.py) script. |
| 2025-10-23 | Use `terraform plan` after `terraform apply` to confirm that the plan is empty after applying the changes. Non-empty plan is a sign of potential bug in either Terraform code or provider and suggests, that configuration might not have been applied as expected or potential problems when implementing future changes | |
| 2025-10-23 | Do not use `data` resource. Even if you must, then still it might be [a bad idea](#avoid-data-resources) |
| 2025-10-23 | when referring other resource prefer using `.id` attribute over names. `.id` is computed field, and will force update when referred resource is replaced. Sometimes this requires explicit `depends_on` - for example for Cloud Run IAM, so it is recreated when parent resource is replaced |
| 2025-10-23 | Maps are the best drivers for `for_each` on the resource level. When using lists, and adding something in the middle of list means that all resources following insertion needs to be replaced |
| 2025-10-21 | Type checking in ternaries requires both sides to have identical types. For objects, it means that they need to define the same fields. And sometimes `null` and `tonumber(null)` don't converge to a common type (citation needed) |
| 2025-10-21 | Terraform dependency graph considers a variable or a local as one node in the graph [adrs/20251013-context-locals.md], you may resolve your dependency cycles by just rearranging your variables / locals. But for resources - the dependency is tracked on attribute level and plan may differ depending on which attribute you depend |
| 2025-10-21 | `create_before_destory` meta-argument is [contagious](https://github.com/hashicorp/terraform/blob/main/docs/destroying.md#forced-create-before-destroy), which means - any resource that any resource depending on CBD resource will also be marked as CBD. This hits hard, when affected resource is silently accepting creation with the same name, even if the object exists (`google_storage_bucket_object`, I'm looking at you). Learn to differentiate `-/+` (destroy then create) from `+/-` (create then destroy) in the pterraform plan. More details in [Dealing with Create Before Destroy](#dealing-with-create-before-destroy) |
## Detailed explanations
@@ -23,3 +25,28 @@ What is considered a safe use case for `data` resource:
* using `data` resource outputs in attributes without `ForceNew` flag - so even if `data` will be read during apply, it will result in spurious update-in-place instead of replacement
In Fabric FAST modules `data` resources are used only by request to simplify calling, but then the above caveats apply to the whole module.
### Dealing with Create-Before-Destroy
If you notice that terraform tries to create a resource, and the resource by this name already exists, but you just changed some attributes of resource that resulted in replacement of the resource check the plan:
```shell
# google_compute_subnetwork.subnetwork must be replaced
+/- resource "google_compute_subnetwork" "subnetwork" {
```
Take note of `+/-`, which means "create then destroy" (CBD). This means that [create_before_destroy](https://developer.hashicorp.com/terraform/language/meta-arguments) strategy was applied to this resource.
This can happen, because the resource has `create_before_destroy` lifecycle argument. But this attribute is also spreading through dependency tree to all resources, that this resource depends (directly or indirectly) on.
For example, if you have a `google_compute_instance_template`, which is usually accompanied by `create_before_destroy` argument, and it references the subnetwork, then the create first, destroy later strategy will be applied to subnetwork.
As long as the name of the network is static and not changing while this change is applied, it will fail, because the subnetwork already exists.
Create-before-destroy flags propagates through dependency tree and keep in mind that what is a node in dependency tree - it is an attribute of resource, a variable, an output or a local.
So if you have a map of subnetworks, and you refer your subnetwork in instance template by accessing one element from this map, then **all subnetworks in this map** are marked as create-before-destroy.
If this is a problem, you can side-step this error, by forcing creation of a new resource under new address:
```shell
resource "google_compute_subnetwork" "subnetwork" {
for_each = local.map
}
```
If the key for your resource stays the same, create-before-destroy will be applied.
But if you change the key in the map, then although CBD is applied, you have one entry that is removed, and other that is created - and they are independence.
This allows to execute these operations in parallel, though this may result in initial failure, after applying again, it will apply cleanly.

85
fast/stages/0-org-setup/README.md
View File

@@ -28,6 +28,7 @@
- [Context-based replacement in the folders factory](#context-based-replacement-in-the-folders-factory)
- [Project factory](#project-factory)
- [CI/CD configuration](#cicd-configuration)
- [Okta](#okta)
- [Leveraging classic FAST Stages](#leveraging-classic-fast-stages)
- [VPC Service Controls](#vpc-service-controls)
- [Security](#security)
@@ -612,6 +613,89 @@ workflows:
plan: $iam_principals:service_accounts/iac-0/iac-org-cicd-ro
```
#### Okta
<details>
<summary>Configure Okta as Workload Identity provider</summary>
Okta is a special case. Unlike providers such as GitHub or GitLab, it's an identity provider that doesn't manage repositories. To use Okta as a Workload Identity provider, configure it in your `datasets/classic/cicd.yaml` file as shown in the following example:
```yaml
workload_identity_federation:
pool_name: iac-0
project: $project_ids:iac-0
providers:
okta:
issuer: okta
provider_id: okta
custom_settings:
audiences:
- <REPLACE_WITH_CUSTOM_AUDIENCE> // Modify this
okta:
auth_server_name: <REPLACE_WITH_SERVER_NAME> // Modify this
organization_name: <REPLACE_WITH_ORG_NAME>.okta.com // Modify this
workflows:
org-setup:
template: okta
workload_identity_provider:
id: $wif_providers:okta
audiences: []
output_files:
storage_bucket: $storage_buckets:iac-0/iac-outputs
providers:
apply: $output_files:providers/0-org-setup
plan: $output_files:providers/0-org-setup-ro
files:
- 0-org-setup.auto.tfvars.json
service_accounts:
apply: $iam_principals:service_accounts/iac-0/iac-org-cicd-rw
plan: $iam_principals:service_accounts/iac-0/iac-org-cicd-ro
```
Finally you will need to modify the following org policies and IAM permissions in `datasets/classic/organization/org-policies/iam.yaml` file:
- Under `org_polices` add your Okta provider URL :
```yaml
org_policies:
iam.workloadIdentityPoolProviders:
rules:
- allow:
values:
- https://token.actions.githubusercontent.com
- https://gitlab.com
- https://app.terraform.io
- https://<REPLACE_WITH_ORG_NAME>.okta.com/oauth2/default // Modify this
```
This configuration adds Okta to the list of allowed Workload Identity providers in your GCP organization.
- Under `iac-org-cicd-ro` and `iac-org-cicd-rw` service accounts add `roles/iam.workloadIdentityUser` to each of them:
```yaml
iac-org-cicd-ro:
display_name: IaC service account for org setup CI/CD (read-only).
iam_sa_roles:
$service_account_ids:iac-0/iac-org-ro:
- roles/iam.workloadIdentityUser
- roles/iam.serviceAccountTokenCreator
iam:
roles/iam.workloadIdentityUser:
- principalSet://iam.googleapis.com/projects/<REPLACE_WITH_IAC_PROJECT_NUMBER>/locations/global/workloadIdentityPools/iac-0/* // Modify this
iac-org-cicd-rw:
display_name: IaC service account for org setup CI/CD (read-write).
iam_sa_roles:
$service_account_ids:iac-0/iac-org-rw:
- roles/iam.workloadIdentityUser
- roles/iam.serviceAccountTokenCreator
iam:
roles/iam.workloadIdentityUser:
- principalSet://iam.googleapis.com/projects/<REPLACE_WITH_IAC_PROJECT_NUMBER>/locations/global/workloadIdentityPools/iac-0/* // Modify this
```
This allows identities from the Workload Identity Pool to impersonate both IaC service accounts.
</details>
## Leveraging classic FAST Stages
Classic Fast stage 2 and 3 can be directly used after applying this if the [Classic FAST layout](#classic-fast-dataset) is used, or similar identities and permissions are implemented in a different design.
@@ -673,3 +757,4 @@ Define values for the `var.environments` variable in a tfvars file.
| [projects](outputs.tf#L22) | Attributes for managed projects. | |
| [tfvars](outputs.tf#L27) | Stage tfvars. | ✓ |
<!-- END TFDOC -->

14
fast/stages/0-org-setup/assets/workflow-okta.yaml Normal file
View File

@@ -0,0 +1,14 @@
# Copyright 2024 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
---

35
fast/stages/0-org-setup/schemas/cicd.schema.json
View File

@@ -147,19 +147,32 @@
},
"jwks_json_path": {
"type": "string"
},
"okta": {
"type": "object",
"additionalProperties": false,
"properties": {
"organization_name": {
"type": "string"
},
"auth_server_name": {
"type": "string"
}
}
}
},
"issuer": {
"type": "string",
"enum": [
"github",
"gitlab",
"terraform",
"okta"
]
},
"provider_id": {
"type": "string"
}
},
"issuer": {
"type": "string",
"enum": [
"github",
"gitlab",
"terraform"
]
},
"provider_id": {
"type": "string"
}
}
}

13
fast/stages/0-org-setup/wif-definitions.tf
View File

@@ -74,5 +74,18 @@ locals {
principal_branch = "principalSet://iam.googleapis.com/%s/attribute.terraform_workspace_id/%s"
principal_repo = "principalSet://iam.googleapis.com/%s/attribute.terraform_project_id/%s"
}
# https://developer.okta.com/docs/api/openapi/okta-oauth/guides/overview/
okta = {
attribute_mapping = {
"google.subject" = "assertion.sub"
"attribute.sub" = "assertion.sub"
}
principal_branch = "principalSet://iam.googleapis.com/%s/attribute.sub/project_path:%s:ref_type:branch:ref:%s"
principal_repo = "principalSet://iam.googleapis.com/%s/attribute.repository/%s"
principal_member = "principalSet://iam.googleapis.com/%s/*"
}
}
}

8
fast/stages/0-org-setup/wif-providers.tf
View File

@@ -50,10 +50,10 @@ resource "google_iam_workload_identity_pool_provider" "default" {
# Setting an empty list configures allowed_audiences to the url of the provider
allowed_audiences = try(each.value.custom_settings.audiences, [])
# If users don't provide an issuer_uri, we set the public one for the platform chosen.
issuer_uri = (
try(each.value.custom_settings.issuer_uri, null) != null
? each.value.custom_settings.issuer_uri
: try(each.value.issuer_uri, null)
issuer_uri = coalesce(
try(each.value.custom_settings.issuer_uri, null),
try(each.value.custom_settings.okta == null ? null : "https://${each.value.custom_settings.okta.organization_name}/oauth2/${each.value.custom_settings.okta.auth_server_name}", null),
try(each.value.issuer_uri, null),
)
# OIDC JWKs in JSON String format. If no value is provided, they key is
# fetched from the `.well-known` path for the issuer_uri