1-resman

Resource hierarchy

This stage performs two important tasks:

• create the top-level hierarchy of folders, and the associated resources used later on to automate each part of the hierarchy (eg. Networking)
• configure resource management tags used in scoping specific IAM roles on the resource hierarchy

The code is intentionally simple, as it's intended to provide a generic initial setup (Networking, Security, etc.), and then allow easy customizations to complete the implementation of the intended hierarchy design.

The following diagram is a high level reference of the resources created and managed here:

• Design overview and choices
  • Department or team folders
  • Multitenancy
  • Workload Identity Federation and CI/CD
• How to run this stage
  • Provider and Terraform variables
  • Impersonating the automation service account
  • Variable configuration
  • Running the stage
• Customizations
  • Toggling features
  • Top-level folders
  • Secure tags
  • IAM
• Files
• Variables
• Outputs

Design overview and choices

Despite its simplicity, this stage implements the basics of a design that we've seen working well for a variety of customers, where the hierarchy is laid out following two conceptually different approaches:

• core or shared resources are grouped in hierarchy branches that map to their type or purpose (e.g. Networking)
• team or application resources are grouped in lower level hierarchy branches that map to management or operational considerations (e.g. which team manages a set of applications, or owns a subset of company data, etc.)

This split approach usually represents well functional and operational patterns, where core resources are centrally managed by individual teams (e.g. networking, security, fleets of similar VMS, etc.), while teams need more granularity to access managed services used by the applications they maintain.

The approach also adapts to different high level requirements:

• it can be used either for single organizations containing multiple environments, or with multiple organizations dedicated to specific environments (e.g. prod/nonprod), as the environment split is implemented at the project or lower folder level
• it adapts to complex scenarios, with different countries or corporate entities using the same GCP organization, as core services are typically shared, and/or an extra layer on top can be used as a drop-in to implement the country/entity separation

Additionally, a few critical benefits are directly provided by this design:

• core services are clearly separated, with very few touchpoints where IAM and security policies need to be applied (typically their top-level folder)
• adding a new set of core services (e.g. shared GKE clusters) is a trivial operation that does not break the existing design
• grouping application resources and services using teams or business logic is a flexible approach, which maps well to typical operational or budget requirements
• automation stages (e.g. Networking) can be segregated in a simple and effective way, by creating the required service accounts and buckets for each stage here, and applying a handful of IAM roles to the relevant folder

For a discussion on naming, please refer to the Bootstrap stage documentation, as the same approach is shared by all stages.

Department or team folders

Top-level folders for teams or departments can be easily created via the top_level_folders variable or the associated factory, which expose the full power of the underlying folder module.

The suggestion is to use this feature sparingly so as to keep the top level of the hierarchy simple, and minimize changes to this stage due to its security implications. One approach is to create a grouping folder (e.g. Departments or Teams) here, and delegate management of lower level folders to the project factory stage.

Top-level folders also support defining associated resources for automation, and auto-created provider files to bootstrap Infrastructure and Code. An example is provided below.

Multitenancy

Multitenancy is supported via a separate stage, which is entirely optional and can be applied after resource management has been deployed. For simpler use cases that do not require complex organization-level multitenancy, top-level folders can be used in combination with the project factory stage support for folder and project management.

Workload Identity Federation and CI/CD

This stage also implements optional support for CI/CD, much in the same way as the bootstrap stage. The only difference is on Workload Identity Federation, which is only configured in bootstrap and made available here via stage interface variables (the automatically generated .tfvars files).

For details on how to configure CI/CD please refer to the relevant section in the bootstrap stage documentation.

How to run this stage

This stage is meant to be executed after the bootstrap stage has run, as it leverages the automation service account and bucket created there. The relevant user groups must also exist, but that's one of the requirements for the previous stage too, so if you ran that successfully, you're good to go.

It's of course possible to run this stage in isolation, but that's outside the scope of this document, and you would need to refer to the code for the bootstrap stage for the actual roles needed.

Before running this stage, you need to make sure you have the correct credentials and permissions, and localize variables by assigning values that match your configuration.

Provider and Terraform variables

As all other FAST stages, the mechanism used to pass variable values and pre-built provider files from one stage to the next is also leveraged here.

The commands to link or copy the provider and terraform variable files can be easily derived from the stage-links.sh script in the FAST root folder, passing it a single argument with the local output files folder (if configured) or the GCS output bucket in the automation project (derived from stage 0 outputs). The following examples demonstrate both cases, and the resulting commands that then need to be copy/pasted and run.

../../stage-links.sh ~/fast-config

# copy and paste the following commands for '1-resman'

ln -s ~/fast-config/providers/1-resman-providers.tf ./
ln -s ~/fast-config/tfvars/0-globals.auto.tfvars.json ./
ln -s ~/fast-config/tfvars/0-bootstrap.auto.tfvars.json ./

../../stage-links.sh gs://xxx-prod-iac-core-outputs-0

# copy and paste the following commands for '1-resman'

gcloud storage cp gs://xxx-prod-iac-core-outputs-0/providers/1-resman-providers.tf ./
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-globals.auto.tfvars.json ./
gcloud storage cp gs://xxx-prod-iac-core-outputs-0/tfvars/0-bootstrap.auto.tfvars.json ./

Impersonating the automation service account

The preconfigured provider file uses impersonation to run with this stage's automation service account's credentials. The gcp-devops and organization-admins groups have the necessary IAM bindings in place to do that, so make sure the current user is a member of one of those groups.

Variable configuration

Variables in this stage -- like most other FAST stages -- are broadly divided into three separate sets:

• variables which refer to global values for the whole organization (org id, billing account id, prefix, etc.), which are pre-populated via the 0-globals.auto.tfvars.json file linked or copied above
• variables which refer to resources managed by previous stage, which are prepopulated here via the 0-bootstrap.auto.tfvars.json file linked or copied above
• and finally variables that optionally control this stage's behaviour and customizations, and can to be set in a custom terraform.tfvars file

The latter set is explained in the Customization sections below, and the full list can be found in the Variables table at the bottom of this document.

Note that the outputs_location variable is disabled by default, you need to explicitly set it in your terraform.tfvars file if you want output files to be generated by this stage. This is a sample terraform.tfvars that configures it, refer to the bootstrap stage documentation for more details:

outputs_location = "~/fast-config"

Running the stage

Once provider and variable values are in place and the correct user is configured, the stage can be run:

terraform init
terraform apply

Customizations

Toggling features

Some FAST features used here and by the following stages can be enabled or disabled using the fast_features variables.

The fast_features variable consists of 5 toggles:

• data_platform controls the creation of required resources (folders, service accounts, buckets, IAM bindings) to deploy the 3-data-platform stage
• gcve controls the creation of required resources (folders, service accounts, buckets, IAM bindings) to deploy the 3-gcve stage
• gke controls the creation of required resources (folders, service accounts, buckets, IAM bindings) to deploy the 3-gke-multitenant stage
• project_factory controls the creation of required resources (folders, service accounts, buckets, IAM bindings) to deploy the 2-project-factory stage
• sandbox controls the creation of a "Sandbox" top level folder with relaxed policies, intended for sandbox environments where users can experiment
• teams controls the creation of the top level "Teams" folder used by the teams feature in resman.

Top-level folders

The top_level_folders variable and associated factory allow simple definition of additional top-level folders, and associated configurations.

The following is an example that creates two folders via tfvars, and also configures the factory to define additional folders via YAML. Folders defined via the variable or factory files support the same interface of the folder module.

factories_config = {
  top_level_folders = "~/fast-config/data/1-resman/folders"
}
top_level_folders = {
  test-1 = {
    name = "Test 1"
    iam = {
      "roles/viewer" = [
        "group:[email protected]"
      ]
    }
  }
  test-2 = {
    # disable creation of the automation SA and bucket
    automation = {
      enable = false
    }
    name = "Test 2"
  }
}

# ~/fast-config/data/1-resman/folders/test-4.yaml
name: Test 4
automation: null
iam:
  roles/browser:
    - domain:example.org

Secure tags

This stage manages Secure Tags at the organization level, via two sets of keys and values:

• a default set of tags used by FAST itself in specific IAM conditions that allow automation service accounts to gain organization-level privileges or specific access to parts of the resource management hierarchy
• an optional set of user-defined tags that can be used in organization policy or IAM conditions

The first set of default tags cannot be overridden and defines the following keys and values (key names can be changed via the tag_names variable):

• context to identify parts of the resource hierarchy, with data, gke, networking, sandbox, security and teams values
• environment to identify folders and projects belonging to specific environments, with development and production values

The second set is optional and allows defining a custom tag hierarchy, including IAM bindings that can refer to specific identities, or to the internally defined automation service accounts via their names, like in the following example:

tags = {
  my-custom-tag = {
    values = {
      eggs = {}
      spam = {
        description = "Example tag value."
        iam = {
          "roles/resourcemanager.tagUser" = ["sandbox"]
        }
      }
    }
  }
}

IAM

The folder_iam variable can be used to manage authoritative bindings for all top-level folders. For additional control, IAM roles can be easily edited in the relevant branch-xxx.tf file, following the best practice outlined in the bootstrap stage documentation of separating user-level and service-account level IAM policies through the IAM-related variables (iam, iam_bindings, iam_bindings_additive) of the relevant modules.

A full reference of IAM roles managed by this stage is available here.

Files

name	description	modules	resources
billing.tf	Billing resources for external billing use cases.		google_billing_account_iam_member
branch-data-platform.tf	Data Platform stages resources.	folder · gcs · iam-service-account
branch-gcve.tf	GCVE stage resources.	folder · gcs · iam-service-account
branch-gke.tf	GKE multitenant stage resources.	folder · gcs · iam-service-account
branch-networking.tf	Networking stage resources.	folder · gcs · iam-service-account
branch-nsec.tf	Network security stage resources.	gcs · iam-service-account
branch-project-factory.tf	Project factory stage resources.	gcs · iam-service-account
branch-sandbox.tf	Sandbox stage resources.	folder · gcs · iam-service-account
branch-security.tf	Security stage resources.	folder · gcs · iam-service-account
checklist.tf	None	folder
cicd-data-platform.tf	CI/CD resources for the data platform branch.	iam-service-account
cicd-gcve.tf	CI/CD resources for the GCVE branch.	iam-service-account
cicd-gke.tf	CI/CD resources for the GKE multitenant branch.	iam-service-account
cicd-netsec.tf	CI/CD resources for the networking branch.	iam-service-account
cicd-networking.tf	CI/CD resources for the networking branch.	iam-service-account
cicd-project-factory.tf	CI/CD resources for the project factories.	iam-service-account
cicd-security.tf	CI/CD resources for the security branch.	iam-service-account
iam.tf	Organization or root node-level IAM bindings.
main.tf	Module-level locals and resources.
organization.tf	Organization policies.	organization
outputs-files.tf	Output files persistence to local filesystem.		local_file
outputs-gcs.tf	Output files persistence to automation GCS bucket.		google_storage_bucket_object
outputs.tf	Module outputs.
tenant-logging.tf	Audit log project and sink for tenant root folder.	bigquery-dataset · gcs · logging-bucket · pubsub
tenant-root.tf	None	folder · project
top-level-folders.tf	None	folder · gcs · iam-service-account
variables-fast.tf	FAST stage interface.
variables.tf	Module variables.

Variables

name	description	type	required	default	producer
automation	Automation resources created by the bootstrap stage.	object({…})	✓		0-bootstrap
billing_account	Billing account id. If billing account is not part of the same org set is_org_level to false. To disable handling of billing IAM roles set no_iam to true.	object({…})	✓		0-bootstrap
logging	Logging configuration for tenants.	object({…})	✓		1-tenant-factory
organization	Organization details.	object({…})	✓		0-bootstrap
prefix	Prefix used for resources that need unique names. Use 9 characters or less.	string	✓		0-bootstrap
cicd_repositories	CI/CD repository configuration. Identity providers reference keys in the automation.federated_identity_providers variable. Set to null to disable, or set individual repositories to null if not needed.	object({…})		null
custom_roles	Custom roles defined at the org level, in key => id format.	object({…})		null	0-bootstrap
factories_config	Configuration for the resource factories or external data.	object({…})		{}
fast_features	Selective control for top-level FAST features.	object({…})		{}
folder_iam	Authoritative IAM for top-level folders.	object({…})		{}
groups	Group names or IAM-format principals to grant organization-level permissions. If just the name is provided, the 'group:' principal and organization domain are interpolated.	object({…})		{}	0-bootstrap
locations	Optional locations for GCS, BigQuery, and logging buckets created here.	object({…})		{}	0-bootstrap
outputs_location	Enable writing provider, tfvars and CI/CD workflow files to local filesystem. Leave null to disable.	string		null
root_node	Root node for the hierarchy, if running in tenant mode.	string		null	0-bootstrap
tag_names	Customized names for resource management tags.	object({…})		{}
tags	Custom secure tags by key name. The iam attribute behaves like the similarly named one at module level.	map(object({…}))		{}
top_level_folders	Additional top-level folders. Keys are used for service account and bucket names, values implement the folders module interface with the addition of the 'automation' attribute.	map(object({…}))		{}

Outputs

name	description	sensitive	consumers
cicd_repositories	WIF configuration for CI/CD repositories.
dataplatform	Data for the Data Platform stage.
folder_ids	Folder ids.
gcve	Data for the GCVE stage.		03-gcve
gke_multitenant	Data for the GKE multitenant stage.		03-gke-multitenant
networking	Data for the networking stage.
project_factories	Data for the project factories stage.
providers	Terraform provider files for this stage and dependent stages.	✓	02-networking · 02-security · 03-dataplatform · 03-network-security
sandbox	Data for the sandbox stage.		xx-sandbox
security	Data for the networking stage.		02-security
tfvars	Terraform variable files for the following stages.	✓