A Guided Tour

Contents

• A Guided Tour
• The Big Picture
• Reference Implementation
  • Accounts & Organizational Units
  • Terraform State
  • Single Sign-On
    • Groups
    • Permission Sets
    • Customer Managed Policies
    • Current Limitations
• Conventions

A Guided Tour

The Metastructure Template Repo is implemented as an NPM package.

This is an unusual choice for a Terraform project. Terraform itself doesn't require it, and while Metastructure is an NPM package, it could easily be installed globally and used in any project context that supports Node.js.

So why an NPM package? In brief, because any infrastructure project is about WAY more than Terraform! You need to release version-controlled code according to some well-defined SDLC and perform other tasks that are best managed within the context of SOME kind of package manager.

Since the goal is to produce a reference implementation that you can put to work right out of the box, I've chosen NPM as the box. This way, you can clone the repo, install your dependencies, and start building infrastructure right away!

The Big Picture

The whole point of using Terraform is to generate very safe, rather WET code from a configuration document and a set of Handlebars templates that are very DRY indeed!

• Why generate WET code? Because Terraform code in general is just not very DRY. If you're creating similar resources in multiple accounts, you'll have to specify them in multiple places with different providers. If you want to alter that pattern, you'll have to change it the SAME way EVERYWHERE. Sooner or later, that's a recipe for disaster... but in Terraform-land, it's just how things work.

• Why the focus on safety? Terraform has features (like iterators and modules) that are intended to DRY up your code base. Unfortunately, these features also introduce an element of risk. Sometimes these risks are worth taking... but Metastructure gives you the option of achieving the same results with code generation templates, leaving your generated code as safe as possible. You get to choose your own adventure.

See The Trouble With Terraform for more on this topic.

So at a high level, here's what you will find:

• The project root contains all of the configuration and machinery required to support the NPM package and utility scripts as well as functions like DevOps. Of key importance is the .metastructure.yml file, which tells Metastructure where to find your project config file. More info on that here.

• The src directory contains all of your infrastructure configuration, templates, and infrastructure code. Within the src directory...

  • metastructure.yml is the project configuration file that drives the code generation process.

  • license.txt contains the text of a licensing header that will be applied to all of your code files.

  • The templates directory contains global templates that will apply to every Terraform workspace.

  • The modules directory is intended to contain your local Terraform modules. Presently it contains a single module, global, whose purpose is to make the contents of your metastructure.yml config file directly accessible from within your Terraform code (your templates already have access to it, of course).

  • Any other directory in src reflects a specific Terraform workspace. Right now these are...

    • bootstrap - Sets up your AWS Organization, accounts, organizational units, and SSO permissions. EVERY workspace directory contains...

      • A templates directory containing the code generation templates specific to that workspace.

      • workspace.local.yml.template is a template file you can use to create local Metastructure CLI overrides specific to that workspace.

      • Your generated code files.

      • Any other artifacts that could be declaratively written rather than generated with Metastructure.

As noted in Design Principles, you could lay out your project differently and Metastructure would still work this fine. But we can't get started without making some choices, so these are the choices I've made.

I'm certainly open to suggestions for improvement. Start a discussion if you'd like to share a better idea!

Reference Implementation

As described in Design Principles, the goal of the Metastructure Template Repo is to serve as a reference implementation of an AWS Organization. This implemetation should embody every element addressed in the AWS Well-Architected framework, and moreover should be sufficiently generic and extensible that as a developer you can easily ADD new functionaility without having to CHANGE existing functionality.

With respect to the feature set, we have a long way to go. But the design principles are 100% in effect, such that you should be able to add key new features without opening the box on existing templates. And if you keep in touch while you're doing it, I'll use your contributions to bring the reference implementation more in line with the ideal!

Meanwhile, the sections below describe the features that already form part of the reference implementation.

Accounts & Organizational Units

Reference Implementation Organization Structure

The image above illustrates the accounts & organizational units currently specified in the Metastructure Template Repo project config.

In addition to creating physical accounts, Metastructure also designates specific physical accounts as key accounts for specific functions. For example, Terraform state is located in the terraform_state key account, which could be assigned to any physical account. These assignments are made in project config at organization.key_accounts.

These accounts, key accounts, organizational units, and the relationships between them are COMPLETELY driven by configuration!

You can specify new accounts (or import existing ones) and manage OU assignments in the accounts section of your project config. The Handlebars template at [src/bootstrap/templates/accounts.hbs] generates the Terraform code that creates & manages these accounts.

You can specify, import & manage Organizational Units in the organizational_units section of your project config. The Handlebars template at [src/bootstrap/templates/organizational_units.hbs] generates the Terraform code that creates & manages these OUs.

Terraform State

Resources related to Terraform state are defined in src/bootstrap/state.tf.

Note that this is a simple Terraform file, not a Handlebars template! The resources defined here are unique, so do not need to be defied across accounts. Also, resource names are generated from tokens passed into the global configuration module from the project config.

This is an importan example of Metastructure's flexibility! You CAN define resources in Handlebars templates, but you don't strictly HAVE to unless you gain a useful efficiency by doing so.

Note also that these resources are defined in the terraform_state key account. In the current configuration, this resolves to the Shared Services account. This assignment could be changed simply by setting the terraform_state key account assignment in the project config to a different value. The names of these resources can be changed by altering the token values at terraform.state.

organization:
  key_accounts:
    # Change this value to assign Terraform state to a different account.
    terraform_state: shared_services
terraform:
  # Change these values to rename Terraform state resources.
  state:
    bucket: '{{organization.tokens.namespace}}-terraform-state'
    key: terraform.tfstate
    lock_table: terraform-state-lock

All of these configurations, plus interactions with other templates, produce the resources illustrated below:

Terraform state resources (click to zoom)

Single Sign-On

AWS SSO Use Cases (click to zoom)

Single Sign-On (SSO) is arguably the most important aspect of AWS Organizations.

In AWS, cross-account permissions are mediated by IAM roles. A user on one account can assume a role on another account, provided the user is granted permission to assume the role on both ends of the relationship. Since this geometry is intrinsically double-ended, in a multi-account environment it can get very complicated, very fast.

IAM Identity Center simplifies this arrangement by abstracting the specification of account & permission relationships away from the provisioning of IAM Roles. If you create the right SSO users, groups, and permission sets with the correct permissions and assign them to the right accounts, Identity Center will automatically create the roles you need in the accounts where they belong.

Metastructure simplifies this even further by automating the (still very complex) creation & assignment of SSO users, groups, permission sets, and Organization account policies. So now, all you have to do is:

• Efficiently specify your SSO users, groups, permission sets, policies & their respective relationships in the sso section of your project config.

• Create aws_iam_policy_document data sources (like these) that lay out the specific permissions you want to assign to your permission sets from the relevant accounts.

Metastructure will generate all the Terraform code required to realize these complex relationships, as illustrated below:

Metastructure and AWS SSO (click to zoom)

Groups

The following SSO Groups are currently configured in the Metastructure Template Repo reference implementation:

Group	Description	Permission Sets	Accounts
TerraformAdmin	Terraform administrators can create & manage all resources in all accounts.	TerraformAdmin	ALL
TerraformDeployment	Terraform deployment users can create & manage all unprotected resources in all accounts.	TerraformDeployment	ALL

Permission Sets

The following SSO Permission Sets are currently configured in the Metastructure Template Repo reference implementation. Policies marked with * are AWS Managed Policies and are not defined withn the implementation:

Permission Set	Description	Policies
TerraformAdmin	Permits creation & management of all resources.	*AdministratorAccess SSOTerraformStateWriter
TerraformDeployment	Permits creation & management of all unprotected resources.	SSOTerraformStateWriter SSOUnprotectedResourceWriter

Customer Managed Policies

The following Customer Managed Policies are currently configured in the Metastructure Template Repo reference implementation. These policies are created in Organization accounts as specified in the Permission Set, Group & Account relationships specified above:

Policy	Description
SSOTerraformStateWriter	Permits writing to the Terraform state bucket & state lock table.
SSOUnprotectedResourceWriter	Permits creation & management of unprotected resources. This policy document's current implementation is a placeholder and requires more work!

Current Limitations

The current SSO implementation does not support external identity providers like Google Workspace or Azure AD. It is limited to AWS SSO users & groups.

This issue addresses this requirement. It isn't a heavy lift, so if you need it, please feel free to contribute! Otherwise I'll get to it myself very soon.

Conventions

As written, the Metastructure Template Repo expresses a number of conventions worth knowing about.

• The structure is as laid out above. If you change it, you'll have to make corresponding changes to your project config so Metastructure can find your templates and generate your code where it belongs.

• Generated files are prefixed with an underscore (e.g. _providers.tf) so they visually sort together in your IDE.

• File names that either have a .local extension or contain .local. in the file name (e.g. workspace.local.yml) are gitignored and will not be persisted to your remote repository. These are intended for local configuration overrides & secrets.

• Files that have the .template extension are NOT gitignored, whether or not they contain the local token. These will be committed to your remote repository and are intended to be used as templates for local files.

• When you update your project config using the -u or --update-config flag (see Config Updates), any comments you have added to your config file will be preserved. You can and should take advantage of this feature to document your config liberally!

• All code generation templates contain a header warning developers not to edit the generated file directly. This is not required, but it is a good practice.

• As described in The Trouble With Terraform, third-party Terraform templates introduce an element of risk. This is a choice you can make, but it isn't one I want to make for you. So the Metastructure Template Repo implementation is restricted to native Terraform data sources & resources.