A Guided Tour

Contents

• A Guided Tour
• The Big Picture
• Reference Implementation
  • Accounts & Organizational Units
• Conventions

A Guided Tour

The Metastructure Template Repo is implemented as an NPM package.

This is an unusual choice for a Terraform project. Terraform itself doesn't require it, and while Metastructure is an NPM package, it could easily be installed globally and used in any project context that supports Node.js.

So why an NPM package? In brief, because any infrastructure project is about WAY more than Terraform! You need to release version-controlled code according to some well-defined SDLC and perform other tasks that are best managed within the context of SOME kind of package manager.

Since the goal is to produce a reference implementation that you can put to work right out of the box, I've chosen NPM as the box. This way, you can clone the repo, install your dependencies, and start building infrastructure right away!

The Big Picture

The whole point of using Terraform is to generate very safe, rather WET code from a configuration document and a set of Handlebars templates that are very DRY indeed!

• Why generate WET code? Because Terraform code in general is just not very DRY. If you're creating similar resources in multiple accounts, you'll have to specify them in multiple places with different providers. If you want to alter that pattern, you'll have to change it the SAME way EVERYWHERE. Sooner or later, that's a recipe for disaster... but in Terraform-land, it's just how things work.

• Why the focus on safety? Terraform has features (like iterators and modules) that are intended to DRY up your code base. Unfortunately, these features also introduce an element of risk. Sometimes these risks are worth taking... but Metastructure gives you the option of achieving the same results with code generation templates, leaving your generated code as safe as possible. You get to choose your own adventure.

See The Trouble With Terraform for more on this topic.

So at a high level, here's what you will find:

• The project root contains all of the configuration and machinery required to support the NPM package and utility scripts as well as functions like DevOps. Of key importance is the .metastructure.yml file, which tells Metastructure where to find your project config file. More info on that here.

• The src directory contains all of your infrastructure configuration, templates, and infrastructure code. Within the src directory...

  • metastructure.yml is the project configuration file that drives the code generation process.

  • license.txt contains the text of a licensing header that will be applied to all of your code files.

  • The templates directory contains global templates that will apply to every Terraform workspace.

  • The modules directory is intended to contain your local Terraform modules. Presently it contains a single module, global, whose purpose is to make the contents of your metastructure.yml config file directly accessible from within your Terraform code (your templates already have access to it, of course).

  • Any other directory in src reflects a specific Terraform workspace. Right now these are...

    • bootstrap - Sets up your AWS Organization, accounts, organizational units, and SSO permissions. EVERY workspace directory contains...

      • A templates directory containing the code generation templates specific to that workspace.

      • workspace.local.yml.template is a template file you can use to create local Metastructure CLI overrides specific to that workspace.

      • Your generated code files.

      • Any other artifacts that could be declaratively written rather than generated with Metastructure.

As noted in Design Principles, you could lay out your project differently and Metastructure would still work this fine. But we can't get started without making some choices, so these are the choices I've made.

I'm certainly open to suggestions for improvement. Start a discussion if you'd like to share a better idea!

Reference Implementation

As described in Design Principles, the goal of the Metastructure Template Repo is to serve as a reference implementation of an AWS Organization. This implemetation should embody every element addressed in the AWS Well-Architected framework, and moreover should be sufficiently generic and extensible that as a developer you can easily ADD new functionaility without having to CHANGE existing functionality.

With respect to the feature set, we have a long way to go. But the design principles are 100% in effect, such that you should be able to add key new features without opening the box on existing templates. And if you keep in touch while you're doing it, I'll use your contributions to bring the reference implementation more in line with the ideal!

Meanwhile, the sections below describe the features that already form part of the reference implementation.

Accounts & Organizational Units

The image above illustrates the accounts & organizational units currently specified in the Metastructure Template Repo project config.

These accounts, organizational units, and the relationships between them are COMPLETELY driven by configuration!

You can specify new accounts (or import existing ones) and manage OU assignments in the accounts section of your project config. The Handlebars template at [src/bootstrap/templates/accounts.hbs] generates the Terraform code that creates & manages these accounts.

You can specify, import & manage Organizational Units in the organizational_units section of your project config. The Handlebars template at [src/bootstrap/templates/organizational_units.hbs] generates the Terraform code that creates & manages these OUs.

Conventions

As written, the Metastructure Template Repo expresses a number of conventions worth knowing about.

• The structure is as laid out above. If you change it, you'll have to make corresponding changes to your project config so Metastructure can find your templates and generate your code where it belongs.

• Generated files are prefixed with an underscore (e.g. _providers.tf) so they visually sort together in your IDE.

• File names that either have a .local extension or contain .local. in the file name (e.g. workspace.local.yml) are gitignored and will not be persisted to your remote repository. These are intended for local configuration overrides & secrets.

• Files that have the .template extension are NOT gitignored, whether or not they contain the local token. These will be committed to your remote repository and are intended to be used as templates for local files.

• When you update your project config using the -u or --update-config flag (see Config Updates), any comments you have added to your config file will be preserved. You can and should take advantage of this feature to document your config liberally!

• All code generation templates contain a header warning developers not to edit the generated file directly. This is not required, but it is a good practice.

• As described in The Trouble With Terraform, third-party Terraform templates introduce an element of risk. This is a choice you can make, but it isn't one I want to make for you. So the Metastructure Template Repo implementation is restricted to native Terraform data sources & resources.