Skip to content

Hello, Enterprise!

Jason Williscroft edited this page Aug 14, 2024 · 1 revision

Contents

Hello, Enterprise!

This is the fun part! In this guide, you will use the Metastructure Template Repo to create a multi-account AWS Organization featuring single sign-on (SSO), remote Terraform state management, and the beginnings of a proper audit logging system.

Expect this paragraph ☝️ to change significantly and soon: before long, this walk-through will include CloudTrail audit logging and a host of other features bringing this template more closely in line with the AWS Well-Architected Framework.

Meanwhile, if you already have dev tools set up and you don't dally, we'll get it all done in about 20 minutes. So let's get started!

The Big Picture

Let's assume we are starting from scratch. We're going to follow these steps:

  1. Set up your development environment & configure your project. More info...

  2. Create a new master AWS account & enable AWS Organizations. More info...

  3. Create a bootstrap-admin IAM user with broad admin permissions across the Organization. You'll use this user to bootstrap SSO, and then you'll switch to SSO authentication. More info...

  4. Run Metastructure to create the accounts, organizational units, and other resources defined in your bootstrap congiguration. More info...

  5. Switch to SSO authentication & migrate your state to the remote S3 Terraform backend. More info...

  6. Log into the AWS Management Console with SSO and verify your implementation. More info...

Setting the Stage

Let's accomplish a couple of preliminary tasks.

First, follow the instructions in Setting Up Your Dev Environment to get a clone of the Metastructure Template Repo up & running in your local environment.

For the sake of simplicity, I will assume you have installed Metastructure as a global package and can run it like metastructure ... from the command line. If not, you will have to run it like npx metastructure ....

Every new AWS account must be created with a unique email address, and these emails can never be reused. For simplicity, the template project config is articulated in a way that supports Gmail's plus-alias feature, which redirects any email of the form owner+alias@gmail.com back to owner@gmail.com. This feature also applies to Google Workspace email accounts.

If you don't have either a Gmail address or a Google Workspace email, for the sake of simplicity you might want to sign up for a new Gmail address now.

We are going to use three short string tokens to create your AWS Organization. We'll derive owner and domain from your email address. For example, if your email is me@gmail.com then owner = 'me' and domain = 'gmail.com'.

Choose a namespace token that will be unique to this specific AWS Organization. For Metastructure development, I use metastructure followed by a numerical iterator, like metastructure-001. Use whatever you like. Just remember you'll probably do this more than once.

Now open up your src/metastructure.yml file. Under organization.tokens, set owner, domain, and namespace to the values you chose above and save the file.

If you were contributing to the Metastructure Template Repo, you'd want to override these values in src/metastructure.local.yml instead of changing them in src/metastructure.yml, in order to preserve the generic values in the template! See Template vs Project Mode for more info.

Create the Master Account

This needs to be done manually. Follow these steps:

  1. Run metastructure -w bootstrap -d and observe the config object output to your console. Find the value at accounts.master.email (near the top). Assuming you completed the steps in Setting The Stage above, this is the email address you will use to create your master account.

  2. While you're at it, find sso.users in your config object. There should be a single entry there. Its key should be your owner token, and its email value should be your email address. If this isn't the case, go back to Setting the Stage and fix your token values.

  3. Sign up for a new AWS account using the master account email you identified above. Remember, you won't be able to use it again to create another account! This process will take a few minutes and you'll need to provide a credit card number, but you won't be charged unless you exceed the free tier limits.

  4. Sign into the new account as the root user (you may be signed in automatically) and choose your desired home region in the upper right corner of the AWS Console.

  5. Visit the IAM Identity Center page and enable IAM Identity Center with AWS Organizations (the default choice).

  6. Make a note of the following values in the IAM Identity Center right sidebar & enter them into src/metastructure.yml at the locations indicated below:

Value Example Location in src/metastructure.yml
Region us-east-1 organization.region
Organization ID o-1234567890 organization.id
AWS SSO access portal URL https://d-1234567890.awsapps.com/start sso.start_url

Create the bootstrap-admin User

This must also be done manually.

  1. In the IAM console (not IAM Identity Center!) create new IAM user bootstrap-admin, attaching AWS managed policy AdministratorAccess. Once the user is created, attach an additional inline policy using the contents of BootstrapAdminPolicy.json.

  2. From the bootstrap-admin user page Security Credentials tab, create an access key and secret key (choose the Other use case or AWS will hassle you with alternatives). Save these in a secure location.

  3. Log out of the AWS console. You shouldn't need to log back in as the root user again.

Create Resources

We're almost ready to run Metastructure! First, complete these steps:

These steps assume you installed suggested VS Code extensions as described in Setting Up Your Dev Environment. If you didn't, either go back & fix it or just follow along using your own tools!

  1. In the VS Code command pallette find AWS: Add New Connection.

  2. Choose IAM Credentials as the sign-in option and enter the access key and secret key for the bootstrap-admin user. Choose a profile name unique to your namespace, like META-BOOTSTRAP-000.

  3. Enter the new profile name in src/metastructure.yml at workspaces.bootstrap.cli_defaults.aws_profile and save the file.

Now run the following command (remember, we're assuming you installed Metastructure as a global package):

metastructure -w bootstrap -g terraform init

Metastructure will generate your bootstrap workspace and initialize your back end. Now run:

metastructure -w bootstrap -g terraform plan

Metastructure will create your bootstrap workspace and generate a Terraform plan.

Now run:

metastructure -w bootstrap -g -u terraform apply

You will see the the Terraform plan again, then the familiar terraform apply confirmation prompt. Once you confirm, Terraform will create your bootstrap accounts & resources.

Terraform may throw an error that looks something like this: Error creating AWS DynamoDB Table ... The AWS Access Key Id needs a subscription for the service. This usually means a new account wasn't fully provisioned before Terraform tried to create resources in it. Just wait a few seconds and run the command again! It will pick up right where it left off.

Once the run succeeds, Metastructure will write the output back to src/metastructure.yml. If you examine this file, you'll see an id value under each Account, Organizational Unit, and SSO User.

You've bootstrapped your AWS Organization! Now let's switch to SSO Authentication and migrate your state.

Switch to SSO & S3 Backend

In your project config file under workspaces.bootstrap, comment out the assume_role & aws_profile keys and uncomment the permission_set key.

Now run:

metastructure -w bootstrap -g -u terraform init -migrate

Metastructure will regenerate your code for SSO authentication and open a browser window for you to log in. Your username will be test. As this is your first login, you'll need to recover your password using the account email we used above.

Once you've reset your password, AWS will walk you through MFA setup.

By the time you complete this process, the Metastructure execution have timed out. If it does, just run the command again!

Once you have successfully authenticated through SSO, Metastructure will migrate your Terraform state to the remote S3 backend.

Now let's verify our implementation!

Verify Your Implementation

Open src/metastructure.yml, copy the url from sso.start_url, and open it in your browser. Log in using the same SSO credentials you used in the previous step.

You should see your AWS access portal, listing all of the accounts in your infrastructure. When you expand any of these accounts, you'll see the permission sets you can use to access these accounts. Right now the only permission set assigned to your account will be TerraformAdmin.

If you choose the master account (it will have the name you chose when you created the Organization), you can visit the IAM Identity center & review all of your Users, Groups & Permission Sets, as well as your tree of Organizational Units & Accounts. These should all reflect the assignmemts in src/metastructure.yml.

If you visit your Shared Services account, you'll find your Terraform S3 state bucket and DynamoDB state lock table.

All of your accounts will have an S3 access log bucket, as well as policy objects to support your SSO permission sets.

If you can see all of this, congratulations! You've successfully bootstrapped your AWS Organization using Metastructure!