Skip to content

Latest commit

 

History

History
724 lines (596 loc) · 74.3 KB

installation.md

File metadata and controls

724 lines (596 loc) · 74.3 KB
Raw

1. Accelerator Installation and Upgrade Guide

We encourage customers installing the Accelerator to get the support of their local AWS account team (SA, TAM, CSM, Proserve) to assist with the installation of the Accelerator, as the Accelerator leverages, deploys, or orchestrates over 30 different AWS services.

Users are strongly encouraged to also read the Accelerator Operations/Troubleshooting Guide before installation. The Operations/Troubleshooting Guide provides details as to what is being performed at each stage of the installation process, including detailed troubleshooting guidance.

These installation instructions assume one of the prescribed architectures is being deployed.

2. Installation

2.1. Prerequisites

2.1.1. General

  • Management or root AWS Organization account (the AWS Accelerator cannot be deployed in an AWS sub-account)
    • No additional AWS accounts need to be pre-created before Accelerator installation
  • If required, a limit increase to support your desired number of new AWS sub-accounts (default limit is 10 sub-accounts)
  • Valid Accelerator configuration file, updated to reflect your requirements (see below)
  • Determine your primary or Accelerator control or home region, this is the AWS region in which you will most often operate
  • Government of Canada customers are still required to do a standalone installation at this time, please request standalone installation instructions from your Account SA or TAM
  • The Accelerator can be installed into existing AWS Organizations - see caveats and notes in section 4 below
  • Existing AWS Landing Zone Solution (ALZ) customers are required to remove their ALZ deployment before deploying the Accelerator. Scripts are available to assist with this process.
  • Changes to the Accelerator codebase are strongly discouraged unless they are contributed and accepted back to the solution. Code customization will block the ability to upgrade to the latest release and upgrades are encouraged to be done between quarterly to semi-annually. The solution was designed to be extremely customizable without changing code, existing customers following these guidelines have been able to upgrade across more than 50 Accelerator releases, while maintaining their customizations and gaining the latest bug fixes, features and enhancements without any developer or professional services based support. Please see this FAQ for more details.

2.2. Production Deployment Planning

2.2.1. General

For any deployment of the Accelerator which is intended to be used for production workloads, you must evaluate all these decisions carefully. Failure to understand these choices could cause challenges down the road. If this is a "test" or "internal" deployment of the Accelerator which will not be used for production workloads, you can leave the default config values.

2.2.2. OU Structure Planning

Plan your OU and core account structure carefully. By default, we suggest: Security, Infrastructure, Central, Sandbox, Dev, Test, Prod.

  • The Security OU will contain the Security account, the Log Archive account, and the Organization Management account.
  • The Infrastructure OU will hold the remainder of the accounts shared or utilized by the rest of the organization (Shared Network, Perimeter, and Operations).
  • The remainder of the OUs correspond with major permission shifts in the SDLC cycle and NOT every stage an organization has in their SDLC cycle (i.e. QA or pre-prod would be included in one of the other OUs).
  • The Central OU is used to hold accounts with workloads shared across Dev, Test, and Prod environments like centralized CI/CD tooling.
  • The v1.5.0 release aligns the Accelerator OU and account structure with AWS multi-account guidance, splitting the core OU into the Security and Infrastructure OUs.

Note: While OUs can be renamed or additional OUs added at a later point in time, deployed AWS accounts CANNOT be moved between top-level OUs (guardrail violation), nor can top-level OUs easily be deleted (requires deleting all AWS accounts from within the OU first).

2.2.3. Network Configuration Planning

If deploying the prescriptive architecture using the Full or Lite sample config files, you will need the following network constructs:

  1. Six (6) RFC1918 Class B address blocks (CIDR's) which do not conflict with your on-premise networks (a single /13 block works well)
  • VPC CIDR blocks cannot be changed after installation, this is simply the way the AWS platform works, given everything is built on top of them. Carefully consider your address block selection.
  • one block for each OU, except Sandbox which is not routable (Sandbox OU will use a 7th non-routed address block)
  • the "core" Class B range will be split to support the Endpoint VPC and Perimeter VPC (with extra addresses remaining for future use)
  • Given a shared VPC architecture is leveraged (prevents stranded islands of CIDR blocks and reduces networking costs), we have assigned a class B address block to each VPC to future proof the deployment. Smaller customers can successfully deploy with a half class B CIDR block per shared VPC.
  1. Two (2) RFC6598 /23 address blocks (Government of Canada (GC) requirement only)
  • Used for AWS Managed Active Directory (MAD) deployment and perimeter underlay network
  • non-GC customers can replace the RFC6598 address space with the extra unused addresses from the above RFC1918 CIDR range above (the App2 subnets in the Central VPC and the Perimeter VPC address space)
  1. BGP ASN's for network routing, one for each of:
    • Transit Gateway (one unique ASN per TGW, multi-region example requires a second ASN)
    • IPSec VPN Firewall Cluster (if deployed)
    • VGW for DirectConnect connectivity (only shown in the config.multi-region-example.json)
  • For example: the Control Tower with Network Firewall example config requires a single BGP ASN for the TGW, the IPSec VPN example requires two BGP ASN's, and the multi-region example requires five unique BGP ASN's.

NOTE: Prior to v1.5.0 CIDR ranges were assigned to each VPC and subnet throughout the config file. This required customers to perform extensive updates across the config file when needing to move to specific IP ranges compatible with a customer's existing on-premise networks. While this is still supported for those wanting to control exactly what address is used on every subnet, the solution has added support for dynamic CIDR assignments and the sample config files have been updated to reflect. New installs will have CIDR's pulled from CIDR pools, defined in the global-options section of the config file with state maintained in DynamoDB. The v1.5.0 custom upgrade guide will provides details on the upgrade process and requirements to migrate to the new CIDR assignment system, if desired. A script was created to assist with this migration.

2.2.4. DNS, Domain Name, TLS Certificate Planning

If deploying the prescriptive architecture, you must decide on:

  1. A unique Windows domain name (organizationaws/organization.aws, organizationcloud/organization.cloud, etc.). Given this is designed as the primary identity store and used to domain join all cloud hosted workloads, changing this in future is difficult. Pick a Windows domain name that does NOT conflict with your on-premise AD domains, ensuring the naming convention conforms to your organizations domain naming standards to ensure you can eventually create a domain trust between the MAD and on-premise domains/forests
  2. DNS Domain names and DNS server IP's for on-premise private DNS zones requiring cloud resolution (can be added in future)
  3. DNS Domain for a cloud hosted public zone "public": ["organization.cloud-nuage.canada.ca"] (can be added in future)
  4. DNS Domain for a cloud hosted private zone "private": ["organization.cloud-nuage.gc.ca"] (can be added in future)
  5. Wildcard TLS certificate for each of the 2 previous zones (can be added/changed in future)

2.2.5. Email Address Planning

  1. While you require a minimum of 6 unique email addresses (1 per sub-account being created), we recommend at least 20 unique email ALIASES associated with a single mailbox, never used before to open AWS accounts, such that you do not need to request new email aliases every time you need to create a new AWS account and they can all be monitored via a single mailbox. These email addresses can never have been used to previously open an AWS account.
  2. You additionally require email addresses for the following additional purposes (these can be existing monitored mailboxes and do not need to be unique):
    • Accelerator execution (state machine) notification events (1 address)
    • High, Medium and Low security alerts (3 addresses if you wish to segregate alerts)
    • Budget notifications

2.2.6. Centralized Ingress/Egress Firewalls

As of v1.5.0 the Accelerator offers multiple automated firewall deployment options:

a) AWS Network Firewall (native AWS Cloud service)

  • Defined in the config file as part of a VPC

b) 3rd party firewalls interconnected to the cloud tenancy via IPSec VPN (Active/Active using BGP + ECMP)

  • Defined in the config file under deployments w/TGW VPN attachments
  • this was the only automated option prior to v1.5.0
  • a sample Fortinet Fortigate configuration is provided (both PAYGO and BYOL supported)
  • For Fortinet BYOL, requires minimum 2 valid license files (evaluation licenses adequate) (can be added in future)

c) 3rd party firewalls interconnected to the cloud tenancy via Gateway Load Balancer (GWLB) in an auto-scaling group

  • Defined in the config file under both deployments and load balancers
  • a sample Checkpoint CloudGuard configuration is provided (both PAYGO and BYOL supported)

d) Customer gateway (CGW) creation, to enable connectivity to on-premises firewalls or manually deployed cloud firewalls

  • Defined in the config file under deployments w/TGW VPN attachments (but without an AMI or VPC association)

Examples of each of the firewall options have been included as variants of the Lite config file example.

Note: While we only provide a single example for each 3rd party implementation today, the implementations are generic and should be usable by any 3rd party firewall vendor, assuming they support the required features and protocols. The two examples were driven by customer demand and heavy lifting by the 3rd party vendor. We look forward to additional vendors developing and contributing additional sample configurations. For new 3rd party integrations, we encourage the use of the GWLB approach.

2.2.7. Other

  1. We recommend installing with the default Accelerator Name (ASEA) and Accelerator Prefix (ASEA-), but allow customization. Prior to v1.5.0 the defaults were (PBMM) and (PBMMAccel-) respectively.
    • the Accelerator name and prefix CANNOT be changed after the initial installation;
    • the Accelerator prefix including the mandatory dash cannot be longer than 10 characters.
  2. New installations, which now leverage Control Tower, require the organization-admin-role be set to AWSControlTowerExecution. Existing standalone installations will continue to utilize their existing role name for the organization-admin-role, typically OrganizationAccountAccessRole, as this role is used by AWS Organizations by default when no role name is specified while creating AWS accounts through the AWS console.
    • the Accelerator leverages this role name to create all new accounts in the organization;
    • this role name, as defined in the config file, MUST be utilized when manually creating all new sub-accounts in the Organization;
    • existing installs wishing to change the role name are required to first deploy a new role with a trust to the root account, in all accounts in the organization.

2.3. Accelerator Pre-Install Steps

2.3.1. General

Before installing, you must first:

  1. Login to the Organization Management (root) AWS account with AdministratorAccess.
  2. Set the region to your desired home region (i.e. ca-central-1)
  3. Install AWS Control Tower:
    • Government of Canada customers are required to skip this step
    • OU and account names can ONLY be customized during initial installation. These values MUST match with the values supplied in the Accelerator config file.
    1. Go to the AWS Control Tower console and click Set up landing zone
    2. Select your home region (i.e. ca-central-1)
      • the Accelerator home region must match the Control Tower home region
    3. Select all regions for Additional AWS Regions for governance, click Next
      • The Control Tower and Accelerator regions MUST be properly aligned
    • If a region is not governed by Control Tower, it must NOT be listed in control-tower-supported-regions
    • To manage a region requires the region:
      • be enabled in Control Tower (if supported)
      • added to the config file control-tower-supported-regions list (if supported)
      • added to the config file supported-regions list (even if not supported by Control Tower, as the Accelerator can manage regions not yet supported by Control Tower, but only when NOT listed in control-tower-supported-regions)
      • While we highly recommend guardrail deployment for all AWS enabled by default regions, at minimum
        • the home region MUST be enabled in Control Tower and must be listed in control-tower-supported-regions
        • both the home-region and ${GBL_REGION} must be listed in supported-regions
    1. For the Foundational OU, leave the default value Security
    2. For the Additional OU provide the value Infrastructure, click Next
    3. Enter the email addresses for your Log Archive and Audit accounts, change the Audit account name to Security, click Next
      • OU and account names can ONLY be customized during initial installation. OU names, account names and email addresses must match identically with the values supplied in the Accelerator config file.
    4. Click setup and wait ~60 minutes for the Control Tower installation to complete
    5. Select Add or register organizational units, Click Add an OU
    6. Type Dev, click Add, wait until the OU is finished provisioning (or it will error)
    7. Repeat step 9 for each OU (i.e. Test, Prod, Central, Sandbox)
    8. Select Account factory, Edit, Subnets: 0, Deselect all regions, click Save
    9. In AWS Organizations, move the Management account from the root OU into the Security OU
  4. Verify:
    1. AWS Organizations is enabled in All features mode
      • if required, navigate to AWS Organizations, click Create Organization, Create Organization
    2. Service Control Policies are enabled
      • if required, in Organizations, select Policies, Service control policies, Enable service control policies
  5. Verify the Organization Management (root) account email address
    • In AWS Organizations, Settings, "Send Verification Request"
    • Once it arrives, complete the validation by clicking the validation link in the email
  6. Create a new KMS key to encrypt your source configuration bucket (you can use an existing key)
    • AWS Key Management Service, Customer Managed Keys, Create Key, Symmetric, and then provide a key name (ASEA-Source-Bucket-Key), Next
    • Select a key administrator (Admin Role or Group for the Organization Management account), Next
    • Select key users (Admin Role or Group for the Organization Management account), Next
    • Validate an entry exists to "Enable IAM User Permissions" (critical step if using an existing key)
      • "arn:aws:iam::123456789012:root", where 123456789012 is your Organization Management account id.
    • Click Finish
    • Select the new key, Select Key Rotation, Automatically rotate this CMK every year, click Save.
  7. Enable "Cost Explorer" (My Account, Cost Explorer, Enable Cost Explorer)
    • With recent platform changes, Cost Explorer may now be auto-enabled (unable to confirm)
  8. Enable "Receive Billing Alerts" (My Account, Billing Preferences, Receive Billing Alerts)
  9. It is extremely important that all the account contact details be validated in the Organization Management (root) account before deploying any new sub-accounts.
  • This information is copied to every new sub-account on creation.
  • Subsequent changes to this information require manually updating it in each sub-account.
  • Go to My Account and verify/update the information lists under both the Contact Information section and the Alternate Contacts section.
  • Please ESPECIALLY make sure the email addresses and Phone numbers are valid and regularly monitored. If we need to reach you due to suspicious account activity, billing issues, or other urgent problems with your account - this is the information that is used. It is CRITICAL it is kept accurate and up to date at all times.

2.3.2. Create GitHub Personal Access Token and Store in Secrets Manager

As of v1.5.0, the Accelerator offers deployment from either GitHub or CodeCommit:

GitHub (recommended)

  1. You require a GitHub access token to access the code repository
  2. Instructions on how to create a personal access token are located here.
  3. Select the scope public_repo underneath the section repo: Full control over private repositories.
  4. Store the personal access token in Secrets Manager as plain text. Name the secret accelerator/github-token (case sensitive).
    • Via AWS console
      • Store a new secret, and select Other type of secrets, Plaintext
      • Paste your secret with no formatting no leading or trailing spaces (i.e. completely remove the example text)
      • Select the key you created above (ASEA-Source-Bucket-Key),
      • Set the secret name to accelerator/github-token (case sensitive)
      • Select Disable rotation

CodeCommit (alternative option)

  • Multiple options exist for downloading the GitHub Accelerator codebase and pushing it into CodeCommit. As this option is only for advanced users, detailed instructions are not provided.
  1. In your AWS Organization Management account, open CodeCommit and create a new repository named aws-secure-environment-accelerator
  2. Go to GitHub and download the repository Source code zip or tarball for the release you wish to deploy
    • Do NOT download the code off the main GitHub branch, this will leave you in a completely unsupported state (and with beta code)
  3. Push the extracted codebase into the newly created CodeCommit repository, maintaining the file/folder hierarchy
  4. Set the default CodeCommit branch for the new repository to main
  5. Create a branch following the Accelerator naming format for your release (i.e. release/v1.5.0)

2.3.3. AWS Internal (Employee) Accounts Only

If deploying to an internal AWS employee account and installing the solution with a 3rd party firewall, you need to enable Private Marketplace (PMP) before starting:

  1. In the Organization Management account go here: https://aws.amazon.com/marketplace/privatemarketplace/create
  2. Click Create a Private Marketplace, and wait for activation to complete
  3. Go to the "Account Groups" sub-menu, click Create account group
  4. Enter an Account Group Title (i.e. Default) and Add the Management (root) account number in Associate AWS account
  5. Associate the default experience New Private Marketplace, then click Create account group and wait for it to create
  6. Go to "Experiences" sub-menu, select New Private Marketplace
  7. Select the "Settings" sub-tab, and click the Not Live slider to make it Live and wait for it to complete
  8. Ensure the "Software requests" slider is set to Requests off and wait for it to complete
  9. Change the name field (i.e. append -PMP) and change the color, so it is clear PMP is enabled for users, click Update
  10. Go to the "Products" sub-tab, then select the All AWS Marketplace products nested sub-tab
  11. Search Private Marketplace for the Fortinet or Checkpoint products and select
  • Fortinet FortiGate (BYOL) Next-Generation Firewall and
  • Fortinet FortiManager (BYOL) Centralized Security Management or
  • CloudGuard Network Security for Gateway Load Balancer - BYOL and
  • Check Point Security Management (BYOL)
  1. Select "Add" in the top right
  • Due to PMP provisioning delays, this sometimes fails when attempted immediately following enablement of PMP or if adding each product individually - retry after 20 minutes.
  1. While not used in this account, you must now subscribe to the two subscriptions and accept the EULA for each product (you will need to do the same in the perimeter account, once provisioned below)
    • To subscribe, select the "Approved products" tab
    • Click on the product you want to subscribe, in this case Fortinet FortiGate (BYOL) Next-Generation Firewall and Fortinet FortiManager (BYOL Centralized Security Management or CloudGuard Network Security for Gateway Load Balancer - BYOL and Check Point Security Management (BYOL)
    • Click on "Continue to Subscribe"
    • Click on "Accept Terms" and wait for subscription to be completed
    • If you are deploying in any region except ca-central-1 or wish to switch to a different license type, you need the new AMI id's. After successfully subscribing, continue one more step and click the “Continue to Configuration”. When you get the below screen, select your region and version (Fortinet v6.4.7, Checkpoint Mgmt R81.10-335.883 and CloudGuard R80.40-294.374 recommended at this time). Marketplace will provide the required AMI id. Document the two AMI id's, as you will need to update them in your config.json file below.

New AMI ID

2.4. Basic Accelerator Configuration

  1. Select a sample config file as a baseline starting point
    • IMPORTANT: Use a config file from the Github code branch you are deploying from, as valid parameters change over time. The main branch is NOT the current release and often will not work with the GA releases.
    • sample config files can be found in this folder;
    • descriptions of the sample config files and customization guidance can be found here;
    • unsure where to start, use the config.lite-CTNFW-example.json, where CTNFW is for Control Tower w/NFW;
    • These configuration files can be used, as-is, with only minor modification to successfully deploy the sample architectures;
    • On upgrades, compare your deployed configuration file with the latest branch configuration file for any new or changed parameters;
  2. At minimum, you MUST update the AWS account names and email addresses in the sample file:
    • For existing accounts, they must match identically to both the account names and email addresses defined in AWS Organizations;
    • For new accounts, they must reflect the new account name/email you want created;
    • All new AWS accounts require a unique email address which has never before been used to create an AWS account;
    • When updating the budget or SNS notification email addresses within the sample config, a single email address for all is sufficient;
    • Update the IP address in the "alarm-not-ip" variable with your on-premise IP ranges (used for the AWS-SSO-Authentication-From-Unapproved-IP alarm);
    • If deploying the Managed AD, update the dns-domain, netbios-domain, log-group-name, as well as the AD users and groups that will be created;
    • For a test deployment, the remainder of the values can be used as-is;
    • While it is generally supported, we recommend not adding more than 1 or 2 workload accounts to the config file during the initial deployment as it will increase risks of hitting a limit. Once the Accelerator is successfully deployed, add the additional accounts to the config file and rerun the state machine.
    • More information here on the fields in the config file that need to be updated.
  3. A successful deployment of the prescriptive architecture requires VPC access to 9 AWS endpoints, you cannot remove both the perimeter firewalls (all public endpoints) and the 9 required central VPC endpoints from the config file (ec2, ec2messages, ssm, ssmmessages, cloudformation, secretsmanager, kms, logs, monitoring).
  4. When deploying to regions other than ca-central-1, you need to modify your config file as follows (for Canada Central 1, the AMI ids are pre-populated for you):
    1. Update the firewall and firewall manager AMI id's to reflect your home regions regional AMI id's (see 2.3.3, item 13), making sure you select the right version and region per the recommendations.
    2. Validate all the Interface Endpoints defined in your config file are supported in your home region (i.e. Endpoint VPC). Remove unsupported endpoints from the config file, add additional endpoints as available.
    3. If you are installing into a home region which is explicitly named in any of the replacements\addl_regions_x, remove it from the list. If deploying in us-east-1, remove ${GBL_REGION}.
  5. Create an S3 bucket in your Organization Management account your-bucket-name
    • you must supply this bucket name in the CFN parameters and in the config file (global-options\central-bucket)
    • the bucket name must be the same in both spots
    • the bucket must have versioning enabled
    • the bucket must be S3-KMS encrypted using the ASEA-Source-Bucket-Key created above
  6. Place your customized config file(s), named config.json (or config.yaml), in your new bucket
  7. If required, place the firewall configuration and license files in the folder and path defined in the config file
    • For AWS Network Firewall: nfw/nfw-example-policy.json
    • For Fortinet: firewall/firewall-example.txt, firewall/license1.lic and firewall/license2.lic
      • We have made several samples available here: ./reference-artifacts/Third-Party/
      • Both samples comprise an active / active firewall pair. Until recently we only brought up one tunnel per firewall, you now also have an example which brings up both tunnels per firewall
      • If you updated your perimeter VPC subnet names, you must also make these changes in your firewall-example.txt file
      • If you don't have any license files, update the config file with an empty array ("license": []). Do NOT use the following: [""].
    • The basic Checkpoint configuration is stored directly in config.json
  8. Place any defined certificate files in the folder and path defined in the config file
    • i.e. certs/example1-cert.key, certs/example1-cert.crt
    • Sample available here: ./reference-artifacts/Certs-Sample/*
    • Ideally you would generate real certificates using your existing certificate authority
    • Should you wish, instructions are provided to aid in generating your own self-signed certificates (Self signed certificates are NOT secure and simply for demo purposes)
    • Use the examples to demonstrate Accelerator TLS functionality only
  9. Detach ALL SCPs (except FullAWSAccess which remains in place) from all OU's and accounts before proceeding
    • For Control Tower based installs do NOT detach Control Tower SCPs (i.e. aws-guardrails-xxxxxx)
    • Installation will fail if this step is skipped

2.5. Installation

  1. You can find the latest release in the repository here.

    • We only support new installations of v1.3.9 or above (older releases continue to function)

  2. Download the CloudFormation (CFN) template for the release you plan to install (either AcceleratorInstallerXXX.template.json for GitHub or AcceleratorInstallerXXX-CodeCommit.template.json for CodeCommit)

  3. Use the provided CloudFormation template to deploy a new stack in your Management (root) AWS account

    • As previously stated we do not support installation in sub-accounts

  4. Login to your Organization Management account and make sure you are in your desired home region (i.e. ca-central-1) (your desired primary or control region)

  5. Navigate to CloudFormation in the AWS Console and click Create stack with new resources (standard), then

    • Select "Template is ready"
    • For the "Specify template" select "Upload a template file"
    • Select the *.template.json file you downloaded in step 2 above
    • Click Next

  6. Fill out the required parameters - LEAVE THE DEFAULTS UNLESS SPECIFIED BELOW

    • Specify Stack Name STARTING with ASEA- (case sensitive) suggest a suffix of orgname or username
    • Change ConfigS3Bucket to the name of the bucket you created above your-bucket-name
    • Add an Email address to be used for State Machine Status notification
    • The GithubBranch should point to the release you selected
      • if upgrading, change it to point to the desired release
      • the latest stable branch is currently release/v1.5.0, case sensitive
      • click Next

  7. Finish deploying the stack

    • Apply a tag on the stack, Key=Accelerator, Value=ASEA (case sensitive).
    • ENABLE STACK TERMINATION PROTECTION under Stack creation options
    • Click Next, Acknowledge resource creation, and click Create stack
    • The stack typically takes under 5 minutes to deploy.

  8. Once deployed, you should see a CodePipeline project named ASEA-InstallerPipeline in your account. This pipeline connects to Github, pulls the code from the prescribed branch and deploys the Accelerator state machine.

    • if the CloudFormation fails to deploy with an Internal Failure, or, if the pipeline fails connecting to GitHub, then:
      • fix the issue with your GitHub secret created in section 2.3.2, then delete the Installer CloudFormation stack you just deployed, and restart at step 3 of this section.

  9. For new stack deployments, when the stack deployment completes, the Accelerator state machine will automatically execute (in Code Pipeline). When upgrading you must manually Release Change to start the pipeline.

  10. While the pipeline is running:

  11. Once the pipeline completes (~10 mins), the main state machine, named ASEA-MainStateMachine_sm, will start in Step Functions

  12. The state machine time is dependent on the quantity of resources being deployed. On an initial installation of a more complex sample configuration files, it takes approximately 2 hours to execute (depending on the configuration file). Timing for subsequent executions depends entirely on what resources are changed in the configuration file, but often takes as little as 20 minutes.

    • While you can watch the state machine in Step Functions, you will also be notified via email when the State Machine completes (or fails). Successful state machine executions include a list of all accounts which were successfully processed by the Accelerator.

  13. The configuration file will be automatically moved into Code Commit (and deleted from S3). From this point forward, you must update your configuration file in CodeCommit.

  14. You will receive an email from the State Machine SNS topic and the 3 SNS alerting topics. Please confirm all four (4) email subscriptions to enable receipt of state machine status and security alert messages. Until completed, you will not receive any email messages (must be completed within 7-days).

  15. If the state machine fails:

    • Refer to the Troubleshooting Guide for instructions on how to inspect and retrieve the error
    • You can also refer to the FAQ and Known Installation Issues
    • Once the error is resolved, re-run the step function ASEA-MainStateMachine_sm using {"scope": "FULL","mode": "APPLY"} as input

  16. If deploying a prescriptive architecture with 3rd party firewalls, after the perimeter account is created in AWS Organizations, but before the Accelerator reaches Stage 2:

    1. NOTE: If you miss the step, or fail to execute it in time, no need to be concerned, you will simply need to re-run the main state machine (ASEA-MainStateMachine_sm) to deploy the firewall (no SM input parameters required)
    2. Login to the perimeter sub-account (Assume your organization-admin-role)
    3. Activate the 3rd party vendor firewall and firewall manager AMI's in the AWS Marketplace
    • Navigate back to your private marketplace
    • Note: Employees should see the private marketplace, including the custom color specified in prerequisite step 4 above.
    • Select "Discover products" from the side bar, then in the "Refine Results" select "Private Marketplace => Approved Products"
    • Subscribe and Accept the Terms for each product (firewall and firewall manager)
    • When complete, you should see the marketplace products as subscriptions in the Perimeter account:

marketplace

  1. If deploying the prescriptive architecture, once the main state machine (ASEA-MainStateMachine_sm) completes successfully, confirm the status of your perimeter firewall deployment
    • If you have t2.micro ec2 instances running in any account which had the account-warming flag set to true, they will be removed on the next state machine execution;
    • If your perimeter firewalls were defined but not deployed on first run, you will need to rerun the state machine. This happens when:
      1. you were unable to activate the firewall AMI's before stage 2 (step 16)
      2. we were not able to fully activate your account before we were ready to deploy your firewalls. This case can be identified by a running EC2 micro instance in the account, or by looking for the following log entry 'Minimum 15 minutes of account warming required for account'.
      3. In these cases, simply select the ASEA-MainStateMachine_sm in Step Functions and select Start Execution (no SM input parameters required)

2.5.1. Known Installation Issues

Current Issues:

  • When a new installation includes AWS Network Firewall (NFW), we are seeing State Machine failures in Phase 1 in the Perimeter account. Timing issues are causing the first deployment of the underlying CloudFormation stack to fail and rollback, when we automatically retry the stacks deployment, we attempt to recreate the NFW CloudWatch Log groups which were retained, causing a failure. A fix is in the works. Manually delete the two NFW log groups from the perimeter account (/ASEA/Nfw/Central-Firewall/Alert and /ASEA/Nfw/Central-Firewall/Flow) using either the Accelerator Pipeline Role or the Org Admin Role and rerun the state machine with the input of {"scope": "FULL", "mode": "APPLY"}.
  • If dns-resolver-logging is enabled, VPC names containing spaces are not supported at this time as the VPC name is used as part of the log group name and spaces are not supported in log group names. By default in many of the sample config files, the VPC name is auto-generated from the OU name using a variable. In this situation, spaces are also not permitted in OU names (i.e. if any account in the OU has a VPC with resolver logging enabled and the VPC is using the OU as part of its name).
  • On larger deployments we are occassionally seeing state machine failures when Creating Config Recorders. Simply rerun the state machine with the input of {"scope": "FULL", "mode": "APPLY"}.
  • Occasionally CloudFormation fails to return a completion signal. After the credentials eventually fail (1 hr), the state machine fails. Simply rerun the state machine with the input of {"scope": "FULL", "mode": "APPLY"}.
  • Applying new Control Tower Detective guardrails fails in v1.5.0. This is fixed in the next release.

Issues in Older Releases:

  • New installs to releases prior to v1.3.9 are no longer supported.
  • Upgrades to releases prior to v1.3.8 are no longer supported.

2.6. Post-Installation

The Accelerator installation is complete, but several manual steps remain:

  1. Enable and configure AWS SSO in your home region (i.e. ca-central-1)
  • Login to the AWS Console using your Organization Management account
  • Navigate to AWS Single Sign-On, click Enable SSO
  • Set the SSO directory to AD ("Settings" => "Identity Source" => "Identity Source" => click Change, Select Active Directory, and select your domain from the list)
  • Under "Identity Source" section, Click Edit beside "Attribute mappings", then set the email attribute to: ${dir:email} and click Save Changes
  • Configure Multi-factor authentication, we recommend the following minimum settings:
    • Every time they sign in (always-on)
    • Security key and built-in authenticators
    • Authenticator apps
    • Require them to provide a one-time password sent by email to sign in
    • Users can add and manage their own MFA devices
  • Create all the default permission sets and any desired custom permission sets
    • e.g. Select AWS accounts from the side bar, select "Permission sets" tab then Create permission set
      • Use an existing job function policy => Next
      • Select job function policy AdministratorAccess
      • Add Tags, Review and Create
      • repeat for each default permission set and any required custom permission sets
  • For Control Tower based installations, remove the orphaned Permission Sets from each AWS accounts (select the account, expand Permission Sets, click Remove for each)
  • Map MAD groups to permission sets and accounts
    • Select AWS accounts from the side bar and select AWS organization tab
    • Select the accounts you want to map to each MAD group and click Assign users
    • Select your DNS domain e.g. example.local, and search for the group you would like to assign (e.g. aws- for the pre-created groups) and click Search connected directory
    • Select the desired group aws-log-archive-View
    • Select the permission set you would like to assign to the MAD group to (e.g. ViewOnlyAccess)
    • Click Finish (Note: if it fails during provisioning, simply select the failed accounts and click on "Retry changes")
  • AWS SSO should be the primary mechanism for all access to all AWS accounts in your Organization, to determine or update the login page for your organization:
    • Click on Dashboard within the AWS SSO console and note the User portal URL
    • Share this url with all your users
    • NOTE: the url prefix can only be changed ONCE (ever) using the settings tab, so update with caution
  • Any pre-created AWS MAD users passwords are available in secrets manager in the AWS management account. To reset these passwords login to the Operations account through AWS SSO, then:
    • Navigate to "Directory Service" , select the directory => Actions => Reset user password
    • Users can change their passwords from any MAD domain connected instance

  1. Configure the new alb-forwarding feature (added in v1.5.0)

    • AWS ALB's are published using DNS names which resolve to backing IP's which could silently change at any time due to a scaling event, maintenance, or a hardware failure. While published as a DNS name, ALB's can only target IP addresses. This presents a challenge as we need the ALB's in the perimeter account to target ALB's in the various back-end workload accounts;
    • Alb-forwarding solves this problem by executing a small snippet of code every 60 seconds which updates managed alb listeners with any IP changes, ensuring any managed flows do not go offline. This removes the requirement to leverage a 3rd party appliance to perform NAT to a DNS name;
    • Alb-forwarding is enabled on the perimeter vpc in the perimeter account in many of the sample configs and can be used with Accelerator created or customer created ALB's. This setting causes a new DynamoDB table named ASEA-Alb-Ip-Forwarding-vpc-* to be created in the perimeter account. Additionally the parameter store in the perimeter account contains the details of all Accelerator deployed alb's across the environment for easy reference;
    • Stepts to configure:
      • First you need to manually create a listener on the front-end alb (without a target group), multiple listeners are supported;
      • Next, for each application that needs to be published, a record needs to be added to the DynamoDB table, see sample below;
      • Records can be added to the table for any alb in the account running the alb-forwarding tool. Records can be added at any time. DDB change logs will trigger the initial creation of the appropriate target group(s) and IP addresses will be verified and updated every 60 seconds therafter.
Sample DynamoDB JSON to add an entry to the table: 
{
    "id": "App1",
    "targetAlbDnsName": "internal-Core-mydevacct1-alb-123456789.ca-central-1.elb.amazonaws.com",
    "targetGroupDestinationPort": 443,
    "targetGroupProtocol": "HTTPS",
    "vpcId": "vpc-0a6f44a80514daaaf",
    "rule": {
      "sourceListenerArn": "arn:aws:elasticloadbalancing:ca-central-1:123456789012:listener/app/Public-DevTest-perimeter-alb/b1b12e7a0c412bf3/ef9b022a4fdd8bdf",
      "condition": {
        "paths": ["/img/*", "/myApp2"],
        "hosts": ["aws.amazon.com"],
        "priority": 30
      }
	}
}
  • where id is any unique text, targetAlbDnsName is the DNS address for the backend alb for this application (found in parameter store), vpcId is the vpc id containing the front-end alb (in this account), sourceListenerArn is the arn of the listener of the front-end alb, paths and hosts are both optional, but one of the two must be supplied. Finally, priority must be unique and is used to order the listener rules. Priorities should be spaced at least 40 apart to allow for easy insertion of new applications and forwarder rules.
  • the provided targetAlbDnsName must resolve to addresses within a supported IP address space.

  1. On a per role basis, you need to enable the CWL Account Selector in the Security and the Operations accounts, in each account:

    • Go to CloudWatch, Settings, Under Cross-account cross-region select Configure, Under View cross-account cross-region select Edit, choose AWS Organization account selector, click Save changes

  2. Configure central Ingress/Egress firewalls, if deployed

    • Layer 3/4 appliance based inspection is an optional feature

    General

    • If deployed, login to any 3rd party firewalls and firewall manager appliances and update any default passwords;
    • Tighten security groups on the 3rd party firewall instances (using the Accelerator configuration file), further limiting access to firewall management interfaces to a set of designated and controlled CIDR ranges;
    • Update the firewall configuration per your organizations security requirements and best practices;
    • Diagrams reflecting perimeter traffic flows when NFW and/or GWLB are used can be found here on slides 6 through 9.

    AWS Network Firewall

    • The AWS Network Firewall policies and rules deployed by the Accelerator, can only be updated using the Accelerator. Customers wishing to manage the AWS Network Firewall from the console GUI, must create a new policy with new rules created through the console and then manually associate this new policy to the Accelerator deployed Network Firewall. Customers can choose either option, but they cannot be mixed to ensures that Accelerator updates do not overwrite console based updates.

    Fortinet

    • Manually update firewall configuration to forward all logs to the Accelerator deployed NLB addresses fronting the rsyslog cluster
      • login to each firewall, select Log Settings, check Send logs to syslog, put the NLB FQDN in the IP Address/FQDN field (stored in parameter store of perimeter account)
    • Manually update the firewall configuration to connect perimeter ALB high port flows through to internal account ALB's
      • Note: while this option is still available, a new alb-forwarding mechanism is available in v1.5.0 (see section 2 above) which simplifies and eliminates this more complicated "NAT to DNS name" option;
      • login to each firewall, switch to FG-traffic vdom, select Policies & Objects, select Addresses, Expand Addresses
        • Set Prod1-ALB-FQDN to point to a reliable sub-account ALB FQDN, this is used for full-path health checks on all ALB's
        • Set additional DevX-ALB-FQDN, TestX-ALB-FQDN and ProdX-ALB-FQDN to point to workload account ALB FQDNs
          • Two of each type of ALB FQDN records have been created, when you need more, you need to create BOTH an additional FQDN and a new VIP, per ALB
          • Each new VIP will use a new high port (i.e. 7007, 7008, etc.), all of which map back to port 443
        • Detailed steps can be read here.

    Checkpoint

    • Download and install Checkpoint SmartConsole client on your PC (only available for Windows, if you have a Mac, you can use Workspaces to install the client)
    • Retrieve the firewall and firewall manager SSL certificate from secrets manager in the perimeter account and save to a pem file (convert to ppk on Windows)
      • Login to the AWS Console for the Perimeter account
      • Navigate to Secrets Manager and select Secrets
      • Look for Secret name ASEA/keypair/ASEA-Appliances and click on it
      • Go to the Secret value section and click on Retrieve secret value
      • Make sure you have the Plaintext tab selected and copy the content to a .pem file using your preferred text editor. (the file will need to be converted to ppk format on Windows)
    • Wait approx. 25 min. after the managers "Launch time" and then SSH into the Firewall Manager using the SSL certificate (.pem file retrieved above) and login user admin
      • The firewall manager EC2 instance name is ASEA-Checkpoint-FirewallMgr
      • Once you SSH successfully, execute the following commands:
        • set user admin password
        • set expert-password
        • set user admin shell /bin/bash
        • save config
    • The following commands are useful for troubleshooting (in expert mode):
      • autoprov_cfg -v (check cme at Take 155 or greater)
      • autoprov_cfg show all (check cme configuration)
      • cat /var/log/aws-user-data.log (validate bootstrap, file should end with "Publish operation" succeeded (100%))
      • tail -f /var/log/CPcme/cme.log (watch to ensure it finds the instances, establishes SIC and adds the nodes)
    • Login to SmartConsole, and update the firewall policy per your organizations security requirements
      • An outbound rule allowing http and https should exist
      • From the RDGW host in Operations, test to see if outbound web browsing is enabled
    • NOTES:
      • No best practice or security configuration has been configured on the Checkpoint firewalls. These firewalls have been configured to work with GWLB, but otherwise have the default/basic Checkpoint out-of-box configuration installed
      • Do NOT reboot the Checkpoint appliances until bootstrap is complete (~25 minutes for the manager), or you will be required to redeploy the instance

  3. Recover root passwords for all sub-accounts and apply strong passwords

    • Process documented here

  4. Enable MFA for all IAM users and all root account users, recommendations:

    • Yubikeys provide the strongest form of MFA protection and are strongly encouraged for all account root users and all IAM users in the Organization Management (root) account
    • the Organization Management (root) account requires a dedicated Yubikey (if access is required to a sub-account root user, we do not want to expose the Organization Management accounts Yubikey)
    • every ~50 sub-accounts requires a dedicated Yubikey (minimize the required number of Yubikeys and the scope of impact should a Yubikey be lost or compromised)
    • each IAM breakglass user requires a dedicated Yubikey, as do any additional IAM users in the Organization Management (root) account. While some CSPs do not recommend MFA on the breakglass users, it is strongly encouraged in AWS
    • all other AWS users (AWS SSO, IAM in sub-accounts, etc.) can leverage virtual MFA devices (like Google Authenticator on a mobile device)

  5. Customers are responsible for the ongoing management and rotation of all passwords on a regular basis per their organizational password policy. This includes the passwords of all IAM users, MAD users, firewall users, or other users, whether deployed by the Accelerator or not. We do NOT automatically rotate any passwords, but strongly encourage customers do so, on a regular basis.

  6. During the installation we request required limit increases, resources dependent on these limits will not be deployed

    1. Limit increase requests are controlled through the Accelerator configuration file "limits":{} setting
    2. The sample configuration file requests increases to your EIP count in the perimeter account and to the VPC count and Interface Endpoint count in the shared-network account
    3. You should receive emails from support confirming the limit increases
    4. On the next state machine execution, resources blocked by limits should be deployed (i.e. additional VPC's and Endpoints)
    5. If more than 2 days elapses without the limits being increased, on the next state machine execution, they will be re-requested

  7. Note: After a successful install the Control Tower Organizational units dashboard will indicate 2 of 3 in the Accounts enrolled column for the Security OU, as it does not enable enrollment of the management account in guardrails. The Accelerator compliments Control Tower and enables guardrails in the management account which is important to high compliance customers.

3. Upgrades

3.1. Considerations

  • Due to some breaking dependency issues, customers can only upgrade to v1.3.8 or above (older releases continue to function, but cannot be installed).
  • While an upgrade path is planned, customers with a standalone Accelerator installation can upgrade to v1.5.0 but need to continue with a standalone installation until the Control Tower upgrade option becomes available.
  • Always compare your configuration file with the config file from the release you are upgrading to in order to validate new or changed parameters or changes in parameter types / formats.
    • do NOT update to the latest firewall AMI - see the the last bullet in section 5.1. Accelerator Design Constraints / Decisions
    • do NOT update the organization-admin-role - see bullet 2 in section 2.2.7. Other
    • do NOT update account-keys (i.e. existing installations cannot change the internal values to management from master)
    • do NOT make changes outside those required for the upgrade (those stated in the release notes or found through the comparison with the sample config file(s)). Customers wishing to change existing Accelerator configuration should either do so before their upgrade, ensuring a clean/successful state machine execution, or after a successful upgrade.
  • The Accelerator name and prefix CANNOT be changed after the initial installation
  • Customers which customized any of the Accelerator provided default configuration files (SCPs, rsyslog config, ssm-documents, iam-policies, etc.) must manually merge the latest Accelerator provided updates with deployed customizations:
    • it is important customers assess the new defaults and integrate them into their custom configuration, or Accelerator functionality could break or Accelerator deployed features may be unprotected from modification
    • if customers don't take action, we continue to utilize the deployed customized files (without the latest updates)
  • The below release specific considerations need to be cumulatively applied (an upgrade from v1.2.3 to v1.2.5 requires you to follow both v1.2.4 and v1.2.5 considerations)

Release Specific Upgrade Considerations:

  • Upgrades to v1.5.0:
    • Due to the size and complexity of this upgrade, we require all customers to upgrade to v1.3.8 or above before beginning this upgrade
    • While v1.5.0 supports Control Tower for NEW installs, existing Accelerator customers CANNOT add Control Tower to their existing installations at this time (planned enhancement for 22H1)
      • Attempts to install Control Tower on top of the Accelerator will corrupt your environment (both Control Tower and the Accelerator need minor enhancements to enable)
    • The v1.5.0 custom upgrade guide can be found here
  • Upgrades to v1.3.9 and above from v1.3.8-b and below:
    • All interface endpoints containing a period must be removed from the config.json file either before or during the upgrade process
      • i.e. ecr.dkr, ecr.api, transfer.server, sagemaker.api, sagemaker.runtime in the full config.json example
      • If you remove them on a pre-upgrade State Machine execution, you can put them back during the upgrade, if you remove them during the upgrade, you can put them back post upgrade.
  • Upgrades to v1.3.3 and above from v1.3.2 and below:
    • Requires mandatory config file schema changes as documented in the release notes.
      • These updates cause the config file change validation to fail and require running the state machine with the following input to override the validation checks on impacted fields: {"scope": "FULL", "mode": "APPLY", "configOverrides": {"ov-ou-vpc": true, "ov-ou-subnet": true, "ov-acct-vpc": true }}
      • Tightens VPC interface endpoint security group permissions and enables customization. If you use VPC interface endpoints that requires ports/protocols other than TCP/443 (such as email-smtp), you must customize your config file as described here
  • Upgrades to v1.3.0 and above from v1.2.6 and below:
    • Please note MAJOR changes to state machine behavior, as documented here.
  • Upgrades to v1.2.6 and above from v1.2.5 and below - Ensure you apply the config file changes described in the release notes:
    • Cut-paste the new "replacements": {}, section at the top of the example config file into your config file, as-is
      • Enables customers to leverage the repo provided SCP's without customization, simplifying upgrades, while allowing SCP region customization
      • the cloud-cidrX/cloud-maskX variables are examples of customer provided values that can be used to consistently auto-replace values throughout config files, these 4 specific variables are all required for the firewalls to successfully deploy
    • The new ${variable} are auto-replaced across your config files, SCP's and firewall config files.
      • as the variables should resolve to their existing values, you can leave your config file using hardcoded region and Accelerator prefix naming, or you can update them to make subsequent file comparisons easier for future upgrades. These are most useful for new installations in non ca-central-1 regions
    • Some repo provide filenames have changed, where they are referenced within the config file, you must update them to their new filenames
    • We do not delete/cleanup old/unused SCP's, in case they were also used by customers for unmanaged OUs or sub-ou's. After the upgrade, you should manually delete any old/extra SCP's which are no longer required
  • Upgrades to v1.2.5 and above from v1.2.4 and below requires the manual removal of the PBMMAccel-PipelineRole StackSet before beginning your upgrade (we have eliminated all use of StackSets in this release)
    • In the root AWS account, go to: CloudFormation, StackSets
    • Find: ASEA-PipelineRole, and Select the: Stack Instances tab
    • Document all the account numbers, comma separated i.e. 123456789012, 234567890123, 345678901234
    • Select: Actions, Delete Stacks from StackSets
    • Paste the above account numbers (comma separated) in the Account numbers box
    • Select the Accelerator installation/home region from the Specify Regions Box (should be the only region in the dropdown)
    • Change: Concurrency to: 8, Next, Submit
    • Wait for operation to complete (refresh the browser several times)
    • Select Actions, Delete StackSet, click Delete StackSet
    • Wait for the operation to complete
  • Upgrades to v1.2.4 and above from v1.2.3 and below - Ensure you apply the config file changes described in the release notes:
    • failure to set "central-endpoint": true directly on the endpoint VPC (instead of in global-options), will result in the removal of your VPC endpoints
    • failure to move your zone definitions to the endpoint VPC, will result in the removal of you Public and Private hosted zones

3.2. Summary of Upgrade Steps (all versions)

  1. Login to your Organization Management (root) AWS account with administrative privileges
  2. Either: a) Ensure a valid Github token is stored in secrets manager (section 2.3.2) b) Ensure the latest release is in a valid branch of CodeCommit in the Organization Management account
  3. Review and implement any relevant tasks noted in the upgrade considerations in section 3.1
  4. Update the config file in CodeCommit with new parameters and updated parameter types based on the version you are upgrading to (this is important as features are iterating rapidly)
    • An automated script is available to help convert config files to the new v1.5.0 format
    • Compare your running config file with the sample config file from the latest release
    • Review the Config file changes section of the release notes for all Accelerator versions since your current deployed release
  5. If you customized any of the other Accelerator default config files by overriding them in your S3 input bucket, merge the latest defaults with your customizations before beginning your upgrade
  6. Download the latest installer template (AcceleratorInstallerXYZ.template.json or AcceleratorInstallerXXX-CodeCommit.template.json) from the Assets section of the latest release
  7. Do NOT accidentally select the ASEA-InitialSetup CloudFormation stack below
  8. If you are replacing your GitHub Token:
    • Take note of the AcceleratorName, AcceleratorPrefix, ConfigS3Bucket and NotificationEmail values from the Parameters tab of your deployed Installer CloudFormation stack (ASEA-what-you-provided)
    • Delete the Installer CloudFormation stack (ASEA-what-you-provided)
    • Redeploy the Installer CloudFormation stack using the template downloaded in step 6, providing the values you just documented (changes to AcceleratorName or AcceleratorPrefix are not supported)
    • The pipeline will automatically run and trigger the upgraded state machine
  9. If you are using a pre-existing GitHub token, or installing from CodeCommit:
  • Update the Installer CloudFormation stack using the template downloaded in step 5, updating the GithubBranch to the latest release (eg. release/v1.5.0)
    • Go to AWS CloudFormation and select the stack: ASEA-what-you-provided
    • Select Update, select Replace current template, Select Upload a template file
    • Select Choose File and select the template you downloaded in step 6 (AcceleratorInstallerXYZ.template.json or AcceleratorInstallerXXX-CodeCommit.template.json)
    • Select Next, Update GithubBranch parameter to release/vX.Y.Z where X.Y.Z represents the latest release
    • Click Next, Next, I acknowledge, Update
    • Wait for the CloudFormation stack to update (Update_Complete status) (Requires manual refresh)
  • Go To Code Pipeline and Release the ASEA-InstallerPipeline

4. Existing Organizations / Accounts

4.1. Considerations: Importing existing AWS Accounts / Deploying Into Existing AWS Organizations

  • The Accelerator can be installed into existing AWS Organizations
    • our early adopters have all successfully deployed into existing organizations
  • Existing AWS accounts can also be imported into an Accelerator managed Organization
  • Caveats:
    • Per AWS Best Practices, the Accelerator deletes the default VPC's in all AWS accounts, worldwide. The inability to delete default VPC's in pre-existing accounts will fail the installation/account import process. Ensure default VPC's can or are deleted before importing existing accounts. On failure, either rectify the situation, or remove the account from Accelerator management and rerun the state machine
    • The Accelerator will NOT alter existing (legacy) constructs (e.g. VPC's, EBS volumes, etc.). For imported and pre-existing accounts, objects the Accelerator prevents from being created using preventative guardrails will continue to exist and not conform to the prescriptive security guidance
      • Existing workloads should be migrated to Accelerator managed VPC's and legacy VPC's deleted to gain the full governance benefits of the Accelerator (centralized flow logging, centralized ingress/egress, no IGW's, Session Manager access, existing non-encrypted EBS volumes, etc.)
    • Existing AWS services will be reconfigured as defined in the Accelerator configuration file (overwriting existing settings)
    • We do NOT support any workloads running or users operating in the Organization Management (root) AWS account. The Organization Management (root) AWS account MUST be tightly controlled
    • Importing existing workload accounts is fully supported, we do NOT support, recommend and strongly discourage importing mandatory accounts, unless they were clean/empty accounts. Mandatory accounts are critical to ensuring governance across the entire solution
    • We've tried to ensure all customer deployments are smooth. Given the breadth and depth of the AWS service offerings and the flexibility in the available deployment options, there may be scenarios that cause deployments into existing Organizations to initially fail. In these situations, simply rectify the conflict and re-run the state machine.
    • If the Firewall Manager administrative account is already set for your organization, it needs to be unset before starting a deployment.

4.2. Process to import existing AWS accounts into an Accelerator managed Organization

  • Newly invited AWS accounts in an Organization will land in the root ou
  • Unlike newly created AWS accounts which immediately have a Deny-All SCP applied, imported accounts are not locked down as we do not want to break existing workloads (these account are already running without Accelerator guardrails)
  • In AWS Organizations, select ALL the newly invited AWS accounts and move them all (preferably at once) to the correct destination OU (assuming the same OU for all accounts)
    • In case you need to move accounts to multiple OU's we have added a 2 minute delay before triggering the State Machine
    • Any accounts moved after the 2 minute window will NOT be properly ingested, and will need to be ingested on a subsequent State Machine Execution
  • This will first trigger an automated update to the config file and then trigger the state machine after a 2 minute delay, automatically importing the moved accounts into the Accelerator per the destination OU configuration
  • As previously documented, accounts CANNOT be moved between OU's to maintain compliance, so select the proper top-level OU with care
  • If you need to customize each of the accounts configurations, you can manually update the configuration file either before or after you move the account to the correct ou
    • if before, you also need to include the standard 4 account config file parameters, if after, you can simply add your new custom parameters to the account entry the Accelerator creates
    • if you add your imported accounts to the config file, moving the first account to the correct ou will trigger the state machine after a 2 minutes delay. If you don't move all accounts to their correct ou's within 2 minutes, your state machine will fail. Simply finish moving all accounts to their correct OU's and then rerun the state machine.
  • If additional accounts are moved into OUs while the state machine is executing, they will not trigger another state machine execution, those accounts will only be ingested on the next execution of the state machine
    • customers can either manually initiate the state machine once the current execution completes, or, the currently running state machine can be stopped and restarted to capture all changes at once
    • Are you unsure if an account had its guardrails applied? The message sent to the state machine Status SNS topic (and corresponding email address) on a successful state machine execution provides a list of all successfully processed accounts.
  • The state machine is both highly parallel and highly resilient, stopping the state machine should not have any negative impact. Importing 1 or 10 accounts generally takes about the same amount of time for the Accelerator to process, so it may be worth stopping the current execution and rerunning to capture all changes in a single execution.
  • We have added a 2 min delay before triggering the state machine, allowing customers to make multiple changes within a short timeframe and have them all captured automatically in the same state machine execution.

4.3. Deploying the Accelerator into an existing Organization

  • As stated above, if the ALZ was previously deployed into the Organization, please work with your AWS account team to find the best mechanism to uninstall the ALZ solution
  • Ensure all existing sub-accounts have the role name defined in organization-admin-role installed and set to trust the Organization Management (root) AWS Organization account
    • prior to v1.2.5, this role must be named: AWSCloudFormationStackSetExecutionRole
    • if using the default role (AWSCloudFormationStackSetExecutionRole) we have provided a CloudFormation stack which can be executed in each sub-account to simplify this process
  • As stated above, we recommend starting with new AWS accounts for the mandatory functions (shared-network, perimeter, security, log-archive accounts).
  • To better ensure a clean initial deployment, we also recommend the installation be completed while ignoring most of your existing AWS sub-accounts, importing them post installation:
    • create a new OU (i.e. Imported-Accounts), placing most of the existing accounts into this OU temporarily, and adding this OU name to the global-options\ignored-ous config parameter;
    • any remaining accounts must be in the correct ou, per the Accelerator config file;
    • install the Accelerator;
    • import the skipped accounts into the Accelerator using the above import process, paying attention to the below notes
  • NOTES:
    • Do NOT move any accounts from any ignored-ous to the root ou, they will immediately be quarantined with a Deny-All SCP, they need to be moved directly to their destination ou
    • As stated above, when importing accounts, there may be situations we are not able to fully handle
      • If doing a mass import, we suggest you take a quick look and if the solution is not immediately obvious, move the account which caused the failure back to ignored-ous and continue importing the remainder of your accounts. Once you have the majority imported, you can circle back and import outstanding problem accounts with the ability to focus on each individual issue
      • The challenge could be as simple as someone has instances running in a default VPC, which may require some cleanup effort before we can import (coming soon, you will be able to exclude single account/region combinations from default VPC deletion to gain the benefits of the rest of the guardrails while you migrate workloads out of the default VPC)

5. Notes

5.1. Accelerator Design Constraints / Decisions

  • The Organization Management (root) account does NOT have any preventative controls to protect the integrity of the Accelerator codebase, deployed objects or guardrails. Do not delete, modify, or change anything in the Organization Management (root) account unless you are certain as to what you are doing. More specifically, do NOT delete, or change any buckets in the Organization Management (root) account.
  • While generally protected, do not delete/update/change s3 buckets with cdk-asea-, or asea- in any sub-accounts.
  • ALB automated deployments only supports Forward and not redirect rules.
  • The Accelerator deploys SNS topics to send email alerts and notifications. Given email is not a secure transport mechanism, we have chosen not to enable SNS encryption on these topics at this time.
  • AWS generally discourages cross-account KMS key usage. As the Accelerator centralizes logs across an entire organization as a security best practice, this is an exception/example of a unique situation where cross-account KMS key access is required.
  • The Accelerator aggregates all logs in the log-archive account using Kinesis Data and Kinesis Firehose as aggregation tools where the logs could persist for up to 24 hours. These logs are encrypted with Customer Managed KMS keys once stored in S3 (ELB logs only support AES256). These logs are also encrypted in transit using TLS encryption. At this time, we have not enabled Kinesis at-rest encryption, we will reconsider this decision based on customer feedback.
  • AWS Config Aggregator is deployed in the Organization Management (root) account as enablement through Organizations is simpler to implement. AWS Organizations only supported deploying the Aggregator in the Organization Management (root) account and not in a designated administrative account when we implemented this feature. We have a backlog item to update the code to move the Aggregator to the security account.
  • An Organization CloudTrail is deployed, which is created in the primary region in the Organization Management (root) AWS account. All AWS account CloudTrails are centralized into this single CloudWatch Log Group. Starting in v1.1.9 this is where we deploy the CloudWatch Alarms which trigger for ALL accounts in the organization. Security Hub will erroneously report that the only account and/or region that is compliant with certain rules is the primary region of the Organization Management (root) account. We are working with the Security Hub team to rectify this situation in future Security Hub/Accelerator releases (resolved in Accelerator v1.5.0).
  • Only 1 auto-deployed MAD in any mandatory-account is supported today.
  • VPC Endpoints have no Name tags applied as CloudFormation does not currently support tagging VPC Endpoints.
  • If the Organization Management (root) account coincidentally already has an ADC with the same domain name, we do not create/deploy a new ADC. You must manually create a new ADC (it won't cause issues).
  • 3rd party firewall updates are to be performed using the firewall OS based update capabilities. To update the AMI using the Accelerator, you must first remove the firewalls and then redeploy them (as the EIP's will block a parallel deployment), or deploy a second parallel FW cluster and de-provision the first cluster when ready.
  • When adding more than 100 accounts to an OU which uses shared VPC's, you must first increase the Quota Participant accounts per VPC in the shared VPC owner account (i.e. shared-network). Trapping this quota before the SM fails has been added to the backlog.
  • The default limit for Directory Sharing is 125 accounts for an Enterprise Managed Active Directory (MAD), a quota increase needs to be manually requested through support from the account containing the MAD before this limit is reached. Standard MAD has a sharing limit of 5 accounts (and only supports a small quota increase). The MAD sharing limit is not available in the Service Quota's tools.

...Return to Accelerator Table of Contents