Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor AWS account setup #137

Merged
merged 10 commits into from
Jan 7, 2025
33 changes: 33 additions & 0 deletions .github/workflows/tofu-checks.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: OpenTofu checks
on:
push:

jobs:
tofu_checks:
name: OpenTofu checks
runs-on: ubuntu-24.04
steps:
- uses: opentofu/setup-opentofu@592200bd4b9bbf4772ace78f887668b1aee8f716 # v1.0.5
with:
tofu_version: 1.8.7

- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

- name: Checks
run: |
set -eu

for d in admin-role capa-controller-role; do (
cd "${d}"

echo "Initializing Tofu for ${d}"
tofu init

echo "Validating Tofu code for ${d}"
tofu validate -no-color

echo "Formatting Tofu code for ${d}"
tofu fmt
echo "Checking differences in Tofu code formatting for ${d}"
git diff --exit-code || { >&2 echo "Tofu code isn't formatted, please run 'tofu fmt'"; exit 1; }
); done
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Changed

- Reduce setup options to only OpenTofu / Terraform

## [4.3.1] - 2024-12-18

### Changed
Expand Down
127 changes: 23 additions & 104 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,129 +1,48 @@
# giantswarm-aws-account-prerequisites

This repo contains Cloud Formation templates and Terraform modules to prepare AWS accounts to run Giant Swarm clusters.
This repository contains OpenTofu / Terraform configuration to prepare AWS accounts for running Giant Swarm clusters based on Cluster API Provider for AWS (CAPA).

# Cluster API
Note that this repo is intended for Giant Swarm customers to setup the initial admin role and to provide some transparency on the different IAM policies that will be used by CAPA and other components. But it's not to be used by Giant Swarm staff to manage AWS IAM resources directly, head to <https://github.com/giantswarm/aws-account-setup/> for that.

## Before starting

Make sure to adjust AWS account limits according to [these docs](https://docs.giantswarm.io/getting-started/cloud-provider-accounts/cluster-api/aws/#limits).
Make sure to adjust AWS account limits according to [these docs](https://docs.giantswarm.io/getting-started/prepare-your-provider-infrastructure/aws/#quotas). Then please create the admin role for Giant Swarm staff access, as shown below.

For Cluster API take a look at these two modules in this repository:
## admin-role

1. [admin-role](./admin-role) which creates a role and a policy for our staff to be able to operate the infrastructure created by our automation in case of failures.
2. [capa-controller-role](./capa-controller-role) which creates the role and policies that the controllers assume to create and manage the kubernetes clusters.
In all AWS accounts where you plan to run a management cluster and workload clusters, Giant Swarm staff need to have access in order to manage, operate and troubleshoot the infrastructure.

## 1. admin-role

For all AWS accounts part of the platform Giant Swarm staff need to have access in order to debug and
manage and operate the infrastructure. To do so, please use one of the following methods to create the necessary role and policy in your AWS account.

### AWS CloudFormation template

You can execute the CloudFormation template directly by clicking [the admin role stack template](https://eu-central-1.console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/quickcreate?templateURL=https://cf-templates-giantswarm.s3.eu-central-1.amazonaws.com/admin-role/cloud-formation-template.yaml&stackName=GiantSwarmAdminRoleBootstrap&&param_AdminRoleName=GiantSwarmAdmin) or uploading the [template file](./admin-role/cloud-formation-template.yaml) when creating a new stack in the AWS console.

You will be asked for the following parameters:

- `AdminRoleName`: The name of the role that will be created. Default is `GiantSwarmAdmin`. You dont need to change this unless you have a specific requirement.

Review the changes and click `Create stack`. In case of any error, please check the `Events` tab in the CloudFormation console and report the error to the Giant Swarm staff.

### Terraform

#### Requirements

- `terraform` installed
- working AWS credentials set to the desired target account
- AWS region has to be set either via aws profile or via env `AWS_REGION`

### Adjust variables

- `admin_role_name` - can be adjusted to be more strict and specify role name. You dent need to change this unless you have a specific requirement.

### Execution
Therefore, please run OpenTofu or Terraform using the configuration provided in the `admin-role` directory, using AWS credentials for the account where the role needs to be created.

```console
export AWS_PROFILE=example
tofu init
tofu apply # review the proposed changes before approving
```
terraform init
terraform apply -var="admin_role_name=GiantSwarmAdmin
```

The created role ARN needs to be supplied to Giant Swarm.

## 2. capa-controller-role

In the AWS account where you plan to run the management cluster, you need to create a role that the Cluster API controllers will assume to create and manage workload clusters and all infrastructure resources.

### AWS CloudFormation template

You can execute directly the CloudFormation template by clicking the [capa controller role stack template](https://eu-central-1.console.aws.amazon.com/cloudformation/home?region=eu-central-1#/stacks/quickcreate?templateURL=https://cf-templates-giantswarm.s3.eu-central-1.amazonaws.com/capa-controller-role/cloud-formation-template.yaml&stackName=CAPAControllerRoleBootstrap&param_InstallationName=CHANGE_THIS_FOR_THE_INSTALLATION_NAME&param_ManagementClusterOidcProviderDomain=MANAGEMENT_CLUSTER_OIDC_PROVIDER_DOMAIN) or uploading the [template file](./capa-controller-role/cloud-formation-template.yaml) when creating a new stack in the AWS console.

You will be asked for the following parameters:

- `InstallationName`: the name of the installation which you have agreed with Giant Swarm upfront.
- `ManagementClusterOidcProviderDomain`: the domain name used by the MC OIDC provider. Normally `irsa.<cluster-base-domain>`.
- `ByoVpc` (optional - defaults to `false`): if `true`, the CAPA role will be created without the permissions needed to manage VPCs
**Once the admin role is created, Giant Swarm staff will take over the maintenance of it and the CAPA controller roles for all the MCs that operate under that account, so there is no further action needed by customers.**

Review the changes and click `Create stack`. In case of any error, please check the `Events` tab in the CloudFormation console and report the error to the Giant Swarm staff.
## capa-controller-role

### Terraform
The Cluster API Provider AWS (CAPA) controller requires an IAM role to assume in order to create and manage clusters and all infrastructure resources in a specific AWS account. As mentioned above, the lifecycle of this role is normally managed by Giant Swarm once the admin role is provisioned.

#### Requirements

- `terraform` installed
- working AWS credentials set to the desired target account
- AWS region has to be set either via aws profile or via env `AWS_REGION`
But in case that for some reason the CAPA controller role needs to be managed individually by the customer, the OpenTofu / Terraform configuration in the `capa-controller-role` directory can be used, using AWS credentials for the account where the role needs to be created.

### Adjust variables

Note that for this stack there are some additional variables that you need to provide:

- `installation_name`: the name of the installation which you have agreed with Giant Swarm upfront.
- `management_cluster_oidc_provider_domain`: the domain name used by the MC OIDC provider. Normally `irsa.<cluster-base-domain>`.
- `byovpc` (optional - defaults to `false`): if `true`, the CAPA role will be created without the permissions needed to manage VPCs

### Execution

```
terraform init
terraform apply -var="installation_name=test" -var="management_cluster_oidc_provider_domain=irsa.test.gaws.gigantic.io"
```

### Import existing resources

To update the policies of an existing role, you can run Terraform with the extra variable `import_existing=true` to import the resources into the state:

```
terraform init
terraform apply -var="installation_name=test" -var="management_cluster_oidc_provider_domain=irsa.test.gaws.gigantic.io" -var="import_existing=true"
```

## AWS cli

### Requirements

- `awscli` installed
- `jq` installed
- working AWS credentials set to the desired target account
- located on the `capa-controller-role` directory of this git repo
- user `${INSTALLATION}-capa-controller` created in GiantSwarm root account `084190472784`

### Setup

```
export INSTALLATION_NAME=test
export MANAGEMENT_CLUSTER_OIDC_PROVIDER_DOMAIN=irsa.test.gaws.gigantic.io
# Optional: only set to true if this installation is going to be used exclusively to create WCs on existing VPCs and subnets
# export BYOVPC=true
# Optional: only set this to aws-cn if the installation is in China
# export AWS_PARTITION=aws-cn
chmod +x setup.sh
./setup.sh
```

__warning__: You may need to modify the `trusted-entities.json` to use `aws-cn` in the `Principal` field when using the China region.

### Cleanup

```
export INSTALLATION_NAME=test
chmod +x cleanup.sh
./cleanup.sh
```console
export AWS_PROFILE=example
export TF_VAR_installation_name=foo
export TF_VAR_management_cluster_oidc_provider_domain=irsa.foo.bar.com
export TF_VAR_byovpc=false
tofu init
tofu apply # review the proposed changes before approving
```
108 changes: 108 additions & 0 deletions admin-role/admin-role.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
variable "gs_user_account" {
type = string
description = "Account of Giant Swarm IAM users (`084190472784`, except for China)"
default = "084190472784"

validation {
condition = can(regex("^[0-9]{12}$", var.gs_user_account))
error_message = "AWS account ID must consist of exactly 12 digits"
}
}

data "aws_partition" "current" {}

data "aws_iam_policy_document" "giantswarm_admin" {
statement {
effect = "Allow"
resources = ["*"]

actions = [
"acm:*",
"autoscaling:*",
"cloudformation:*",
"cloudfront:*",
"cloudwatch:*",
"dynamodb:*",
"ec2:*",
"ecr:*",
"elasticfilesystem:*",
"elasticloadbalancing:*",
"events:*",
"ram:*",
"iam:AddRoleToInstanceProfile",
"iam:AttachRolePolicy",
"iam:CreateAccessKey",
"iam:CreateInstanceProfile",
"iam:CreatePolicy",
"iam:CreatePolicyVersion",
"iam:CreateRole",
"iam:CreateServiceLinkedRole",
"iam:DeleteAccessKey",
"iam:DeleteInstanceProfile",
"iam:DeletePolicy",
"iam:DeletePolicyVersion",
"iam:DeleteRole",
"iam:DeleteRolePolicy",
"iam:DeleteServiceLinkedRole",
"iam:DetachRolePolicy",
"iam:GenerateServiceLastAccessedDetails",
"iam:Get*",
"iam:List*",
"iam:PassRole",
"iam:PutRolePolicy",
"iam:RemoveRoleFromInstanceProfile",
"iam:TagPolicy",
"iam:TagRole",
"iam:UpdateAccessKey",
"iam:UpdateAssumeRolePolicy",
"iam:UpdateRoleDescription",
"kms:*",
"logs:*",
"route53:*",
"route53domains:*",
"route53resolver:*",
"s3:*",
"sts:AssumeRole",
"sts:DecodeAuthorizationMessage",
"sts:GetFederationToken",
"support:*",
"trustedadvisor:*",
"sqs:*",
"iam:CreateOpenIDConnectProvider",
"iam:DeleteOpenIDConnectProvider",
"iam:TagOpenIDConnectProvider",
"iam:UntagOpenIDConnectProvider",
"iam:UpdateOpenIDConnectProviderThumbprint",
"iam:RemoveClientIDFromOpenIDConnectProvider",
"iam:AddClientIDToOpenIDConnectProvider"
]
}
}

data "aws_iam_policy_document" "giantswarm_admin_assume" {
statement {
effect = "Allow"

principals {
type = "AWS"
identifiers = ["arn:${data.aws_partition.current.partition}:iam::${var.gs_user_account}:root"]
}

actions = ["sts:AssumeRole"]
}
}

resource "aws_iam_role" "giantswarm_admin" {
name = "GiantSwarmAdmin"
assume_role_policy = data.aws_iam_policy_document.giantswarm_admin_assume.json
}

resource "aws_iam_policy" "giantswarm_admin_policy" {
name = "GiantSwarmAdmin"
policy = data.aws_iam_policy_document.giantswarm_admin.json
}

resource "aws_iam_role_policy_attachment" "giantswarm_policy_attachment" {
role = aws_iam_role.giantswarm_admin.name
policy_arn = aws_iam_policy.giantswarm_admin_policy.arn
}
Loading
Loading