Skip to content

Commit

Permalink
Merge pull request #29 from jfallows/nitro-enclaves-acm
Browse files Browse the repository at this point in the history
Changes to support nitro enclaves acm
  • Loading branch information
jfallows authored Oct 2, 2024
2 parents 5216f44 + 3ccc518 commit bc28146
Show file tree
Hide file tree
Showing 6 changed files with 2,370 additions and 2,247 deletions.
38 changes: 30 additions & 8 deletions amazon-msk/cdktf/secure-public-access/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,23 @@ cp terraform.tfvars.example terraform.tfvars
To get a list all MSK clusters run:

```bash
aws kafka list-clusters --query 'ClusterInfoList[*].[ClusterName,ClusterArn]' --output table
aws kafka list-clusters --query 'ClusterInfoList[*].{Name:ClusterName, Arn:ClusterArn, Iam:ClientAuthentication.Iam.Enabled, Scram:ClientAuthentication.Sasl.Scram.Enabled, Tls:ClientAuthentication.Tls.Enabled, mTls:ClientAuthentication.Tls.CertificateAuthorityArnList[*] | join(`,`, @) || None, Unauthenticated:ClientAuthentication.Unauthenticated.Enabled}' --output table
```

Use the `ClusterName` of your desired MSK cluster for this variable.
Set the desired client authentication method based on the MSK cluster setup, using `MSK_ACCESS_METHOD` environment variable.

### `public_tls_certificate_key`: Public TLS Certificate Key

You need the ARN of the Secrets Manager secret that contains your public TLS certificate private key.
You need the ARN of either the Certificte Manager certificate or the Secrets Manager secret that contains your public TLS certificate private key.

List all certificates in Certificate Manager:

```bash
aws acm list-certificates --certificate-statuses ISSUED --query 'CertificateSummaryList[*].[DomainName,CertificateArn]' --output table
```

Find and note down the ARN of your public TLS certificate.

List all secrets in Secrets Manager:

Expand All @@ -58,6 +67,7 @@ Find and note down the ARN of the secret that contains your public TLS certifica
### `public_wildcard_dns`: Public Wildcard DNS

This variable defines the public wildcard DNS pattern for bootstrap servers to be used by Kafka clients.
It should match the wildcard DNS of the public TLS certificate.

### `zilla_plus_capacity`: Zilla Plus Capacity

Expand Down Expand Up @@ -109,13 +119,11 @@ cp .env.example .env

### MSK Client Authentication Method

By default Zilla Plus will choose the most secure way configured for your MSK cluster. Order from most to least secure:
To specify which client authentication method Zilla should use set the `MSK_ACCESS_METHOD` environment variable to the desired access method (mTLS, SASL/SCRAM or Unauthorized).

1. mTLS
1. SASL/SCRAM
1. Unauthorized
### Public TLS Certificate Via ACM

If you want to specify which client authentication method Zilla should use set the `MSK_ACCESS_METHOD` environment variable to the desired access method (mTLS, SASL/SCRAM or Unauthorized).
By default Zilla Plus will assume TLS certificate coming from Secrets Manager. You can use Zilla Plus with TLS certificate via ACM. To enable this set `PUBLIC_TLS_CERTIFICATE_VIA_ACM` to `true`.

### Custom Zilla Plus Role

Expand Down Expand Up @@ -202,6 +210,16 @@ Install the node.js dependencies specified in the `package.json` file:
npm install
```

### Generate external Terraform Provider constructs

Navigate to the CDKTF project directory.

Run the following command to generate the constructs for external providers:

```bash
npm run get
```

### Synthesize the Terraform Configuration

First, you need to synthesize the Terraform configuration from the CDKTF code.
Expand Down Expand Up @@ -240,7 +258,11 @@ terraform -chdir=cdktf.out/stacks/secure-public-access apply

### Configure Global DNS

This ensures that any new Kafka brokers added to the cluster can still be reached via the Zilla proxy. When using a wildcard DNS name for your own domain, such as `*.example.aklivity.io` then the DNS entries are setup in your DNS provider. After deploying the stack, check the outputs, where you can find the NetworkLoadBalancer DNS. `NetworkLoadBalancerOutput = "network-load-balancer-******.elb.us-east-1.amazonaws.com"` Lookup the IP addresses of your load balancer using `nslookup` and the DNS of the NetworkLoadBalancer.
This ensures that any new Kafka brokers added to the cluster can still be reached via the Zilla proxy. When using a wildcard DNS name for your own domain, such as `*.example.aklivity.io` then the DNS entries are setup in your DNS provider. After deploying the stack, check the outputs, where you can find the NetworkLoadBalancer DNS.
```
NetworkLoadBalancerOutput = "network-load-balancer-******.elb.us-east-1.amazonaws.com"
```
Lookup the IP addresses of your load balancer using `nslookup` and the DNS of the NetworkLoadBalancer.

```bash
nslookup network-load-balancer-86334a80cbd16ec2.elb.us-east-2.amazonaws.com
Expand Down
4 changes: 3 additions & 1 deletion amazon-msk/cdktf/secure-public-access/cdktf.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
"app": "npx ts-node main.ts",
"projectId": "cf699bc3-a88e-425b-bb8f-29b9e1181022",
"sendCrashReports": "false",
"terraformProviders": [],
"terraformProviders": [
"hashicorp/awscc@~> 1.14.0"
],
"terraformModules": [],
"context": {

Expand Down
163 changes: 115 additions & 48 deletions amazon-msk/cdktf/secure-public-access/main.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
import { Construct } from "constructs";
import { App, TerraformStack, TerraformOutput, TerraformVariable, Fn, Op } from "cdktf";
import instanceTypes from "./instance-types";
import { AwsProvider } from "@cdktf/provider-aws/lib/provider";
import { Lb } from "@cdktf/provider-aws/lib/lb";
import { LbListener } from "@cdktf/provider-aws/lib/lb-listener";
import { dataAwsAmi, launchTemplate } from "@cdktf/provider-aws";
import { autoscalingGroup } from "@cdktf/provider-aws";
import { autoscalingGroup, dataAwsAmi, launchTemplate } from "@cdktf/provider-aws";
import { LbTargetGroup } from "@cdktf/provider-aws/lib/lb-target-group";
import { DataAwsAcmpcaCertificateAuthority } from "@cdktf/provider-aws/lib/data-aws-acmpca-certificate-authority";
import { DataAwsSecretsmanagerSecretVersion } from "@cdktf/provider-aws/lib/data-aws-secretsmanager-secret-version";
Expand All @@ -28,13 +26,18 @@ import { DataAwsSubnets } from "@cdktf/provider-aws/lib/data-aws-subnets";
import { IamInstanceProfile } from "@cdktf/provider-aws/lib/iam-instance-profile";

import { UserVariables } from "./variables";
import { AwsProvider } from "@cdktf/provider-aws/lib/provider";
import { ec2EnclaveCertificateIamRoleAssociation } from "./.gen/providers/awscc"
import { AwsccProvider } from "./.gen/providers/awscc/provider";


export class ZillaPlusSecurePublicAccessStack extends TerraformStack {
constructor(scope: Construct, id: string) {
super(scope, id);
const userVariables = new UserVariables(this, "main");

const awsProvider = new AwsProvider(this, "AWS", {});
const awsProvider = new AwsProvider(this, "AWS", { });
new AwsccProvider(this, "AWSCC", { });

const region = new DataAwsRegion(this, "CurrentRegion", {
provider: awsProvider,
Expand Down Expand Up @@ -129,15 +132,6 @@ export class ZillaPlusSecurePublicAccessStack extends TerraformStack {
}

let mskClientAuthentication = userVariables.mskClientAuthentication;
if (userVariables.mskClientAuthentication === "Unknown") {
mskClientAuthentication = mskCluster.bootstrapBrokersTls
? "mTLS"
: mskCluster.bootstrapBrokersSaslScram
? "SASL/SCRAM"
: mskCluster.bootstrapBrokers
? "Unauthorized"
: userVariables.mskClientAuthentication;
}

const bootstrapServers =
mskClientAuthentication === "mTLS"
Expand Down Expand Up @@ -189,6 +183,13 @@ export class ZillaPlusSecurePublicAccessStack extends TerraformStack {
- ${mskCertificateAuthority}`;
}

const publicTlsCertificateKey = new TerraformVariable(this, "public_tls_certificate_key", {
type: "string",
description: "TLS Certificate SecretsManager or CertificateManager ARN",
});

const publicTlsCertificateViaAcm = userVariables.publicTlsCertificateViaAcm;

let zillaPlusRole;
if (!userVariables.createZillaPlusRole) {
const zillaPlusRoleVar = new TerraformVariable(this, "zilla_plus_role_name", {
Expand Down Expand Up @@ -259,19 +260,49 @@ export class ZillaPlusSecurePublicAccessStack extends TerraformStack {
role: iamRole.name,
});

const iamPolicy = {
Version: "2012-10-17",
Statement: [
{
Sid: "secretStatement",
Effect: "Allow",
Action: ["secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret"],
Resource: ["arn:aws:secretsmanager:*:*:secret:*"],
},
],
};

if (publicTlsCertificateViaAcm) {
iamPolicy.Statement = iamPolicy.Statement.concat([
{
"Sid": "s3Statement",
"Effect": "Allow",
"Action": [ "s3:GetObject" ],
"Resource": ["arn:aws:s3:::*/*"]
},
{
"Sid": "kmsDecryptStatement",
"Effect": "Allow",
"Action": [ "kms:Decrypt" ],
"Resource": ["arn:aws:kms:*:*:key/*"]
},
{
"Sid": "getRoleStatement",
"Effect": "Allow",
"Action": [ "iam:GetRole" ],
"Resource": [ `arn:aws:iam::*:role/${iamRole.name}` ]
}]
);

new ec2EnclaveCertificateIamRoleAssociation.Ec2EnclaveCertificateIamRoleAssociation(this, "ZillaPlusEnclaveIamRoleAssociation", {
roleArn: iamRole.arn,
certificateArn: publicTlsCertificateKey.stringValue
});
}

new IamRolePolicy(this, "ZillaPlusRolePolicy", {
role: iamRole.name,
policy: JSON.stringify({
Version: "2012-10-17",
Statement: [
{
Sid: "VisualEditor0",
Effect: "Allow",
Action: ["secretsmanager:GetSecretValue", "secretsmanager:DescribeSecret"],
Resource: ["arn:aws:secretsmanager:*:*:secret:*"],
},
],
}),
policy: JSON.stringify(iamPolicy),
});

zillaPlusRole = iamInstanceProfile.name;
Expand Down Expand Up @@ -329,14 +360,12 @@ export class ZillaPlusSecurePublicAccessStack extends TerraformStack {
description: "The public wildcard DNS pattern for bootstrap servers to be used by Kafka clients",
});

const publicTlsCertificateKey = new TerraformVariable(this, "public_tls_certificate_key", {
type: "string",
description: "TLS Certificate Private Key Secret ARN",
});
// Validate that the Certificate Key exists
new DataAwsSecretsmanagerSecretVersion(this, "publicTlsCertificate", {
secretId: publicTlsCertificateKey.stringValue,
});
if (!publicTlsCertificateViaAcm) {
// Validate that the Certificate Key exists
new DataAwsSecretsmanagerSecretVersion(this, "publicTlsCertificate", {
secretId: publicTlsCertificateKey.stringValue,
});
}

let keyName = "";

Expand All @@ -348,9 +377,34 @@ export class ZillaPlusSecurePublicAccessStack extends TerraformStack {
keyName = keyNameVar.stringValue;
}

let acmYamlContent = "";
let enclavesAcmServiceStart = "";
let zillaTelemetryContent = "";
let bindingTelemetryContent = "";

if (publicTlsCertificateViaAcm) {
acmYamlContent = `
enclave:
cpu_count: 2
memory_mib: 256
options:
sync_interval_secs: 600
tokens:
- label: acm-token-example
source:
Acm:
certificate_arn: '${publicTlsCertificateKey}'
refresh_interval_secs: 43200
pin: 1234
`
enclavesAcmServiceStart = `
systemctl enable nitro-enclaves-acm.service
systemctl start nitro-enclaves-acm.service
`
}

if (!userVariables.cloudwatchDisabled) {
const defaultLogGroupName = `${id}-group`;
const defaultMetricNamespace = `${id}-namespace`;
Expand Down Expand Up @@ -410,27 +464,32 @@ ${metricsSection}`;

const instanceType = new TerraformVariable(this, "zilla_plus_instance_type", {
type: "string",
default: "t3.small",
default: publicTlsCertificateViaAcm ? "c6i.xlarge" : "t3.small",
description: "Zilla Plus EC2 instance type",
});
instanceType.addValidation({
condition: `${Fn.contains(instanceTypes.instanceTypes, instanceType.stringValue)}`,
errorMessage: "must be a valid EC2 instance type.",
});

const ami = new dataAwsAmi.DataAwsAmi(this, "LatestAmi", {
mostRecent: true,
filter: [
{
name: "product-code",
values: ["ca5mgk85pjtbyuhtfluzisgzy"],
},
{
name: "is-public",
values: ["true"],
},
],
});
let imageId = userVariables.zillaPlusAmi;
if (!imageId)
{
const ami = new dataAwsAmi.DataAwsAmi(this, "LatestAmi", {
mostRecent: true,
filter: [
{
name: "product-code",
values: ["ca5mgk85pjtbyuhtfluzisgzy"],
},
{
name: "is-public",
values: ["true"],
},
],
});
imageId = ami.imageId;
}

const nlb = new Lb(this, `NetworkLoadBalancer-${id}`, {
name: "network-load-balancer",
Expand Down Expand Up @@ -467,7 +526,7 @@ ${metricsSection}`;
name: public
vaults:
secure:
type: aws
type: ${publicTlsCertificateViaAcm ? "aws-acm" : "aws-secrets"}
${zillaTelemetryContent}
bindings:
tcp_server:
Expand Down Expand Up @@ -540,6 +599,10 @@ cat <<EOF > /etc/zilla/zilla.yaml
${zillaYamlContent}
EOF
cat <<EOF > /etc/nitro_enclaves/acm.yaml
${acmYamlContent}
EOF
chown ec2-user:ec2-user /etc/zilla/zilla.yaml
mkdir /etc/cfn
Expand All @@ -562,13 +625,14 @@ systemctl enable cfn-hup
systemctl start cfn-hup
systemctl enable amazon-ssm-agent
systemctl start amazon-ssm-agent
${enclavesAcmServiceStart}
systemctl enable zilla-plus
systemctl start zilla-plus
`;

const ZillaPlusLaunchTemplate = new launchTemplate.LaunchTemplate(this, "ZillaPlusLaunchTemplate", {
imageId: ami.imageId,
imageId: imageId,
instanceType: instanceType.stringValue,
networkInterfaces: [
{
Expand All @@ -580,6 +644,9 @@ systemctl start zilla-plus
iamInstanceProfile: {
name: zillaPlusRole,
},
enclaveOptions: {
enabled: publicTlsCertificateViaAcm
},
keyName: keyName,
userData: Fn.base64encode(userData),
});
Expand Down
Loading

0 comments on commit bc28146

Please sign in to comment.