From ad36eabe04547f2263ea5a74805a252b523ea6ed Mon Sep 17 00:00:00 2001 From: Yong Wen Chua Date: Mon, 19 Mar 2018 10:30:17 +0800 Subject: [PATCH] Add documentation --- modules/consul-cluster/README.md | 170 ++++++++++++++++++++----------- 1 file changed, 108 insertions(+), 62 deletions(-) diff --git a/modules/consul-cluster/README.md b/modules/consul-cluster/README.md index e4727eed..e550c560 100644 --- a/modules/consul-cluster/README.md +++ b/modules/consul-cluster/README.md @@ -1,8 +1,8 @@ # Consul Cluster -This folder contains a [Terraform](https://www.terraform.io/) module to deploy a -[Consul](https://www.consul.io/) cluster in [AWS](https://aws.amazon.com/) on top of an Auto Scaling Group. This module -is designed to deploy an [Amazon Machine Image (AMI)](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) +This folder contains a [Terraform](https://www.terraform.io/) module to deploy a +[Consul](https://www.consul.io/) cluster in [AWS](https://aws.amazon.com/) on top of an Auto Scaling Group. This module +is designed to deploy an [Amazon Machine Image (AMI)](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) that has Consul installed via the [install-consul](https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/install-consul) module in this Module. @@ -20,42 +20,42 @@ module "consul_cluster" { # Specify the ID of the Consul AMI. You should build this using the scripts in the install-consul module. ami_id = "ami-abcd1234" - + # Add this tag to each node in the cluster cluster_tag_key = "consul-cluster" cluster_tag_value = "consul-cluster-example" - - # Configure and start Consul during boot. It will automatically form a cluster with all nodes that have that same tag. + + # Configure and start Consul during boot. It will automatically form a cluster with all nodes that have that same tag. user_data = <<-EOF #!/bin/bash /opt/consul/bin/run-consul --server --cluster-tag-key consul-cluster EOF - + # ... See variables.tf for the other parameters you must define for the consul-cluster module } ``` Note the following parameters: -* `source`: Use this parameter to specify the URL of the consul-cluster module. The double slash (`//`) is intentional - and required. Terraform uses it to specify subfolders within a Git repo (see [module - sources](https://www.terraform.io/docs/modules/sources.html)). The `ref` parameter specifies a specific Git tag in - this repo. That way, instead of using the latest version of this module from the `master` branch, which +* `source`: Use this parameter to specify the URL of the consul-cluster module. The double slash (`//`) is intentional + and required. Terraform uses it to specify subfolders within a Git repo (see [module + sources](https://www.terraform.io/docs/modules/sources.html)). The `ref` parameter specifies a specific Git tag in + this repo. That way, instead of using the latest version of this module from the `master` branch, which will change every time you run Terraform, you're using a fixed version of the repo. -* `ami_id`: Use this parameter to specify the ID of a Consul [Amazon Machine Image +* `ami_id`: Use this parameter to specify the ID of a Consul [Amazon Machine Image (AMI)](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) to deploy on each server in the cluster. You should install Consul in this AMI using the scripts in the [install-consul](https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/install-consul) module. - -* `user_data`: Use this parameter to specify a [User + +* `user_data`: Use this parameter to specify a [User Data](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html#user-data-shell-scripts) script that each - server will run during boot. This is where you can use the [run-consul script](https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/run-consul) to configure and - run Consul. The `run-consul` script is one of the scripts installed by the [install-consul](https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/install-consul) - module. + server will run during boot. This is where you can use the [run-consul script](https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/run-consul) to configure and + run Consul. The `run-consul` script is one of the scripts installed by the [install-consul](https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/install-consul) + module. You can find the other parameters in [variables.tf](variables.tf). -Check out the [consul-cluster example](https://github.com/hashicorp/terraform-aws-consul/tree/master/MAIN.md) for fully-working sample code. +Check out the [consul-cluster example](https://github.com/hashicorp/terraform-aws-consul/tree/master/MAIN.md) for fully-working sample code. @@ -64,15 +64,15 @@ Check out the [consul-cluster example](https://github.com/hashicorp/terraform-aw ### Using the HTTP API from your own computer -If you want to connect to the cluster from your own computer, the easiest way is to use the [HTTP -API](https://www.consul.io/docs/agent/http.html). Note that this only works if the Consul cluster is running in public +If you want to connect to the cluster from your own computer, the easiest way is to use the [HTTP +API](https://www.consul.io/docs/agent/http.html). Note that this only works if the Consul cluster is running in public subnets and/or your default VPC (as in the [consul-cluster example](https://github.com/hashicorp/terraform-aws-consul/tree/master/MAIN.md)), which is OK for testing and experimentation, but NOT recommended for production usage. -To use the HTTP API, you first need to get the public IP address of one of the Consul Servers. You can find Consul -servers by using AWS tags. If you're running the [consul-cluster example](https://github.com/hashicorp/terraform-aws-consul/tree/master/MAIN.md), the -[consul-examples-helper.sh script](https://github.com/hashicorp/terraform-aws-consul/tree/master/examples/consul-examples-helper/consul-examples-helper.sh) will do the tag lookup -for you automatically (note, you must have the [AWS CLI](https://aws.amazon.com/cli/), +To use the HTTP API, you first need to get the public IP address of one of the Consul Servers. You can find Consul +servers by using AWS tags. If you're running the [consul-cluster example](https://github.com/hashicorp/terraform-aws-consul/tree/master/MAIN.md), the +[consul-examples-helper.sh script](https://github.com/hashicorp/terraform-aws-consul/tree/master/examples/consul-examples-helper/consul-examples-helper.sh) will do the tag lookup +for you automatically (note, you must have the [AWS CLI](https://aws.amazon.com/cli/), [jq](https://stedolan.github.io/jq/), and the [Consul agent](https://www.consul.io/) installed locally): ``` @@ -111,7 +111,7 @@ Success! Data written to: foo ``` And reading that value back: - + ``` > consul kv get -http-addr=11.22.33.44:8500 foo @@ -125,8 +125,8 @@ Finally, you can try opening up the Consul UI in your browser at the URL `http:/ ### Using the Consul agent on another EC2 Instance -The easiest way to run [Consul agent](https://www.consul.io/docs/agent/basics.html) and have it connect to the Consul -cluster is to use the same EC2 tags the Consul servers use to discover each other during bootstrapping. +The easiest way to run [Consul agent](https://www.consul.io/docs/agent/basics.html) and have it connect to the Consul +cluster is to use the same EC2 tags the Consul servers use to discover each other during bootstrapping. For example, imagine you deployed a Consul cluster in `us-east-1` as follows: @@ -139,12 +139,12 @@ module "consul_cluster" { # Add this tag to each node in the cluster cluster_tag_key = "consul-cluster" cluster_tag_value = "consul-cluster-example" - - # ... Other params omitted ... + + # ... Other params omitted ... } ``` -Using the `retry-join-ec2-xxx` params, you can connect run a Consul agent on an EC2 Instance as follows: +Using the `retry-join-ec2-xxx` params, you can connect run a Consul agent on an EC2 Instance as follows: ``` consul agent -retry-join-ec2-tag-key=consul-cluster -retry-join-ec2-tag-value=consul-cluster-example -data-dir=/tmp/consul @@ -152,13 +152,59 @@ consul agent -retry-join-ec2-tag-key=consul-cluster -retry-join-ec2-tag-value=co Two important notes about this command: -1. By default, the Consul cluster nodes advertise their *private* IP addresses, so the command above only works from +1. By default, the Consul cluster nodes advertise their *private* IP addresses, so the command above only works from EC2 Instances inside the same VPC (or any VPC with proper peering connections and route table entries). 1. In order to look up the EC2 tags, the EC2 Instance where you're running this command must have an IAM role with the `ec2:DescribeInstances` permission. +## How do you connect load balancers to the Auto Scaling Group (ASG)? + +You can use the [`aws_autoscaling_attachment`](https://www.terraform.io/docs/providers/aws/r/autoscaling_attachment.html) resource. + +For example, if you are using the new application or network load balancers: + +```hcl +resource "aws_lb_target_group" "test" { + // ... +} + +# Create a new Consul Cluster +module "consul" { + source ="..." + // ... +} + +# Create a new load balancer attachment +resource "aws_autoscaling_attachment" "asg_attachment_bar" { + autoscaling_group_name = "${module.consul.asg_name}" + alb_target_group_arn = "${aws_alb_target_group.test.arn}" +} +``` + +If you are using a "classic" load balancer: + +```hcl +# Create a new load balancer +resource "aws_elb" "bar" { + // ... +} + +# Create a new Consul Cluster +module "consul" { + source ="..." + // ... +} + +# Create a new load balancer attachment +resource "aws_autoscaling_attachment" "asg_attachment_bar" { + autoscaling_group_name = "${module.consul.asg_name}" + elb = "${aws_elb.bar.id}" +} +``` + + ## What's included in this module? @@ -177,7 +223,7 @@ This architecture consists of the following resources: ### Auto Scaling Group This module runs Consul on top of an [Auto Scaling Group (ASG)](https://aws.amazon.com/autoscaling/). Typically, you -should run the ASG with 3 or 5 EC2 Instances spread across multiple [Availability +should run the ASG with 3 or 5 EC2 Instances spread across multiple [Availability Zones](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html). Each of the EC2 Instances should be running an AMI that has Consul installed via the [install-consul](https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/install-consul) module. You pass in the ID of the AMI to run using the `ami_id` input parameter. @@ -186,30 +232,30 @@ module. You pass in the ID of the AMI to run using the `ami_id` input parameter. ### EC2 Instance Tags This module allows you to specify a tag to add to each EC2 instance in the ASG. We recommend using this tag with the -[retry_join_ec2](https://www.consul.io/docs/agent/options.html?#retry_join_ec2) configuration to allow the EC2 -Instances to find each other and automatically form a cluster. +[retry_join_ec2](https://www.consul.io/docs/agent/options.html?#retry_join_ec2) configuration to allow the EC2 +Instances to find each other and automatically form a cluster. ### Security Group Each EC2 Instance in the ASG has a Security Group that allows: - + * All outbound requests * All the inbound ports specified in the [Consul documentation](https://www.consul.io/docs/agent/options.html?#ports-used) -The Security Group ID is exported as an output variable if you need to add additional rules. +The Security Group ID is exported as an output variable if you need to add additional rules. -Check out the [Security section](#security) for more details. +Check out the [Security section](#security) for more details. ### IAM Role and Permissions -Each EC2 Instance in the ASG has an [IAM Role](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) attached. -We give this IAM role a small set of IAM permissions that each EC2 Instance can use to automatically discover the other -Instances in its ASG and form a cluster with them. See the [run-consul required permissions +Each EC2 Instance in the ASG has an [IAM Role](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) attached. +We give this IAM role a small set of IAM permissions that each EC2 Instance can use to automatically discover the other +Instances in its ASG and form a cluster with them. See the [run-consul required permissions docs](https://github.com/hashicorp/terraform-aws-consul/tree/master/modules/run-consul#required-permissions) for details. -The IAM Role ARN is exported as an output variable if you need to add additional permissions. +The IAM Role ARN is exported as an output variable if you need to add additional permissions. @@ -229,9 +275,9 @@ NOT actually deploy those new instances. To make that happen, you should do the ``` curl -X PUT :8500/v1/agent/leave ``` - + 1. Once the instance has left the cluster, terminate it: - + ``` aws ec2 terminate-instances --instance-ids ``` @@ -241,7 +287,7 @@ NOT actually deploy those new instances. To make that happen, you should do the 1. Wait for the new Instance to boot and join the cluster. 1. Repeat these steps for each of the other old Instances in the ASG. - + We will add a script in the future to automate this process (PRs are welcome!). @@ -250,13 +296,13 @@ We will add a script in the future to automate this process (PRs are welcome!). ## What happens if a node crashes? There are two ways a Consul node may go down: - + 1. The Consul process may crash. In that case, `supervisor` should restart it automatically. -1. The EC2 Instance running Consul dies. In that case, the Auto Scaling Group should launch a replacement automatically. +1. The EC2 Instance running Consul dies. In that case, the Auto Scaling Group should launch a replacement automatically. Note that in this case, since the Consul agent did not exit gracefully, and the replacement will have a different ID, you may have to manually clean out the old nodes using the [force-leave - command](https://www.consul.io/docs/commands/force-leave.html). We may add a script to do this - automatically in the future. For more info, see the [Consul Outage + command](https://www.consul.io/docs/commands/force-leave.html). We may add a script to do this + automatically in the future. For more info, see the [Consul Outage documentation](https://www.consul.io/docs/guides/outage.html). @@ -283,30 +329,30 @@ Consul can encrypt all of its network traffic. For instructions on enabling netw The EC2 Instances in the cluster store all their data on the root EBS Volume. To enable encryption for the data at rest, you must enable encryption in your Consul AMI. If you're creating the AMI using Packer (e.g. as shown in -the [consul-ami example](https://github.com/hashicorp/terraform-aws-consul/tree/master/examples/consul-ami)), you need to set the [encrypt_boot -parameter](https://www.packer.io/docs/builders/amazon-ebs.html#encrypt_boot) to `true`. +the [consul-ami example](https://github.com/hashicorp/terraform-aws-consul/tree/master/examples/consul-ami)), you need to set the [encrypt_boot +parameter](https://www.packer.io/docs/builders/amazon-ebs.html#encrypt_boot) to `true`. ### Dedicated instances -If you wish to use dedicated instances, you can set the `tenancy` parameter to `"dedicated"` in this module. +If you wish to use dedicated instances, you can set the `tenancy` parameter to `"dedicated"` in this module. ### Security groups This module attaches a security group to each EC2 Instance that allows inbound requests as follows: -* **Consul**: For all the [ports used by Consul](https://www.consul.io/docs/agent/options.html#ports), you can - use the `allowed_inbound_cidr_blocks` parameter to control the list of +* **Consul**: For all the [ports used by Consul](https://www.consul.io/docs/agent/options.html#ports), you can + use the `allowed_inbound_cidr_blocks` parameter to control the list of [CIDR blocks](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) that will be allowed access and the `allowed_inbound_security_group_ids` parameter to control the security groups that will be allowed access. -* **SSH**: For the SSH port (default: 22), you can use the `allowed_ssh_cidr_blocks` parameter to control the list of +* **SSH**: For the SSH port (default: 22), you can use the `allowed_ssh_cidr_blocks` parameter to control the list of [CIDR blocks](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing) that will be allowed access. You can use the `allowed_inbound_ssh_security_group_ids` parameter to control the list of source Security Groups that will be allowed access. - + Note that all the ports mentioned above are configurable via the `xxx_port` variables (e.g. `server_rpc_port`). See -[variables.tf](variables.tf) for the full list. - - +[variables.tf](variables.tf) for the full list. + + ### SSH access @@ -329,15 +375,15 @@ This module does NOT handle the following items, which you may want to provide o ### Monitoring, alerting, log aggregation -This module does not include anything for monitoring, alerting, or log aggregation. All ASGs and EC2 Instances come -with limited [CloudWatch](https://aws.amazon.com/cloudwatch/) metrics built-in, but beyond that, you will have to +This module does not include anything for monitoring, alerting, or log aggregation. All ASGs and EC2 Instances come +with limited [CloudWatch](https://aws.amazon.com/cloudwatch/) metrics built-in, but beyond that, you will have to provide your own solutions. ### VPCs, subnets, route tables -This module assumes you've already created your network topology (VPC, subnets, route tables, etc). You will need to -pass in the the relevant info about your network topology (e.g. `vpc_id`, `subnet_ids`) as input variables to this +This module assumes you've already created your network topology (VPC, subnets, route tables, etc). You will need to +pass in the the relevant info about your network topology (e.g. `vpc_id`, `subnet_ids`) as input variables to this module.