Merge branch 'develop' of github.com:npalm/terraform-aws-gitlab-runner

CHANGELOG.md

@@ -6,6 +6,13 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 
 
 ## Unreleased
+- Change: default version of the runner to 12.6.0
+- Fix: External references from destroy provisioners are deprecated (examples)
+- Fix: typos cache bucket (#172) @@thorec
+- Fix: missing double quotes (#171) ggrangia
+- Change: default for gitlab_url to https://gitlab.com (#170) @riccardomc
+- Change: Encrypt runner root device by default (#168) …
+- Added: allow eip for runner (#166) 
 
 ## 4.9.0 - 2019-11-14
 - Make use of on-demand instances in docker-machine #158 @skorfmann

README.md

@@ -231,7 +231,7 @@ terraform destroy
 | ami\_owners | The list of owners used to select the AMI of Gitlab runner agent instances. | list(string) | `<list>` | no |
 | aws\_region | AWS region. | string | n/a | yes |
 | aws\_zone | AWS availability zone (typically 'a', 'b', or 'c'). | string | `"a"` | no |
-| cache\_bucket | Configuration to control the creation of the cache bucket. By default the bucket will be created and used as shared cache. To use the same cache cross multiple runners disable the cration of the cache and provice a policy and bucket name. See the public runner example for more details. | map | `<map>` | no |
+| cache\_bucket | Configuration to control the creation of the cache bucket. By default the bucket will be created and used as shared cache. To use the same cache across multiple runners disable the creation of the cache and provide a policy and bucket name. See the public runner example for more details. | map | `<map>` | no |
 | cache\_bucket\_name\_include\_account\_id | Boolean to add current account ID to cache bucket name. | bool | `"true"` | no |
 | cache\_bucket\_prefix | Prefix for s3 cache bucket name. | string | `""` | no |
 | cache\_bucket\_versioning | Boolean used to enable versioning on the cache bucket, false by default. | bool | `"false"` | no |
@@ -244,6 +244,7 @@ terraform destroy
 | docker\_machine\_spot\_price\_bid | Spot price bid. | string | `"0.06"` | no |
 | docker\_machine\_version | Version of docker-machine. | string | `"0.16.2"` | no |
 | enable\_cloudwatch\_logging | Boolean used to enable or disable the CloudWatch logging. | bool | `"true"` | no |
+| enable\_eip | Enable the assignment of an EIP to the gitlab runner instance | bool | `"false"` | no |
 | enable\_gitlab\_runner\_ssh\_access | Enables SSH Access to the gitlab runner instance. | bool | `"false"` | no |
 | enable\_kms | Let the module manage a KMS key, logs will be encrypted via KMS. Be-aware of the costs of an custom key. | bool | `"false"` | no |
 | enable\_manage\_gitlab\_token | Boolean to enable the management of the GitLab token in SSM. If `true` the token will be stored in SSM, which means the SSM property is a terraform managed resource. If `false` the Gitlab token will be stored in the SSM by the user-data script during creation of the the instance. However the SSM parameter is not managed by terraform and will remain in SSM after a `terraform destroy`. | bool | `"true"` | no |
@@ -253,7 +254,7 @@ terraform destroy
 | environment | A name that identifies the environment, used as prefix and for tagging. | string | n/a | yes |
 | gitlab\_runner\_registration\_config | Configuration used to register the runner. See the README for an example, or reference the examples in the examples directory of this repo. | map(string) | `<map>` | no |
 | gitlab\_runner\_ssh\_cidr\_blocks | List of CIDR blocks to allow SSH Access to the gitlab runner instance. | list(string) | `<list>` | no |
-| gitlab\_runner\_version | Version of the GitLab runner. | string | `"12.4.1"` | no |
+| gitlab\_runner\_version | Version of the GitLab runner. | string | `"12.6.0"` | no |
 | instance\_role\_json | Default runner instance override policy, expected to be in JSON format. | string | `""` | no |
 | instance\_type | Instance type used for the GitLab runner. | string | `"t3.micro"` | no |
 | kms\_deletion\_window\_in\_days | Key rotation window, set to 0 for no rotation. Only used when `enable_kms` is set to `true`. | number | `"7"` | no |
@@ -262,7 +263,7 @@ terraform destroy
 | runner\_ami\_filter | List of maps used to create the AMI filter for the Gitlab runner docker-machine AMI. | map(list(string)) | `<map>` | no |
 | runner\_ami\_owners | The list of owners used to select the AMI of Gitlab runner docker-machine instances. | list(string) | `<list>` | no |
 | runner\_instance\_spot\_price | By setting a spot price bid price the runner agent will be created via a spot request. Be aware that spot instances can be stopped by AWS. | string | `""` | no |
-| runner\_root\_block\_device | The EC2 instance root block device configuration. Takes the following keys: `delete_on_termination`, `volume_type`, `volume_size`, `iops` | map(string) | `<map>` | no |
+| runner\_root\_block\_device | The EC2 instance root block device configuration. Takes the following keys: `delete_on_termination`, `volume_type`, `volume_size`, `encrypted`, `iops` | map(string) | `<map>` | no |
 | runners\_additional\_volumes | Additional volumes that will be used in the runner config.toml, e.g Docker socket | list | `<list>` | no |
 | runners\_concurrent | Concurrent value for the runners, will be used in the runner config.toml. | number | `"10"` | no |
 | runners\_environment\_vars | Environment variables during build execution, e.g. KEY=Value, see runner-public example. Will be used in the runner config.toml | list(string) | `<list>` | no |
@@ -315,6 +316,7 @@ terraform destroy
 | runner\_as\_group\_name | Name of the autoscaling group for the gitlab-runner instance |
 | runner\_cache\_bucket\_arn | ARN of the S3 for the build cache. |
 | runner\_cache\_bucket\_name | Name of the S3 for the build cache. |
+| runner\_eip | EIP of the Gitlab Runner |
 | runner\_role\_arn | ARN of the role used for the docker machine runners. |
 | runner\_role\_name | Name of the role used for the docker machine runners. |
 | runner\_sg\_id | ID of the security group attached to the docker machine runners. |

_docs/TF_MODULE.md

@@ -7,7 +7,7 @@
 | ami\_owners | The list of owners used to select the AMI of Gitlab runner agent instances. | list(string) | `<list>` | no |
 | aws\_region | AWS region. | string | n/a | yes |
 | aws\_zone | AWS availability zone (typically 'a', 'b', or 'c'). | string | `"a"` | no |
-| cache\_bucket | Configuration to control the creation of the cache bucket. By default the bucket will be created and used as shared cache. To use the same cache cross multiple runners disable the cration of the cache and provice a policy and bucket name. See the public runner example for more details. | map | `<map>` | no |
+| cache\_bucket | Configuration to control the creation of the cache bucket. By default the bucket will be created and used as shared cache. To use the same cache across multiple runners disable the creation of the cache and provide a policy and bucket name. See the public runner example for more details. | map | `<map>` | no |
 | cache\_bucket\_name\_include\_account\_id | Boolean to add current account ID to cache bucket name. | bool | `"true"` | no |
 | cache\_bucket\_prefix | Prefix for s3 cache bucket name. | string | `""` | no |
 | cache\_bucket\_versioning | Boolean used to enable versioning on the cache bucket, false by default. | bool | `"false"` | no |
@@ -20,6 +20,7 @@
 | docker\_machine\_spot\_price\_bid | Spot price bid. | string | `"0.06"` | no |
 | docker\_machine\_version | Version of docker-machine. | string | `"0.16.2"` | no |
 | enable\_cloudwatch\_logging | Boolean used to enable or disable the CloudWatch logging. | bool | `"true"` | no |
+| enable\_eip | Enable the assignment of an EIP to the gitlab runner instance | bool | `"false"` | no |
 | enable\_gitlab\_runner\_ssh\_access | Enables SSH Access to the gitlab runner instance. | bool | `"false"` | no |
 | enable\_kms | Let the module manage a KMS key, logs will be encrypted via KMS. Be-aware of the costs of an custom key. | bool | `"false"` | no |
 | enable\_manage\_gitlab\_token | Boolean to enable the management of the GitLab token in SSM. If `true` the token will be stored in SSM, which means the SSM property is a terraform managed resource. If `false` the Gitlab token will be stored in the SSM by the user-data script during creation of the the instance. However the SSM parameter is not managed by terraform and will remain in SSM after a `terraform destroy`. | bool | `"true"` | no |
@@ -29,7 +30,7 @@
 | environment | A name that identifies the environment, used as prefix and for tagging. | string | n/a | yes |
 | gitlab\_runner\_registration\_config | Configuration used to register the runner. See the README for an example, or reference the examples in the examples directory of this repo. | map(string) | `<map>` | no |
 | gitlab\_runner\_ssh\_cidr\_blocks | List of CIDR blocks to allow SSH Access to the gitlab runner instance. | list(string) | `<list>` | no |
-| gitlab\_runner\_version | Version of the GitLab runner. | string | `"12.4.1"` | no |
+| gitlab\_runner\_version | Version of the GitLab runner. | string | `"12.6.0"` | no |
 | instance\_role\_json | Default runner instance override policy, expected to be in JSON format. | string | `""` | no |
 | instance\_type | Instance type used for the GitLab runner. | string | `"t3.micro"` | no |
 | kms\_deletion\_window\_in\_days | Key rotation window, set to 0 for no rotation. Only used when `enable_kms` is set to `true`. | number | `"7"` | no |
@@ -38,7 +39,7 @@
 | runner\_ami\_filter | List of maps used to create the AMI filter for the Gitlab runner docker-machine AMI. | map(list(string)) | `<map>` | no |
 | runner\_ami\_owners | The list of owners used to select the AMI of Gitlab runner docker-machine instances. | list(string) | `<list>` | no |
 | runner\_instance\_spot\_price | By setting a spot price bid price the runner agent will be created via a spot request. Be aware that spot instances can be stopped by AWS. | string | `""` | no |
-| runner\_root\_block\_device | The EC2 instance root block device configuration. Takes the following keys: `delete_on_termination`, `volume_type`, `volume_size`, `iops` | map(string) | `<map>` | no |
+| runner\_root\_block\_device | The EC2 instance root block device configuration. Takes the following keys: `delete_on_termination`, `volume_type`, `volume_size`, `encrypted`, `iops` | map(string) | `<map>` | no |
 | runners\_additional\_volumes | Additional volumes that will be used in the runner config.toml, e.g Docker socket | list | `<list>` | no |
 | runners\_concurrent | Concurrent value for the runners, will be used in the runner config.toml. | number | `"10"` | no |
 | runners\_environment\_vars | Environment variables during build execution, e.g. KEY=Value, see runner-public example. Will be used in the runner config.toml | list(string) | `<list>` | no |
@@ -91,6 +92,7 @@
 | runner\_as\_group\_name | Name of the autoscaling group for the gitlab-runner instance |
 | runner\_cache\_bucket\_arn | ARN of the S3 for the build cache. |
 | runner\_cache\_bucket\_name | Name of the S3 for the build cache. |
+| runner\_eip | EIP of the Gitlab Runner |
 | runner\_role\_arn | ARN of the role used for the docker machine runners. |
 | runner\_role\_name | Name of the role used for the docker machine runners. |
 | runner\_sg\_id | ID of the security group attached to the docker machine runners. |

examples/runner-default/.terraform-version

@@ -1 +1 @@
-0.12.13
+0.12.18

examples/runner-default/main.tf

@@ -4,7 +4,7 @@ data "aws_availability_zones" "available" {
 
 module "vpc" {
   source  = "terraform-aws-modules/vpc/aws"
-  version = "2.17"
+  version = "2.21"
 
   name = "vpc-${var.environment}"
   cidr = "10.0.0.0/16"
@@ -35,6 +35,7 @@ module "runner" {
   runners_name             = var.runner_name
   runners_gitlab_url       = var.gitlab_url
   enable_runner_ssm_access = true
+  enable_eip               = true
 
   docker_machine_spot_price_bid = "0.06"
 
@@ -70,13 +71,14 @@ module "runner" {
   runners_off_peak_periods = "[\"* * 0-9,17-23 * * mon-fri *\", \"* * * * * sat,sun *\"]"
 }
 
-
-
 resource "null_resource" "cancel_spot_requests" {
   # Cancel active and open spot requests, terminate instances
+  triggers = {
+    environment = var.environment
+  }
 
   provisioner "local-exec" {
-    when    = "destroy"
-    command = "../../ci/bin/cancel-spot-instances.sh ${var.environment}"
+    when    = destroy
+    command = "../../ci/bin/cancel-spot-instances.sh ${self.triggers.environment}"
   }
 }

examples/runner-default/providers.tf

@@ -1,6 +1,6 @@
 provider "aws" {
   region  = var.aws_region
-  version = "2.34"
+  version = "2.41"
 }
 
 provider "template" {
@@ -18,4 +18,3 @@ provider "null" {
 provider "tls" {
   version = "2.1.1"
 }
-

examples/runner-docker/.terraform-version

@@ -1 +1 @@
-0.12.13
+0.12.18

examples/runner-docker/README.md

@@ -27,7 +27,7 @@ The terraform version is managed using [tfenv](https://github.com/Zordrak/tfenv)
 |------|-------------|:----:|:-----:|:-----:|
 | aws\_region | AWS region. | string | `"eu-west-1"` | no |
 | environment | A name that identifies the environment, will used as prefix and for tagging. | string | `"runners-docker"` | no |
-| gitlab\_url | URL of the gitlab instance to connect to. | string | `"https://www.gitlab.com"` | no |
+| gitlab\_url | URL of the gitlab instance to connect to. | string | `"https://gitlab.com"` | no |
 | private\_ssh\_key\_filename |  | string | `"generated/id_rsa"` | no |
 | public\_ssh\_key\_filename |  | string | `"generated/id_rsa.pub"` | no |
 | registration\_token |  | string | n/a | yes |

examples/runner-docker/_docs/TF_MODULE.md

@@ -4,7 +4,7 @@
 |------|-------------|:----:|:-----:|:-----:|
 | aws\_region | AWS region. | string | `"eu-west-1"` | no |
 | environment | A name that identifies the environment, will used as prefix and for tagging. | string | `"runners-docker"` | no |
-| gitlab\_url | URL of the gitlab instance to connect to. | string | `"https://www.gitlab.com"` | no |
+| gitlab\_url | URL of the gitlab instance to connect to. | string | `"https://gitlab.com"` | no |
 | private\_ssh\_key\_filename |  | string | `"generated/id_rsa"` | no |
 | public\_ssh\_key\_filename |  | string | `"generated/id_rsa.pub"` | no |
 | registration\_token |  | string | n/a | yes |

examples/runner-docker/main.tf

@@ -4,14 +4,15 @@ data "aws_availability_zones" "available" {
 
 module "vpc" {
   source  = "terraform-aws-modules/vpc/aws"
-  version = "2.17"
+  version = "2.21"
 
   name = "vpc-${var.environment}"
   cidr = "10.1.0.0/16"
 
-  azs                = [data.aws_availability_zones.available.names[0]]
-  public_subnets     = ["10.1.101.0/24"]
-  enable_s3_endpoint = true
+  azs                     = [data.aws_availability_zones.available.names[0]]
+  public_subnets          = ["10.1.101.0/24"]
+  enable_s3_endpoint      = true
+  map_public_ip_on_launch = false
 
   tags = {
     Environment = var.environment
@@ -24,9 +25,8 @@ module "runner" {
   aws_region  = var.aws_region
   environment = var.environment
 
-  ssh_public_key = local_file.public_ssh_key.content
-
   runners_use_private_address = false
+  enable_eip                  = true
 
   vpc_id                   = module.vpc.vpc_id
   subnet_ids_gitlab_runner = module.vpc.public_subnets

examples/runner-docker/providers.tf

@@ -1,6 +1,6 @@
 provider "aws" {
   region  = var.aws_region
-  version = "2.34"
+  version = "2.41"
 }
 
 provider "template" {
@@ -18,4 +18,3 @@ provider "null" {
 provider "tls" {
   version = "2.1.1"
 }
-

examples/runner-docker/variables.tf

@@ -27,7 +27,7 @@ variable "runner_name" {
 variable "gitlab_url" {
   description = "URL of the gitlab instance to connect to."
   type        = string
-  default     = "https://www.gitlab.com"
+  default     = "https://gitlab.com"
 }
 
 variable "registration_token" {

examples/runner-pre-registered/.terraform-version

@@ -1 +1 @@
-0.12.13
+0.12.18

examples/runner-pre-registered/main.tf

@@ -4,7 +4,7 @@ data "aws_availability_zones" "available" {
 
 module "vpc" {
   source  = "terraform-aws-modules/vpc/aws"
-  version = "2.17"
+  version = "2.21"
 
   name = "vpc-${var.environment}"
   cidr = "10.0.0.0/16"
@@ -46,4 +46,3 @@ module "runner" {
   # working 9 to 5 :)
   runners_off_peak_periods = "[\"* * 0-9,17-23 * * mon-fri *\", \"* * * * * sat,sun *\"]"
 }
-

examples/runner-pre-registered/providers.tf

@@ -1,6 +1,6 @@
 provider "aws" {
   region  = var.aws_region
-  version = "2.34"
+  version = "2.41"
 }
 
 provider "template" {
@@ -18,4 +18,3 @@ provider "null" {
 provider "tls" {
   version = "2.1.1"
 }
-

examples/runner-public/.terraform-version

@@ -1 +1 @@
-0.12.13
+0.12.18

examples/runner-public/main.tf

@@ -4,7 +4,7 @@ data "aws_availability_zones" "available" {
 
 module "vpc" {
   source  = "terraform-aws-modules/vpc/aws"
-  version = "2.17"
+  version = "2.21"
 
   name = "vpc-${var.environment}"
   cidr = "10.1.0.0/16"
@@ -109,9 +109,12 @@ module "runner2" {
 
 resource "null_resource" "cancel_spot_requests" {
   # Cancel active and open spot requests, terminate instances
+  triggers = {
+    environment = var.environment
+  }
 
   provisioner "local-exec" {
-    when    = "destroy"
-    command = "../../ci/bin/cancel-spot-instances.sh ${var.environment}"
+    when    = destroy
+    command = "../../ci/bin/cancel-spot-instances.sh ${self.triggers.environment}"
   }
 }

examples/runner-public/providers.tf

@@ -1,6 +1,6 @@
 provider "aws" {
   region  = var.aws_region
-  version = "2.34"
+  version = "2.41"
 }
 
 provider "template" {
@@ -18,4 +18,3 @@ provider "null" {
 provider "tls" {
   version = "2.1.1"
 }
-

main.tf

@@ -116,12 +116,21 @@ data "template_file" "user_data" {
   template = file("${path.module}/template/user-data.tpl")
 
   vars = {
+    eip                 = var.enable_eip ? data.template_file.eip.rendered : ""
     logging             = var.enable_cloudwatch_logging ? data.template_file.logging.rendered : ""
     gitlab_runner       = data.template_file.gitlab_runner.rendered
     user_data_trace_log = var.enable_runner_user_data_trace_log
   }
 }
 
+data "template_file" "eip" {
+  template = file("${path.module}/template/eip.tpl")
+
+  vars = {
+    eip = join(",", aws_eip.gitlab_runner.*.public_ip)
+  }
+}
+
 data "template_file" "logging" {
   template = file("${path.module}/template/logging.tpl")
 
@@ -324,6 +333,7 @@ resource "aws_launch_configuration" "gitlab_runner_instance" {
       delete_on_termination = lookup(root_block_device.value, "delete_on_termination", true)
       volume_type           = lookup(root_block_device.value, "volume_type", "gp2")
       volume_size           = lookup(root_block_device.value, "volume_size", 8)
+      encrypted             = lookup(root_block_device.value, "encrypted", true)
       iops                  = lookup(root_block_device.value, "iops", null)
     }
   }
@@ -486,6 +496,10 @@ resource "aws_iam_role_policy_attachment" "service_linked_role" {
   policy_arn = aws_iam_policy.service_linked_role[0].arn
 }
 
+resource "aws_eip" "gitlab_runner" {
+  count = var.enable_eip ? 1 : 0
+}
+
 ################################################################################
 ### AWS Systems Manager access to store runner token once registered
 ################################################################################
@@ -513,3 +527,27 @@ resource "aws_iam_role_policy_attachment" "ssm" {
   role       = aws_iam_role.instance.name
   policy_arn = aws_iam_policy.ssm[0].arn
 }
+
+################################################################################
+### AWS assign EIP
+################################################################################
+data "template_file" "eip_policy" {
+  template = file("${path.module}/policies/instance-eip.json")
+}
+
+resource "aws_iam_policy" "eip" {
+  count = var.enable_eip ? 1 : 0
+
+  name        = "${var.environment}-eip"
+  path        = "/"
+  description = "Policy for runner to assign EIP"
+
+  policy = data.template_file.eip_policy.rendered
+}
+
+resource "aws_iam_role_policy_attachment" "eip" {
+  count = var.enable_eip ? 1 : 0
+
+  role       = aws_iam_role.instance.name
+  policy_arn = aws_iam_policy.eip[0].arn
+}

outputs.tf

@@ -42,3 +42,8 @@ output "runner_sg_id" {
   description = "ID of the security group attached to the docker machine runners."
   value       = aws_security_group.docker_machine.id
 }
+
+output "runner_eip" {
+  description = "EIP of the Gitlab Runner"
+  value       = element(concat(aws_eip.gitlab_runner.*.public_ip, [""]), 0)
+}

policies/instance-eip.json

@@ -0,0 +1,13 @@
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Action": [
+                "ec2:AssociateAddress",
+                "ec2:Describe*"
+            ],
+            "Resource": "*"
+        }
+    ]
+}