Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8s: add deployment for EKS, GKS and DOKS #11

Merged
merged 1 commit into from
Aug 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ This repo is using [Opentofu](https://opentofu.org/) and other tools to create c
| | VM | Docker | K8s |
|-----------------------|------------|-----------------|--------------|
| Microsoft Azure | VM ✅ | ACI ✅ | AKS ✅ |
| Amazon Web Services | EC2 ✅ | ECS(Fargate) ✅ | EKS 🚧 |
| Google Cloud Platform | GCE ✅ | Cloud Run ✅ | GKE 🚧 |
| Amazon Web Services | EC2 ✅ | ECS(Fargate) ✅ | EKS |
| Google Cloud Platform | GCE ✅ | Cloud Run ✅ | GKE |
| Render | ❌ | DOCKER ✅(See 1) | ❌ |
| DigitalOcean | Droplets ✅ | AP ⚠️(See 3) | DOKS 🚧 |
| DigitalOcean | Droplets ✅ | AP ⚠️(See 3) | DOKS |
| Native K8s | ❌ | ❌ | K8S ✅(See 2) |

The following icons are used to represent the status of support for each platform:
Expand Down
4 changes: 2 additions & 2 deletions charts/onebusaway/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,14 +77,14 @@ spec:
path: /onebusaway-api-webapp/api/where/current-time.json?key=org.onebusaway.iphone
port: 8080
initialDelaySeconds: 60
periodSeconds: 10
periodSeconds: 30
failureThreshold: 3
readinessProbe:
httpGet:
path: /onebusaway-api-webapp/api/where/current-time.json?key=org.onebusaway.iphone
port: 8080
initialDelaySeconds: 60
periodSeconds: 10
periodSeconds: 30
failureThreshold: 3
volumes:
- name: bundle-volume
Expand Down
109 changes: 109 additions & 0 deletions modules/aws-eks/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

126 changes: 126 additions & 0 deletions modules/aws-eks/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.47.0"
}

random = {
source = "hashicorp/random"
version = "~> 3.6.1"
}

tls = {
source = "hashicorp/tls"
version = "~> 4.0.5"
}

cloudinit = {
source = "hashicorp/cloudinit"
version = "~> 2.3.4"
}
}

required_version = "~> 1.3"
}

provider "aws" {
region = var.region
}

# Filter out local zones, which are not currently supported
# with managed node groups
data "aws_availability_zones" "available" {
filter {
name = "opt-in-status"
values = ["opt-in-not-required"]
}
}

locals {
cluster_name = "education-eks-${random_string.suffix.result}"
}

resource "random_string" "suffix" {
length = 8
special = false
}

module "vpc" {
source = "terraform-aws-modules/vpc/aws"
version = "5.8.1"

name = "eks-vpc"

cidr = "10.0.0.0/16"
azs = slice(data.aws_availability_zones.available.names, 0, 3)

private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
public_subnets = ["10.0.4.0/24", "10.0.5.0/24", "10.0.6.0/24"]

enable_nat_gateway = true
single_nat_gateway = true
enable_dns_hostnames = true

public_subnet_tags = {
"kubernetes.io/role/elb" = 1
}

private_subnet_tags = {
"kubernetes.io/role/internal-elb" = 1
}
}

module "eks" {
source = "terraform-aws-modules/eks/aws"
version = "20.8.5"

cluster_name = local.cluster_name
cluster_version = "1.29"

cluster_endpoint_public_access = true
enable_cluster_creator_admin_permissions = true

cluster_addons = {
aws-ebs-csi-driver = {
service_account_role_arn = module.irsa-ebs-csi.iam_role_arn
}
}

vpc_id = module.vpc.vpc_id
subnet_ids = module.vpc.private_subnets

eks_managed_node_group_defaults = {
ami_type = "AL2_x86_64"

}

eks_managed_node_groups = {
one = {
name = "node-group-1"

instance_types = [var.instance_type]

min_size = var.min_node_count
max_size = var.max_node_count
desired_size = var.node_count
}
}
}


# https://aws.amazon.com/blogs/containers/amazon-ebs-csi-driver-is-now-generally-available-in-amazon-eks-add-ons/
data "aws_iam_policy" "ebs_csi_policy" {
arn = "arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy"
}

module "irsa-ebs-csi" {
source = "terraform-aws-modules/iam/aws//modules/iam-assumable-role-with-oidc"
version = "5.39.0"

create_role = true
role_name = "AmazonEKSTFEBSCSIRole-${module.eks.cluster_name}"
provider_url = module.eks.oidc_provider
role_policy_arns = [data.aws_iam_policy.ebs_csi_policy.arn]
oidc_fully_qualified_subjects = ["system:serviceaccount:kube-system:ebs-csi-controller-sa"]
}
19 changes: 19 additions & 0 deletions modules/aws-eks/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
output "cluster_endpoint" {
description = "Endpoint for EKS control plane"
value = module.eks.cluster_endpoint
}

output "cluster_security_group_id" {
description = "Security group ids attached to the cluster control plane"
value = module.eks.cluster_security_group_id
}

output "region" {
description = "AWS region"
value = var.region
}

output "cluster_name" {
description = "Kubernetes Cluster Name"
value = module.eks.cluster_name
}
81 changes: 81 additions & 0 deletions modules/aws-eks/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Deployment Guide for Amazon Elastic Kubernetes Service (EKS)

Amazon Elastic Kubernetes Service (EKS) is a managed Kubernetes service provided by AWS, offering a scalable and secure way to run Kubernetes clusters. This guide will show you how to deploy the OneBusAway server on EKS.

## Prerequisites

1. An AWS account. If you don't have one, you can create a free account [here](https://aws.amazon.com/free/).
2. AWS CLI, which includes `kubectl` configured to work with EKS. You can install it by following the instructions [here](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html).
3. Opentofu, an open-source Terraform alternative. You can install it by following the instructions [here](https://opentofu.org/docs/intro/install/).
4. Ensure that all prerequisites are installed before starting the deployment.

## Steps

1. Clone this repository to your local machine. You can run:
```bash
git clone
```

2. Change the directory to `modules/eks`, using the command:
```bash
cd onebusaway-deployment/modules/aws-eks
```

3. Configure your AWS CLI with your credentials, you can follow the instructions [here](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html):

4. Initialize the project. This will download the necessary plugins and providers for the project:
```bash
tofu init
```

5. Configure the cluster by copying the example variables file and modifying it:
```bash
cp terraform.tfvars.example terraform.tfvars
```
Then modify the `terraform.tfvars` file to configure the cluster. You can find the meaning of each parameter in the `variables.tf` file.

6. Deploy the project:
```bash
tofu apply
```

7. Configure `kubectl` to connect to the EKS cluster:
```bash
aws eks --region $(tofu output -raw region) update-kubeconfig \
--name $(tofu output -raw cluster_name)

# Check context
kubectl config get-contexts
```

8. Install Ingress Nginx:

Here is the [recommended way](https://kubernetes.github.io/ingress-nginx/deploy/#aws) to install Ingress Nginx on EKS:
```bash
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.11.2/deploy/static/provider/aws/deploy.yaml
```

9. Configure your domain:

You should add a DNS record to your domain to point to the address of the Ingress Nginx controller. EKS use an ELB to expose the Ingress Nginx controller. You can find the address by running:
```bash
kubectl get svc -n ingress-nginx
```
If you are using Route 53, you can follow the instructions [here](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/routing-to-elb-load-balancer.html).

10. Deploy OneBusAway server:

Modify the `onebusaway/values.yaml` file to configure the application. You can find the parameter meanings in [onebusaway-docker](https://github.com/OneBusAway/onebusaway-docker/#deployment-parameters).
You can use this command to deploy the application:
```bash
helm install onebusaway ../../charts/onebusaway
```
Then you can check the status of the deployment by running:
```bash
kubectl get pods -n oba
```

11. Access the OneBusAway server:

You can access the OneBusAway server by visiting the domain you configured in step 9.

5 changes: 5 additions & 0 deletions modules/aws-eks/terraform.tfvars.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
region = "us-east-1"
instance_type = "t3.medium"
min_node_count = 1
max_node_count = 3
node_count = 2
Loading
Loading