DevOps Project

Herein, the main project's objective is to make a web application go through the main CI/CD steps, to be deployed using (i) a virtual environment with vagrant, (ii) docker and docker compose and (iii) Kubernetes. A list of completed tasks can be seen below:

• Complete the application code
  • Start a web server
  • Create a user
  • Get user data
  • Testing (One test missing in test/user.router.js)
• Apply CI/CD
  • Testing on Travis CI
  • Deployment on Heroku
• Configure and provision a virtual environment and run your application using IaC approach
  • Use vagrant
  • Provision the VM with Ansible to install and run (language runtime, database, git, ssh forwarding for private repository, app)
  • Provision Ansible playbook for Healthchecks
• Build Docker image of your application
  • Create Docker image of your application
  • Push the image to Docker Hub.
• Make docker orchestration using Kubernetes
  • Install Kubernetes cluster using Minikube
  • Create a Kubernetes Manifest yaml files
• Make a service mesh using Istio
• Describe your project in the README.md file

User API web application

It is a basic NodeJS web application exposing REST API that creates and stores user parameters in Redis database.

Functionality

1. Start a web server
2. Create a user
3. Get user data

Installation and usage

This application is written on NodeJS and it uses Redis database. There are different scenarios to install and run the application. See below for their descriptions.

Local installation and usage

Installation

First check that both NodeJS and Redis are installed in your system. If not follow these two links and their respective installation guides:

• Install NodeJS

• Install Redis

Once done you can clone or download the repository. Once in the root directory of the application (where package.json file is located) run the following command:

npm install

Usage

1. Start a web server

From the root directory of the project run:

npm start

It will start a web server available in your browser at http://localhost:3000.

1. Create a user

Send a POST (REST protocol) request using terminal:

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"username":"fbraza","firstname":"faouzi","lastname":"braza"}' \
  http://localhost:3000/user

It will output:

{"status":"success","msg":"OK"}

2. Get user's data

curl http://localhost:3000/user/fbraza

It will output:

{"status":"success","msg":{"firstname":"Faouzi","lastname":"Braza"}}

Testing

From the root directory of the project, run:

npm test

Installation and usage in a virtual machine

Installation

First check that Virtual Box and Vagrant are installed in your system:

• Install Virtual Box
• install Vagrant

Once done, open a terminal inside the iac folder, and run the following command:

vagrant up

This will fire-up a virtual machine with a CentOS/7 installation. Additionally this will install all required packages, software and run necessary services (notably Redis database). At the end of the process you should have something a bit similar to this:

Usage

To enter the VM use, stay inside the iac folder and run:

vagrant ssh

You can then move to the cloned repository using the following command:

cd private-repo

Once inside the folder you can basically do what you would do locally:

# First install node packages
npm install

# run the webserver
npm start

Once the web server started you can access the app in your browser using the following link http://localhost:3000/ . The difficulty here is that when started the terminal is occupied. So if you want to curl you app just open a second terminal and ssh again inside the VM. From there you can create and get users' data.

Testing

In your VM move to the private-repo folder and run:

npm test

Docker installation and usage

Installation

First check that docker and docker-compose are installed in your system if not follow these links:

• Install docker
• Install docker-compose

docker-compose needs to be installed separately only for Linux user. It is included as part of Mac and Windows desktop installs

Once done you can clone or download the repository. Once in the root directory of the application (where package.json file located) run the following command:

COMPOSE_DOCKER_CLI_BUILD=1 docker-compose build

This will pull two images from the docker Hub (Redis, our application), build them, expose the relevant ports, mount the storage volumes.

COMPOSE_DOCKER_CLI_BUILD=1 force the use of the Buildkit to build the containers faster. It also takes into account the dockerignore file by default. This command is available from docker-compose 1.25.1. If you have older version use the classical docker-compose build which may not take into account the dockerignore file.

Next you can run the following command:

docker-compose up -d

This will make the two containers running in background.

Usage

After installation, a web server will be available in your browser at http://localhost:3000. You can also use the previous commands described previously to create users and get user's data.

Testing

The test suite has not been included in the docker image. All tests have been performed locally and then run on Travis CI before containerizing the app.

Installation & deployment with Kubernetes

Installation

To be able to install and deploy the app on a local Kubernetes cluster first check that kubectl (Kubernetes CLI) and minikube (CLI to set up local Kubernetes cluster) are installed in your system. I chose to run minikube with the docker driver (so check that docker is installed if you follow the same path) but if you prefer you can use others container / virtual machine managers.

• Install kubectl
• Install minikube
• Install docker

Once all required softwares are installed go to the root directory and run:

minikube start

This will build and start you local Kubernetes cluster. Then run:

kubectl apply -f k8s_manifests_distinct

This will set up two distinct pods in our cluster. One pod will run our application container and the other one will run the Redis container. A persistent storage will be attributed to the Redis container to save data.

Usage

To access the web server, create and get user data you first need to get the IP address of the running cluster. Because you are going to use it several time in your curl command, you can save it in a bash variable by running the following command:

CLUSTER_IP=$(minikube ip)

To expose the container in our pods we created Kubernetes services manifests and open some NodePort. To get the node port of our app we can run the following command:

kubectl get svc

This should output something similar to:

NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
faouzi-webapp-service   NodePort    10.102.58.222   <none>        3000:30867/TCP   5s
kubernetes              ClusterIP   10.96.0.1       <none>        443/TCP          2d9h
redis-service           NodePort    10.98.127.167   <none>        6379:31048/TCP   5s

Looking at the service named faouzi-webapp-service we can quickly find the node port which is 30867. Save it to a bash variable

PORT=30867

Then to interact with the web server you can run:

curl http://${CLUSTER_IP}:${PORT}

This should return:

Hello World!

You can then use the commands described previously to create users and get users' data but be careful to put the right ip and port. Using the variables initialized previously should simplify the process:

1. To create user

curl --header "Content-Type: application/json" \
  --request POST \
  --data '{"username":"fbraza","firstname":"faouzi","lastname":"braza"}' \
  http://${CLUSTER_IP}:${PORT}/user

2. To get user's data

curl http://${CLUSTER_IP}:${PORT}/user/fbraza

Author

Faouzi Braza

faouzi.braza@edu.dsti.institute