Calling for all soapers! Do not struggle with your soap recipe anymore! Just refresh the page and it will generate a random soap recipe for you. How to use it? simple, just refresh the page.
This time, I had a chance to try something new. The project subject was to create 3 APIs which generate random data and render it on the frontend. I'm a soaper and produce homemade soap from time to time, and I thought it would be nice to have a recipe generator. This application works very simply, Just hit refresh and it will generate a new recipe each time. I have build 3 APIs with Python, then rendered the frontend with Python, Flask, and Jinja. There are 4 VM instances used for Jenkins, Load balancer(Nginx), Docker swarm manager, and worker.
- Trello Board
- Google Cloud Platform for VM instances
- SQL Database on GCP
- Python and Flask programming + Jinja2 for HTML rendering
- Unit testing with Pytest
- Git for version control
- Nginx for the Load balancer
- Jenkins server (pipeline)
- Ansible
- Docker Compose and Swarm
- ERD
- Risk Assessment
The frontend page is handling API calls in between all 3 backend services and also stores the generated data to MySQL DB. I've set up Docker swarm manager, worker and Nginx instance for load balancer where user can access the application through the reverse proxy.
This project required lots of tools and work. I was swamped with the amount of the stuff I had to manage and came up with a Trello board to organize the tasks.
This is my Trello board at the end of the project.
Below is an entity diagram design for this project.
As a nature of the project, I only needed a simple DB structure where I could store 4 fields of data.
I have used GCP SQL manager this time rather than setting one on my own on a VM instance.
I have used 4 instances for this project, 'project' instance for Docker swarm manager node, 'swarm-worker' for swarm worker node, 'nginx' for load balancer instance and 'jenkins' for CI server pipeline and Ansible.
After building up the APIs and connect my application to the SQL database, I ran the unit test with Pytest as my code was written in Python. The purpose of the test was to ensure API applications are working without an error.
3 out of 4 applications had 100% of the pass result, however, Service1 (frontend) hadn't passed the test as it had a DB connection issue which is one of the ongoing issues.
For service2 & service3, I have tested APIs are giving the correct data listed. and for service4, I have tested whether it was giving the corresponding data depending on service2's outcome. I have tested several times to ensure all 3 services for the test consistency.
For load balancer, I have set an instance with Nginx configuration. Any users accessing the web page will reach this server first, and Nginx will split the traffic and distribute it to either Swarm manager or Swarm worker.
Ansible was used for this project to join Swarm manager and Swarm worker.
This is how it looks when Ansible is running on the Jenkins instance manually.
I have included Ansible on Jenkins pipeline and when it's successfully connected you will see this outcome.
There is 4 stage on Jenkins Pipeline to test, set Ansible, build and deploy. On the Test stage, Jenkins will test 3 backend applications and when it passes, it will set trigger Ansible. Ansible will join the Swarm manager instance and Swarm worker instance. Build stage was added to simply build docker images and push them to Dockerhub as Docker Swarm will only work when images are pre-build and on Dockerhub. For the last, It's now ready to deploy the application. Firstly it will copy the docker-compose.yaml file to the Swarm manager instance and SSH into the manager node. then it will stack deploy the application meaning it's now up and running on both Swarm manager and Swarm worker node.
In this project, I have automated Test - Build - Deploy using Ansible, Docker, Jenkins. I usually work on the code on VScode on my local machine. Each time code is pushed to my GitHub, webhook will trigger the build on the Jenkins CI server. Once the build is triggered, it will automatically test the code, Join Swarm manager and Swarm worker using Ansible, build Docker images and push them to Dockerhub, and will stack deploy it.
| Risk Description | Assessment | Impact | Action to take |
|---|---|---|---|
| Unit testing failure | Service1(frontend) unit testing failed could lead to unstable service | Medium | Debug unit testing |
| Security issue | DB password is open | High | Implement environment variables |
| SQL database failure | In case of DB failure, all the data stored would be wiped | Low | Create a backup DB |
| GCP instance failure | In case of instance failure, the entire application wouldn't work as SQL and Jenkins are running on the instance | High | Create a backup instance |
- Implement environment variables
- Unit testing
Bora Kim