Dispatch305 Flask Backend API

Table of Contents

1. Description
2. Install (Run) with Docker
3. Installation without Docker
4. Run with the React js Frontend (with and without Docker)
5. Structure and apps
6. Deploying in VPS
7. Screenshots of the Admin Panel
8. Screenshots of the Frontend React App
9. Useful Links

Description

Dispatch305 is a service that helps dispatchers to communicate with both drivers and brokers. From now on, we call DISPATCH305 to the company and all its components, and so we will often call drivers to the clients, while the dispatchers will be called staff members (of the company). In addition, the services offered by the website will be referred as:

• Searching a Cargo: When a client requests the dispatcher to look out for an Agency with a Cargo to transport. In this case the staff member should contact a broker (Agency) and make the arraignments.

• Sending Analytics: Every Friday, a bill should be sent to every client with the weekly pending bill amount and a description of the charges. In addition, for the clients with a VIP account, an analytics resume of the week should also be sent.

• Managing POD: When a Cargo is delivered, the VIP clients may request the staff member assigned to them to manage the sending of the Rate Conf and the POD (this are 2 PDF files needed as proof of service and delivery).

• How does Dispatch305 work?

1. User Registration: First, the client registers at dispatch305.com (the Frontend website created using REACT js). During this step, the client should provide basic account information such as name, company, ..., and should also upload the 4 basic PDF files that most brokers require for hiring them to deliver cargo.
2. Account Activation: When the new account is created, it is set to inactive and the Admin User is notified. The, the Admin User assigns a staff member to the client. From this point on, the staff member is said to be the client's dispatcher. The client must download the PDF that is under the section Agreement in the Frontend app. Once this steps are completed, the client's account is set to active.
3. Requesting Services and Billing: Now and while the account is active, a client with a plan BASICO (basic account) can request the dispatcher to offer the service of Searching a Cargo, while VIP clients can ask for the services of Searching a Cargo, Sending Analytics, and Managing POD. Regardless of which type of account a client has, a Bill will be sent to them with the pending amount to pay for the services offered in that week from DISPATCH305. When a user fails to pay the weekly bill (usually a timeline of 3 days offered), the account becomes inactive until the bill is paid.
4. Factoring: After the user requests a Searching a Cargo service, and the Cargo is found by the staff member, and later delivered by the client, if the client has a VIP account, the staff member can be requested to offer a Managing POD service. In order to do that, the client must send the PDF files required, and the staff member should upload them to DISPATCH305's Admin Panel.

Install (Run) with Docker

1. Clone the repo:

   git clone https://github.com/Ceci-Aguilera/dispatch305_flask_backend_api.git

2. Install Docker and Docker Compose

3. Configure the environment variables using one of the following methods:

   i. Create an .env file inside the config folder and set up the following environment variables:

    SECRET_KEY                      (for example "someSecurityPassword")
    JWT_SECRET_KEY                  (use secrets.token_hex(12) from python secrets)
    SECURITY_PASSWORD_SALT          (for example "someSecurityPassword")
    ADMIN_EMAIL_CREDENTIAL          (email to use to create a Admin user)
    ADMIN_PASSWORD_CREDENTIAL       (the password for the Admin user)
    MAIL_SERVER                     (the server for sending emails using Flask-Mail)
    MAIL_PORT
    MAIL_PASSWORD
    MAIL_STRING_ID                  (a short random string to pass when using office 365)
    FRONTEND_APP                    (an allowed origin)
   

   or

   ii. Copy and modify the content of the .example.env file to the .env file:

   cp config/.example.env config/.env

4. Run the command:

   docker-compose up -d --build

5. Congratulations =) !!! the app should be running in localhost:5000

Installation without Docker

1. Clone the repo:

   git clone https://github.com/Ceci-Aguilera/dispatch305_flask_backend_api.git

2. Create a virtual env and Pip install dependencies:

   pip install -r requirements.txt

3. Open the app.py file and change the parameter of create_app to 'development' (by default it is set to production)

4. Set up postgresql database (See Useful Links). For development the default credentials are:

       Database name: test_db
       Database user: test_user
       Database password: test_pass
       Database host: localhost
       Database port: 5432
   

5. Config the environment variables using one of the following methods:

   i. Create an .env file inside the config folder and set up the following environment variables:

    SECRET_KEY                      (for example "someSecurityPassword")
    JWT_SECRET_KEY                  (use secrets.token_hex(12) from python secrets)
    SECURITY_PASSWORD_SALT          (for example "someSecurityPassword")
    ADMIN_EMAIL_CREDENTIAL          (email to use to create a Admin user)
    ADMIN_PASSWORD_CREDENTIAL       (the password for the Admin user)
    FRONTEND_APP                    (An allowed origin)
    MAIL_SERVER                     (the server for sending emails using Flask-Mail)
    MAIL_PORT
    MAIL_PASSWORD
    MAIL_STRING_ID                  (a short random string to pass when using office 365)
    FRONTEND_APP                    (an allowed origin)
   

   or

   ii. Copy and modify the content of the .example.env file to the .env file:

   cp config/.example.env config/.env

6. Run the migrations

   flask db init
   flask db migrate
   flask db upgrade

   NOTE: In case of an error regarding revision of migration, run:

   flask db revision --rev-id <revision_id_in_error>
   flask db migrate
   flask db upgrade

7. Run the app

   python app.py

   NOTE: To change the initial configuration edit the files app.py and config/config.py files. The env variables for the config/config.py files are retrieved from config/.env using decouple.config.

8. Congratulations =) !!! the app should be running in localhost:5050

Run with the React js Frontend (with and without Docker)

Note: Before following these steps clone this repository. From now on the selected folder that contains the clone will be referred as project_root. So far, it should look like this:

   project_root
   └── dispatch305_flask_backend_api

1. Assuming that your are at the project_root, clone the React Frontend repository:

      git clone https://github.com/Ceci-Aguilera/dispatch305_react_frontend.git

   Now the project_root folder should look like:

   project_root
   ├── dispatch305_flask_backend_api
   └── dispatch305_react_frontend

• If Using Docker and Docker Compose

  1. Copy the content of the docker-compose-connect.yml to a new file docker-compose.yml in the project_root. The docker-compose-connect.yml file can be found at the root of this repository and also at the root of the React Frontend repository (Either file is fine to copy).

  2. Follow the instruction to configure the environment variables of the Flask backend API that can be found in the section Install (Run) with Docker in the Readme.md of the React Frontend repository. The only env variable needed is the Flask Backend url, which by default should be http://localhost:5000.

  3. Follow the instructions on the Install (Run) with Docker section of this Readme.md to configure the environment variables for this repo. Note: Right now the project_root should look like:

     project_root
     ├── dispatch305_flask_backend_api
     ├── dispatch305_react_frontend
     └── docker-compose.yml

  4. Run the command:

     docker-compose up --build

  5. Congratulations =) !!! the frontend app should be running in localhost:80 while the backend is at localhost:5000

• Running without Docker and Docker Compose

  1. Follow the instructions of the Installation without Docker section in the Readme.md of the React Frontend repository to configure and run the frontend. Modify the REACT_APP_API_DOMAIN_NAME to be the url of the Flask Backend API (by default it is http://localhost:5050.
  2. Follow the instructions of section Installation without Docker of this Readme.md.
  3. Congratulations =) !!! the frontend app should be running in localhost:3000 while the backend is at localhost:5050

---

Structure and Apps

Folders and Endpoints

Dispatch305 Backend-Api is divided in the following folders:

.
├── api
│   ├── admin
│   ├── message
│   ├── static
│   │   ├── assets
│   │   └── css
│   ├── templates
│   │   ├── admin
│   │   ├── driver
│   │   ├── security
│   │   ├── staff
│   │   └── trucks_cargos
│   ├── trucks_cargo
│   └── user_account
├── config
├── migrations
└── uploads

• At the root is located a file named app.py which together with the config/config.py file determine the behavior of the Flask app, and so the .env file is located inside the config folder. Most of the weight of the configuration is actually in the init.py file inside the api folder. This init.py file initializes most of the different pieces of the app such as the Database, Flask-Secure, Flask-Admin, .... Thus, in case of wanting to change what services are included this file will probably be the one to be altered.
• The migrations folder contains all files auto-generated by Flask-SQLAlchemy.
• The uploads folder has all the content uploaded from the clients, which is later divided into 2 type of sub-folders: trucks-cargos which contains the Rate Confirmation.pdf and the POD.pdf files for each Cargo registered through the TrucksCargo model, and the {user-email} folders that are dynamically generated when a client registers and they are meant to store the 4 principal pdf files for the drivers setup with a new broker.
• Finally, the api folder contains the main apps/folders of the project, each of them has a models.py file and a views.py file. In addition, it has the static and template folders that manage static files and html (jinja2) templates respectively.
• The api/admin folder:
  • models.py: manages the UserAdmin model (in this case, the AdminUser model is different from the User model) and the Role model that are required by Flask-Security .
  • The views.py file that is in the same folder has all the views that an AdminUser with role admin or with the role staff can see. In this case, not all views are available to both roles, as the admin user is allowed to view/edit all models, while staff users (dispatchers are the only AdminUser with staff role) can only see their clients (which are instances of the User model, not the AdminUser model) that are active, and the TrucksCargo instances created by them (by the dispatchers). However, Brokers are available to be viewed/created by any dispatcher. Therefore, the only models available to the dispatchers are Users, TrucksCargo, and Brokers (with the limitations listed above)
• The api/message folder has the Message model that manages the comments submitted by the clients that visit the website (not necessary users of Dispatch305) _ The api/user_account folder manages the User (client) model and the api endpoints that are called by the React Frontend
• The api/trucks_cargo folder has the TrucksCargo model which is the Cargo that the dispatcher must hire, and the Broker model which is the company that offers the Cargo. The views that are from the view.py file are registered, however never used

Deploy to VPS using PostgreSQL, Nginx, and Gunicorn

1. Clone the repo:

   git clone https://github.com/Ceci-Aguilera/dispatch305_flask_backend_api.git

2. Install the dependencies:

   sudo apt-get update
   sudo apt-get install python3-pip python3-dev libpq-dev postgresql postgresql-contrib nginx

3. Set up the postgresql database (See Useful Links)

4. Create an .env file and configure the environment variables

5. Create a virtual env and activate it:

   virtualenv myprojectenv
   source myprojectenv/bin/activate

6. Pip install the requirements:

   pip install -r requirements.txt

7. Pip install gunicorn:

   pip install gunicorn

8. Open app.py and add host='0.0.0.0' to the create_app() function

9. Delete the migrations folder in case it exits, and create an empty folder with the same name

10. Test configuration so far:

    flask db init
    flask db migrate
    flask db upgrade
    
    python app.py

11. Change the frontend domain for reset password in the api/templates/user-account/send-request-reset-password.html file (currently http://localhost:3000)

12. Create wsgi.py file:

    sudo vim wsgi.py

    and copy and paste this:

    from app import app
     
    if __name__ == "__main__":
    app.run()

   and then run

gunicorn --bind 0.0.0.0:5050 wsgi:app

12. Complete the setup of the website with this link

13. Set up Cors to allow the frontend to fetch and post requests (See Useful Links)

• Connecting to the React frontend

1. Configure Cors to allow React js axios calls (See Link). If running in development mode and with the React app running in localhost:3000, then no config is needed in Flask
2. Follow the instructions of the REACT Frontend repo: Dipatch305 Frontend to run the React js app

Screenshots of the Admin Panel

---

---

Screenshots of the Frontend React App

Useful Links

Database (PostgreSQL and SQLAlchemy)

• Set up postgreSQL in Ubuntu (install + create database, user, and alter roles): Link from DigitalOcean about deploying Django + Postgresql
• Create models and connecting them to the db using SQLAlchemy: Link to Flask-SQLAlchemy official documentation
  • Managing Foreign Keys: One to One, One to Many, and Many to One: Link to Flask-SQLAlchemy

Authentication and Admin Panel

• Flask with JWT Authentication (For calls coming from the REACT frontend): Link to Flask-JWT-Extended official documentation
• Flask Security for the Admin Panel (Using Flask-Security-Too instead of Flask Security as it is deprecated): Link to Flask-Security-Too official documentation
  • Useful Flask-Security-Too links from Github and Stackoverflow:

    • FS Uniqufiers is mandatory for User model, and get_user function eliminated from version 4.0

    • Custom Login html and overriding other templates

    • Flask-Security + Flask Admin when Authenticating user Resource 1

    • Flask-Security + Flask Admin when Authenticating user Resource 2

    • Flask Admin Panel:

      • Link to Flask-Admin official documentation
      • Link to Flask-Admin github
      • Link to Custom Formatter in Stackoverflow to implement a custom field

Rest Api using Restx

• Link to Flask Restx official documentation
• Link to Flask Restx github

Managing PDFs and other files

• Save pdf and other file types from React Frontend
• Send pdf file to Frontend
• Render PDF file in Browser using Flask

Docker and Docker Compose with Flask + Postgresql

• Dockerize Flask app with Postgresql, Guinicorn and Nginx
• Python Slim Buster error with gcc
• Using sh file in docker to init flask and run migrations: Fix slim-buster with netcat, gcc, and g++
• Why not using volumes in docker-compose for flask files: Fix migrations folder is created and not empty error

Sending Emails with Flask-Mail

• Flask-Mail official documentation
• Flask-Mail with office 365 services issue with ID Stackoverflow Fix

Cors Headers Configuration

• Example of simple Cors config for React js Frontend app