DOCID Integration Backend

Table of Contents

• Project Overview
  • Structure Overview
  • Software
  • QuickStart
  • Linting
  • Testing
  • REST API

Project Overview

Considering the fast and growing need for scholarly data infrastructures in Africa, and the diverse economic development in the continent. The reality is that some universities can afford DOIs but not for all scholarly and research output, while the bulk of research content is in gray literature in Africa. Adding to that the unicity of what the APA RA would like to specialize in, which is indigenous knowledge and cultural heritage, and patent digital object containers.

Structure Overview

This project is split into two parts – the back end (written in Python & Flask), and the front-end (written in JavaScript & React) in a separate repository.

The general folder structure for the project can be seen below:

├── docid_app              # Project folder with project files
├── .dockerignore          # Includes files/folders to be excluded from image building
├── .editorconfig          # Configuration defining consistent coding styles and formatting
├── .env.sample          # Includes environment variables for the local environment
├── .gitignore             # Includes files/folders to be excluded from git commits
├── Dockerfile.dev         # Instructions specifying how to build docker images.
├── LICENSE                # License for this project
├── Makefile               # List of utility commands to help developers
├── Pipfile                # Configuration for virtual environments, and dependencies for Python
├── Pipfile.lock           # Includes versions of the dependencies and transitive dependencies
├── README.adoc            # Readme file for this project
├── docker-compose.dev.yml # Configuration defining multiple Docker containers used in this app
├── pytest.ini             # Configuration file for unit testing using pytest framework

To perform some operations, a Makefile with a set of make commands is provided. In order to run these commands, the GNU make utility needs to be installed. Some of the commands are listed below.

System Prerequisites

This application leverages microservices approach where each service runs in its own container. The services include:

• PostgreSQL - The database that facilitates storage, retrieval and updating of information.

• Python/Flask - This is the backend service for the application which provides a rich application programming interface (API) endpoints through which the frontend app communicates.

• Nginx - A highly scalable server that serves the front end and backend applications via a proxy server. It is suited for both test and production deployment environments.

You need docker installed to deploy this app. Please refer to the docker documentation for installation details.

Tip	Please refer to Docker installation docs and Docker Compose installation docs to install them.

Software

This back-end project consists of Python/Flask, Nginx and PostgreSQL services. There is a list of Python third-party libraries listed in Pipfile. The initial requirements include:

• Flask a lightweight and flexible web framework for Python. It is designed to make it easy to build web applications and APIs quickly and with minimal boilerplate code.

• python-dotenv for management of environment variables

• requests for handling of HTTP requests

• flask-sqlalchemy for adding SQLAlchemy funtionality such as Object Relational Mapper (ORM) capabilities to the application

• flask-migrate for handling database migrations

• flask-cors for handling Cross Origin Resource Sharing (CORS) requests.

• psycopg2-binary to provide PostgreSQL database connectivity and integration

• gunicorn for serving the application via HTTP and WSGI protocols

• black for code formatting

• coverage measures code coverage, typically during test execution.

• pytest for testing, from simple to complex tests on application code.

QuickStart

As a first mandatory step, rename the .env_sample to .env and add the necessary values to each environment variable.

To build and setup the application from the ground up, ignoring previously build cached layers, just type:

make build-with-no-cache

You can also build with the usual build command by typing:

make build

This will create the necessary Docker containers and install the required Python packages.

To start the services while seeing the log, type:

make start-services

Alternatively, to start the services in the background (detached mode) where you will not see the logs in the terminal, type:

make start-services-detached

Once the services have started, Flask should be running Postgres, and Nginx in production. You can change these values in the docker-compose.dev file.

To create database tables, run:

make create-db

To populate tables and lookups, run:

make insert-data

To generate and populate PID lookup table:

make populate-pids

To stop the services, type:

make stop-services

Alternatively, to stop services and remove volumes that contain persisted data, type:

make stop-delete-volumes

To print out the logs, type:

make show-logs

Alternatively, to print out logs interactively showing what is happening in the services, type:

make show-interactive-logs

The other commands include:

• make python-shell for entering into the flask app container bash shell.

• make db-shell for entering into database container bash shell

Linting

Black is used to automatically format and lint files. The make format-python command checks that the project is correctly formatted.

Testing

Work-in-progress

REST API

Work-in-progress

API Documentation

Work-in-progress

Endpoints

The endpoints provided by the app include:

• Root - / this can be utilized to confirm that the app is running. It should display a JSON output which is a welcome message

• Login - auth/login/ which provides login

• Logout - auth/logout/ which provides logout

• DataCite DOI - doi/get-datacite-doi/ which provides generation of a Datacite DOI from the DataCite API (credentials required)

• DOCID DOI - doi/get-ddocid-doi/ which provides DocID DOI from prepopulated data