Setup DB and Add Pagination to Conform to OGC Specs

[ar-siddiqui]

This PR makes changes to the app that were required to comply with the pagination requirements of OGC specs: '/req/job-list/limit-definitionand/req/core/pl-limit-definition`. The solution implemented also overcame the shortcoming of the cache (as discussed in the paper).

The following enhancements have been implemented:

• /jobs and processes routes now support offset and limit parameters and include links to previous and next pages. This was a requirement of OGC Specs.
  

• Add GitHub Actions like symbols to job statuses
  

• Update HTML pages to be sync with recent code changes

• Update tests to reduce testing time from ~5 min to ~4 min

• Make echo logs colorful to easily identify requests that were failed or bad
  

• Use of SQLite database in place of snapshots to persist job records.

• Get rid of the JobsCache hashmap and introduce the ActiveJobs hashmap. ActiveJobs keep a record of currently active jobs. As soon as a job reaches a terminated state (SUCCESSFUL, FAILED, DISMISSED). The job is being removed from the ActiveJobs and status, logs, etc are updated in the database.

• Introduce a hit and miss strategy in handlers. A job id is first checked in ActiveJobs, if it is there (hit) then return a response immediately without checking the database, else if job is not in ActiveJobs (miss) then look in the database. This is similar to the cache hit-and-miss concept.

System design before and after:

Benefits achieved with this PR:

• Use of the database allows easy pagination, before we were using hashmap to store jobs which has no particular order.

• SQLite is a file-based database and will be in sync with the system all the time. Hence it solves the issue of losing data if the server crashes in between snapshots without adding the complexity of standing up a new database server.

• SQLite file can be viewed and updated by software such as DBeaver previously the only way to look at jobs was through API, providing no control over raw data.

• JobsCache was never a cache, to begin with, it was serving as a data store.

• No need to calculate and track the size of jobs in memory as only active jobs are stored in memory. The remaining jobs are in the database (disk).

Fix #37 Fix #27 Fix #18

[mxkpp]

I see there is mutex lock/unlock going on and have some questions:

1. Does the current Mutex behavior protect the sqlite file or only the in-memory job lists?
2. Are deadlocks possible with the current Mutexes?
3. Are deadlocks possible with the current sqlite interactions?
4. Should we add timeouts?

[ar-siddiqui]

I see there is mutex lock/unlock going on and have some questions:

1. Does the current Mutex behavior protect the sqlite file or only the in-memory job lists?
2. Are deadlocks possible with the current Mutexes?
3. Are deadlocks possible with the current sqlite interactions?
4. Should we add timeouts?

@mxkpp I will make this an issue to research later.

[mxkpp]

I see there is mutex lock/unlock going on and have some questions:

1. Does the current Mutex behavior protect the sqlite file or only the in-memory job lists?
2. Are deadlocks possible with the current Mutexes?
3. Are deadlocks possible with the current sqlite interactions?
4. Should we add timeouts?

@mxkpp I will make this an issue to research later.

@ar-siddiqui was this resolved, or an issue opened to track?

EDIT: nvm I just made this one, please lmk if it is duplicated.
#86