atd-postgrest

This repository contains configuration files for orchestrating Austin Transportation's PostgREST services.

Table of contents

• atd-postgrest
  • Table of contents
  • Design
  • Services
  • Configuration
    • Environment variables
    • Docker
    • HAProxy
    • haproxy/config/haproxy.cfg
    • haproxy/maps/routes.map
  • Get it running
  • Deployment
  • Maintenance

Design

ATD relies on multiple postgREST services. These services are fronted by a single HAProxy load balancer, which functions as a reverse proxy to route requests to each postgREST instance.

The root endpoint is available at http://atd-postgrest.austinmobility.io/.

Services

name	repo	route
knack services	atd-knack-services	/knack-services/
legacy scripts	atd-data-deploy	/legacy-scripts/
parking	atd-parking-data	/parking/
CTR data lake	atd-data-lake	/ctr-data-lake/
road conditions	atd-road-conditions	/road-conditions/
bond reporting	atd-bond-reporting	/bond-reporting/

Configuration

Environment variables

Because each of the services's databases are hosted in the same RDS cluster, the following environment variables are applied to all postgREST services.

• PG_HOST: the postgres server host name
• PG_USER: the postgres user name
• PG_PASSWORD: the postgres password
• PGREST_MAX_ROWS: the maximum rows to be returned by a postgREST request

As well, each postgREST service requires a unique (32-char minimum) JWT secret. The var can be named as you wish, and must be referenced in docker-compose.yaml (look there for example).

Docker

Each postgREST service must be defined in docker-compose.yaml. The service name defined in the compose yaml wil be referenced as a host in the HA proxy config.

HAProxy

A dockerized HAProxy service reverse-proxy routing to the available postgREST instances.

happroxy/config/haproxy.cfg

This file defines the HAProxy configuration. Each "backend" postgREST service must be fined here. See the comments in that file for details.

happroxy/maps/routes.map

This filed defines which how an inbound HTTP request's path will be mapped to the various backend postgREST services. See the comments in that file for details.

Get it running

1. Modify docker-compose.yaml, haproxy.cfg, and routes.map as needed.
2. Create an environment file in the root directory. Name it env.
3. Start the services: $ docker-compose --env-file env up -d

Deployment

Docker is configured on the production server to restart on boot.

The script /scripts/docker-keepalive.sh checks if the HAProxy service is running, and restarts all containers if not. This script is deployed to the prod server's crontab to run every hour.

You can inspect the crontab with sudo crontab -l.

Maintenance

Any changes to this repository must be manually pulled on the prod server.

If you make changes to the schema, permissions, or secrets of any of the running postgREST services, you will need to restart the docker-compose service.

$ docker-compose --env-file env restart