Schema Genie Stack

Recommended Tools

• Docker
• Docker Compose
• PgAdmin4 - Optional
  • Install via Homebrew

Overview

This contains the components required to run Schema Genie.

The Schema Genie stack consists of a docker-compose.yml file which specifies a list of docker images to be run as containers.

1. Backend Service
   • The Backend service is a docker image built from the schema-genie-backend repository.
   • This is the main schema-genie service which relies on the other services to perform data processing or persistence.
2. XML-to-XSD Service
   • The XML-to-XSD service is a docker image built from the schema-genie-xml-to-xsd repository.
   • This processes lists of XML data into Xml Schema Definitions for the backend service, which is important for generating Hitman Schemas.
3. OpenApi-Lint Service
   • The OpenApi-Lint service is a docker image built from the schema-genie-openapi-lint repository.
   • This processes an OpenApi document and produces a lint report on how the document can be improved, based on a set of static rules.
4. Postgres
   • Postgres is a COTS relational database which is used by schema genie to persist application data.
   • This image is not built locally and is instead pulled from dockerhub which is the central docker registry for many COTS and open-source docker images.

These images can either be built locally from the individual repositories or pulled down from a docker image registry.

In the docker-compose.yml file, change the 'image:' field's value to the location of the image in the docker registry you are using, or to the name of the image on your local machine.

image: schema-genie-backend

Building the images locally

For each of the first three services above, if you are not pulling the docker images from a docker registry you can build them locally with Docker installed.

1. Clone the git repository.
2. Open a command terminal in the root directory of the repository.
3. Run docker build . -t {image-name}
4. Ensure the {image-name} value matches the image field in the docker-compose.yml file in schema-genie-stack.

You should then see the image in the list of local docker images when doing a docker images command.

Unpackaging pre-built images

In some circumstances, you may not have the ability to build the images locally, as you may be unable to pull libraries from the internet. You can unpackage the images with docker if you have been provided with the relevant '.tar' files.

1. Download the .tar files to your local machine.
2. Open a command terminal in the directory your .tar file is saved.
3. Run docker load --input {image.tar}
4. Ensure the new image (seen when using the command docker images) matches the image field in the docker-compose.yml file in schema-genie-stack.

Optional - Running Services Locally.

Using the docker stack, you have the option of deploying services locally instead of using Docker. For example deploying the backend locally. In the case of the backend, this can be done by simply commenting out the backend service in the docker-compose.yml and following the instructions in the schema-genie-backend repository to run the service locally.

Launching

The following instructions allow for the schema genie stack to be launched.

docker compose up

Generate a HITMAN Schema using HAR files

1. Navigate to Schema Genie Swagger UI

2. Create an application using the POST /api/v1/applications endpoint

3. Upload HAR files using the POST /api/v1/applications/{applicationId}/test-data endpoint.

applicationId can be found in the location header of the response from the POST /api/v1/applications endpoint

4. Generate a schema using the POST /api/v1/applications/{applicationId}/generate-schema endpoint

5. Export the HITMAN schema using the POST /api/v1/applications/{applicationId}/schema/{schemaId}/export endpoint

schemaId can be a specific revision based on the response from the POST /api/v1/applications/{applicationId}/generate-schema endpoint or can be the latest by using openapi.

FAQs

How do I prevent Schema Genie over parameterising the endpoints?

Schema Genie will try to parameterise as much as possible by default.

This behaviour can be altered by either supplying ?nounPrecendenceEnabled=true query string in the generate schema endpoint.

Alternatively, you can supply Schema Genie with a list of (partial) paths that you know are not by parameters by using the POST /api/v1/applications/{applicationId}/known-path-values endpoint.