[major] Make a single queue subscriber middleware for other microservices #43
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #14
Some microservices need more information than just the source media file and the REST endpoint to place the derivative (i.e. the default context sent by Islandora's alpaca derivative middleware). Sometimes you need additional metadata around a media and/or its parent node to make the derivative. For such cases, alpaca's middleware model is not sufficient.
So we're creating a new middleware model that can read the message sent to activemq (which contains the necessary information) and send the entire event to a microservice. This new middleware has its own scyllaridae spec to define what queue to read from, and a URL to forward the entire Islandora event to. This new middleware is meant to be deployed alongside the ISLE stack so it can read from activemq without opening ports outside the localhost/docker network/firewall. This approach is implemented in opposition of alpaca's approach of parsing the Islandora event and sending its disparate components to an endpoint via HTTP headers. Instead, this middleware will send the entire event so those microservices that need additional context can access that information without having to subscribe directly to the activmq queue to read it. This unlocks having microservices that need this additional context from running on the same hardware as the ISLE stack; instead, these types of microservices can be distributed to other compute locations similar to other microservices with less context/metadata requirements.
This PR also adds concurrency to allow for multi-threaded execution to the respective microservices the middleware acts on behalf of.