Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[major] Make a single queue subscriber middleware for other microservices #43

Merged
merged 5 commits into from
Oct 2, 2024

Conversation

joecorall
Copy link
Member

@joecorall joecorall commented Oct 1, 2024

Closes #14

Some microservices need more information than just the source media file and the REST endpoint to place the derivative (i.e. the default context sent by Islandora's alpaca derivative middleware). Sometimes you need additional metadata around a media and/or its parent node to make the derivative. For such cases, alpaca's middleware model is not sufficient.

So we're creating a new middleware model that can read the message sent to activemq (which contains the necessary information) and send the entire event to a microservice. This new middleware has its own scyllaridae spec to define what queue to read from, and a URL to forward the entire Islandora event to. This new middleware is meant to be deployed alongside the ISLE stack so it can read from activemq without opening ports outside the localhost/docker network/firewall. This approach is implemented in opposition of alpaca's approach of parsing the Islandora event and sending its disparate components to an endpoint via HTTP headers. Instead, this middleware will send the entire event so those microservices that need additional context can access that information without having to subscribe directly to the activmq queue to read it. This unlocks having microservices that need this additional context from running on the same hardware as the ISLE stack; instead, these types of microservices can be distributed to other compute locations similar to other microservices with less context/metadata requirements.

This PR also adds concurrency to allow for multi-threaded execution to the respective microservices the middleware acts on behalf of.

@joecorall joecorall changed the title Make a single queue subscriber middleware for other microservices [major] Make a single queue subscriber middleware for other microservices Oct 1, 2024
@joecorall joecorall merged commit f6f5610 into main Oct 2, 2024
15 checks passed
@joecorall joecorall deleted the queue-as-middleware branch October 2, 2024 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Create a channel buffer in the stomp subscriber
1 participant