Kafka Executor
Responsible for implementing the "ProcessExecutor" interface and is capable of managing the shutdown behaviour of any local components. Although at minimum requires a "ResultsListener" to return result futures.
To "execute" a process it generates a PiInstance with a new ID locally, sends a ReduceRequest with that PiInstance then uses a promise listener to wait and respond with the result or exception.
Contains PiInstance which need reducing, along with any returned AtomicProcess results which need to be posted to that instance.
Consumes the ReduceRequest topic, posts the results into the PiInstances and reduces them. Produces Assignment messages for AtomicProcesses when opening new threads, otherwise posts to "Results" topic with results. It also posts the latest PiInstances to the "PiiHistory" topic.
Stores AtomicProcess calls which need computing.
Consumes the Assignment topic, executes AtomicProcesses, and posts SequenceRequests.
Stores:
- PiiUpdate: The latest PiInstance state for a given PiInstance.
- SequenceRequest: All the results of AtomicProcess calls which need posting to its PiInstance.
- SequenceFailures: The outstanding tasks which need mopping up after an exception.
Consumes the PiiHistory topic, matches up any pending SequenceRequests with the latest PiInstance and posts ReduceRequests for the Reducer.
Should really be called "PiiLogs" or "PiEvents", stores the history of all PiEvents .
"ResultsListener"s does no work per se, each one has it's own ConsumerGroup so each ResultsListener receives all PiEvents (a broadcast/subscribe pattern). Implements the "PiObservable" trait for the cluster and the KafkaExecutor.