Consumer: let the user choose cluster topology #22
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
put registry names scoped to parent module instead of under the global `Registry` name this way 1. registry ids are scoped to their parent module 2. it is consistent with the rest of the code, like `MessageTracker.Supervisor` Signed-off-by: Francesco Noacco <francesco.noacco@secomind.com>
Replace Kernel's Registry and DynamicSupervisor modules with Horde's distributed versions. We still need to set up clustering with libcluster next. A fork of horde is used instead of the version from hex because of a bug where `extra_arguments` is not passed as a parameter to the processes in a DynamicSupervisor Signed-off-by: Francesco Noacco <francesco.noacco@secomind.com>
set up clustering using libcluster the topologies will be defined in a second iteration Signed-off-by: Francesco Noacco <francesco.noacco@secomind.com>
Use Horde's `process_redistribution` flag to enable automatic sharding (and resharding) of processes on cluster events. This allows for almost seamless autoscaling, as processes will be terminated and restarted on different Erlang nodes. Use the grain-like nature of AMQPDataConsumers to support spawining (re-)sharded consumers both on single and multi node Erlang clusters. Based on secomind#15. Signed-off-by: Arnaldo Cesco <arnaldo.cesco@secomind.com>
Allow users of this framework to setup the clustering as they prefer. This way, we don't force a choice. Expose the `cluster_topologies` option that can be populated with libcluster topologies. Signed-off-by: Arnaldo Cesco <arnaldo.cesco@secomind.com>
Signed-off-by: Arnaldo Cesco <arnaldo.cesco@secomind.com>
|
Superseded by progress in #15. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Allow users of this framework to setup the clustering as they prefer. This way, we don't force a choice.
Expose the
cluster_topologiesoption that can be populated with libcluster topologies.Based on #21