Enable rate limits to automatically update based on the amount of available preprocessors

[kyle-sammons]

Summary

This PR enables rate limits to automatically update based on the amount of available preprocessors. It accomplishes this by creating a new MetadataStore for preprocessors, and having each preprocessor add an ephemeral node into that store on boot. Each preprocessor then adds a listener to that store, so that when a new preprocessor comes online (i.e. adds a new ephemeral node to the store), or goes offline (i.e. an ephemeral node is removed), the rate limit gets updated simply by dividing the cluster-wide rate limit by total number of ephemeral nodes present. Just to prevent preprocessors for somehow running on outdated information (for example if they didn't properly get the ZK event when a new node comes online), the preprocessors will get the preprocessor counts and update their rate limits at configurable intervals (default is 60 seconds).

This should effectively unlock the ability to enable auto-scaling on preprocessors as a whole.

This PR also fixes a couple of hidden bugs surrounding synchronization in ZK that I ran into while building it out.

Requirements

• I've read and understood the Contributing Guidelines and have done my best effort to follow them.
• I've read and agree to the Code of Conduct.