Vtgateproxy v19 fix build #633
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This doesn't actually do anything yet except spark up the mysql server and start listening for connections.
* First draft of discovery * Fix address list collection * Fix nebula discovery
* Stash the connection attributes on the conn struct * Clean up code style around upstream server changes
* Cleanup discovery. Prep for load balancing
* first draft of az affinity
* Restore git deps
* Honor num connections
* fix bugs
* Don't try to register channelz (this should be done elsewhere)
lis, err := net.Listen("tcp", "localhost:8153")
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
service.RegisterChannelzServiceToServer(s)
go s.Serve(lis)
* Draft: very messy and doesn't compile * Simplifyy * less log, plz * simplify more * Simplified by a lot - much simpler now pick fewer addresses * fixy * Account for infinite * copyright nonsense * clean up debug logging * round_robin works! * use rw mutex to serialize creation * rework the filtering to make everything parameterized and more explicit Change all the config so that instead of hard coded constants we set the various connection attributes, json field names, etc using command line flags. Then make the pool type and affinity arguments more explicit and less generic. * no longer needed * update comments * only pass through the URL params we need * affinity is actually optional --------- Co-authored-by: Michael Demmer <mdemmer@slack-corp.com>
* Fix locking in getConnection Signed-off-by: Henry Robinson <hrobinson@slack-corp.com> * Comment Signed-off-by: Henry Robinson <hrobinson@slack-corp.com> * Undo bash change Signed-off-by: Henry Robinson <hrobinson@slack-corp.com> --------- Signed-off-by: Henry Robinson <hrobinson@slack-corp.com>
* refactor a bit more to consolidate the command line flags * rework again to have a single parser and move around all the logic This way we only have a single entity watching the file and dispatching out to the resolvers when things change rather than a bunch of tickers watching the file. Also cleaned up the code a bunch. * redo the shuffle to be a single pass * actually use the builder rand not the global one * add some metrics * only stat once per second * split out counter for unknown affinity
* reinitalise targets when parsing host list * remove metrics and logging changes
* move organization of target hosts to parse time * rework metrics and logging of parse errors * add discovery bits to debug status page * reset parseErr in the right place * add sync and change debug page to do the shuffle * unrelated but just move some code around --------- Co-authored-by: Michael Demmer <mdemmer@slack-corp.com>
Signed-off-by: Henry Robinson <hrobinson@slack-corp.com> Signed-off-by: Esme Lamb <dlamb@slack-corp.com>
* shuffle targets during parse phase * don't use global rand
* add firstready balancer * fail if an invalid balancer is picked * maintain the currentConn even if another one becomes ready * remove unnecessary globals * normalize the names
* return a custom error to fail fast * fatal if the resolver cannot initialize Signed-off-by: Esme Lamb <dlamb@slack-corp.com>
Signed-off-by: Esme Lamb <dlamb@slack-corp.com>
* Add end to end tests for vtgateproxy * Fix linter warnings and unhandled errors --------- Signed-off-by: Riley Laine <rlaine@slack-corp.com> Signed-off-by: Esme Lamb <dlamb@slack-corp.com>
Change the discovery layer to pass in the pool type as part of the Address Attributes, and use that to maintain a map of current connections for each pool type.
Signed-off-by: Riley Laine <rlaine@slack-corp.com>
Signed-off-by: Riley Laine <rlaine@slack-corp.com>
Signed-off-by: Riley Laine <rlaine@slack-corp.com>
When a connection is closed or reset before it is properly established, the existing ::getSession will log a spurious error since the connection has no attributes to extract for the pool type. Fix this by refactoring so that when closing a connection we only act on the session object if one already existed, i.e. the conn was used before.
* Add vtgateproxy tests to CI * Refactor tests to make them less flappy --------- Signed-off-by: Riley Laine <rlaine@slack-corp.com> Signed-off-by: Esme Lamb <dlamb@slack-corp.com>
Signed-off-by: Henry Robinson <hrobinson@slack-corp.com>
When there are a lot more targets than the number of connections in the pool, then it's possible that if the list of hosts changes, the builder might pick a totally new set of hosts than the previously selected ones, none of which will have established subconns. Instead of giving this new list to the picker immediately, first combine it with the list of hosts that were previously selected, so that those subconns have some time to warm up while the current set is still in the list.
Signed-off-by: Esme Lamb <dlamb@slack-corp.com>
* get v19 to build (WIP) * sort of builds? * misnamed flag * flag name * remove binary Signed-off-by: Esme Lamb <dlamb@slack-corp.com> * update go-mysql-driver Signed-off-by: Esme Lamb <dlamb@slack-corp.com> * return ErrNoSubConnAvailable for no available connections Signed-off-by: Esme Lamb <dlamb@slack-corp.com> --------- Signed-off-by: Esme Lamb <dlamb@slack-corp.com>
…s v19 (#598) * vtgateproxy: sticky random balancer and remote zone backup connections (#593) * gist of the idea of a sticky_random balancer * make sticky_random actually work * include the zone locality in the address attributes Make it available to the picker layer along with the pool type. Also rework how the locality is computed to do it as part of building the target host rather than deferring the comparisons to the sorter. * include IsLocal in debug output * add support for local zone affinity to sticky_random Using the zone local attribute inhjected by discovery (if it exists), update sticky_random so that it biases to only use the local zone connections if there are any available, otherwise fall back to remote. * add num_backup_conns option to force discovery of non-zone-local vtgates * sticky_random failure test * and the rest * sticky_random rebalance --------- Co-authored-by: Michael Demmer <mdemmer@slack-corp.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.