You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We want to quickly determine where a particular tablet id is running (or will be running soon), and currently only hive is the authoritative source of that info. This may be used to quickly select where an external clients should redirect their requests for maximum locality. It's currently possible to do via pipe cache, but with multiple drawbacks:
Pipe cache has to connect the pipe first, which is usually cheap when idle (for leader tablets), but it may have an extremely negative effect during request storms, when cache is cold and pipes are not connected yet.
Pipe cache connects to some (closest) instance when used with followers, so it cannot be used to gather node ids for all followers. Worse, follower pipes are not cheap, since pipe cache uses short lived follower pipes and keeps reconnecting periodically (to rediscover better instances that may have appeared since the previous connection).
It's already a service that has to receive and forward all tablet messages, adding more traffic that is not directly related to requests may affect latency.
Connected pipes require connected interconnect, which may cause all nodes to connect to all other nodes unnecessarily (it's ok when it's done by a local kqp session, it has to gather and forward results anyway, but for external redirects it's entirely avoidable).
Thus we want some HiveCache service that takes a list of tablet ids and returns corresponding node ids (possibly with all followers), ideally without any network round trips from a coherent local cache. A rough sketch of what it does:
Connects to a root hive and caches a coherent mapping from tablet range to an owner hive
For each relevant hive (where relevant means we needed that info at least once) it connects and caches a mapping of some basic info for each tablet (leader and follower nodes, maybe something else very basic). This info must be easy to cache without unnecessary traffic on reconnects (we don't want to redownload millions of tablets on every hive restart/disconnect), so it should be versioned with rollup using easily computed deltas.
For each relevant hive it must cache which tablet ids are actually in use (using some sparse bitmaps), so it is possible to detect deleted tablets very efficiently (including subscriptions and notifications as soon as a given tablet is deleted). See Tablet pipes with CheckAliveness may be too slow for volatile transactions #4797.
This would allow us to efficiently and locally answer a bunch of questions about tablets (with info that is almost always up-to-date). Ideally there should be a C++ object for getting cached info without expensive cross-actor messaging.
The text was updated successfully, but these errors were encountered:
We want to quickly determine where a particular tablet id is running (or will be running soon), and currently only hive is the authoritative source of that info. This may be used to quickly select where an external clients should redirect their requests for maximum locality. It's currently possible to do via pipe cache, but with multiple drawbacks:
Thus we want some HiveCache service that takes a list of tablet ids and returns corresponding node ids (possibly with all followers), ideally without any network round trips from a coherent local cache. A rough sketch of what it does:
This would allow us to efficiently and locally answer a bunch of questions about tablets (with info that is almost always up-to-date). Ideally there should be a C++ object for getting cached info without expensive cross-actor messaging.
The text was updated successfully, but these errors were encountered: