JSON-RPC: performance problem with chainHead_v1_storage
queries using descendantValues
#5589
Labels
I3-annoyance
The node behaves within expectations, however this “expected behaviour” itself is at issue.
T3-RPC_API
This PR/Issue is related to RPC APIs.
We’ve encountered a performance issue when executing
chainHead_v1_storage
queries with thedescendantValues
option in the new JSON-RPC API.When performing such a query, the RPC node sends an
operationStorageItems
notification containing only 5 items. This is immediately followed by awaitingForContinue
notification. Upon receiving this, we immediately respond with achainHead_v1_continue
request, and this cycle repeats.This results in certain queries taking an excessively long time to resolve. For example, requesting the descendant values of
NominationPools.PoolMembers
on the Polkadot relay-chain can take 6 to 10 minutes to complete using the new JSON-RPC API, while the same request takes only a few seconds with the legacy RPC API.Expected Behavior
The node should efficiently return a larger number of items per
operationStorageItems
notification, especially when there is no sign of back-pressure (e.g., if thechainHead_v1_continue
response is received promptly). Ideally, the node could send hundreds of items at once, and dynamically adjust the number of items sent based on the system's responsiveness.Current Behavior
The node currently sends only 5 items per
operationStorageItems
notification (and always requesting awaitingForContinue
notification), significantly slowing down the resolution of large storage queries.Proposed Solution
operationStorageItems
notification, potentially sending several hundred at a time.chainHead_v1_continue
response is received quickly, send more items in the next notification).Logs
slowOperationStorageItems.log
The text was updated successfully, but these errors were encountered: