Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement POST validators/validator_balances APIs #4872

Merged
merged 4 commits into from
Dec 8, 2023

Conversation

michaelsproul
Copy link
Member

@michaelsproul michaelsproul commented Oct 23, 2023

Issue Addressed

Some API users want to fetch large chunks of the validator set, and currently do so using the id param in the query string. However, query strings are usually subject to length limits on both the client and server side. For Lighthouse, we can't extend the length limit without forking our HTTP server (see #4333 (comment)).

Proposed Changes

Bypass the issue of query string limits by adding a POST variant of the /eth/v1/beacon/states/{state_id}/validators method. It takes the exact same parameters in a JSON body that looks like this:

{
  "ids": ["1"],
  "statuses": ["active"]
}

Update (Dec 2023): this is part of the beacon API spec now.

Additional Info

I would like the RocketPool analytics tools to use this, as they're currently using individual validator queries and large batches in parallel, which frustrate Lighthouse's state load mechanisms. Even with tree-states, large numbers of parallel queries are hard to deal with, because we have to load the entire state, even if only one validator is required. We have caches, but these typically only take effect after one of the requests completes. In practice this is several thousands times slower than if the end-user just loaded the whole validator set once. However users are (rightly) reluctant to do this, because the size of the JSON response is unwieldy. The best of both worlds is to get all the desired validators (and nothing else) in one request: i.e. using a POST request enabled by this PR.

@michaelsproul michaelsproul added enhancement New feature or request do-not-merge optimization Something to make Lighthouse run more efficiently. HTTP-API labels Oct 23, 2023
@michaelsproul michaelsproul changed the title Add POST API for fetching validators from state Implement POST validators/validator_balances APIs Dec 7, 2023
@michaelsproul michaelsproul added v4.6.0 ETA Q1 2024 ready-for-review The code is ready for review and removed do-not-merge labels Dec 7, 2023
@paulhauner paulhauner self-requested a review December 8, 2023 00:14
Copy link
Member

@paulhauner paulhauner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, simple and valuable!

@michaelsproul michaelsproul added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Dec 8, 2023
@michaelsproul michaelsproul merged commit b882519 into sigp:unstable Dec 8, 2023
25 checks passed
@michaelsproul michaelsproul deleted the post-state-validators branch December 8, 2023 01:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request HTTP-API optimization Something to make Lighthouse run more efficiently. ready-for-merge This PR is ready to merge. v4.6.0 ETA Q1 2024
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants