feat: add endpoint for Altair block reward

[ensi321]

Description
Introduce a beacon API endpoint to return block reward info according to the spec. This is one of the three reward endpoints mentioned in #5694

Note that the endpoint currently only supports Altair block or later. Attempts to query for phase0 block will return error.

TODO
Will open follow-up PRs for:

• Support block reward api for phase0 blocks
• Add finalized property to the reward response mentioning in Add finalized property to HTTP API responses #5693 which is currently blocked

Steps to test or reproduce

curl http://localhost:9596/eth/v1/beacon/rewards/blocks/head | jq
{
  "data": {
    "proposer_index": "0",
    "total": "0",
    "attestations": "0",
    "sync_aggregate": "0",
    "proposer_slashings": "0",
    "attester_slashings": "0"
  },
  "execution_optimistic": false
}

[codecov]

Codecov Report

Merging #6178 (eceea02) into unstable (ed43a98) will increase coverage by 1.53%.
Report is 59 commits behind head on unstable.
The diff coverage is 75.52%.

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #6178      +/-   ##
============================================
+ Coverage     60.15%   61.69%   +1.53%     
============================================
  Files           407      554     +147     
  Lines         46511    58090   +11579     
  Branches       1550     1832     +282     
============================================
+ Hits          27980    35840    +7860     
- Misses        18499    22213    +3714     
- Partials         32       37       +5     

[nflaig]

Looks pretty good! Just few remarks / questions regarding the units we use for the ssz type

I tested this on my goerli node (slot 7466188)

> curl -s http://localhost:9596/eth/v1/beacon/rewards/blocks/7466188 | jq
{
  "data": {
    "proposer_index": "2232",
    "total": "25570228",
    "attestations": "24656876",
    "sync_aggregate": "913352",
    "proposer_slashings": "0",
    "attester_slashings": "0"
  },
  "execution_optimistic": false
}

I haven't confirmed that the data is correct but 0,0255 ETH consensus reward sounds about right. We might wanna cross check our rewards APIs against other clients (Teku / Lighthouse) at some point.

Not sure if @tuyennhv wants to give this another review, all previous comments have been addressed

[wemeetagain]

A little sketchy that this can trigger a regen.
Is this the only non-debug endpoint that can do that?

[ensi321]

A little sketchy that this can trigger a regen.

What's your concern in particular? I don't think getPreState modifies anything under the hood. To get the preState of a block I don't see a way without calling regen. At some point regen.getState() must be called.

Is this the only non-debug endpoint that can do that?

This is the only one that I know of cc. @tuyennhv

[nflaig]

Is this the only non-debug endpoint that can do that?

It looks like this is the only one, can be confirmed by searching for allowRegen which is only set to true for debug state apis

What's your concern in particular?

The main concern is probably that you can use this API to easily DoS our public nodes as regen is quite expensive.

What about adding rewards APIs as their own namespace and disabling them by default?

e.g. we also have light client APIs on their own namespace even though those are part of /beacon as per spec

lodestar/packages/api/src/beacon/server/index.ts

Line 41 in 347c95f

lightclient: () => lightclient.getRoutes(config, api.lightclient),

we could then simply not enable rewards APIs by default here

lodestar/packages/beacon-node/src/api/rest/index.ts

Line 19 in 347c95f

api: ["beacon", "config", "events", "node", "validator", "lightclient"],

[ensi321]

What about adding rewards APIs as their own namespace and disabling them by default?

I read the documentation of other CL clients and looks like everyone has rewards endpoint enabled by default. Don’t know if the users expect this endpoint would be enabled on the public nodes.

Alternatively, we can limit the queried block to any block from last finalized checkpoint to head similar to what Teku behaves under prune mode: https://docs.teku.consensys.io/how-to/use-rewards-api#limitations . This way we only need to check stateCache and checkpointStateCache without triggering a regen request

The main concern is probably that you can use this API to easily DoS our public nodes as regen is quite expensive.

My wishful thinking is to have two priority tiers for RegenRequest. One being essential and one being non-essential. Any debug and reward endpoint that triggers regen should have a low priority such that when JobItemQueue size reaches a certain threshold, it rejects new RegenRequest that has low priority.

[wemeetagain]

Alternatively, we can limit the queried block from last finalized checkpoint to head

This is already the case now, it isn't fetching historical states, but this can still trigger block / epoch transitions if the requested state isn't already in a cache. In practice, in healthy network conditions, we have all states between finalized and head in cache. But it can become a problem in periods of non-finality.

We could add and use a regen.getPreStateSync that acts like regen.getStateSync in that it only checks cached state values and returns undefined if the state isn't cached. IMO this is a good compromise, since it allows us to serve the data in most cases, but doesn't open us up to any DoS issues.

[wemeetagain]

As an aside, It may be worth thinking deeper about our strategy around serving more expensive queries.

Should we support expensive queries? Behind a feature flag or flags? If so, what does that look like?
Behind extra namespaces for each additional feature-set in the APIs?
What does the architecture of generating expensive data look like? Queue that deprioritizes non-urgent work? Separate worker that handles expensive queries? Other?

[twoeths]

yes this is really a concern because with PersistentCheckpointStateCache, using getPreState means it will reload checkpoint state if needed

We could add and use a regen.getPreStateSync that acts like regen.getStateSync in that it only checks cached state values and returns undefined if the state isn't cached. IMO this is a good compromise, since it allows us to serve the data in most cases, but doesn't open us up to any DoS issues.

in this specific scenario, if we can get a cached pre state, we can also get post state and get the cached reward from post state, so it's no use to have getPreStateSync()

I think we only want to support getting cached reward in post state for lodestar (it means it'll work for 64-96 blocks for now). If demand arises from any specific use case, we can enhance later

[ensi321]

in this specific scenario, if we can get a cached pre state, we can also get post state and get the cached reward from post state, so it's no use to have getPreStateSync()

For the purpose of this endpoint, I echo with @wemeetagain 's idea to only serve blocks which its corresponding postState is cached.
So we should just drop the preState parameter in computeBlockRewards(block, preState, postState), and also all the reward calculation in blockRewards.ts and solely rely on cached values
Update: Looks like we still need the preState to calculate proposer and attester slashing rewards instead of simply checking the cache because RewardCache combines both into a single slashing value. Will still need getPreStateSync() for this PR

so it's no use to have getPreStateSync()

The other two rewards endpoint (attestation and sync committee) both require preState so we will need to have getPreStateSync() implemented at some point. We can avoid getPreStateSync() for this endpoint because we have block rewards cached in BeaconStateCache.proposerRewards: RewardCache but that's not the case for the other endpoints.

[ensi321]

Please take a closer look at getPreStateSync() and its usage. Pretty much moved the majority of getPreState() to this function.
Looks to me checkpointStateCache.getLatest() wouldn't trigger a reload in the absence of state.
@tuyennhv @wemeetagain @nflaig

[twoeths]

yes looks good to me

[ensi321]

@wemeetagain Would love a second pass from you being merging this

[wemeetagain]

my main concern about this endpoint triggering regen has been resolved
gave another review, LGTM

[wemeetagain]

🎉 This PR is included in v1.17.0 🎉