Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use blocks v3 endpoint in the VC #4813

Merged
merged 78 commits into from
Jan 8, 2024
Merged

Conversation

eserilev
Copy link
Collaborator

@eserilev eserilev commented Oct 7, 2023

Issue Addressed

#4812

Proposed Changes

Use the blocks v3 endpoint in the vc if post deneb or if enabled via a config flag

Additional Info

This also includes changes from #4629. It'll probably be tough to review this without merging #4629 to unstable first

  • add block_v3 logic to the vc
  • enable block_v3 logic by default post deneb
  • enable block_v3 logic in vc via config (pre deneb)

@realbigsean
Copy link
Member

At a bare minimum I think we need to add a new VC flag --builder-boost-factor

Agree!

There might also be some additional flags we could add to improve UX

--builder-always sets builder_boost_factor to u64::MAX, ignoring local payloads as long as the builder payload is valid.
--local-always sets builder_boost_factor to 0, ignoring builder payloads as long as the local payload is valid

We have a BN flag --always-prefer-builder-payload, so re-using the name in the VC could be good? and --always-prefer-local-payload as an alias for 0? But maybe we leave these for a follow up PR

I asked other teams about whether they plan to switch endpoints at the fork or start using V3 right away or stick with V2 through the fork. Teku is still using V2 by default and has a flag for V3, so seems like we're ok to move forward with the same.

The Goerli fork for Deneb is gonna be on Jan 17, so if we're able to get this in by early next week (no pressure though @eserilev , we can bump it this is too tight a timeline), we'd be able to include it in our pre-release for that fork. That would give us a good opportunity to start getting people prepared for the deprecation of --builder-proposal. I've reviewed #5035 and it looks good to go apart from one small change.

@jimmygchen
Copy link
Member

jimmygchen commented Jan 5, 2024

At a bare minimum I think we need to add a new VC flag --builder-boost-factor

It would be a good feature, but I see this as a new and separate feature, i.e. we don't need a new flag to make our VC compatible with block v3. I think we also want to think about potentially removing some of the logic in the BN before we add these new flags, as we have the same flags in the BN, and it's already quite complex.

We need to do a pre-release next week for the Goerli Deneb fork and it would be great to have block V3 support (builder-boost-factor) included in the release, so we should probably consider making it work before adding additional features.

EDIT: ^ probably applies more to the beacon node PR (#5035), this one is probably lower priority. If we were to remove some of the BN flags in the future, I think we should bundle those changes together with the new VC flags in the same release, so users still have the ability to maintain the same Lighthouse behaviour without a workaround.

@jimmygchen
Copy link
Member

btw with the new --builder-boost-factor, we will be able to keep the --builder-proposal flag, and support per validator configuration, so we shouldn't need to deprecate the flag.

@realbigsean
Copy link
Member

btw with the new --builder-boost-factor, we will be able to keep the --builder-proposal flag, and support per validator configuration, so we shouldn't need to deprecate the flag.

Oh right! nice

@paulhauner paulhauner added v4.6.0 ETA Q1 2024 and removed v5.0.0 Q1 2024 labels Jan 7, 2024
@paulhauner
Copy link
Member

I pushed this to v4.6.0 based on a DM from @realbigsean in another channel. Please feel free to push it back if you didn't intend it for v4.6.0.

@michaelsproul
Copy link
Member

michaelsproul commented Jan 8, 2024

I've updated this branch to build on #5035. I would like someone to double-check my translation of builder_proposals (see commit a9304c0).

I've also removed the automatic activation at Deneb, as that's what I understood Sean desired from this comment: #4813 (comment).

I also removed some TODOs from the code and opened #5039 and #5040 to track the remaining tasks.

Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just some other small changes I noticed

validator_client/src/block_service.rs Outdated Show resolved Hide resolved
validator_client/src/block_service.rs Outdated Show resolved Hide resolved
validator_client/src/block_service.rs Outdated Show resolved Hide resolved
validator_client/src/block_service.rs Outdated Show resolved Hide resolved
@michaelsproul michaelsproul removed the backwards-incompat Backwards-incompatible API change label Jan 8, 2024
@jimmygchen jimmygchen added ready-for-review The code is ready for review and removed work-in-progress PR is a work-in-progress labels Jan 8, 2024
Copy link
Member

@michaelsproul michaelsproul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Copy link
Member

@jimmygchen jimmygchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other changes also look good to me!

paulhauner added a commit to paulhauner/lighthouse that referenced this pull request Jan 8, 2024
Squashed commit of the following:

commit f579a04
Author: Jimmy Chen <jchen.tc@gmail.com>
Date:   Mon Jan 8 15:46:10 2024 +1100

    Revert changes on making block errors irrecoverable.

commit 32cad11
Author: Jimmy Chen <jchen.tc@gmail.com>
Date:   Mon Jan 8 15:06:42 2024 +1100

    Remove duplicate `current_slot` parameter in `publish_block` function, and remove unnecessary clone.

commit b93f16d
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 8 12:59:28 2024 +1100

    Update CLI flags & book

commit a9304c0
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 8 12:36:45 2024 +1100

    Don't activate at fork; support builder_proposals

commit 56c565b
Merge: 3f86e3d db05d37
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 8 12:22:59 2024 +1100

    Merge remote-tracking branch 'origin/unstable' into block-v3-vc

commit 3f86e3d
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Jan 2 15:08:40 2024 +0200

    merge conflicts

commit 2e912f8
Merge: 8d5d5a1 01994c4
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Jan 2 15:01:10 2024 +0200

    merge conflicts

commit 8d5d5a1
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Mon Dec 4 14:44:35 2023 -0800

    fixed issues from merge

commit bc1a594
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Dec 3 13:49:33 2023 -0800

    comments

commit f0f2442
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Dec 3 13:47:50 2023 -0800

    comments

commit d108edc
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Dec 3 12:13:55 2023 -0800

    publish_block_v3 should return irrecoveerable errors

commit 6dc3273
Merge: 13942bf 44aaf13
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Dec 3 12:11:37 2023 -0800

    todos and flag comments

commit 13942bf
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Nov 24 07:19:52 2023 -0800

    remove block v3 fallback to v2

commit ad978f6
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Nov 22 14:24:38 2023 -0800

    revert

commit 8f3ab53
Merge: d9a50ad e856a90
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Nov 22 14:24:04 2023 -0800

    Merge branch 'unstable' of https://github.com/sigp/lighthouse into block-v3-vc

commit d9a50ad
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Nov 22 14:22:33 2023 -0800

    changes based on feedback

commit 282f92b
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Nov 3 13:04:11 2023 +0200

    lint

commit 412d7ca
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Nov 3 12:15:32 2023 +0200

    cleanup

commit 1a50c7f
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Nov 3 12:04:24 2023 +0200

    refactor

commit fb5e60b
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Nov 3 11:47:35 2023 +0200

    refactor

commit d346356
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Nov 3 11:18:29 2023 +0200

    fix merge

commit e2d4fea
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Nov 3 11:16:12 2023 +0200

    fix merge

commit be8a2be
Merge: 4715e26 36d8849
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Nov 3 11:09:27 2023 +0200

    merge conflicts

commit 4715e26
Merge: d2e7790 a9f9dc2
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Oct 29 16:44:32 2023 +0200

    Merge branch 'unstable' of https://github.com/sigp/lighthouse into block-v3-vc

commit d2e7790
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Oct 29 16:44:27 2023 +0200

    resolve merge conflicts

commit 229893a
Merge: 9562d8c cb5f56e
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Oct 26 19:36:25 2023 +0300

    Merge branch 'block-v3-endpoint' of https://github.com/eserilev/lighthouse into block-v3-vc

commit cb5f56e
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Oct 26 19:23:05 2023 +0300

    fmt

commit 219ad17
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Oct 26 19:19:54 2023 +0300

    header bugfix

commit 34a08a1
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Oct 22 01:24:41 2023 +0300

    refactor

commit 59e1a18
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Oct 22 00:06:05 2023 +0300

    remove comments

commit 7dc7ece
Merge: 4d3621e 074c495
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sat Oct 21 22:40:15 2023 +0300

    resolve merge conflicts

commit 9562d8c
Merge: a8ded56 4d3621e
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Oct 15 00:18:22 2023 +0300

    Merge branch 'block-v3-endpoint' of https://github.com/eserilev/lighthouse into block-v3-vc

commit 4d3621e
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Oct 15 00:09:31 2023 +0300

    update

commit 1667b32
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sat Oct 14 23:23:22 2023 +0300

    refactor based on feedback

commit a8ded56
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Mon Oct 9 16:34:26 2023 +0300

    add produce-block-v3

commit 33f3411
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sat Oct 7 18:52:14 2023 +0300

    add v3 logic into vc

commit dc746c4
Merge: 1d7cbf0 c3321dd
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sat Oct 7 18:24:09 2023 +0300

    Merge branch 'unstable' of https://github.com/sigp/lighthouse into block-v3-endpoint

commit d2c7dba
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sat Oct 7 18:23:49 2023 +0300

    add block v3 to vc

commit 1b20398
Merge: 1d7cbf0 c3321dd
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sat Oct 7 18:23:23 2023 +0300

    Merge branch 'unstable' of https://github.com/sigp/lighthouse into block-v3-vc

commit 1d7cbf0
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Oct 6 11:26:24 2023 +0300

    clone epoch participation outside of the loop

commit 0a70967
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Oct 6 01:57:28 2023 +0300

    fmt

commit 518c6a0
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Fri Oct 6 01:57:04 2023 +0300

    clone get_epoch_participation result

commit 4b1f396
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Oct 5 23:14:37 2023 +0300

    prevent setting the participation flag

commit 8a674dd
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Oct 1 17:34:17 2023 +0300

    update header name to consensus block value

commit 33254e7
Merge: 6878abe 441fc16
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Sun Oct 1 17:24:32 2023 +0300

    Merge branch 'unstable' of https://github.com/sigp/lighthouse into block-v3-endpoint

commit 6878abe
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Sep 28 17:03:45 2023 +0300

    lint

commit 91d37e9
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Sep 28 16:51:01 2023 +0300

    add consensus block value

commit df25083
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Sep 27 23:23:20 2023 +0300

    refactor

commit c68e7af
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Sep 27 23:03:15 2023 +0300

    merge unstable

commit 6441c52
Merge: 36f863a 0f05499
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Mon Sep 25 11:37:36 2023 +0300

    initial merge

commit 36f863a
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Sep 6 09:16:20 2023 +0300

    linting

commit d5cb547
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Sep 6 08:53:19 2023 +0300

    block value fix

commit a59a966
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Sep 6 08:05:18 2023 +0300

    add block value

commit 8021687
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Sep 6 07:31:43 2023 +0300

    fixed logs

commit f285d16
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Sep 6 00:27:39 2023 +0300

    remove dead code

commit 0f0c41e
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Sep 5 23:46:25 2023 +0300

    cleanup

commit 2599fec
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Sep 5 23:35:01 2023 +0300

    cleanup

commit 0edfcbd
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Sep 5 23:20:06 2023 +0300

    fix clippy issues

commit ae2b1d3
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Sep 5 18:27:35 2023 +0300

    blinded payload test case passing

commit 5be3113
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Sep 5 01:28:56 2023 +0300

    initial v3 test

commit 877973e
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Sep 5 01:07:04 2023 +0300

    revert cargo lock

commit 03f442f
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Sep 5 01:04:36 2023 +0300

    tests passing

commit 682eb61
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Sep 5 01:03:54 2023 +0300

    tests passing

commit 272cb76
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Mon Sep 4 20:56:36 2023 +0300

    debugging

commit 133c289
Merge: 17f00f3 2841f60
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Mon Sep 4 16:52:08 2023 +0300

    Merge branch 'unstable' of https://github.com/sigp/lighthouse into block-v3-endpoint

commit 17f00f3
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Aug 31 20:57:11 2023 +0300

    merge v2 flow with v3

commit 50b0d13
Merge: 54860ac 74eb267
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Aug 31 19:46:09 2023 +0300

    merge conflicts

commit 54860ac
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Mon Aug 21 12:49:50 2023 +0300

    some debugging

commit bac744e
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Aug 17 00:48:11 2023 +0300

    added v3 endpoint

commit 281d88d
Merge: 1802e45 59c24bc
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Aug 17 00:14:23 2023 +0300

    Merge branch 'unstable' of https://github.com/sigp/lighthouse into block-v3-endpoint

commit 1802e45
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Thu Aug 17 00:14:20 2023 +0300

    add new endpoint version

commit ae690fb
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Aug 16 18:40:35 2023 +0300

    add api logic

commit 48661b4
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Aug 16 18:34:53 2023 +0300

    add api logic

commit 5d984f0
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Wed Aug 16 14:20:03 2023 +0300

    the full flow...

commit a55e91d
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Aug 15 23:14:02 2023 +0300

    continue refactor

commit 6515ce4
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Aug 15 00:46:58 2023 +0300

    block v3 flow

commit cd36be5
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Tue Aug 15 00:44:11 2023 +0300

    block v3 flow

commit 2cf2dec
Author: Eitan Seri-Levi <eserilev@ucsc.edu>
Date:   Mon Aug 14 20:36:34 2023 +0300

    block v3 endpoint init
@paulhauner paulhauner mentioned this pull request Jan 8, 2024
@jimmygchen jimmygchen added ready-for-merge This PR is ready to merge. and removed ready-for-review The code is ready for review labels Jan 8, 2024
@realbigsean realbigsean merged commit 5c8c8da into sigp:unstable Jan 8, 2024
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
deneb ready-for-merge This PR is ready to merge. v4.6.0 ETA Q1 2024 val-client Relates to the validator client binary
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use validator/blocks v3 endpoint in the VC
5 participants