updating the goethereum dependency #14351

james-prysm · 2024-08-15T21:26:10Z

What type of PR is this?

Other

What does this PR do? Why is it needed?

upgrades geth to 1.14.x ~
exchanges the usb third party dependency for hid
starts e2e from Capella instead of phase 0 ( we need to use some other way of testing phase 0 to capella)

related PRs that depend on geth changes for e2e

Which issues(s) does this PR fix?

Fixes #

Other notes for review

WORKSPACE

…distance due to time issues

james-prysm · 2024-11-04T22:11:37Z

beacon-chain/sync/subscriber_beacon_blocks.go

@@ -62,6 +63,9 @@ func (s *Service) beaconBlockSubscriber(ctx context.Context, msg proto.Message)
 // This function reconstructs the blob sidecars from the EL using the block's KZG commitments,
 // broadcasts the reconstructed blobs over P2P, and saves them into the blob storage.
 func (s *Service) reconstructAndBroadcastBlobs(ctx context.Context, block interfaces.ReadOnlySignedBeaconBlock) {
+	if block.Version() < version.Deneb {


temporary, it's addressed in terence's PR #14605

james-prysm · 2024-11-04T22:13:24Z

runtime/interop/premine-state.go

@@ -601,7 +601,7 @@ func (s *PremineGenesisConfig) setExecutionPayload(g state.BeaconState) error {
 			GasLimit:      gb.GasLimit(),
 			GasUsed:       gb.GasUsed(),
 			Timestamp:     gb.Time(),
-			ExtraData:     gb.Extra()[:32],
+			ExtraData:     gb.Extra(),


Extra data is not passed anymore due to removal of clique

james-prysm · 2024-11-04T22:16:13Z

testing/endtoend/component_handler_test.go

@@ -169,6 +169,9 @@ func (c *componentHandler) setup() {
 		c.eth1Proxy = proxies
 	}

+	// timeout while we wait for the eth1 client to sync
+	time.Sleep(10 * time.Second)


wonder if I should add a constant for this

james-prysm · 2024-11-04T22:16:43Z

testing/endtoend/minimal_builder_e2e_test.go

@@ -9,11 +9,11 @@ import (
 )

 func TestEndToEnd_MinimalConfig_WithBuilder(t *testing.T) {
-	r := e2eMinimal(t, types.InitForkCfg(version.Phase0, version.Deneb, params.E2ETestConfig()), types.WithCheckpointSync(), types.WithBuilder())
+	r := e2eMinimal(t, types.InitForkCfg(version.Capella, version.Deneb, params.E2ETestConfig()), types.WithCheckpointSync(), types.WithBuilder())


easier to start at capella than at the merge

james-prysm · 2024-11-05T17:15:43Z

runtime/interop/genesis.go

 	shanghaiTime := GethShanghaiTime(genesisTime, cfg)
+	if cfg.CapellaForkEpoch == 0 {


wondering if this is the best way to handle this instead of updating the function above. it's used elsewhere so perhaps it's not good as it changes the behavior

james-prysm · 2024-11-05T17:16:27Z

beacon-chain/p2p/fork_test.go

@@ -44,11 +46,17 @@ func TestStartDiscv5_DifferentForkDigests(t *testing.T) {
 	require.NoError(t, err)
 	defer bootListener.Close()

+	// Allow bootnode's table to have its initial refresh. This allows
+	// inbound nodes to be added in.
+	time.Sleep(5 * time.Second)


these 5 second timeouts on tests suck

james-prysm · 2024-11-05T17:16:43Z

testing/endtoend/components/eth1/miner.go

@@ -143,11 +143,9 @@ func (m *Miner) initAttempt(ctx context.Context, attempt int) (*os.File, error)
 		"--ws.origins=\"*\"",
 		"--ipcdisable",
 		"--verbosity=4",
-		"--mine",


not supported anymore

james-prysm · 2024-11-05T17:17:18Z

testing/endtoend/components/eth1/transactions.go

+			Data:     code,
+		})
+
+		// TODO: replace call with al, err := txfuzz.CreateAccessList(rpc, tx, sender) when txfuzz is fixed in new release


tx-fuzz doesn't work correctly in 1.4.0 so I just manually do the ethcall below

james-prysm · 2024-11-05T17:18:06Z

testing/middleware/builder/builder.go

@@ -798,7 +798,13 @@ func executableDataToBlock(params engine.ExecutableData, prevBeaconRoot []byte)
 		pRoot := common.Hash(prevBeaconRoot)
 		header.ParentBeaconRoot = &pRoot
 	}
-	block := gethTypes.NewBlockWithHeader(header).WithBody(txs, nil /* uncles */).WithWithdrawals(params.Withdrawals)
+	// TODO: update requests with requests for electra


when we support electra E2E we need to update this

james-prysm · 2024-11-05T17:20:36Z

time/slots/slottime.go

@@ -265,7 +265,7 @@ func SyncCommitteePeriodStartEpoch(e primitives.Epoch) (primitives.Epoch, error)
 // given slot start time
 func SecondsSinceSlotStart(s primitives.Slot, genesisTime, timeStamp uint64) (uint64, error) {
 	if timeStamp < genesisTime+uint64(s)*params.BeaconConfig().SecondsPerSlot {
-		return 0, errors.New("could not compute seconds since slot start: invalid timestamp")
+		return 0, fmt.Errorf("could not compute seconds since slot %d start: invalid timestamp, got %d < want %d", s, timeStamp, genesisTime+uint64(s)*params.BeaconConfig().SecondsPerSlot)


hope it's ok to update the error in this way.

james-prysm · 2024-11-05T22:23:16Z

testing/endtoend/evaluators/builder.go

@@ -64,7 +66,7 @@ func builderActive(_ *e2etypes.EvaluationContext, conns ...*grpc.ClientConn) err
 		if err != nil {
 			return err
 		}
-		if forkStartSlot == b.Block().Slot() || forkStartSlot+1 == b.Block().Slot() {
+		if forkStartSlot == b.Block().Slot() || forkStartSlot+1 == b.Block().Slot() || lowestBound <= 1 {


I had to add this due to this error

assertions.go:145: endtoend_test.go:607 Evaluation failed for epoch 1: capella block with slot 6 was not built by the builder. It has an extra data of and txRoot of 0xd13b70ff02dac0503e4103e33b9b4030d4d2409d71579efb4793434fddbe58d2: capella block with slot 6 was not built by the builder. It has an extra data of and txRoot of 0xd13b70ff02dac0503e4103e33b9b4030d4d2409d71579efb4793434fddbe58d2

james-prysm · 2024-11-05T22:24:47Z

time/slots/slottime.go

@@ -265,7 +265,7 @@ func SyncCommitteePeriodStartEpoch(e primitives.Epoch) (primitives.Epoch, error)
 // given slot start time
 func SecondsSinceSlotStart(s primitives.Slot, genesisTime, timeStamp uint64) (uint64, error) {
 	if timeStamp < genesisTime+uint64(s)*params.BeaconConfig().SecondsPerSlot {
-		return 0, errors.New("could not compute seconds since slot start: invalid timestamp")
+		return 0, fmt.Errorf("could not compute seconds since slot %d start: invalid timestamp, got %d < want %d", s, timeStamp, genesisTime+uint64(s)*params.BeaconConfig().SecondsPerSlot)


this error still happens Occasionally on e2e startup

helpers.go:242: time="2024-11-05 21:18:14" level=error msg="could not check if block arrived early" error="could not compute seconds since slot 0 start: invalid timestamp, got 1730841480 < want 1730841484" prefix=forkchoice-doublylinkedtree helpers.go:242: time="2024-11-05 21:18:14" level=error msg="Could not get builder payload" error="builder returned header with 0 bid amount" prefix="rpc/validator" ```

james-prysm and others added 2 commits August 15, 2024 16:22

updating the goethereum dependency

0e5a86e

Merge branch 'develop' into update-geth-to-v1.14

ba32681

prestonvanloon reviewed Aug 16, 2024

View reviewed changes

WORKSPACE Outdated Show resolved Hide resolved

james-prysm and others added 11 commits August 16, 2024 15:03

fixing dependencies

4a19a7e

Merge branch 'develop' into update-geth-to-v1.14

4f8b842

reverting workspace

8c853fc

more fixes, work in progress

412fa61

trying with upgraded geth version

3540b4b

fixing deprecated functions except for the time related ones on eth1 …

7b72d09

…distance due to time issues

fixing time issues

2fb1599

Merge branch 'develop' into update-geth-to-v1.14

9e75d25

gaz

dc92e95

fixing test and upgrading some dependencies and reverting others

b7e326e

Disable cgo in hid, delete old vendored usb library

d8b13a4

prestonvanloon force-pushed the update-geth-to-v1.14 branch from ab89634 to d8b13a4 Compare August 20, 2024 20:39

james-prysm added 3 commits August 20, 2024 15:46

Merge branch 'develop' into update-geth-to-v1.14

611be64

Merge branch 'develop' into update-geth-to-v1.14

86abac1

Merge branch 'develop' into update-geth-to-v1.14

a58c1d7

james-prysm added Blocked Blocked by research or external factors Breaking Changes Breaking changes for a major release labels Sep 11, 2024

james-prysm and others added 10 commits September 16, 2024 09:51

Merge branch 'develop' into update-geth-to-v1.14

0b67dc9

changelog

56b4ec1

rolling back dependencies

c943260

fixing go mod tidy

e47892d

Geth v1.13.6

902d029

fix tests

6e16bb5

Add ping interval, set to 500ms for tests. This didnt work

95d2a70

Update to v1.14.8

2c81dd9

Spread it out to different bootnodes

c468f65

Fix it

2edc281

james-prysm and others added 10 commits October 30, 2024 15:12

setting everything to capella

c83cd76

Merge branch 'develop' into update-geth-to-v1.14

0905831

more partial fixes

d0a2e10

Merge branch 'develop' into update-geth-to-v1.14

6f2dc61

Merge branch 'develop' into update-geth-to-v1.14

4345016

more fixes but still WIP

e80f522

Merge branch 'develop' into update-geth-to-v1.14

cde5d63

Merge branch 'develop' into update-geth-to-v1.14

5326ac9

fixing access list transactions"

9d891bb

Merge branch 'develop' into update-geth-to-v1.14

29b93ec

james-prysm commented Nov 4, 2024

View reviewed changes

james-prysm and others added 3 commits November 4, 2024 17:16

some cleanup

608eb1a

making configs dynamic

2939be1

Merge branch 'develop' into update-geth-to-v1.14

098254b

james-prysm commented Nov 5, 2024

View reviewed changes

james-prysm added 2 commits November 5, 2024 11:21

reverting time

f94cc3b

skip lower bound in builder

92b03a3

james-prysm commented Nov 5, 2024

View reviewed changes

james-prysm marked this pull request as ready for review November 5, 2024 22:55

james-prysm requested a review from a team as a code owner November 5, 2024 22:55

james-prysm requested review from kasey, rkapka and nisdas November 5, 2024 22:55

james-prysm added the Ready For Review A pull request ready for code review label Nov 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

updating the goethereum dependency #14351

updating the goethereum dependency #14351

james-prysm commented Aug 15, 2024 •

edited

Loading

james-prysm Nov 4, 2024

james-prysm Nov 4, 2024

james-prysm Nov 4, 2024

james-prysm Nov 4, 2024

james-prysm Nov 5, 2024

james-prysm Nov 5, 2024

james-prysm Nov 5, 2024

james-prysm Nov 5, 2024

james-prysm Nov 5, 2024

james-prysm Nov 5, 2024

james-prysm Nov 5, 2024

james-prysm Nov 5, 2024

		shanghaiTime := GethShanghaiTime(genesisTime, cfg)
		if cfg.CapellaForkEpoch == 0 {

updating the goethereum dependency #14351

Are you sure you want to change the base?

updating the goethereum dependency #14351

Conversation

james-prysm commented Aug 15, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

james-prysm commented Aug 15, 2024 •

edited

Loading