Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: enable mixed beacon-validator sim tests #5940

Merged
merged 4 commits into from
Sep 8, 2023
Merged

ci: enable mixed beacon-validator sim tests #5940

merged 4 commits into from
Sep 8, 2023

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented Sep 6, 2023

Motivation

At the moment mixed client tests are disabled because the assumption was that those will only work once #5553 is fixed but that is not the case as the issue is not an actual problem which would prevent Lodestar BN <> Lighthouse VC combinations from producing attestations and aggregates.

Description

Further considerations

We could update sim tests to run all containers with --network host flag and not create a custom docker network

@nazarhussain are there any other reasons / trade offs for using a custom docker network instead of host network?

@nflaig nflaig requested a review from a team as a code owner September 6, 2023 17:07
@nazarhussain
Copy link
Contributor

@nflaig Using custom network allows us to set a submit and then we can assign any IP to our services in that range from code. There is no drawback I see using it.

@nflaig
Copy link
Member Author

nflaig commented Sep 6, 2023

There is no drawback I see using it

Yeah, the only drawback is that we have to use host.docker.internal but that's not really a big deal, should work on all platforms now.

@github-actions
Copy link
Contributor

github-actions bot commented Sep 6, 2023 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: cd4aa18 Previous: 6ac8c07 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 812.63 us/op 596.90 us/op 1.36
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 48.356 us/op 47.207 us/op 1.02
BLS verify - blst-native 1.1106 ms/op 1.0834 ms/op 1.03
BLS verifyMultipleSignatures 3 - blst-native 2.3696 ms/op 2.2999 ms/op 1.03
BLS verifyMultipleSignatures 8 - blst-native 5.2460 ms/op 5.0980 ms/op 1.03
BLS verifyMultipleSignatures 32 - blst-native 19.252 ms/op 18.673 ms/op 1.03
BLS verifyMultipleSignatures 64 - blst-native 37.931 ms/op 36.814 ms/op 1.03
BLS verifyMultipleSignatures 128 - blst-native 75.296 ms/op 73.138 ms/op 1.03
BLS deserializing 10000 signatures 782.44 ms/op 751.70 ms/op 1.04
BLS deserializing 100000 signatures 7.7023 s/op 7.7848 s/op 0.99
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.0922 ms/op 1.1119 ms/op 0.98
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.2344 ms/op 1.2618 ms/op 0.98
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.1112 ms/op 2.1537 ms/op 0.98
BLS verifyMultipleSignatures - same message - 64 - blst-native 2.8847 ms/op 3.1955 ms/op 0.90
BLS verifyMultipleSignatures - same message - 128 - blst-native 4.7669 ms/op 4.8771 ms/op 0.98
BLS aggregatePubkeys 32 - blst-native 21.654 us/op 22.065 us/op 0.98
BLS aggregatePubkeys 128 - blst-native 84.983 us/op 86.862 us/op 0.98
getAttestationsForBlock 29.561 ms/op 26.495 ms/op 1.12
isKnown best case - 1 super set check 300.00 ns/op 198.00 ns/op 1.52
isKnown normal case - 2 super set checks 291.00 ns/op 196.00 ns/op 1.48
isKnown worse case - 16 super set checks 291.00 ns/op 194.00 ns/op 1.50
CheckpointStateCache - add get delete 3.3170 us/op 3.3520 us/op 0.99
validate api signedAggregateAndProof - struct 2.2923 ms/op 2.3931 ms/op 0.96
validate gossip signedAggregateAndProof - struct 2.3000 ms/op 2.3808 ms/op 0.97
validate gossip attestation - vc 640000 1.0866 ms/op 1.0738 ms/op 1.01
batch validate gossip attestation - vc 640000 - chunk 32 119.54 us/op 117.20 us/op 1.02
batch validate gossip attestation - vc 640000 - chunk 64 104.47 us/op 102.53 us/op 1.02
batch validate gossip attestation - vc 640000 - chunk 128 96.856 us/op 94.440 us/op 1.03
batch validate gossip attestation - vc 640000 - chunk 256 92.744 us/op 94.462 us/op 0.98
pickEth1Vote - no votes 798.50 us/op 853.19 us/op 0.94
pickEth1Vote - max votes 8.5780 ms/op 8.3455 ms/op 1.03
pickEth1Vote - Eth1Data hashTreeRoot value x2048 5.2098 ms/op 5.1542 ms/op 1.01
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 12.034 ms/op 10.207 ms/op 1.18
pickEth1Vote - Eth1Data fastSerialize value x2048 368.24 us/op 361.06 us/op 1.02
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.9648 ms/op 4.8209 ms/op 1.24
bytes32 toHexString 396.00 ns/op 335.00 ns/op 1.18
bytes32 Buffer.toString(hex) 295.00 ns/op 193.00 ns/op 1.53
bytes32 Buffer.toString(hex) from Uint8Array 408.00 ns/op 287.00 ns/op 1.42
bytes32 Buffer.toString(hex) + 0x 294.00 ns/op 195.00 ns/op 1.51
Object access 1 prop 0.18000 ns/op 0.10800 ns/op 1.67
Map access 1 prop 0.17700 ns/op 0.10400 ns/op 1.70
Object get x1000 4.8350 ns/op 5.4190 ns/op 0.89
Map get x1000 0.49300 ns/op 0.41900 ns/op 1.18
Object set x1000 26.154 ns/op 22.859 ns/op 1.14
Map set x1000 16.099 ns/op 15.808 ns/op 1.02
Return object 10000 times 0.21570 ns/op 0.21460 ns/op 1.01
Throw Error 10000 times 2.6446 us/op 2.8771 us/op 0.92
fastMsgIdFn sha256 / 200 bytes 1.8480 us/op 1.8210 us/op 1.01
fastMsgIdFn h32 xxhash / 200 bytes 285.00 ns/op 181.00 ns/op 1.57
fastMsgIdFn h64 xxhash / 200 bytes 341.00 ns/op 222.00 ns/op 1.54
fastMsgIdFn sha256 / 1000 bytes 5.8250 us/op 6.0800 us/op 0.96
fastMsgIdFn h32 xxhash / 1000 bytes 397.00 ns/op 301.00 ns/op 1.32
fastMsgIdFn h64 xxhash / 1000 bytes 400.00 ns/op 285.00 ns/op 1.40
fastMsgIdFn sha256 / 10000 bytes 50.114 us/op 51.930 us/op 0.97
fastMsgIdFn h32 xxhash / 10000 bytes 1.7250 us/op 1.6500 us/op 1.05
fastMsgIdFn h64 xxhash / 10000 bytes 1.1780 us/op 1.0790 us/op 1.09
enrSubnets - fastDeserialize 64 bits 865.00 ns/op 794.00 ns/op 1.09
enrSubnets - ssz BitVector 64 bits 415.00 ns/op 303.00 ns/op 1.37
enrSubnets - fastDeserialize 4 bits 187.00 ns/op 113.00 ns/op 1.65
enrSubnets - ssz BitVector 4 bits 422.00 ns/op 311.00 ns/op 1.36
prioritizePeers score -10:0 att 32-0.1 sync 2-0 63.928 us/op 61.551 us/op 1.04
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 77.399 us/op 72.300 us/op 1.07
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 103.99 us/op 103.87 us/op 1.00
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 174.49 us/op 175.26 us/op 1.00
prioritizePeers score 0:0 att 64-1 sync 4-1 196.76 us/op 197.34 us/op 1.00
array of 16000 items push then shift 1.2264 us/op 1.1941 us/op 1.03
LinkedList of 16000 items push then shift 6.6930 ns/op 6.4470 ns/op 1.04
array of 16000 items push then pop 67.886 ns/op 65.280 ns/op 1.04
LinkedList of 16000 items push then pop 5.6630 ns/op 5.3680 ns/op 1.05
array of 24000 items push then shift 1.8796 us/op 1.7494 us/op 1.07
LinkedList of 24000 items push then shift 5.8850 ns/op 6.1730 ns/op 0.95
array of 24000 items push then pop 80.848 ns/op 73.429 ns/op 1.10
LinkedList of 24000 items push then pop 5.6930 ns/op 5.5070 ns/op 1.03
intersect bitArray bitLen 8 5.4250 ns/op 5.4140 ns/op 1.00
intersect array and set length 8 40.827 ns/op 41.462 ns/op 0.98
intersect bitArray bitLen 128 26.048 ns/op 25.960 ns/op 1.00
intersect array and set length 128 576.24 ns/op 584.40 ns/op 0.99
bitArray.getTrueBitIndexes() bitLen 128 1.1510 us/op 1.0560 us/op 1.09
bitArray.getTrueBitIndexes() bitLen 248 1.9180 us/op 1.7550 us/op 1.09
bitArray.getTrueBitIndexes() bitLen 512 3.6460 us/op 3.3270 us/op 1.10
Buffer.concat 32 items 810.00 ns/op 751.00 ns/op 1.08
Uint8Array.set 32 items 2.2320 us/op 1.7600 us/op 1.27
Set add up to 64 items then delete first 1.5854 us/op 1.7083 us/op 0.93
OrderedSet add up to 64 items then delete first 2.4129 us/op 2.5913 us/op 0.93
Set add up to 64 items then delete last 1.8072 us/op 1.9302 us/op 0.94
OrderedSet add up to 64 items then delete last 2.7022 us/op 2.8711 us/op 0.94
Set add up to 64 items then delete middle 1.8053 us/op 1.9393 us/op 0.93
OrderedSet add up to 64 items then delete middle 3.9420 us/op 4.0968 us/op 0.96
Set add up to 128 items then delete first 3.5702 us/op 3.8324 us/op 0.93
OrderedSet add up to 128 items then delete first 5.6425 us/op 5.9752 us/op 0.94
Set add up to 128 items then delete last 3.4524 us/op 3.6927 us/op 0.93
OrderedSet add up to 128 items then delete last 5.2462 us/op 5.5201 us/op 0.95
Set add up to 128 items then delete middle 3.4473 us/op 3.6938 us/op 0.93
OrderedSet add up to 128 items then delete middle 10.057 us/op 10.453 us/op 0.96
Set add up to 256 items then delete first 7.0370 us/op 7.4750 us/op 0.94
OrderedSet add up to 256 items then delete first 11.285 us/op 11.798 us/op 0.96
Set add up to 256 items then delete last 6.9802 us/op 7.2812 us/op 0.96
OrderedSet add up to 256 items then delete last 10.495 us/op 10.971 us/op 0.96
Set add up to 256 items then delete middle 7.0364 us/op 7.2040 us/op 0.98
OrderedSet add up to 256 items then delete middle 29.132 us/op 29.852 us/op 0.98
transfer serialized Status (84 B) 1.3850 us/op 1.3390 us/op 1.03
copy serialized Status (84 B) 1.3360 us/op 1.1680 us/op 1.14
transfer serialized SignedVoluntaryExit (112 B) 1.6120 us/op 1.4110 us/op 1.14
copy serialized SignedVoluntaryExit (112 B) 1.3980 us/op 1.2330 us/op 1.13
transfer serialized ProposerSlashing (416 B) 2.4350 us/op 2.3820 us/op 1.02
copy serialized ProposerSlashing (416 B) 2.4820 us/op 2.4060 us/op 1.03
transfer serialized Attestation (485 B) 2.5110 us/op 2.4440 us/op 1.03
copy serialized Attestation (485 B) 2.5710 us/op 2.3540 us/op 1.09
transfer serialized AttesterSlashing (33232 B) 2.5240 us/op 2.5440 us/op 0.99
copy serialized AttesterSlashing (33232 B) 5.3440 us/op 4.2040 us/op 1.27
transfer serialized Small SignedBeaconBlock (128000 B) 2.5910 us/op 2.1730 us/op 1.19
copy serialized Small SignedBeaconBlock (128000 B) 10.659 us/op 8.2430 us/op 1.29
transfer serialized Avg SignedBeaconBlock (200000 B) 2.5530 us/op 2.0320 us/op 1.26
copy serialized Avg SignedBeaconBlock (200000 B) 15.314 us/op 11.314 us/op 1.35
transfer serialized BlobsSidecar (524380 B) 2.3620 us/op 2.1700 us/op 1.09
copy serialized BlobsSidecar (524380 B) 65.189 us/op 99.711 us/op 0.65
transfer serialized Big SignedBeaconBlock (1000000 B) 2.6970 us/op 2.5460 us/op 1.06
copy serialized Big SignedBeaconBlock (1000000 B) 127.44 us/op 158.47 us/op 0.80
pass gossip attestations to forkchoice per slot 2.7683 ms/op 2.6508 ms/op 1.04
forkChoice updateHead vc 100000 bc 64 eq 0 418.54 us/op 451.93 us/op 0.93
forkChoice updateHead vc 600000 bc 64 eq 0 2.7910 ms/op 3.3245 ms/op 0.84
forkChoice updateHead vc 1000000 bc 64 eq 0 4.4775 ms/op 4.4762 ms/op 1.00
forkChoice updateHead vc 600000 bc 320 eq 0 2.5778 ms/op 2.6247 ms/op 0.98
forkChoice updateHead vc 600000 bc 1200 eq 0 2.7219 ms/op 2.6293 ms/op 1.04
forkChoice updateHead vc 600000 bc 7200 eq 0 3.5648 ms/op 3.4832 ms/op 1.02
forkChoice updateHead vc 600000 bc 64 eq 1000 9.9321 ms/op 9.1897 ms/op 1.08
forkChoice updateHead vc 600000 bc 64 eq 10000 9.4158 ms/op 9.3269 ms/op 1.01
forkChoice updateHead vc 600000 bc 64 eq 300000 12.036 ms/op 11.807 ms/op 1.02
computeDeltas 500000 validators 300 proto nodes 2.8688 ms/op 2.8269 ms/op 1.01
computeDeltas 500000 validators 1200 proto nodes 2.8543 ms/op 2.8637 ms/op 1.00
computeDeltas 500000 validators 7200 proto nodes 2.8554 ms/op 2.8662 ms/op 1.00
computeDeltas 750000 validators 300 proto nodes 4.3046 ms/op 4.3251 ms/op 1.00
computeDeltas 750000 validators 1200 proto nodes 4.3048 ms/op 4.3042 ms/op 1.00
computeDeltas 750000 validators 7200 proto nodes 4.3259 ms/op 4.3199 ms/op 1.00
computeDeltas 1400000 validators 300 proto nodes 8.3359 ms/op 8.3967 ms/op 0.99
computeDeltas 1400000 validators 1200 proto nodes 8.4363 ms/op 8.3870 ms/op 1.01
computeDeltas 1400000 validators 7200 proto nodes 8.3686 ms/op 8.5752 ms/op 0.98
computeDeltas 2100000 validators 300 proto nodes 12.645 ms/op 12.872 ms/op 0.98
computeDeltas 2100000 validators 1200 proto nodes 12.655 ms/op 12.628 ms/op 1.00
computeDeltas 2100000 validators 7200 proto nodes 12.869 ms/op 12.642 ms/op 1.02
computeProposerBoostScoreFromBalances 500000 validators 2.6898 ms/op 2.7020 ms/op 1.00
computeProposerBoostScoreFromBalances 750000 validators 2.6896 ms/op 2.6383 ms/op 1.02
computeProposerBoostScoreFromBalances 1400000 validators 2.6901 ms/op 2.6506 ms/op 1.01
computeProposerBoostScoreFromBalances 2100000 validators 2.6920 ms/op 2.6788 ms/op 1.00
altair processAttestation - 250000 vs - 7PWei normalcase 1.5984 ms/op 1.5604 ms/op 1.02
altair processAttestation - 250000 vs - 7PWei worstcase 2.1955 ms/op 2.2253 ms/op 0.99
altair processAttestation - setStatus - 1/6 committees join 67.714 us/op 103.08 us/op 0.66
altair processAttestation - setStatus - 1/3 committees join 140.15 us/op 200.89 us/op 0.70
altair processAttestation - setStatus - 1/2 committees join 195.82 us/op 267.69 us/op 0.73
altair processAttestation - setStatus - 2/3 committees join 274.49 us/op 372.92 us/op 0.74
altair processAttestation - setStatus - 4/5 committees join 391.88 us/op 471.32 us/op 0.83
altair processAttestation - setStatus - 100% committees join 452.06 us/op 590.78 us/op 0.77
altair processBlock - 250000 vs - 7PWei normalcase 8.3042 ms/op 7.1342 ms/op 1.16
altair processBlock - 250000 vs - 7PWei normalcase hashState 12.160 ms/op 12.158 ms/op 1.00
altair processBlock - 250000 vs - 7PWei worstcase 31.770 ms/op 29.986 ms/op 1.06
altair processBlock - 250000 vs - 7PWei worstcase hashState 46.340 ms/op 44.183 ms/op 1.05
phase0 processBlock - 250000 vs - 7PWei normalcase 2.6484 ms/op 1.8932 ms/op 1.40
phase0 processBlock - 250000 vs - 7PWei worstcase 26.636 ms/op 23.607 ms/op 1.13
altair processEth1Data - 250000 vs - 7PWei normalcase 395.11 us/op 276.96 us/op 1.43
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 7.3670 us/op 6.6190 us/op 1.11
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 33.114 us/op 16.169 us/op 2.05
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 5.5060 us/op 9.3670 us/op 0.59
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 6.1960 us/op 6.3800 us/op 0.97
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 136.67 us/op 94.588 us/op 1.44
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.0270 ms/op 580.60 us/op 1.77
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.4128 ms/op 887.35 us/op 1.59
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.3666 ms/op 838.43 us/op 1.63
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 1.9794 ms/op 2.0192 ms/op 0.98
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.6868 ms/op 1.4204 ms/op 1.19
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.3880 ms/op 3.4749 ms/op 0.98
Tree 40 250000 create 230.36 ms/op 227.01 ms/op 1.01
Tree 40 250000 get(125000) 111.88 ns/op 104.51 ns/op 1.07
Tree 40 250000 set(125000) 675.34 ns/op 696.00 ns/op 0.97
Tree 40 250000 toArray() 9.1012 ms/op 9.2125 ms/op 0.99
Tree 40 250000 iterate all - toArray() + loop 9.1017 ms/op 9.2841 ms/op 0.98
Tree 40 250000 iterate all - get(i) 38.811 ms/op 39.692 ms/op 0.98
MutableVector 250000 create 9.7549 ms/op 7.6758 ms/op 1.27
MutableVector 250000 get(125000) 5.5810 ns/op 5.4970 ns/op 1.02
MutableVector 250000 set(125000) 189.12 ns/op 182.12 ns/op 1.04
MutableVector 250000 toArray() 2.3158 ms/op 1.9053 ms/op 1.22
MutableVector 250000 iterate all - toArray() + loop 2.4258 ms/op 2.2974 ms/op 1.06
MutableVector 250000 iterate all - get(i) 1.3669 ms/op 1.3427 ms/op 1.02
Array 250000 create 2.1301 ms/op 2.1184 ms/op 1.01
Array 250000 clone - spread 998.36 us/op 897.65 us/op 1.11
Array 250000 get(125000) 0.56400 ns/op 0.46100 ns/op 1.22
Array 250000 set(125000) 0.63600 ns/op 0.53100 ns/op 1.20
Array 250000 iterate all - loop 76.968 us/op 77.807 us/op 0.99
effectiveBalanceIncrements clone Uint8Array 300000 12.567 us/op 11.558 us/op 1.09
effectiveBalanceIncrements clone MutableVector 300000 358.00 ns/op 249.00 ns/op 1.44
effectiveBalanceIncrements rw all Uint8Array 300000 168.76 us/op 170.11 us/op 0.99
effectiveBalanceIncrements rw all MutableVector 300000 61.543 ms/op 57.680 ms/op 1.07
phase0 afterProcessEpoch - 250000 vs - 7PWei 78.284 ms/op 75.229 ms/op 1.04
phase0 beforeProcessEpoch - 250000 vs - 7PWei 27.327 ms/op 30.341 ms/op 0.90
altair processEpoch - mainnet_e81889 258.30 ms/op 231.73 ms/op 1.11
mainnet_e81889 - altair beforeProcessEpoch 49.967 ms/op 45.877 ms/op 1.09
mainnet_e81889 - altair processJustificationAndFinalization 6.1500 us/op 5.8410 us/op 1.05
mainnet_e81889 - altair processInactivityUpdates 4.5392 ms/op 3.4838 ms/op 1.30
mainnet_e81889 - altair processRewardsAndPenalties 66.963 ms/op 59.475 ms/op 1.13
mainnet_e81889 - altair processRegistryUpdates 1.7790 us/op 958.00 ns/op 1.86
mainnet_e81889 - altair processSlashings 521.00 ns/op 165.00 ns/op 3.16
mainnet_e81889 - altair processEth1DataReset 651.00 ns/op 306.00 ns/op 2.13
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.4924 ms/op 852.00 us/op 1.75
mainnet_e81889 - altair processSlashingsReset 2.2810 us/op 1.7730 us/op 1.29
mainnet_e81889 - altair processRandaoMixesReset 3.1770 us/op 2.0710 us/op 1.53
mainnet_e81889 - altair processHistoricalRootsUpdate 680.00 ns/op 457.00 ns/op 1.49
mainnet_e81889 - altair processParticipationFlagUpdates 1.1250 us/op 1.0780 us/op 1.04
mainnet_e81889 - altair processSyncCommitteeUpdates 536.00 ns/op 245.00 ns/op 2.19
mainnet_e81889 - altair afterProcessEpoch 81.447 ms/op 83.586 ms/op 0.97
capella processEpoch - mainnet_e217614 799.45 ms/op 747.39 ms/op 1.07
mainnet_e217614 - capella beforeProcessEpoch 191.52 ms/op 185.54 ms/op 1.03
mainnet_e217614 - capella processJustificationAndFinalization 6.7270 us/op 6.4180 us/op 1.05
mainnet_e217614 - capella processInactivityUpdates 15.000 ms/op 14.825 ms/op 1.01
mainnet_e217614 - capella processRewardsAndPenalties 289.95 ms/op 255.61 ms/op 1.13
mainnet_e217614 - capella processRegistryUpdates 17.450 us/op 11.568 us/op 1.51
mainnet_e217614 - capella processSlashings 592.00 ns/op 285.00 ns/op 2.08
mainnet_e217614 - capella processEth1DataReset 540.00 ns/op 203.00 ns/op 2.66
mainnet_e217614 - capella processEffectiveBalanceUpdates 2.9643 ms/op 3.2326 ms/op 0.92
mainnet_e217614 - capella processSlashingsReset 1.8110 us/op 1.3260 us/op 1.37
mainnet_e217614 - capella processRandaoMixesReset 3.0970 us/op 1.7100 us/op 1.81
mainnet_e217614 - capella processHistoricalRootsUpdate 567.00 ns/op 265.00 ns/op 2.14
mainnet_e217614 - capella processParticipationFlagUpdates 1.6610 us/op 525.00 ns/op 3.16
mainnet_e217614 - capella afterProcessEpoch 194.67 ms/op 185.05 ms/op 1.05
phase0 processEpoch - mainnet_e58758 299.12 ms/op 242.99 ms/op 1.23
mainnet_e58758 - phase0 beforeProcessEpoch 119.26 ms/op 93.505 ms/op 1.28
mainnet_e58758 - phase0 processJustificationAndFinalization 9.1630 us/op 6.6700 us/op 1.37
mainnet_e58758 - phase0 processRewardsAndPenalties 65.542 ms/op 48.938 ms/op 1.34
mainnet_e58758 - phase0 processRegistryUpdates 8.8260 us/op 4.6610 us/op 1.89
mainnet_e58758 - phase0 processSlashings 568.00 ns/op 315.00 ns/op 1.80
mainnet_e58758 - phase0 processEth1DataReset 516.00 ns/op 258.00 ns/op 2.00
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.2285 ms/op 725.02 us/op 1.69
mainnet_e58758 - phase0 processSlashingsReset 1.9210 us/op 1.0090 us/op 1.90
mainnet_e58758 - phase0 processRandaoMixesReset 2.8980 us/op 1.4430 us/op 2.01
mainnet_e58758 - phase0 processHistoricalRootsUpdate 677.00 ns/op 262.00 ns/op 2.58
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.3700 us/op 1.6940 us/op 1.99
mainnet_e58758 - phase0 afterProcessEpoch 70.486 ms/op 69.070 ms/op 1.02
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.5585 ms/op 954.69 us/op 1.63
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.9153 ms/op 1.1353 ms/op 1.69
altair processInactivityUpdates - 250000 normalcase 22.200 ms/op 18.776 ms/op 1.18
altair processInactivityUpdates - 250000 worstcase 23.131 ms/op 18.775 ms/op 1.23
phase0 processRegistryUpdates - 250000 normalcase 3.7090 us/op 3.1760 us/op 1.17
phase0 processRegistryUpdates - 250000 badcase_full_deposits 340.24 us/op 222.37 us/op 1.53
phase0 processRegistryUpdates - 250000 worstcase 0.5 109.89 ms/op 100.90 ms/op 1.09
altair processRewardsAndPenalties - 250000 normalcase 58.938 ms/op 63.930 ms/op 0.92
altair processRewardsAndPenalties - 250000 worstcase 65.168 ms/op 63.713 ms/op 1.02
phase0 getAttestationDeltas - 250000 normalcase 5.1039 ms/op 5.2456 ms/op 0.97
phase0 getAttestationDeltas - 250000 worstcase 5.1324 ms/op 5.2618 ms/op 0.98
phase0 processSlashings - 250000 worstcase 1.3884 ms/op 1.4430 ms/op 0.96
altair processSyncCommitteeUpdates - 250000 105.82 ms/op 104.30 ms/op 1.01
BeaconState.hashTreeRoot - No change 322.00 ns/op 196.00 ns/op 1.64
BeaconState.hashTreeRoot - 1 full validator 30.781 us/op 32.462 us/op 0.95
BeaconState.hashTreeRoot - 32 full validator 335.16 us/op 300.33 us/op 1.12
BeaconState.hashTreeRoot - 512 full validator 3.5822 ms/op 3.1962 ms/op 1.12
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 37.709 us/op 36.340 us/op 1.04
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 557.82 us/op 509.32 us/op 1.10
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 7.4305 ms/op 7.1849 ms/op 1.03
BeaconState.hashTreeRoot - 1 balances 29.870 us/op 28.998 us/op 1.03
BeaconState.hashTreeRoot - 32 balances 283.60 us/op 271.92 us/op 1.04
BeaconState.hashTreeRoot - 512 balances 2.3621 ms/op 2.4519 ms/op 0.96
BeaconState.hashTreeRoot - 250000 balances 45.148 ms/op 44.136 ms/op 1.02
aggregationBits - 2048 els - zipIndexesInBitList 10.445 us/op 11.182 us/op 0.93
regular array get 100000 times 29.664 us/op 29.824 us/op 0.99
wrappedArray get 100000 times 29.659 us/op 29.730 us/op 1.00
arrayWithProxy get 100000 times 10.177 ms/op 11.082 ms/op 0.92
ssz.Root.equals 234.00 ns/op 155.00 ns/op 1.51
byteArrayEquals 220.00 ns/op 145.00 ns/op 1.52
shuffle list - 16384 els 4.7475 ms/op 4.3661 ms/op 1.09
shuffle list - 250000 els 67.327 ms/op 65.288 ms/op 1.03
processSlot - 1 slots 5.6220 us/op 5.8190 us/op 0.97
processSlot - 32 slots 749.43 us/op 698.37 us/op 1.07
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 52.309 ms/op 48.090 ms/op 1.09
getCommitteeAssignments - req 1 vs - 250000 vc 2.1611 ms/op 2.1236 ms/op 1.02
getCommitteeAssignments - req 100 vs - 250000 vc 3.2518 ms/op 3.2047 ms/op 1.01
getCommitteeAssignments - req 1000 vs - 250000 vc 3.5295 ms/op 3.4628 ms/op 1.02
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.4400 ns/op 3.4600 ns/op 1.28
state getBlockRootAtSlot - 250000 vs - 7PWei 938.56 ns/op 858.87 ns/op 1.09
computeProposers - vc 250000 6.2218 ms/op 6.3487 ms/op 0.98
computeEpochShuffling - vc 250000 70.314 ms/op 66.573 ms/op 1.06
getNextSyncCommittee - vc 250000 100.85 ms/op 107.48 ms/op 0.94
computeSigningRoot for AttestationData 8.5370 us/op 7.9898 us/op 1.07
hash AttestationData serialized data then Buffer.toString(base64) 1.2340 us/op 1.2172 us/op 1.01
toHexString serialized data 797.86 ns/op 740.56 ns/op 1.08
Buffer.toString(base64) 155.91 ns/op 144.63 ns/op 1.08

by benchmarkbot/action

@nflaig nflaig changed the title test: enable mixed beacon-validator sim tests ci: enable mixed beacon-validator sim tests Sep 6, 2023
@nflaig nflaig mentioned this pull request Sep 7, 2023
Closed
@wemeetagain wemeetagain merged commit 0f38c05 into unstable Sep 8, 2023
11 of 12 checks passed
@wemeetagain wemeetagain deleted the nflaig/enable-multiclient-sim-tests branch September 8, 2023 15:30
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.12.0 🎉

@nflaig nflaig mentioned this pull request Apr 23, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain approved these changes

@nazarhussain nazarhussain nazarhussain approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nflaig @nazarhussain @wemeetagain