Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: implement shuffling cache #6030

Merged
merged 3 commits into from
Nov 30, 2023
Merged

feat: implement shuffling cache #6030

merged 3 commits into from
Nov 30, 2023

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Oct 11, 2023 •
edited by dapplion
Loading

Motivation

  • To verify attestations, we actually don't need the state but only shuffling
  • In case the head state of attestation is not available, lodestar may have to do a regen, or not able to validate attestation

Description

  • This PR started with feat: n historical states #6008 and addressed comments from there
  • Implement a ShufflingCache, add new shufflings when we pass epoch transitions
    • Every shuffling is keyed by epoch and dependent root which is the last block root of the previous 2 epochs
  • Support regen state which is tracked as a promise inside ShufflingCache
    • this is not needed with the default chain option of maxSkippedSlot = 32, handle just in case
    • if there are multiple requests of the same shuffling, only do the regen once
    • also support 1 regen at a time, bound inside ShufflingCache

this is a prerequisite for #6008

Closes #2848

@twoeths twoeths mentioned this pull request Oct 11, 2023
Closed
9 tasks
@github-actions
Copy link
Contributor

github-actions bot commented Oct 11, 2023 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 5ff38b4 Previous: 8afbc98 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 550.38 us/op 665.09 us/op 0.83
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 48.749 us/op 53.691 us/op 0.91
BLS verify - blst-native 1.1288 ms/op 1.0988 ms/op 1.03
BLS verifyMultipleSignatures 3 - blst-native 2.3941 ms/op 2.3464 ms/op 1.02
BLS verifyMultipleSignatures 8 - blst-native 5.2946 ms/op 5.0602 ms/op 1.05
BLS verifyMultipleSignatures 32 - blst-native 19.470 ms/op 18.187 ms/op 1.07
BLS verifyMultipleSignatures 64 - blst-native 38.261 ms/op 35.775 ms/op 1.07
BLS verifyMultipleSignatures 128 - blst-native 75.891 ms/op 71.179 ms/op 1.07
BLS deserializing 10000 signatures 805.54 ms/op 769.27 ms/op 1.05
BLS deserializing 100000 signatures 8.0045 s/op 7.8165 s/op 1.02
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.1303 ms/op 1.0378 ms/op 1.09
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.2921 ms/op 1.1974 ms/op 1.08
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.0441 ms/op 1.8353 ms/op 1.11
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.5840 ms/op 2.8077 ms/op 1.28
BLS verifyMultipleSignatures - same message - 128 - blst-native 4.9923 ms/op 6.1567 ms/op 0.81
BLS aggregatePubkeys 32 - blst-native 22.394 us/op 21.136 us/op 1.06
BLS aggregatePubkeys 128 - blst-native 87.960 us/op 82.219 us/op 1.07
getAttestationsForBlock 23.344 ms/op 25.662 ms/op 0.91
isKnown best case - 1 super set check 222.00 ns/op 337.00 ns/op 0.66
isKnown normal case - 2 super set checks 223.00 ns/op 325.00 ns/op 0.69
isKnown worse case - 16 super set checks 221.00 ns/op 328.00 ns/op 0.67
CheckpointStateCache - add get delete 3.7400 us/op 3.7880 us/op 0.99
validate api signedAggregateAndProof - struct 2.4016 ms/op 2.2659 ms/op 1.06
validate gossip signedAggregateAndProof - struct 2.4014 ms/op 2.3535 ms/op 1.02
validate gossip attestation - vc 640000 1.1470 ms/op 1.1269 ms/op 1.02
batch validate gossip attestation - vc 640000 - chunk 32 136.26 us/op 133.19 us/op 1.02
batch validate gossip attestation - vc 640000 - chunk 64 121.49 us/op 117.94 us/op 1.03
batch validate gossip attestation - vc 640000 - chunk 128 115.06 us/op 113.31 us/op 1.02
batch validate gossip attestation - vc 640000 - chunk 256 108.01 us/op 105.30 us/op 1.03
pickEth1Vote - no votes 857.73 us/op 760.81 us/op 1.13
pickEth1Vote - max votes 8.7974 ms/op 8.1926 ms/op 1.07
pickEth1Vote - Eth1Data hashTreeRoot value x2048 15.319 ms/op 17.542 ms/op 0.87
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 28.061 ms/op 24.954 ms/op 1.12
pickEth1Vote - Eth1Data fastSerialize value x2048 399.82 us/op 408.58 us/op 0.98
pickEth1Vote - Eth1Data fastSerialize tree x2048 6.7809 ms/op 6.7670 ms/op 1.00
bytes32 toHexString 340.00 ns/op 440.00 ns/op 0.77
bytes32 Buffer.toString(hex) 194.00 ns/op 298.00 ns/op 0.65
bytes32 Buffer.toString(hex) from Uint8Array 307.00 ns/op 419.00 ns/op 0.73
bytes32 Buffer.toString(hex) + 0x 196.00 ns/op 299.00 ns/op 0.66
Object access 1 prop 0.10400 ns/op 0.19300 ns/op 0.54
Map access 1 prop 0.093000 ns/op 0.18400 ns/op 0.51
Object get x1000 5.0660 ns/op 5.3170 ns/op 0.95
Map get x1000 0.63500 ns/op 0.69900 ns/op 0.91
Object set x1000 24.384 ns/op 24.380 ns/op 1.00
Map set x1000 16.757 ns/op 16.346 ns/op 1.03
Return object 10000 times 0.21770 ns/op 0.22020 ns/op 0.99
Throw Error 10000 times 2.7253 us/op 2.6793 us/op 1.02
fastMsgIdFn sha256 / 200 bytes 1.8100 us/op 1.9040 us/op 0.95
fastMsgIdFn h32 xxhash / 200 bytes 193.00 ns/op 293.00 ns/op 0.66
fastMsgIdFn h64 xxhash / 200 bytes 237.00 ns/op 329.00 ns/op 0.72
fastMsgIdFn sha256 / 1000 bytes 6.0580 us/op 6.1860 us/op 0.98
fastMsgIdFn h32 xxhash / 1000 bytes 313.00 ns/op 401.00 ns/op 0.78
fastMsgIdFn h64 xxhash / 1000 bytes 304.00 ns/op 389.00 ns/op 0.78
fastMsgIdFn sha256 / 10000 bytes 52.905 us/op 52.632 us/op 1.01
fastMsgIdFn h32 xxhash / 10000 bytes 1.7370 us/op 1.7500 us/op 0.99
fastMsgIdFn h64 xxhash / 10000 bytes 1.1120 us/op 1.1900 us/op 0.93
send data - 1000 256B messages 11.329 ms/op 11.042 ms/op 1.03
send data - 1000 512B messages 16.229 ms/op 14.452 ms/op 1.12
send data - 1000 1024B messages 21.943 ms/op 21.770 ms/op 1.01
send data - 1000 1200B messages 15.599 ms/op 26.780 ms/op 0.58
send data - 1000 2048B messages 30.882 ms/op 31.016 ms/op 1.00
send data - 1000 4096B messages 31.359 ms/op 30.532 ms/op 1.03
send data - 1000 16384B messages 83.590 ms/op 74.874 ms/op 1.12
send data - 1000 65536B messages 372.66 ms/op 342.03 ms/op 1.09
enrSubnets - fastDeserialize 64 bits 986.00 ns/op 957.00 ns/op 1.03
enrSubnets - ssz BitVector 64 bits 316.00 ns/op 401.00 ns/op 0.79
enrSubnets - fastDeserialize 4 bits 118.00 ns/op 191.00 ns/op 0.62
enrSubnets - ssz BitVector 4 bits 318.00 ns/op 406.00 ns/op 0.78
prioritizePeers score -10:0 att 32-0.1 sync 2-0 66.948 us/op 65.318 us/op 1.02
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 81.417 us/op 78.496 us/op 1.04
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 108.34 us/op 107.62 us/op 1.01
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 182.83 us/op 175.89 us/op 1.04
prioritizePeers score 0:0 att 64-1 sync 4-1 204.35 us/op 195.88 us/op 1.04
array of 16000 items push then shift 1.2999 us/op 1.2592 us/op 1.03
LinkedList of 16000 items push then shift 5.8410 ns/op 5.7600 ns/op 1.01
array of 16000 items push then pop 49.598 ns/op 56.398 ns/op 0.88
LinkedList of 16000 items push then pop 5.6120 ns/op 5.5260 ns/op 1.02
array of 24000 items push then shift 1.8897 us/op 1.7293 us/op 1.09
LinkedList of 24000 items push then shift 5.7430 ns/op 6.0380 ns/op 0.95
array of 24000 items push then pop 86.954 ns/op 88.871 ns/op 0.98
LinkedList of 24000 items push then pop 5.5500 ns/op 5.6120 ns/op 0.99
intersect bitArray bitLen 8 5.4770 ns/op 6.0990 ns/op 0.90
intersect array and set length 8 48.236 ns/op 47.987 ns/op 1.01
intersect bitArray bitLen 128 27.743 ns/op 27.661 ns/op 1.00
intersect array and set length 128 678.60 ns/op 682.40 ns/op 0.99
bitArray.getTrueBitIndexes() bitLen 128 1.0690 us/op 1.2560 us/op 0.85
bitArray.getTrueBitIndexes() bitLen 248 1.8080 us/op 1.8680 us/op 0.97
bitArray.getTrueBitIndexes() bitLen 512 3.4060 us/op 3.4990 us/op 0.97
Buffer.concat 32 items 778.00 ns/op 865.00 ns/op 0.90
Uint8Array.set 32 items 1.8780 us/op 2.5070 us/op 0.75
Set add up to 64 items then delete first 1.7217 us/op 1.7325 us/op 0.99
OrderedSet add up to 64 items then delete first 2.6187 us/op 2.6676 us/op 0.98
Set add up to 64 items then delete last 1.9588 us/op 1.9640 us/op 1.00
OrderedSet add up to 64 items then delete last 3.0603 us/op 2.9393 us/op 1.04
Set add up to 64 items then delete middle 1.9713 us/op 1.9913 us/op 0.99
OrderedSet add up to 64 items then delete middle 4.2930 us/op 4.1359 us/op 1.04
Set add up to 128 items then delete first 3.8035 us/op 3.8847 us/op 0.98
OrderedSet add up to 128 items then delete first 5.8029 us/op 6.3944 us/op 0.91
Set add up to 128 items then delete last 3.7478 us/op 3.7351 us/op 1.00
OrderedSet add up to 128 items then delete last 5.8530 us/op 5.6584 us/op 1.03
Set add up to 128 items then delete middle 3.7282 us/op 3.7307 us/op 1.00
OrderedSet add up to 128 items then delete middle 10.646 us/op 10.654 us/op 1.00
Set add up to 256 items then delete first 7.3394 us/op 7.6264 us/op 0.96
OrderedSet add up to 256 items then delete first 11.135 us/op 12.114 us/op 0.92
Set add up to 256 items then delete last 7.3270 us/op 7.4531 us/op 0.98
OrderedSet add up to 256 items then delete last 11.637 us/op 11.205 us/op 1.04
Set add up to 256 items then delete middle 7.2788 us/op 7.2883 us/op 1.00
OrderedSet add up to 256 items then delete middle 30.395 us/op 30.490 us/op 1.00
transfer serialized Status (84 B) 1.3240 us/op 1.5630 us/op 0.85
copy serialized Status (84 B) 1.1930 us/op 1.4110 us/op 0.85
transfer serialized SignedVoluntaryExit (112 B) 1.5050 us/op 1.8310 us/op 0.82
copy serialized SignedVoluntaryExit (112 B) 1.2740 us/op 1.5870 us/op 0.80
transfer serialized ProposerSlashing (416 B) 2.0320 us/op 2.7820 us/op 0.73
copy serialized ProposerSlashing (416 B) 2.1230 us/op 2.8770 us/op 0.74
transfer serialized Attestation (485 B) 2.4930 us/op 2.8340 us/op 0.88
copy serialized Attestation (485 B) 2.2730 us/op 2.8780 us/op 0.79
transfer serialized AttesterSlashing (33232 B) 2.0620 us/op 2.7680 us/op 0.74
copy serialized AttesterSlashing (33232 B) 4.3710 us/op 5.3720 us/op 0.81
transfer serialized Small SignedBeaconBlock (128000 B) 2.1480 us/op 3.1820 us/op 0.68
copy serialized Small SignedBeaconBlock (128000 B) 8.7630 us/op 10.108 us/op 0.87
transfer serialized Avg SignedBeaconBlock (200000 B) 2.4740 us/op 2.7530 us/op 0.90
copy serialized Avg SignedBeaconBlock (200000 B) 12.173 us/op 13.069 us/op 0.93
transfer serialized BlobsSidecar (524380 B) 2.3340 us/op 2.7070 us/op 0.86
copy serialized BlobsSidecar (524380 B) 72.308 us/op 108.06 us/op 0.67
transfer serialized Big SignedBeaconBlock (1000000 B) 2.3540 us/op 3.0000 us/op 0.78
copy serialized Big SignedBeaconBlock (1000000 B) 187.07 us/op 196.37 us/op 0.95
pass gossip attestations to forkchoice per slot 2.7358 ms/op 2.7156 ms/op 1.01
forkChoice updateHead vc 100000 bc 64 eq 0 474.55 us/op 447.87 us/op 1.06
forkChoice updateHead vc 600000 bc 64 eq 0 2.7851 ms/op 2.8429 ms/op 0.98
forkChoice updateHead vc 1000000 bc 64 eq 0 4.3167 ms/op 4.2752 ms/op 1.01
forkChoice updateHead vc 600000 bc 320 eq 0 2.6091 ms/op 2.5577 ms/op 1.02
forkChoice updateHead vc 600000 bc 1200 eq 0 2.6923 ms/op 2.6334 ms/op 1.02
forkChoice updateHead vc 600000 bc 7200 eq 0 3.2299 ms/op 3.1831 ms/op 1.01
forkChoice updateHead vc 600000 bc 64 eq 1000 9.9833 ms/op 9.6866 ms/op 1.03
forkChoice updateHead vc 600000 bc 64 eq 10000 9.7344 ms/op 9.1028 ms/op 1.07
forkChoice updateHead vc 600000 bc 64 eq 300000 11.958 ms/op 11.297 ms/op 1.06
computeDeltas 500000 validators 300 proto nodes 3.0549 ms/op 3.0339 ms/op 1.01
computeDeltas 500000 validators 1200 proto nodes 2.9562 ms/op 2.9947 ms/op 0.99
computeDeltas 500000 validators 7200 proto nodes 2.9437 ms/op 2.9075 ms/op 1.01
computeDeltas 750000 validators 300 proto nodes 4.5491 ms/op 4.4391 ms/op 1.02
computeDeltas 750000 validators 1200 proto nodes 4.5061 ms/op 4.5137 ms/op 1.00
computeDeltas 750000 validators 7200 proto nodes 4.6529 ms/op 4.4437 ms/op 1.05
computeDeltas 1400000 validators 300 proto nodes 8.9651 ms/op 8.3989 ms/op 1.07
computeDeltas 1400000 validators 1200 proto nodes 9.1607 ms/op 8.4565 ms/op 1.08
computeDeltas 1400000 validators 7200 proto nodes 8.7324 ms/op 8.3350 ms/op 1.05
computeDeltas 2100000 validators 300 proto nodes 13.802 ms/op 13.213 ms/op 1.04
computeDeltas 2100000 validators 1200 proto nodes 13.701 ms/op 13.382 ms/op 1.02
computeDeltas 2100000 validators 7200 proto nodes 13.916 ms/op 13.463 ms/op 1.03
computeProposerBoostScoreFromBalances 500000 validators 3.2910 ms/op 3.2634 ms/op 1.01
computeProposerBoostScoreFromBalances 750000 validators 3.1441 ms/op 3.2525 ms/op 0.97
computeProposerBoostScoreFromBalances 1400000 validators 3.1485 ms/op 3.2462 ms/op 0.97
computeProposerBoostScoreFromBalances 2100000 validators 3.1481 ms/op 3.0259 ms/op 1.04
altair processAttestation - 250000 vs - 7PWei normalcase 1.5632 ms/op 1.4879 ms/op 1.05
altair processAttestation - 250000 vs - 7PWei worstcase 2.2771 ms/op 2.1576 ms/op 1.06
altair processAttestation - setStatus - 1/6 committees join 97.384 us/op 96.238 us/op 1.01
altair processAttestation - setStatus - 1/3 committees join 195.48 us/op 189.99 us/op 1.03
altair processAttestation - setStatus - 1/2 committees join 278.01 us/op 265.94 us/op 1.05
altair processAttestation - setStatus - 2/3 committees join 371.90 us/op 377.26 us/op 0.99
altair processAttestation - setStatus - 4/5 committees join 485.86 us/op 461.48 us/op 1.05
altair processAttestation - setStatus - 100% committees join 585.27 us/op 579.94 us/op 1.01
altair processBlock - 250000 vs - 7PWei normalcase 9.0361 ms/op 7.7043 ms/op 1.17
altair processBlock - 250000 vs - 7PWei normalcase hashState 28.706 ms/op 37.841 ms/op 0.76
altair processBlock - 250000 vs - 7PWei worstcase 33.952 ms/op 31.808 ms/op 1.07
altair processBlock - 250000 vs - 7PWei worstcase hashState 92.722 ms/op 85.293 ms/op 1.09
phase0 processBlock - 250000 vs - 7PWei normalcase 2.5204 ms/op 2.6929 ms/op 0.94
phase0 processBlock - 250000 vs - 7PWei worstcase 27.692 ms/op 26.069 ms/op 1.06
altair processEth1Data - 250000 vs - 7PWei normalcase 413.61 us/op 372.22 us/op 1.11
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 9.5410 us/op 8.0540 us/op 1.18
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 60.143 us/op 39.907 us/op 1.51
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 19.525 us/op 17.818 us/op 1.10
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 13.126 us/op 8.4080 us/op 1.56
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 209.35 us/op 154.24 us/op 1.36
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 691.16 us/op 1.2277 ms/op 0.56
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.4173 ms/op 1.5196 ms/op 0.93
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.4057 ms/op 1.2827 ms/op 1.10
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.0642 ms/op 1.9756 ms/op 1.04
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.4099 ms/op 1.3957 ms/op 1.01
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.5271 ms/op 3.2415 ms/op 1.09
Tree 40 250000 create 254.84 ms/op 267.42 ms/op 0.95
Tree 40 250000 get(125000) 105.71 ns/op 100.32 ns/op 1.05
Tree 40 250000 set(125000) 666.22 ns/op 700.32 ns/op 0.95
Tree 40 250000 toArray() 14.507 ms/op 14.676 ms/op 0.99
Tree 40 250000 iterate all - toArray() + loop 14.651 ms/op 14.470 ms/op 1.01
Tree 40 250000 iterate all - get(i) 39.691 ms/op 39.159 ms/op 1.01
MutableVector 250000 create 10.949 ms/op 11.343 ms/op 0.97
MutableVector 250000 get(125000) 5.4720 ns/op 5.2280 ns/op 1.05
MutableVector 250000 set(125000) 208.92 ns/op 195.07 ns/op 1.07
MutableVector 250000 toArray() 2.3584 ms/op 2.5943 ms/op 0.91
MutableVector 250000 iterate all - toArray() + loop 2.8663 ms/op 3.1052 ms/op 0.92
MutableVector 250000 iterate all - get(i) 1.2784 ms/op 1.3184 ms/op 0.97
Array 250000 create 2.4300 ms/op 2.3281 ms/op 1.04
Array 250000 clone - spread 992.66 us/op 1.0218 ms/op 0.97
Array 250000 get(125000) 0.85900 ns/op 0.97200 ns/op 0.88
Array 250000 set(125000) 1.0550 ns/op 1.1790 ns/op 0.89
Array 250000 iterate all - loop 145.32 us/op 152.86 us/op 0.95
effectiveBalanceIncrements clone Uint8Array 300000 17.777 us/op 12.691 us/op 1.40
effectiveBalanceIncrements clone MutableVector 300000 268.00 ns/op 361.00 ns/op 0.74
effectiveBalanceIncrements rw all Uint8Array 300000 175.24 us/op 183.21 us/op 0.96
effectiveBalanceIncrements rw all MutableVector 300000 60.366 ms/op 60.432 ms/op 1.00
phase0 afterProcessEpoch - 250000 vs - 7PWei 71.877 ms/op 75.065 ms/op 0.96
phase0 beforeProcessEpoch - 250000 vs - 7PWei 40.769 ms/op 39.237 ms/op 1.04
altair processEpoch - mainnet_e81889 418.88 ms/op 402.71 ms/op 1.04
mainnet_e81889 - altair beforeProcessEpoch 66.849 ms/op 62.956 ms/op 1.06
mainnet_e81889 - altair processJustificationAndFinalization 9.6870 us/op 10.239 us/op 0.95
mainnet_e81889 - altair processInactivityUpdates 3.8553 ms/op 4.6698 ms/op 0.83
mainnet_e81889 - altair processRewardsAndPenalties 56.578 ms/op 71.883 ms/op 0.79
mainnet_e81889 - altair processRegistryUpdates 1.1210 us/op 1.5220 us/op 0.74
mainnet_e81889 - altair processSlashings 411.00 ns/op 420.00 ns/op 0.98
mainnet_e81889 - altair processEth1DataReset 399.00 ns/op 677.00 ns/op 0.59
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.6427 ms/op 1.7690 ms/op 0.93
mainnet_e81889 - altair processSlashingsReset 2.1830 us/op 2.1230 us/op 1.03
mainnet_e81889 - altair processRandaoMixesReset 3.4070 us/op 3.6060 us/op 0.94
mainnet_e81889 - altair processHistoricalRootsUpdate 549.00 ns/op 731.00 ns/op 0.75
mainnet_e81889 - altair processParticipationFlagUpdates 1.6610 us/op 1.7460 us/op 0.95
mainnet_e81889 - altair processSyncCommitteeUpdates 479.00 ns/op 636.00 ns/op 0.75
mainnet_e81889 - altair afterProcessEpoch 75.187 ms/op 75.183 ms/op 1.00
capella processEpoch - mainnet_e217614 1.8016 s/op 1.7245 s/op 1.04
mainnet_e217614 - capella beforeProcessEpoch 391.17 ms/op 374.11 ms/op 1.05
mainnet_e217614 - capella processJustificationAndFinalization 6.3780 us/op 6.2450 us/op 1.02
mainnet_e217614 - capella processInactivityUpdates 19.916 ms/op 17.182 ms/op 1.16
mainnet_e217614 - capella processRewardsAndPenalties 457.33 ms/op 461.09 ms/op 0.99
mainnet_e217614 - capella processRegistryUpdates 12.127 us/op 15.311 us/op 0.79
mainnet_e217614 - capella processSlashings 403.00 ns/op 525.00 ns/op 0.77
mainnet_e217614 - capella processEth1DataReset 543.00 ns/op 492.00 ns/op 1.10
mainnet_e217614 - capella processEffectiveBalanceUpdates 3.4392 ms/op 3.4689 ms/op 0.99
mainnet_e217614 - capella processSlashingsReset 1.6460 us/op 1.8130 us/op 0.91
mainnet_e217614 - capella processRandaoMixesReset 3.7650 us/op 3.1150 us/op 1.21
mainnet_e217614 - capella processHistoricalRootsUpdate 428.00 ns/op 606.00 ns/op 0.71
mainnet_e217614 - capella processParticipationFlagUpdates 1.4370 us/op 1.2620 us/op 1.14
mainnet_e217614 - capella afterProcessEpoch 219.93 ms/op 193.11 ms/op 1.14
phase0 processEpoch - mainnet_e58758 396.25 ms/op 375.83 ms/op 1.05
mainnet_e58758 - phase0 beforeProcessEpoch 109.42 ms/op 111.20 ms/op 0.98
mainnet_e58758 - phase0 processJustificationAndFinalization 12.611 us/op 13.020 us/op 0.97
mainnet_e58758 - phase0 processRewardsAndPenalties 57.709 ms/op 61.210 ms/op 0.94
mainnet_e58758 - phase0 processRegistryUpdates 3.9470 us/op 7.3250 us/op 0.54
mainnet_e58758 - phase0 processSlashings 403.00 ns/op 585.00 ns/op 0.69
mainnet_e58758 - phase0 processEth1DataReset 375.00 ns/op 541.00 ns/op 0.69
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 870.05 us/op 1.3532 ms/op 0.64
mainnet_e58758 - phase0 processSlashingsReset 1.7260 us/op 2.3930 us/op 0.72
mainnet_e58758 - phase0 processRandaoMixesReset 2.4070 us/op 3.1630 us/op 0.76
mainnet_e58758 - phase0 processHistoricalRootsUpdate 352.00 ns/op 648.00 ns/op 0.54
mainnet_e58758 - phase0 processParticipationRecordUpdates 2.2950 us/op 2.7440 us/op 0.84
mainnet_e58758 - phase0 afterProcessEpoch 65.030 ms/op 62.437 ms/op 1.04
phase0 processEffectiveBalanceUpdates - 250000 normalcase 976.08 us/op 978.77 us/op 1.00
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.1097 ms/op 1.1075 ms/op 1.00
altair processInactivityUpdates - 250000 normalcase 20.250 ms/op 20.395 ms/op 0.99
altair processInactivityUpdates - 250000 worstcase 21.224 ms/op 20.681 ms/op 1.03
phase0 processRegistryUpdates - 250000 normalcase 5.9250 us/op 3.4970 us/op 1.69
phase0 processRegistryUpdates - 250000 badcase_full_deposits 444.99 us/op 356.54 us/op 1.25
phase0 processRegistryUpdates - 250000 worstcase 0.5 115.13 ms/op 119.08 ms/op 0.97
altair processRewardsAndPenalties - 250000 normalcase 63.793 ms/op 61.931 ms/op 1.03
altair processRewardsAndPenalties - 250000 worstcase 61.264 ms/op 64.175 ms/op 0.95
phase0 getAttestationDeltas - 250000 normalcase 5.1656 ms/op 5.0631 ms/op 1.02
phase0 getAttestationDeltas - 250000 worstcase 5.6999 ms/op 5.5319 ms/op 1.03
phase0 processSlashings - 250000 worstcase 1.6148 ms/op 1.3655 ms/op 1.18
altair processSyncCommitteeUpdates - 250000 112.98 ms/op 106.34 ms/op 1.06
BeaconState.hashTreeRoot - No change 190.00 ns/op 294.00 ns/op 0.65
BeaconState.hashTreeRoot - 1 full validator 119.76 us/op 120.15 us/op 1.00
BeaconState.hashTreeRoot - 32 full validator 1.4987 ms/op 1.4845 ms/op 1.01
BeaconState.hashTreeRoot - 512 full validator 15.877 ms/op 15.481 ms/op 1.03
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 184.04 us/op 159.82 us/op 1.15
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.1017 ms/op 2.2157 ms/op 0.95
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 27.875 ms/op 28.359 ms/op 0.98
BeaconState.hashTreeRoot - 1 balances 133.85 us/op 133.41 us/op 1.00
BeaconState.hashTreeRoot - 32 balances 1.1283 ms/op 1.1036 ms/op 1.02
BeaconState.hashTreeRoot - 512 balances 12.685 ms/op 11.618 ms/op 1.09
BeaconState.hashTreeRoot - 250000 balances 192.31 ms/op 195.72 ms/op 0.98
aggregationBits - 2048 els - zipIndexesInBitList 11.058 us/op 12.521 us/op 0.88
byteArrayEquals 32 64.758 ns/op 59.654 ns/op 1.09
Buffer.compare 32 38.159 ns/op 34.546 ns/op 1.10
byteArrayEquals 1024 1.7677 us/op 1.5879 us/op 1.11
Buffer.compare 1024 44.733 ns/op 39.375 ns/op 1.14
byteArrayEquals 16384 28.195 us/op 25.302 us/op 1.11
Buffer.compare 16384 208.95 ns/op 185.73 ns/op 1.13
byteArrayEquals 123687377 199.54 ms/op 194.94 ms/op 1.02
Buffer.compare 123687377 3.9272 ms/op 5.3088 ms/op 0.74
byteArrayEquals 32 - diff last byte 61.924 ns/op 59.482 ns/op 1.04
Buffer.compare 32 - diff last byte 39.124 ns/op 35.246 ns/op 1.11
byteArrayEquals 1024 - diff last byte 1.7814 us/op 1.6533 us/op 1.08
Buffer.compare 1024 - diff last byte 44.941 ns/op 41.529 ns/op 1.08
byteArrayEquals 16384 - diff last byte 28.147 us/op 26.273 us/op 1.07
Buffer.compare 16384 - diff last byte 203.70 ns/op 183.17 ns/op 1.11
byteArrayEquals 123687377 - diff last byte 200.98 ms/op 196.54 ms/op 1.02
Buffer.compare 123687377 - diff last byte 4.5351 ms/op 5.3104 ms/op 0.85
byteArrayEquals 32 - random bytes 4.4340 ns/op 4.3310 ns/op 1.02
Buffer.compare 32 - random bytes 40.120 ns/op 36.978 ns/op 1.08
byteArrayEquals 1024 - random bytes 4.4100 ns/op 4.2600 ns/op 1.04
Buffer.compare 1024 - random bytes 38.281 ns/op 34.952 ns/op 1.10
byteArrayEquals 16384 - random bytes 4.3940 ns/op 4.2620 ns/op 1.03
Buffer.compare 16384 - random bytes 34.996 ns/op 34.466 ns/op 1.02
byteArrayEquals 123687377 - random bytes 7.0300 ns/op 7.4100 ns/op 0.95
Buffer.compare 123687377 - random bytes 40.760 ns/op 37.150 ns/op 1.10
regular array get 100000 times 41.979 us/op 39.331 us/op 1.07
wrappedArray get 100000 times 41.320 us/op 39.317 us/op 1.05
arrayWithProxy get 100000 times 10.317 ms/op 9.6725 ms/op 1.07
ssz.Root.equals 55.393 ns/op 52.222 ns/op 1.06
byteArrayEquals 53.975 ns/op 51.450 ns/op 1.05
Buffer.compare 9.1500 ns/op 8.6640 ns/op 1.06
shuffle list - 16384 els 4.3556 ms/op 4.3340 ms/op 1.00
shuffle list - 250000 els 64.512 ms/op 63.461 ms/op 1.02
processSlot - 1 slots 15.132 us/op 14.262 us/op 1.06
processSlot - 32 slots 3.3802 ms/op 2.9526 ms/op 1.14
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 57.574 ms/op 52.476 ms/op 1.10
getCommitteeAssignments - req 1 vs - 250000 vc 2.1763 ms/op 2.1275 ms/op 1.02
getCommitteeAssignments - req 100 vs - 250000 vc 3.2684 ms/op 3.1986 ms/op 1.02
getCommitteeAssignments - req 1000 vs - 250000 vc 3.5723 ms/op 3.4750 ms/op 1.03
findModifiedValidators - 10000 modified validators 414.20 ms/op 436.19 ms/op 0.95
findModifiedValidators - 1000 modified validators 300.47 ms/op 317.42 ms/op 0.95
findModifiedValidators - 100 modified validators 339.82 ms/op 297.67 ms/op 1.14
findModifiedValidators - 10 modified validators 281.13 ms/op 296.64 ms/op 0.95
findModifiedValidators - 1 modified validators 316.66 ms/op 295.23 ms/op 1.07
findModifiedValidators - no difference 318.72 ms/op 286.89 ms/op 1.11
compare ViewDUs 4.1102 s/op 3.9360 s/op 1.04
compare each validator Uint8Array 1.4975 s/op 1.4996 s/op 1.00
compare ViewDU to Uint8Array 803.42 ms/op 752.89 ms/op 1.07
migrate state 1000000 validators, 24 modified, 0 new 763.69 ms/op 690.08 ms/op 1.11
migrate state 1000000 validators, 1700 modified, 1000 new 1.0428 s/op 938.25 ms/op 1.11
migrate state 1000000 validators, 3400 modified, 2000 new 1.3096 s/op 1.2531 s/op 1.05
migrate state 1500000 validators, 24 modified, 0 new 753.03 ms/op 754.20 ms/op 1.00
migrate state 1500000 validators, 1700 modified, 1000 new 1.0364 s/op 1.0347 s/op 1.00
migrate state 1500000 validators, 3400 modified, 2000 new 1.2972 s/op 1.2650 s/op 1.03
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 3.4600 ns/op 4.2600 ns/op 0.81
state getBlockRootAtSlot - 250000 vs - 7PWei 730.81 ns/op 654.80 ns/op 1.12
computeProposers - vc 250000 6.7142 ms/op 6.2416 ms/op 1.08
computeEpochShuffling - vc 250000 68.763 ms/op 65.072 ms/op 1.06
getNextSyncCommittee - vc 250000 103.79 ms/op 106.97 ms/op 0.97
computeSigningRoot for AttestationData 26.385 us/op 22.204 us/op 1.19
hash AttestationData serialized data then Buffer.toString(base64) 1.2082 us/op 1.1529 us/op 1.05
toHexString serialized data 729.70 ns/op 724.07 ns/op 1.01
Buffer.toString(base64) 149.29 ns/op 147.06 ns/op 1.02

by benchmarkbot/action

@twoeths twoeths added this to the v1.13.0 milestone Oct 11, 2023
@twoeths twoeths added the status-do-not-merge Merging this issue will break the build. Do not merge! label Oct 11, 2023
@twoeths twoeths marked this pull request as ready for review October 11, 2023 12:36
@twoeths twoeths requested a review from a team as a code owner October 11, 2023 12:36
@twoeths twoeths requested a review from dapplion October 13, 2023 08:45
@twoeths twoeths marked this pull request as draft October 31, 2023 07:52
@philknows philknows mentioned this pull request Nov 5, 2023
Closed
@twoeths twoeths force-pushed the tuyen/shuffling_cache branch 2 times, most recently from 29be86f to a4b3378 Compare November 11, 2023 02:41
@twoeths twoeths removed the status-do-not-merge Merging this issue will break the build. Do not merge! label Nov 14, 2023
@twoeths twoeths marked this pull request as ready for review November 24, 2023 06:48
@twoeths
Copy link
Contributor Author

twoeths commented Nov 27, 2023

after 4 days testing, metrics are the same to unstable in term of:

  • memory
  • attestation job time + job wait time
  • forwarded messages in beacon_attestation message
  • subnet mesh peers

ready for review @dapplion

wemeetagain
wemeetagain reviewed Nov 27, 2023
View reviewed changes
packages/beacon-node/src/chain/shufflingCache.ts Outdated
* - skip computing shuffling when loading state bytes from disk
*/
export class ShufflingCache {
/** LRU cache implemented as an array, pruned every time we add an item */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment looks wrong, its implemented as a map

packages/beacon-node/src/chain/shufflingCache.ts Outdated
*/
insertPromise(shufflingEpoch: Epoch, decisionRootHex: RootHex): void {
const promiseCount = Array.from(this.itemsByDecisionRootByEpoch.values())
.map((innerMap) => Array.from(innerMap.values()))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think theres a flatMap now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @wemeetagain that looks great 👍

dapplion
dapplion approved these changes Nov 28, 2023
View reviewed changes
Copy link
Contributor

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feature looks correct would like to get this deployed. There are good metrics that should proof it works well

@twoeths
Copy link
Contributor Author

twoeths commented Nov 30, 2023

after 4 days testing, metrics are the same to unstable in term of:

  • memory
  • attestation job time + job wait time
  • forwarded messages in beacon_attestation message
  • subnet mesh peers

ready for review @dapplion

adding metrics here:

  • Shuffling cache size is consistently 4 or 5
Screenshot 2023-11-30 at 10 30 25
  • No promise at all because it always hit the cache
Screenshot 2023-11-30 at 10 31 05
  • Miss metric hasn't populated ever
Screenshot 2023-11-30 at 10 31 41
  • No regen data which is great
Screenshot 2023-11-30 at 10 32 20

@twoeths
Copy link
Contributor Author

twoeths commented Nov 30, 2023

also beacon_attestation job time is the same before/after I deploy the branch

Screenshot 2023-11-30 at 10 36 28

@twoeths twoeths merged commit c39e12a into unstable Nov 30, 2023
15 checks passed
@twoeths twoeths deleted the tuyen/shuffling_cache branch November 30, 2023 03:39
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.13.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@wemeetagain wemeetagain wemeetagain left review comments

@dapplion dapplion dapplion approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
v1.13.0
Development

Successfully merging this pull request may close these issues.

Cache shufflings outside of regen state cache
3 participants
@twoeths @wemeetagain @dapplion