Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

app/eth2wrap: deep validator cache #3114

Merged
merged 6 commits into from
Jun 4, 2024
Merged

Conversation

gsora
Copy link
Collaborator

@gsora gsora commented Jun 3, 2024

Implement a validator cache that caches the complete Validators endpoint request.

Short-cut the duty selection for the DV validator set to the known-active, cached list instead: this will reduce the load on the beacon node drastically, since before this change we were calling the BN once every slot.

Try to reduce at a minimum upstream BN requests when the VC calls for validator state.

All Validators() calls to the upstream BN, if greater than 200 validators, will have an exponential timeout calculated as (50ms * validator amount).

category: feature
ticket: none

Implement a validator cache that caches the complete Validators endpoint request.

Short-cut the duty selection for the DV validator set to the known-active, cached list instead: this will reduce the load on the beacon node drastically, since before this change we were calling the BN once every slot.

Try to reduce at a minimum upstream BN requests when the VC calls for validator state.

All Validators() calls to the upstream BN, if greater than 200 validators, will have an exponential timeout calculated as `(50ms * validator amount)`.
Copy link

codecov bot commented Jun 3, 2024

Codecov Report

Attention: Patch coverage is 32.47863% with 79 lines in your changes missing coverage. Please review.

Project coverage is 56.68%. Comparing base (278bcf7) to head (e2cff5a).
Report is 4 commits behind head on main.

Current head e2cff5a differs from pull request most recent head 23843cb

Please upload reports for the commit 23843cb to get more accurate results.

Files Patch % Lines
app/eth2wrap/httpwrap.go 0.00% 29 Missing ⚠️
core/validatorapi/validatorapi.go 40.90% 21 Missing and 5 partials ⚠️
app/eth2wrap/multi.go 7.14% 13 Missing ⚠️
app/eth2wrap/lazy.go 14.28% 6 Missing ⚠️
app/eth2wrap/valcache.go 78.94% 3 Missing and 1 partial ⚠️
app/app.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3114      +/-   ##
==========================================
- Coverage   56.82%   56.68%   -0.15%     
==========================================
  Files         208      208              
  Lines       29153    29234      +81     
==========================================
+ Hits        16566    16570       +4     
- Misses      10749    10821      +72     
- Partials     1838     1843       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

core/validatorapi/validatorapi.go Show resolved Hide resolved
app/eth2wrap/valcache.go Show resolved Hide resolved
core/validatorapi/validatorapi.go Show resolved Hide resolved
@pinebit
Copy link
Contributor

pinebit commented Jun 4, 2024

All Validators() calls to the upstream BN, if greater than 200 validators, will have an exponential timeout calculated as (50ms * validator amount).

Is this done in this PR? I did not find changes implementing this logic.

@gsora
Copy link
Collaborator Author

gsora commented Jun 4, 2024

@pinebit you can find it here: https://github.com/ObolNetwork/charon/pull/3114/files#diff-9496d7e5dde67e0e9b5aef94238c8031bb904688be029b2e7683d871a0b02320R115

Copy link

sonarcloud bot commented Jun 4, 2024

Quality Gate Passed Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@gsora gsora added the merge when ready Indicates bulldozer bot may merge when all checks pass label Jun 4, 2024
@obol-bulldozer obol-bulldozer bot merged commit 470c3b0 into main Jun 4, 2024
12 checks passed
@obol-bulldozer obol-bulldozer bot deleted the gsora/deep-validator-cache branch June 4, 2024 17:16
KaloyanTanev pushed a commit that referenced this pull request Jun 5, 2024
Implement a validator cache that caches the complete Validators endpoint request.

Short-cut the duty selection for the DV validator set to the known-active, cached list instead: this will reduce the load on the beacon node drastically, since before this change we were calling the BN once every slot.

Try to reduce at a minimum upstream BN requests when the VC calls for validator state.

All Validators() calls to the upstream BN, if greater than 200 validators, will have an exponential timeout calculated as `(50ms * validator amount)`.

category: feature
ticket: none
obol-bulldozer bot pushed a commit that referenced this pull request Jun 5, 2024
- #3084
- #3092  
- #3098 
- #3099 
- #3100 
- #3106 
- #3110 
- #3111 
- #3114 
- #3118 

category: misc
ticket: #3103 

closes: #3103
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merge when ready Indicates bulldozer bot may merge when all checks pass
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants