Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add canonical cache for live sync #114

Closed

Conversation

forcodedancing
Copy link
Collaborator

@forcodedancing forcodedancing commented Aug 19, 2024

Description

Add cache layer to cache historical committed accounts & storages.

Rationale

To improve the execution of live sync.

Example

Test Result - Live sync 20K blocks on BSC, from 41413200 to 41433100.

Note: There are reboots before each tests, to make sure the test environment is clean.

  • Metrics 1 - hit rate
image

account hit rate (the l3 account in the figure): 36.2%

storage hit rate (the l3 storage in the figure): 43.3%

Note: l1 account & storage caches refer to the revm state cache.

  • Metrics 2 - the execution time for every 100 blocks
image image

The time decreases about 21% after steady. (due to reboot, at the beginning, the performance is bad for both).

The time to apply bundle state to cache (update cache after db commit) is included for the cached version.

  • Metrics 3 - the gas throughput for [41432701, 41433100]
image

The cached version is 356 Mgas/s for execution on avg, while the original version is 277 Mgas/s.

The gas throughput increases about 28%. Why it is not aligned to the execution time?

  • The avg Mgas/s is calculated from the log (each block has a log about it).
  • The cached version introduces some extra cost for applying state to cache. This extra time is included in the Metrics 2.

Changes

Notable changes:

  • add each change in a bullet point here
  • ...

Potential Impacts

  • add potential impacts for other components here
  • ...

@forcodedancing forcodedancing force-pushed the livesync_cache branch 3 times, most recently from a9106ae to d080d77 Compare August 19, 2024 12:17
let mut to_wipe = false;
for storage in &change_set.storage {
if storage.wipe_storage {
to_wipe = true;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

storage.wipe_storage seems only mean to the storage of an account, but why we clean all storage cache here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is that: the quick cache is very simple there is no function for iterate or find keys with a prefix.
One way is that we can make storage cache to use nested structure, e.g., a map or another layer of cache. However, it seems a bit complex. Do you think should we implement in this way?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should have some test, like create/destroy a contract to ensure our cache works well

@forcodedancing forcodedancing force-pushed the livesync_cache branch 14 times, most recently from a837f30 to d121876 Compare September 1, 2024 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants