Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split storage ranges at the end of snap ranges sync to parallelize execution #7688

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

damian-orzechowski
Copy link
Contributor

Changes

At the end of state ranges sync, when only large storage trie is left to be synced, snap batches will be processed sequentially. This change adds a mechanism to detect such scenario and partition the space left to sync in a similar way that account ranges are partition at the start of snap sync. Parallel execution of the batches improves last phase - usually manifesting itself as "stuck" at 100% progress.

Types of changes

What types of changes does your code introduce?

  • Bugfix (a non-breaking change that fixes an issue)
  • New feature (a non-breaking change that adds functionality)
  • Breaking change (a change that causes existing functionality not to work as expected)
  • Optimization
  • Refactoring
  • Documentation update
  • Build-related changes
  • Other: Description

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Notes on testing

Optional. Remove if not applicable.

Documentation

Requires documentation update

  • Yes
  • No

If yes, link the PR to the docs update or the issue with the details labeled docs. Remove if not applicable.

Requires explanation in Release Notes

  • Yes
  • No

If yes, fill in the details here. Remove if not applicable.

@LukaszRozmej
Copy link
Member

Why not just partition storage tries by default always?

@damian-orzechowski
Copy link
Contributor Author

Why not just partition storage tries by default always?

Created a different approach here: #7733

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants