Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prune blobs #3852

Merged
merged 64 commits into from
Feb 8, 2023
Merged

Prune blobs #3852

merged 64 commits into from
Feb 8, 2023

Commits on Feb 8, 2023

  1. Boiler plate code for blobs pruning

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    e2a6da4 View commit details
    Browse the repository at this point in the history

  2. Prune blobs before data availability breakpoint

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    7bf88c2 View commit details
    Browse the repository at this point in the history

  3. Plug in pruning of blobs into app

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    fe0c911 View commit details
    Browse the repository at this point in the history

  4. fixup! Prune blobs before data availability breakpoint

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    2a41f25 View commit details
    Browse the repository at this point in the history

  5. fixup! Plug in pruning of blobs into app

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    b88d888 View commit details
    Browse the repository at this point in the history

  6. fixup! Plug in pruning of blobs into app

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    d21c66d View commit details
    Browse the repository at this point in the history

  7. Remove inaccurate guess for db index

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    934f3ab View commit details
    Browse the repository at this point in the history

  8. fixup! Prune blobs before data availability breakpoint

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    d3b94d8 View commit details
    Browse the repository at this point in the history

  9. Fix rebase conflict

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    28e1e63 View commit details
    Browse the repository at this point in the history

  10. Fix rebase conflict

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    a211e6a View commit details
    Browse the repository at this point in the history

  11. Fix rebase conflict

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    ce2db35 View commit details
    Browse the repository at this point in the history

  12. Fix typo 

    Co-authored-by: realbigsean <seananderson33@GMAIL.com>
    @emhane @realbigsean
    emhane and realbigsean committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    82ffec3 View commit details
    Browse the repository at this point in the history

  13. Prune blobs on migration in addition to start-up

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    d67468d View commit details
    Browse the repository at this point in the history

  14. Fix try_prune_blobs to use state root

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    667cca5 View commit details
    Browse the repository at this point in the history

  15. Improve syntax 

    Co-authored-by: Michael Sproul <micsproul@gmail.com>
    @emhane @michaelsproul
    emhane and michaelsproul committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    6f5ca02 View commit details
    Browse the repository at this point in the history

  16. Save fetching state for blobs pruning

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    a2b8c6e View commit details
    Browse the repository at this point in the history

  17. Log info loaded from disk

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    94aa2ce View commit details
    Browse the repository at this point in the history

  18. Delete blobs that conflict with finalization

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    c7f53a9 View commit details
    Browse the repository at this point in the history

  19. Store orphan block roots

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    8752dee View commit details
    Browse the repository at this point in the history

  20. Prune blobs in bg after canonical head update

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    2f565d2 View commit details
    Browse the repository at this point in the history

  21. Enable skipping blob pruning at each epoch

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    6346c30 View commit details
    Browse the repository at this point in the history

  22. fixup! Store orphan block roots

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    d58a30b View commit details
    Browse the repository at this point in the history

  23. Avoid repeteadly updating blob info for multiple head candidates

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    fb2ce90 View commit details
    Browse the repository at this point in the history

  24. Convert epochs_per_blob_prune to Epoch once

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    b5abfe6 View commit details
    Browse the repository at this point in the history

  25. Fix epoch constructor misconception

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    0d13932 View commit details
    Browse the repository at this point in the history

  26. Simplify conceptual design

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    7103a25 View commit details
    Browse the repository at this point in the history

  27. fixup! Simplify conceptual design

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    2056775 View commit details
    Browse the repository at this point in the history

  28. fixup! Simplify conceptual design

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    44ec331 View commit details
    Browse the repository at this point in the history

  29. Fix type bug 

    Co-authored-by: realbigsean <seananderson33@GMAIL.com>
    @emhane @realbigsean
    emhane and realbigsean committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    3d93dad View commit details
    Browse the repository at this point in the history

  30. Ignore IDE file

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    74172ed View commit details
    Browse the repository at this point in the history

  31. Clarify hybrid blob prune solution and fix error handling

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    83a9520 View commit details
    Browse the repository at this point in the history

  32. fixup! Clarify hybrid blob prune solution and fix error handling

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    54699f8 View commit details
    Browse the repository at this point in the history

  33. Fix typo

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    3bede06 View commit details
    Browse the repository at this point in the history

  34. Fix blobs store bug

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    a875bec View commit details
    Browse the repository at this point in the history

  35. Run prune blobs on migrator thread

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    caa04db View commit details
    Browse the repository at this point in the history

  36. Only store non-empty orphaned blobs

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    0bdc291 View commit details
    Browse the repository at this point in the history

  37. Fix typo

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    d1b75e2 View commit details
    Browse the repository at this point in the history

  38. Plug in running blob pruning in migrator, related bug fixes and add t… 

    …odos
    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    d7fc24a View commit details
    Browse the repository at this point in the history

  39. Allow user to set an epoch margin for pruning

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    1812301 View commit details
    Browse the repository at this point in the history

  40. fixup! Allow user to set an epoch margin for pruning

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    4de523f View commit details
    Browse the repository at this point in the history

  41. Keep uniform size small keys 

    Co-authored-by: Michael Sproul <micsproul@gmail.com>
    @emhane @michaelsproul
    emhane and michaelsproul committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    756c881 View commit details
    Browse the repository at this point in the history

  42. Clarify wording 

    Co-authored-by: Michael Sproul <micsproul@gmail.com>
    @emhane @michaelsproul
    emhane and michaelsproul committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    e4b4473 View commit details
    Browse the repository at this point in the history

  43. Clarify comment 

    Co-authored-by: Michael Sproul <micsproul@gmail.com>
    @emhane @michaelsproul
    emhane and michaelsproul committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    f6346f8 View commit details
    Browse the repository at this point in the history

  44. Fix wording 

    Co-authored-by: Michael Sproul <micsproul@gmail.com>
    @emhane @michaelsproul
    emhane and michaelsproul committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    c50f831 View commit details
    Browse the repository at this point in the history

  45. Prune from highest data availability boundary

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    63ca3bf View commit details
    Browse the repository at this point in the history

  46. fixup! Fix blobs store bug

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    43c3c74 View commit details
    Browse the repository at this point in the history

  47. fixup! Prune from highest data availability boundary

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    d479560 View commit details
    Browse the repository at this point in the history

  48. Reflect use of prune margin epochs at import

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    9c2e623 View commit details
    Browse the repository at this point in the history

  49. Improve naming

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    5d2480c View commit details
    Browse the repository at this point in the history

  50. Atomically update blob info with pruned blobs

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    6dff69b View commit details
    Browse the repository at this point in the history

  51. Remove unused stuff

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    9ee9b6d View commit details
    Browse the repository at this point in the history

  52. Add tests for blob pruning flags

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    1e59cb9 View commit details
    Browse the repository at this point in the history

  53. Correct comment

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    a2eda76 View commit details
    Browse the repository at this point in the history

  54. fixup! Allow user to set an epoch margin for pruning

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    8f137df View commit details
    Browse the repository at this point in the history

  55. Make implementation of BlobInfo more coder friendly

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    00ca21e View commit details
    Browse the repository at this point in the history

  56. Verify StoreConfig

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    b2abec5 View commit details
    Browse the repository at this point in the history

  57. Fix conflicts rebasing eip4844

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    56c8417 View commit details
    Browse the repository at this point in the history

  58. Improve use of whitespace 

    Co-authored-by: Michael Sproul <micsproul@gmail.com>
    @emhane @michaelsproul
    emhane and michaelsproul committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    577262c View commit details
    Browse the repository at this point in the history

  59. Remove debug comment 

    Co-authored-by: Michael Sproul <micsproul@gmail.com>
    @emhane @michaelsproul
    emhane and michaelsproul committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    d599e41 View commit details
    Browse the repository at this point in the history

  60. Reorder loading of db metadata from disk to allow for future changes … 

    …to schema
    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    d7eb944 View commit details
    Browse the repository at this point in the history

  61. Removed unused code

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    9d91991 View commit details
    Browse the repository at this point in the history

  62. Fix regression in DB write atomicity

    @michaelsproul @emhane
    michaelsproul authored and emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    ac4b5b5 View commit details
    Browse the repository at this point in the history

  63. fixup! Improve use of whitespace

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    bc468b4 View commit details
    Browse the repository at this point in the history

  64. fixup! Fix regression in DB write atomicity

    @emhane
    emhane committed Feb 8, 2023
    Configuration menu
    Copy the full SHA
    6a37e84 View commit details
    Browse the repository at this point in the history