Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

proof, trie: refactor MPTProofCache and add revive tests #129

Open
wants to merge 66 commits into
base: state_expiry_mvp0.1_dev
Choose a base branch
from
Open

proof, trie: refactor MPTProofCache and add revive tests #129

wants to merge 66 commits into from

Commits on Oct 17, 2023

  1. flags: add state expiry enable flag;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    fc19bf5 View commit details
    Browse the repository at this point in the history

  2. feat: add meta info

    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    e146978 View commit details
    Browse the repository at this point in the history

  3. feat: add path proof generation

    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    29adf96 View commit details
    Browse the repository at this point in the history

  4. feat: add revive operation 

    fix: verify proof first then only revive

remove comments

edit function comments

verify child hash while constructing trie from proof

remove unnecessary comments and parameters
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    887dfdc View commit details
    Browse the repository at this point in the history

  5. state/snapshot: support snapshot prune, epoch meta;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    3a98bb6 View commit details
    Browse the repository at this point in the history

  6. state/snapshot: support get/put in statedb; 

    state/statedb: support state expiry;
state/state_object: add revive trie, access state, etc.;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    48859f7 View commit details
    Browse the repository at this point in the history

  7. feat: implement epoch-based trie 

    fix minor test bug
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    abed248 View commit details
    Browse the repository at this point in the history

  8. feat: add eth_getStorageReviveProof 

    minor fix

change rpc blockNum to hash
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    7ce6beb View commit details
    Browse the repository at this point in the history

  9. ethdb/fullstatedb: add FullStateDB interface; 

    state/statedb: fetch expired state from remote;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    d700e95 View commit details
    Browse the repository at this point in the history

  10. core/state: implement state object revive storage trie 

    minor change to key

remove journal codes
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    48d0784 View commit details
    Browse the repository at this point in the history

  11. core/types: modify metadata type

    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    f0435dd View commit details
    Browse the repository at this point in the history

  12. core/state, trie: access trie updates with current epoch

    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    b393825 View commit details
    Browse the repository at this point in the history

  13. feat: add trie prune logic 

    apply comments
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    2c7c459 View commit details
    Browse the repository at this point in the history

  14. trie/epochmeta: add trie epoch meta storage; 

    trie/epochmeta: trie integrate epoch meta storage;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    22fc2ab View commit details
    Browse the repository at this point in the history

  15. trie/committer: support epoch meta commit;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    19fd26c View commit details
    Browse the repository at this point in the history

  16. pruner: support prune expired trie/snapshot kv;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    2350ed4 View commit details
    Browse the repository at this point in the history

  17. pruner: support prune expired trie/snapshot kv;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    cca3319 View commit details
    Browse the repository at this point in the history

  18. state/trieprefetcher: support handle expired state; 

    ethdb/fullstatedb: support proof cache;
trie/epochmeta: opt query & commit logic;
bugfix: fix remote full state db params;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    b8ce056 View commit details
    Browse the repository at this point in the history

  19. fix: revive storage trie from remoteDB error 

    more fixes

minor
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    794273c View commit details
    Browse the repository at this point in the history

  20. trie/proof: fix child hash validate bug; 

    bugfix: fix state copy issues, remote cache;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    7633607 View commit details
    Browse the repository at this point in the history

  21. fix: revive trie use nubs method 

    more fixes

Squashed commit of the following:

commit 45e7deacede8ebecd2625cc09c609d0b92f96515
Author: 0xbundler <124862913+0xbundler@users.noreply.github.com>
Date:   Mon Sep 4 11:50:49 2023 +0800

    trie/proof: fix child hash validate bug;

    bugfix: fix state copy issues, remote cache;
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    a6e4e30 View commit details
    Browse the repository at this point in the history

  22. core/state: fix revive error, trie prefetcher copy bug; 

    trie: commit account meta, revive state bug;
ethdb/fullstatedb: fix cache bug, cannot share by key;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    58e80ab View commit details
    Browse the repository at this point in the history

  23. logs: opt some state expiry logs level to debug; 

    pruner: fix some epoch meta db error;
metrics: add more metrics for state expiry;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    e0de4ac View commit details
    Browse the repository at this point in the history

  24. trie/trie: fix compile error;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    753658c View commit details
    Browse the repository at this point in the history

  25. bugfix: fix some proof generate, revive bugs; 

    fullstatedb: opt storage trie init method;
metrics: add more cost time metrics;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    0c86308 View commit details
    Browse the repository at this point in the history

  26. trie/epochmeta: opt graceful shutdown logic; 

    pruner: refactor expired prune;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    001a2c5 View commit details
    Browse the repository at this point in the history

  27. trie/inspect: add inspect trie tools;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    079a1c6 View commit details
    Browse the repository at this point in the history

  28. metrics: opt some timer, add more miner metrics;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    7ac0a0b View commit details
    Browse the repository at this point in the history

  29. fixbugs: fix prune initial； 

    trie/trie: fix some migrate issues;
trie: fix compile error;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    c9bd5b4 View commit details
    Browse the repository at this point in the history

  30. bugfix: fix trie update;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    91577fb View commit details
    Browse the repository at this point in the history

  31. bugfix: fix state prefetcher concurrent bugs; 

    bugfix: fix trie epoch update bugs;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    16f91a8 View commit details
    Browse the repository at this point in the history

  32. fix: rebase bsc/pbss_active conflicts

    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    4b78013 View commit details
    Browse the repository at this point in the history

  33. trie/proof: fix pbss trie proof generate bug;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    26e7967 View commit details
    Browse the repository at this point in the history

  34. pruner: support PBSS expired state prune;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    7dde14a View commit details
    Browse the repository at this point in the history

  35. rpc(revive): get from latest block num 

    modify

modify
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    85db375 View commit details
    Browse the repository at this point in the history

  36. core/state, trie: fix bad block 

    add TODO

api fix
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    43dde20 View commit details
    Browse the repository at this point in the history

  37. feat: insert new slot in snapshot, prevent execution touch trie; 

    feat: reuse prefetcher tire? it could fetch from remoteDB;
feat: revive from local first, then fetch from remote;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    c05a729 View commit details
    Browse the repository at this point in the history

  38. core/state: subfetcher revive state in batch mode

    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    e4235be View commit details
    Browse the repository at this point in the history

  39. trie: add unit tests

    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    634d117 View commit details
    Browse the repository at this point in the history

  40. api: add cache for storage revive proof 

    remove log
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    d9e7568 View commit details
    Browse the repository at this point in the history

  41. chore(cmd/geth): enable state expiry only with PBSS

    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    2bc3c9c View commit details
    Browse the repository at this point in the history

  42. pruner: opt size statistic; 

    trie/inspect: opt inspect in PBSS mode;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    c3a31d7 View commit details
    Browse the repository at this point in the history

  43. flags: refactor state expiry config;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    bdc2fc9 View commit details
    Browse the repository at this point in the history

  44. feat(snap): add state expiry support to snap sync 

    add server related for snap sync

edit
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    b9a07a1 View commit details
    Browse the repository at this point in the history

  45. pruner: fix some prune bugs;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    0772f61 View commit details
    Browse the repository at this point in the history

  46. trie/epochmeta: add bloom filter, opt lock;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    2032bfa View commit details
    Browse the repository at this point in the history

  47. metrics: add some trace metrics for epoch meta; 

    trie/trie: fix some update epoch bugs;
state/state_object: fix epoch update issue;
state/statedb: fix oom issue;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    5f776c2 View commit details
    Browse the repository at this point in the history

  48. core/state: fix snapshot recovery 

    a
    @asyukii @0xbundler
    asyukii authored and 0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    4b05dc4 View commit details
    Browse the repository at this point in the history

  49. trie: fix rebase error;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    4e7a438 View commit details
    Browse the repository at this point in the history

  50. trie: opt trie update; 

    trie: fix rebase error;
    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    811f37a View commit details
    Browse the repository at this point in the history

  51. pruner: opt expired prune, add more logs;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    7a7abb9 View commit details
    Browse the repository at this point in the history

  52. state: add some revive metrics;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    5a5f371 View commit details
    Browse the repository at this point in the history

  53. fix: revert to master change;

    @0xbundler
    0xbundler committed Oct 17, 2023
    Configuration menu
    Copy the full SHA
    74169f7 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2023

  1. trie/pathdb: support account meta/epoch meta query; 

    trie/trie: merge epoch meta & trie store in PBSS;
    @0xbundler
    0xbundler committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    e98a619 View commit details
    Browse the repository at this point in the history

  2. fix: missing epochMaps init

    @asyukii
    asyukii committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    a1a0d4d View commit details
    Browse the repository at this point in the history

Commits on Nov 3, 2023

  1. trie/pathdb: fix clean cache low hit rate issue; 

    trie/trie: reuse node cache, prevent resolve again;
    @0xbundler
    0xbundler committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    d2ad0ec View commit details
    Browse the repository at this point in the history

  2. prune: add more concurrent logics;

    @0xbundler
    0xbundler committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    272522b View commit details
    Browse the repository at this point in the history

  3. trie/typednode: opt encode/decode performance; 

    pruner: opt concurrent logic;
state/state_object: add access state in prefetch;
    @0xbundler
    0xbundler committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    7fb891c View commit details
    Browse the repository at this point in the history

  4. state/state_object: add more expired metrics; 

    state/state_object: add more expired metrics;
    @0xbundler
    0xbundler committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    c502191 View commit details
    Browse the repository at this point in the history

  5. eth/downloader: state expiry remotedb keep behind the latest;

    @0xbundler
    0xbundler committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    749bcc2 View commit details
    Browse the repository at this point in the history

  6. ethapi: opt storage proof logic;

    @0xbundler
    0xbundler committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    d24d379 View commit details
    Browse the repository at this point in the history

  7. state/stateobject: add dirtystorage touch & revive logic; 

    trie: fix local review leaf expand bug;
    @0xbundler
    0xbundler committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    52d8606 View commit details
    Browse the repository at this point in the history

  8. state/stateobject: fix revive in diff & delete shrink miss node bug; 

    trie: support revive from prefix;
    @0xbundler
    0xbundler committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    40a6e30 View commit details
    Browse the repository at this point in the history

  9. ut: fix some broken ut; 

    lint: fix some golang lint;
    @0xbundler
    0xbundler committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    128eb98 View commit details
    Browse the repository at this point in the history

  10. refactor: remove MPTProofCache 

    test(trie): fix TestRandom error

fix

fix

Revert "test(trie): fix TestRandom error"

This reverts commit c0bd559.

fix

msg
    @asyukii
    asyukii committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    7196140 View commit details
    Browse the repository at this point in the history

  11. test(trie): add random cases

    @asyukii
    asyukii committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    acea451 View commit details
    Browse the repository at this point in the history