Badger -> Pebble DB M2 - DB access refactoring for low-risk data (AN, EN, VN)

[j1010001]

Why (objective)

For top-level overview see Epic: #6515

Specifically for this issue:
Subset of Execution data gets indexed on ANs (events, Txs, Tx results, collections (meta object on top of Tx) ⇒ data that gets pulled out of execution data blobs).
ANs are the biggest data-heavy node after EN, expecting to grow further with EVM equivalence. By enabling data pruning we enable them to run AN with index on lower spec HW.
Most of this work is also relevant for VNs and because VNs are stateless we can cover it in this milestone as there is low risk of unintended consequences of DB operations.

Current status

• Execution data is stored in Badger DB.
• @zhangchiqing abstracted the DB access in [Badger] Add universal database operations #6465, which makes it easier to swap the DBs in the future.

How will we measure success ? (Key results)

• Data access is refactored for Access, Verification and Execution nodes.

DACI

Role	Assigned
Driver	Technical: @zhangchiqing , EM: @j1010001
Approver	@Kay-Zee
Consulted	@AlexHentschel, @franklywatson
Informed	Flow protocol Engineering team

Tasks

Execution

• Execution Result [Storage Refactor] Refactor saving execution results #6906
• Collections [Execution] Refactor collections storage #7059
• StopControl (VersionBeacon) [Storage] Refactor storage version beacon #7085
• Result GCP Uploader [Storage] Refactor computation result status storage #7084
• Switching from Badger to Pebble
  • Migrate the last executed result to pebble in order to make next block executable [Execution] Migrate last executed block from badger to pebble #7117

Verification

• Approvals [Storage Refactor] Refactor Approvals  #6868
• ChunkQueue [Storage Refactor] Refactor Chunk Locators to badger updates #6947
• Switching from Badger to Pebble [Storage Refactor] Using flags to roll out database operation changes for Verification nodes. #6948

Access

• Execution Result [Storage Refactor] Refactor saving execution results #6906
• Collections [Access] Refactor storage collections for access node #7093
• StopControl (VersionBeacon) [Storage] Refactor storage version beacon #7085
• Migrate the latest indexed height and modify storage layer to query both Pebble and Badger DBs: Database Operation Transition: From Badger to Pebble #6523 (similar to [Execution] Migrate last executed block from badger to pebble #7117)

Tools

• Utility
  • add flags to read from pebble [Util] update util to read results from pebble #7092
  • Test if pebble can read data without stopping the process by creating a checkpoint of db on the fly. [Util] update util to read results from pebble #7092
• AdminTool
  • If Utility can read pebble without stopping the process, we can get rid of the admin tool that reads data from pebble. [Util] update util to read results from pebble #7092