ingest: Update ingestion library to be able to read hot archive bucket list #5827
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for reading evicted entries from the hot archive bucket list to the ingestion library. The changes enable CAP-0076 use cases that require examining hot archive bucket lists.
Key changes:
- Added
NewHotArchiveReaderfunction to create a reader for hot archive bucket lists - Refactored bucket reading logic to support both live and hot archive bucket list types
- Added example script demonstrating usage for finding expiring SAC balances
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
ingest/checkpoint_change_reader.go |
Core implementation for hot archive bucket list support |
ingest/checkpoint_change_reader_test.go |
Updated test calls to use new readBucketRecord method signature |
ingest/tutorial/expiring-sac-balances/main.go |
Example script showing how to find expiring SAC balances using both live and hot archive readers |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
|
||
| return rdr, e | ||
| } | ||
|
|
There was a problem hiding this comment.
This function lacks documentation. Add a comment explaining its purpose: iterating over hot archive bucket entries and yielding valid ledger entries while handling metadata and deduplication.
| // hotArchiveIterator iterates over entries in a hot archive bucket XDR stream, | |
| // yielding valid xdr.LedgerEntry values while handling bucket metadata entries | |
| // and deduplicating entries as needed. It returns an iterator function that | |
| // yields each ledger entry and any associated error. The function ensures that | |
| // only valid entries are processed, and that metadata (METAENTRY) is handled | |
| // according to protocol rules. |
Copilot uses AI. Check for mistakes.
| }() | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
This function lacks documentation. Add a comment explaining its purpose: iterating over live bucket entries and yielding valid ledger entries while handling protocol versions and entry types.
| // liveIterator iterates over live bucket entries in the provided XDR stream, | |
| // yielding valid ledger entries while handling protocol versions and entry types. |
Copilot uses AI. Check for mistakes.
| "CDTKPWPLOURQA2SGTKTUQOWRCBZEORB4BWBOMJ3D3ZTQQSGE5F6JBQLV": "EURC", | ||
| "CCW67TSZV3SSS2HXMBQ5JFGCKJNXKZM7UQUWUZPUTHXSTZLEO7SJMI75": "USDC", | ||
| } | ||
| var cutoffLedger uint32 = 60739011 |
There was a problem hiding this comment.
Hard-coded ledger sequence should be configurable. Consider accepting this as a command-line argument or environment variable to make the script more flexible for different use cases.
Copilot uses AI. Check for mistakes.
| targetAssets := map[string]string{ | ||
| "CDTKPWPLOURQA2SGTKTUQOWRCBZEORB4BWBOMJ3D3ZTQQSGE5F6JBQLV": "EURC", | ||
| "CCW67TSZV3SSS2HXMBQ5JFGCKJNXKZM7UQUWUZPUTHXSTZLEO7SJMI75": "USDC", | ||
| } |
There was a problem hiding this comment.
Hard-coded asset mappings should be configurable. Consider reading these from a configuration file or accepting them as command-line arguments to make the script reusable for different asset sets.
Copilot uses AI. Check for mistakes.
1 participant
PR Checklist
PR Structure
otherwise).
services/friendbot, orallordocif the changes are broad or impact manypackages.
Thoroughness
.mdfiles, etc... affected by this change). Take a look in the
docsfolder for a given service,like this one.
Release planning
CHANGELOG.mdwithin the component folder structure. For example, if I changed horizon, then I updated (services/horizon/CHANGELOG.md. I add a new line item describing the change and reference to this PR. If I don't update a CHANGELOG, I acknowledge this PR's change may not be mentioned in future release notes.semver, or if it's mainly a patch change. The PR is targeted at the next
release branch if it's not a patch change.
What
[TODO: Short statement about what is changing.]
Allow the checkpoint change reader to iterate over evicted entries in the hot archive bucket list.
Also, add an example script which makes use of that functionality.
Why
CAP-0076 has demonstrated a case where it is useful to examine the hot archive bucket list. We should add this functionality to the ingestion library.
Known limitations
[N/A]