Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(sequencer)!: update storage keys locations and values (ENG-898) #1616

Merged
merged 8 commits into from
Oct 9, 2024

Conversation

Fraser999
Copy link
Contributor

Summary

The storage keys for all components were moved from the top of state_ext modules into new storage::keys modules. The key values were updated to a consistent style.

Background

We wanted the keys to be easy to find and to follow a consistent pattern in terms of their values and locations.

Changes

  • All key consts and functions were moved to <COMPONENT>/storage/keys.
  • All keys were given a prefix matching the component name followed by a /.
  • All keys now use full words (except for a few instances of well-know abbreviations like "init") and _ as word separators.
  • Existing helper structs in various state_ext modules for formatting keys were unified into a new struct AddressPrefixer in crate::storage.
  • The Asset functionality was moved from crate::storage_keys to crate::storage.

Testing

  • All keys have snapshot tests (including const keys - this might be considered overkill?)
  • All keys have tests ensuring they have a component name as prefix.
  • All prefix consts or functions which are used in state_ext modules are tested to ensure they are actually prefixes of the given keys.

Breaking Changelist

  • This breaks the on-disk format of both the verifiable and non-verifiable store.

Related Issues

Closes #1611.

@github-actions github-actions bot added the sequencer pertaining to the astria-sequencer crate label Oct 2, 2024
Copy link
Member

@SuperFluffy SuperFluffy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thank you for going through all the keys and streamlining them.

I very much like that we have full (I think?) coverage of all keys and prefixes.

Since this is a breaking change and a rework of how our fees are structured, I think we should break with our previous key and make their types and (where applicable) byte-encoding consistent everywhere.

Some open points that I think should be addressed before merge:

  1. make the type of all keys consistently &str/String. Right now we have a string representation in some places, and raw byte slices in others. It's not clear why that's done. Yet especially for snapshot tests it's very valuable to have a human readable representation.
  2. Apart from ibc-prefixed assets (which are "ibc/<hex-encoded-32-bytes>", all key/prefix parts that involve raw bytes (like account) should be base64 encoded to save space.
  3. I don't have a strong opinion on only using the byte part of ibc-prefixed assets when constructing keys from them. I mildly lean toward encoding the full string (including "ibc/") and not just the bytes.

crates/astria-core/src/primitive/v1/asset/denom.rs Outdated Show resolved Hide resolved
crates/astria-sequencer/src/storage/keys.rs Outdated Show resolved Hide resolved
crates/astria-sequencer/src/storage/keys.rs Outdated Show resolved Hide resolved
crates/astria-sequencer/src/ibc/storage/keys.rs Outdated Show resolved Hide resolved
crates/astria-sequencer/src/ibc/state_ext.rs Outdated Show resolved Hide resolved
crates/astria-sequencer/src/assets/storage/keys.rs Outdated Show resolved Hide resolved
crates/astria-sequencer/src/assets/storage/keys.rs Outdated Show resolved Hide resolved
crates/astria-sequencer/src/assets/storage/keys.rs Outdated Show resolved Hide resolved
crates/astria-sequencer/src/assets/state_ext.rs Outdated Show resolved Hide resolved
crates/astria-sequencer/src/app/storage/keys.rs Outdated Show resolved Hide resolved
Copy link
Member

@SuperFluffy SuperFluffy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thank you.

I wonder if in this PR we should just change base64 encoding to the url safe alphabet everywhere (so that we don't have a different encoding in the storage keys and the normal display impl).

crates/astria-sequencer/src/grpc/storage/keys.rs Outdated Show resolved Hide resolved
@Fraser999 Fraser999 added this pull request to the merge queue Oct 9, 2024
Merged via the queue into main with commit bb6c435 Oct 9, 2024
44 checks passed
@Fraser999 Fraser999 deleted the fraser/update-storage-keys branch October 9, 2024 10:03
@Fraser999
Copy link
Contributor Author

I wonder if in this PR we should just change base64 encoding to the url safe alphabet everywhere (so that we don't have a different encoding in the storage keys and the normal display impl).

Ticketed for follow-up at #1649.

steezeburger added a commit that referenced this pull request Oct 11, 2024
* main:
  feat(sequencer)!: enforce block ordering by transaction group  (#1618)
  fix(cli): ensure checkTx passes before waiting for inclusion (#1636)
  chore(sequencer)!: update storage keys locations and values (ENG-898) (#1616)
  chore: cargo audit warning (#1644)
  chore(proto, core)!: remove action suffix from all action types (#1630)
  feat(sequencer)!: add limit to total amount of transactions in parked  (#1638)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sequencer pertaining to the astria-sequencer crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update storage keys in sequencer
2 participants