Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better zero-copy support for keys and values that can be trivially converted into Slice #100

Merged
merged 2 commits into from
Dec 20, 2024

Conversation

carlsverre
Copy link
Contributor

@carlsverre carlsverre commented Dec 3, 2024

closes #99
depends on fjall-rs/value-log#11
depends on fjall-rs/lsm-tree#85

@carlsverre
Copy link
Contributor Author

Tests will fail until fjall-rs/value-log#11 ships. Unless you'd prefer me change the tests.

@marvin-j97
Copy link
Collaborator

marvin-j97 commented Dec 3, 2024

Into<Slice> should probably be used also in some other places:

https://github.com/fjall-rs/fjall/blob/2d804edf992a600059db96f1b1b2dcc637a50271/src/partition/mod.rs#L857C9-L857C18

pub fn remove<K: AsRef<[u8]>>(&self, key: K) -> crate::Result<()> {

pub(super) fn insert<K: AsRef<[u8]>, V: AsRef<[u8]>>(

https://github.com/fjall-rs/fjall/blob/2d804edf992a600059db96f1b1b2dcc637a50271/src/tx/write/mod.rs#L379C8-L382C30

pub fn insert<K: AsRef<[u8]>, V: AsRef<[u8]>>(

etc. Let me know if you find more...

And actually this may bubble down all the way down to lsm-tree 🥹: https://github.com/fjall-rs/lsm-tree/blob/11479d7a9c24531fb9174c7c3f309735931db4a8/src/abstract.rs#L584C5-L584C14

@carlsverre
Copy link
Contributor Author

I went through and updated all writing methods to support Into<Slice> on both key and value. I did not change read methods as they mostly just read keys and take references. And Keys are fairly small so the cost is less. To update those methods to take Into<Slice> would require more work in the range conversion code.

@carlsverre carlsverre changed the title Zero-copy support in Batch Better zero-copy support for keys and values that can be triviallyl converted into Slice Dec 4, 2024
@carlsverre carlsverre changed the title Better zero-copy support for keys and values that can be triviallyl converted into Slice Better zero-copy support for keys and values that can be trivially converted into Slice Dec 4, 2024
@marvin-j97 marvin-j97 changed the base branch from main to 2.5.0 December 20, 2024 14:05
@marvin-j97 marvin-j97 merged commit 84afe43 into fjall-rs:2.5.0 Dec 20, 2024
0 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

zero-copy support for Batch
2 participants