Adding file mode option to serialize() methods#1071
Closed
chatman wants to merge 1 commit intorapidsai:branch-25.08from
Closed
Adding file mode option to serialize() methods#1071chatman wants to merge 1 commit intorapidsai:branch-25.08from
chatman wants to merge 1 commit intorapidsai:branch-25.08from
Conversation
mythrocks
reviewed
Jul 15, 2025
| endif() | ||
|
|
||
| list(APPEND CUVS_CUDA_FLAGS --expt-extended-lambda --expt-relaxed-constexpr) | ||
| list(APPEND CUVS_CUDA_FLAGS -allow-unsupported-compiler) |
Contributor
There was a problem hiding this comment.
Question: Why was this option required for nvcc?
mythrocks
reviewed
Jul 15, 2025
Comment on lines
+197
to
+199
| const char* filename, | ||
| cuvsBruteForceIndex_t index, | ||
| char file_mode); |
Contributor
There was a problem hiding this comment.
Is the indent off by 1 here?
Member
|
@chatman is this still relevant given the more recent move to serializing cagra out to Lucene HNSW format directly? |
Member
|
@chatman ping ^^ |
Contributor
Author
Contributor
Author
|
Closing this in favour of #1085. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In serialize() methods of CAGRA, Brute Force and HNSW, it would be preferable to add the ability to append to a file (ios::app). This will be useful for Lucene, where the file written to the segment contains a header, post which the serialized index could be written, by opening the file in append mode.
Also, useful for composite files (file containing multiple indexes). For this usecase, though, a deserialize() method where indexes from the same file can be loaded using offsets+length. This is not in scope for this PR, though.
In this case, the additional file_mode parameter can have 'a' for ios::app or 'w' for ios::out.