Skip to content

Fix defragment_symbol_data dropping user metadata (#9889436827)#2991

Open
himsheda wants to merge 3 commits intoman-group:masterfrom
himsheda:fix/defragment-symbol-data-drops-metadata
Open

Fix defragment_symbol_data dropping user metadata (#9889436827)#2991
himsheda wants to merge 3 commits intoman-group:masterfrom
himsheda:fix/defragment-symbol-data-drops-metadata

Conversation

@himsheda
Copy link
Copy Markdown

@himsheda himsheda commented Mar 28, 2026

defragment_symbol_data_impl passed std::nullopt to collate_and_write for user_meta, silently discarding the metadata that had been correctly read into pipeline_context->user_meta_ from the previous version. Fix by passing the actual user_meta_ptr from the pipeline context.

Also fix the Python wrapper which hardcoded metadata=None in the returned VersionedItem regardless of what was stored; it now reads the metadata back from the newly-written version using the existing read_metadata pattern.

Add regression test test_defragment_preserves_metadata.

Reference Issues/PRs

#2611

What does this implement or fix?

Any other comments?

Checklist

Checklist for code changes...
  • Have you updated the relevant docstrings, documentation and copyright notice?
  • Is this contribution tested against all ArcticDB's features?
  • Do all exceptions introduced raise appropriate error messages?
  • Are API changes highlighted in the PR description?
  • Is the PR labelled as enhancement or bug so it appears in autogenerated release notes?

@himsheda himsheda force-pushed the fix/defragment-symbol-data-drops-metadata branch from 765da9e to b313aeb Compare March 30, 2026 10:02
timestamp=result.timestamp,
)
version_query = self._get_version_query(result.version)
_, udm = self.version_store.read_metadata(symbol, version_query)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The metadata should already available in the C++ layer at this point. Consider returning it directly to avoid extra API call and storage read.

Copy link
Copy Markdown
Collaborator

@phoebusm phoebusm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#2991 (comment)
Returning the metadata directly from C++ maybe too much scope for this PR. It would be sufficient to just update the docstring in Library to document that the returned metadata will be None

@himsheda himsheda requested a review from phoebusm March 30, 2026 20:19
@himsheda himsheda force-pushed the fix/defragment-symbol-data-drops-metadata branch from f6539a8 to 8ca9081 Compare April 1, 2026 15:53
@himsheda
Copy link
Copy Markdown
Author

himsheda commented Apr 1, 2026

#2991 (comment) Returning the metadata directly from C++ maybe too much scope for this PR. It would be sufficient to just update the docstring in Library to document that the returned metadata will be None

Handled. @phoebusm Please have a look

himsheda added 3 commits April 2, 2026 21:09
defragment_symbol_data_impl passed std::nullopt to collate_and_write for
user_meta, silently discarding the metadata that had been correctly read
into pipeline_context->user_meta_ from the previous version. Fix by
passing the actual user_meta_ptr from the pipeline context.

Also fix the Python wrapper which hardcoded metadata=None in the returned
VersionedItem regardless of what was stored; it now reads the metadata back
from the newly-written version using the existing read_metadata pattern.

Add regression test test_defragment_preserves_metadata.
@himsheda himsheda force-pushed the fix/defragment-symbol-data-drops-metadata branch from 8ca9081 to 2ff5427 Compare April 2, 2026 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants