feat/fix: Support for FixedSizeList vectors, nested schemas, and UI text wrapping#14
feat/fix: Support for FixedSizeList vectors, nested schemas, and UI text wrapping#14mattiabonzi wants to merge 3 commits intolance-format:mainfrom
Conversation
c45d742 to
19b0dcc
Compare
|
Hey @mattiabonzi thanks for this PR, really nice work. The FixedSizeList fix is a genuine gap in the current code, the null scalar handling prevents real crashes, and the nested object renderer is a great addition. All 10 CI checks passed, but the PR now has merge conflicts against While you're in there, a small thing I noticed Inline styles in The actual functionality looks good. Happy to merge once the conflicts are resolved. Thanks again for taking the time on this! |
Description
This PR introduces several fixes for vector serialization and adds UI enhancements for viewing complex data structures. Specifically, it adds support for FixedSizeList vector types and improves the rendering of nested structs and lists. It also includes UI controls for text wrapping.
Bug Fixes (Commit:
fix: Enhance serialization and vector handling)FixedSizeListVector Support: Updatedapp.pyto check forpa.types.is_fixed_size_list(). Previously, strictly typed vectors (which are required for LanceDBIVF_PQindexes) were bypassed and fell back to raw text rendering. They now correctly trigger the vector preview/heatstrip UI.is_validcheck inserialize_value.pyto preventCannot export buffer from null Arrow Scalarerrors when encountering empty cells.StructScalar has no attribute 'field'errors, and refined the binary check so standard text strings (StringScalar) aren't accidentally Base64 encoded.New Features
1. Nested Schemas & Complex Objects (Commit:
feat: manage nested vectors and complex objects in the UI)app.pynow recursively walks through PyArrowStructsandLists, allowing it to detect and format vectors even when they are deeply nested inside complex objects.[object Object]JavaScript rendering with a clean, recursive DOM builder inapp.js. Complex structs and arrays now render as neatly indented, syntax-highlighted blocks.styles.css(.complex-object-wrapper,.co-list, etc.) keeping the JS DOM construction clean of inline styles.2. Dynamic Text Wrapping (Commit:
feat: add text wrapping).wrap-textclass to the data table, limiting cell width to600pxand enforcingbreak-wordwrapping. This allows users to instantly switch between a compact "spreadsheet" view and a readable "document" view for long text fields.Testing
FixedSizeList<Float32>vectors, and complex nested schemas (e.g., lists of structs containing vectors).0.24.3.Screenshot