Releases: opensearch-project/k-NN
Releases · opensearch-project/k-NN
2.17.0.0
Version 2.17.0.0 Release Notes
Compatible with OpenSearch 2.17.0
Features
- Integrate Lucene Vector field with native engines to use KNNVectorFormat during segment creation #1945
- k-NN query rescore support for native engines #1984
- Add support for byte vector with Faiss Engine HNSW algorithm #1823
- Add support for byte vector with Faiss Engine IVF algorithm #2002
- Add mode/compression configuration support for disk-based vector search #2034
- Add spaceType as a top level optional parameter while creating vector field. #2044
Enhancements
- Adds iterative graph build capability into a faiss index to improve the memory footprint during indexing and Integrates KNNVectorsFormat for native engines#1950
- Add model version to model metadata and change model metadata reads to be from cluster metadata #2005
Bug Fixes
- Corrected search logic for scenario with non-existent fields in filter #1874
- Add script_fields context to KNNAllowlist [#1917] (#1917)
- Fix graph merge stats size calculation #1844
- Disallow a vector field to have an invalid character for a physical file name. #1936
- Fix memory overflow caused by cache behavior #2015
- Use correct type for binary vector in ivf training #2086
- Switch MINGW32 to MINGW64 #2090
Infrastructure
Maintenance
- Fix a flaky unit test:testMultiFieldsKnnIndex, which was failing due to inconsistent merge behaviors #1924
Refactoring
- Introduce KNNVectorValues interface to iterate on different types of Vector values during indexing and search #1897
- Integrate KNNVectorValues with vector ANN Search flow #1952
- Clean up parsing for query #1824
- Refactor engine package structure #1913
- Refactor method structure and definitions #1920
- Refactor KNNVectorFieldType from KNNVectorFieldMapper to a separate class for better readability. #1931
- Generalize lib interface to return context objects #1925
- Restructure mappers to better handle null cases and avoid branching in parsing #1939
- Added Quantization Framework and implemented 1Bit and multibit quantizer#1889
- Encapsulate dimension, vector data type validation/processing inside Library #1957
- Add quantization state cache #1960
- Add quantization state reader and writer #1997
2.16.0.0
Version 2.16.0.0 Release Notes
Compatible with OpenSearch 2.16.0
Features
- Adds dynamic query parameter ef_search #1783
- Adds dynamic query parameter ef_search in radial search faiss engine #1790
- Adds dynamic query parameter nprobes #1792
- Add binary format support with HNSW method in Faiss Engine #1781
- Add script scoring support for knn field with binary data type #1826
- Add painless script support for hamming with binary vector data type #1839
- Add binary format support with IVF method in Faiss Engine #1784
- Add support for Lucene inbuilt Scalar Quantizer #1848
Enhancements
- Switch from byte stream to byte ref for serde #1825
Bug Fixes
- Fixing the arithmetic to find the number of vectors to stream from java to jni layer.#1804
- Fixed LeafReaders casting errors to SegmentReaders when segment replication is enabled during search.#1808
- Release memory properly for an array type #1820
- FIX Same Suffix Cause Recall Drop to zero #1802
Infrastructure
- Apply custom patch only once by comparing the last patch id #1833
Documentation
- Update dev guide to fix clang linking issue on arm #1746
Maintenance
- Bump faiss commit to 33c0ba5 #1796
2.15.0.0
Version 2.15.0.0 Release Notes
Compatible with OpenSearch 2.15.0
Features
- Use the Lucene Distance Calculation Function in Script Scoring for doing exact search #1699
Enhancements
- Add KnnCircuitBreakerException and modify exception message #1688
- Add stats for radial search #1684
- Support script score when doc value is disabled and fix misusing DISI #1696
- Add validation for pq m parameter before training starts #1713
- Block delete model requests if an index uses the model #1722
Bug Fixes
2.14.0.0
Version 2.14.0.0 Release Notes
Compatible with OpenSearch 2.14.0
Features
- Add k-NN clear cache api #740
- Support radial search in k-NN plugin #1617
- Support filter and nested field in faiss engine radial search #1652
Enhancements
- Make the HitQueue size more appropriate for exact search #1549
- Implement the Streaming Feature to stream vectors from Java to JNI layer to enable creation of larger segments for vector indices #1604
- Remove unnecessary toString conversion of vector field and added some minor optimization in KNNCodec 1613
- Serialize all models into cluster metadata #1499
Bug Fixes
- Add stored fields for knn_vector type #1630
- Enable script score to work with model based indices #1649
Infrastructure
2.13.0.0
Version 2.13.0.0 Release Notes
Compatible with OpenSearch 2.13.0
Enhancements
- Optize Faiss Query With Filters: Reduce iteration and memory for id filter #1402
- Detect AVX2 Dynamically on the System #1502
- Validate zero vector when using cosine metric #1501
- Persist model definition in model metadata [#1527] (#1527)
- Added Inner Product Space type support for Lucene Engine #1551
- Add Range Validation for Faiss SQFP16 #1493
- SQFP16 Range Validation for Faiss IVF Models #1557
Bug Fixes
- Disable sdc table for HNSWPQ read-only indices #1518
- Switch SpaceType.INNERPRODUCT's vector similarity function to MAXIMUM_INNER_PRODUCT #1532
- Add patch to fix arm segfault in nmslib during ingestion #1541
- Share ivfpq-l2 table allocations across indices on load #1558
Infrastructure
- Manually install zlib for win CI #1513
- Update k-NN build artifact script to enable SIMD on ARM for Faiss #1543
Maintenance
2.12.0.0
Version 2.12.0.0 Release Notes
Compatible with OpenSearch 2.12.0
Features
Enhancements
- Increase Lucene max dimension limit to 16,000 #1346
- Tuned default values for ef_search and ef_construction for better indexing and search performance for vector search #1353
- Enabled Filtering on Nested Vector fields with top level filters #1372
- Throw proper exception to invalid k-NN query #1380
Bug Fixes
- Fix use-after-free case on nmslib search path #1305
- Allow nested knn field mapping when train model #1318
- Properly designate model state for actively training models when nodes crash or leave cluster #1317
- Fix script score queries not getting cached #1367
- Fix KNNScorer to apply boost #1403
- Fix equals and hashCode methods for KNNQuery and KNNQueryBuilder #1397
- Pass correct value on IDSelectorBitmap initialization #1444
Infrastructure
- Upgrade gradle to 8.4 1289
- Refactor security testing to install from individual components #1307
- Refactor integ tests that access model index #1423
- Fix flaky model tests #1429
Maintenance
2.11.0.0
2.10.0.0
Version 2.10.0.0 Release Notes
Compatible with OpenSearch 2.10.0
Features
- Add Clear Cache API (#740)
Enhancements
- Enabled the IVF algorithm to work with Filters of K-NN Query. (#1013)
- Improved the logic to switch to exact search for restrictive filters search for better recall. (#1059)
- Added max distance computation logic to enhance the switch to exact search in filtered Nearest Neighbor Search. (#1066)
Bug Fixes
- Update Faiss parameter construction to allow HNSW+PQ to work (#1074)
Maintenance
- Update Guava Version to 32.0.1 (#1019)
Refactoring
- Fix TransportAddress Refactoring Changes in Core (#1020)