Skip to content

Conversation

normen662
Copy link
Contributor

@normen662 normen662 commented Sep 16, 2025

This PR implements a HNSW (Hierarchical Navigable Small World) structure as an part of fdb-extension according to https://arxiv.org/pdf/1603.09320.

  • the implementation is a basic structure that lives inside of a subspace in fdb
  • currently we store and retrieve vectors of class Vector that holds component data using Halfs which are FP16 encoded.
  • we currently only support insertion, batch insertion and retrieval of data via approximative nearest neighbor searches
  • we currently support the following options:
    • M the optimal number of connections between nodes within a layer of the HNSW
    • MMax the maximum number of connections between nodes within a layer of the HNSW
    • MMax0 the maximum number of connections between nodes within layer 0 of the HNSW
    • efConstruction the size of the search queue during thorough (non-greedy) searches during insertion
    • extendCandidates a boolean indicating if we should extend the candidates we can connect to to by also searching within the neighbors of already found candidates
    • keepPrunedConnections a boolean indicating whether under certain circumstances discarded candidates to connect to are reevaluated
  • deletes are not implemented (yet)
  • tests include basic testing using the SIFTsmall data set that is comprised of 10k vectors, 100 queries

@normen662 normen662 force-pushed the hnsw branch 2 times, most recently from 8a3c53c to 87c5674 Compare September 19, 2025 10:43
@normen662 normen662 added the enhancement New feature or request label Sep 19, 2025
@normen662 normen662 force-pushed the hnsw branch 2 times, most recently from fde3d47 to 597d08b Compare September 23, 2025 16:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant