Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Integrating live document information during search in the faiss engine #2275

Closed
heemin32 opened this issue Nov 15, 2024 · 1 comment
Labels
duplicate This issue or pull request already exists enhancement

Comments

@heemin32
Copy link
Collaborator

In the Faiss engine, when no filtering clause is applied, the search initially considers all available documents in a segment, and deleted documents are filtered out afterward.

This approach affects both recall and latency when there are many deleted documents. For ANN search, recall decreases because the results may include deleted documents among the top k. For exact search, latency increases due to the additional computation of distances for deleted documents.

This applies only with Faiss engine. With Lucene engine the live document information is already being used during search.

@navneet1v
Copy link
Collaborator

@heemin32 this same feature is discussed in this github issue: #1491 . Lets track this feature at 1 place only.

@navneet1v navneet1v added duplicate This issue or pull request already exists and removed untriaged labels Nov 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
duplicate This issue or pull request already exists enhancement
Projects
None yet
Development

No branches or pull requests

2 participants