Description
Bug description
RedisVectorStore
return incorrect amount of results when calling similaritySearch
. It will only return the first 10 documents when topK
is greater than 10.
Environment
Spring Ai version: 1.0.0-SNAPSHOT
Java version: 17
Spring Boot version: 3.3.4
Steps to reproduce
Search with topK
greater than 10, then RedisVectorStore
will only return the first 10 documents.
@Test
void testSearch() {
List<Document> documents = List.of(
new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")),
new Document("The World is Big and Salvation Lurks Around the Corner"),
new Document("AAAAAA"),
new Document("BBBBBB"),
new Document("CCCCCC"),
new Document("DDDDDD"),
new Document("EEEEEE"),
new Document("FFFFFF"),
new Document("GGGGGG"),
new Document("HHHHHH"),
new Document("IIIIII"),
new Document("JJJJJJ"),
new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")));
// Add the documents to Redis
vectorStore.add(documents);
// Retrieve documents similar to a query
List<Document> results = vectorStore.similaritySearch(SearchRequest.query("Spring").withTopK(12));
results.forEach(System.out::println);
assertThat(results).hasSize(12);
}
Expected behavior
Return correct amount of documents.
Minimal Complete Reproducible example
@Test
void testSearch() {
List<Document> documents = List.of(
new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")),
new Document("The World is Big and Salvation Lurks Around the Corner"),
new Document("AAAAAA"),
new Document("BBBBBB"),
new Document("CCCCCC"),
new Document("DDDDDD"),
new Document("EEEEEE"),
new Document("FFFFFF"),
new Document("GGGGGG"),
new Document("HHHHHH"),
new Document("IIIIII"),
new Document("JJJJJJ"),
new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")));
// Add the documents to Redis
vectorStore.add(documents);
Fix suggest
Should add limit
to Query
according to Redis Document
limits the results to the offset and number of results given. Note that the offset is zero-indexed. The default is 0 10, which returns 10 items starting from the first result. You can use LIMIT 0 0 to count the number of documents in the result set without actually returning them.
Query query = new Query(queryString).addParam(EMBEDDING_PARAM_NAME, RediSearchUtil.toByteArray(embedding))
.returnFields(returnFields.toArray(new String[0]))
.setSortBy(DISTANCE_FIELD_NAME, true)
// Below line should be added
.limit(0, request.getTopK())
.dialect(2);
Activity
dafriz commentedon Sep 26, 2024
Confirmed bug and that your suggested fix works. Raised #1423 with fix.
markpollack commentedon Oct 6, 2024
thanks, closing this issue as it has been merged.