Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] RedisVectorStore return incorrect amount of results. #1415

Open
Martin7-1 opened this issue Sep 26, 2024 · 1 comment · May be fixed by #1423
Open

[Bug] RedisVectorStore return incorrect amount of results. #1415

Martin7-1 opened this issue Sep 26, 2024 · 1 comment · May be fixed by #1423

Comments

@Martin7-1
Copy link

Martin7-1 commented Sep 26, 2024

Bug description

RedisVectorStore return incorrect amount of results when calling similaritySearch. It will only return the first 10 documents when topK is greater than 10.

Environment

Spring Ai version: 1.0.0-SNAPSHOT
Java version: 17
Spring Boot version: 3.3.4

Steps to reproduce

Search with topK greater than 10, then RedisVectorStore will only return the first 10 documents.

    @Test
    void testSearch() {

        List<Document> documents = List.of(
                new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")),
                new Document("The World is Big and Salvation Lurks Around the Corner"),
                new Document("AAAAAA"),
                new Document("BBBBBB"),
                new Document("CCCCCC"),
                new Document("DDDDDD"),
                new Document("EEEEEE"),
                new Document("FFFFFF"),
                new Document("GGGGGG"),
                new Document("HHHHHH"),
                new Document("IIIIII"),
                new Document("JJJJJJ"),
                new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")));

        // Add the documents to Redis
        vectorStore.add(documents);

        // Retrieve documents similar to a query
        List<Document> results = vectorStore.similaritySearch(SearchRequest.query("Spring").withTopK(12));

        results.forEach(System.out::println);

        assertThat(results).hasSize(12);
    }

Expected behavior

Return correct amount of documents.

Minimal Complete Reproducible example

    @Test
    void testSearch() {

        List<Document> documents = List.of(
                new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")),
                new Document("The World is Big and Salvation Lurks Around the Corner"),
                new Document("AAAAAA"),
                new Document("BBBBBB"),
                new Document("CCCCCC"),
                new Document("DDDDDD"),
                new Document("EEEEEE"),
                new Document("FFFFFF"),
                new Document("GGGGGG"),
                new Document("HHHHHH"),
                new Document("IIIIII"),
                new Document("JJJJJJ"),
                new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")));

        // Add the documents to Redis
        vectorStore.add(documents);

Fix suggest

Should add limit to Query according to Redis Document

limits the results to the offset and number of results given. Note that the offset is zero-indexed. The default is 0 10, which returns 10 items starting from the first result. You can use LIMIT 0 0 to count the number of documents in the result set without actually returning them.

		Query query = new Query(queryString).addParam(EMBEDDING_PARAM_NAME, RediSearchUtil.toByteArray(embedding))
				.returnFields(returnFields.toArray(new String[0]))
				.setSortBy(DISTANCE_FIELD_NAME, true)
				// Below line should be added
				.limit(0, request.getTopK())
				.dialect(2);
@dafriz
Copy link
Contributor

dafriz commented Sep 26, 2024

Confirmed bug and that your suggested fix works. Raised #1423 with fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants