Skip to content

[Bug] RedisVectorStore return incorrect amount of results. #1415

Closed
@Martin7-1

Description

@Martin7-1

Bug description

RedisVectorStore return incorrect amount of results when calling similaritySearch. It will only return the first 10 documents when topK is greater than 10.

Environment

Spring Ai version: 1.0.0-SNAPSHOT
Java version: 17
Spring Boot version: 3.3.4

Steps to reproduce

Search with topK greater than 10, then RedisVectorStore will only return the first 10 documents.

    @Test
    void testSearch() {

        List<Document> documents = List.of(
                new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")),
                new Document("The World is Big and Salvation Lurks Around the Corner"),
                new Document("AAAAAA"),
                new Document("BBBBBB"),
                new Document("CCCCCC"),
                new Document("DDDDDD"),
                new Document("EEEEEE"),
                new Document("FFFFFF"),
                new Document("GGGGGG"),
                new Document("HHHHHH"),
                new Document("IIIIII"),
                new Document("JJJJJJ"),
                new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")));

        // Add the documents to Redis
        vectorStore.add(documents);

        // Retrieve documents similar to a query
        List<Document> results = vectorStore.similaritySearch(SearchRequest.query("Spring").withTopK(12));

        results.forEach(System.out::println);

        assertThat(results).hasSize(12);
    }

Expected behavior

Return correct amount of documents.

Minimal Complete Reproducible example

    @Test
    void testSearch() {

        List<Document> documents = List.of(
                new Document("Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!! Spring AI rocks!!", Map.of("meta1", "meta1")),
                new Document("The World is Big and Salvation Lurks Around the Corner"),
                new Document("AAAAAA"),
                new Document("BBBBBB"),
                new Document("CCCCCC"),
                new Document("DDDDDD"),
                new Document("EEEEEE"),
                new Document("FFFFFF"),
                new Document("GGGGGG"),
                new Document("HHHHHH"),
                new Document("IIIIII"),
                new Document("JJJJJJ"),
                new Document("You walk forward facing the past and you turn back toward the future.", Map.of("meta2", "meta2")));

        // Add the documents to Redis
        vectorStore.add(documents);

Fix suggest

Should add limit to Query according to Redis Document

limits the results to the offset and number of results given. Note that the offset is zero-indexed. The default is 0 10, which returns 10 items starting from the first result. You can use LIMIT 0 0 to count the number of documents in the result set without actually returning them.

		Query query = new Query(queryString).addParam(EMBEDDING_PARAM_NAME, RediSearchUtil.toByteArray(embedding))
				.returnFields(returnFields.toArray(new String[0]))
				.setSortBy(DISTANCE_FIELD_NAME, true)
				// Below line should be added
				.limit(0, request.getTopK())
				.dialect(2);

Activity

dafriz

dafriz commented on Sep 26, 2024

@dafriz
Contributor

Confirmed bug and that your suggested fix works. Raised #1423 with fix.

self-assigned this
on Oct 3, 2024
added this to the 1.0.0-M3 milestone on Oct 3, 2024
markpollack

markpollack commented on Oct 6, 2024

@markpollack
Member

thanks, closing this issue as it has been merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

    Participants

    @markpollack@dafriz@Martin7-1

    Issue actions

      [Bug] `RedisVectorStore` return incorrect amount of results. · Issue #1415 · spring-projects/spring-ai