Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce batching support for embedding documents #1214

Closed
sobychacko opened this issue Aug 13, 2024 · 3 comments
Closed

Introduce batching support for embedding documents #1214

sobychacko opened this issue Aug 13, 2024 · 3 comments
Assignees
@sobychacko

Comments

@sobychacko
Copy link
Contributor

Introduce a way for EmbeddingModel implementations to embed Document objects using a batching strategy.
@sobychacko
Copy link
Contributor Author

Using this #1140 as a starting point, introduce a new API for batching for all vector stores and embedding models to use.

@sobychacko sobychacko self-assigned this Aug 13, 2024
sobychacko added a commit to sobychacko/spring-ai that referenced this issue Aug 13, 2024
@sobychacko
spring-projectsGH-1214: Batching strategy for embedding documents
3999067 
 - When embedding documents, allow batching the documents using some criteria.
 - `BatchingStrategy` interface with a `TokenCountBatchingStrategy` implementation that uses
   the openai max input token size of 8191 as the default.
 - Add a default method in EmbeddingModel to embed document using this new batching strategy.
 - Change `MilvusVectorStore` to make use of this new batching API.
 - Adding unit tests for `TokenCountBatchingStrategy`.
 - Adding openai integration test to call the embed API that uses batching.

Resolves spring-projects#1214
@sobychacko sobychacko mentioned this issue Aug 13, 2024
Closed
@impactCn
Copy link
Contributor

@sobychacko hi, I also encountered the same problem in PG. The segmentation process I use , #1200 .
Can you guide me on how to use this batch API?

svrc pushed a commit to svrc/spring-ai that referenced this issue Aug 21, 2024
@sobychacko
Add Batching strategy for embedding documents
e44be0b 
 - When embedding documents, allow batching the documents using some criteria.
 - `BatchingStrategy` interface with a `TokenCountBatchingStrategy` implementation that uses
   the openai max input token size of 8191 as the default.
 - Add a default method in EmbeddingModel to embed document using this new batching strategy.
 - Change `MilvusVectorStore` to make use of this new batching API.
 - Adding unit tests for `TokenCountBatchingStrategy`.
 - Adding openai integration test to call the embed API that uses batching.

Resolves spring-projects#1214

Other vector stores will be updated seperately
@sobychacko sobychacko mentioned this issue Sep 3, 2024
Open
15 tasks
@sobychacko
Copy link
Contributor Author

@impactCn Sorry for the delay in responding. We just merged similar batching changes for the PG vector store. Can you take a look and see if that satisfies your use case? If there is a gap, we can improve on that. Thanks!

@sobychacko sobychacko mentioned this issue Sep 11, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sobychacko sobychacko

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

GH-1214: Batching strategy for embedding documents sobychacko/spring-ai
2 participants
@sobychacko @impactCn