Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add batching strategy for all vector store implementations #1261

Open
8 of 15 tasks
sobychacko opened this issue Aug 21, 2024 · 0 comments
Open
8 of 15 tasks

Add batching strategy for all vector store implementations #1261

sobychacko opened this issue Aug 21, 2024 · 0 comments
Labels
vector store

Comments

@sobychacko
Copy link
Contributor

sobychacko commented Aug 21, 2024 •
edited by markpollack
Loading

See this for more details: #1214

  • Cassandra
  • Chroma
  • ElasticSearch
  • Gemfire
  • Hanadb
  • MongoDB Atlas
  • Neo4j
  • Opensearch
  • Oracle
  • PgVectorStore
  • Pinecone
  • Qdrant
  • Redis
  • Typesense
  • Weaviate
sobychacko added a commit to sobychacko/spring-ai that referenced this issue Sep 3, 2024
@sobychacko
PgVectorStore: Add batching strategry for adding documents
d4353cd 
Sub batch documents based on a BatchingStrategy when adding documents to PgVectorStore

Related to spring-projects#1261
@sobychacko sobychacko mentioned this issue Sep 3, 2024
Closed
markpollack pushed a commit that referenced this issue Sep 4, 2024
@sobychacko @markpollack
Add batching strategy for embedding documents in PgVectorStore
087de16 
- Precompute all embeddings using a BatchingStrategy before inserting into the vector store

This optimization improves efficiency when adding multiple documents

Related to #1261
markpollack pushed a commit that referenced this issue Sep 9, 2024
@sobychacko @markpollack
Add batching for more vector stores
15fdd05 
Apply batching when adding Documents to the following vector stores:

- Chroma
- ElasticSearch
- Neo4j
- Qdrant
- Redis
- Typesense
- Weaviate

This improves efficiency by processing multiple Documents at once instead of individually, reducing the overhead for each operation.

Related to #1261
sobychacko added a commit to sobychacko/spring-ai that referenced this issue Sep 9, 2024
@sobychacko
Add batching strategy to more vector stores
8c00297 
 Apply batching when adding Documents to the following vector stores:

- Azure vectore store
- Cassanra
- MongoDB Atlas
- OpenSearch
- Oracle
- Pinecone

 This improves efficiency by processing multiple Documents at once instead of individually, reducing the overhead for each operation.

Related to spring-projects#1261
sobychacko added a commit to sobychacko/spring-ai that referenced this issue Sep 9, 2024
@sobychacko
Add batching strategy to more vector stores
1c7754e 
 Apply batching when adding Documents to the following vector stores:

- Azure vector store
- Cassandra
- MongoDB Atlas
- OpenSearch
- Oracle
- Pinecone

 This improves efficiency by processing multiple Documents at once instead of individually, reducing the overhead for each operation.

Related to spring-projects#1261
@sobychacko sobychacko mentioned this issue Sep 9, 2024
Closed
markpollack pushed a commit that referenced this issue Sep 17, 2024
@sobychacko
Add batching strategy to more vector stores
66455b9 
 Apply batching when adding Documents to the following vector stores:

- Azure vector store
- Cassandra
- MongoDB Atlas
- OpenSearch
- Oracle
- Pinecone

 This improves efficiency by processing multiple Documents at once instead of individually, reducing the overhead for each operation.

Related to #1261
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
vector store
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@markpollack @sobychacko