Skip to content

Commit

Permalink
GH-1316: Elasticsearch vector store - Wrong error reported fix
Browse files Browse the repository at this point in the history
Fixes: #1316

* Handling missing index case
* Docs update
* User header for observability
* Same exception management for delete
  • Loading branch information
l-trotta committed Sep 9, 2024
1 parent 5b6a60d commit 4f62ac1
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 12 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -50,12 +50,14 @@ TIP: Refer to the xref:getting-started.adoc#repositories[Repositories] section t


The vector store implementation can initialize the requisite schema for you, but you must opt-in by specifying the `initializeSchema` boolean in the appropriate constructor or by setting `...initialize-schema=true` in the `application.properties` file.
Alternatively you can opt-out the initialization and create the index manually using the Elasticsearch client, which can be useful if the index needs advanced mapping or additional configuration.

NOTE: this is a breaking change! In earlier versions of Spring AI, this schema initialization happened by default.



Please have a look at the list of <<elasticsearchvector-properties,configuration parameters>> for the vector store to learn about the default values and configuration options.
These properties can be also set by configuring the `ElasticsearchVectorStoreOptions` bean.

Additionally, you will need a configured `EmbeddingModel` bean. Refer to the xref:api/embeddings.adoc#available-implementations[EmbeddingModel] section for more information.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -105,27 +105,28 @@ public ElasticsearchVectorStore(ElasticsearchVectorStoreOptions options, RestCli
Objects.requireNonNull(embeddingModel, "EmbeddingModel must not be null");
this.elasticsearchClient = new ElasticsearchClient(new RestClientTransport(restClient, new JacksonJsonpMapper(
new ObjectMapper().configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false))));
elasticsearchClient.withTransportOptions(t -> t.addHeader("user-agent", "spring-ai"));
this.embeddingModel = embeddingModel;
this.options = options;
this.filterExpressionConverter = new ElasticsearchAiSearchFilterExpressionConverter();
}

@Override
public void doAdd(List<Document> documents) {
// For the index to be present, either it must be pre-created or set the
// initializeSchema to true.
if (!indexExists()) {
throw new IllegalArgumentException("Index not found");
}
BulkRequest.Builder bulkRequestBuilder = new BulkRequest.Builder();

for (Document document : documents) {
if (Objects.isNull(document.getEmbedding()) || document.getEmbedding().length == 0) {
logger.debug("Calling EmbeddingModel for document id = " + document.getId());
document.setEmbedding(this.embeddingModel.embed(document));
}
// We call operations on BulkRequest.Builder only if the index exists.
// For the index to be present, either it must be pre-created or set the
// initializeSchema to true.
if (indexExists()) {
bulkRequestBuilder.operations(op -> op
.index(idx -> idx.index(this.options.getIndexName()).id(document.getId()).document(document)));
}
bulkRequestBuilder.operations(op -> op
.index(idx -> idx.index(this.options.getIndexName()).id(document.getId()).document(document)));
}
BulkResponse bulkRequest = bulkRequest(bulkRequestBuilder.build());
if (bulkRequest.errors()) {
Expand All @@ -141,13 +142,13 @@ public void doAdd(List<Document> documents) {
@Override
public Optional<Boolean> doDelete(List<String> idList) {
BulkRequest.Builder bulkRequestBuilder = new BulkRequest.Builder();
// We call operations on BulkRequest.Builder only if the index exists.
// For the index to be present, either it must be pre-created or set the
// initializeSchema to true.
if (indexExists()) {
for (String id : idList) {
bulkRequestBuilder.operations(op -> op.delete(idx -> idx.index(this.options.getIndexName()).id(id)));
}
if (!indexExists()) {
throw new IllegalArgumentException("Index not found");
}
for (String id : idList) {
bulkRequestBuilder.operations(op -> op.delete(idx -> idx.index(this.options.getIndexName()).id(id)));
}
return Optional.of(bulkRequest(bulkRequestBuilder.build()).errors());
}
Expand Down

0 comments on commit 4f62ac1

Please sign in to comment.