Skip to content

Commit

Permalink
docs(onnx): Add a note when the onnxruntime_gpu dependency is missing.
Browse files Browse the repository at this point in the history
  • Loading branch information
dongfeng authored and tzolov committed Aug 11, 2024
1 parent 66e4b88 commit aeba9c2
Showing 1 changed file with 14 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ The complete list of supported properties are:
| spring.ai.embedding.transformer.cache.directory | Directory path to cache remote resources, such as the ONNX models | ${java.io.tmpdir}/spring-ai-onnx-model
| spring.ai.embedding.transformer.onnx.modelUri | Existing, pre-trained ONNX model. | onnx/all-MiniLM-L6-v2/model.onnx
| spring.ai.embedding.transformer.onnx.modelOutputName | The ONNX model's output node name, which we'll use for embedding calculation. | last_hidden_state
| spring.ai.embedding.transformer.onnx.gpuDeviceId | The GPU device ID to execute on. Only applicable if >= 0. Ignored otherwise. | -1
| spring.ai.embedding.transformer.onnx.gpuDeviceId | The GPU device ID to execute on. Only applicable if >= 0. Ignored otherwise.(Requires additional onnxruntime_gpu dependency) | -1
| spring.ai.embedding.transformer.metadataMode | Specifies what parts of the Documents content and metadata will be used for computing the embeddings. | NONE
|===

Expand Down Expand Up @@ -114,6 +114,19 @@ The `model.onnx_data` is called link:https://onnx.ai/onnx/repo-docs/ExternalData
Currently the only workaround is to copy the large `model.onnx_data` in the folder you run your Boot applicaiton.
====

[NOTE]
====
If you get an error like `ai.onnxruntime.OrtException: Error code - ORT_EP_FAIL - message: Failed to find CUDA shared provider`,
that means that you are using the GPU parameters `spring.ai.embedding.transformer.onnx.gpuDeviceId` , but the onnxruntime_gpu dependency are missing.
----
<dependency>
<groupId>com.microsoft.onnxruntime</groupId>
<artifactId>onnxruntime_gpu</artifactId>
</dependency>
----
Please select the appropriate onnxruntime_gpu version based on the CUDA version(link:https://onnxruntime.ai/docs/get-started/with-java.html[ONNX Java Runtime]).
====

== Manual Configuration

If you are not using Spring Boot, you can manually configure the Onnx Transformers Embedding Model.
Expand Down

0 comments on commit aeba9c2

Please sign in to comment.