Skip to content

Commit

Permalink
Update docs/source/openvino/inference.mdx
Browse files Browse the repository at this point in the history
Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com>
  • Loading branch information
echarlaix and helena-intel authored Jul 9, 2024
1 parent 2f28fb4 commit 06c3304
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/source/openvino/inference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ model = OVModelForSequenceClassification.from_pretrained(model_id, ov_config=ov_
Optimum Intel leverages OpenVINO's model caching to speed up model compiling on GPU. By default a `model_cache` directory is created in the model's directory in the [Hugging Face Hub cache](https://huggingface.co/docs/huggingface_hub/main/en/guides/manage-cache). To override this, use the ov_config parameter and set `CACHE_DIR` to a different value. To disable model caching on GPU, set `CACHE_DIR` to an empty string.

```python
ov_config = {"PERFORMANCE_HINT": "LATENCY", "CACHE_DIR":""}
ov_config = {"CACHE_DIR":""}
model = OVModelForSequenceClassification.from_pretrained(model_id, device="gpu", ov_config=ov_config)
```

Expand Down

0 comments on commit 06c3304

Please sign in to comment.