From 06c33046b46acd715978ec3086bed1f716ba30c0 Mon Sep 17 00:00:00 2001 From: Ella Charlaix <80481427+echarlaix@users.noreply.github.com> Date: Tue, 9 Jul 2024 10:58:33 +0200 Subject: [PATCH] Update docs/source/openvino/inference.mdx Co-authored-by: Helena Kloosterman --- docs/source/openvino/inference.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/openvino/inference.mdx b/docs/source/openvino/inference.mdx index 9416fb8c05..b838044698 100644 --- a/docs/source/openvino/inference.mdx +++ b/docs/source/openvino/inference.mdx @@ -163,7 +163,7 @@ model = OVModelForSequenceClassification.from_pretrained(model_id, ov_config=ov_ Optimum Intel leverages OpenVINO's model caching to speed up model compiling on GPU. By default a `model_cache` directory is created in the model's directory in the [Hugging Face Hub cache](https://huggingface.co/docs/huggingface_hub/main/en/guides/manage-cache). To override this, use the ov_config parameter and set `CACHE_DIR` to a different value. To disable model caching on GPU, set `CACHE_DIR` to an empty string. ```python -ov_config = {"PERFORMANCE_HINT": "LATENCY", "CACHE_DIR":""} +ov_config = {"CACHE_DIR":""} model = OVModelForSequenceClassification.from_pretrained(model_id, device="gpu", ov_config=ov_config) ```