Skip to content

Commit

Permalink
Udpate openvino export CLI documentation (#746)
Browse files Browse the repository at this point in the history
* udpate openvino documentation

* rephrase
  • Loading branch information
echarlaix authored Jun 5, 2024
1 parent 52875b9 commit 8dd2a89
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions docs/source/inference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ As shown in the table below, each task is associated with a class enabling to au
| `fill-mask` | `OVModelForMaskedLM` |
| `image-classification` | `OVModelForImageClassification` |
| `audio-classification` | `OVModelForAudioClassification` |
| `text-generation` | `OVModelForCausalLM` |
| `text2text-generation` | `OVModelForSeq2SeqLM` |
| `text-generation-with-past` | `OVModelForCausalLM` |
| `text2text-generation-with-past` | `OVModelForSeq2SeqLM` |
| `automatic-speech-recognition` | `OVModelForSpeechSeq2Seq` |
| `image-to-text` | `OVModelForVision2Seq` |

Expand All @@ -46,7 +46,7 @@ optimum-cli export openvino --model gpt2 ov_model

The example above illustrates exporting a checkpoint from the 🤗 Hub. When exporting a local model, first make sure that you saved both the model’s weights and tokenizer files in the same directory (`local_path`).
When using CLI, pass the `local_path` to the model argument instead of the checkpoint name of the model hosted on the Hub and provide the `--task` argument. You can review the list of supported tasks in the 🤗 [Optimum documentation](https://huggingface.co/docs/optimum/exporters/task_manager). If task argument is not provided, it will default to the model architecture without any task specific head.
Here we set the `task` to `text-generation-with-past`, with the `-with-past` suffix enabling the re-use of the pre-computed key/values hidden-states `use_cache=True`.
The `-with-past` suffix enable the re-use of the pre-computed key/values hidden-states and is the recommended option, to export the model without (equivalent to `use_cache=False`), you will need to remove this suffix.

```bash
optimum-cli export openvino --model local_path --task text-generation-with-past ov_model
Expand Down

0 comments on commit 8dd2a89

Please sign in to comment.