Skip to content

Commit

Permalink
fix
Browse files Browse the repository at this point in the history
  • Loading branch information
echarlaix authored Jul 8, 2024
1 parent 0316e3d commit 8fdf5c7
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/source/openvino/export.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ Models larger than 1 billion parameters are exported to the OpenVINO format with
optimum-cli export openvino --model meta-llama/Meta-Llama-3-8B --task text-generation-with-past ov_llama/
```

The `-with-past` suffix enable the re-use of past keys and values. This allows to avoid recomputing the same intermediate activations during the generation. to export the model without, you will need to remove this suffix.
The `-with-past` suffix enable the re-use of past keys and values. This allows to avoid recomputing the same intermediate activations at each generation step. To export the model without, you will need to remove this suffix.

| With K-V cache | Without K-V cache |
|------------------------------------------|--------------------------------------|
Expand Down

0 comments on commit 8fdf5c7

Please sign in to comment.