Add speculative decoding example to OpenVINO quantized generation not… · huggingface/optimum-intel@7c1d38b · GitHub

Commit

Add speculative decoding example to OpenVINO quantized generation not…

Browse files

…ebook (#635)

* Add assisted generation example to notebook

* Bug fixes

* Add acceptance rate measurement

* Fix explanation of AR formula

* Fix norm AR calc to ignore last window

* Minor fixes

Loading branch information

ofirzaf authored Apr 4, 2024

1 parent a48e0ca commit 7c1d38b