Ipex doc #828
Ipex doc #828
Conversation
|
All ipex docs have been added, please take a review @echarlaix cc @rbrugaro |
Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
|
@echarlaix Please trigger the tests. Thx. |
|
Hi @echarlaix . For the IPEX doc, I have removed export and anything related to jit. It will not confuse users even we change jit.trance to torch.compile because we didn't mention it in the docs, and the user's behaviors will not be changed. We need an ipex doc to show users how to use IPEXModel and don't need to tell users the details, WDYT? |
docs/source/ipex/inference.mdx
Outdated
|
|
||
| # Inference | ||
|
|
||
| Optimum Intel can be used to load models from the [Hub](https://huggingface.co/models) and create pipelines to run inference with IPEX optimization (including patching, weight prepack and graph mode) on a variety of Intel processors (currently only support for CPU) |
There was a problem hiding this comment.
What's the plan for the models format in the next release ? From my understanding we will stop using TorchScript, I'd prefer to wait for the next release once we have something that won't get deprecated, before adding it to the documentation
There was a problem hiding this comment.
Do you mean the Pytorch release? If so, it depends on the models performance under torch.compile. If all models can have an acceptable speed-up under torch.compile, we will remove jit.trace and apply torch.compile; otherwise, we will keep torchscript or convert parts of models to torch.compile, I will discuss with you before I take any actions.
I don't think waiting for the next release is the best option since the torch.compile is not under our control. Currently, torch.compile is not working on all models for all tasks; many performance regressions need to be fixed. We will apply the torch.compile one by one for models. Changing all models to compile without any issues is impossible for now, so it's a long-term project. Besides, we will not change the API in IPEXModel, so it's okay to deliver this ipex doc to users.
…will be changed to compile in the future
Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
|
Hi @echarlaix . Do you think this PR could be merged since all tests passed? |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Hi @echarlaix . I just noticed that we haven't got any ipex doc on huggingface doc hub.
This is the 1st step that enables the optimmu-cli command to export models.
I will add ipex doc in the next step (in this PR), please review the optimmu-cli command for ipex. Thx!