Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ipex doc #828

Merged
merged 32 commits into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
d7b0fc4
change readme, source/index, source/installation
jiqing-feng Jul 16, 2024
78f7c61
add ipex doc 1st step
jiqing-feng Jul 17, 2024
b531a72
update readme for command line usage
jiqing-feng Jul 17, 2024
90d9000
fix bug for ipex readme
jiqing-feng Jul 17, 2024
b39be97
add export doc
jiqing-feng Jul 17, 2024
a90cb23
update all ipex docs
jiqing-feng Jul 17, 2024
e884158
rm diffusers
jiqing-feng Jul 17, 2024
2100cd9
change register
jiqing-feng Jul 17, 2024
84305bc
Update README.md
jiqing-feng Jul 17, 2024
23f8756
Update docs/source/installation.mdx
jiqing-feng Jul 17, 2024
644d197
fix readme
jiqing-feng Jul 17, 2024
fde311c
fix ipex exporter args comments
jiqing-feng Jul 17, 2024
a9a2c38
extend ipex export explain
jiqing-feng Jul 17, 2024
4368205
fix ipex reference.mdx
jiqing-feng Jul 18, 2024
c31696d
add comments for auto doc
jiqing-feng Jul 18, 2024
c5412da
rm cli export
jiqing-feng Jul 18, 2024
291c73d
Update optimum/commands/export/ipex.py
jiqing-feng Jul 18, 2024
8772c51
rm commit hash in export command
jiqing-feng Jul 18, 2024
39f27dd
rm export
jiqing-feng Jul 22, 2024
1d8fc29
rm jit
jiqing-feng Jul 22, 2024
02fa235
add ipex on doc's docker file
jiqing-feng Jul 26, 2024
4ed2620
indicate that ipex model only supports for cpu and the export format …
jiqing-feng Jul 26, 2024
770d82f
Update docs/source/ipex/inference.mdx
jiqing-feng Jul 29, 2024
0a9ce3d
explain patching
jiqing-feng Jul 29, 2024
378144e
rm ipex reference
jiqing-feng Aug 6, 2024
be7097d
Update docs/source/ipex/inference.mdx
echarlaix Aug 26, 2024
21f06cf
Update docs/source/ipex/inference.mdx
echarlaix Aug 26, 2024
ae8143a
Update docs/source/ipex/inference.mdx
echarlaix Aug 26, 2024
9a25ac7
Update docs/source/index.mdx
echarlaix Aug 26, 2024
52bec25
Update docs/source/ipex/inference.mdx
echarlaix Aug 26, 2024
d6153b7
Update docs/source/ipex/models.mdx
echarlaix Aug 26, 2024
8115bf6
Update docs/Dockerfile
echarlaix Aug 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,11 @@ You can find more examples in the [documentation](https://huggingface.co/docs/op


## IPEX
IPEX export can be used through the Optimum command-line interface:
```bash
optimum-cli export ipex -m gpt2 --torch_dtype bfloat16 ipex-gpt2
```

jiqing-feng marked this conversation as resolved.
Show resolved Hide resolved
To load your IPEX model, you can just replace your `AutoModelForXxx` class with the corresponding `IPEXModelForXxx` class. You can set `export=True` to load a PyTorch checkpoint, export your model via TorchScript and apply IPEX optimizations : both operators optimization (replaced with customized IPEX operators) and graph-level optimization (like operators fusion) will be applied on your model.
```diff
from transformers import AutoTokenizer, pipeline
Expand All @@ -224,14 +229,18 @@ To load your IPEX model, you can just replace your `AutoModelForXxx` class with
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
results = pipe("He's a dreadful magician and")

+ # You can also use the model exported by Optimum command-line interface
+ exported_model = IPEXModelForCausalLM.from_pretrained("ipex-gpt2")
+ pipe.model = exported_model
jiqing-feng marked this conversation as resolved.
Show resolved Hide resolved
+ results = pipe("He's a dreadful magician and")
```

For more details, please refer to the [documentation](https://intel.github.io/intel-extension-for-pytorch/#introduction).


## Running the examples

Check out the [`examples`](https://github.com/huggingface/optimum-intel/tree/main/examples) directory to see how 🤗 Optimum Intel can be used to optimize models and accelerate inference.
Check out the [`examples`](https://github.com/huggingface/optimum-intel/tree/main/examples) and [`notebooks`](https://github.com/huggingface/optimum-intel/tree/main/notebooks) directory to see how 🤗 Optimum Intel can be used to optimize models and accelerate inference.

Do not forget to install requirements for every example:

Expand Down
15 changes: 15 additions & 0 deletions docs/source/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,5 +30,20 @@
title: Tutorials
isExpanded: false
title: OpenVINO
- sections:
- local: ipex/export
title: Export
- local: ipex/inference
title: Inference
- local: ipex/models
title: Supported Models
- local: ipex/reference
title: Reference
- sections:
- local: ipex/tutorials/notebooks
title: Notebooks
title: Tutorials
isExpanded: false
title: IPEX
title: Optimum Intel
isExpanded: false
2 changes: 2 additions & 0 deletions docs/source/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ limitations under the License.

🤗 Optimum Intel is the interface between the 🤗 Transformers and Diffusers libraries and the different tools and libraries provided by Intel to accelerate end-to-end pipelines on Intel architectures.

[Intel Extension for PyTorch](https://intel.github.io/intel-extension-for-pytorch/#introduction) is an open-source library which provides optimizations for both eager mode and graph mode, however, compared to eager mode, graph mode in PyTorch* normally yields better performance from optimization techniques, such as operation fusion.
echarlaix marked this conversation as resolved.
Show resolved Hide resolved

[Intel Neural Compressor](https://www.intel.com/content/www/us/en/developer/tools/oneapi/neural-compressor.html) is an open-source library enabling the usage of the most popular compression techniques such as quantization, pruning and knowledge distillation. It supports automatic accuracy-driven tuning strategies in order for users to easily generate quantized model. The users can easily apply static, dynamic and aware-training quantization approaches while giving an expected accuracy criteria. It also supports different weight pruning techniques enabling the creation of pruned model giving a predefined sparsity target.

[OpenVINO](https://docs.openvino.ai) is an open-source toolkit that enables high performance inference capabilities for Intel CPUs, GPUs, and special DL inference accelerators ([see](https://docs.openvino.ai/2024/about-openvino/compatibility-and-support/supported-devices.html) the full list of supported devices). It is supplied with a set of tools to optimize your models with compression techniques such as quantization, pruning and knowledge distillation. Optimum Intel provides a simple interface to optimize your Transformers and Diffusers models, convert them to the OpenVINO Intermediate Representation (IR) format and run inference using OpenVINO Runtime.
Expand Down
3 changes: 2 additions & 1 deletion docs/source/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ To install the latest release of 🤗 Optimum Intel with the corresponding requi
|:-----------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------|
| [Intel Neural Compressor (INC)](https://www.intel.com/content/www/us/en/developer/tools/oneapi/neural-compressor.html) | `pip install --upgrade --upgrade-strategy eager "optimum[neural-compressor]"`|
| [Intel OpenVINO](https://docs.openvino.ai ) | `pip install --upgrade --upgrade-strategy eager "optimum[openvino]"` |
| [Intel Extension for PyTorch](https://intel.github.io/intel-extension-for-pytorch/#introduction) | `pip install --upgrade --upgrade-strategy eager "optimum[ipex]"` |

The `--upgrade-strategy eager` option is needed to ensure `optimum-intel` is upgraded to the latest version.

Expand All @@ -42,4 +43,4 @@ or to install from source including dependencies:
python -m pip install "optimum-intel[extras]"@git+https://github.com/huggingface/optimum-intel.git
```

where `extras` can be one or more of `neural-compressor`, `openvino`, `nncf`.
where `extras` can be one or more of `neural-compressor`, `openvino`, `ipex`.
61 changes: 61 additions & 0 deletions docs/source/ipex/export.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# Export your model

## Using the CLI

To export your model to the IPEX format with the CLI :
jiqing-feng marked this conversation as resolved.
Show resolved Hide resolved

```bash
optimum-cli export ipex -m gpt2 --torch_dtype bfloat16 ipex-gpt2
```

The model argument can either be the model ID of a model hosted on the [Hub](https://huggingface.co/models) or a path to a model hosted locally. For local models, you need to specify the task for which the model should be loaded before export, among the list of the [supported tasks](https://github.com/huggingface/optimum-intel/blob/main/optimum/intel/ipex/utils.py).

```bash
optimum-cli export ipex -m local_gpt2 --task text-generation --torch_dtype bfloat16 ipex-gpt2
```

Check out the help for more options:

```bash
optimum-cli export ipex --help

usage: optimum-cli export ipex [-h] -m MODEL [--task TASK] [--trust_remote_code] [--revision REVISION] [--token TOKEN] [--cache_dir CACHE_DIR] [--subfolder SUBFOLDER]
[--local_files_only LOCAL_FILES_ONLY] [--force_download FORCE_DOWNLOAD] [--commit_hash COMMIT_HASH] [--torch_dtype TORCH_DTYPE]
output

options:
-h, --help show this help message and exit

Required arguments:
-m MODEL, --model MODEL
Model ID on huggingface.co or path on disk to load model from.
output Path indicating the directory where to store the generated IPEX model.

Optional arguments:
--task TASK The task to export the model for. If not specified, the task will be auto-inferred based on the model. Available tasks depend on the model.
--trust_remote_code Allows to use custom code for the modeling hosted in the model repository. This option should only be set for repositories you trust and in which you have read the code, as it
will execute on your local machine arbitrary code present in the model repository.
--revision REVISION model kwargs
--token TOKEN model kwargs
--cache_dir CACHE_DIR
model kwargs
--subfolder SUBFOLDER
model kwargs
--local_files_only LOCAL_FILES_ONLY
model kwargs
--force_download FORCE_DOWNLOAD
model kwargs
--commit_hash COMMIT_HASH
model kwargs
--torch_dtype TORCH_DTYPE
model kwargs
jiqing-feng marked this conversation as resolved.
Show resolved Hide resolved
```
46 changes: 46 additions & 0 deletions docs/source/ipex/inference.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# Inference

Optimum Intel can be used to load models from the [Hub](https://huggingface.co/models) and create pipelines to run inference with IPEX optimization (including patching, weight prepack and jit trace) on a variety of Intel processors (currently only support for CPU)


## Loading

### Transformers models

echarlaix marked this conversation as resolved.
Show resolved Hide resolved
You can load models from HuggingFace Model Hub, it will be optimized by ipex during loading.
jiqing-feng marked this conversation as resolved.
Show resolved Hide resolved

```diff
import torch
from transformers import AutoTokenizer, pipeline
- from transformers import AutoModelForCausalLM
+ from optimum.intel import IPEXModelForCausalLM

model_id = "gpt2"
- model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16)
+ model = IPEXModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, export=True)
jiqing-feng marked this conversation as resolved.
Show resolved Hide resolved
tokenizer = AutoTokenizer.from_pretrained(model_id)
pipe = pipeline("text-generation", model=model, tokenizer=tokenizer)
results = pipe("He's a dreadful magician and")
```

As shown in the table below, each task is associated with a class enabling to automatically load your model.

| Auto Class | Task |
|--------------------------------------|--------------------------------------|
| `IPEXModelForSequenceClassification` | `text-classification` |
| `IPEXModelForTokenClassification` | `token-classification` |
| `IPEXModelForQuestionAnswering` | `question-answering` |
| `IPEXModelForImageClassification` | `image-classification` |
| `IPEXModel` | `feature-extraction` |
| `IPEXModelForMaskedLM` | `fill-mask` |
| `IPEXModelForAudioClassification` | `audio-classification` |
| `IPEXModelForCausalLM` | `text-generation` |
46 changes: 46 additions & 0 deletions docs/source/ipex/models.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# Supported models

🤗 Optimum handles the export of models to IPEX in the `exporters.ipex` module. It provides classes, functions, and a command line interface to perform the export easily.
echarlaix marked this conversation as resolved.
Show resolved Hide resolved
Here is the list of the supported architectures :

## [Transformers](https://huggingface.co/docs/transformers/index)

- Albert
- Bart
- Beit
- Bert
- BlenderBot
- BlenderBotSmall
- Bloom
- CodeGen
- DistilBert
- Electra
- Flaubert
- GPT-2
- GPT-BigCode
- GPT-Neo
- GPT-NeoX
- Llama
- MPT
- Mistral
- MobileNet v1
- MobileNet v2
- MobileVit
- OPT
- ResNet
- Roberta
- Roformer
- SqueezeBert
- UniSpeech
- Vit
- Wav2Vec2
- XLM
72 changes: 72 additions & 0 deletions docs/source/ipex/reference.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# Models

## Generic model classes

[[autodoc]] ipex.modeling_base.IPEXModel
- _from_pretrained
- reshape

## Natural Language Processing

The following classes are available for the following natural language processing tasks.

### IPEXModelForCausalLM

[[autodoc]] ipex.modeling_base.IPEXModelForCausalLM
- forward
- generate

### IPEXModelForMaskedLM

[[autodoc]] ipex.modeling_base.IPEXModelForMaskedLM
- forward

### IPEXModelForQuestionAnswering

[[autodoc]] ipex.modeling_base.IPEXModelForQuestionAnswering
- forward

### IPEXModelForSequenceClassification

[[autodoc]] ipex.modeling_base.IPEXModelForSequenceClassification
- forward

### IPEXModelForTokenClassification

[[autodoc]] ipex.modeling_base.IPEXModelForTokenClassification
- forward


## Audio

The following classes are available for the following audio tasks.

### IPEXModelForAudioClassification

[[autodoc]] ipex.modeling_base.IPEXModelForAudioClassification
- forward


## Computer Vision

The following classes are available for the following computer vision tasks.

### IPEXModelForImageClassification

[[autodoc]] ipex.modeling_base.IPEXModelForImageClassification
- forward


### IPEXModelForFeatureExtraction

[[autodoc]] ipex.modeling_base.IPEXModelForFeatureExtraction
- forward
16 changes: 16 additions & 0 deletions docs/source/ipex/tutorials/notebooks.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
<!--Copyright 2024 The HuggingFace Team. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
-->

# Notebooks

## Inference

| Notebook | Description | | |
|:---------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------- |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|------:|
| [How to run inference with the IPEX](https://github.com/huggingface/optimum-intel/tree/main/notebooks/ipex) | Explains how to export your model to IPEX and to run inference with IPEX model on text-generation task | [![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/huggingface/optimum-intel/blob/main/notebooks/ipex/text_generation.ipynb) | [![Open in AWS Studio](https://studiolab.sagemaker.aws/studiolab.svg)](https://studiolab.sagemaker.aws/import/github/huggingface/optimum-intel/blob/main/notebooks/ipex/text_generation.ipynb) |
Loading