Add an initial warmup step to IPEXModels

[ofirzaf]

What does this PR do?

The first 2 forwards of an IPEXModel after trace/load includes background optimizations steps that make the output of these forwards unpredictable and non consistent with the model after the optimizations. To fix that, an initial warmup step was added to the __init__ of IPEXModels

Depends on PR #542

@echarlaix

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you make sure to update the documentation with your changes?
• Did you write any new necessary tests?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[echarlaix]

LGTM

[echarlaix]

why is this needed ?

[ofirzaf]

The prepare_jit_inputs looks at the signature of the function and the wraps and super help avoid code copy

[echarlaix]

would prefer we avoid as it will fail in case outputs is not a dict

optimum-intel/optimum/intel/ipex/modeling_base.py

Line 193 in 8ee487d

return ModelOutput(**outputs) if isinstance(outputs, dict) else ModelOutput(logits=outputs[0])

[echarlaix]

also not sure to see the link with prepare_jit_inputs

[ofirzaf]

1. In _init_warmup we call prepare_jit_inputs which examines the passed model's forward signature to see which dummy inputs exists in the signature. If we don't use wraps we get the signature of

   self, *args, **kwargs
   

   instead of

   self, input_ids: torch.Tensor, attention_mask: torch.Tensor, token_type_ids: torch.Tensor = None, **kwargs,
   

2. outputs will always be a dict because this is the output of IPEXModel.forward, no?

[echarlaix]

1. OK I understand, was thinking that prepare_jit_inputs was only used for the torchscript export but I see that it's also used in _init_warmup, thanks for the clarification
2. here I'm talking about outputs https://github.com/huggingface/optimum-intel/blob/8ee487dc2ade5bd0023d1bbe0a0103d6af8821e0/optimum/intel/ipex/modeling_base.py#L192C9-L192C16