Introduce OVPipelineQuantizationConfig#1310
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
@l-bat Could you please review this PR? |
| model, | ||
| calibration_datasets["model"], | ||
| subset_size=quantization_config.num_samples, | ||
| subset_size=quantization_config.num_samples or 128, |
There was a problem hiding this comment.
why is this value hidden here ?
There was a problem hiding this comment.
128 was previously a default value for num_samples argument in OVQuantizationConfig. In this PR I've removed it so it is now None by default. That's why 128 has appeared here.
Ideally, we should transition to providing arguments via quantization_config.to_nncf_dict() here, as it is done for OV case. But I propose to do this in a separate PR.
|
Hi @echarlaix! Do you mind us merging this or you would like to take a look? |
|
apologies for the delay @nikita-savelyevv, taking a look right now! |
echarlaix
left a comment
There was a problem hiding this comment.
Looks great thanks @nikita-savelyevv !!
What does this PR do?
Changes:
OVPipelineQuantizationConfigallowing to specify quantization parameters per model component. Add corresponding tests._quantization_config_from_dict().For example the code below applies int8 PTQ to
lm_model, int8 WC totext_embeddings_modeland no optimization tovision_embeddings_model.Before submitting