Refactor and Add Tests #2
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* add llamd and bloom in ipex inference tests * only add llama tests
* add position_ids in forward * check if jit model need position_ids * use MODEL_TYPES_REQUIRING_POSITION_IDS * fix has_position_ids * fix position_ids length * rm useless params * check model inputs by input names * fix format * check input names in graph model * fix style * consider eager model in input_names * add input names * add text input names * fix styl;e * Update optimum/intel/generation/modeling.py * fix format * Update optimum/intel/generation/modeling.py --------- Co-authored-by: Ella Charlaix <ella@huggingface.co> Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
* f32 precision for compare-with-transformers tests Allow these tests to pass locally on devices where inference is run by default in FP16 (GPU) or BF16 (latest Xeon). * Add F32_CONFIG constant for modeling tests * Replace get_version with is_openvino_version
3 tasks
|
maybe it's better to re-create PR to optimum-intel from apaniukov:openvino_tokenizers ? |
…quest-wrapper-typo Fix typo inside InferRequestWrapper
* Add try for get_property * refine code --------- Co-authored-by: fishbell <wangwang.wang@intel.com>
…i-fix Fix error with optimum-cli export openvino --help
* Bump min torch version * comment * add torch min version
* Allow loading of stateful models (no patching yet) * Stateful models support * Fix forward for chatglm * Passing stateful as a dedicated parameter * Fixed possibly misaligned types in ShapeOf Concat sub-expression * Fixed critical typo in infer_request invocation * Apply bettertransfomer when model is converted in stateful mode * Correct default value handling for stateful flag * Apply bettertransformer under try-except to avoid crashes when model is not supported * Added --stateful option in optimum-cli * Raise if too old version of opevino is used ans stateful=True * Fix openvino version check to be compatible with openvino-nightly * Fix for bloom family * Allow loading of stateful models (no patching yet) * Stateful models support * Fix forward for chatglm * Passing stateful as a dedicated parameter * Fixed possibly misaligned types in ShapeOf Concat sub-expression * Fixed critical typo in infer_request invocation * Apply bettertransfomer when model is converted in stateful mode * Correct default value handling for stateful flag * Apply bettertransformer under try-except to avoid crashes when model is not supported * Added --stateful option in optimum-cli * Raise if too old version of opevino is used ans stateful=True * Fix openvino version check to be compatible with openvino-nightly * Fix for bloom family * Fix general code style and appliy renaming suggestions * fix version checking if openvino not in site-packages * use reset_stateif available * remove input patch in bettertransformer apply * add tests * add type hints and update doc strings * added more tests * Fixed outdated signature of InferRequest wrapper to fix one of the quantizer tests. * Switch to stateful model by default * Allow loading of stateful models (no patching yet) * Stateful models support * Fix forward for chatglm * Passing stateful as a dedicated parameter * Fixed possibly misaligned types in ShapeOf Concat sub-expression * Apply bettertransfomer when model is converted in stateful mode * Correct default value handling for stateful flag * Apply bettertransformer under try-except to avoid crashes when model is not supported * Added --stateful option in optimum-cli * Raise if too old version of opevino is used ans stateful=True * Fix openvino version check to be compatible with openvino-nightly * Fix for bloom family * Fix general code style and appliy renaming suggestions * fix version checking if openvino not in site-packages * use reset_stateif available * remove input patch in bettertransformer apply * add tests * add type hints and update doc strings * added more tests * Fixed outdated signature of InferRequest wrapper to fix one of the quantizer tests. * Stateful models support * Fix forward for chatglm * Passing stateful as a dedicated parameter * Apply bettertransfomer when model is converted in stateful mode * Raise if too old version of opevino is used ans stateful=True * Fix openvino version check to be compatible with openvino-nightly * Fix for bloom family * fix test and add beam_idx attribute * apply review comments * stateful by default fixes * less agressive stateful * ensure that task support stateful * remove debug print * Apply suggestions from code review Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com> * Apply suggestions from code review Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com> * update requirements and warning messages * Apply suggestions from code review Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com> * fix cli export * Update optimum/exporters/openvino/__main__.py Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com> --------- Co-authored-by: Sergey Lyalin <sergey.lyalin@intel.com> Co-authored-by: Helena Kloosterman <helena.kloosterman@intel.com> Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
…uggingface#518) * Use f32 precision for compare-to-diffusers tests * Add f32 ov_config to training/stable diffusion tests
apaniukov
force-pushed
the
openvino_tokenizers
branch
from
f2a2877 to
57782d1
Compare
Reuse existing preprocessors
* add IPEX model and README update ipex modeling and add case for text-generation and text-classification Signed-off-by: Wang, Yi A <yi.a.wang@intel.com> * fix style * IPEX modeling refactorization * typo * remove use cache arg when loading model * fix style * move tests * remove readme * add test * add warning if use_cache mismatch * fix * format * update setup * add use_cache attribute --------- Signed-off-by: Wang, Yi A <yi.a.wang@intel.com> Co-authored-by: Feng, Jiqing <jiqing.feng@intel.com>
* add IPEX model for QA task * add fix
…huggingface#533) * Expose InferRequestWrapper class so it can be imported from elsewhere * Fix
…#536) * add test * format * Add image classification task * Add test
huggingface#537) * relax requirements to have registered normalized config for usage converted decoder models * add property for access to normalized config
* fix torch version for ipex tests * disbale tests for incompatible torch version with ipex * fix
…ce#544) * Refactor IPEX CausalLM for better model arch scale * Fix style
* Handle autocast in IPEXModel.forward * Handle missing torch_dtype in config
* Handle autocast in IPEXModel.forward * Handle missing torch_dtype in config * Warmup IPEX models at init * Minor fix * Fix _init_warmup use_cache condition * Fix output handling in IPEX question answering
…ommit Fix openvino/test_training.py
* Initial code for load_in_4_bit * Dataset does not work * Intermediate changes * Make it working with dataset * Style * Fixed small issue * Fixed failed tests * Style * Comment failed tests due to NNCF 2.8 * Commented failed tests until new NNCF release * Added tests for load_in_4bit * Added awq option. Included NNCF package into openvino extra. * Rolled back including nncf into openvino extra * Style * Fixed tests * Fixed issues with models larger than 1B. Added tests. * Style * Fixed issues. Applied comments. * Removed unnecessary exception * Applied more comments * Fixed issue * Make quantization_config a part of OVConfig in OVQuantizer * Fixed issue with Transformers * Fixed test * Changed the naming. Added additional tests * Fixed tests * Fixed tests * Applied more comments * Style
…#535) * skip compression weights tests for nncf==2.8.0 and reworked logic of optimization stateful PyTorch models * black happy * ruff happy * updated nncf version * replied to comments * replied comments * typo * cherry pick fixes for tests from PR 538 * replied to comments --------- Co-authored-by: Ella Charlaix <80481427+echarlaix@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes # (issue)
Before submitting