f32 precision for compare-with-transformers tests #508
Conversation
Allow these tests to pass locally on devices where inference is run by default in FP16 (GPU) or BF16 (latest Xeon).
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
tests/openvino/test_modeling.py
Outdated
| @@ -430,7 +440,9 @@ class OVModelForFeatureExtractionIntegrationTest(unittest.TestCase): | |||
| def test_compare_to_transformers(self, model_arch): | |||
| model_id = MODEL_NAMES[model_arch] | |||
| set_seed(SEED) | |||
| ov_model = OVModelForFeatureExtraction.from_pretrained(model_id, export=True) | |||
| ov_model = OVModelForFeatureExtraction.from_pretrained( | |||
| model_id, export=True, ov_config={"CACHE_DIR": "", "INFERENCE_PRECISION_HINT": "f32"} | |||
There was a problem hiding this comment.
I suggest using OV_CONFIG const in utils and loading it via:
ov_model = OVModelForFeatureExtraction.from_pretrained(
model_id, export=True, ov_config=OV_CONFIG)
There was a problem hiding this comment.
also setting NUN_STREAMS = 1 could be useful
There was a problem hiding this comment.
I suggest using OV_CONFIG const in utils and loading it via:
ov_model = OVModelForFeatureExtraction.from_pretrained(
model_id, export=True, ov_config=OV_CONFIG)
I like being explicit in tests, especially when we deviate from defaults. But I can add an F32_CONFIG at the top of this file.
also setting NUN_STREAMS = 1 could be useful
Agreed that this can be useful, but we should run some tests and assuming no issues, set this by default in optimum-intel, not just for the tests.
There was a problem hiding this comment.
I like being explicit in tests, especially when we deviate from defaults. But I can add an F32_CONFIG at the top of this file.
Done
There was a problem hiding this comment.
the same will be needed also in stable diffusion tests
tests/openvino/test_modeling.py
Outdated
| @@ -125,7 +128,10 @@ def test_load_from_hub_and_save_model(self): | |||
| loaded_model = OVModelForSequenceClassification.from_pretrained(self.OV_MODEL_ID, ov_config=ov_config) | |||
| self.assertTrue(manual_openvino_cache_dir.is_dir()) | |||
| self.assertGreaterEqual(len(list(manual_openvino_cache_dir.glob("*.blob"))), 1) | |||
| self.assertEqual(loaded_model.request.get_property("PERFORMANCE_HINT").name, "THROUGHPUT") | |||
| if get_version() < "2023.3": | |||
There was a problem hiding this comment.
minor comment : it could be replaced with is_openvino_version for clarity
| if get_version() < "2023.3": | |
| if is_openvino_version("<", "2023.3"): |
There was a problem hiding this comment.
Thanks! Changed.
|
Thanks a lot @helena-intel |
* f32 precision for compare-with-transformers tests Allow these tests to pass locally on devices where inference is run by default in FP16 (GPU) or BF16 (latest Xeon). * Add F32_CONFIG constant for modeling tests * Replace get_version with is_openvino_version
Allow these tests to pass locally on devices where inference is run by default in FP16 (GPU) or BF16 (latest Xeon).
Also added an OpenVINO version check to make a test pass in 2023.3 (get_property now returns a string instead of an object).