Skip to content

Commit 13f2836

Browse files
Allow to infer VAE in f16 precision (#859)
* Added custom ov_configs for vae encoder and decoder * Simplify usage
1 parent 0c577d1 commit 13f2836

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

optimum/intel/openvino/modeling_diffusion.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -718,7 +718,7 @@ def __call__(self, latent_sample: np.ndarray):
718718
return list(outputs.values())
719719

720720
def _compile(self):
721-
if "GPU" in self._device:
721+
if "GPU" in self._device and "INFERENCE_PRECISION_HINT" not in self.ov_config:
722722
self.ov_config.update({"INFERENCE_PRECISION_HINT": "f32"})
723723
super()._compile()
724724

@@ -739,7 +739,7 @@ def __call__(self, sample: np.ndarray):
739739
return list(outputs.values())
740740

741741
def _compile(self):
742-
if "GPU" in self._device:
742+
if "GPU" in self._device and "INFERENCE_PRECISION_HINT" not in self.ov_config:
743743
self.ov_config.update({"INFERENCE_PRECISION_HINT": "f32"})
744744
super()._compile()
745745

0 commit comments

Comments
 (0)