[Error] Load fp16

[phamkhactu]

I successfully converted my model to fp16:

import onnx
from onnxconverter_common import float16

model = onnx.load("tts_quantized.onnx")
model_fp16 = float16.convert_float_to_float16(model)
onnx.save(model_fp16, "tts_quantized_fp16.onnx")

But when I load fp16 model, I get error:

    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/home/tupk/anaconda3/envs/dl/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 397, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from tts_quantized_fp16.onnx failed:Type Error: Type (tensor(float16)) of output arg (/RandomNormalLike_output_0) of node (/RandomNormalLike) does not match expected type (tensor(float)).

Here is my fp16 model.

[xadupre]

The implementation in onnxruntime does not support float16 (https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/providers/cpu/generator/random.cc#L39) but it does on CUDA. The best here is probably to surround RandomNormalLike with Cast operators. float16 is still not part of C++ standard so the most simple implementation C++ would probably do the same. You should be able to do that by adding this node to the blocklist (parameter op_block_list).

[phamkhactu]

Thanks for your reply

I'm a newbie in this field.I had tried, but i still get error. I'm happy if you can give me some code example.