FP16 conversion yields an unusable model

[eddieevt-DXC]

I'm working with a model in Sagemaker (Resnet50 640x640, size [1, -1, -1, 3]) converted to ONNX. When trying to get more performance out of it by converting it to FP16, the conversion succeeds but trying to run the model gives this error:

E0907 08:27:25.823138 1379 model_lifecycle.cc:626] failed to load 'sagemaker' version 1: Internal: onnx runtime error 1: 
Load model from /models/sagemaker/1/model.onnx failed:Node (StatefulPartitionedCall/map/while_loop) Op (Loop) TypeInferenceError] 
Graph attribute inferencing failed: Node (Resize__59) Op (Resize) [ShapeInferenceError] 
Either `sizes` or `scales` must be provided, but not both of them

Trying out mixed precision instead fails at shape inferencing:

Traceback (most recent call last):
  File "/workspace/fp-16-onnx-converter.py", line 15, in <module>
    model_fp16 = auto_mixed_precision.auto_convert_mixed_precision(model, input_feed, rtol=0.01, atol=0.001, keep_io_types=True)
  File "/usr/local/lib/python3.10/dist-packages/onnxconverter_common/auto_mixed_precision.py", line 80, in auto_convert_mixed_precision
    if not run_attempt(node_names):
  File "/usr/local/lib/python3.10/dist-packages/onnxconverter_common/auto_mixed_precision.py", line 72, in run_attempt
    res1 = get_tensor_values_using_ort(model, feed_dict)
  File "/usr/local/lib/python3.10/dist-packages/onnxconverter_common/auto_mixed_precision.py", line 132, in get_tensor_values_using_ort
    sess = ort.InferenceSession(model.SerializeToString(), sess_options, providers=['CUDAExecutionProvider'])
  File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 383, in __init__
    self._create_inference_session(providers, provider_options, disabled_optimizers)
  File "/usr/local/lib/python3.10/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 426, in _create_inference_session
    sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Node (StatefulPartitionedCall/map/while_loop) Op (Loop) [TypeInferenceError] Graph attribute inferencing failed: Node (Resize__59) Op (Resize) [ShapeInferenceError] Either `sizes` or `scales` must be provided, but not both of them

It gives the same error with the latest shape inferencing script from Github. I am not sure where I need to post this issue as multiple parts of the ONNX stack seem involved and not working.

[skottmckay]

As the user reports the original model has no issue but the converted model has an invalid Resize node, it appears this is a converter bug.

microsoft/onnxruntime#17447

[eddieevt-DXC]

Tested EfficientDet D2 768x768 and it also exhibits the same behavior.

What I did:

• Download TF model
• Convert using tf2onnx
  -- At this point loading the model in an ONNX session works
• Downcast to FP16 using the same script
  -- Loading the ONNX FP16 model is a session gives the same error