Support for FDAT models

[Kuronoe-Ookami]

I would like to know if vs-mlrt has support for FDAT models or fp16 models?

I tried using the following model with TRT in 720x480, but I got some errors and when I did, the image looked weird, black and white with lots of cracks.
2x_animefilm_light_161k_fp16_static_720x480.onnx

src = core.ffms2.Source(r'C:\anime_720x480.mkv')
clipes = core.resize.Spline64(src,720, 480, format=vs.RGBS, matrix_in_s="709")
clipes = core.trt.Model(clipes, engine_path=r"C:\2x_animefilm_light_161k_fp16_static_720x480.engine")
clipes = core.resize.Spline64(clipes, 1280, 720, format=vs.YUV420P10, matrix_in_s="470bg", matrix_s="709")
clipe.set_output()

I used the following code to generate the engine.
--fp16 --inputIOFormats=fp32:chw --outputIOFormats=fp32:chw --tacticSources=+CUDNN,+CUBLAS --builderOptimizationLevel=2 --skipInference

if it works directly with onnx, that would be good too. I tested it with TRT because it technically works with VideoJanai, which references vs-mlrt for this.

I'd appreciate it if someone with more experience could test it and point out where I'm going wrong.

Thank you.

[WolframRhodium]

(Sorry for my late response.)

These kinds of precision issue is common for these transformer-based models. There is no simple solution AFAIK, and you may have to disable --fp16.

[Kuronoe-Ookami]

Hi, that's so kind, thanks for your help.

Actually, removing --fp16 made the model work, I didn't imagine that could be it, since the trained model is fp16.
Although the model seems slow anyway. I don't know if it's a consequence of this change, as I use other models in fp32 and they are stupidly fast even when combined with other filters.
AnimeJanai or AniSD can often reach 20fps with x264 and other filters, for example. I understand that compact and SPAN are fast, but the idea behind using fp16 (according to the model's creators) is that, in theory, it would be faster than fp32.

Anyway, I appreciate the help. It seems to have worked without the errors I experienced previously.
Ty~

[WolframRhodium]

Which version of vs-mlrt (or TensorRT in particular) and what GPU are you using? With TensorRT 10.3, I cannot reproduce the issue.

[Kuronoe-Ookami]

After your --fp16 tip the problem was solved.

I'm using the latest version of vsmlrt, CPU i7 13700kf and GPU 3060TI 8GiB.

what happened before was something like this image.