Skip to content

OpenVINO + FP16 + Waifu2xModel.cunet == ERROR #156

@WhitePr

Description

@WhitePr

Arch linux
kernel: linux 6.18.2-arch2-1
CPU ryzen 7900x
intel arc b580
openvino 2025.4.0-2 from AUR

This is vapoursynth script

import vapoursynth as vs
from vapoursynth import core
import mvsfunc as mvf
import vsmlrt

vsmlrt.Waifu2x(clip = core.std.BlankClip(format=vs.RGBS),
               scale = 1, 
               noise = 3, 
               model = vsmlrt.Waifu2xModel.cunet, 
               backend = vsmlrt.Backend.OV_GPU(fp16=1, num_streams=2)).set_output()

error log from Terminal:

/usr/include/c++/15.2.1/bits/stl_vector.h:1282: std::vector<_Tp, _Alloc>::const_reference std::vector<_Tp, _Alloc>::operator[](size_type) const [with _Tp = kernel_selector::Tensor::DataTensor; _Alloc = std::allocator<kernel_selector::Tensor::DataTensor>; const_reference = const kernel_selector::Tensor::DataTensor&; size_type = long unsigned int]: Assertion '__n < this->size()' failed.

I tested other models (except swin) without problems. Also tried different noise and scale parameters, but the problem remained. Once I set OPENVINO to FP32, the problem disappeared. Checking the WIKI documentation, I found that the A770 is able to run the CUNET model in FP16 mode, but reports errors on my B580.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions