tu-convnext_* with UNetPlusPlus throws runtime errors due to invalid weights with shape [0,x,y,y]

[tniranjan]

I tried using any of the Timm convnext encoders with UNetPlusPlus, and always get the following runtime error: Given groups=1, expected weight to be at least 1 at dimension 0, but got weight of size [0,96,3,3] . (The 96 part changes depending on what size the encoder is). I believe this has to do with how the DecoderBlock out_channels are initialized. I poked around a bit and think this has to do with a difference in how encoder levels are generally initialized in backbones like ResNet/EffNet vs timm_encoder. If initialized with an encoder of depth 2, the problem disappears, but deeper Us have the problem.

[qubvel]

Hey @tniranjan, thanks for opening discussion! At the moment Unet++ does not support transformers-like and convnext encoders, because feature map with H/2 resolution does no exist for them (the number of channels is 0).

Would appreciate a fix, in case you have bandwidth. The decoder should be adapted

[tniranjan]

Thanks for the clarification. I am going to try.