pretrained models for SD1.5 Canny

[qiushu-chen]

While using ControlNeXt’s SD1.5 and SDXL versions for training my dataset, I encountered issues with recovering CT images from Canny-extracted edges. The SDXL version, despite having a pre-trained model, consistently encounters dimension mismatch errors, rendering it unusable. Conversely, the SD1.5 model lacks a pre-trained Canny-related model. After training my dataset from scratch for 5 epochs, the output fails to correspond to the conditioned edges, displaying erratic results. Noting the previous experiments with SD1.5 and Canny images on your website, I inquire if it’s possible to obtain the relevant pre-trained model.

[qiushu-chen]

[qiushu-chen]

And when I used the pretrained models in Pbihao
/
ControlNeXt , the following errors occored:
RuntimeError: Error(s) in loading state_dict for ControlNeXtModel:
size mismatch for down_res.0.norm1.weight: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for down_res.0.norm1.bias: copying a param with shape torch.Size([3]) from checkpoint, the shape in current model is torch.Size([128]).
size mismatch for down_res.0.conv1.weight: copying a param with shape torch.Size([128, 3, 3, 3]) from checkpoint, the shape in current model is torch.Size([128, 128, 3, 3]).
size mismatch for mid_convs.0.2.weight: copying a param with shape torch.Size([256, 256, 3, 3]) from checkpoint, the shape in current model is torch.Size([256]).
size mismatch for mid_convs.1.weight: copying a param with shape torch.Size([1280, 256, 1, 1]) from checkpoint, the shape in current model is torch.Size([320, 256, 1, 1]).
size mismatch for mid_convs.1.bias: copying a param with shape torch.Size([1280]) from checkpoint, the shape in current model is torch.Size([320]).

[qiushu-chen]

[qiushu-chen]

So the pretrained model cannot be referred to or be fine-tuned? Are there any solutions?