Problem of inferring image resolution

[tzayuan]

I'm encountering unusual inference outputs when using images smaller than 512×512 resolution. Results become relatively normal only at 512×512 and larger resolutions.

Questions:​​

1. What could be the potential reasons for this behavior?
2. Is it possible to achieve normal inference for resolutions below 512×512?

Environment:

ckpt: 1_2_16/lc.ckpt
command: python inference_partition.py --ckpt_sd ./weight/v2-1_512-ema-pruned.ckpt --ckpt_lc ./weight/1_2_16/lc.ckpt --config configs/model/diffeic.yaml --input /kodak/input_all --output ./outputs/kodak_256 --steps 50 --device cuda

Thanks for your insights!

[huai-chang]

Hi, sorry for the late reply😭. Regarding this issue, I think there are three main reasons:

1. SD 2.1 was trained on 512×512 resolutions, so its denoising UNet and VAE are naturally optimized for that scale. When the input resolution is significantly smaller, the model’s learned priors may not generalize well, leading to abnormal inference results.
2. The SD VAE is lossy, and from my experience, the reconstruction quality becomes worse as the input resolution decreases.
3. Lower-resolution images contain less redundancy, making high-ratio compression much more challenging.