Possibility of removing the limit for tile overlap

[milancelanmi]

Could you please remove the limit for tile overlap (currently bellow 1/2 of tile size)? I have models that are fixed to 192 and 256 fixed input dimensions. When I enable tiling, the seam artifacts (or: overlapping edges) are clearly visible. However, increasing the overlap would remove these artifacts (I've tried this with the same models on images and with ONNX scripts). Could you remove this restriction in your vs-mlrt project? Thank you very much in advance!

[milancelanmi]

Additionaly
For models that support only static dimensions (fixed tiling), like DiQP (512x512) and similar: When the input image is smaller than the required dimensions, could VSMLRT pad the image to the necessary size using an appropriate padding technique (ideally one that maintains edge continuity), then perform inference, and finally, crop the result to get the upscaled version of the original image area?
The process would look like this:
a. Check the input dimensions. If they are smaller than the dimensions the model expects, generate a fixed-size image canvas where the original image will be placed (ideally centered, but adjusting margins could also be an option if needed).
b. Fill the empty areas (padding) with an appropriate value (reflection or replicate padding is recommended, depending on how the model was trained, to minimize artifacts).
c. Apply the model (run inference) on this padded image.
d. After inference, perform the appropriate cropping on the output to obtain only the part corresponding to the original input area (taking into account any scaling – if upscaling, coordinates need to be transformed correctly).

[WolframRhodium]

Could you please remove the limit for tile overlap (currently bellow 1/2 of tile size)?

If the overlap is too large, it is not doing anything useful. Consider a 64x1 image with tilesize=(61, 1) and overlap=(30, 0), the following tiles are fed into the inference engine: x=[0:61], x=[1:62], x=[2:63], x=[3:64], because the step between tiles is calculated by

vs-mlrt/vsov/vs_openvino.cpp

Line 345 in c7794d5

auto step_w = src_tile_w - 2 * d->overlap_w;

as $\textsf{step} = = 61 - 2 * 30 = 1$.

With a larger value of overlap, the step will be negative.

When the input image is smaller than the required dimensions, could VSMLRT pad the image to the necessary size using an appropriate padding technique

Because this is a very simple task that can be implemented by the user of this library, while there is a vast number of potential options, I prefer to make this a user's problem and keeping the library minimal.

[milancelanmi]

@WolframRhodium
It would be great to implement the Gaussian weighted tiling technique as used in this script.
https://pastebin.com/W4s2b5xm

Specifically, it's a very elegant approach where:

A Gaussian weight mask is created for each tile (center has weight 1.0, edges close to 0)
Overlap regions are blended using weighted accumulation
Final result is normalized by dividing the accumulated result by the total weight

This technique eliminates seams between tiles and provides extremely smooth transitions, which is critical for high-quality AI upscaling results.

Regarding the second item, I asked Selur to implement it in Hybrid but he doesn't want to, unfortunately. If it's fairly easy for you to implement, I would kindly ask you to implement it once more.

Parameters that work excellently:
Overlap: 50% of tile size (e.g., 384px for 768px tile)
Sigma: tile_size/4 for Gaussian distribution
Weighted blending instead of regular linear blending

Thank you very much in advance! 🙏

[WolframRhodium]

I also don't want to implement it because it takes time and generally we don't use tiling at all.