sequential image stitching getting blurred

[kxg3030]

Hello, this library is great. I am planning to apply it to my project, but I have found some issues and would like to seek help.
My application scenario is to continuously capture images from the screen and then stitch them together in a loop. I found that as the number of stitches increases, the previously synthesized parts become increasingly blurry, while the newly stitched parts do not experience this situation. I found a similar problem online:
https://answers.opencv.org/question/54891/sequential-image-stitching/

Is there any way to solve this problem

[kxg3030]

my code

setting = {
            "confidence_threshold": 1,
            "detector"            : "sift",
            "try_use_gpu"         : True,
            "crop"                : False,
        }
        self.stitcher = AffineStitcher(**setting)
        self.captureIns = mss.mss()
        while True:
            try:
                screenshot = self.captureIns.grab({
                    'left'  : start.x(),
                    'top'   : start.y(),
                    'width' : end.x() - start.x(),
                    'height': end.y() - start.y()
                })
                self.number += 1
                img = Image.frombytes('RGB', screenshot.size, screenshot.bgra, 'raw', 'BGRX')
                img = img.convert("RGBA")

                cv2.imwrite("y.png", img)
                before = cv2.imread("x.png")
                current = cv2.imread("y.png")

                if True:
                    stitchAfter = self.stitcher.stitch([before, current])
                    cv2.imwrite("x.png", stitchAfter)
                    self.dataSlot.emit(current, stitchAfter)
                self.lastCapture = current
            except Exception as e:
                pass

[lukasalexanderweber]

This was also reported in #84. The most important thing would be sample images. They could be shared here

[kxg3030]

I have submitted my sample PR
https://github.com/OpenStitching/images/pull/3

[lukasalexanderweber]

Can you provide a code snippet that works standalone? Which libraries must be installed? What is mss.mss()?

[kxg3030]

• prepare the sample images first, for example: ./Image

• run code

import glob
import os.path
import re

import cv2
from stitching import AffineStitcher


def extract_number(filename):
    match = re.search(r'(\d+)', filename)
    return int(match.group(0)) if match else 0


if __name__ == "__main__":
    setting = {
        "confidence_threshold": 1,
        "detector"            : "sift",
        "try_use_gpu"         : True,
        "crop"                : False,
    }
    stitcher = AffineStitcher(**setting)
    # get sample image list
    sample = glob.glob("Image/*.png")
    images = sorted(sample, key = extract_number)
    del images[0]
    for _, i in enumerate(images):
        current = cv2.imread(i)
        if not os.path.exists("first.png"):
            cv2.imwrite("first.png", current)
            continue
        try:
            first = cv2.imread("first.png")
            res = stitcher.stitch([first, current])
            cv2.imwrite("first.png", res)
            cv2.imshow("new", res)
            cv2.waitKey(0)
        except Exception as e:
            pass

The stitched result of each image will be displayed in a window, and you can continuously close the window to see the changes in the composite image. The final stitched result will be saved to first.png

[kxg3030]

Hello, is there any way to solve this problem?

[lukasalexanderweber]

You can change

for idx, i in enumerate(images):

and then below

res = stitcher.stitch([first, current])

add

stitcher.stitch_verbose([first, current], verbose_dir=idx)

Monitor how the resolutions of the intermediate Images (medium, low) and the resolutions of the final result after each step evolves. I did have no time to look at it myself. I suspect that the difference of resolution between the big image and the new frame grows bigger and bigger leads to issues we are not aware yet. But you can help figuring this out

[lukasalexanderweber]

stitch_verbose creates a directory with intermediate results

[kxg3030]

Thank you for following your method, but based on this log, I don't know how to adjust it. I'm not very good at image processing, and I hope to be able to solve this problem in the future