[QUESTION] Recognized text is at wrong position

[j0na1han]

Hi,

I try to mark the text which your library recognized.

The code:
`
const frameProcessor = useSkiaFrameProcessor((frame) => {
'worklet'

    frame.render()
    const data = scanOCR(frame)

    const paint = Skia.Paint()
    paint.setColor(Skia.Color('red'))

    if (Object.keys(data).length != 0) {
        for (const block of data.result.blocks) {
            const rect = Skia.XYWHRect(block.frame.x, block.frame.y, block.frame.width, block.frame.height)
            frame.drawRect(rect, paint)
        }
    }

})
`

I added a video with the behavior. Do you have an idea why this so?

text.MP4

react-native-vision-camera: 4.5.3
@ismaelmoreiraa/vision-camera-ocr: 3.0.2-1

[ismaelsousa]

hello, it should be related to the rotation of the frame.
what is the ratio of the frame and the ratio of the camera view?

[j0na1han]

Hi,
the pixel ratio of the camera view is 3.

How can I get the ratio of the frame?

Here again a video without my special case.
https://github.com/user-attachments/assets/634aa738-3bae-4d68-b0e3-38d468b2142b

[ismaelsousa]

You can get the ratio from the frame inside the frame processor

But yeah, I need to take a look at that.
We need to convert the bounding boxes to the same ratio and scale of the camera view.

[ismaelsousa]

Wait, hmm, odd, are you using skia to draw the bounding boxes?

I never tried this. You can check the coordinates to see what is happening when you start to move the text to the middle of the screen.

[j0na1han]

Yes I do. You mean it is a problem with react-native-vision camera?

At the top right corner is 0,0. I would expect that in the bottom left corner is 1920,1080. Somehow the max value is not greater than 1000,1000.

[ismaelsousa]

hmm, I guess in ML Kit the (0,0) is at top left