Any changes I can make to improve recognition of screenshots

[ianpmurphy]

I've written a small application which takes screenshots of the screen and OCRs parts of it to make decisions on when to send keystrokes.

If you are wondering why I don't use Windows Automation for this, I do, but it hasn't been suitable in some cases as it fails to give access to the text on some controls.

So, Tesseract works fairly well but fails to correctly recognise some texts in oddly simple situations. I'm reading label fields on windows apps in what should be pretty much perfect conditions - clear black text using a san serif font on a white or light grey background.

Problems I'm seeing when I pass it a bitmap with two words, one on top of another:

• it will fail to read the second word even though its clear and well within the boundries of the bitmap, so 'Abced EFG' will return just 'Abced'
• Letters which are upper case will be read as lower in unexpected ways, for example reading 'Abced EFG' as 'AbcedefG'
• Line breaks are not passed back (I assume this is an option somewhere or other)
• Line breaks are not passed through as spaces, so words get jammed together.

is there anything I should be reading which will help with this?

[SarahAmagno]

Hi :)

I think I had a similar issue, my texts were not recognized correctly, because the "words" in it were rather gibberish or unusual. As Tesseract "validates" the recognized words with LTSM (if I understood it right), it helped to disable the dictionary validation for scenarios, in which I expected no ordinary words to be recognized. Follow the instructions in this StackOverflow answer to set up a config: https://stackoverflow.com/a/26878952/10512697
or do

var engine = new TesseractEngine(dataPath, language, EngineMode.Default);
engine.SetVariable("load_system_dawg", false);
engine.SetVariable("load_freq_dawg", false);

Maybe this helps for your situation.
However, I am unsure about the line breaks.

[ianpmurphy]

Thanks for that. I had only found some docs on lowering the strictness using the defultpagesegmode setting.

I'll give it a run and let you know if it improves things.

[PeterExtrapreneur]

Did it do better?

[ianpmurphy]

Apologies, I completely forgot I said I would come back to you let you know.

Some things worked a lot better, but I'm still mystified at things it completely mis-reads, especially given that they're screen copies so the letters are crisp and clear. Luckily it doesn't particularly matter so long as its consistently incorrect. I do regex matching on the text scanned and my solution has been to put 'Expected text|' followed by whatever the ocr engine returns. I should dedicate some time to work out why some text is perfectly ocr'd and others are completely mis-read.