split up prediction to avoid overly large batches (causing OOM)

[bertsky]

No description provided.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 82.56881% with 19 lines in your changes missing coverage. Please review.

Project coverage is 69.50%. Comparing base (4adf09f) to head (842bd92).

Files with missing lines	Patch %	Lines
ocrd_calamari/recognize.py	82.56%	10 Missing and 9 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #116      +/-   ##
==========================================
- Coverage   71.07%   69.50%   -1.58%     
==========================================
  Files           5        5              
  Lines         204      223      +19     
  Branches       50       57       +7     
==========================================
+ Hits          145      155      +10     
- Misses         48       53       +5     
- Partials       11       15       +4     

Flag	Coverage Δ
	69.50% <82.56%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[bertsky]

Assuming we can soon rely on Calamari-OCR/calamari#361, this PR should be changed to re-use the batch_size kwarg of predict_raw instead.

[mikegerber]

Also includes part of #104

[bertsky]

Also includes part of #104

You mean removes that part?

That's because I merged your upstream fixes into Calamari v1.0.7 so we won't need the workaround.

[bertsky]

Assuming we can soon rely on Calamari-OCR/calamari#361, this PR should be changed to re-use the batch_size kwarg of predict_raw instead.

Actually, Calamari1's predict_raw with delegated batch_size param is still inferior to Calamari2's setup with length bucketing based on the corresponding feature of TF2. The problem is that even a single long line (esp. if it is also low in height and thus needs to be scaled up for target height) can extremely increase the memory requirements for the entire batch due to zero padding. So as long as we use Calamari1 we should probably rather do our own "bucketing", or at least split up batches with huge lengths to avoid OOM...

[mikegerber]

You mean removes that part?
That's because I merged your upstream fixes into Calamari v1.0.7 so we won't need the workaround.

#104 is about removing the workaround, so it overlaps with/is superseded by this PR.

[bertsky]

Actually, Calamari1's predict_raw with delegated batch_size param is still inferior to Calamari2's setup with length bucketing based on the corresponding feature of TF2. The problem is that even a single long line (esp. if it is also low in height and thus needs to be scaled up for target height) can extremely increase the memory requirements for the entire batch due to zero padding. So as long as we use Calamari1 we should probably rather do our own "bucketing", or at least split up batches with huge lengths to avoid OOM...

Implemented in bertsky@45e20b1

[bertsky]

#104 is about removing the workaround, so it overlaps with/is superseded by this PR.

ah, yes, but it does not point to 1.0.7 but to the commit ref – so we should merge both PRs

[mikegerber]

#104 is about removing the workaround, so it overlaps with/is superseded by this PR.

ah, yes, but it does not point to 1.0.7 but to the commit ref – so we should merge both PRs

Should probably have written: "Note to self:", my comment just served as a reminder for me to have a look at both PRs.

[mikegerber]

LGTM.

I'll merge it after a manual test.

[mikegerber]

Manual test looks OK, merging!

[mikegerber]

@stweil This may fix the OOM you have seen, but it's hard to tell without the data!