Alternative pdf processing

[schoeppe]

Maybe this alternative implementation is useful:
Because Elasticsearch with the ingest-attachment plugin/Apache Tika does a good job indexing the text parts of PDFs we only OCR the image parts.

This implementation depends on the Linux command line tool pdfimages (sudo apt-get install poppler-utils) instead of ImageMagick and ghostscript for PDF processing. ImageMagick is still needed for processing images though, but the extra configuration for PDF processing (mentioned here) is not neccessary.

This could lead to a significant speedup when processing "real" PDFs.
Tested with the Nextcloud Manual the speedup was 350%.

[ArtificialOwl]

Hello @schoeppe and tanks for your patience.

I did some test on my side of your solution, but I have the same result. 179s with your PR against 183 with master. Are you sure about your speedup ?

[schoeppe]

Hi @daita,
if I remember correctly this implementation is not faster for PDFs containing only images, e.g. scanned documents. For these PDFs the performance should be about the same as the original implementation.

But for PDFs containing Text and Images, e.g. the "Nextcloud Manual" there should be a significant performance gain. Did you get your results from indexing the "Nextcloud Manual"?

[ArtificialOwl]

yes, I did some comparaison using Nextcloud Manual.pdf.

The thing is that your edit only affect the way to generate the image files from pdf, not the OCR itself which is done via tesseract, right ?

[schoeppe]

My version extracts all images embedded in the PDF. The OCR is the same.
I will do some more benchmarks when I have time and get back to you with the results!