Process multiple images at the same time

[hienphan161]

Hi team,

Is it possible to process multiple images simultaneously using the ocr_predictor? Currently, when I test with 3 identical images, the processing time is nearly three times longer compared to running the process with just one image. The code I used for testing is provided below.

import os
import time

import numpy as np
import torch
from PIL import Image
from doctr.models import ocr_predictor

os.environ["ONEDNN_PRIMITIVE_CACHE_CAPACITY "] = "1"
os.environ["DOCTR_MULTIPROCESSING_DISABLE"] = "TRUE"

with torch.no_grad():
    image = np.asarray(Image.open("test.jpg").convert("RGB"))
    predictor = ocr_predictor(
        det_arch="fast_base",
        reco_arch="parseq",
        pretrained=True,
        assume_straight_pages=False,
        detect_orientation=True,
        resolve_blocks=False,
        straighten_pages=False,
    )

    start_time = time.time()
    predictor([image])
    end_time = time.time()
    print(
        "Time taken for 1 image:", end_time - start_time
    )  # 2.3408219814300537 seconds

    start_time = time.time()
    predictor([image, image, image])
    end_time = time.time()
    print(
        "Time taken for 3 images:", end_time - start_time
    )  # 7.097523927688599 seconds

[felixdittrich92]

Hi @hienphan161 👋,

This would require to load the model n times in a multiproc pool

I would suggest to tune the det_bs and rec_bs to your hardware that's more efficient :)

[felixdittrich92]

Otherwise you can try: https://github.com/felixdittrich92/OnnxTR

That's more prod / runtime focused
It's the same as docTR but with Onnxruntime - especially on CPU it provides a massive latency boost