[Bug]:GPU SLOWNESS

[yepyhun]

Do you need to file an issue?

• I have searched the existing issues and this bug is not already filed.
• I believe this is a legitimate bug, not just a question or feature request.

Describe the bug

The Docker implementation of DeepTutor uses a CPU-only base image (python:3.11-slim), which prevents GPU-accelerated PDF processing in MinerU. This results in extreme slowness (approx. 40 seconds per page on a system with an RTX 3070 Ti) as the system falls back to CPU inference.
Log output showing high inference times:

INFO:raganything.parser:[MinerU] Predict:  13%|█▎ | 464/3696 [10:22<36:49:49, 41.02s/it]

Steps to reproduce

1. Start DeepTutor using the provided docker-compose.yml and Dockerfile.
2. Upload a large PDF for ingestion.
3. Observe the logs during the MinerU Predict phase.
4. Inference speed will be extremely slow due to CPU usage.

Expected Behavior

The Docker image should support NVIDIA GPU acceleration by default (or via a specific build target) to enable fast VLM inference, especially when the host machine has a compatible GPU.

Related Module

• Dockerfile
• docker-compose.yml

Configuration Used

• DeepTutor Version: v0.5.0
• Base Image: python:3.11-slim (original)
• GPU: NVIDIA GeForce RTX 3070 Ti
  deeptutor version 0.5

Additional Information

• Root Cause: The python:3.11-slim image does not contain the necessary NVIDIA CUDA drivers/libraries, and the default pip install -r requirements.txt installs the CPU version of PyTorch.
• Proposed Fix:
  1. Switch the base image to nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04.
  2. Install Python 3.11 on top of the CUDA base.
  3. Explicitly install the CUDA-enabled version of PyTorch during build:
     pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
  4. Ensure docker-compose.yml includes the GPU reservation block.

[scrrlt]

cuda doesn't come with python, it is more complex to manually install python. you can also change the image to fit what you need

FROM python:3.11-slim

FROM nvidia/cuda:12.4.1-cudnn-runtime-ubuntu22.04

[yepyhun]

With all due respect, is there any point in filing bug reports if the response is essentially a list of excuses for why the system doesn't work out of the box?

This is an AI-driven multimodal RAG pipeline. Shipping it with a CPU-only Docker image by default causing 10x to 20x slower inference times makes the container implementation practically useless for real-world scenarios. Claiming that installing Python on a CUDA base image is 'too complex' is a non-argument for any serious developer; it is a standard industry practice for every major AI project (e.g., Ollama, LocalAI, vLLM, or even official PyTorch images).

I'm not saying the CPU image should be removed. But for an AI tool, providing only a CPU-bound environment and then telling users to 'change the image themselves' for a core performance requirement shows a lack of interest in accessibility. I have spent 3 full days just trying to make this project functional because the foundation is so fragile.

Right now, the project feels structurally incomplete: there is no robust installer, no basic entry script or launcher for end-users, and critical performance bottlenecks are dismissed as "user-side customization". If the goal for v0.5.0 is to be a usable tool and not just a research experiment, you should provide a Docker environment (or at least a separate GPU tag) that actually utilizes the hardware it was designed for.

[scrrlt]

With all due respect, is there any point in filing bug reports if the response is essentially a list of excuses for why the system doesn't work out of the box?

This is an AI-driven multimodal RAG pipeline. Shipping it with a CPU-only Docker image by default causing 10x to 20x slower inference times makes the container implementation practically useless for real-world scenarios. Claiming that installing Python on a CUDA base image is 'too complex' is a non-argument for any serious developer; it is a standard industry practice for every major AI project (e.g., Ollama, LocalAI, vLLM, or even official PyTorch images).

I'm not saying the CPU image should be removed. But for an AI tool, providing only a CPU-bound environment and then telling users to 'change the image themselves' for a core performance requirement shows a lack of interest in accessibility. I have spent 3 full days just trying to make this project functional because the foundation is so fragile.

Right now, the project feels structurally incomplete: there is no robust installer, no basic entry script or launcher for end-users, and critical performance bottlenecks are dismissed as "user-side customization". If the goal for v0.5.0 is to be a usable tool and not just a research experiment, you should provide a Docker environment (or at least a separate GPU tag) that actually utilizes the hardware it was designed for.

respectfully if you list 7 issues in 24 hours, you're likely to be given an interim workaround specifically so it can remain functional to you. when users apply the workaround, it verifies the solution and root cause before it is implemented. this makes fixing it quicker. this is also an open-source project and by definition not out of the box. humanly you can't address all of these within 24 hours. hardware-specific optimisation firstly requires the core architectural work that is being done to make everything not fragile. you can't just chuck in a cheeky 3gb image with massive overhead. tldr; this all takes time and please have patience

[scrrlt]

just to clarify the intent here: the workaround I provided is not a dismissal of the issue. It’s a temporary path so the system remains usable while the underlying fix is designed and implemented. GPU support isn’t a one‑line change — it requires restructuring the base image, dependency chain, and build process. that work is already underway, but it can’t be completed within a single day, especially across multiple open issues. interim workarounds are standard practice in open‑source so users aren’t blocked while the long‑term solution is built. once the architecture stabilises, a proper GPU-enabled image (or build target) can be added without introducing fragility or unnecessary overhead. it’s about sequencing the work so the project remains maintainable.

[yepyhun]

Just a personal feedback.
I post bugs / whatever cause I think this is a valuable project.
It's a good thing.

But every time I get comments like, okey it's your problem it's not our problem, the program is great and you are dumb..
Maybe I am cause I'm not a programmer just a geek guy...

Maybe you should phrase your response a bit different.
But it's just my opinion.

If my feedbacks are not helpful feel free to communicate it straight out..then I won't open issues.

[scrrlt]

open issues, but i recommend consolidating into a singular so they are easier to process and address. it's not a this isnt a problem, rather there is significant core work that needs to be done to be able to do all of these things. testing is very valuable, it'd be great if you work using the dev branch because identifying root causes of issues is incredibly helpful

[scrrlt]

wait join the discord and use the channels in there so we can also converse about it?

[yepyhun]

after 5 days of trying to make it work.... with 0 success i gave up.
This project is not ready to use.

I can't make it work with llama encoder... cant rag the 100page pdf...

[pancacake]

after 5 days of trying to make it work.... with 0 success i gave up. This project is not ready to use.

I can't make it work with llama encoder... cant rag the 100page pdf...

please try the newest version if you like. We will keep improving it.