[Bug]: GPU CPU torch cuda problem

[yepyhun]

Do you need to file an issue?

• I have searched the existing issues and this bug is not already filed.
• I believe this is a legitimate bug, not just a question or feature request.

Describe the bug

MinerU (used by RAGAnything) fails to detect the available GPU memory (8GB RTX 3070 Ti) and defaults to 1 GB VRAM with batch_size: 1. This occurs because the environment has a CPU-only version of PyTorch installed (2.9.1+cpu), forcing MinerU's VLM inference engine to run entirely on the CPU.

This results in:

• 100% CPU usage.
• Extremely slow document processing (approx. 3s per inference step).
• Inability to utilize the dedicated hardware.

Steps to reproduce

1. Set up a Python environment with a CPU-only PyTorch version (e.g., pip install torch --index-url https://download.pytorch.org/whl/cpu).
2. Install raganything and mineru.
3. Run the RAG indexing pipeline on a document containing images/tables.
4. Observe the backend logs during the MinerU VLM phase.

Expected Behavior

The system should detect the presence of the NVIDIA GPU (via torch.cuda.is_available()) and correctly identify the available VRAM (8GB). The batch_size should automatically adjust (e.g., to 4 or 8) for optimal performance, and inference should run on the GPU.

Related Module

mineru.backend.vlm.utils (GPU detection logic) / raganything

Configuration Used

• Operating System: Windows 11
• GPU: NVIDIA RTX 3070 Ti (8GB VRAM)
• Python Version: 3.11
• PyTorch Version: 2.9.1+cpu (Incorrect)

Logs and screenshots

Backend Log:

[Backend]  INFO:raganything.parser:[MinerU] ... Using transformers as the inference engine for VLM.
[Backend]  INFO:raganything.parser:[MinerU] ... gpu_memory: 1 GB, batch_size: 1
[Backend]  INFO:raganything.parser:[MinerU] ... get transformers predictor cost: 3.02s

Additional Information

• Root Cause: The mineru utilities rely on torch.cuda to query device properties. If torch is CPU-only, these calls return false/default values.
• Fix: Reinstalling PyTorch with CUDA support resolves the issue:

  pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124 --force-reinstall

[scrrlt]

cpu optimisation is also a work in progress. if you have gpu constraints i recommend docker. rag systems are heavy

[pancacake]

hey, please pull the newest version on main branch, and try the lightwise pipeline such as using docling, or llamaindex. Let me know if there's any other problems

[yepyhun]

As I said I gave up to make it work.
I worked on it like 10+ hours to be able to study...

I rather study from the book.... Or use notebook LLM by Google.

This app is not working reliable.... Sorry.

Maybe in some circumstances it is... But not for me...