Acceleration effect on Qwen2.5VL

[zry13]

Thank you for your contribution to the open-source community! I noticed that you open-sourced the implementation for Qwen2.5VL. However, I encountered some issues during testing and hope to get some advice.

I used your open-source code to evaluate VisionZip's acceleration effect on Qwen2.5VL using the MMStar benchmark, but it actually took more time than expected. I'm wondering what the acceleration ratio of VisionZip for Qwen2.5VL was approximately in your experiments, and whether the evaluation code could be made publicly available?

[Yangsenqiao]

Thanks for your interest in our VisionZip.

Since computing attention weights is necessary for our approach—and Flash Attention does not expose them directly—it becomes inevitable to recompute attention in the vision encoder or adopt alternative methods. Therefore, when comparing VisionZip on Qwen2.5-VL with Flash Attention 2 to the vanilla Qwen2.5-VL with Flash Attention 2, our method may not yield significant speed gains when the number of visual tokens is small.

Besides, the community has made great progress—frameworks like VLLM and SGLang are becoming increasingly popular. You might try running inference with VLLM using a maximized batch size to assess VisionZip speed improvements over the full dataset.

Also, VisionZip was originally designed for the LLaVA family, and integration with Qwen models remains preliminary. We believe there’s substantial room for optimization—e.g., exploring low-rank approximations of attention weights rather than recomputing them exactly.

Recently, we've also experimented with reinforcement learning to train more efficient reasoning VLMs, VisionThink.

And we're also excited to see the community develop better efficient VLM algorithms, making VLMs more practical and accessible in real-world applications.