qwen2.5-vl speed

[vmmm123]

Thanks you for bringing the incredible job!
I wonder why the speed on qwen2.5-vl is nearly the same as the original model. I test it on llama-factory dataset format by pytorch. Could you please provider the infer script or more detailed instrcutions.

[Yangsenqiao]

Thanks for your interest in our VisionZip.

Our scripts follow the lmms-eval framework. Regarding speed, here are some of my thoughts:

Since computing attention weights is necessary for our approach—and Flash Attention does not expose them directly—it becomes inevitable to recompute attention in the vision encoder or adopt alternative methods. Therefore, when comparing VisionZip on Qwen2.5-VL with Flash Attention 2 to the vanilla Qwen2.5-VL with Flash Attention 2, our method may not yield significant speed gains when the number of visual tokens is small.

Besides, the community has made great progress—frameworks like VLLM and SGLang are becoming increasingly popular. You might try running inference with VLLM using a maximized batch size to assess VisionZip speed improvements over the full dataset.

Also, VisionZip was originally designed for the LLaVA family, and integration with Qwen models remains preliminary. We believe there’s substantial room for optimization—e.g., exploring low-rank approximations of attention weights rather than recomputing them exactly.

Recently, we've also experimented with reinforcement learning to train more efficient reasoning VLMs, VisionThink.

And we're also excited to see the community develop better efficient VLM algorithms, making VLMs more practical and accessible in real-world applications.