Different results among many runs

[LinZichuan]

Hi @Yangsenqiao ,

I encountered inconsistent results when running the script run_efficient_gpt4o_judge.sh twice independently:

1. First run: The success tool call ratio increased rapidly, reaching ~1.0 by the 10th step. This suggests the model may have collapsed to using tools for nearly all samples This behavior does not match the green curve in Fig. 3(a) of the paper.

2. Second run: The ratio increased more slowly, reaching at ~0.65 by the 5th step.

Did you observe similar phenomenons in your experiments?

Look forward to your reply. Thanks!

[Yangsenqiao]

Dear @LinZichuan,

Thanks for your interest in VisionThink!

From our experiments, we found that setting the threshold at 0.2 generally yields stable results. I’d suggest checking the distribution of data within the effective batch—specifically, whether in some iterations the entire batch either requires image resizing or does not require it at all. I’ll also look into this on my side.