The motivation

[Germany321]

Can your model generate COT just like Deepseek-R1? If not, I am curious what is the motivation to use RL for training?

[CaptainEven]

Good question! I 'm curious abot the CoT SFT or R1 training too! Looking forward to the answer!

[xrc10]

The model can generate COT. And that is the motivation of using GRPO because the training data does not have ground truth COT annotation. Please check out the demo here: https://huggingface.co/spaces/omlab/VLM-R1-Referral-Expression

[Germany321]

Thanks for your reply. I have tried you demo and the model can really generate thought. It is interesting. Are SFT results trained on COT data? Or you just supervise the model with the IOU + format reward and let the model learn to generate COT by themselves?

[snakeztc]

There is no COT SFT involved. We just do GPRO on the Qwen-2.5 VL 3B directly and the reasoning process is emerged automatically.

[CaptainEven]

There is no COT SFT involved. We just do GPRO on the Qwen-2.5 VL 3B directly and the reasoning process is emerged automatically.

Excellent!

[Germany321]

There is no COT SFT involved. We just do GPRO on the Qwen-2.5 VL 3B directly and the reasoning process is emerged automatically.

That sounds very interesting!