How to reproduce the experimental results in the paper?

[huang2444586]

After completing the beauty_align stage, I merged the model with checkpoint-5591 and launched the beauty-sid-rec training.
When I tested directly with checkpoint-67092 (without CoT), I obtained:
hit@1: 0.0181
hit@5: 0.0459
hit@10: 0.0471
ndcg@5: 0.0322
ndcg@10: 0.0380
Are these numbers far below the reported results simply because I skipped CoT, or could there also be a problem with my experimental settings (e.g., checkpoint selection)?

[zhangrz22]

If your training setup—such as the number of machines or certain hyperparameter configurations—differs from mine, the issue might be related to checkpoint selection. I'd recommend testing several checkpoints to observe the performance trend.

[huang2444586]

If your training setup—such as the number of machines or certain hyperparameter configurations—differs from mine, the issue might be related to checkpoint selection. I'd recommend testing several checkpoints to observe the performance trend.

By adjusting the checkpoint settings, I achieved results close to those in the paper (without CoT). Thank you for your reply.
hit@1: 0.0207
hit@5: 0.0537
hit@10: 0.0742
ndcg@5: 0.0375
ndcg@10: 0.0442

[yuanxuns]

If your training setup—such as the number of machines or certain hyperparameter configurations—differs from mine, the issue might be related to checkpoint selection. I'd recommend testing several checkpoints to observe the performance trend.

By adjusting the checkpoint settings, I achieved results close to those in the paper (without CoT). Thank you for your reply. hit@1: 0.0207 hit@5: 0.0537 hit@10: 0.0742 ndcg@5: 0.0375 ndcg@10: 0.0442

How did you adjust the checkpoint settings?