-
Beta Was this translation helpful? Give feedback.
Answered by
yingfhu
Dec 12, 2023
Replies: 1 comment 1 reply
-
|
Yes, the leaderboard only gives pass@1. We also support pass@k eval following https://opencompass.readthedocs.io/en/latest/advanced_guides/code_eval.html |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Thanks, hopefully this configuration could be added to the existing documentation :) |
Beta Was this translation helpful? Give feedback.
Answer selected by
zhimin-z
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Yes, the leaderboard only gives pass@1. We also support pass@k eval following https://opencompass.readthedocs.io/en/latest/advanced_guides/code_eval.html