Discussions

• 1 You must be logged in to vote

  💡

  Providing the full-scale model testing cmd for each model

  abse4411 started Sep 25, 2023 in Ideas

  

  

  0
• 1 You must be logged in to vote

  🙏

  请问命令行模式进行测评时怎么输出summary

  L-hongbin asked Sep 8, 2023 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  --debug命令可以正常推理，但是不使用debug则无法推理

  ZzzybEric asked Sep 22, 2023 in Q&A · Closed · Answered

  

  

  4
• 1 You must be logged in to vote

  💡

  Will opencompass support model EleutherAI / pythia?

  taomanwai started Sep 22, 2023 in Ideas

  

  

  1
• 1 You must be logged in to vote

  🙏

  增加predictions文件中保存的字段

  PamKing7 asked Sep 21, 2023 in Q&A · Unanswered

  

  

  2
• 1 You must be logged in to vote

  #️⃣

  Add Text Classification dataset: CoLA 添加文本分类数据集支持 CoLA

  Enhancement New feature or request StageAwaitingAssignment SIG Medium

  vansin started Aug 29, 2023 in Community Task

  

  

  2
• 1 You must be logged in to vote

  🙏

  Questions about MMbench

  crystraldo asked Sep 15, 2023 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  🙏

  如何设置模型推理时的参数，如top_k? How to set the generation config of the model, like the temperature.

  royny asked Aug 29, 2023 in Q&A · Answered

  

  

  

  

  4
• 1 You must be logged in to vote

  🙏

  有Llama2-70B评测结果在Humaneval上的prompt输入和模型输出结果吗？评测没有复现，想借鉴下

  gongjunjin asked Sep 13, 2023 in Q&A · Unanswered

  

  

  1
• 1 You must be logged in to vote

  #️⃣

  Add text generation dataset WikiText. 添加文本生成数据集WikiText

  Enhancement New feature or request StageAssigned SIG Medium

  vansin started Aug 29, 2023 in Community Task

  

  

  2
• 1 You must be logged in to vote

  🙏

  Can the evaluation continue after modifying the model configuration? 模型配置更改后，评测是否能够继续？

  niexufei asked Sep 11, 2023 in Q&A · Answered

  

  

  2
• 1 You must be logged in to vote

  🙏

  对于ppl/loglikelihood的评测方式，就单选题而言，需要给模型看到其他选项吗？

  RanchiZhao asked Sep 7, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  huggingface openLLM 跑分和 opencompass 的跑分不一致的原因是什么？

  amulil started Sep 4, 2023 in View all discussions

  

  

  1
• 1 You must be logged in to vote

  💬

  Plan for upgrading faiss-gpu

  yeoedward started Sep 4, 2023 in View all discussions

  

  

  1
• 1 You must be logged in to vote

  🙏

  test

  gaotongxiao asked Sep 4, 2023 in Q&A · Closed · Unanswered

  

  

  0
• 1 You must be logged in to vote

  #️⃣

  Support evaluation of HumanEval dataset when k > 1 in pass@k 支持HumanEval k > 1 in pass@k的情况

  StageAwaitingAssignment SIG Hard Algorithm Reproduction

  vansin started Aug 29, 2023 in Community Task

  

  

  0
• 1 You must be logged in to vote

  #️⃣

  Add text generation dataset ToxiGen. 添加文本生成数据集ToxiGen

  Enhancement New feature or request StageAwaitingAssignment SIG Medium

  vansin started Aug 29, 2023 in Community Task

  

  

  0
• 1 You must be logged in to vote

  #️⃣

  Add Multi-Turn Question Answering dataset HotpotQA. 添加Multi-Turn Question Answering数据集HotpotQA

  Enhancement New feature or request StageAwaitingAssignment SIG Medium

  vansin started Aug 29, 2023 in Community Task

  

  

  0
• 1 You must be logged in to vote

  #️⃣

  Add General Question Answering dataset WebQuestions. 添加General Question Answering数据集WebQuestions

  Enhancement New feature or request StageAwaitingAssignment SIG Medium

  vansin started Aug 29, 2023 in Community Task

  

  

  0
• 1 You must be logged in to vote

  #️⃣

  Add coreference resolution dataset: XWinoGrad(both Chinese and English) 添加Coreference Resolution数据集支持 XWinograd(中英文)

  Enhancement New feature or request StageAwaitingAssignment SIG Medium

  vansin started Aug 29, 2023 in Community Task

  

  

  0
• 1 You must be logged in to vote

  #️⃣

  Add NLI dataset: XNLI(both Chinese and English) 添加NLI数据集支持 XNLI(中英文)

  Enhancement New feature or request StageAwaitingAssignment SIG Medium

  vansin started Aug 29, 2023 in Community Task

  

  

  0
• 1 You must be logged in to vote

  🙏

  How to run IWSLT2017Dataset?

  zhangyikaii asked Sep 2, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  🙏

  About the evaluation of OpenChat-13b-V3.2 etc.

  niexufei asked Sep 1, 2023 in Q&A · Answered

  

  

  1
• 1 You must be logged in to vote

  🙏

  怎么一次导入排行榜上的所有任务进行评分，看文档只有导入单一任务的教程?

  amulil asked Aug 30, 2023 in Q&A · Unanswered

  

  

  0
• 1 You must be logged in to vote

  💬

  generation args

  shiweiba started Aug 29, 2023 in View all discussions

  

  

  1