Skip to content

Commit b4cf1ac

Browse files
Update README
1 parent 398492a commit b4cf1ac

File tree

1 file changed

+5
-3
lines changed

1 file changed

+5
-3
lines changed

README.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
1-
# Expedition Aya: Reward Model Multilingual
1+
# M-RewardBench: Evaluating Reward Models in Multilingual Settings
22

3-
In this project, we evaluate reward models on their multilingual capabilities.
3+
This repository contains the source code for M-RewardBench, a benchmark and toolkit for evaluating reward models in multilingual settings.
44
We translated [RewardBench](https://huggingface.co/datasets/allenai/reward-bench) into 23 diverse languages and evaluated several open-source and multilingual LLMs on their chat, safety, and reasoning capabilities.
5-
This project is part of [Cohere for AI's Expedition Aya 2024](https://sites.google.com/cohere.com/expedition-aya/home), a 6-week open build challenge.
5+
This project was part of [Cohere for AI's Expedition Aya 2024](https://sites.google.com/cohere.com/expedition-aya/home), a 6-week open build challenge.
66

77
<p align="center">
88
<b><a href="https://huggingface.co/datasets/aya-rm-multilingual/multilingual-reward-bench">🤗 Dataset</a></b>
@@ -16,6 +16,8 @@ This project is part of [Cohere for AI's Expedition Aya 2024](https://sites.goog
1616

1717
## News
1818

19+
- [2024-10-28] We've published our research, M-RewardBench: Evaluating Reward Models in Multilingual Settings, as an arXiv [**preprint!**](https://arxiv.org/abs/2410.15522)
20+
- [2024-10-20] Added a **Translation** sub-category to evaluate RM preferences on translation tasks (de<->en, zh<->en). We also improved the translation quality of the benchmark by using the Google Translate API and performing manual filtering and verification.
1921
- [2024-08-28] We won **Silver Prize** in Expedition Aya 2024! We're also releasing the v1 of the multilingual RewardBench on [HuggingFace](https://huggingface.co/datasets/aya-rm-multilingual/multilingual-reward-bench).
2022

2123
## Setup and installation

0 commit comments

Comments
 (0)