Skip to content

Releases: amazon-science/mxeval

v1.2

09 Jun 22:58
32aad3e
Compare
Choose a tag to compare

Canonical solutions release

We have released canonical solutions for certain popular langauges (v1.2). The detailed numbers of the solutions for each langauge are listed below.

Dataset # solutions # problems
MBCPP 773 848
MBCSP 725 968
MBJP 874 966
MBJSP 938 966
MBKP 796 966
MBPHP 950 966
MBPP 960 974
MBRBP 784 966
MBTSP 967 968

@wshiqi-aws 👏 🎉 thank you for the PR!

v1.1

27 Dec 21:54
c035b0a
Compare
Choose a tag to compare

v1.1

Bugfix:

Fix unit test cases for 47 problems’ test assertions for C#/TypeScript/Go, which represents ~5% of all problems:

  • Root cause of the issue is a possibility for the input parameters to the canonical solutions get mutated as a side-affect which cause the captured input to mismatch.
  • We fix this issue by saving another copy of the function input before passing it for execution.
  • Update README with new results for MBGP, MBTSP, MBCSP with the fix.

First release

25 Oct 19:49
Compare
Choose a tag to compare
v1.0

Release v1