Releases: amazon-science/mxeval
Releases · amazon-science/mxeval
v1.2
Canonical solutions release
We have released canonical solutions for certain popular langauges (v1.2). The detailed numbers of the solutions for each langauge are listed below.
Dataset | # solutions | # problems |
---|---|---|
MBCPP | 773 | 848 |
MBCSP | 725 | 968 |
MBJP | 874 | 966 |
MBJSP | 938 | 966 |
MBKP | 796 | 966 |
MBPHP | 950 | 966 |
MBPP | 960 | 974 |
MBRBP | 784 | 966 |
MBTSP | 967 | 968 |
@wshiqi-aws 👏 🎉 thank you for the PR!
v1.1
v1.1
Bugfix:
Fix unit test cases for 47 problems’ test assertions for C#/TypeScript/Go, which represents ~5% of all problems:
- Root cause of the issue is a possibility for the input parameters to the canonical solutions get mutated as a side-affect which cause the captured input to mismatch.
- We fix this issue by saving another copy of the function input before passing it for execution.
- Update README with new results for MBGP, MBTSP, MBCSP with the fix.
First release
v1.0 Release v1