New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate Berkeley Function Calling Leaderboard into Evalchemy #28

Open

CharlieJCJ opened this issue Nov 21, 2024 · 2 comments

Assignees

Collaborator

CharlieJCJ commented Nov 21, 2024

Source:
https://gorilla.cs.berkeley.edu/leaderboard.html
https://github.com/ShishirPatil/gorilla/tree/main/berkeley-function-call-leaderboard

CharlieJCJ self-assigned this

Collaborator Author

CharlieJCJ commented Nov 21, 2024 •

edited

Loading

I'll lay down the TODOs here, I'll be main point of contact for the integration.

For visibility @HuanzhiMao @Fanjia-Yan @ShishirPatil

Collaborator Author

CharlieJCJ commented Nov 21, 2024 •

edited

Loading

FYI, evalchemy use github subtree for updating external codebase https://github.com/mlfoundations/evalchemy?tab=readme-ov-file#adding-external-evaluation-repositories, so our changes in BFCL codebase can be actively reflected in evalchemy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment