Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create dataset loader for Okapi m-Hellaswag #478

Open
SamuelCahyawijaya opened this issue Mar 3, 2024 · 2 comments
Open

Create dataset loader for Okapi m-Hellaswag #478

SamuelCahyawijaya opened this issue Mar 3, 2024 · 2 comments
Assignees

Comments

@SamuelCahyawijaya
Copy link
Collaborator

Dataloader name: okapi_m_hellaswag/okapi_m_hellaswag.py
DataCatalogue: http://seacrowd.github.io/seacrowd-catalogue/card.html?okapi_m_hellaswag

Dataset okapi_m_hellaswag
Description m-Hellaswag is a multi-lingual version of Hellaswag, a commonsense inference challenge dataset. Though its questions are trivial for humans (>95% accuracy), state-of-the-art models struggle (
Subsets -
Languages ind, vie
Tasks Question Answering
License Creative Commons Attribution Non Commercial 4.0 (cc-by-nc-4.0)
Homepage http://nlp.uoregon.edu/download/okapi-eval/datasets/
HF URL https://huggingface.co/datasets/jon-tow/okapi_hellaswag
Paper URL https://arxiv.org/abs/2307.16039
@SamuelCahyawijaya SamuelCahyawijaya converted this from a draft issue Mar 3, 2024
@tellarin
Copy link
Collaborator

tellarin commented Mar 3, 2024

#self-assign

@tellarin
Copy link
Collaborator

Back working on this. Sorry for the delay.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

2 participants