Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Machine Translation Multilingual Evaluation #1

Open
wants to merge 89 commits into
base: master
Choose a base branch
from
Open

Machine Translation Multilingual Evaluation #1

wants to merge 89 commits into from

Conversation

hadyelsahar
Copy link

Work in progress

  • Integration of XGLM model
  • Flores 101 dataset

jon-tow and others added 30 commits October 30, 2021 17:04
@jon-tow
Replace the fewshot_description API with a description_dict based…
8ac9926 
… interface
@jon-tow
Merge branch 'master' of https://github.com/EleutherAI/lm-evaluation-…
57e064d 
…harness into evaluator-description-option
@leogao2
Merge remote-tracking branch 'origin/master' into evaluator-descripti…
792a765 
…on-option
@leogao2
Merge remote-tracking branch 'origin/master' into evaluator-descripti…
0b7a792 
…on-option
@jon-tow
Merge branch 'master' of https://github.com/EleutherAI/lm-evaluation-…
564e061 
…harness into evaluator-description-option
@jon-tow
Merge branch 'master' of https://github.com/EleutherAI/lm-evaluation-…
ff314d6 
…harness into evaluator-description-option
@jon-tow
Merge
1d04c42
@jon-tow
Merge branch 'evaluator-description-option' of https://github.com/jon…
7a35797 
…-tow/lm-evaluation-harness into evaluator-description-option
@jon-tow
Add provide_description arg for backward compat
ee53be2
@jon-tow
Add basic description_dict
d7a8ab2 
test
@jon-tow
Remove print from test
3fdff22
@jon-tow
Fix assertion error string
09cd76c
@jon-tow
Add newline to end of file
d131995
@jon-tow
Make evaluate and simple_evaluate description args consistent
10dd7d3
@jon-tow
Remove needless call
e3ddcfc
@jon-tow
Fix assertion error message
09e1de9
@jon-tow
Remove unused import
3f06b60
@jon-tow
Fix task example link
e54380d
@jon-tow
Fix task example link
744482b
@jon-tow
Add type info to doc-string
1bc6cdb
@jon-tow
Add description_dict docs and update task-guide
acf76b5
@jon-tow
Fix doc reference
70f9273
@jon-tow
Add description_dict to results config
d34ae3c
@jon-tow
Add positional arg deprecation decorator
8ebe36b
@jon-tow
Format for consistency
aea963a
@thomasw21
Allow users to specify en headqa or es
5855f48
@leogao2
headqa: maintain backwards compatibility
cdab2c0
@leogao2
Add new testdata
22c4124
@leogao2
Make simple_evaluate take LM and Task objects directly too
7b2b2a2
@leogao2
more changes
d86aabc
leogao2 and others added 26 commits January 3, 2022 22:57
@leogao2
Merge pull request EleutherAI#244 from rokosbasilisk/asdiv
8728710 
add asdiv task
@leogao2
Update description_guide.md
d09561f
@jon-tow
Enforce rnd args with assertions
d2636b4
@jon-tow
Merge branch 'evaluator-description-option' of https://github.com/jon…
c3bec45 
…-tow/lm-evaluation-harness into evaluator-description-option
@thomasw21
Best-download have backward compatibility issue
76dc609
@leogao2
Fix fewshot_context method handling
b2f6bce
@leogao2
Merge branch 'master' of github.com:EleutherAI/lm-evaluation-harness …
5792030 
…into evaluator-description-option
@thomasw21
Actually it shouldn't be hard to fix it to be compatible with future …
c65412e 
…version
@leogao2
Update blimp fewshot_context
02a4def
@leogao2
Merge pull request EleutherAI#226 from jon-tow/evaluator-description-…
170ae09 
…option

Replace `fewshot_description` API with a `description_dict` based interface
@leogao2
Merge pull request EleutherAI#243 from bigscience-workshop/thomas/fix…
cc23812 
…_multirc

Fix multirc
@thomasw21
Merge branch 'master' into thomas/fix_best_download_version
78824d7
@leogao2
Merge pull request EleutherAI#250 from bigscience-workshop/thomas/fix…
ed6931e 
…_best_download_version

Best-download have backward compatibility issue
@thomasw21
Missed asdiv
2d9fc25
@thomasw21
Fix CB
ea3fd79
@thomasw21
Bump version
b421f05
@leogao2
Merge pull request EleutherAI#252 from bigscience-workshop/thomas/fix…
be40299 
…_best_download_version_1

Best download fix
@jon-tow
Conform asidv to the new description api
caff4f1
@leogao2
Merge pull request EleutherAI#256 from jon-tow/update-asdiv
4849702 
Conform `asidv` to the new description api
@leogao2
Merge pull request EleutherAI#254 from bigscience-workshop/thomas/fix_cb
c3d941b 
Fix CB
@leogao2
cb: add testdata
1c52e91
@jon-tow
Update headqa deprecation warning to display on init only
03c15e0
@leogao2
Merge pull request EleutherAI#259 from jon-tow/headqa-ux
26f0233 
Update `headqa` deprecation warning to display on init only
@hadyelsahar
adding support of XGLM model
1dc6eb0
@hadyelsahar
Fix new lines tokenizer issue in XGLM
c3b7267
@hadyelsahar
integration of bigscience model
fbdc61c
@StellaAthena
Copy link
Collaborator

StellaAthena commented Apr 25, 2022 •
edited
Loading

@hadyelsahar I'm taking responsibility for maintaining this repo now. Can we find some time to chat about this PR?

@hadyelsahar
Copy link
Author

Yup, will ping you over slack.

StellaAthena pushed a commit that referenced this pull request Apr 27, 2022
@jon-tow
Merge pull request #1 from cjlovering/cjlovering/gen_max_len
4941a8b 
Add optional max length to generation.
jon-tow pushed a commit that referenced this pull request May 23, 2022
@KhalidAlt
Merge pull request #1 from bigscience-workshop/master
acd329e 
Update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@hadyelsahar @StellaAthena @jon-tow @leogao2 @thomasw21 @igor0 @xagi-dev