[Release] LMMs-Eval v0.3.0, Audio Evaluation#426

Merged

KairuiHu merged 48 commits intomainfrom feat-dev

Nov 27, 2024

+9,828-18

Commits on Nov 27, 2024

[Feat] Add qwen2_audio model support and Automatic speech recognition task with LibriSpeech dataset (#289 )

authored and
kcz358
committed
add clotho_aqa task

pbcong
authored and
kcz358
committed
Apply black formatting

pbcong
authored and
kcz358
committed
formatting

pbcong
authored and
kcz358
committed
excluding xl due to downloading issue.

Luodian
authored and
kcz358
committed
[Feat] add audiobench version of clothoaqa (#302 )

pbcong
authored and
kcz358
committed
Add AIR_bench task (#315 )

pbcong
authored and
kcz358
committed
add common_voice_15 and people_speech tasks (#316 )

authored and
kcz358
committed
add indent to yaml

Yingluo-momo
authored and
kcz358
committed
Add openhermes task (#323 )

pbcong
authored and
kcz358
committed
[Refactor] Fixing doc to audio return type, qwen_audio revise (#329 )
kcz358
committed
add muchomusic and vocalsound task (#331 )

pbcong
authored and
kcz358
committed
add alpaca audio task (#333 )

pbcong
authored and
kcz358
committed
[feat] added gigaspeech config (#334 )

authored and
kcz358
committed
add tedlium_long_form and tedlium_dev_test tasks (#345 )

authored and
kcz358
committed
[Feat] add-wavcaps (#349 )

authored and
kcz358
committed
Update dep and fix log samples for audio (#355 )
kcz358
committed
fix vocalsound (#362 )

pbcong
authored and
kcz358
committed
Add using simple prompt for Qwen2 Audio to align (#360 )
kcz358
committed
Add retry for gpt api call and improve air_bench aggregation function (#376 )

pbcong
authored and
kcz358
committed
[Feat] Add mix_evals audio2text (#420 )
kcz358
committed
Gemini Audio (#421 )

pufanyi
authored and
kcz358
committed
Revise prompt
kcz358
committed
delete redundant tasks in gigaspeech

KairuiHu
authored and
kcz358
committed
Fix wavcaps bugs
kcz358
committed
Add lmms-eval-0.3 docs
kcz358
authored
Update lmms-eval-0.3.md
kcz358
authored
fix errors in markdown and add hyperlinks
KairuiHu
committed
proofread markdown and fix errors
KairuiHu
committed
rewrite some parts to fix errors
KairuiHu
committed
rewrite some parts to fix errors
KairuiHu
committed
rewrite some parts to fix errors
KairuiHu
committed
try optimize the table format using html
KairuiHu
committed
try optimize the table format using html
KairuiHu
committed
try optimize the table 2 format
KairuiHu
committed
final proofread
KairuiHu
committed
final proofread
KairuiHu
committed
final proofread
KairuiHu
committed
add explanantion for AIF and ASR
KairuiHu
committed
standardize WER to WER(↓)
KairuiHu
committed
final proofread
KairuiHu
committed
final proofread
KairuiHu
committed
final proofread
KairuiHu
committed
final proofread
KairuiHu
committed
correct hyperlink errors
KairuiHu
committed
modify readme to support lmms-eval0.3.0 release
KairuiHu
committed
modify icon
KairuiHu
committed
fix typos
KairuiHu
committed