Skip to content

Commit

Permalink
Update lmms-eval-0.3.md
Browse files Browse the repository at this point in the history
  • Loading branch information
Luodian authored Dec 3, 2024
1 parent 1572e6f commit 0589d0f
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion docs/lmms-eval-0.3.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ While many models have made progress in integrating audio understanding, there i

To address this, we introduce an upgrade to the `lmms-eval` framework, focusing on audio understanding. Building on the success of `lmms-eval/v0.2.0`, the new `lmms-eval/v0.3.0` includes dedicated modules and designs for audio tasks, ensuring consistent evaluation across audio and visual modalities.

This upgrade includes multiple benchmarks for audio understanding and instruction following, enabling standardized and reproducible comparisons of various audio models.
Our upgrade also includes many new features, including faster model and task loading, SGLang/GPT4 Batch API integrations, and LMMs-Eval analysis tool for improving evaluation efficiency and comprehensiveness.

## Audio Evaluation Pipeline

Expand Down

0 comments on commit 0589d0f

Please sign in to comment.