Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LLaVA v1.5 7b very low performance on TextVQA #554

Open
wufeim opened this issue Oct 28, 2024 · 0 comments
Open

LLaVA v1.5 7b very low performance on TextVQA #554

wufeim opened this issue Oct 28, 2024 · 0 comments

Comments

@wufeim
Copy link

wufeim commented Oct 28, 2024

Dear authors,

Thanks for sharing this great codebase.

I tried to test LLaVA-v1.5-7b (llava_v1.5_7b) model on TextVQA_VAL and only get about 21.88 accuracy, which is much lower than the 58.2 performance reported in the paper. I understand that the codebase is not aimed to reproduce the exact results but it seems that the gap is too big.

Is there any reason why this happens and any quick fix?

Thanks very much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant