Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The leaderboard is broken #14

Open
constanzafierro opened this issue Nov 5, 2024 · 2 comments
Open

The leaderboard is broken #14

constanzafierro opened this issue Nov 5, 2024 · 2 comments

Comments

@constanzafierro
Copy link

The HF leaderboard https://huggingface.co/spaces/uonlp/open_multilingual_llm_leaderboard has a Runtime error.

@souzatharsis
Copy link

Following this - leaderboard is still broken

@robbiemu
Copy link

robbiemu commented Nov 13, 2024

can confirm, and have a solution. First, to confirm:

docker run -it -p 7860:7860 --platform=linux/amd64 \
        registry.hf.space/uonlp-open-multilingual-llm-leaderboard:latest python app.py
Unable to find image 'registry.hf.space/uonlp-open-multilingual-llm-leaderboard:latest' locally
latest: Pulling from uonlp-open-multilingual-llm-leaderboard
6e840ff50b0d: Pull complete
aed53d313d12: Pull complete
14eb3354604d: Pull complete
7b8a2f6ba895: Pull complete
2d36fb591704: Pull complete
8b61c7e85d4a: Pull complete
790d36a0aa57: Pull complete
Digest: sha256:d39d89be3ea9ca110f24318dd735a9e6e83aa810a0f4f08799592f03ac5c76ab
Status: Downloaded newer image for registry.hf.space/uonlp-open-multilingual-llm-leaderboard:latest
/usr/local/lib/python3.10/site-packages/gradio/analytics.py:106: UserWarning: IMPORTANT: You are using gradio version 4.38.1, however version 4.44.1 is available, please upgrade.
--------
  warnings.warn(
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.

Solution (for those who want to see it before the fix it):

  • exec bash on the running container and run apt-get update && apt-get install -y vim
  • vi app.py and go to the launch() command to insert "share=True" in the launch command.
  • restart the container

visit the url in the logs, replacing "0.0.0.0" with "localhost"

FOR THE MAINTANERS:
there are scores in your PRs so I don't know if you guys are leaving this on the back burner but if you do repair the container, you might also audit the security of the image. there are some old commands/installs in there:

~/trivvy_summary registry.hf.space/uonlp-open-multilingual-llm-leaderboard:latest
Total Vulnerabilities: 2135, Critical: 25, High: 316, Top 3: "It was found that apt-key in apt, all versions, do not correctly valid ..., [Privilege escalation possible to other user than root], binutils: Memory leak with the C++ symbol demangler routine in libiberty"

(here's that script: https://gist.github.com/robbiemu/1047ff78047a6c25909c06eceafa1965 )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants