How to launch alpaca-13b with 6gb of VRAM ? #1746
Closed
vivelespatess
announced in
Announcements
Replies: 2 comments 1 reply
-
You don't. Not on gpu. This isn't magic, even if 4bit appears so at first glance. Try GGML that will run on your cpu and ram(not vram) |
Beta Was this translation helpful? Give feedback.
1 reply
-
@vivelespatess could you please close this issue? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Describe the bug
I want to run gpt4-x-alpaca-13b with only 6gb of VRAM, but i've this error :
Is there an existing issue for this?
Reproduction
i use the last version of oobabooga
Screenshot
No response
Logs
System Info
i use the last version of oobabooga with : Ryzen 7 3800x GTX 1060 6GB vram 32GB ram ddr4
Beta Was this translation helpful? Give feedback.
All reactions