Experts per token option missing at model load screen for GGUF (reg and _hff loader) #6607
Open
1 task done
Labels
bug
Something isn't working
Describe the bug
No option to select "number" of experts to use for MOE models, in GGUF format.
Is there an existing issue for this?
Reproduction
Attempt to load a MOE GGUF model - either loader -> llamacpp OR llamacpp_HF
Screenshot
No response
Logs
System Info
Windows 11, Nvidia 4060Ti Vram. Also reported by other users, attempt to load any of my MOE models (10), from my repo. (DavidAU at hug..face). NOTE: Moes are built using NEWEST llamacpp versions, not older llamacpp/may be "broken" moes. For llama-server.exe I use this hack: ./llama-server -m i:/llm/David_AU/testfiles/Grand-Horror-MOE-4X8-series1-Q4_K_S.gguf -c 4096 -ngl 99 --override-kv llama.expert_used_count=int:4 Example using PowerShell.
The text was updated successfully, but these errors were encountered: