Loving the simplified cmake files! Just a quick Q on switching CPU & GPU backends #10690
-
|
Are my assumptions correct that I can now compile for any number of GPU and CPU backends and use whichever combination I want together (given the correct hardware of course)? Is it as simple as passing in an option to llama-server or llama-cli to instruct which CPU and GPU backends I want to use? What are these options? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
You can build any number of backends, although there may be some incompatibilities with backends that require using a different C/C++ compiler. In that case, it would still be possible to build these backends separately using |
Beta Was this translation helpful? Give feedback.
You can build any number of backends, although there may be some incompatibilities with backends that require using a different C/C++ compiler. In that case, it would still be possible to build these backends separately using
GGML_BACKEND_DLand then bundle them together. You can use--deviceto select the devices you want to use, and--list-devicesto view a list of available devices.