Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add integration/CMake for SYCL backend #656

Open
Jacoby1218 opened this issue Feb 2, 2024 · 5 comments
Open

Add integration/CMake for SYCL backend #656

Jacoby1218 opened this issue Feb 2, 2024 · 5 comments
Labels
enhancement New feature or request KIV for now Some issues prevent this from being merged

Comments

@Jacoby1218
Copy link

At this point, given the >4GB allocation bug was fixed, if possible, I'd like to see the SYCL backend integrated into koboldcpp. While Intel GPUs are supported by Vulkan, SYCL is much faster at prompt processing (4-7x in some cases, though Vulkan is getting faster.) Even if it's not included by default with koboldcpp (much like hipBLAS) it should be supported as a backend.

@LostRuins
Copy link
Owner

Unfortunately I have no way to test it as I have not gotten SYCL to run on my device.

@LostRuins LostRuins added enhancement New feature or request KIV for now Some issues prevent this from being merged labels Feb 8, 2024
@Jacoby1218
Copy link
Author

once ggerganov#5357 is done upstream it should at least build to test on non-Intel hardware, but understandable.

@tpimh
Copy link

tpimh commented Oct 11, 2024

I have an A770 16Gb, is there anything I can help with?

@ChristophHaag
Copy link

amd sycl support has been merged upstream btw. ggerganov#9658

Seems to only work with the sycl implementation in intel's oneapi sdk and not with any other sycl implementation but at least it's something.

@DarwinAnim8or
Copy link

I can confirm that the oneapi sycl implementation works, I've been using it myself on a Arc A380; I might see if I can adapt it to koboldcpp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request KIV for now Some issues prevent this from being merged
Projects
None yet
Development

No branches or pull requests

5 participants