Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specific suggested wiki "Feature Matrix" updates pertaining to SYCL. #7415

Open
ghchris2021 opened this issue May 20, 2024 · 2 comments
Open
Labels
enhancement New feature or request

Comments

@ghchris2021
Copy link

Specific suggested wiki "Feature Matrix" updates pertaining to SYCL (q.v. attached draft).

Modified / added footnotes 1,7, tried to make consistent with README-sycl.md and various
pertinent SYCL pull requests / issues active in recent weeks.
Some additional clarifications of which quantization / data types besides the IQ ones could
be updated by someone who has clarity about the status quo (I do not) wrt.
F16/F32/Q_n_K_x/I8/BF16/....etc.

IDK how using the GH web interface to submit a PR for the Feature-matrix.md file in the wiki section git so I thought I'd
try this since it's just a few simple changes to the one markdown file as a basis.

https://github.com/ghchris2021/tmpwiki0520/blob/a58564aa18210814fdc07dadb54018f7f612b099/Feature-matrix.md

@ghchris2021 ghchris2021 added the enhancement New feature or request label May 20, 2024
@NeoZhangJianyu
Copy link
Collaborator

NeoZhangJianyu commented May 21, 2024

@ghchris2021
Here is the update as my understanding:

I-quants: SYCL- Yes.

  • iq1_s is fault (big error) in one case. That won't impact to support the feature.

Multi-GPU: SYCL -Yes.

  • not support [row]: in fact, it's supported as function, but with lower performance.

K cache quants: SYCL -Yes

MoE architecture: SYCL - Yes.

  • There is known bug, I will fix it recently.

ghchris2021 added a commit to ghchris2021/tmpwiki0520 that referenced this issue May 21, 2024
Updating / editing per. information shared in

ggerganov/llama.cpp#7415 (comment)

And README-sycl.md and other issues.
@ghchris2021
Copy link
Author

@NeoZhangJianyu
Thank you very much for the information you have shared about the update, also for your
interest and work on the SYCL bug you mentioned!

I am very glad to see that there have been made the capabilities / improvements you mentioned about
SYCL support relative to what the older "Feature Matrix" listed, it seems like it is pretty well supporting
many things now.

I edited my copy of the "Feature Matrix" markdown document according to
what you have shared with me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants