Skip to content

Commit

Permalink
sycl-exp : Revert "Minor arithmetic improvement to mmvq wrapper kernel (
Browse files Browse the repository at this point in the history
ggerganov#7172)" (ggerganov#7980)

* Revert "Minor arithmetic improvement to mmvq wrapper kernel (ggerganov#7172)"

This reverts commit 8c570c9.

* Update ggml-sycl.cpp
  • Loading branch information
joeatodd authored and Alcpz committed Jun 20, 2024
1 parent 8ee4082 commit 7b168f7
Showing 1 changed file with 8 additions and 10 deletions.
18 changes: 8 additions & 10 deletions ggml-sycl.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -7984,26 +7984,24 @@ static void mul_mat_vec_q(const void * __restrict__ vx, const void * __restrict_
const int blocks_per_row = ncols / qk;
const int blocks_per_warp = vdr * WARP_SIZE / qi;

const int qi_vdr = (qi / vdr); // N_threads processing 1 qk block

// partial sum for each thread
float tmp = 0.0f;

const block_q_t * x = (const block_q_t *) vx;
const block_q8_1 * y = (const block_q8_1 *) vy;

for (int i = item_ct1.get_local_id(2) / qi_vdr; i < blocks_per_row;
for (int i = item_ct1.get_local_id(2) / (qi / vdr); i < blocks_per_row;
i += blocks_per_warp) {
const int ibx = row * blocks_per_row + i; // x block index
const int ibx = row*blocks_per_row + i; // x block index

const int iby = i * (qk / QK8_1); // y block index that aligns with ibx
const int iby = i * (qk/QK8_1); // y block index that aligns with ibx

const int iqs =
vdr *
(item_ct1.get_local_id(2) -
i * qi_vdr); // x block quant index when casting the quants to int
const int iqs =
vdr *
(item_ct1.get_local_id(2) %
(qi / vdr)); // x block quant index when casting the quants to int

tmp += vec_dot_q_sycl(&x[ibx], &y[iby], iqs);
tmp += vec_dot_q_sycl(&x[ibx], &y[iby], iqs);
}

// sum up partial sums and write back result
Expand Down

0 comments on commit 7b168f7

Please sign in to comment.