Skip to content

Commit

Permalink
manually undefine i8mm support for normal gguf
Browse files Browse the repository at this point in the history
  • Loading branch information
l3utterfly committed Aug 26, 2024
1 parent 8a2f512 commit c4934bc
Showing 1 changed file with 7 additions and 0 deletions.
7 changes: 7 additions & 0 deletions ggml/src/ggml-quants.c
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,13 @@

#define UNUSED GGML_UNUSED

// we force undef the i8mm feature here
// this is because we want to compile our app WITH the i8mm feature to support faster calculations for devices that support it
// those devices will use the special Q4_048 quants, whose code is in the ggml-arrch64.c file
// this file contains code for "normal" quants, so we disable the i8mm feature explicitly
// this makes running normal quants slightly slower for devices that support i8mm, but it's a small price to pay to support a wider range of deices
#undef __ARM_FEATURE_MATMUL_INT8

// some compilers don't provide _mm256_set_m128i, e.g. gcc 7
#define MM256_SET_M128I(a, b) _mm256_insertf128_si256(_mm256_castsi128_si256(b), (a), 1)

Expand Down

0 comments on commit c4934bc

Please sign in to comment.