Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash running on iPhone 14 Pro: unsupported op 'MUL_MAT' #1786

Open
keropudding opened this issue Jan 19, 2024 · 3 comments
Open

Crash running on iPhone 14 Pro: unsupported op 'MUL_MAT' #1786

keropudding opened this issue Jan 19, 2024 · 3 comments

Comments

@keropudding
Copy link

Hi,

I am getting this error when running whisper.swiftui.demo app on my iPhone 14 Pro:
ggml_metal_graph_compute_block_invoke: error: unsupported op 'MUL_MAT'

If I set params.use_gpu = false, the app works fine (and for this reason, the app works fine in simulator mode since use_gpu is false there)

I saw a similar issue on llama.cpp reported: ggerganov/llama.cpp#4998 <-- this mentioned possibly device missing Apple7 feature set, but I checked that iPhone 14 Pro should be Apple8 feature set.

Thanks!
@ggerganov
Copy link
Owner

Can you show the ggml_metal_init log from the console?

@keropudding
Copy link
Author

Here you go:
ggml_metal_init: allocating
ggml_metal_init: picking default device: Apple A16 GPU
ggml_metal_init: loading '/var/containers/Bundle/Application/61EFBC8F-E73B-4B8C-9A3E-3B845ECFBA47/risejournal.app/ggml_ggml.bundle/default.metallib'
ggml_metal_init: skipping kernel_soft_max (not supported)
ggml_metal_init: skipping kernel_soft_max_4 (not supported)
ggml_metal_init: skipping kernel_rms_norm (not supported)
ggml_metal_init: skipping kernel_group_norm (not supported)
ggml_metal_init: skipping kernel_mul_mv_f32_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_f16_f16 (not supported)
ggml_metal_init: skipping kernel_mul_mv_f16_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_f16_f32_1row (not supported)
ggml_metal_init: skipping kernel_mul_mv_f16_f32_l4 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q4_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q4_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q5_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q5_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q8_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q2_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q3_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q4_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q5_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q6_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_iq2_xxs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_iq2_xs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_f32_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_f16_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q4_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q4_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q5_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q5_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q8_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q2_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q3_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q4_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q5_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q6_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_iq2_xxs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_iq2_xs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_f32_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_f16_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q4_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q4_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q5_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q5_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q8_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q2_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q3_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q4_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q5_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q6_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_iq2_xxs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_iq2_xs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_f32_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_f16_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q4_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q4_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q5_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q5_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q8_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q2_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q3_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q4_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q5_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q6_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_iq2_xxs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_iq2_xs_f32 (not supported)
ggml_metal_init: allocating
ggml_metal_init: picking default device: Apple A16 GPU
ggml_metal_init: loading '/var/containers/Bundle/Application/61EFBC8F-E73B-4B8C-9A3E-3B845ECFBA47/risejournal.app/ggml_ggml.bundle/default.metallib'
ggml_metal_init: skipping kernel_soft_max (not supported)
ggml_metal_init: skipping kernel_soft_max_4 (not supported)
ggml_metal_init: skipping kernel_rms_norm (not supported)
ggml_metal_init: skipping kernel_group_norm (not supported)
ggml_metal_init: skipping kernel_mul_mv_f32_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_f16_f16 (not supported)
ggml_metal_init: skipping kernel_mul_mv_f16_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_f16_f32_1row (not supported)
ggml_metal_init: skipping kernel_mul_mv_f16_f32_l4 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q4_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q4_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q5_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q5_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q8_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q2_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q3_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q4_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q5_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_q6_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_iq2_xxs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_iq2_xs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_f32_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_f16_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q4_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q4_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q5_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q5_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q8_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q2_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q3_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q4_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q5_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_q6_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_iq2_xxs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mv_id_iq2_xs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_f32_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_f16_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q4_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q4_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q5_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q5_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q8_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q2_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q3_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q4_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q5_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_q6_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_iq2_xxs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_iq2_xs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_f32_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_f16_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q4_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q4_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q5_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q5_1_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q8_0_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q2_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q3_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q4_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q5_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_q6_K_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_iq2_xxs_f32 (not supported)
ggml_metal_init: skipping kernel_mul_mm_id_iq2_xs_f32 (not supported)

@ggerganov
Copy link
Owner

This log is missing some lines about the detected GPU.
Here is how it looks on my iPhone 13 Mini:

whisper_backend_init: using Metal backend
ggml_metal_init: allocating
ggml_metal_init: picking default device: Apple A15 GPU
ggml_metal_init: loading '/var/containers/Bundle/Application/55FED7F2-D013-4C3E-B971-942E00188AE0/whisper.swiftui.app/ggml_ggml.bundle/default.metallib'
ggml_metal_init: GPU name:   Apple A15 GPU
ggml_metal_init: GPU family: MTLGPUFamilyApple8  (1008)
ggml_metal_init: GPU family: MTLGPUFamilyCommon3 (3003)
ggml_metal_init: GPU family: MTLGPUFamilyMetal3  (5001)
ggml_metal_init: simdgroup reduction support   = true
ggml_metal_init: simdgroup matrix mul. support = true
ggml_metal_init: hasUnifiedMemory              = true
ggml_metal_init: recommendedMaxWorkingSetSize  =  2863.32 MB

Are you running latest whiper.cpp? Maybe try with a fresh clone

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ggerganov @keropudding