Getting segfault on exit using libllama.so after metal refactor

[crasm]

I'm dynamically linking to libllama.so from a dart program. After commit b38b5e9 for #4794, I get a segfault after the normal control flow of my program exits. Before that, everything exits cleanly.

Because it happens only on exit, and only when using dart run and not dart build, I suspect it's an issue with some resources not being fully cleaned up.

vczf@brick ~/git/ensemble/llamacpp (git)-[master] % dart run bin/tokenize.dart 'decorate'
   0:     1 = <s>
   1: 10200 = ▁decor
   2:   403 = ate

===== CRASH =====
si_signo=Segmentation fault: 11(11), si_code=SEGV_ACCERR(2), si_addr=0x75a3e512be60
version=3.3.0-279.0.dev (dev) (Thu Jan 4 06:34:46 2024 -0800) on "macos_arm64"
pid=15639, thread=12291, isolate_group=(nil)(0x0), isolate=(nil)(0x0)
os=macos, arch=arm64, comp=no, sim=no
isolate_instructions=0, vm_instructions=102890f80
fp=16dd0aed0, sp=16dd0ae90, pc=184563ff0
  pc 0x0000000184563ff0 fp 0x000000016dd0aed0 objc_release+0x10
  pc 0x2912800184567adc fp 0x000000016dd0af00 objc_autoreleasePoolPop+0x2912800000000104
  pc 0xb532800184598e30 fp 0x000000016dd0af30 objc_tls_direct_base<AutoreleasePoolPage*, (tls_key)3, AutoreleasePoolPage::HotPageDealloc>::dtor_(void*)+0xb5328000000000a8
  pc 0x1b3f00018492f9fc fp 0x000000016dd0af90 _pthread_tsd_cleanup+0x1b3f00000000026c
  pc 0x1766800184932724 fp 0x000000016dd0afc0 _pthread_exit+0x1766800000000054
  pc 0x2613000184932040 fp 0x000000016dd0afe0 _pthread_start+0x2613000000000094
-- End of DumpStackTrace

===== CRASH =====
si_signo=Trace/BPT trap: 5(5), si_code=TRAP_BRKPT(1), si_addr=0x18483daf4
Aborting reentrant request for stack trace.

===== CRASH =====
si_signo=Trace/BPT trap: 5(5), si_code=?(0), si_addr=0x18483daf4
Aborting reentrant request for stack trace.

[...continues crashing...]

[ggerganov]

Hard to guess - try to make a repro with some of the llama.cpp examples so we can investigate.

It works on my end with the main example and building with cmake -DBUILD_SHARED_LIBS=ON .. on macOS

[KrishDevCrafting]

hmmmmmmmmmm

[crasm]

@ggerganov It's not triggered dynamic loading. It's from using a metal context inside a pthread.

• crasm_segfault-on-pthread
• test-pthread.cpp

(Btw is another test that would be really good to have automated with ci/run.sh! I haven't forgotten about working on the CI script.)

[ggerganov]

@crasm Could you check that the issue is fixed in #4970

[crasm]

@ggerganov fixed and working nicely!