Code

merge from upstream#49

Code

Merged

l3utterfly merged 92 commits intolayla-buildl3utterfly/llama.cpp:layla-buildfrom mergel3utterfly/llama.cpp:mergeCopy head branch name to clipboardDec 28, 2024

+21,563-4,236

Commits

Commits on Dec 11, 2024

• docs: fix server documentation formatting (ggerganov#10776)

  CentricStorm

  authored
  4b4d92b

• bug-fix: snprintf prints NULL in place of the last character (ggerganov#10419)

  Show description for 484d2f3
  kallewoof

  authored
  484d2f3

• ci : pin nodejs to 22.11.0 (ggerganov#10779)

  ngxson

  authored
  92f77a6

• Update README.md (ggerganov#10772)

  qingy1337

  authored
  1a31d0d

• server : (UI) add tok/s, get rid of completion.js (ggerganov#10786)

  Show description for 235f6e1
  ngxson

  authored
  235f6e1

• gguf-py : bump version to 0.11.0

  ggerganov

  committed
  fb18934

• Merge pull request ggerganov#10788 from ggerganov/gg/gguf-py-0.11.0

  ggerganov

  authored
  973f328

• docs: update server streaming mode documentation (ggerganov#9519)

  Show description for 5555c0c
  CentricStorm

  authored
  5555c0c

Commits on Dec 12, 2024

• common : add missing env var for speculative (ggerganov#10801)

  ngxson

  authored
  9fdb124

• Vulkan: Add VK_EXT_subgroup_size_control support to ensure full subgroups for coopmats (ggerganov#10721)

  Show description for dc5301d
  0cc4m

  authored
  dc5301d

• Vulkan: Use improved q4_k and q5_k dequant code in dequant shaders (ggerganov#10798)

  0cc4m

  authored
  4064c0e

• remove CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS (ggerganov#10797)

  Show description for cb13ef8
  slaren

  authored
  cb13ef8

• CUDA: faster non-contiguous concat (ggerganov#10760)

  Show description for 8faa1d4
  

  A3shTnT

  and

  slaren

  authored
  8faa1d4

• contrib : add ngxson as codeowner (ggerganov#10804)

  ngxson

  authored
  274ec65

• common : improve -ctv -ctk CLI arguments (ggerganov#10806)

  Show description for adffa6f
  ngxson

  authored
  adffa6f

Commits on Dec 13, 2024

• ggml : Fix compilation issues on ARM platform when building without fp16 (ggerganov#10811)

  kkontny

  authored
  d583cd0

• SYCL: Reduce most of the compiler warnings (ggerganov#10748)

  Show description for 83ed24a
  

  qnixsynapse

  and

  abhilash1910

  authored
  83ed24a

• vulkan: small mul_mat_vec optimizations (ggerganov#10665)

  Show description for 64ae065
  netrunnereve

  authored
  64ae065

• Fix crash caused by ggml_backend_load_all when launching on Android Activity (ggerganov#10812)

  Show description for 9f35e44
  

  sienaiwun

  and

  slaren

  authored
  9f35e44

• gguf-py : numpy 2 newbyteorder fix (ggerganov#9772)

  jettjaniak

  authored
  4601a8b

• fix: graceful shutdown for Docker images (ggerganov#10815)

  co42

  authored
  11e07fd

• Opt class for positional argument handling (ggerganov#10508)

  Show description for c27ac67
  ericcurtin

  authored
  c27ac67

• Introducing experimental OpenCL backend with support for Qualcomm Adreno GPUs (ggerganov#10693)

  Show description for a76c56f
  

  6 peopleauthored
  a76c56f

• Removes spurious \r in output that causes logging in journalctl to treat lines as binary and therefore hidden by default (ggerganov#10771)

  Show description for 56eea07
  

  cduk

  and

  Charles Darke

  authored
  56eea07

Commits on Dec 14, 2024

• llama : add Qwen2VL support + multimodal RoPE (ggerganov#10361)

  Show description for ba1cb19
  

  HimariO

  and

  ggerganov

  authored
  ba1cb19

• nix: allow to override rocm gpu targets (ggerganov#10794)

  Show description for e52aba5
  kurnevsky

  authored
  e52aba5

• server: Fix has_next_line in JSON response (ggerganov#10818)

  Show description for 89d604f
  MichelleTanPY

  authored
  89d604f

Commits on Dec 15, 2024

• gguf-py : bump to v0.13.0

  ggerganov

  committed
  b5ae1dd

• server: (UI) add syntax highlighting and latex math rendering (ggerganov#10808)

  Show description for 5478bbc
  

  VJHack

  and

  ngxson

  authored
  5478bbc

• scripts : change build path to "build-bench" for compare-commits.sh (ggerganov#10836)

  ggerganov

  authored
  87cf323

• llama : add Deepseek MoE v1 & GigaChat models (ggerganov#10827)

  Show description for a097415
  Inf1delis

  authored
  a097415

• llava : Allow locally downloaded models for QwenVL (ggerganov#10833)

  Show description for 4ddd199
  

  bartowski1182

  and

  ngxson

  authored
  4ddd199

Commits on Dec 16, 2024

• sampling : refactor + optimize penalties sampler (ggerganov#10803)

  Show description for 644fd71
  

  ggerganov

  and

  slaren

  authored
  644fd71

• unicode : improve naming style (ggerganov#10838)

  Show description for 08ea539
  ggerganov

  authored
  08ea539

• rwkv6: add wkv6 support for Vulkan backend (ggerganov#10829)

  Show description for 160bc03
  

  uniartisan

  and

  MollySophia

  authored
  160bc03

Commits on Dec 17, 2024

• vulkan: bugfixes for small subgroup size systems + llvmpipe test (ggerganov#10809)

  Show description for 7b1ec53
  netrunnereve

  authored
  7b1ec53

• server : (UI) fix missing async generator on safari (ggerganov#10857)

  Show description for 227d7c5
  ngxson

  authored
  227d7c5

• readme : update typos (ggerganov#10863)

  ruanych

  authored
  4f51968

• llama : add Falcon3 support (ggerganov#10864)

  mokeddembillel

  authored
  382bc7f

• server : fill usage info in embeddings and rerank responses (ggerganov#10852)

  Show description for 05c3a44
  krystiancha

  authored
  05c3a44

• ggml : update ggml_backend_cpu_device_supports_op (ggerganov#10867)

  Show description for 0006f5a
  ggerganov

  authored
  0006f5a

• ggml : add check for grad_accs (ggml/1046)

  Show description for 3919da8
  

  danbev

  authored and

  ggerganov

  committed
  3919da8

• ggml : remove return from ggml_gallocr_allocate_node (ggml/1048)

  Show description for 130d0c9
  

  danbev

  authored and

  ggerganov

  committed
  130d0c9

• vulkan : fix soft_max.comp division by zero (whisper/2633)

  Show description for 8dd19a4
  

  gn64

  authored and

  ggerganov

  committed
  8dd19a4

• cmake : fix "amd64" processor string (whisper/2638)

  ggerganov

  committed
  78f7667

• sync : ggml

  ggerganov

  committed
  5437d4a

• tests: add tests for GGUF (ggerganov#10830)

  JohannesGaessler

  authored
  081b29b

• Use model->gguf_kv for loading the template instead of using the C API. (ggerganov#10868)

  Show description for d62b532
  dranger003

  authored
  d62b532

Commits on Dec 18, 2024

• Revert "llama : add Falcon3 support (ggerganov#10864)" (ggerganov#10876)

  Show description for 4da69d1
  slaren

  authored
  4da69d1

• docs: Fix HIP (née hipBLAS) in README (ggerganov#10880)

  Show description for 6b064c9
  brianredbeard

  authored
  6b064c9

• server : (embeddings) using same format for "input" and "content" (ggerganov#10872)

  Show description for 4682887
  ngxson

  authored
  4682887

• server : add "tokens" output (ggerganov#10853)

  Show description for 0e70ba6
  

  ggerganov

  and

  ngxson

  authored
  0e70ba6

• server : output embeddings for all tokens when pooling = none (ggerganov#10861)

  Show description for 152610e
  

  ggerganov

  and

  ngxson

  authored
  152610e

• server: avoid overwriting Authorization header (ggerganov#10878)

  Show description for 7bbb5ac
  

  vesath

  and

  ngxson

  authored
  7bbb5ac

• tts : add OuteTTS support (ggerganov#10784)

  Show description for 0bf2d10
  ggerganov

  authored
  0bf2d10

• ggml : fix arm build (ggerganov#10890)

  Show description for 9177484
  

  slaren

  and

  angt

  authored
  9177484

Commits on Dec 19, 2024

• llama-run : improve progress bar (ggerganov#10821)

  Show description for 7909e85
  ericcurtin

  authored
  7909e85

• tests: disable GGUF test for bad value size (ggerganov#10886)

  JohannesGaessler

  authored
  cd920d0

• convert : Add support for Microsoft Phi-4 model (ggerganov#10817)

  Show description for 7585edb
  

  fairydreaming

  and

  sszymczy

  authored
  7585edb

• llama : fix Roberta embeddings (ggerganov#10856)

  Show description for 2fffc52
  

  Ssukriti

  and

  gabe-l-hart

  authored
  2fffc52

• ggml: fix arm build with gcc (ggerganov#10895)

  Show description for a3c33b1
  angt

  authored
  a3c33b1

• server : fix logprobs, make it OAI-compatible (ggerganov#10783)

  Show description for 57bb2c4
  ngxson

  authored
  57bb2c4

• tts : small QoL for easy model fetch (ggerganov#10903)

  ggerganov

  authored
  36319de

• llama : minor grammar refactor (ggerganov#10897)

  Show description for 5cab3e4
  ggerganov

  authored
  5cab3e4

• clip : disable GPU support (ggerganov#10896)

  Show description for d408bb9
  ggerganov

  authored
  d408bb9

Commits on Dec 20, 2024

• convert : fix RWKV v6 model conversion (ggerganov#10913)

  Show description for 0a11f8b
  MollySophia

  authored
  0a11f8b

• ggml : add test for SVE and disable when it fails (ggerganov#10906)

  slaren

  authored
  21ae3b9

• server : (UI) fix copy to clipboard function (ggerganov#10916)

  ngxson

  authored
  0ca416c

• SYCL: Migrate away from deprecated ggml_tensor->backend (ggerganov#10840)

  Show description for eb5c3dc
  qnixsynapse

  authored
  eb5c3dc

• ggml-cpu: replace NEON asm with intrinsics in ggml_gemv_q4_0_4x8_q8_0() (ggerganov#10874)

  Show description for e34c5af
  angt

  authored
  e34c5af

Commits on Dec 21, 2024

• vulkan: optimize coopmat2 dequant functions (ggerganov#10855)

  Show description for a91a413
  jeffbolznv

  authored
  a91a413

• convert : add BertForMaskedLM (ggerganov#10919)

  ggerganov

  authored
  5cd85b5

Commits on Dec 22, 2024

• vulkan: build fixes for 32b (ggerganov#10927)

  Show description for ebdee94
  jeffbolznv

  authored
  ebdee94

• llama : add Falcon3 support (ggerganov#10883)

  Show description for 7ae33a6
  mokeddembillel

  authored
  7ae33a6

• devops : add docker-multi-stage builds (ggerganov#10832)

  rudiservo

  authored
  7c0e285

Commits on Dec 23, 2024

• ggml : fix run-time on FreeBSD in get_executable_path() (ggerganov#10948)

  yurivict

  authored
  7024d59

• llama-run : include temperature option (ggerganov#10899)

  Show description for dab76c9
  ericcurtin

  authored
  dab76c9

• llama : support for Llama-3_1-Nemotron-51B (ggerganov#10669)

  Show description for 6f0c9e0
  ymcki

  authored
  6f0c9e0

• llama : support InfiniAI Megrez 3b (ggerganov#10893)

  Show description for b92a14a
  dixyes

  authored
  b92a14a

• rpc-server : add support for the SYCL backend (ggerganov#10934)

  rgerganov

  authored
  86bf31c

• server : add system_fingerprint to chat/completion (ggerganov#10917)

  Show description for 485dc01
  ngxson

  authored
  485dc01

• server : fix missing model id in /model endpoint (ggerganov#10957)

  Show description for 14b699e
  ngxson

  authored
  14b699e

• ggml : fix const usage in SSE path (ggerganov#10962)

  slaren

  authored
  32d6ee6

Commits on Dec 24, 2024

• ggml : fix arm enabled features check (ggerganov#10961)

  slaren

  authored
  3327bb0

• ggml : use wstring for backend search paths (ggerganov#10960)

  Show description for 60cfa72
  slaren

  authored
  60cfa72

• llama : the WPM vocabs use the CLS token as BOS (ggerganov#10930)

  Show description for 30caac3
  ggerganov

  authored
  30caac3

• server: allow filtering llama server response fields (ggerganov#10940)

  Show description for 09fe2e7
  

  nvrxq

  and

  ngxson

  authored
  09fe2e7

• ggml : more perfo with llamafile tinyblas on x86_64 (ggerganov#10714)

  Show description for 2cd43f4
  Djip007

  authored
  2cd43f4

• server : add support for "encoding_format": "base64" to the */embeddings endpoints (ggerganov#10967)

  Show description for 9ba399d
  

  elk-cloner

  and

  ngxson

  authored
  9ba399d

Commits on Dec 26, 2024

• examples, ggml : fix GCC compiler warnings (ggerganov#10983)

  Show description for d283d02
  peter277

  authored
  d283d02

• vulkan: multi-row k quants (ggerganov#10846)

  Show description for d79d8f3
  netrunnereve

  authored
  d79d8f3

Commits on Dec 28, 2024

• Merge branch 'layla-build' into merge

  l3utterfly

  authored
  5c8aa73