Code

merge upstream#34

Code

Merged

l3utterfly merged 70 commits intolayla-buildl3utterfly/llama.cpp:layla-buildfrom masterl3utterfly/llama.cpp:masterCopy head branch name to clipboardAug 28, 2024

+12,969-9,038

Commits

Commits on Aug 12, 2024

• server : handle models with missing EOS token (ggerganov#8997)

  Show description for 5ef07e2
  ggerganov

  authored
  5ef07e2

  Copy full SHA for 5ef07e2

• py : fix requirements check '==' -> '~=' (ggerganov#8982)

  Show description for d3ae0ee
  ggerganov

  authored
  d3ae0ee

  Copy full SHA for d3ae0ee

• Fix a spelling mistake (ggerganov#9001)

  Septa2112

  authored
  2589292

  Copy full SHA for 2589292

• ggml: fix div-by-zero (ggerganov#9003)

  Show description for df5478f
  DavidKorczynski

  authored
  df5478f

  Copy full SHA for df5478f

• grammar-parser : fix possible null-deref (ggerganov#9004)

  Show description for 1262e7e
  DavidKorczynski

  authored
  1262e7e

  Copy full SHA for 1262e7e

• docs: introduce gpustack and gguf-parser (ggerganov#8873)

  Show description for 84eb2f4
  thxCode

  authored
  84eb2f4

  Copy full SHA for 84eb2f4

• llama : model-based max number of graph nodes calculation (ggerganov#8970)

  Show description for 0fd93cd
  

  nicoboss

  and

  slaren

  authored
  0fd93cd

  Copy full SHA for 0fd93cd

• ci : enable RPC in all of the released builds (ggerganov#9006)

  Show description for 1f67436
  rgerganov

  authored
  1f67436

  Copy full SHA for 1f67436

• ci : fix github workflow vulnerable to script injection (ggerganov#9008)

  Show description for fc4ca27
  diogoteles08

  authored
  fc4ca27

  Copy full SHA for fc4ca27

Commits on Aug 13, 2024

• export-lora : throw error if lora is quantized (ggerganov#9002)

  ngxson

  authored
  828d6ff

  Copy full SHA for 828d6ff

• ggml : move rope type enum to ggml.h (ggerganov#8949)

  Show description for 06943a6
  

  danbev

  and

  slaren

  authored
  06943a6

  Copy full SHA for 06943a6

Commits on Aug 14, 2024

• cmake : remove unused option GGML_CURL (ggerganov#9011)

  ggerganov

  authored
  43bdd3c

  Copy full SHA for 43bdd3c

• server : fix segfault on long system prompt (ggerganov#8987)

  Show description for 98a532d
  compilade

  authored
  98a532d

  Copy full SHA for 98a532d

• Vulkan Optimizations and Fixes (ggerganov#8959)

  Show description for 5fd89a7
  0cc4m

  authored
  5fd89a7

  Copy full SHA for 5fd89a7

Commits on Aug 15, 2024

• server : init stop and error fields of the result struct (ggerganov#9026)

  Show description for 234b306
  jpodivin

  authored
  234b306

  Copy full SHA for 234b306

• ci : disable bench workflow (ggerganov#9010)

  ggerganov

  authored
  d5492f0

  Copy full SHA for d5492f0

• llama : add pre-tokenizer regexes for BLOOM and gpt3-finnish (ggerganov#8850)

  Exploder98

  authored
  6bda7ce

  Copy full SHA for 6bda7ce

• common : remove duplicate function llama_should_add_bos_token (ggerganov#8778)

  kylo5aby

  authored
  4af8420

  Copy full SHA for 4af8420

• server : fix duplicated n_predict key in the generation_settings (ggerganov#8994)

  snowyu

  authored
  37501d9

  Copy full SHA for 37501d9

• retrieval : fix memory leak in retrieval query handling (ggerganov#8955)

  Show description for 4b9afbb
  gtygo

  authored
  4b9afbb

  Copy full SHA for 4b9afbb

Commits on Aug 16, 2024

• ggml : dynamic ggml_sched_max_splits based on graph_size (ggerganov#9047)

  Show description for e3f6fd5
  nicoboss

  authored
  e3f6fd5

  Copy full SHA for e3f6fd5

• Add Nemotron/Minitron GGUF Conversion & Inference Support (ggerganov#8922)

  Show description for 2a24c8c
  

  suhara

  and

  compilade

  authored
  2a24c8c

  Copy full SHA for 2a24c8c

• common : add support for cpu_get_num_physical_cores() on Windows (ggerganov#8771)

  Show description for fb487bb
  Septa2112

  authored
  fb487bb

  Copy full SHA for fb487bb

• llama : add EXAONE model support (ggerganov#9025)

  Show description for c679e0c
  

  3 peopleauthored
  c679e0c

  Copy full SHA for c679e0c

• gguf-py : bump version from 0.9.1 to 0.10.0 (ggerganov#9051)

  compilade

  authored
  23fd453

  Copy full SHA for 23fd453

• Fix inference example lacks required parameters (ggerganov#9035)

  Show description for c8ddce8
  Aisuko

  authored
  c8ddce8

  Copy full SHA for c8ddce8

• py : fix wrong input type for raw_dtype in ggml to gguf scripts (ggerganov#8928)

  Show description for ee2984b
  

  farbodbj

  and

  farbod

  authored
  ee2984b

  Copy full SHA for ee2984b

• llava : support MiniCPM-V-2.6 (ggerganov#8967)

  Show description for d565bb2
  

  3 peopleauthored
  d565bb2

  Copy full SHA for d565bb2

• server : refactor middleware and /health endpoint (ggerganov#9056)

  Show description for 8b3befc
  

  ngxson

  and

  ggerganov

  authored
  8b3befc

  Copy full SHA for 8b3befc

Commits on Aug 17, 2024

• Fix incorrect use of ctx_split for bias tensors (ggerganov#9063)

  suhara

  authored
  2fb9267

  Copy full SHA for 2fb9267

Commits on Aug 18, 2024

• tests : add integration test for lora adapters (ggerganov#8957)

  Show description for 2339a0b
  

  ltoniazzi

  and

  ngxson

  authored
  2339a0b

  Copy full SHA for 2339a0b

• flake.lock: Update (ggerganov#9068)

  ggerganov

  authored
  554b049

  Copy full SHA for 554b049

Commits on Aug 19, 2024

• rpc : prevent crashes on invalid input (ggerganov#9040)

  Show description for 18eaf29
  rgerganov

  authored
  18eaf29

  Copy full SHA for 18eaf29

• rpc : print error message when failed to connect endpoint (ggerganov#9042)

  rgerganov

  authored
  1b6ff90

  Copy full SHA for 1b6ff90

• cann: add doc for cann backend (ggerganov#8867)

  Show description for cfac111
  

  3 peopleauthored
  cfac111

  Copy full SHA for cfac111

Commits on Aug 20, 2024

• tests : add missing comma in grammar integration tests (ggerganov#9099)

  Show description for 90db814
  

  fairydreaming

  and

  sszymczy

  authored
  90db814

  Copy full SHA for 90db814

• [SYCL] Fix SYCL im2col and convert Overflow with Large Dims (ggerganov#9052)

  Show description for 4f8d19f
  zhentaoyu

  authored
  4f8d19f

  Copy full SHA for 4f8d19f

• [SYCL] fallback mmvq (ggerganov#9088)

  Show description for 50addec
  

  airMeng

  and

  Alcpz

  authored
  50addec

  Copy full SHA for 50addec

• llava: Add ACC OP for GPU acceleration to the Vulkan backend in the LLAVA CLIP model. (ggerganov#8984)

  Show description for 2f3c146
  

  cyzero-kim

  and

  0cc4m

  authored
  2f3c146

  Copy full SHA for 2f3c146

Commits on Aug 21, 2024

• llama : std::move llm_bigram_bpe from work_queue (ggerganov#9062)

  Show description for 8455340
  danbev

  authored
  8455340

  Copy full SHA for 8455340

• llava : zero-initialize clip_ctx structure fields with aggregate initialization 908)

  Show description for f63f603
  

  fairydreaming

  and

  sszymczy

  authored
  f63f603

  Copy full SHA for f63f603

• llama : support for falcon-mamba architecture (ggerganov#9074)

  Show description for b40eb84
  

  younesbelkada

  and

  compilade

  authored
  b40eb84

  Copy full SHA for b40eb84

• server : support reading arguments from environment variables (ggerganov#9105)

  Show description for fc54ef0
  ngxson

  authored
  fc54ef0

  Copy full SHA for fc54ef0

• llama : simplify Mamba with advanced batch splits (ggerganov#8526)

  Show description for a1631e5
  

  compilade

  and

  ggerganov

  authored
  a1631e5

  Copy full SHA for a1631e5

Commits on Aug 22, 2024

• [SYCL] Add oneDNN primitive support (ggerganov#9091)

  Show description for 1731d42
  luoyu-intel

  authored
  1731d42

  Copy full SHA for 1731d42

• [SYCL] Add a space to supress a cmake warning (ggerganov#9133)

  qnixsynapse

  authored
  11b84eb

  Copy full SHA for 11b84eb

Commits on Aug 23, 2024

• llama : use F32 precision in GLM4 attention and no FA (ggerganov#9130)

  piDack

  authored
  a07c32e

  Copy full SHA for a07c32e

• lora : fix llama conversion script with ROPE_FREQS (ggerganov#9117)

  ngxson

  authored
  3ba780e

  Copy full SHA for 3ba780e

Commits on Aug 24, 2024

• quantize : fix typo in usage help of quantize.cpp (ggerganov#9145)

  joaodinissf

  authored
  8f824ff

  Copy full SHA for 8f824ff

• CPU/CUDA: Gemma 2 FlashAttention support (ggerganov#8542)

  Show description for e11bd85
  JohannesGaessler

  authored
  e11bd85

  Copy full SHA for e11bd85

Commits on Aug 25, 2024

• CUDA: fix Gemma 2 numerical issues for FA (ggerganov#9166)

  JohannesGaessler

  authored
  f91fc56

  Copy full SHA for f91fc56

• common: fixed not working find argument --n-gpu-layers-draft (ggerganov#9175)

  GermanAizek

  authored
  93bc383

  Copy full SHA for 93bc383

Commits on Aug 26, 2024

• llama : fix time complexity of string replacement (ggerganov#9163)

  Show description for 436787f
  jart

  authored
  436787f

  Copy full SHA for 436787f

• ggml-ci : try to improve build time (ggerganov#9160)

  slaren

  authored
  f12ceac

  Copy full SHA for f12ceac

• metal : gemma2 flash attention support (ggerganov#9159)

  slaren

  authored
  0c41e03

  Copy full SHA for 0c41e03

• server : update deps (ggerganov#9183)

  ggerganov

  authored
  e5edb21

  Copy full SHA for e5edb21

• ci : add VULKAN support to ggml-ci (ggerganov#9055)

  ggerganov

  authored
  7a3df79

  Copy full SHA for 7a3df79

• tests : fix compile warnings for unreachable code (ggerganov#9185)

  Show description for 879275a
  ggerganov

  authored
  879275a

  Copy full SHA for 879275a

• ggml : add SSM Metal kernels (ggerganov#8546)

  Show description for fc18425
  ggerganov

  authored
  fc18425

  Copy full SHA for fc18425

• metal : separate scale and mask from QKT in FA kernel (ggerganov#9189)

  Show description for 06658ad
  ggerganov

  authored
  06658ad

  Copy full SHA for 06658ad

• ggml : do not crash when quantizing q4_x_x with an imatrix (ggerganov#9192)

  slaren

  authored
  7d787ed

  Copy full SHA for 7d787ed

Commits on Aug 27, 2024

• common : Update stb_image.h to latest version (ggerganov#9161)

  Show description for ad76569
  arch-btw

  authored
  ad76569

  Copy full SHA for ad76569

• llama : fix llama3.1 rope_freqs not respecting custom head_dim (ggerganov#9141)

  Show description for 75e1dbb
  nyxkrage

  authored
  75e1dbb

  Copy full SHA for 75e1dbb

• llama : fix ChatGLM4 wrong shape (ggerganov#9194)

  Show description for 2e59d61
  CausalLM

  authored
  2e59d61

  Copy full SHA for 2e59d61

• server : add some missing env variables (ggerganov#9116)

  Show description for a77feb5
  ngxson

  authored
  a77feb5

  Copy full SHA for a77feb5

• llama : fix qs.n_attention_wv for DeepSeek-V2 (ggerganov#9156)

  compilade

  authored
  78eb487

  Copy full SHA for 78eb487

• Fix minicpm example directory (ggerganov#9111)

  xyb

  authored
  3246fe8

  Copy full SHA for 3246fe8

• sync : ggml

  ggerganov

  committed
  231cff5

  Copy full SHA for 231cff5

• vulkan : fix build (#0)

  Show description for 20f1789
  ggerganov

  committed
  20f1789

  Copy full SHA for 20f1789

Commits on Aug 28, 2024

• Merge branch 'ggerganov:master' into master

  l3utterfly

  authored
  23e298c

  Copy full SHA for 23e298c