Code

merge upstream#36

Code

Merged

l3utterfly merged 49 commits intolayla-buildl3utterfly/llama.cpp:layla-buildfrom masterl3utterfly/llama.cpp:masterCopy head branch name to clipboardSep 8, 2024

+9,662-6,008

Commits

Commits on Sep 2, 2024

• llama : minor style

  ggerganov

  committed
  c6d4cb4

  Copy full SHA for c6d4cb4

• build(nix): Package gguf-py (ggerganov#5664)

  Show description for 9c1ba55
  ditsuke

  authored
  9c1ba55

  Copy full SHA for 9c1ba55

• llama-cli : remove duplicated log message (ggerganov#9275)

  nbcsm

  authored
  b60074f

  Copy full SHA for b60074f

• server : refactor multitask handling (ggerganov#9274)

  Show description for 6e7d133
  

  ngxson

  and

  ggerganov

  authored
  6e7d133

  Copy full SHA for 6e7d133

• ggml : add pthread includes on FreeBSD (ggerganov#9258)

  yurivict

  authored
  f771d06

  Copy full SHA for f771d06

• docker : fix missing binaries in full-cuda image (ggerganov#9278)

  slaren

  authored
  048de84

  Copy full SHA for 048de84

• src: make tail invalid when kv cell is intersection for mamba (ggerganov#9249)

  kylo5aby

  authored
  f148516

  Copy full SHA for f148516

• server : test script : add timeout for all requests (ggerganov#9282)

  ngxson

  authored
  48baa61

  Copy full SHA for 48baa61

Commits on Sep 3, 2024

• readme : refactor API section + remove old hot topics

  ggerganov

  authored
  b69a480

  Copy full SHA for b69a480

• llama-bench : add JSONL (NDJSON) output mode (ggerganov#9288)

  Show description for 8962422
  akx

  authored
  8962422

  Copy full SHA for 8962422

• flake.lock: Update (ggerganov#9261)

  Show description for 7605ae7
  

  ggerganov

  and

  github-actions[bot]

  authored
  7605ae7

  Copy full SHA for 7605ae7

Commits on Sep 4, 2024

• readme : rename result_format to response_format (ggerganov#9300)

  iscy

  authored
  9379d3c

  Copy full SHA for 9379d3c

• rpc : make RPC servers come first in the device list (ggerganov#9296)

  Show description for 82e3b03
  rgerganov

  authored
  82e3b03

  Copy full SHA for 82e3b03

• Fix broken links in docker.md (ggerganov#9306)

  carlory

  authored
  c8671ae

  Copy full SHA for c8671ae

• [SYCL] Fix DMMV dequantization (ggerganov#9279)

  Show description for 5910ea9
  OuadiElfarouki

  authored
  5910ea9

  Copy full SHA for 5910ea9

• ggml : AVX2 support for Q4_0_8_8 (ggerganov#8713)

  Show description for 581c305
  Srihari-mcw

  authored
  581c305

  Copy full SHA for 581c305

Commits on Sep 5, 2024

• llama-bench : fix NUL terminators in CPU name (ggerganov#9313)

  slaren

  authored
  bdf314f

  Copy full SHA for bdf314f

• cuda : fix defrag with quantized KV (ggerganov#9319)

  slaren

  authored
  4db0478

  Copy full SHA for 4db0478

• CMake fix: host for msvc compiler can only be x86 or x64 (ggerganov#8624)

  Xarbirus

  authored
  1031771

  Copy full SHA for 1031771

• Update build.yml (ggerganov#9184)

  Show description for 32b2ec8
  awatuna

  authored
  32b2ec8

  Copy full SHA for 32b2ec8

Commits on Sep 6, 2024

• ggml-quants : ternary packing for TriLMs and BitNet b1.58 (ggerganov#8151)

  Show description for 9bc6db2
  compilade

  authored
  9bc6db2

  Copy full SHA for 9bc6db2

• Improve Vulkan shader build system (ggerganov#9239)

  Show description for 8ebe8dd
  mtavenrath

  authored
  8ebe8dd

  Copy full SHA for 8ebe8dd

• server : fix missing lock (ggerganov#9334)

  ngxson

  authored
  4a1411b

  Copy full SHA for 4a1411b

• ggml : fix build break for the vulkan-debug (ggerganov#9265)

  Show description for 409dc4f
  cyzero-kim

  authored
  409dc4f

  Copy full SHA for 409dc4f

• batched-bench : add --output-format jsonl option (ggerganov#9293)

  Show description for 815b1fb
  akx

  authored
  815b1fb

  Copy full SHA for 815b1fb

• llama-bench : log benchmark progress (ggerganov#9287)

  Show description for 134bc38
  akx

  authored
  134bc38

  Copy full SHA for 134bc38

• server : simplify state machine for slot (ggerganov#9283)

  Show description for 9b2c24c
  

  ngxson

  and

  ggerganov

  authored
  9b2c24c

  Copy full SHA for 9b2c24c

Commits on Sep 7, 2024

• ci : disable rocm image creation (ggerganov#9340)

  slaren

  authored
  6c89eb0

  Copy full SHA for 6c89eb0

• ggml : fix missing cpu_set_t on emscripten (ggerganov#9336)

  Show description for 947538a
  ngxson

  authored
  947538a

  Copy full SHA for 947538a

• llama : refactor sampling v2 (ggerganov#9294)

  Show description for df270ef
  ggerganov

  authored
  df270ef

  Copy full SHA for df270ef

• ggml : always check bounds on get_rows operations (ggerganov#9354)

  slaren

  authored
  e32d081

  Copy full SHA for e32d081

• common : refactor arg parser (ggerganov#9308)

  Show description for 1b9ae51
  ngxson

  authored
  1b9ae51

  Copy full SHA for 1b9ae51

• llamafile : disable sgemm for batch-size 1 (ggerganov#9330)

  netrunnereve

  authored
  e536426

  Copy full SHA for e536426

• llama : sanitize invalid tokens (ggerganov#9357)

  Show description for faf69d4
  ggerganov

  authored
  faf69d4

  Copy full SHA for faf69d4

• llama : fix empty ring buffer push (ggerganov#9358)

  ggerganov

  authored
  f12295b

  Copy full SHA for f12295b

• llama.android : fix build (ggerganov#9350)

  ggerganov

  authored
  a5b5d9a

  Copy full SHA for a5b5d9a

Commits on Sep 8, 2024

• llama : set attrs of mislabelled EOT/EOM tokens (ggerganov#9348)

  bakkot

  authored
  fbb7fcf

  Copy full SHA for fbb7fcf

• ggml : fix cont with transposed tensors when one dimension is 1 (ggml/934)

  Show description for efe6a83
  

  smeso

  and

  ggerganov

  committed
  efe6a83

  Copy full SHA for efe6a83

• cuda : mark BF16 CONT as unsupported

  ggerganov

  committed
  51d964a

  Copy full SHA for 51d964a

• cann : add Ascend NPU support (whisper/2336)

  Show description for d2d3200
  

  MengqingCao

  authored and

  ggerganov

  committed
  d2d3200

  Copy full SHA for d2d3200

• cann : fix doxy (ggml/0)

  ggerganov

  committed
  ba1cf84

  Copy full SHA for ba1cf84

• ggml: fix ggml_graph_cpy undefined behavior (ggml/943)

  

  JohannesGaessler

  authored and

  ggerganov

  committed
  dbbebca

  Copy full SHA for dbbebca

• tests: add gradient tests for all backends (ggml/932)

  Show description for 202084d
  

  JohannesGaessler

  authored and

  ggerganov

  committed
  202084d

  Copy full SHA for 202084d

• vulkan: correctly report support for OP_CONT (ggml/946)

  Show description for 9cb9260
  

  smeso

  authored and

  ggerganov

  committed
  9cb9260

  Copy full SHA for 9cb9260

• vulkan: add dryrun support to sin and cos ops (ggml/947)

  Show description for 406c1a3
  

  smeso

  authored and

  ggerganov

  committed
  406c1a3

  Copy full SHA for 406c1a3

• scripts : option to increase git patch context

  ggerganov

  committed
  60a3107

  Copy full SHA for 60a3107

• sync : ggml

  ggerganov

  committed
  385decb

  Copy full SHA for 385decb

• metal : update support condition for im2col + fix warning (#0)

  ggerganov

  committed
  a876861

  Copy full SHA for a876861

• Merge branch 'master' of https://github.com/l3utterfly/llama.cpp

  l3utterfly

  committed
  d11bd3b

  Copy full SHA for d11bd3b