Code

merge upstream#43

Code

Merged

l3utterfly merged 80 commits intolayla-buildl3utterfly/llama.cpp:layla-buildfrom masterl3utterfly/llama.cpp:masterCopy head branch name to clipboardOct 27, 2024

+14,982-7,933

Commits

Commits on Oct 10, 2024

• examples : do not use common library in simple example (ggerganov#9803)

  Show description for c7499c5
  slaren

  authored
  c7499c5

• musa: add docker image support (ggerganov#9685)

  Show description for cf8e0a3
  yeahdongcn

  authored
  cf8e0a3

• rpc : add backend registry / device interfaces (ggerganov#9812)

  Show description for 0e9f760
  slaren

  authored
  0e9f760

• common : use common_ prefix for common library functions (ggerganov#9805)

  Show description for 7eee341
  

  slaren

  and

  ggerganov

  authored
  7eee341

Commits on Oct 11, 2024

• ggml : move more prints to the ggml log system (ggerganov#9839)

  Show description for 9677640
  slaren

  authored
  9677640

Commits on Oct 12, 2024

• musa : update doc (ggerganov#9856)

  Show description for 943d20b
  yeahdongcn

  authored
  943d20b

• llama : improve infill support and special token detection (ggerganov#9798)

  Show description for 11ac980
  ggerganov

  authored
  11ac980

• server : remove legacy system_prompt feature (ggerganov#9857)

  Show description for 95c76e8
  ggerganov

  authored
  95c76e8

• server : remove self-extend features (ggerganov#9860)

  Show description for 1bde94d
  ggerganov

  authored
  1bde94d

• server : add option to time limit the generation phase (ggerganov#9865)

  Show description for edc2656
  ggerganov

  authored
  edc2656

Commits on Oct 13, 2024

• flake.lock: Update (ggerganov#9870)

  Show description for 92be9f1
  

  ggerganov

  and

  github-actions[bot]

  authored
  92be9f1

• server : reuse cached context chunks (ggerganov#9866)

  Show description for c7181bd
  ggerganov

  authored
  c7181bd

• server : accept extra_context for the infill endpoint (ggerganov#9874)

  Show description for d4c19c0
  ggerganov

  authored
  d4c19c0

Commits on Oct 14, 2024

• Vectorize load instructions in dmmv f16 CUDA kernel (ggerganov#9816)

  Show description for 13dca2a
  

  agray3

  and

  JohannesGaessler

  authored
  13dca2a

• server : handle "logprobs" field with false value (ggerganov#9871)

  Show description for a89f75e
  

  VoidIsVoid

  and

  Gimling

  authored
  a89f75e

Commits on Oct 15, 2024

• readme : update bindings list (ggerganov#9889)

  srgtuszy

  authored
  4c42f93

• server : update preact (ggerganov#9895)

  ggerganov

  authored
  dcdd535

• sampling : add XTC sampler (ggerganov#9742)

  Show description for fbc98b7
  MaggotHATE

  authored
  fbc98b7

• server : improve infill context reuse (ggerganov#9894)

  Show description for 223c25a
  ggerganov

  authored
  223c25a

• llama : add infill sampler (ggerganov#9896)

  Show description for 755a9b2
  ggerganov

  authored
  755a9b2

Commits on Oct 16, 2024

• [CANN] Fix cann compilation error (ggerganov#9891)

  Show description for becfd38
  leo-pony

  authored
  becfd38

• ggml-alloc : remove buffer_id from leaf_alloc (ggml/987)

  Show description for cd60b88
  

  danbev

  authored and

  ggerganov

  committed
  cd60b88

• sync : ggml

  ggerganov

  committed
  0e41b30

• server : fix the disappearance of the end of the text (ggerganov#9867)

  Show description for 1f66b69
  z80maniac

  authored
  1f66b69

• llama : add tensor name for "result_norm" (ggerganov#9907)

  Show description for 10433e8
  MollySophia

  authored
  10433e8

• grammar : fix JSON Schema for string regex with top-level alt. (ggerganov#9903)

  Show description for 66c2c93
  jemc

  authored
  66c2c93

• llava : fix typo in error message [no ci] (ggerganov#9884)

  danbev

  authored
  dbf18e4

• llama : suppress conversion from 'size_t' to 'int' (ggerganov#9046)

  Show description for 9e04102
  danbev

  authored
  9e04102

• fix: use vm_allocate to allocate CPU backend buffer on macOS (ggerganov#9875)

  Show description for 73afe68
  giladgd

  authored
  73afe68

• fix: allocating CPU buffer with size 0 (ggerganov#9917)

  giladgd

  authored
  2194200

Commits on Oct 17, 2024

• vulkan : add backend registry / device interfaces (ggerganov#9721)

  Show description for f010b77
  slaren

  authored
  f010b77

• readme : update bindings list (ggerganov#9918)

  Show description for 3752217
  

  ShenghaiWang

  and

  Tim Wang

  authored
  3752217

• llama : infill sampling handle very long tokens (ggerganov#9924)

  Show description for 99bd4ac
  ggerganov

  authored
  99bd4ac

• llama : change warning to debug log

  ggerganov

  committed
  9f45fc1

• readme : remove --memory-f32 references (ggerganov#9925)

  ggerganov

  committed
  17bb928

• llama : rename batch_all to batch (ggerganov#8881)

  Show description for 6f55bcc
  danbev

  authored
  6f55bcc

Commits on Oct 18, 2024

• server : add n_indent parameter for line indentation requirement (ggerganov#9929)

  Show description for 8901755
  ggerganov

  authored
  8901755

• add amx kernel for gemm (ggerganov#8998)

  Show description for 60ce97c
  mingfeima

  authored
  60ce97c

• [SYCL] Add SYCL Backend registry, device and Event Interfaces (ggerganov#9705)

  Show description for 87421a2
  OuadiElfarouki

  authored
  87421a2

• rpc : backend refactoring (ggerganov#9912)

  Show description for afd9909
  rgerganov

  authored
  afd9909

• llama : remove all_pos_0, all_pos_1, all_seq_id from llama_batch (ggerganov#9745)

  Show description for cda0e4b
  ngxson

  authored
  cda0e4b

Commits on Oct 20, 2024

• readme : update infra list (ggerganov#9942)

  Show description for 7cab208
  icppWorld

  authored
  7cab208

• readme : update bindings list (ggerganov#9951)

  Show description for 45f0976
  lcarrere

  authored
  45f0976

Commits on Oct 21, 2024

• fix mul_mat_vec_q and *_vec_q error (ggerganov#9939)

  Show description for 1db8c84
  

  NeoZhangJianyu

  and

  arthw

  authored
  1db8c84

• speculative : fix handling of some input params (ggerganov#9963)

  Show description for bc21975
  ggerganov

  authored
  bc21975

• llama : default sampling changes + greedy update (ggerganov#9897)

  Show description for 55e4778
  ggerganov

  authored
  55e4778

• rpc : pack only RPC structs (ggerganov#9959)

  rgerganov

  authored
  d5ebd79

• ggml : add asserts for type conversion in fattn kernels (ggerganov#9971)

  Show description for f594bc8
  ggerganov

  authored
  f594bc8

• llama.vim : plugin for Neovim (ggerganov#9787)

  ggerganov

  authored
  dbd5f2f

• arg : fix attention non-causal arg value hint (ggerganov#9985)

  Show description for 94008cc
  danbev

  authored
  94008cc

• readme : update UI list (ggerganov#9972)

  Show description for 994cfb1
  a-ghorbani

  authored
  994cfb1

• llama.vim : move info to the right of screen [no ci] (ggerganov#9787)

  Show description for e01c67a
  ggerganov

  committed
  e01c67a

• llama.vim : fix info text display [no ci] (ggerganov#9787)

  ggerganov

  committed
  e94a138

Commits on Oct 22, 2024

• arg : fix typo in embeddings argument help [no ci] (ggerganov#9994)

  Show description for 674804a
  danbev

  authored
  674804a

• [CANN] Adapt to dynamically loadable backends mechanism (ggerganov#9970)

  Show description for 6b84473
  leo-pony

  authored
  6b84473

• llama : add chat template for RWKV-World + fix EOT (ggerganov#9968)

  Show description for 4ff7fe1
  MollySophia

  authored
  4ff7fe1

• lora : warn user if new token is added in the adapter (ggerganov#9948)

  ngxson

  authored
  c421ac0

• Rwkv chat template fix (ggerganov#10001)

  Show description for 11d4705
  

  MollySophia

  and

  ngxson

  authored
  11d4705

• llama : rename batch to ubatch (ggerganov#9950)

  Show description for 19d900a
  danbev

  authored
  19d900a

• llama : fix empty batch causing llama_batch_allocr to crash (ggerganov#9966)

  Show description for c8c07d6
  

  ngxson

  and

  ggerganov

  authored
  c8c07d6

Commits on Oct 23, 2024

• flake.lock: Update

  Show description for 873279b
  

  github-actions[bot]

  authored and

  SomeoneSerge

  committed
  873279b

• metal : add POOL2D and fix IM2COL (ggerganov#9943)

  Show description for 4c9388f
  junhee-yoo

  authored
  4c9388f

• llama.vim : add classic vim support (ggerganov#9995)

  Show description for ac113a0
  

  m18coppola

  and

  Michael Coppola

  authored
  ac113a0

• ggml : remove redundant set of contexts used field (ggml/978)

  Show description for c19af0a
  

  danbev

  authored and

  ggerganov

  committed
  c19af0a

• CUDA: fix 1D im2col, add tests (ggml/993)

  

  JohannesGaessler

  authored and

  ggerganov

  committed
  80273a3

• llama.vim : bump generation time limit to 3s [no ci]

  ggerganov

  committed
  2d3aba9

• sync : ggml

  ggerganov

  committed
  190a37d

• server : samplers accept the prompt correctly (ggerganov#10019)

  wwoodsTM

  authored
  0a1c750

Commits on Oct 24, 2024

• CUDA: fix MMQ for non-contiguous src0, add tests (ggerganov#10021)

  Show description for c39665f
  JohannesGaessler

  authored
  c39665f

• CUDA: fix insufficient buffer clearing for MMQ (ggerganov#10032)

  JohannesGaessler

  authored
  167a515

• ci : fix cmake flags for SYCL

  ggerganov

  committed
  40f2555

• server : refactor slot input data, move tokenizer to HTTP thread (ggerganov#10023)

  Show description for 958367b
  ngxson

  authored
  958367b

Commits on Oct 25, 2024

• server : check that the prompt fits in the slot's context (ggerganov#10030)

  Show description for bc5ba00
  ggerganov

  authored
  bc5ba00

• llamafile : extend sgemm.cpp support for Q5_0 models (ggerganov#10010)

  Srihari-mcw

  authored
  2f8bd2b

• llama: string_split fix (ggerganov#10022)

  Show description for d80fb71
  Xarbirus

  authored
  d80fb71

• llama : add DRY sampler (ggerganov#9702)

  Show description for ff252ea
  

  3 peopleauthored
  ff252ea

• metal : support permuted matrix multiplicaions (ggerganov#10033)

  Show description for 6687503
  ggerganov

  authored
  6687503

Commits on Oct 26, 2024

• scripts : fix amx sync [no ci]

  ggerganov

  committed
  9e4a256

• increase cuda_cpy block size (ggml/996)

  Show description for 8c60a8a
  

  2 peopleauthored and

  ggerganov

  committed
  8c60a8a

• sync : ggml

  ggerganov

  committed
  cc2983d