Code

merge upstream#39

Code

Merged

l3utterfly merged 22 commits intolayla-buildl3utterfly/llama.cpp:layla-buildfrom masterl3utterfly/llama.cpp:masterCopy head branch name to clipboardSep 26, 2024

+2,508-1,612

Commits

Commits on Sep 23, 2024

• metal : use F32 prec for K*Q in vec FA (ggerganov#9595)

  Show description for bf9c101
  ggerganov

  authored
  bf9c101

• perplexity : remove extra new lines after chunks (ggerganov#9596)

  ggerganov

  authored
  37f8c7b

• ggml : AVX512 gemm for Q4_0_8_8 (ggerganov#9532)

  Show description for 1e7b929
  

  Srihari-mcw

  and

  ggerganov

  authored
  1e7b929

• flake.lock: Update (ggerganov#9586)

  Show description for f3979df
  

  ggerganov

  and

  github-actions[bot]

  authored
  f3979df

• readme : add programmable prompt engine language CLI (ggerganov#9599)

  snowyu

  authored
  1d48e98

• threads: improve ggml_barrier scaling with large number of threads (ggerganov#9598)

  Show description for f0c7b5e
  

  max-krasnyansky

  and

  wtarreau

  authored
  f0c7b5e

• server : add --no-context-shift option (ggerganov#9607)

  Show description for 0b3bf96
  

  ngxson

  and

  ggerganov

  authored
  0b3bf96

Commits on Sep 24, 2024

• cuda: add q8_0->f32 cpy operation (ggerganov#9571)

  Show description for 116efee
  Nekotekina

  authored
  116efee

• threads: fix msvc build without openmp (ggerganov#9615)

  Show description for c087b6f
  max-krasnyansky

  authored
  c087b6f

• sampling : avoid expensive softmax during greedy sampling (ggerganov#9605)

  Show description for b0f2736
  

  ggerganov

  and

  slaren

  authored
  b0f2736

• server : add newline after chat example (ggerganov#9616)

  StrangeBytesDev

  authored
  0aa1501

• log : add CONT level for continuing previous log entry (ggerganov#9610)

  ggerganov

  authored
  cea1486

• llama : keep track of all EOG tokens in the vocab (ggerganov#9609)

  Show description for 31ac583
  ggerganov

  authored
  31ac583

• examples : adapt to ggml.h changes (ggml/0)

  Show description for c038931
  ggerganov

  committed
  c038931

• sync : ggml

  ggerganov

  committed
  bb5f819

• ggml : add AVX512DQ requirement for AVX512 builds (ggerganov#9622)

  EZForever

  authored
  70392f1

Commits on Sep 25, 2024

• cann: fix crash when llama-bench is running on multiple cann devices (ggerganov#9627)

  bachelor-dou

  authored
  904837e

• llama : add IBM Granite MoE architecture (ggerganov#9438)

  Show description for 3d6bf69
  

  gabe-l-hart

  and

  ggerganov

  authored
  3d6bf69

• server : add more env vars, improve gen-docs (ggerganov#9635)

  Show description for afbbfaa
  ngxson

  authored
  afbbfaa

• ggml : remove assert for AArch64 GEMV and GEMM Q4 kernels (ggerganov#9217)

  Show description for 1e43630
  chaxu01

  authored
  1e43630

• ci : fix docker build number and tag name (ggerganov#9638)

  Show description for ea9c32b
  ngxson

  authored
  ea9c32b

Commits on Sep 26, 2024

• mtgpu: enable VMM (ggerganov#9597)

  Show description for 7691654
  yeahdongcn

  authored
  7691654