Skip to content

Commit

Permalink
llama : fix not enough space in buffer with Qwen (ggerganov#5086)
Browse files Browse the repository at this point in the history
  • Loading branch information
slaren authored Jan 22, 2024
1 parent 6f9939d commit 011e8ec
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion llama.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -4440,9 +4440,9 @@ static struct ggml_tensor * llm_build_kv(

// these nodes are added to the graph together so that they are not reordered
// by doing so, the number of splits in the graph is reduced
ggml_build_forward_expand(graph, q_cur);
ggml_build_forward_expand(graph, k_cur);
ggml_build_forward_expand(graph, v_cur);
ggml_build_forward_expand(graph, q_cur);

llm_build_kv_store(ctx, hparams, kv, graph, k_cur, v_cur, n_ctx, n_tokens, kv_head, cb, il);

Expand Down

0 comments on commit 011e8ec

Please sign in to comment.