Skip to content

Commit

Permalink
common : ensure llama_batch size does not exceed max size (ggerganov#…
Browse files Browse the repository at this point in the history
…9668)

A crash was observed when the number of tokens added to a batch exceeds
llama_batch size. An assertion in llama_batch_add was added to protect
against llama_batch size overflow.
  • Loading branch information
matiaslin authored and arthw committed Nov 18, 2024
1 parent db440f2 commit 1720949
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions common/common.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1437,6 +1437,8 @@ void llama_batch_add(
llama_pos pos,
const std::vector<llama_seq_id> & seq_ids,
bool logits) {
GGML_ASSERT(batch.seq_id[batch.n_tokens] && "llama_batch size exceeded");

batch.token [batch.n_tokens] = id;
batch.pos [batch.n_tokens] = pos;
batch.n_seq_id[batch.n_tokens] = seq_ids.size();
Expand Down

0 comments on commit 1720949

Please sign in to comment.