Skip to content

Commit

Permalink
llama: rwkv6: Avoid division by zero
Browse files Browse the repository at this point in the history
Co-authored-by: compilade <[email protected]>
  • Loading branch information
MollySophia and compilade authored Aug 31, 2024
1 parent 59dc2e7 commit 5175375
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/llama.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -15156,7 +15156,7 @@ struct llm_build_context {
)
);

if ((il + 1) % hparams.rescale_every_n_layers == 0) {
if (hparams.rescale_every_n_layers != 0 && (il + 1) % hparams.rescale_every_n_layers == 0) {
cur = ggml_scale(ctx0, cur, 0.5F);
}

Expand Down

0 comments on commit 5175375

Please sign in to comment.