Skip to content

Dynamic Temp (updated, December 15)
Compare
Choose a tag to compare
@kalomaze kalomaze released this 15 Dec 19:47
· 88 commits to exp-dynatemp-minp-latest since this release
dec-15-updated
062f668

Another maintenance build for my custom koboldcpp build, this one adds support for Mixtral.

Noisy sampling is still included.

Override values have changed for the different Dynamic Temp options.

  • 3.9 Temp for Greedy Dynamic Temp
  • 2.2 Temp for HHI Dynamic Temp
  • 1.84 Temp for Entropy Dynamic Temp (this one remains the same as before)

I still think Entropy Dynamic Temp is the best one. 0.0 minTemp and 2.0 maxTemp seems to scale well as a default, but the maxTemp can go higher especially with a decent minP setting

EDIT: Hotfix for latest koboldcpp build that has improved prompt processing is now included. I recommend BLAS Batch size 128 for Mixtral at the moment, until further improvements are made to prompt processing.

Assets 4
Loading