How do I lengthen the responses the AI gives ?

[Katajan]

I am still very new to this, and I am playing around with different models and etc, but I have noticed that my responses are always extremely short (not longer than 200 words).

Does the length depend on the model I use, and if yes what are models you could recommend that write long responses.

[TechScorpion]

In the help for parameters it says this:

min_length: Minimum generation length in tokens. This is a built-in parameter in the transformers library that has never been very useful. Typically you want to check "Ban the eos_token" instead.
...
Ban the eos_token: One of the possible tokens that a model can generate is the EOS (End of Sequence) token. When it is generated, the generation stops prematurely. When this parameter is checked, that token is banned from being generated, and the generation will always generate "max_new_tokens" tokens.

I don't see the "min length" option currently in the parameters even with "all model loaders" option set. Maybe it doesn't get shown now or it depends on the AI model, but it recommends to use the "ban the eos_token" option.

You could also try putting in the prompt to ask it to generate a long response, eg. "write a long reply to this message responding to each point...".

Also make sure your "max new tokens" in the parameters tab is high enough.

[iboyles]

Yeah I noticed this as well recently after an update is there anyway we can set min_token_length like in huggingface transformers?, why did they take min token generation away from params. I swear min token used to be an option.

[guispfilho]

Also curious