Completely lost, would like directions

[gruevy]

I've never done any AI chat before other than a little chatGPT and I'm not a coder. I found this by reading about LLaMA and doing some googling, and I was able to use some directions on Reddit to get 13b running just fine on my Windows pc with a 4090.

I don't know what any of the sliders do. I'm not sure what the presets are, exactly, or why I'd pick one over another. It doesn't seem to respond to my chats like chatgpt does, and I'm not sure why or how to get it to do different things. I can change the 'context' field to get it to speak like different characters, but I'm not sure if it's affecting personality or what. The AI seems to forget things it said only one or two messages back, and I don't know if that's normal.

I also see that this will run a variety of different chat models, but I don't know what any of them are or why I'd pick one over another.

In short, I'm almost totally lost. Any chance we can get some sort of guides or tutorials added to the wiki to explain the features and how to use it properly? I'll keep poking around, and I appreciate the software in the first place.

[oobabooga]

You can find the definitions for the parameters here: https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig

In practical terms, the only thing that you have to understand about these models is that they predict 1 token at a time. Given your input prompt, the model will output a list of probabilities for the next token. These parameters that you see in the UI (temperature, top_k, top_k, etc) control how the next token will be sampled from this list of probabilities.

For instance, top_k restricts the sampling to the top k most likely tokens (usually k=50). Temperature flattens out the distribution and makes low probability tokens more likely to be sampled. Etc.

[gruevy]

Thanks, I'll read that. Looks helpful.

Do you have any quick tips for finding the 'sweet spot' with settings? Answers that aren't all short, or too long and rambling? More coherent responses, stuff like that? Where can I read how to create or save a character, and what I'd need to include? Sorry, feels like these are all really basic questions.

[pbosh]

These settings work well for my purposes.

[gruevy]

Thanks, I'll give it a try!

[jfranecki]

How do you get 'Pygmalion' as a Generation parameters preset?

[ghost]

It's likely a custom preset placed in the "presets" folder.