Chat mode updates

[oobabooga]

• Instruct/chat mode separation: when the UI automatically selects "Instruct" mode after loading an instruct model, your character data is no longer lost. Just click on "Chat" in the menu below and the character data will reappear unchanged in the "Character" tab.

• Custom chat styles are now supported, and a new style by Reddit user TheEncrypted777 has been added: https://www.reddit.com/r/Oobabooga/comments/12xe6vq/updated_css_styling_with_color_customization_for/:

The styles can be selected in the following new menu:

Here is the documentation on how to create a new style (pull requests are welcome): https://github.com/oobabooga/text-generation-webui/blob/main/docs/Chat-mode.md#chat-styles

• superbooga (SuperBIG) support in chat mode:

This new extension sorts the chat history by similarity rather than by chronological order. It does that using ChromaDB to query relevant message/reply pairs in the history relative to the current user input. The idea is to have a long term memory where old exchanges that are relevant are brought back into view. I haven't tested this much, and feedback on its usefulness would be appreciated.

The methodology was entirely developed by @kaiokendev, who is working on improvements to this method. I merely adapted it in simplified form. See the PR below and his repository for more details:

#1548
https://github.com/kaiokendev/superbig

To use the extension:

pip install -r extensions/superbooga/requirements.txt
python server.py --listen --chat --extensions superbooga

[valden80]

IMHO, it will be very cool, if this extension will allow to load context from files/urls/text and use tag <|injection-point|> in the chat mode too. Now, there is enough models who can get clear benefit from describing characters details and chat background in "context" or "greeting" fields. (instruct-chat mode).
I used this approach many times (add specific details to setup through added instruction text in "context"), and it's give decent result. Typically I just add something like in "context":

This is chat between player and Anna. You response as Anna. Always stay in character and carefully use background information from the provided text block:
... some data about character and place, can be pretty long and detailed ...
Anna:

And it's work's pretty good for most vicuna based models. If only there can be possibility to add <|injection-point|> to this block...

Also, a button to reset/clear database from already loaded text with one click, will be very useful too...

[eucdee]

funny, i asked chatgpt to modify the colors of his most recent html_cai_style.css to something futuristic and it came up with its own grey colors xD

i just have a problem with codeblocks now, they come out miniaturized.
maybe a good time to mention codeblocks need an update, copy button, language interpretation, color coded and all those little helpers, who is able to do it :)

i know models use the right synthas like ```pyhon but the chat ui cant handle it...

[cornpo]

The UI for file upload only appears below the directions in default mode? I'm not seeing it in chat mode.

[LitenBuzzTh]

• Instruct/chat mode separation: when the UI automatically selects "Instruct" mode after loading an instruct model, your character data is no longer lost. Just click on "Chat" in the menu below and the character data will reappear unchanged in the "Character" tab.

* Custom chat styles are now supported, and a new style by Reddit user TheEncrypted777 has been added: https://www.reddit.com/r/Oobabooga/comments/12xe6vq/updated_css_styling_with_color_customization_for/:

The styles can be selected in the following new menu:

Here is the documentation on how to create a new style (pull requests are welcome): https://github.com/oobabooga/text-generation-webui/blob/main/docs/Chat-mode.md#chat-styles

* superbooga (SuperBIG) support in chat mode:

This new extension sorts the chat history by similarity rather than by chronological order. It does that using ChromaDB to query relevant message/reply pairs in the history relative to the current user input. The idea is to have a long term memory where old exchanges that are relevant are brought back into view. I haven't tested this much, and feedback on its usefulness would be appreciated.

The methodology was entirely developed by @kaiokendev, who is working on improvements to this method. I merely adapted it in simplified form. See the PR below and his repository for more details:

#1548 https://github.com/kaiokendev/superbig

To use the extension:

pip install -r extensions/superbooga/requirements.txt
python server.py --listen --chat --extensions superbooga

what should I put in settings.yaml to start the webui in chat-instruct mode?

[oobabooga]

Make this change

 truncation_length: 2048
 truncation_length_min: 0
 truncation_length_max: 16384
-mode: chat
+mode: chat-instruct
 start_with: ''
 chat_style: cai-chat
 instruction_template: None

[LitenBuzzTh]

That is exactly what I already did, but it always starts in instruct mode