chat.append_message() should allow choosing content_type

[gadenbuie]

Currently, chat.append_message() allows a message to be a dictionary with items "role" and "content" and the message's content_type is ultimately determined from the role.

I believe we should make it possible to customize the content_type directly in the message dictionary.

Here's a small reprex from shiny/api-examples/chat/app-core.py:

from shiny import App, ui

app_ui = ui.page_fillable(
    ui.panel_title("Hello Shiny Chat"),
    ui.chat_ui("chat"),
    fillable_mobile=True,
)

# Create a welcome message
welcome = ui.markdown(
    """
    Hi! This is a simple Shiny `Chat` UI. Enter a message below and I will
    simply repeat it back to you. For more examples, see this
    [folder of examples](https://github.com/posit-dev/py-shiny/tree/main/examples/chat).
    """
)


def server(input, output, session):
    chat = ui.Chat(id="chat", messages=[welcome])

    # Define a callback to run when the user submits a message
    @chat.on_user_submit
    async def _():
        # Get the user's input
        user = chat.user_input()
        await chat.append_message({
            "role": "assistant",
            "content": f"You said:\n\n{user}",
            "content_type": "html"
        })
        await chat.append_message({
            "role": "user",
            "content": f"You said:\n\n{user}",
            "content_type": "html"
        })


app = App(app_ui, server)

Put any text in the prompt in the browser and then run this in the dev console:

$$("shiny-user-message, shiny-chat-message").map(el => el.getAttribute("content_type"))
// ['html', null, 'markdown', 'markdown']

Ideally, the content type of the two added message could be customized based on the content_type of the message dictionary.

[cpsievert]

the message's content_type is ultimately determined from the role.

There's a bit more to this. With role: "user" we do currently fix it to "semi-markdown", but for role: "assistant" you can switch from markdown to html by marking the content string with ui.HTML()

[cpsievert]

Also, providing control over the user's content type from .append_message() doesn't feel quite right since user messages are automatically appended purely client-side (i.e., the only reason you should be appending user messages from the server is to restore a conversation). That said, it doesn't seem unreasonable to think we could provide some control over this in the ui.Chat() constructor

[gadenbuie]

Also, providing control over the user's content type from .append_message() doesn't feel quite right since user messages are automatically appended purely client-side

elmer appends messages server-side via shinychat which might have been copied from chatlas? idk, but it does happen. But also if you want to do something very custom where you mess with user input or assemble it from other UI elements, you'd want to have more control.