How to get response content and timings info in stream mode?

[waltersamson]

I'm using llama.cpp server with SillyTavern frontend.

In non-stream mode, a response json, formed by to_json_non_oaicompat(), is sent to the frontend, it contains detailed information including response content and timings.

However, when I enable stream mode, the response is empty. The verbose log just says:

response:
srv  remove_waiti: remove task 1231 from waiting list. current waiting = 1 (before remove)

How can I have the response json sent to the frontend in stream mode? Or at least visible in the logs?

There are sparse information on this in the server README, but those seems outdated.

Is the lack of information in stream mode a limitation, or a bug?

Thanks!

[ngxson]

The detailed response only available in the last chunk in the stream response.

You should test it with curl instead of looking at the log.