API call auditing

[dmwyatt]

I'm messing around with using simpleaichat rather than directly using the openai library and there's one blocker we're encountering.

Because of requirements From Above™️ we need to log all the calls to/from openai with as much detail as possible. When using the openai client we log the args/kwargs passed into the client and a json dump of the whole response object.

simpleaichat doesn't expose the request or the response object. It's kinda ok logging a ai.model_dump(), but the data in that becomes pretty sparse when using output_schema since it doesn't keep track of the messages with self.add_messages.

I'm not sure the best solution. I think I'd like to have access to the raw request/response, but failing that at least keep the messages even when using the output schema.

[minimaxir]

A good solution might be to have some sort of verbose toggle that stores the full JSON calls and responses for each ChatSession.