Integrate Observability

[bonk1t]

Updated PR Description

Documentation Reference

Key Implementation Details

1. Abstract Tracking Interface (AbstractTracker)
   Introduces a provider-agnostic interface for tracking usage (tokens), tool calls, and errors. This abstraction allows seamless swapping between different trackers—such as SQLiteTracker for local/streaming tests and LangfuseTracker for cloud-based observability—without code changes in the main logic.
2. Multiple Integration Approaches

• Decorator-Based (e.g. Langfuse): Uses @observe and langfuse_context as per Langfuse’s official recommendations. This approach requires minimal patching, leverages the Assistants API, and cleanly integrates into the code.
• Client Patching (e.g. Helicone): Accommodates platforms requiring custom headers or direct client instrumentation.
• Callbacks (Future Enhancements): Allows compatibility with callback-based observability frameworks for broader ecosystem support.

3. Streaming & Local Development Support
   A simple SQLiteTracker provides a fast, local setup for development and interactive demos (e.g., Gradio). While some platforms limit streaming support, the abstract design allows incremental improvements as APIs evolve.
4. Captured Usage Data
   Tracks prompt/completion tokens, messages, and session metadata, supporting both non-streaming and streaming scenarios. This lays the groundwork for additional event types (tool calls, errors) and more sophisticated observability features.

Default Configuration

• Local & Streaming Demos: Use SQLiteTracker by default for immediate feedback.
• Easy Switch: set_tracker("langfuse") or other providers as they are integrated.