Enhance Logging Infrastructure with Structured Logging and Correlation IDs

[monilpat]

Is your feature request related to a problem? Please describe.

Debugging and observability are challenging due to the lack of structured logging and correlation IDs in the existing logging setup. This makes it difficult to trace requests and understand the system's behavior effectively.

Describe the solution you'd like

Implement a structured logging framework that:

• Uses JSON format for all logs
• Includes standard fields (timestamp, severity, correlation ID)
• Supports context injection
• Has different log levels (DEBUG, INFO, WARN, ERROR)
• Allows adding custom fields
• Provides performance logging utilities

Describe alternatives you've considered

• Using plain text logs with grep
• Manual JSON formatting
• Application Performance Monitoring (APM) tools only

Additional context

The current logging system primarily uses console logs, which lack the necessary context and structure for effective debugging and monitoring. By adopting structured logging, we can improve log readability, facilitate easier log aggregation, and enable better monitoring and observability across the system.