Middleware logging is a technique used in software development, particularly in web and microservices applications, to log important information about incoming requests, outgoing responses, and the operations performed by the application.
-
core-go/middleware is designed to integrate with middleware logging seamlessly for existing Go libraries: Echo, Gin, or net/http (Gorilla mux, Go-chi), with any logging libraries (zap, logrus), to log request headers, request body, response status code, body content, response time, and size
-
Especially, core-go/middleware supported to encrypt sensitive data, which is useful for Financial Products (to comply with PCI-DSS standards) and Healthcare (to comply with HIPAA regulations)
-
You can refer to middleware-log-tracing at my Linked In for more details.
- When you zoom one micro service, the flow is as below, and you can see "middleware" in the full picture:
- Log Request Method and URL: Log the HTTP method (GET, POST, etc.) and the requested URL.
- Log Request Headers: Option to log request headers for debugging purposes.
- Log Request Body: Option to log the request body (with configurable size limits to avoid logging large payloads).
- Debugging: Helps in tracing and debugging issues by providing complete information about incoming requests.
- Monitoring: Provides visibility into the types of requests being received.
- Log Response Status Code: Log the HTTP status code of the response.
- Log Response Headers: Option to log response headers.
- Log Response Body: Option to log the response body (with configurable size limits to avoid logging large payloads).
- Debugging: Assists in diagnosing issues by providing complete information about the responses sent by the server.
- Auditing: Helps in auditing and reviewing server responses for compliance and monitoring purposes.
- Log Response Time: Calculate and log the time taken to process each request.
- Performance Monitoring: Helps in identifying slow requests and performance bottlenecks.
- Optimization: Provides data to optimize and improve server response times.
- Log Response Size: Log the size of the response payload in bytes.
- Bandwidth Monitoring: Helps in monitoring and managing bandwidth usage.
- Optimization: Provides insights into the response sizes to optimize payloads and improve performance.
- Middleware Function: Designed to integrate seamlessly with existing Go libraries: Echo, Gin, or net/http (Gorilla mux, Go-chi).
- Sample for Echo is at go-echo-sql-sample
- Sample for Gin is at go-gin-sql-sample
- Sample for Gorilla mux is at go-sql-sample
- Context Handling: Pass context to handle request-specific data throughout the middleware chain.
- Ease of Use: Simplifies the integration of logging into existing web applications.
- Consistency: Ensures consistent logging across different parts of the application.
- Do not depend on any logging libraries.
- Already supported to integrate with zap, logrus
- Can be integrated with any logging library.
- Mask/Encrypt sensitive data in the request and response bodies.
- Sensitive Data Identification: identify and encrypt specific fields in JSON payloads.
- Security: Protects sensitive information from being exposed in logs.
- Compliance: Helps meet security and compliance requirements by safeguarding sensitive data.
- Ease of Use: Simplifies the integration of encryption/masking into any existing applications.
- Consistency: Ensures that sensitive data is consistently encrypted or masked across all logged requests and responses
- Sample for Echo is at go-echo-sql-sample
- Sample for Gin is at go-gin-sql-sample
- Sample for Gorilla mux is at go-sql-sample
- Enable/Disable Logging: Allow users to turn on or off logging for requests, responses, headers, and bodies independently.
- Logging Levels: Support different logging levels (e.g., INFO, DEBUG, ERROR) to control the verbosity of logs.
- Flexibility: Provides users with the flexibility to configure logging based on their needs and environment.
- Efficiency: Reduces overhead by allowing selective logging, especially in production environments.
- Non-Blocking Logs: Implement asynchronous logging to ensure that logging does not block request processing.
- Log Buffering: Use buffering to improve logging performance and reduce latency.
- Performance: Improves the overall performance of the application by reducing logging overhead.
- Scalability: Allows the application to handle high-throughput logging without performance degradation.
- Benefit: Encrypting sensitive financial data, such as credit card numbers and transaction details, helps comply with PCI-DSS standards and secures financial transactions from exposure in logs.
- Benefit: Encrypting patient data such as medical records and health information in logs ensures compliance with HIPAA regulations and protects patient privacy.
- Benefit: Protecting customer information, such as addresses and payment details, enhances customer trust and protects the e-commerce platform from potential data breaches.
- Provides detailed logs that help developers debug and troubleshoot issues in the application by tracing the flow of requests and responses.
- Enables monitoring of application performance and behavior, allowing for real-time alerting on errors, slow responses, and unusual activity.
- Logs performance metrics that can be analyzed to identify bottlenecks, optimize resource usage, and improve overall application performance.
- Helps in tracking access and usage patterns, detecting security incidents, and complying with regulatory requirements by logging relevant information.
- Provides an audit trail of user actions and system operations, which is essential for security audits and forensic analysis.
Middleware logging is a critical aspect of building robust, maintainable, and secure applications, providing valuable insights and aiding in the continuous improvement of the software.
Please make sure to initialize a Go module before installing core-go/middleware:
go get -u github.com/core-go/middlewareImport:
import "github.com/core-go/middleware"
- "middleware" in the full picture of cross-cutting concerns
