add self logs debug severity grep filter #1437
Conversation
| @@ -89,8 +89,8 @@ func generateHealthChecksLogsComponents(ctx context.Context) []fluentbit.Compone | |||
|
|
|||
| // This method creates a file input for the `logging-module.log` file, a regex parser for the | |||
| // fluent-bit self logs and a translator of severity to the logging api format. | |||
There was a problem hiding this comment.
The part of the comment and a translator of severity to the logging api format. should be removed from here (missed to do this update in my previous PR) since this "translation" now happens in generateSelfLogsProcessingComponents() for all self logs.
| Config: map[string]string{ | ||
| "Name": "grep", | ||
| "Match": fluentBitSelfLogsTag, | ||
| "Exclude": "severity DEBUG", |
There was a problem hiding this comment.
I believe this exclusion filter won't work since the severity field at this point hasn't been transformed correctly to a logging api friendly format (this func comment is misleading too, see other comment). The translation happens in generateSelfLogsProcessingComponents() to make the translation for all self logs once.
Some options for this to work :
- Move filter to
generateSelfLogsProcessingComponents()after severity field transform. The exclude filter would look like (may fail due to backslashes if this hasn't been fixed in the current fluent-bit version in the ops agent) :
"Exclude": "logging.googleapis.com/severity DEBUG",
- Keep filter here in
generateFluentBitSelfLogsComponents()and use lowercasedebug:
"Exclude": "severity debug",
| @@ -108,6 +108,16 @@ func generateFluentBitSelfLogsComponents(ctx context.Context) []fluentbit.Compon | |||
| }, | |||
| }, | |||
| }.Components(ctx, fluentBitSelfLogsTag, "fluent-bit-self-log-regex-parsing")...) | |||
| if logLevel == "debug" { | |||
There was a problem hiding this comment.
This would disable completely all fluent-bit debug logs. Is there any situation where we would want debug logs ? From my fluent-bit self logs exploration they haven't been useful at all, but there may be some case i'm not aware off.
|
Closing in favor of #1674 |
Description
Currently, Fluent Bit self logs absolutely explode if you turn on debug logging. This PR will add a filter to drop
DEBUGFluent Bit logs when the logging level is set todebugin the Ops Agent config.Related issue
b/272779619
How has this been tested?
Tested the generated config in my own environment to ensure debug logs weren't sent to Cloud Logging.
Checklist: