Add the prometheus-log-parser optional component
#473
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
component/THREDDS
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2870/Result ✅ SUCCESSBIRDHOUSE_DEPLOY_BRANCH : prometheus-log-parser DACCS_IAC_BRANCH : master DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-216.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1735/NOTEBOOK TEST RESULTS |
huard
approved these changes
Nov 6, 2024
birdhouse/optional-components/prometheus-log-parser/config/thredds/prometheus-log-exporter.py
Outdated
Show resolved
Hide resolved
birdhouse/optional-components/prometheus-log-parser/config/thredds/prometheus-log-exporter.py
Outdated
Show resolved
Hide resolved
This was referenced Nov 7, 2024
fmigneault
reviewed
Nov 8, 2024
Co-authored-by: David Huard <[email protected]>
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2876/Result ❌ FAILUREBIRDHOUSE_DEPLOY_BRANCH : prometheus-log-parser DACCS_IAC_BRANCH : master DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-216.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1738/NOTEBOOK TEST RESULTS |
fmigneault
approved these changes
Nov 8, 2024
tlvu
approved these changes
Nov 13, 2024
There was a problem hiding this comment.
Very nice implementation, separating the generic of I/O line parsing and our specific proxy logs format parsing. I love how you "steal" the logs from inside the proxy container :D
I just have a question. I see you have logs in your generic log-parser. Where are those logs going to? To stdout which will be in docker logs output? How can we increase or decrease the logging level?
|
Oh and please also add a comment to mentionning that if this proxy log format changes, have to update the corresponding regex inoptional-components/prometheus-log-parser/config/thredds/prometheus-log-exporter.py.
|
Yeah it goes to stdout by default but can be logged to a file by setting the Log level can be set with an environment variable as well. I'll add in an option so that we can set it in the default.env file if you'd like by setting
There's already a comment by the regex but I'll add one in the nginx config file as well. |
I wasn't asking for a log to file either. Logs going to |
E2E Test ResultsDACCS-iac Pipeline ResultsBuild URL : http://daccs-jenkins.crim.ca:80/job/DACCS-iac-birdhouse/2886/Result ❌ FAILUREBIRDHOUSE_DEPLOY_BRANCH : prometheus-log-parser DACCS_IAC_BRANCH : master DACCS_CONFIGS_BRANCH : master PAVICS_E2E_WORKFLOW_TESTS_BRANCH : master PAVICS_SDI_BRANCH : master DESTROY_INFRA_ON_EXIT : true PAVICS_HOST : https://host-140-216.rdext.crim.ca PAVICS-e2e-workflow-tests Pipeline ResultsTests URL : http://daccs-jenkins.crim.ca:80/job/PAVICS-e2e-workflow-tests/job/master/1744/NOTEBOOK TEST RESULTS |
Wouldn't it be logged to file automatically because of x-logging: &default-logging
driver: "json-file"
options:
max-size: "50m"
max-file: "10"Logging to file would be redundant, unless an alternate log representation is desired. |
|
@fmigneault |
|
Note that this update bumps the minor version since it creates a new component. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
This component parses log files from other components and converts their logs to prometheus metrics that are then ingested by the monitoring Prometheus instance (the one created by the
components/monitoringcomponent).For more information on how this component reads log files and converts them to prometheus components see the log-parser documentation.
To configure this component:
PROMETHEUS_LOG_PARSER_POLL_DELAYvariable to a number of seconds to set how often the log parser checks if new lines have been added to log files (default: 1)PROMETHEUS_LOG_PARSER_TAILvariable to"true"to only parse new lines in log files. If unset, this will parse all existing lines in the log file as well (default:"true")To view all metrics exported by the log parser:
https://<BIRDHOUSE_FQDN>/prometheus/graphsearch page{job="log_parser"}in the search bar and click the "Execute" buttonUpdate the prometheus version to the current latest
v2.53.3. This is required to support loading multiple prometheus scrape configuration files with thescrape_config_filesconfiguration option.Changes
Non-breaking changes
Breaking changes
Related Issue / Discussion
Additional Information
implements parser given as an example here: 💡 [Feature] Log download stats from THREDDS server #444 (comment)
this is an alternative to Add the
promtailandlokioptional components #474. See discussion in 💡 [Feature] Log download stats from THREDDS server #444 to help decide which we should pick.CI Operations
birdhouse_daccs_configs_branch: master
birdhouse_skip_ci: false