Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add Datadog search script #864

Merged
merged 2 commits into from
Dec 3, 2024
Merged

feat: add Datadog search script #864

merged 2 commits into from
Dec 3, 2024

Conversation

robrap
Copy link
Contributor

@robrap robrap commented Dec 2, 2024

A search script for searching Datadog
monitors and dashboards.

Implements:

Sample script output:

Note: This sample output is slightly out of date.

  • It is missing the final output of match counts.
  • There may be other minor differences.
% python datadog_search.py --regex "code_owner[^_]"
Searching for regex code_owner([^_]|\b) in all monitors:
.........................................................

...

- 99999999 "LMS missing code owner transactions" ['team:arch-bom', 'service:edx-edxapp-lms', 'operation_name:django.request']
  - match_path: monitor.query
    - match: trace-analytics("@_top_level:1 env:(prod OR edge) service:edx-edxapp-lms -@code_owner:* [email protected]_code:(30* OR 40*) @http.status_code:* operation_name:django.request -resource_name:*404*").rollup("count").by("env,status,service").last("1h") > 10


- 99999999 "LMS unrecognized code_owner_squad" ['team:arch-bom', 'env:prod', 'service:edx-edxapp-lms', 'env:(prod', '-@code_owner:*', '[email protected]_code:(30*', 'operation_name:django.request', '_top_level:1', '_trace_root:1', '[email protected]_code:*', 'http.status_code:*', 'code_owner:*', "code_owner_squad:''", "code_owner_squad:('always-available'", "-@code_owner_squad:('always-available'", 'status:error', '-@code_owner_squad:(always-available', 'env:edge']
  - match_path: monitor.tags[4]
    - match: -@code_owner:*
  - match_path: monitor.query
    - match: trace-analytics("env:(prod OR edge) service:edx-edxapp-lms [email protected]_code:(30* OR 40*) @http.status_code:* operation_name:django.request @code_owner:* -@code_owner_squad:(...

...

Searching for regex code_owner([^_]|\b) in all dashboards:

...

- xxx-xxx-xxx "LMS - missing code_owner" ['team:arch-bom']
  - match_path: dashboard_details.title
    - match: LMS - missing code_owner
  - match_path: dashboard_details.widgets[0]['definition']['content']
    - match: # README
This dashboard displays LMS Transactions and Errors that do not yet have a code_owner.
  - match_path: dashboard_details.widgets[1]['definition']['title']
    - match: LMS Successful Transactions (missing code_owner)
  - match_path: dashboard_details.widgets[2]['definition']['content']
    - match: # LMS Successful Transactions
...

- xxx-xxx-xxx "Edx Studio (CMS) dashboard" ['team:tnl']
  - match_path: dashboard_details.widgets[5]['definition']['requests'][0]['queries'][0]['search']['query']
    - match: status:error @code_owner:*tnl* service:(edx-edxapp-cms OR edx-edxapp-cms-workers)

...

Skipping xxx-xxx-xxx due to error: Invalid value for `x`, must be a value greater than or equal to `0`
...

Author concern:

The original script had not unit tests. I'm ok with that, but are you?

Merge checklist:
Check off if complete or not applicable:

  • Version bumped
  • Changelog record added
  • Documentation updated (not only docstrings)
  • Fixup commits are squashed away
  • Unit tests added/updated
  • Manual testing instructions provided
  • Noted any: Concerns, dependencies, migration issues, deadlines, tickets

A search script for searching Datadog
monitors and dashboards.

Implements:
- #786
- adds match counts to final output
- fixes review feedback and help output formatting
@robrap robrap merged commit a95fe3d into main Dec 3, 2024
6 checks passed
@robrap robrap deleted the robrap/add-dd-search-script branch December 3, 2024 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants