Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add alert for new agent version in prometheus collector #654

Merged
merged 24 commits into from
Oct 23, 2023
Merged
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
538ec92
Removing duplicate alerts from ci recommended alerts
Sohamdg081992 Jun 30, 2023
52afdc9
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Jul 8, 2023
667fa62
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Jul 14, 2023
5ed84bc
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Jul 18, 2023
4280706
Remove test branch
Sohamdg081992 Jul 18, 2023
5244d82
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Jul 26, 2023
c0e61b9
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Jul 26, 2023
def76a5
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Aug 2, 2023
94a94f8
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Aug 10, 2023
12ad6c4
Remove preview keyword from policy readme
Sohamdg081992 Aug 10, 2023
d882600
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Aug 11, 2023
2d85282
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Aug 16, 2023
61cd57b
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Aug 19, 2023
431e85d
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Aug 22, 2023
fad0d47
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Aug 28, 2023
aa2b1ec
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Aug 30, 2023
ad196bd
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Sep 8, 2023
2dbe39d
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Sep 28, 2023
05c5435
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Oct 3, 2023
f657b5e
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Oct 13, 2023
3aea1dd
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Oct 17, 2023
1d2c4fa
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Oct 19, 2023
5806218
Merge branch 'main' of https://github.com/Azure/prometheus-collector
Sohamdg081992 Oct 20, 2023
4cb1d4e
Add alert for new agent version in prometheus collector
Sohamdg081992 Oct 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions internal/alerts/example-alert-template.json
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,24 @@
"actionGroupId": "/subscriptions/13d371f9-5a39-46d5-8e1b-60158c49db84/resourceGroups/ContainerInsightsPrometheusCollector-Prod/providers/microsoft.insights/actiongroups/AMA-PrometheusCollectorAlertGroup"
}
]
},
{
"alert": "New agent version found for prometheus collector",
"expression": "count(count (kube_pod_container_info{image=~\"mcr.microsoft.com/azuremonitor/containerinsights/ciprod/prometheus-collector.*\"}) by (image)) > 2",
vishiy marked this conversation as resolved.
Show resolved Hide resolved
"for": "PT30M",
"annotations": {
"description": "New agent version found for prometheus collector. This alert is only used in near ring regions for prod monitoring clusters"
},
"severity": 4,
"resolveConfiguration": {
"autoResolved": true,
"timeToResolve": "PT10M"
},
"actions": [
{
"actionGroupId": "/subscriptions/13d371f9-5a39-46d5-8e1b-60158c49db84/resourceGroups/ContainerInsightsPrometheusCollector-Prod/providers/microsoft.insights/actiongroups/AMA-PrometheusCollectorAlertGroup"
}
]
}
]
}
Expand Down