Skip to content

Commit

Permalink
Point stream monitoring to microsite
Browse files Browse the repository at this point in the history
  • Loading branch information
chrisjs committed May 17, 2019
1 parent 58c213c commit b95b50d
Show file tree
Hide file tree
Showing 4 changed files with 1 addition and 353 deletions.

This file was deleted.

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,62 +1,5 @@
[[streams-monitoring]]
= Stream Monitoring

This section describes how to monitor the applications that were deployed as part of a Stream.
The setup for each platform is different but the general architecture is the same across the platforms.

The Data Flow 2.x metrics architecture is designed around the https://micrometer.io/[Micrometer library] which is a Vendor-neutral application metrics facade.
It provides a simple facade over the instrumentation clients for the most popular monitoring systems.
See the https://micrometer.io/docs[Micrometer documentation] for the list of supported monitoring systems.
Starting with Spring Boot 2.0, Micrometer is the instrumentation library https://docs.spring.io/spring-integration/docs/current/reference/html/system-management-chapter.html#micrometer-integration[powering the delivery of application metrics from Spring Boot].
Spring Integration provides https://docs.spring.io/spring-integration/docs/current/reference/html/system-management-chapter.html#micrometer-integration[additional integration] to expose metrics around message rates and errors which is critical to the monitoring of deployed Streams.

All Spring Cloud Stream App Starters are configured to support two of the most popular monitoring systems, Prometheus and InfluxDB.
You can declaratively select which monitoring system to use.
If you are not using Prometheus or InfluxDB, you can customise the App starters to use a different monitoring system as well as include your preferred micrometer monitoring system library in your own custom applications.
To help you get started monitoring Streams, Data Flow provides https://grafana.com/[Grafana Dashboards] you can install and customize for your needs. Support for monitoring Tasks is on the roadmap.

The general architecture of how applications are monitored is shown below.

.The Spring Cloud Data Flow Monitoring Architecture
image::{dataflow-asciidoc}/images/micrometer-arch.png[Micrometer Monitoring Architecture, scaledwidth="80%"]

.Each Spring Cloud Stream application sends send metrics to a monitoring system, often a Time Series Database (TSDB).
.Connect Grafana to the selected monitoring system and install the provided Grafana dashboards that visualize different aspects of the running stream applications..
.The Data Flow UI provides buttons to open the Grafana dashboard for each stream.

To allow aggregating metrics per application type, per instance or per stream the Spring <<applications,Cloud Stream Application Starters>> are configured to use the following Micrometer tags:

[width="100%",frame="topbot",options="header"]
|===
|Tag Name| Description

|stream.name
|Name of the Stream that contains the applications sending the metrics

|application.name
|Name or label of the application reporting the metrics

|application.type
|The type (Sourcer, Processor or SInk) of the application reporting the metrics.

|application.guid
|Unique instance identifier of the application instance reporting the metrics

|application.index
|application instance id (when available)

|===

If the Data Flow server is started with the `spring.cloud.dataflow.grafana-info.url` property pointing to your Grafana URL, then the Grafana feature is enabled and Data Flow UI will provide you with Grafana-buttons that can open particular dashboard for given stream, application or application instance. Following screenshot illustrates these buttons:

image::{dataflow-asciidoc}/images/grafana-scdf-ui-buttons-streams.png[Data Flow UI Grafana Buttons - Streams, scaledwidth="70%"]

image::{dataflow-asciidoc}/images/grafana-scdf-ui-buttons-apps.png[Data Flow UI Grafana Buttons - Applications, scaledwidth="70%"]

As setting up Prometheus and InfluxDB is different depending on the platform you are running on, we provide instructions for each platform.
In Spring Cloud Data Flow 2.x, local server and Kubernetes instructions have been provided.

include::streams-monitoring-local.adoc[]
include::streams-monitoring-kubernetes.adoc[]

Please see the link:https://dataflow.spring.io/docs/feature-guides/streams/monitoring/[Stream Monitoring] Guide of the microsite for more information on how to monitor the applications that were deployed as part of a Stream.

0 comments on commit b95b50d

Please sign in to comment.