Skip to content
This repository has been archived by the owner on Jan 29, 2024. It is now read-only.

Commit

Permalink
Kafka datadog: clarify Kafka metrics vs Kafka consumer metrics
Browse files Browse the repository at this point in the history
Split the chapter of customising metrics into two - first for the main Kafka integration and the other for the  Consumer Integration.

Add an example how to use exclude_topics.

The issue was that documentation failed to mention that the four keys include_topics, exclude_topics, include_consumer_groups and exclude_consumer_groups only affect the consumer integration. And that kafka_custom_metrics applies to Kafka integration.
  • Loading branch information
juha-aiven committed Dec 20, 2023
1 parent 87f9041 commit 50d2a74
Showing 1 changed file with 24 additions and 6 deletions.
30 changes: 24 additions & 6 deletions docs/products/kafka/howto/datadog-customised-metrics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,26 +42,44 @@ Before customising the metrics, make sure that you have a Datadog endpoint confi
To customise the metrics sent to Datadog, you can use the ``service integration-update`` passing the following customised parameters:

* ``kafka_custom_metrics``: defining the comma separated list of custom metrics to include (within ``kafka.log.log_size``, ``kafka.log.log_start_offset`` and ``kafka.log.log_end_offset``)

As example to sent the ``kafka.log.log_size`` and ``kafka.log.log_end_offset`` metrics execute the following code:

.. code::
avn service integration-update \
-c kafka_custom_metrics=['kafka.log.log_size','kafka.log.log_end_offset'] \
INTEGRATION_ID
Once the update is successful and metrics have been collected and pushed, you should see them in your Datadog explorer.

.. seealso:: Learn more about :doc:`/docs/integrations/datadog`.


Customise Apache Kafka® Consumer Integration metrics sent to Datadog

Check failure on line 59 in docs/products/kafka/howto/datadog-customised-metrics.rst

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/products/kafka/howto/datadog-customised-metrics.rst#L59

[Aiven.capitalization_headings] 'Customise Apache Kafka® Consumer Integration metrics sent to Datadog' should be in sentence case
Raw output
{"message": "[Aiven.capitalization_headings] 'Customise Apache Kafka® Consumer Integration metrics sent to Datadog' should be in sentence case", "location": {"path": "docs/products/kafka/howto/datadog-customised-metrics.rst", "range": {"start": {"line": 59, "column": 1}}}, "severity": "ERROR"}
====================================================================

`Kafka Consumer Integration <https://docs.datadoghq.com/integrations/kafka/?tab=host#kafka-consumer-integration>`_ collects metrics for message offsets.

To customise the metrics sent from this Datadog integration to Datadog, you can use the ``service integration-update`` passing the following customised parameters:

* ``include_topics``: defining the comma separated list of topics to include

.. Tip::
.. Tip::

By default, all topics are included.

* ``exclude_topics``: defining the comma separated list of topics to exclude
* ``include_consumer_groups``: defining the comma separated list of consumer groups to include
* ``exclude_consumer_groups``: defining the comma separated list of consumer groups to include


As example to sent the ``kafka.log.log_size`` and ``kafka.log.log_end_offset`` metrics for ``topic1`` and ``topic2`` execute the following code:
As example to include topics ``topic1`` and ``topic2`` and exclude topic ``topic3`` execute the following code:

.. code::
avn service integration-update \
-c kafka_custom_metrics=['kafka.log.log_size','kafka.log.log_end_offset'] \
-c include_topics=['topic1','topic2'] \
-c exclude_topics=['3'] \
INTEGRATION_ID
Once the update is successful and metrics have been collected and pushed, you should see them in your Datadog explorer.

.. seealso:: Learn more about :doc:`/docs/integrations/datadog`.

0 comments on commit 50d2a74

Please sign in to comment.