Merge pull request #2418 from aiven/harshini-kafka-prometheus-private…

…link

Configure Apache Kafka multi-node servic-Prometheus via Privatelink

_redirects

@@ -65,6 +65,7 @@
 /docs/platform/concepts/service-level-agreement                        /docs/platform/howto/project-support-center
 /docs/products/postgresql/reference/list-of-advanced-params            /docs/products/postgresql/reference/advanced-params
 /docs/products/kafka/reference/kstream-data-write-issue                https://aiven.io/changelog#06-12-2023
+/docs/tools/api/examples                                               /docs/tools/api
 
 
 # Moved to https://aiven.io/developer
@@ -91,6 +92,8 @@
 /docs/community/challenge/the-rolling-challenge                                 https://aiven.io/community/
 /docs/tools/cli/account/account-authentication-method                           /docs/tools/cli/account
 /docs/tools/cli/card                                                            /docs/tools/cli/account
+
+
 /docs/tools/api/examples                                                        /docs/tools/api
 /docs/products/postgresql/getting-started                                       /docs/products/postgresql/get-started
 /docs/products/m3db/getting-started                                             /docs/products/m3db/get-started
@@ -103,6 +106,7 @@
 /docs/products/opensearch/dashboards/getting-started                            /docs/products/opensearch/dashboards/get-started
 /docs/products/kafka/kafka-mirrormaker/getting-started                          /docs/products/kafka/kafka-mirrormaker/get-started
 
+
 # Redirect from .index.html to specific page names for landing
 
 # with one section and no subsections, i. e. docs/platform

_toc.yml

@@ -416,6 +416,7 @@ entries:
               - file: docs/products/kafka/howto/kafka-streams-with-aiven-for-kafka
               - file: docs/products/kafka/howto/flink-with-aiven-for-kafka
               - file: docs/products/kafka/howto/datadog-customised-metrics
+              - file: docs/products/kafka/howto/kafka-prometheus-privatelink
               - file: docs/products/kafka/howto/ksql-docker
                 title: Use ksqlDB with Aiven for Apache Kafka
               - file: docs/products/kafka/howto/add-missing-producer-consumer-metrics

docs/platform/howto/integrations/prometheus-metrics.rst

@@ -48,7 +48,8 @@ Aiven offers Prometheus endpoints for your services. To enable this feature, tak
 
       At the top of the **Integrations** page, you will see the Prometheus integration listed and status ``active``. 
 
-#. From the **Integrations** page, go to the **Overview** page > the **Connection information** section > the **Prometheus** tab.
+#. Next, navigate to the service's **Overview** page, and then locate the **Connection information** section. 
+#. Click on the **Prometheus** tab.
 #. Copy **Service URI**, and use it in your browser to access the Prometheus dashboard.
 
 .. topic:: Result
@@ -59,10 +60,29 @@ Aiven offers Prometheus endpoints for your services. To enable this feature, tak
 
    There might be a slight delay of approximately one minute before the metrics become available.
 
+
 Accessing Prometheus in a VPC
 ''''''''''''''''''''''''''''''
 
-If you use a VPC in your project, to access Prometheus, you need to go to `Aiven Console <https://console.aiven.io/>`_ > your project > the service you want to monitor using Prometheus > the **Service settings** page > the **Cloud and network** section > the actions (**...**) menu > the **More network configurations**. In the **Network configuration** window, select **Add configuration options**, search for property ``public_access.prometheus``, enable it, and select **Save configuration**.
+If you use a VPC in your project, follow these steps to access Prometheus:
+
+1. Access `Aiven Console <https://console.aiven.io/>`_.
+
+2. Select your project, and select the service you want to monitor using Prometheus.
+
+3. Click **Service settings** from the sidebar. 
+
+4. In the **Cloud and network** section, click on the actions (**...**) menu.
+
+5. Choose **More network configurations**.
+
+6. In the **Network configuration** window, select **Add configuration options**.
+
+7. Search for the ``public_access.prometheus`` property and enable it.
+
+8. Click **Save configuration**.
+
+
 
 Configure Prometheus
 --------------------
@@ -133,7 +153,7 @@ Multi-node services
 
 .. note::
 
-   For Aiven services with multiple nodes and a Replica URI, the primary DNS name does not include standby IP addresses. To track those, make sure to include the replica DNS names in the list. If you have ``<PROMETHEUS_SERVICE_URI>`` as ``public-example.aivencloud.com``, then you will need to add ``public-replica-example.aivencloud.com``. This applies to PostgreSQL®, MySQL®, and Redis®* services.
+   For Aiven services with multiple nodes and a Replica URI, the primary DNS name does not include standby IP addresses. To track those, make sure to include the replica DNS names in the list. If you have ``<PROMETHEUS_SERVICE_URI>`` as ``public-example.aivencloud.com``, then you will need to add ``public-replica-example.aivencloud.com``. This applies to PostgreSQL®, MySQL®, Apache Kafka®, and Redis®* services.
 
 View full list of metrics
 ''''''''''''''''''''''''''
@@ -148,10 +168,11 @@ You can preview the full list of metrics in :doc:`Prometheus system metrics </do
 
    For some services the metrics provided by different hosts may vary depending on the host role. Most notably for Kafka® only one of the nodes provides metrics related to consumer group offsets.
 
-More integrations
+Related pages
 -----------------
 
 Learn more about integrations with Aiven:
 
 * :doc:`Aiven integrations </docs/platform/concepts/service-integration>`
 * :doc:`Datadog integration </docs/integrations/datadog>`
+* Configure Prometheus for Aiven for Apache Kafka® via Privatelink

docs/products/kafka/howto/kafka-prometheus-privatelink.rst

@@ -0,0 +1,84 @@
+Configure Prometheus for Aiven for Apache Kafka® using Privatelink
+====================================================================
+
+You can integrate Prometheus with your Aiven for Apache Kafka® service using Privatelink for secure monitoring. This setup uses a Privatelink load balancer, which allows for efficient service discovery of Apache Kafka nodes and enables you to connect to your Aiven for Apache Kafka service using a private endpoint in your network or VPCs.
+
+
+Prerequisites
+-------------
+
+Before you start, ensure you have the following:
+
+- :doc:`Aiven for Apache Kafka® </docs/products/kafka/get-started>` service running.
+- :doc:`Prometheus integration </docs/platform/howto/integrations/prometheus-metrics>` set up for your Aiven for Apache Kafka for extracting metrics.
+- Necessary permissions to modify service configurations.
+
+
+Configuration steps
+--------------------
+
+Basic configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Begin by configuring Prometheus to scrape metrics from your Aiven for Apache Kafka service. This setup involves specifying various parameters for secure data retrieval. Following is an example configuration:
+
+.. code-block:: yaml
+
+    scrape_configs:
+      - job_name: aivenmetrics
+        scheme: https
+        tls_config:
+          insecure_skip_verify: true
+        basic_auth:
+          username: <PROMETHEUS_USERNAME>
+          password: <PROMETHEUS_PASSWORD>
+        http_sd_configs:
+          - url: <PROMETHEUS_PRIVATELINK_ACCESS_SERVICE_URI>
+            refresh_interval: 120s
+            tls_config:
+              insecure_skip_verify: true
+            basic_auth:
+              username: <PROMETHEUS_USERNAME>
+              password: <PROMETHEUS_PASSWORD>
+
+**Configuration details**:
+
+- ``job_name``: Identifies the set of targets, e.g., ``aivenmetrics``.
+- ``scheme``: Specifies the protocol, typically ``https``.
+- ``tls_config``: Manages TLS settings. 
+
+  .. note::
+    Setting ``insecure_skip_verify: true`` is crucial, as it permits Prometheus to disregard TLS certificate validation against host IP addresses, facilitating seamless connectivity.
+
+- ``basic_auth``: Provides authentication credentials for Apache Kafka service access.
+- ``http_sd_configs``: Configures HTTP Service Discovery. Includes:
+
+  - ``url``: The URI for Prometheus Privatelink service access.
+  - ``refresh_interval``: The frequency of target list refresh, e.g., ``120s``.
+
+.. note::
+    The ``basic_auth`` and ``tls_config`` are specified twice - first for scraping the HTTP SD response and then to retrieve service metrics. This duplication is necessary because the same authentication and security settings are used to retrieve the service discovery information and scrape the metrics.
+
+(Optional) Metadata and relabeling 
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If your setup involves multiple Privatelink connections, you can leverage Prometheus's relabeling for better target management. This approach allows you to dynamically modify target label sets before scraping. 
+
+To manage metrics from different Privatelink connections, include the ``__meta_privatelink_connection_id`` label in your configuration. This setup helps categorize and filter relevant metrics for each connection.
+
+.. code-block:: yaml
+
+    relabel_configs:
+      - source_labels: [__meta_privatelink_connection_id]
+        regex: 1
+        action: keep
+
+
+The ``regex: 1`` in the configuration is a placeholder. Make sure to replace ``1`` with the actual Privatelink connection ID that you wish to monitor.
+
+
+
+Related pages
+--------------
+
+* :doc:`Aiven for Apache Kafka® metrics available via Prometheus </docs/products/kafka/reference/kafka-metrics-prometheus>`