Skip to content
This repository has been archived by the owner on Jan 29, 2024. It is now read-only.

Updated service settings for more service topics #2390

Merged
merged 3 commits into from
Dec 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 6 additions & 10 deletions docs/products/grafana/howto/dashboard-previews.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,12 @@ Enable dashboard previews

Follow these steps to enable dashboard previews for your Aiven for Grafana service:

1. Log in to the `Aiven Console <https://console.aiven.io/>`_.
2. On the **Services** page, click the Grafana service for which you want to enable dashboard previews.
3. On the **Services overview** page, scroll down to the **Advanced configuration** section.
4. Click the **Change** button.
5. In the **Edit advanced configuration** pop-up screen, turn the toggle on next to ``dashboad_previews_enabled`` to enable the feature.
6. Click the **Save advanced configuration** button. You will notice the status next to ``dashboad_previews_enabled`` change from ``not synced`` to ``synced``.

.. image:: /images/products/grafana/enable-dashboard-previews.png
:alt: Enable dashboard previews in Advanced configuration

1. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Grafana® service.
2. In the service page, select **Service settings** from the sidebar.
3. On the **Service settings** page, scroll down to the **Advanced configuration** section, and click **Configure**.
4. In the **Advanced configuration** dialog, click **Add configuration option**.
5. Find and set ``dashboad_previews_enabled`` to **Enabled** position.
6. Click the **Save configuration**. You will notice the status next to ``dashboad_previews_enabled`` change from ``not synced`` to ``synced``.
7. Using the **Service URI**, open the Grafana login page.
8. Enter the username and password, and click **Log in**.
9. Click **Dashboards** on the left side menu, and select the grid layout to view dashboard previews of all the dashboards. Dashboard previews are rendered as thumbnails and can be sorted alphabetically.
Expand Down
5 changes: 1 addition & 4 deletions docs/products/grafana/howto/pitr-process-for-grafana.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,7 @@ The Point-in-Time Recovery (PITR) process allows you to restore your Grafana ser

2. Click **Fork & restore**.

.. image:: /images/products/grafana/grafana-pitr-fork-restore.png
:alt: click 'Fork & restore' from backup tab of Grafana service from Aiven console

3. In the **New Database Fork** screen,
3. In the **New Database Fork** dialog,

- Provide a name for the new service.
- Verify that the appropriate Project name is chosen.
Expand Down
19 changes: 10 additions & 9 deletions docs/products/kafka/howto/change-retention-period.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ For a single topic

To change the retention period for a single topic, follow these steps:

#. Log in to `Aiven Console <https://console.aiven.io/>`_ and select your Aiven for Apache Kafka® service.
#. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.

#. Select **Topics** from the left sidebar.

Expand Down Expand Up @@ -38,15 +38,16 @@ To change the retention period for a single topic, follow these steps:
At a service level
~~~~~~~~~~~~~~~~~~~

#. Log in to `Aiven Console <https://console.aiven.io/>`_ and select your Aiven for Apache Kafka® service.
#. On the **Overview** page, scroll down to **Advanced configuration** and select **Change**.
#. In the **Edit advanced configuration** screen, select **Add configuration option**.
#. You have two options to configure the retention period for Apache Kafka® logs.
#. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.
#. In the service page, select **Service settings** from the sidebar.
#. On the **Service settings** page, scroll down to the **Advanced configuration** section, and click **Configure**.
#. In the **Advanced configuration** dialog, click **Add configuration options**.
#. You have two options to configure the retention period for Apache Kafka® logs:

* You can either select ``kafka.log_retention_hours`` or ``kafka.log_retention_ms`` and set the desired length of time for retention.
* Alternatively, if you prefer to limit the amount of data retained based on storage usage, you can specify the value for ``kafka.log_retention_bytes``.
* You can either find ``kafka.log_retention_hours`` or ``kafka.log_retention_ms`` and set the desired length of time for retention.
* Alternatively, if you prefer to limit the amount of data retained based on storage usage, you can specify the value for ``kafka.log_retention_bytes``.

#. Click on **Save advanced configuration**.
#. Click on **Save configuration**.

Unlimited retention
~~~~~~~~~~~~~~~~~~~~~
Expand All @@ -55,7 +56,7 @@ We do not limit the maximum retention period in any way, and in order to disable

.. Warning::

Using high retention periods without monitoring the available storage space can cause your service to run out of disk space. These situations are not covered by our SLA.
Using high retention periods without monitoring the available storage space can cause your service to run out of disk space. These situations are not covered by our SLA.



10 changes: 6 additions & 4 deletions docs/products/kafka/howto/configure-log-cleaner.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,20 @@ The log cleaner serves the purpose of preserving only the latest value associate

Enable log compaction for all topics
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#. Log in to `Aiven Console <https://console.aiven.io/>`_ and select your Aiven for Apache Kafka service.
#. On the **Overview** page, scroll down to **Advanced configuration** and select **Add configuration option**.
#. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.
#. In the service page, select **Service settings** from the sidebar.
#. On the **Service settings** page, scroll down to the **Advanced configuration** section, and click **Configure**.
#. In the **Advanced configuration** dialog, click **Add configuration options**.
#. Find ``log.cleanup.policy`` in the list and select it.
#. Set the value to ``compact``.
#. Select **Save advanced configuration**.
#. Click **Save configuration**.

.. warning:: This change will affect all topics in the cluster that do not have a configuration override in place.

Enable log compaction for a specific topic
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#. Log in to `Aiven Console <https://console.aiven.io/>`_ and select your Aiven for Apache Kafka service.
#. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.
#. Select **Topics** from the left sidebar.
#. Select a topic you want to modify and select **Modify** in the context menu.
#. From the drop-down options for the **Cleanup policy**, select the value ``compact``.
Expand Down
6 changes: 3 additions & 3 deletions docs/products/kafka/howto/create-topic.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@ To create a new topic using the `Aiven Console <https://console.aiven.io/>`_, fo

1. Log in to `Aiven Console <https://console.aiven.io/>`_ and select the Aiven for Apache Kafka® service where you want to create the topic.
2. From the left sidebar, select **Topics**.
3. Select **Add topic** to create a new topic and enter a name for the topic.
4. If required, enable advanced configurations for the topic by toggling the corresponding option.
3. Select **Create topic** to create a new topic and enter a name for the topic.
4. If required, set the advanced configuration option to **Yes**.
5. In the **Topic advanced configuration** section, you can set properties such as the replication factor, number of partitions, and other settings. These settings can be modified later if needed.
6. Select **Add topic**.
6. Select **Create topic**.
The new topic will be visible immediately, but may take a few minutes before you can update its settings.


Expand Down
11 changes: 6 additions & 5 deletions docs/products/kafka/howto/create-topics-automatically.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,12 @@ Enable automatic topic creation using Aiven Console

To enable automatic topic creation through the Aiven Console, follow these steps:

1. Log in to `Aiven Console <https://console.aiven.io/>`_ and select your Aiven for Apache Kafka service.
2. On the **Overview** page, scroll down to **Advanced configuration** and select **Change**.
3. In the **Edit advanced configuration** screen, select **Add configuration option**.
4. Find the ``auto_create_topics_enable`` parameter and set it to true to enable automatic topic creation.
5. Select **Save advanced configuration**.
1. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.
2. In the service page, select **Service settings** from the sidebar.
3. On the **Service settings** page, scroll down to the **Advanced configuration** section, and click **Configure**.
4. In the **Advanced configuration** dialog, click **Add configuration options**.
5. Find the ``auto_create_topics_enable`` parameter and set it to true to enable automatic topic creation.
6. Select **Save configuration**.

.. Warning::

Expand Down
11 changes: 6 additions & 5 deletions docs/products/kafka/howto/enable-oidc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,11 @@ Before proceeding with the setup, ensure you have:
Enable OAuth2/OIDC via Aiven Console
-------------------------------------------------------

1. In `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.
2. On the **Overview** page, scroll down to **Advanced configuration** and select **Configure**.
3. In the **Advanced configuration** screen, select **Add configuration options**.
4. Set the following OIDC parameters:
1. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.
2. In the service page, select **Service settings** from the sidebar.
3. On the **Service settings** page, scroll down to the **Advanced configuration** section, and click **Configure**.
4. In the **Advanced configuration** dialog, select **Add configuration options**.
5. Set the following OIDC parameters:

* ``kafka.sasl_oauthbearer_jwks_endpoint_url``

Expand Down Expand Up @@ -60,7 +61,7 @@ Enable OAuth2/OIDC via Aiven Console
Adjusting OIDC configurations, such as enabling, disabling, or modifying settings, can lead to a rolling restart of Kafka brokers. As a result, the brokers may temporarily operate with different configurations. To minimize any operational disruptions, plan to implement these changes during a maintenance window or at a time that ensures a minimal impact on your operations.


5. Select **Save configurations** to save your changes
6. Select **Save configurations** to save your changes



Expand Down
6 changes: 3 additions & 3 deletions docs/products/kafka/howto/enabled-consumer-lag-predictor.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,9 @@ Before you start, ensure you have the following:
Enable via Aiven Console
----------------------------------------------------

1. In `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.
1. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.

2. Click **Service settings** on the sidebar.
2. In the service page, click **Service settings** on the sidebar.

3. Scroll to the **Advanced configuration** section, and select **Configure**.

Expand All @@ -37,7 +37,7 @@ Enable via Aiven Console

By default, the consumer lag predictor calculates the lag of all consumer groups. To restrict the calculation to specific groups, use the ``kafka_lag_predictor.group_filters`` option.

6. Select **Save configurations** to save your changes and enable consumer lag prediction.
6. Select **Save configuration** to save your changes and enable consumer lag prediction.

Enable via Aiven CLI
------------------------------------------------
Expand Down
15 changes: 6 additions & 9 deletions docs/products/kafka/howto/kafka-sasl-auth.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,12 @@ Use SASL authentication with Aiven for Apache Kafka®

Aiven offers a selection of :doc:`authentication methods for Apache Kafka® <../concepts/auth-types>`, including `SASL <https://en.wikipedia.org/wiki/Simple_Authentication_and_Security_Layer>`_ (Simple Authentication and Security Layer).

1. Log in to `Aiven Console <https://console.aiven.io/>`_ and choose your project.
2. From the list of services, choose the Aiven for Apache Kafka service for which you wish to enable SASL.
3. On the **Overview** page of the selected service, scroll down to the **Advanced configuration** section.
4. Select **Change**.
5. Enable the ``kafka_authentication_methods.sasl`` setting, and then select **Save advanced configuration**.

.. image:: /images/products/kafka/enable-sasl.png
:alt: Enable SASL authentication for Apache Kafka
:width: 100%
1. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.
2. In the service page, select **Service settings** from the sidebar.
3. On the **Service settings** page, scroll down to the **Advanced configuration** section.
4. Click **Configure**.
5. In the **Advanced configuration** dialog, set the ``kafka_authentication_methods.sasl`` toggle to the enabled position.
6. Click **Save configuration**.

The **Connection information** at the top of the **Overview** page will now offer the ability to connect via SASL or via Client Certificate.

Expand Down
58 changes: 33 additions & 25 deletions docs/products/kafka/howto/keystore-truststore.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,44 +7,52 @@ right tools to be able to communicate with the Aiven services.
Keystores and truststores are password-protected files accessible by the client that interacts with the service.
To create these files:

1. Log in to `Aiven Console <https://console.aiven.io/>`_ and select your Aiven for Apache Kafka service.
Access service certificates
``````````````````````````````````````````
* Log in to `Aiven Console <https://console.aiven.io/>`_ and select your Apache Kafka service.
* Download the **Access Key**, **Access Certificate**, and **CA Certificate**. The files "service.key", "service.cert", and "ca.pem" are necessary for the following steps.

2. Download the **Access Key**, **Access Certificate** and **CA Certificate**. The resulting ``service.key``, ``service.cert`` and ``ca.pem`` are going to be used in the following steps.
.. image:: /images/products/kafka/ssl-certificates-download.png
:alt: Access Key, Access Certificate, and CA Certificate download from Aiven Console

.. image:: /images/products/kafka/ssl-certificates-download.png
:alt: Download the Access Key, Access Certificate and CA Certificate from the Aiven console
Create the keystore
````````````````````````````````````

3. Use the ``openssl`` utility to create the keystore with the ``service.key`` and
``service.cert`` files downloaded previously:
* Use the ``openssl`` utility to create a keystore using the downloaded ``service.key`` and ``service.cert``:

.. code::
.. code-block::

openssl pkcs12 -export \
-inkey service.key \
-in service.cert \
-out client.keystore.p12 \
openssl pkcs12 -export \
-inkey service.key \
-in service.cert \
-out client.keystore.p12 \
-name service_key

.. Note::
The format has to be ``PKCS12`` , which is the default since Java 9.
.. Note::
Ensure the keystore format is ``PKCS12``, the default since Java 9.

5. Enter a password to protect the keystore and the key, when prompted
* Set a password for the keystore and key when prompted.

6. In the folder where the certificates are stored, use the ``keytool`` utility to create the truststore with the ``ca.pem`` file as input:
Create the truststore
``````````````````````````````

.. code::

keytool -import \
-file ca.pem \
-alias CA \
-keystore client.truststore.jks
* In the directory containing the certificates, use the ``keytool`` utility to create a truststore with the ``ca.pem`` file:

.. code-block::

7. Enter a password to protect the truststores, when prompted
keytool -import \
-file ca.pem \
-alias CA \
-keystore client.truststore.jks

8. Reply to ``yes`` to confirm trusting the CA certificate, when prompted
* When prompted, enter a password for the truststore and confirm trust in the CA certificate.

The result are the keystore named ``client.keystore.p12`` and truststore named ``client.truststore.jks`` that can be used for client applications configuration.
Resulting configuration files
``````````````````````````````
The process generates two files: "client.keystore.p12" (keystore) and "client.truststore.jks" (truststore). These files are ready for client configuration.

.. Tip::

You can use :doc:`Aiven CLI </docs/tools/cli>` ``avn service user-kafka-java-creds`` to automate the creation of both the keystore and the truststore. Check the :ref:`dedicated page <avn_service_user_kafka_java_creds>` for more details.
Use the :doc:`Aiven CLI </docs/tools/cli>` command ``avn service user-kafka-java-creds`` to automate keystore and truststore creation. For more information, see :ref:`avn_service_user_kafka_java_creds`.


18 changes: 9 additions & 9 deletions docs/products/kafka/howto/prevent-full-disks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,13 @@
Upgrade to a larger service plan
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#. Login to the `Aiven Console <https://console.aiven.io/>`_ and select your service.
#. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.

#. On the **Overview** page, scroll down to **Service plan** and select **Change plan**.
#. In the service page, select **Service settings** from the sidebar.

#. Select your new service plan and select **Change**.
#. On the **Service settings** page, scroll to **Service plan** and select **Change plan** from the **Actions (…)** menu.

#. In the **Change service plan** dialog, select your new service plan and select **Change**.

This will deploy new nodes with increased disk space. Once the data is migrated from the old nodes to the new ones, disk usage will return to an acceptable level, and write operations will be allowed again.

Expand All @@ -36,9 +38,9 @@
Delete one or more topics
~~~~~~~~~~~~~~~~~~~~~~~~~

#. Login to the `Aiven Console <https://console.aiven.io/>`__ and select your service.
#. In the `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service.

#. Select **Topics** from the left sidebar.
#. Select **Topics** from the sidebar.

#. Select the topic you want to remove, and in the **Topic info** screen, select **Remove**.

Expand All @@ -53,8 +55,6 @@
Decrease retention time/size
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Another way to make more space available without deleting an entire
topic is to reduce the retention time or size for one or more topics. If
you know how old the oldest messages are in a topic, you can lower the
retention time for the topic to make more space available. Follow the instructions :doc:`to change retention period </docs/products/kafka/howto/change-retention-period>`.
Another way to make more space available without deleting an entire topic is to reduce the retention time or size for one or more topics. If
you know how old the oldest messages are in a topic, you can lower the retention time for the topic to make more space available. Follow the instructions :doc:`to change retention period </docs/products/kafka/howto/change-retention-period>`.

Check failure on line 59 in docs/products/kafka/howto/prevent-full-disks.rst

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/products/kafka/howto/prevent-full-disks.rst#L59

[Aiven.common_replacements] Use 'Kafka' instead of 'kafka'.
Raw output
{"message": "[Aiven.common_replacements] Use 'Kafka' instead of 'kafka'.", "location": {"path": "docs/products/kafka/howto/prevent-full-disks.rst", "range": {"start": {"line": 59, "column": 204}}}, "severity": "ERROR"}

Loading
Loading