Skip to content
This repository has been archived by the owner on Jan 29, 2024. It is now read-only.

Commit

Permalink
Merge pull request #1546 from aiven/fix-warnings-couchbsase
Browse files Browse the repository at this point in the history
Fixed some warnings related to connectors docs and mysql
  • Loading branch information
anelook authored Nov 10, 2022
2 parents 257b8f1 + 13ce0fa commit 3582957
Show file tree
Hide file tree
Showing 12 changed files with 68 additions and 46 deletions.
3 changes: 3 additions & 0 deletions .github/vale/styles/Aiven/capitalization_headings.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ exceptions:
- Apache
- AWS Transit Gateway
- Auth0
- Azure
- Azure Marketplace
- Boot
- Business
Expand Down Expand Up @@ -57,6 +58,8 @@ exceptions:
- InnoDB
- IPsec
- Java
- JMX
- Jolokia
- Kafdrop
- Kafka
- Karapace
Expand Down
28 changes: 21 additions & 7 deletions docs/platform/concepts/service-power-cycle.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@ Service power cycle

Aiven service power off and power on is more than stopping and starting a service on nodes. For better utilisation of resources on Aiven platform, idle resources will be released and only the necessary data will be kept after power off. The impact on the service is different depending on the service type and plan.

.. warning:: Depending on service type and plan, data loss may happen during a service power off, so it is important for users to understand the consequences before powering off a service.
.. Warning::

Depending on service type and plan, data loss may happen during a service power off, so it is important for users to understand the consequences before powering off a service.

Aiven service power off and power on can be done on `Aiven Console <https://console.aiven.io>`_ or through :doc:`Aiven CLI </docs/platform/howto/pause-from-cli>`.

Expand All @@ -14,15 +16,25 @@ Whenever an Aiven service is powered off:

* All virtual machine(s) of the service will be **removed** from the public cloud.
* The service information and configuration will be stored on Aiven Platform, while service data will be lost if there's no backup available .
* If the service has **time-based** or **PITR (point in time recovery)** backups, they will be kept on Aiven Platform. The backups are listed in the ``Backups`` tab of the service on Aiven Console. Absence of the tab means the service has no backups. For details on backups for different Aiven services on different plans, please refer to :doc:`Backups at Aiven </docs/platform/concepts/service_backups>`.
.. warning:: Aiven does `periodic cleanup of powered-off services <https://help.aiven.io/en/articles/4578430-periodic-cleanup-of-powered-off-services>`_ on services powered off for longer than **180** consecutive days. Notification emails will be sent before actions are taken.
* The message in the **Power Off Confirmation** window will give some hints on the consquence of the power off. Below is an example of powering off an Aiven for Redis service whose data since the latest backup will be lost because the service only has time-based but not PITR backups.
* If the service has **time-based** or **PITR (point in time recovery)** backups, they will be kept on Aiven Platform. The backups are listed in the **Backups** tab of the service on Aiven Console. Absence of the tab means the service has no backups. For details on backups for different Aiven services on different plans, please refer to :doc:`Backups at Aiven </docs/platform/concepts/service_backups>`.

.. Warning::

Aiven does `periodic cleanup of powered-off services <https://help.aiven.io/en/articles/4578430-periodic-cleanup-of-powered-off-services>`_ on services powered off for longer than **180** consecutive days. Notification emails will be sent before actions are taken.

* The message in the **Power Off Confirmation** window will give some hints on the consequence of the power off. Below is an example of powering off an Aiven for Redis®* service whose data since the latest backup will be lost because the service only has time-based but not PITR backups.

.. image:: /images/platform/power-off-confirmation.png
:alt: Power Off Confirmation

* Moreover, under the ``Backups`` tab, hovering the mouse over the help icon (if it's available) can present some details on the content of the backups. This information suggests what can be restored if the service is powered on later.

.. image:: /images/platform/backup-help-info.png
:alt: Backup Help Information
.. warning:: For backup enabled Aiven for Apache Kafka® services, topic configuration, schemas and connectors are all backed up, but not the data in topics. Therefore all topic data will be lost on power off. For Kafka services without backups, topic configurations together with all topic data will be lost on power off.

.. Warning::

For backup enabled Aiven for Apache Kafka® services, topic configuration, schemas and connectors are all backed up, but not the data in topics. Therefore all topic data will be lost on power off. For Kafka services without backups, topic configurations together with all topic data will be lost on power off.


Power on
Expand All @@ -33,8 +45,10 @@ When a service is powered on, the following things will happen:
* New virtual machine(s) will be created on the specified public cloud for the service.
* Service will be started with the stored configuration parameters.
* The latest time-based backup that is available will be restored. The restore time depends on the network bandwidth and disk IOPS allocated to the service plan as well as the size of the backup. It could take from minutes to hours. Smaller plans with larger backups take longer time than bigger plans with smaller backups. Restore progress can be checked by Aiven support with Aiven Admin CLI.
* If PITR backup is avilable, the database transaction log (e.g. ``WAL`` for PostgreSQL, ``binlog`` for MySQL) will be replayed to recover the service data to a specific point in time.
* If PITR backup is available, the database transaction log (e.g. ``WAL`` for PostgreSQL®, ``binlog`` for MySQL) will be replayed to recover the service data to a specific point in time.
* Service will be ready for serving.

.. warning:: Depending on the service plan, backups have different retention periods. Data will be lost after the retention period.
.. Warning::

Depending on the service plan, backups have different retention periods. Data will be lost after the retention period.

30 changes: 18 additions & 12 deletions docs/platform/howto/manage-vpc-peering.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@ Manage Virtual Private Cloud (VPC) peering

Virtual Private Cloud (VPC) peering is a method of connecting separate AWS, Google Cloud, or Azure private networks with each other. This makes it possible for the virtual machines in the different VPC to talk to each other directly without going through the public internet.

.. _platform_howto_setup_vpc_peering:

Configure VPC peering
----------------------------------------

In Aiven, VPC peering is configured as a project and region-specific setting. This means that all services created and running use the same VPC peering connection. If necessary, you can use different connections for VPC peering across multiple projects.

To set up VPC peering for your Aiven project:
Expand All @@ -14,9 +19,10 @@ To set up VPC peering for your Aiven project:
3. On the right, click **Create VPC** button.

.. note::
You'll need either an **admin** or an **operator** user role to be able to create a VPC. For more information about Aiven project members and roles, refer to :doc:`../concepts/projects_accounts_access`.

4. Enter the IP range that you want to use for the VPC connection. Use an IP range that does not overlap with any networks that you want to connect via VPC peering. For example, if your own networks use the range 10.0.0.0/8, you could set the range for your Aiven project's VPC to 192.168.0.0/24.
You'll need either an **admin** or an **operator** user role to be able to create a VPC. For more information about Aiven project members and roles, refer to :doc:`projects, accounts, and managing access permissions </docs/platform/concepts/projects_accounts_access>`.

4. Enter the IP range that you want to use for the VPC connection. Use an IP range that does not overlap with any networks that you want to connect via VPC peering. For example, if your own networks use the range `10.0.0.0/8`, you could set the range for your Aiven project's VPC to `192.168.0.0/24`.

5. Click **Create VPC**.

Expand All @@ -25,16 +31,17 @@ Once you have created the VPC, Aiven automatically sets it up and updates the st
When you create a new service, you can then place it in the VPC. The **VPC** tab in the *Select Service Cloud Region* section lists the available VPC. This also allows you to migrate a service to or from a VPC.

.. note::

Depending on the cloud provider that you selected for the VPC connection, you also have to accept a VPC peering connection request or set up a corresponding VPC peering connection to Aiven.

Cloud-specific VPC peering instructions
-----------------------------------------------------

- :doc:`Set up VPC peering on Amazon Web Services (AWS) <vpc-peering-aws>`
- :doc:`Set up VPC peering on Google Cloud Platform (GCP) <vpc-peering-gcp>`
- :doc:`Set up VNet (VPC) peering on Microsoft Azure <vnet-peering-azure>`
- :doc:`Set up VPC peering on Amazon Web Services (AWS) </docs/platform/howto/vpc-peering-aws>`
- :doc:`Set up VPC peering on Google Cloud Platform (GCP) </docs/platform/howto/vpc-peering-gcp>`
- :doc:`Set up VNet (VPC) peering on Microsoft Azure </docs/platform/howto/vnet-peering-azure>`

Deploying new services to a VPC
Deploy new services to a VPC
-------------------------------

When you create a new service, your peered VPC is available as a new geolocation on the **VPC** tab under *Select Service Cloud Region*.
Expand All @@ -44,13 +51,13 @@ It might take a few minutes for newly created VPC to appear for service deployme

The service nodes use firewall rules to allow only connections from private IP ranges that originate from networks on the other end of VPC peering connections. You can only deploy services to a VPC if they belong to the project where that specific VPC was created.

Deleting an existing VPC and VPC peering
Delete an existing VPC and VPC peering
----------------------------------------

Before deleting an existing VPC from Aiven console, you should move out any active services from that VPC. To delete a VPC, navigate to the Aiven console under the VPC section. You can find the **Delete** button as the last column for each VPC.
Once the VPC is deleted, the cloud provider side of the peering connection will go to an inactive or deleted state.

Migrating a public service to a VPC
Migrate a public service to a VPC
-----------------------------------

You can migrate any Aiven service to or from a VPC.
Expand All @@ -70,13 +77,12 @@ You can migrate any Aiven service to or from a VPC.
Once you migrate your service to an Aiven project-specific VPC, you can no longer access the service over the public internet. You can only access it from clients that are in a VPC that is peered to the VPC for the Aiven project.


Accessing VPC services from the public internet
Access VPC services from the public internet
-----------------------------------------------

When you move your service to a VPC, access from public networks is blocked by default unless you switch on public access, which generates a separate endpoint with a public- prefix that you can use.
You can switch on public internet access for your services in the service's *Overview* > *Advanced Configuration* section, but this option is switched off by default. As an example, see :doc:`how to enable public access in a VPC <public-access-in-vpc>`.
You can switch on public internet access for your services in the service's *Overview* > *Advanced Configuration* section, but this option is switched off by default. As an example, see :doc:`how to enable public access in a VPC </docs/platform/howto/public-access-in-vpc>`.

IP filtering (the Allowed IP Addresses list on the service overview page) is still available for a service deployed to a VPC where both public and private access are allowed. We recommend that you use IP filtering when your VPC service is also exposed to the public internet.

Also note that safelisting applies to both internal and external traffic. If you safelist an external IP address and want to keep traffic flowing with the internal (peered) connections, make sure that you safelist the CIDR blocks of the peered networks as well to avoid disruptions to the service.

Also note that safelisting applies to both internal and external traffic. If you safelist an external IP address and want to keep traffic flowing with the internal (peered) connections, make sure that you safelist the CIDR blocks of the peered networks as well to avoid disruptions to the service.
4 changes: 2 additions & 2 deletions docs/platform/howto/restrict-access.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Restrict network access to your service
=======================================
========================================

It is possible to restrict access to your service to a single IP, and address block, or any combination of both. By default the service is publicly accessible.

Expand All @@ -8,4 +8,4 @@ It is possible to restrict access to your service to a single IP, and address bl
3. Enter your address or address block using the CIDR notation. You can specify several addresses or blocks or combination of both in a single change.
4. Click **Save Changes**.

You can also use the :ref:`dedicated service update function <avn-cli-service-update>` to create or update the IP filter for your service via the :doc:`Aiven CLI </docs/tools/cli>`.
You can also use the :ref:`dedicated service update function <avn-cli-service-update>` to create or update the IP filter for your service via the :doc:`Aiven CLI </docs/tools/cli>`.
30 changes: 15 additions & 15 deletions docs/platform/howto/vnet-peering-azure.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Preparation
Please install the `Azure CLI <https://docs.microsoft.com/en-us/cli/azure/?view=azure-cli-latest>`__
as well as the :doc:`Aiven CLI </docs/tools/cli>` to follow this guide.

1. Log in with an Azure admin account
1. log in with an Azure admin account
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Using the Azure CLI:
Expand All @@ -57,7 +57,7 @@ is not needed if there's only one subscription:
az account set --subscription <subscription name or id> 


2. Create application object
2. create application object
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Create an application object in your AD tenant. Using the Azure CLI,
Expand All @@ -73,7 +73,7 @@ tenant (the tenant the app was created in) has the credentials to
authenticate the app. Save the ``appId`` field from the output - this
will be referred to as ``$user_app_id``

3. Create a service principal for your app object
3. create a service principal for your app object
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Create a service principal for the app object you created. The service
Expand All @@ -90,7 +90,7 @@ output - this will be referred to as ``$user_sp_id`` . Notice that this
is different from the ``$user_app_id`` value earlier, which is also
shown in the output.

4. Set a password for your app object
4. set a password for your app object
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

::
Expand All @@ -100,7 +100,7 @@ shown in the output.
Save the ``password`` field from the output - this will be referred to
as ``$user_app_secret`` below

5. Find the id properties of your virtual network
5. find the id properties of your virtual network
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This can be found in the Azure portal in "Virtual networks" -> name of
Expand All @@ -127,7 +127,7 @@ Also grab
``$user_vnet_id`` should have the format
``/subscriptions/$user_subscription_id/resourceGroups/$user_resource_group/providers/Microsoft.Network/virtualNetworks/$user_vnet_name``

6. Grant your service principal permissions to peer
6. grant your service principal permissions to peer
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The service principal created in step 3 needs to be assigned a role that
Expand Down Expand Up @@ -156,7 +156,7 @@ you, it may also be given permission for the scope of an entire resource
group, or the whole subscription to allow create other peerings later
without assigning the role again for each VNet separately.

7. Create a service principal for the Aiven application object
7. create a service principal for the Aiven application object
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The Aiven AD tenant contains an application object (similar to the one
Expand All @@ -179,7 +179,7 @@ tenant" then your account does not have the correct permissions. Please
use an account with at least the **Application administrator** role
assigned.

8. Create a custom role for the Aiven application object
8. create a custom role for the Aiven application object
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The Aiven application now has a service principal that can be given
Expand All @@ -198,7 +198,7 @@ include. Save the ``id`` field from the output - this will be referred
to as ``$aiven_role_id``


9. Assign the custom role to the Aiven service principal
9. assign the custom role to the Aiven service principal
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To give the Aiven application object's service principal permissions to
Expand All @@ -211,7 +211,7 @@ with
az role assignment create --role $aiven_role_id --assignee-object-id $aiven_sp_id --scope $user_vnet_id


10. Find your AD tenant id
10. find your AD tenant id
~~~~~~~~~~~~~~~~~~~~~~~~~~

The ID of your AD tenant will be needed in the next step. Find it from
Expand All @@ -226,7 +226,7 @@ saving the ``tenantId`` field from the output. It will be referred to as
``$user_tenant_id`` later


11. Create a peering connection from the Aiven Project VPC
11. create a peering connection from the Aiven Project VPC
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This leads to the Aiven platform creating a peering from the VNet in the
Expand All @@ -252,7 +252,7 @@ currently only accepts names in lower case. If no error is shown, the
peering connection is being set up by the Aiven platform.


12. Wait for the Aiven platform to set up the connection
12. wait for the Aiven platform to set up the connection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Run the following command until the state is no longer ``APPROVED`` ,
Expand All @@ -276,8 +276,8 @@ Save the ``to-tenant-id`` field from the output. It will be referred to
as the ``aiven_tenant_id`` later. The ``to-network-id`` field from the
output is referred to as the ``$aiven_vnet_id``

13. Create peering from your VNet to the VNet of the project VPC
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
13. create peering from your VNet to the VNet of the project VPC
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Log out the Azure user you logged in with in step 1 using

Expand Down Expand Up @@ -322,7 +322,7 @@ the role assignment in step 6 was correct.
The client '<random uuid>' with object id '<another random uuid>' does not have authorization to perform action 'Microsoft.Network/virtualNetworks/virtualNetworkPeerings/write' over scope '$user_vnet_id' If access was recently granted, please refresh your credentials.


14. Wait until the Aiven peering connection is active
14. wait until the Aiven peering connection is active
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The Aiven platform polls peering connections in state ``PENDING_PEER``
Expand Down
2 changes: 1 addition & 1 deletion docs/products/clickhouse/howto/connect-to-grafana.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,6 @@ Integrate ClickHouse® with Grafana®
#. Set *URL* to ``CLICKHOUSE_HTTPS_URI``.
#. In *Auth* section, enable **Basic auth** and **With Credentials**.
#. In *Basic Auth Details*, set your ``CLICKHOUSE_USER`` and ``CLICKHOUSE_PASSWORD``.
#. Selec **Save & test**.
#. Select **Save & test**.

Now you can create a dashboard and panels to work with the data from your Aiven for ClickHouse® service.
2 changes: 1 addition & 1 deletion docs/products/kafka/kafka-connect/howto/couchbase-sink.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Define the connector configurations in a file (we'll refer to it with the name `
"couchbase.password": "COUCHBASE_PASSWORD",
"couchbase.bucket": "COUCHBASE_BUCKET",
"topics": "TOPIC_LIST"
}
}
The configuration file contains the following entries:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Define the connector configurations in a file (we'll refer to it with the name `
"couchbase.collections": "COUCHBASE_COLLECTIONS",
"couchbase.source.handler": "com.couchbase.connect.kafka.handler.source.RawJsonSourceHandler",
"value.converter": "org.apache.kafka.connect.converters.ByteArrayConverter",
}
}
The configuration file contains the following entries:

Expand Down
Loading

0 comments on commit 3582957

Please sign in to comment.