Skip to content
This repository has been archived by the owner on Jan 29, 2024. It is now read-only.

Commit

Permalink
Addressed review feedback
Browse files Browse the repository at this point in the history
  • Loading branch information
harshini-rangaswamy committed Jan 24, 2023
1 parent 30d0a52 commit 241e314
Show file tree
Hide file tree
Showing 4 changed files with 12 additions and 9 deletions.
5 changes: 4 additions & 1 deletion docs/products/flink/concepts/flink-architecture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,10 @@ Some of the responsibilities of the JobManager include:

In a high-availability setup, there may be multiple JobManagers running in the cluster, with one JobManager designated as the leader and the others as standby JobManagers.

The JobManager in Apache Flink consists of three main components: **ResourceManager**, **Dispatcher**, and **JobMaster**. The ResourceManager is responsible for managing the allocation and deallocation of resources in the Flink cluster. Additionally, ResourceMaanger is responsible for managing **Task slots** - the unit of resource scheduling in a Flink cluster.
The JobManager in Apache Flink consists of three main components:
- **ResourceManager:** The ResourceManager is responsible for managing the allocation and deallocation of resources in the Flink cluster. Additionally, ResourceManger is responsible for managing **Task slots** - the unit of resource scheduling in a Flink cluster.
- **Dispatcher:** The Dispatcher in Apache Flink ensures tasks run smoothly on the cluster by scheduling them on available task slots and ensuring that tasks are executed efficiently.
- **JobMaster: ** The JobMaster in Apache Flink makes sure a specific job runs smoothly on the cluster by coordinating the tasks and executing them correctly and efficiently.
TaskManager
-----------
Expand Down
4 changes: 2 additions & 2 deletions docs/products/flink/concepts/flink-features.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Aiven for Apache Flink® features
================================
Aiven for Apache Flink is powered by the open-source framework Apache Flink, a distributed processing engine for stateful computations over data streams. It enables you to easily get started with real-time stream processing using SQL.
Aiven for Apache Flink® is powered by the open-source framework Apache Flink, a distributed processing engine for stateful computations over data streams. It enables you to easily get started with real-time stream processing using SQL.

Some of the key features of Aiven for Apache Flink® include:

Expand All @@ -15,7 +15,7 @@ Applications in the Aiven for Apache Flink service are containers that hold ever

Interactive queries
-------------------
The interactive query feature in Aiven for Apache Flink allows you to preview the data of a Flink table or job without outputting the rows to a sink table such as Kafka. This can be useful for testing and debugging purposes, as it allows you to quickly and easily see the data that is being processed by your Flink application.
The interactive query feature in Aiven for Apache Flink allows you to preview the data of a Flink table or job without outputting the rows to a sink table such as Apache Kafka®. This can be useful for testing and debugging purposes, as it allows you to quickly and easily see the data that is being processed by your Flink application.

Built-in data flow integration with Aiven for Apache Kafka®
-----------------------------------------------------------
Expand Down
10 changes: 5 additions & 5 deletions docs/products/flink/concepts/managed-service-features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,27 @@ As a fully managed distributed data stream processing platform, deployable in th

Cluster deployment mode
-----------------------
Aiven for Apache Flink® is configured to use the `HashMap state backend <https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/runtime/state/hashmap/HashMapStateBackend.html>`_. This means that the `state <https://ci.apache.org/projects/flink/flink-docs-release-1.15/docs/concepts/stateful-stream-processing/#what-is-state>`_ is stored in memory, which can impact the performance of jobs that require keeping a very large state. We recommend you provision your platform accordingly.
Aiven for Apache Flink® is configured to use the `HashMap state backend <https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/runtime/state/hashmap/HashMapStateBackend.html>`_. This means that the `state <https://nightlies.apache.org/flink/flink-docs-stable/docs/concepts/stateful-stream-processing/#what-is-state>`_ is stored in memory, which can impact the performance of jobs that require keeping a very large state. We recommend you provision your platform accordingly.

The Flink cluster executes applications in `session mode <https://nightlies.apache.org/flink/flink-docs-release-1.13/docs/deployment/overview/#session-mode>`_ so you can deploy multiple Flink jobs on the same cluster, thus effectively utilizing the available resources.
The Flink cluster executes applications in `session mode <https://nightlies.apache.org/flink/flink-docs-stable/docs/deployment/overview/#session-mode>`_ so you can deploy multiple Flink jobs on the same cluster, thus effectively utilizing the available resources.

Scaling a Flink cluster
-----------------------
Each node is equipped with a TaskManager and JobManager. We recommend scaling up your cluster to add more CPU and memory for the TaskManager before attempting to scale out, so you make the best use of the resources with a minimum number of nodes.

By default, each TaskManager is configured with a single slot for maximum job isolation. It is highly recommended that you modify this option to match your requirements.

.. note:: Adjusting the task slots per TaskManager requires a cluster restart.
.. warning:: Adjusting the task slots per TaskManager requires a cluster restart.

Cluster restart strategy
------------------------
The cluster's default restart strategy is configured to Failure Rate. This controls Flink's restart behavior in cases of failures during the execution of jobs. Administrators can overwrite this setting in the service's advanced configuration options.
The default restart strategy of the cluster is set to `Failure Rate`. This controls how Apache Flink restarts in case of failures during job execution. Administrators can change this setting in the advanced configuration options of the service.

For more information on available options, refer to `Apache Flink's fault tolerance <https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#fault-tolerance>`_ documentation.

Disaster recovery
-----------------
Periodic checkpoints have been configured to be persisted externally in object storage. They allow Flink to recover states and positions in the streams by giving the application the same semantics as a failure-free execution.
Periodic checkpoints have been configured to be persisted externally in object storage. They allow Flink to recover states and positions in the streams by giving the application the same semantics as a failure-free execution. For information on checkpoints, see

Security considerations
-----------------------
Expand Down
2 changes: 1 addition & 1 deletion docs/products/flink/reference/flink-limitations.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ There are likely differences between Aiven for Apache Flink and an Apache Flink
Some of the differences you might see between Aiven for Apache Flink and a self-managed Flink service include:

- **User-defined functions:** Aiven for Apache Flink does not currently support using user-defined functions (UDFs).
- **Apache Flink CLI tool:** The Apache Flink® CLI tool is not currently supported as it requires access to the - JobManager in production, which is not exposed to customers.
- **Apache Flink CLI tool:** The Apache Flink® CLI tool is not currently supported as it requires access to the JobManager in production, which is not exposed to customers.
- **Job-level settings:** In Aiven for Apache Flink, each job inherits the cluster-level settings, and job-level settings are not yet supported. You cannot specify separate settings for individual jobs within the same cluster.
- **Flame graphs:** Flame graphs are marked as an experimental feature in Apache Flink 1.15 and are not currently enabled in the Aiven for Apache Flink web UI.

0 comments on commit 241e314

Please sign in to comment.