Skip to content
This repository has been archived by the owner on Jan 29, 2024. It is now read-only.

Added Kafka key concepts from blog into Apache Kafka® glossary for SEO enhancement. #2421

Merged
merged 5 commits into from
Jan 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions _toc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -318,6 +318,7 @@ entries:
- file: docs/products/kafka/howto/fake-sample-data
title: Sample data generator
- file: docs/products/kafka/concepts
title: Concepts
entries:
- file: docs/products/kafka/concepts/upgrade-procedure
title: Upgrade procedure
Expand Down
85 changes: 81 additions & 4 deletions docs/products/kafka/concepts.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,83 @@
Concepts
========
Apache Kafka® concepts
======================

Explanations and supporting concepts to help you succeed with Aiven for Apache Kafka®.
A comprehensive glossary of essential Apache Kafka® terms and their meanings.

.. tableofcontents::
.. _Broker:

Broker
------

A server that operates Apache Kafka, responsible for message storage, processing, and delivery. Typically part of a cluster for enhanced scalability and reliability, each broker functions independently but is integral to Kafka's overall operations, separate from tools like Apache Kafka Connect.

Consumer
--------

An application that reads data from Apache Kafka, often processing or acting upon it. Various tools used with Apache Kafka ultimately function as either a producer or a consumer when communicating with Apache Kafka.

Consumer groups
---------------

Groups of consumers in Apache Kafka are used to scale beyond a single application instance. Multiple instances of an application coordinate to handle messages, with each group allocated to different partitions for even workload distribution.

Event-driven architecture
-------------------------

Application architecture centered around responding to and processing events.

.. _Event:

Event
-----

A single discrete data unit in Apache Kafka, consisting of a ``value`` (the message body) and often a ``key`` (for quick identification) and ``headers`` (metadata about the message).

Kafka node
----------

See :ref:`Broker`

Kafka server
------------

See :ref:`Broker`

Message
-------

See :ref:`Event`

Partitioning
------------

A method used by Apache Kafka to distribute a topic across multiple servers. Each server acts as the ``leader`` for a partition, ensuring data sharding and message order within each partition.

Producer
--------

An application that writes data into Apache Kafka without concern for the data's consumers. The data can range from well-structured to simple text, often accompanied by metadata.

Check failure on line 58 in docs/products/kafka/concepts.rst

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/products/kafka/concepts.rst#L58

[Aiven.aiven_spelling] 'data's' does not seem to be a recognised word
Raw output
{"message": "[Aiven.aiven_spelling] 'data's' does not seem to be a recognised word", "location": {"path": "docs/products/kafka/concepts.rst", "range": {"start": {"line": 58, "column": 75}}}, "severity": "ERROR"}

Pub/sub
-------

A publish-subscribe messaging architecture where messages are broadcasted by publishers and received by any listening subscribers, unlike point-to-point systems.

Check failure on line 63 in docs/products/kafka/concepts.rst

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/products/kafka/concepts.rst#L63

[Aiven.aiven_spelling] 'broadcasted' does not seem to be a recognised word
Raw output
{"message": "[Aiven.aiven_spelling] 'broadcasted' does not seem to be a recognised word", "location": {"path": "docs/products/kafka/concepts.rst", "range": {"start": {"line": 63, "column": 63}}}, "severity": "ERROR"}

Queueing
--------

A messaging system where messages are sent and received in the order they are produced. Apache Kafka maintains a watermark for each consumer to track the most recent message read.

Record
------

See :ref:`Event`

Replication
-----------

Apache Kafka's feature for data replication across multiple servers, ensuring data preservation even if a server fails. This is configurable per topic.

Topic
-----

Logical channels in Apache Kafka through which messages are organized. Topics are named in a human-readable manner, like ``sensor-readings`` or ``kubernetes-logs``.
Loading