Skip to content

Commit

Permalink
[DPE-6046] Update terminology regarding Apache Kafka trademarks (#280)
Browse files Browse the repository at this point in the history
Co-authored-by: discourse-gatekeeper-docs-bot <[email protected]>
  • Loading branch information
github-actions[bot] and discourse-gatekeeper-docs-bot authored Dec 19, 2024
1 parent 723effa commit 9ca899a
Show file tree
Hide file tree
Showing 35 changed files with 615 additions and 421 deletions.
23 changes: 12 additions & 11 deletions docs/explanation/e-cluster-configuration.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
# Overview of a cluster configuration content
# Cluster configuration

[Apache Kafka](https://kafka.apache.org) is an open-source distributed event streaming platform that requires an external solution to coordinate and sync metadata between all active brokers.
One of such solutions is [ZooKeeper](https://zookeeper.apache.org).
One of such solutions is [Apache ZooKeeper](https://zookeeper.apache.org).

Here are some of the responsibilities of ZooKeeper in a Kafka cluster:
Here are some of the responsibilities of Apache ZooKeeper in an Apache Kafka cluster:

- **Cluster membership**: through regular heartbeats, it keeps tracks of the brokers entering and leaving the cluster, providing an up-to-date list of brokers.
- **Controller election**: one of the Kafka brokers is responsible for managing the leader/follower status for all the partitions. ZooKeeper is used to elect a controller and to make sure there is only one of it.
- **Topic configuration**: each topic can be replicated on multiple partitions. ZooKeeper keeps track of the locations of the partitions and replicas, so that high-availability is still attained when a broker shuts down. Topic-specific configuration overrides (e.g. message retention and size) are also stored in ZooKeeper.
- **Access control and authentication**: ZooKeeper stores access control lists (ACL) for Kafka resources, to ensure only the proper, authorized, users or groups can read or write on each topic.
- **Cluster membership**: through regular heartbeats, it keeps track of the brokers entering and leaving the cluster, providing an up-to-date list of brokers.
- **Controller election**: one of the Apache Kafka brokers is responsible for managing the leader/follower status for all the partitions. Apache ZooKeeper is used to elect a controller and to make sure there is only one of it.
- **Topic configuration**: each topic can be replicated on multiple partitions. Apache ZooKeeper keeps track of the locations of the partitions and replicas so that high availability is still attained when a broker shuts down. Topic-specific configuration overrides (e.g. message retention and size) are also stored in Apache ZooKeeper.
- **Access control and authentication**: Apache ZooKeeper stores access control lists (ACL) for Apache Kafka resources, to ensure only the proper, authorized, users or groups can read or write on each topic.

The values for the configuration parameters mentioned above are stored in znodes, the hierarchical unit data structure in ZooKeeper.
The values for the configuration parameters mentioned above are stored in znodes, the hierarchical unit data structure in Apache ZooKeeper.
A znode is represented by its path and can both have data associated with it and children nodes.
ZooKeeper clients interact with its data structure similarly to a remote file system that would be sync-ed between the ZooKeeper units for high availability.
For a Charmed Kafka related to a Charmed ZooKeeper:
Apache ZooKeeper clients interact with its data structure similarly to a remote file system that would be synced between the Apache ZooKeeper units for high availability.
For a Charmed Apache Kafka related to a Charmed Apache ZooKeeper:

- the list of the broker ids of the cluster can be found in `/kafka/brokers/ids`
- the endpoint used to access the broker with id `0` can be found in `/kafka/brokers/ids/0`
- the credentials for the Charmed Kafka users can be found in `/kafka/config/users`
- the credentials for the Apache Kafka users can be found in `/kafka/config/users`
49 changes: 25 additions & 24 deletions docs/explanation/e-hardening.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Security Hardening Guide

This document provides guidance and instructions to achieve
a secure deployment of [Charmed Kafka](https://github.com/canonical/kafka-bundle), including setting up and managing a secure environment.
a secure deployment of [Charmed Apache Kafka](https://github.com/canonical/kafka-bundle), including setting up and managing a secure environment.
The document is divided into the following sections:

1. Environment, outlining the recommendation for deploying a secure environment
2. Applications, outlining the product features that enable a secure deployment of a Kafka cluster
2. Applications, outlining the product features that enable a secure deployment of an Apache Kafka cluster
3. Additional resources, providing any further information about security and compliance

## Environment
Expand All @@ -17,7 +17,7 @@ The environment where applications operate can be divided in two components:

### Cloud

Charmed Kafka can be deployed on top of several clouds and virtualization layers:
Charmed Apache Kafka can be deployed on top of several clouds and virtualization layers:

| Cloud | Security guide |
|-----------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
Expand All @@ -36,7 +36,7 @@ all applications. Therefore, it is imperative that it is set up securely. Please
#### Cloud credentials

When configuring the cloud credentials to be used with Juju, ensure that the users have correct permissions to operate at the required level.
Juju superusers responsible for bootstrapping and managing controllers require elevated permissions to manage several kind of resources, such as
Juju superusers responsible for bootstrapping and managing controllers require elevated permissions to manage several kinds of resources, such as
virtual machines, networks, storages, etc. Please refer to the references below for more information on the policies required to be used depending on the cloud.

| Cloud | Cloud user policies |
Expand All @@ -47,47 +47,47 @@ virtual machines, networks, storages, etc. Please refer to the references below

#### Juju users

It is very important that the different juju users are set up with minimal permission depending on the scope for their operations.
It is very important that the different Juju users are set up with minimal permissions depending on the scope of their operations.
Please refer to the [User access levels](https://juju.is/docs/juju/user-permissions) documentation for more information on the access level and corresponding abilities
that the different users can be granted.

Juju user credentials must be stored securely and rotated regularly to limit the chances of unauthorized access due to credentials leakage.

## Applications

In the following we provide guidance on how to harden your deployment using:
In the following, we provide guidance on how to harden your deployment using:

1. Operating System
2. Kafka and ZooKeeper Security Upgrades
1. Operating system
2. Security upgrades
3. Encryption
4. Authentication
5. Monitoring and Auditing
5. Monitoring and auditing

### Operating System
### Operating system

Charmed Kafka and Charmed ZooKeeper currently run on top of Ubuntu 22.04. Deploy a [Landscape Client Charm](https://charmhub.io/landscape-client?) in order to
Charmed Apache Kafka and Charmed Apache ZooKeeper currently run on top of Ubuntu 22.04. Deploy a [Landscape Client Charm](https://charmhub.io/landscape-client?) in order to
connect the underlying VM to a Landscape User Account to manage security upgrades and integrate Ubuntu Pro subscriptions.

### Kafka and ZooKeeper Security Upgrades
### Security upgrades

Charmed Kafka and Charmed ZooKeeper operators install a pinned revision of the [Charmed Kafka snap](https://snapcraft.io/charmed-kafka)
and [Charmed ZooKeeper snap](https://snapcraft.io/charmed-zookeeper), respectively, in order to provide reproducible and secure environments.
New versions of Charmed Kafka and Charmed ZooKeeper may be released to provide patching of vulnerabilities (CVEs).
Charmed Apache Kafka and Charmed Apache ZooKeeper operators install a pinned revision of the [Charmed Apache Kafka snap](https://snapcraft.io/charmed-kafka)
and [Charmed ZooKeeper snap](https://snapcraft.io/charmed-zookeeper), respectively, to provide reproducible and secure environments.
New versions of Charmed Apache Kafka and Charmed Apache ZooKeeper may be released to provide patching of vulnerabilities (CVEs).
It is important to refresh the charm regularly to make sure the workload is as secure as possible.
For more information on how to refresh the charm, see the [how-to upgrade](https://charmhub.io/kafka/docs/h-upgrade) guide.

### Encryption

Charmed Kafka must be deployed with encryption enabled.
To do that, you need to relate Kafka and ZooKeeper charms to one of the TLS certificate operator charms.
Charmed Apache Kafka must be deployed with encryption enabled.
To do that, you need to relate Charmed Apache Kafka and Charmed Apache ZooKeeper to one of the TLS certificate operator charms.
Please refer to the [Charming Security page](https://charmhub.io/topics/security-with-x-509-certificates) for more information on how to select the right certificate
provider for your use-case.
provider for your use case.

For more information on encryption setup, see the [How to enable encryption](https://charmhub.io/kafka/docs/h-enable-encryption) guide.

### Authentication

Charmed Kafka supports the following authentication layers:
Charmed Apache Kafka supports the following authentication layers:

1. [SCRAM-based SASL Authentication](/t/charmed-kafka-how-to-manage-app/10285)
2. [certificate-base Authentication (mTLS)](/t/create-mtls-client-credentials/11079)
Expand All @@ -98,21 +98,22 @@ Please refer to the [listener reference documentation](/t/charmed-kafka-document

### Monitoring and Auditing

Charmed Kafka provides native integration with the [Canonical Observability Stack (COS)](https://charmhub.io/topics/canonical-observability-stack).
Charmed Apache Kafka provides native integration with the [Canonical Observability Stack (COS)](https://charmhub.io/topics/canonical-observability-stack).
To reduce the blast radius of infrastructure disruptions, the general recommendation is to deploy COS and the observed application into
separate environments, isolated one another. Refer to the [COS production deployments best practices](https://charmhub.io/topics/canonical-observability-stack/reference/best-practices)
for more information.

Refer to How-To user guide for more information on:
* [how to integrate the Charmed Kafka deployment with COS](/t/charmed-kafka-how-to-enable-monitoring/10283)

* [how to integrate the Charmed Apache Kafka deployment with COS](/t/charmed-kafka-how-to-enable-monitoring/10283)
* [how to customise the alerting rules and dashboards](/t/charmed-kafka-documentation-how-to-integrate-custom-alerting-rules-and-dashboards/13431)

External user access to Kafka is logged to the `kafka-authorizer.log` that is pushes to [Loki endpoint](https://charmhub.io/loki-k8s) and exposed via [Grafana](https://charmhub.io/grafana), both components being part of the COS stack.
Access denials are logged at INFO level, whereas allowed accesses are logged at DEBUG level. Depending on the auditing needs,
External user access to Apache Kafka is logged to the `kafka-authorizer.log` that is pushed to [Loki endpoint](https://charmhub.io/loki-k8s) and exposed via [Grafana](https://charmhub.io/grafana), both components being part of the COS stack.
Access denials are logged at the `INFO` level, whereas allowed accesses are logged at the `DEBUG` level. Depending on the auditing needs,
customize the logging level either for all logs via the [`log_level`](https://charmhub.io/kafka/configurations?channel=3/stable#log_level) config option or
only tune the logging level of the `authorizerAppender` in the `log4j.properties` file. Refer to the Reference documentation, for more information about
the [file system paths](/t/charmed-kafka-documentation-reference-file-system-paths/13262).

## Additional Resources

For further information and details on the security and cryptographic specifications used by Charmed Kafka, please refer to the [Security Explanation page](/t/charmed-kafka-documentation-explanation-security/15714).
For further information and details on the security and cryptographic specifications used by Charmed Apache Kafka, please refer to the [Security Explanation page](/t/charmed-kafka-documentation-explanation-security/15714).
Loading

0 comments on commit 9ca899a

Please sign in to comment.