Update to kafka 3.4

[sophokles73]

Updated to bitnami/kafka chart version 21.x which uses Kafka 3.4.
This aligns better with the Kafka version being used for testing in Hono
2.4.

[calohmn]

Do we explicitly want to use ca-cert.pem and not trusted-certs.pem (which contains also root-cert.pem and is used in example-trust-store.yaml) here?
(When running mosquitto_sub for example, I have to use the trusted-certs.pem for its --cafile param (e.g. in setCloud2EdgeEnv.sh), not ca-cert.pem.)

[sophokles73]

You're right, we should use the trusted-certs.pem instead.

[calohmn]

In kafka-example-keys.yaml there is also the ca-cert.pem being used currently. Should we change that as well? The Secret is being set in the existingSecrets in the Kafka chart.

[sophokles73]

Hmmm, let's take a step back and reconsider this.
The example keys/certs are based on a (self-signed) root certificate, a CA certificate that has been signed using the root key and certificates for each of Hono's components which have been signed using the CA key.
So, in general, it should actually be sufficient for all of Hono's components that need to talk to each other to use the CA cert as their trust store because all other server certs have been signed using the CA key.
When using mosquitto_sub for connecting to the MQTT adapter it should also be sufficient to use the CA cert as the trust store because the MQTT adapter's server cert has been signed by the CA key as well.

[calohmn]

When using mosquitto_sub for connecting to the MQTT adapter it should also be sufficient to use the CA cert as the trust store because the MQTT adapter's server cert has been signed by the CA key as well.

In practise, this doesn't work.
When having installed the Hono chart in the default configuration and running

export MQTT_ADAPTER_IP=$(kubectl get service eclipse-hono-adapter-mqtt --output="jsonpath={.status.loadBalancer.ingress[0]['hostname','ip']}" -n hono)
cd <iot-packages-dir>
export MOSQUITTO_OPTIONS="--cafile $(pwd)/charts/hono/example/certs/ca-cert.pem --insecure"
mosquitto_sub -d -v -h ${MQTT_ADAPTER_IP} -p 8883 -u sensor1@DEFAULT_TENANT -P hono-secret ${MOSQUITTO_OPTIONS} -t command///req/#

there is this error:

Client null sending CONNECT
OpenSSL Error[0]: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
Error: Protocol error

When using
export MOSQUITTO_OPTIONS="--cafile $(pwd)/charts/hono/example/certs/root-cert.pem --insecure"
or
export MOSQUITTO_OPTIONS="--cafile $(pwd)/charts/hono/example/certs/trusted-certs.pem --insecure"
instead, it works:

Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: command///req/#, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0

See also:

$ openssl verify -verbose -CAfile root-cert.pem ca-cert.pem
ca-cert.pem: OK

$ openssl verify -verbose -CAfile ca-cert.pem mqtt-adapter-cert.pem
C = CA, L = Ottawa, O = Eclipse IoT, OU = Hono, CN = ca
error 2 at 1 depth lookup: unable to get issuer certificate
error mqtt-adapter-cert.pem: verification failed

$ openssl verify -verbose -CAfile root-cert.pem mqtt-adapter-cert.pem
C = CA, L = Ottawa, O = Eclipse IoT, OU = Hono, CN = mqtt-adapter
error 20 at 0 depth lookup: unable to get local issuer certificate
error mqtt-adapter-cert.pem: verification failed

$ openssl verify -verbose -CAfile <(cat ca-cert.pem root-cert.pem) mqtt-adapter-cert.pem
mqtt-adapter-cert.pem: OK

[sophokles73]

Ok, I have taken another look. Actually, there is no need to add the trust store to the example keys secret because we already mount the generic hono-example-trust-store configmap into all containers, which contains the ca.crt (based on trusted-certs.pem).
I have therefore removed the change to the dispatch-router-example-keys ...

[sophokles73]

@calohmn would you mind taking another look?

[calohmn]

Minor:
The helm template directive causes a bash syntax error to be displayed in the editor (preventing also shellcheck checks of the rest of the script).
This could be avoided by using something like:

URL_SCHEME="{{- if ( eq .Values.deviceRegistryExample.hono.registry.http.insecurePortEnabled true ) }}http{{ else }}https{{ end }}"
URL_PORT=$([ "$URL_SCHEME" = "http" ] && echo "8080" || echo "8443")
HTTP_BASE_URL="${URL_SCHEME}://{{ include "hono.fullname" . }}-service-device-registry:${URL_PORT}/v1"

[calohmn]

I have therefore removed the change to the dispatch-router-example-keys ...

The change is still mentioned in the commit message.

[sophokles73]

@calohmn fixed the commit message

[calohmn]

LGTM