Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Difference between HONO (running) and CLOUD2EDGE (not running) installation #480

Closed
paulocabrita-ionos opened this issue Jul 26, 2023 · 3 comments
Closed

Difference between HONO (running) and CLOUD2EDGE (not running) installation #480

paulocabrita-ionos opened this issue Jul 26, 2023 · 3 comments
Assignees
@calohmn
Labels
Cloud2Edge Improvements to the Cloud2Edge package Hono

Comments

@paulocabrita-ionos
Copy link
Contributor

I'm trying to do the deployment of Hono and Ditto using the cloud2edge charts but without success.

I made some research and find some differences between the installation of ONLY Hono and the installation of Hono and Ditto.

Here goes my findings and what I did.

helm repo update

Installation of Hono

Install

helm install -n digitaltwin-iot eclipse-hono ./hono - Before, I did the pull of the chart.

Installed pods

eclipse-hono-adapter-amqp-7f846f9d44-mzc4n             1/1     Running       0          5m2s    10.214.249.97    iot-node-cgo7y7fhzc   <none>           <none>
eclipse-hono-adapter-http-78797b54f9-mpzhx             1/1     Running       0          5m2s    10.214.249.119   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-hono-adapter-mqtt-59d8d96c6-fggvs              1/1     Running       0          5m2s    10.214.249.103   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-hono-kafka-0                                   1/1     Running       0          5m2s    10.214.249.91    iot-node-cgo7y7fhzc   <none>           <none>
eclipse-hono-service-auth-65c9c4fbf4-9lqb9             1/1     Running       0          5m2s    10.214.249.125   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-hono-service-command-router-5849db4cd8-zm7vb   1/1     Running       0          5m2s    10.214.249.78    iot-node-cgo7y7fhzc   <none>           <none>
eclipse-hono-service-device-registry-0                 1/1     Running       0          5m2s    10.214.249.121   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-hono-zookeeper-0                               1/1     Running       0          5m2s    10.214.249.99    iot-node-cgo7y7fhzc   <none>           <none>

I checked the logs and it seems that everything is running fine.

Installation of Hono with cloud2edge

Install

helm install -n digitaltwin-iot eclipse-iot ./cloud2edge/ --wait - Before, I did the pull of the chart.

Installed pods

eclipse-iot-adapter-amqp-57449c44b-gksw8               0/1     Running       0          5m59s   10.214.249.110   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-adapter-http-d558c5c4f-gln5r               0/1     Running       0          6m      10.214.249.116   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-adapter-mqtt-55b6bf4878-qj7qc              0/1     Running       0          5m59s   10.214.249.122   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-ditto-connectivity-559d7fd9ff-v2wtq        1/1     Running       0          6m      10.214.249.88    iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-ditto-gateway-8b5764f59-9rcqd              1/1     Running       0          5m59s   10.214.249.93    iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-ditto-nginx-776f9cb494-hx7tc               1/1     Running       0          5m58s   10.214.249.111   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-ditto-policies-698b467789-9wb92            1/1     Running       0          6m      10.214.249.96    iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-ditto-swaggerui-7889c477f4-w4svj           1/1     Running       0          6m      10.214.249.127   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-ditto-things-c4f995b89-tql6h               1/1     Running       0          5m59s   10.214.249.114   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-ditto-thingssearch-56c69886bf-78dnp        1/1     Running       0          6m      10.214.249.107   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-kafka-0                                    1/1     Running       0          5m58s   10.214.249.83    iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-service-auth-ccc947c64-xr5dw               1/1     Running       0          6m      10.214.249.94    iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-service-command-router-84d696fdbb-hxhzq    0/1     Running       0          6m      10.214.249.102   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-service-device-registry-6bf7cd6b76-g6r2s   1/1     Running       0          6m      10.214.249.118   iot-node-cgo7y7fhzc   <none>           <none>
eclipse-iot-zookeeper-0                                1/1     Running       0          5m58s   10.214.249.79    iot-node-cgo7y7fhzc   <none>           <none>

Some logs

eclipse-iot-adapter-amqp-57449c44b-gksw8

...
2023-07-26 13:44:42,118 DEBUG [org.ecl.hon.cli.amq.con.imp.ConnectionFactoryImpl] (vert.x-eventloop-thread-1) can't connect to AMQP 1.0 container [amqps://eclipse-iot-service-command-router:5671, role: Command Router]: finishConnect(..) failed: Connection refused: eclipse-iot-service-command-router/10.233.54.90:5671
2023-07-26 13:44:42,118 DEBUG [org.ecl.hon.cli.amq.con.imp.HonoConnectionImpl] (vert.x-eventloop-thread-1) attempt [#99] to connect to server [eclipse-iot-service-command-router:5671, role: Command Router] failed: io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: eclipse-iot-service-command-router/10.233.54.90:5671
Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused
        at io.netty.channel.unix.Errors.newConnectException0(Errors.java:155)
        at io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128)
        at io.netty.channel.unix.Socket.finishConnect(Socket.java:321)
        at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:710)
        at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:687)
        at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567)
        at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:477)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:385)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Unknown Source)

2023-07-26 13:44:42,138 INFO  [org.apa.kaf.com.net.Selector] (kafka-producer-network-thread | event-eclipse-iot-adapter-amqp-57449c44b-gksw8_c7fa15ece004_184) [Producer clientId=event-eclipse-iot-adapter-amqp-57449c44b-gksw8_c7fa15ece004_184] Failed authentication with eclipse-iot-kafka-0.eclipse-iot-kafka-headless/10.214.249.83 (channelId=-1) (SSL handshake failed)

eclipse-iot-adapter-http-d558c5c4f-gln5r

2023-07-26 13:46:09,732 ERROR [org.ecl.hon.cli.kaf.con.HonoKafkaConsumer] (vert.x-kafka-consumer-thread-0) consumer error occurred [client-id: notification-eclipse-iot-adapter-http-d558c5c4f-gln5r_7d31a9418421_2]: org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
        at java.base/sun.security.ssl.Alert.createSSLException(Unknown Source)
        at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
        at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
        at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(Unknown Source)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(Unknown Source)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(Unknown Source)
        at java.base/sun.security.ssl.SSLHandshake.consume(Unknown Source)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(Unknown Source)
        at java.base/java.security.AccessController.doPrivileged(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(Unknown Source)
        at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:435)
        at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:523)
        at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:373)
        at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:293)
        at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:178)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:543)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
        at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:560)
        at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
        at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
        at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215)
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:252)
        at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.coordinatorUnknownAndUnready(ConsumerCoordinator.java:493)
        at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:521)
        at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1271)
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1215)
        at io.vertx.kafka.client.consumer.impl.KafkaReadStreamImpl.lambda$pollRecords$6(KafkaReadStreamImpl.java:155)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
        at java.base/sun.security.validator.PKIXValidator.doValidate(Unknown Source)
        at java.base/sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
        at java.base/sun.security.validator.Validator.validate(Unknown Source)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
        ... 32 more

eclipse-iot-adapter-mqtt-55b6bf4878-qj7qc

...
2023-07-26 13:47:26,591 DEBUG [org.ecl.hon.cli.amq.con.imp.ConnectionFactoryImpl] (vert.x-eventloop-thread-1) can't connect to AMQP 1.0 container [amqps://eclipse-iot-service-device-registry:5671, role: Credentials]: Failed to create SSL connection
2023-07-26 13:47:26,591 WARN  [org.ecl.hon.cli.amq.con.imp.HonoConnectionImpl] (vert.x-eventloop-thread-1) attempt [#164] to connect to server [eclipse-iot-service-device-registry:5671, role: Credentials] failed: javax.net.ssl.SSLHandshakeException: Failed to create SSL connection
        at io.vertx.core.net.impl.ChannelProvider$1.userEventTriggered(ChannelProvider.java:129)
        at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:346)
        at io.netty.channel.AbstractChannelHandlerContext.invokeUserEventTriggered(AbstractChannelHandlerContext.java:332)
        at io.netty.channel.AbstractChannelHandlerContext.fireUserEventTriggered(AbstractChannelHandlerContext.java:324)
        at io.netty.handler.ssl.SslHandler.handleUnwrapThrowable(SslHandler.java:1260)
        at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1241)
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1285)
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:510)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:449)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:279)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
        at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:487)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:385)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
        at java.base/sun.security.ssl.Alert.createSSLException(Unknown Source)
        at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
        at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
        at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(Unknown Source)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(Unknown Source)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(Unknown Source)
        at java.base/sun.security.ssl.SSLHandshake.consume(Unknown Source)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(Unknown Source)
        at java.base/java.security.AccessController.doPrivileged(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(Unknown Source)
        at io.netty.handler.ssl.SslHandler.runDelegatedTasks(SslHandler.java:1549)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1395)
        at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1236)
        ... 18 more
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
        at java.base/sun.security.validator.PKIXValidator.doValidate(Unknown Source)
        at java.base/sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
        at java.base/sun.security.validator.Validator.validate(Unknown Source)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
        ... 30 more
Caused by: java.security.cert.CertPathValidatorException: validity check failed
        at java.base/sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(Unknown Source)
        at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(Unknown Source)
        at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(Unknown Source)
        at java.base/sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(Unknown Source)
        at java.base/java.security.cert.CertPathValidator.validate(Unknown Source)
        ... 35 more
Caused by: java.security.cert.CertificateExpiredException: NotAfter: Thu Jun 22 06:33:51 UTC 2023
        at java.base/sun.security.x509.CertificateValidity.valid(Unknown Source)
        at java.base/sun.security.x509.X509CertImpl.checkValidity(Unknown Source)
        at java.base/sun.security.provider.certpath.BasicChecker.verifyValidity(Unknown Source)
        at java.base/sun.security.provider.certpath.BasicChecker.check(Unknown Source)
        ... 40 more

2023-07-26 13:47:26,754 INFO  [org.apa.kaf.com.net.Selector] (kafka-producer-network-thread | event-eclipse-iot-adapter-mqtt-55b6bf4878-qj7qc_f4ed6f3f7e4e_181) [Producer clientId=event-eclipse-iot-adapter-mqtt-55b6bf4878-qj7qc_f4ed6f3f7e4e_181] Failed authentication with eclipse-iot-kafka-0.eclipse-iot-kafka-headless/10.214.249.83 (channelId=-1) (SSL handshake failed)
2023-07-26 13:47:26,754 ERROR [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | event-eclipse-iot-adapter-mqtt-55b6bf4878-qj7qc_f4ed6f3f7e4e_181) [Producer clientId=event-eclipse-iot-adapter-mqtt-55b6bf4878-qj7qc_f4ed6f3f7e4e_181] Connection to node -1 (eclipse-iot-kafka-0.eclipse-iot-kafka-headless/10.214.249.83:9092) failed authentication due to: SSL handshake failed
2023-07-26 13:47:26,754 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | event-eclipse-iot-adapter-mqtt-55b6bf4878-qj7qc_f4ed6f3f7e4e_181) [Producer clientId=event-eclipse-iot-adapter-mqtt-55b6bf4878-qj7qc_f4ed6f3f7e4e_181] Bootstrap broker eclipse-iot-kafka-0.eclipse-iot-kafka-headless:9092 (id: -1 rack: null) disconnected

eclipse-iot-service-command-router-84d696fdbb-hxhzq

...
2023-07-26 13:49:21,433 ERROR [org.ecl.hon.cli.kaf.con.HonoKafkaConsumer] (vert.x-kafka-consumer-thread-1) consumer error occurred [client-id: notification-eclipse-iot-service-command-router-84d696fdbb-hxhzq_270cb8b22976_0]: org.apache.kafka.common.errors.SslAuthenticationException: SSL handshake failed
Caused by: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
        at java.base/sun.security.ssl.Alert.createSSLException(Unknown Source)
        at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
        at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
        at java.base/sun.security.ssl.TransportContext.fatal(Unknown Source)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(Unknown Source)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(Unknown Source)
        at java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(Unknown Source)
        at java.base/sun.security.ssl.SSLHandshake.consume(Unknown Source)
        at java.base/sun.security.ssl.HandshakeContext.dispatch(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(Unknown Source)
        at java.base/java.security.AccessController.doPrivileged(Unknown Source)
        at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(Unknown Source)
        at org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks(SslTransportLayer.java:435)
        at org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap(SslTransportLayer.java:523)
        at org.apache.kafka.common.network.SslTransportLayer.doHandshake(SslTransportLayer.java:373)
        at org.apache.kafka.common.network.SslTransportLayer.handshake(SslTransportLayer.java:293)
        at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:178)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:543)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:481)
        at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:560)
        at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:265)
        at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:236)
        at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:215)
        at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureCoordinatorReady(AbstractCoordinator.java:252)
        at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.coordinatorUnknownAndUnready(ConsumerCoordinator.java:493)
        at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:521)
        at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1271)
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1235)
        at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1215)
        at io.vertx.kafka.client.consumer.impl.KafkaReadStreamImpl.lambda$pollRecords$6(KafkaReadStreamImpl.java:155)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
        at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.base/java.lang.Thread.run(Unknown Source)
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed
        at java.base/sun.security.validator.PKIXValidator.doValidate(Unknown Source)
        at java.base/sun.security.validator.PKIXValidator.engineValidate(Unknown Source)
        at java.base/sun.security.validator.Validator.validate(Unknown Source)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(Unknown Source)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(Unknown Source)
        ... 32 more
Caused by: java.security.cert.CertPathValidatorException: validity check failed
        at java.base/sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(Unknown Source)
        at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(Unknown Source)
        at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(Unknown Source)
        at java.base/sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(Unknown Source)
        at java.base/java.security.cert.CertPathValidator.validate(Unknown Source)
        ... 37 more
Caused by: java.security.cert.CertificateExpiredException: NotAfter: Thu Jun 22 06:33:52 UTC 2023
        at java.base/sun.security.x509.CertificateValidity.valid(Unknown Source)
        at java.base/sun.security.x509.X509CertImpl.checkValidity(Unknown Source)
        at java.base/sun.security.provider.certpath.BasicChecker.verifyValidity(Unknown Source)
        at java.base/sun.security.provider.certpath.BasicChecker.check(Unknown Source)
        ... 42 more

2023-07-26 13:49:21,600 INFO  [org.apa.kaf.com.net.Selector] (kafka-producer-network-thread | internal-cmd-sender-eclipse-iot-service-command-router-84d696fdbb-hxhzq_270cb8b22976_101) [Producer clientId=internal-cmd-sender-eclipse-iot-service-command-router-84d696fdbb-hxhzq_270cb8b22976_101] Failed authentication with eclipse-iot-kafka-0.eclipse-iot-kafka-headless/10.214.249.83 (channelId=-1) (SSL handshake failed)
2023-07-26 13:49:21,600 ERROR [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | internal-cmd-sender-eclipse-iot-service-command-router-84d696fdbb-hxhzq_270cb8b22976_101) [Producer clientId=internal-cmd-sender-eclipse-iot-service-command-router-84d696fdbb-hxhzq_270cb8b22976_101] Connection to node -1 (eclipse-iot-kafka-0.eclipse-iot-kafka-headless/10.214.249.83:9092) failed authentication due to: SSL handshake failed
2023-07-26 13:49:21,600 WARN  [org.apa.kaf.cli.NetworkClient] (kafka-producer-network-thread | internal-cmd-sender-eclipse-iot-service-command-router-84d696fdbb-hxhzq_270cb8b22976_101) [Producer clientId=internal-cmd-sender-eclipse-iot-service-command-router-84d696fdbb-hxhzq_270cb8b22976_101] Bootstrap broker eclipse-iot-kafka-0.eclipse-iot-kafka-headless:9092 (id: -1 rack: null) disconnected

eclipse-iot-kafka-0

...
[2023-07-26 13:50:05,272] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.102 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,275] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.122 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,276] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.122 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,311] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.122 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,332] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.116 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,343] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.110 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,515] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.110 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,527] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.116 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,587] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.102 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,591] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.122 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,603] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.110 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,629] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.116 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,708] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.102 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,858] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.110 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
[2023-07-26 13:50:05,968] INFO [SocketServer listenerType=ZK_BROKER, nodeId=0] Failed authentication with /10.214.249.110 (SSL handshake failed) (org.apache.kafka.common.network.Selector)

Some notes

  • when I install only Hono, the Kafka version is 3.4.0
  • when I use the cloud2edge, the Kafka version is 3.1.1
  • I think the issue is related to some initial configuration of Kafka but I can't figure it out;

Any idea?
@calohmn calohmn self-assigned this Aug 9, 2023
@calohmn calohmn added Hono Cloud2Edge Improvements to the Cloud2Edge package labels Aug 9, 2023
@calohmn
Copy link
Contributor

calohmn commented Aug 10, 2023 •
edited
Loading

Can you check the Hono chart version that you are using for the cloud2edge chart deployment by checking the archive filename in your local packages/cloud2edge/charts directory?

The CertificateExpiredException errors in Hono pods let's me assume that you are not using the latest Hono chart. The Hono chart version 2.4.2 contains re-generated certificates (see #473).
To get the newest chart version, you can change to the packages/cloud2edge directory and do helm dependency update.

Note: There currently is an issue with the cloud2edge chart using the newest ditto chart (#482).
As a workaround, before doing the dependency update, you can change the ditto chart version to be used, as described here. (Or you can use the fix from PR #483.)
EDIT: this issue is resolved in the current cloud2edge chart version.

@calohmn
Copy link
Contributor

calohmn commented Sep 22, 2023

@paulocabrita-ionos Do you still see an issue here or can this be closed?

@paulocabrita-ionos
Copy link
Contributor Author

Going to close it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@calohmn calohmn

Labels
Cloud2Edge Improvements to the Cloud2Edge package Hono
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@calohmn @paulocabrita-ionos