Kakfa Producer as Data sink

[iamaccosta]

Hi Team,

Thank you for the excellent work on Databridge!

I wanted to ask if it's possible to publish data to a Kafka topic using the Databridge component. The examples provided only showcase Kafka as a source through kafkaconsumer.json.

If this functionality isn't currently available, do you have any plans to introduce it in the future?

Thank you in advance for your help!

[aaronzi]

Hi @iamaccosta,

we are happy you like the Databridge.
Kafka is currently only available as a data consumer.

A Kafka producer isn't on our roadmap right now. But if you like, you can implement this functionality yourself. A good starting point for this is the kafka consumer and the mqtt producer as a blueprint to how producers are implemented.

[iamaccosta]

Sorry to bother...

I tried to implement it by myself. I added the required files following other components that already support bidirectional communication and used similar config parameters as Kafka Consumer. I took consideration the abstract classes and got the Build Success.

When testing, I provided all the configuration parameters as I defined and I got the following error:

2024-10-29 16:18:08 java.lang.NullPointerException
2024-10-29 16:18:08     at org.eclipse.digitaltwin.basyx.databridge.core.configuration.route.core.RouteCreatorHelper.getDataSinkEndpoints(RouteCreatorHelper.java:45)
2024-10-29 16:18:08     at org.eclipse.digitaltwin.basyx.databridge.core.configuration.route.core.AbstractRouteCreator.addRouteToRouteBuilder(AbstractRouteCreator.java:55)
2024-10-29 16:18:08     at org.eclipse.digitaltwin.basyx.databridge.core.routebuilder.DataBridgeRouteBuilder.configure(DataBridgeRouteBuilder.java:58)
2024-10-29 16:18:08     at org.apache.camel.builder.RouteBuilder.checkInitialized(RouteBuilder.java:726)
2024-10-29 16:18:08     at org.apache.camel.builder.RouteBuilder.configureRoutes(RouteBuilder.java:668)
2024-10-29 16:18:08     at org.apache.camel.builder.RouteBuilder.addRoutesToCamelContext(RouteBuilder.java:600)
2024-10-29 16:18:08     at org.apache.camel.impl.engine.AbstractCamelContext.addRoutes(AbstractCamelContext.java:1213)
2024-10-29 16:18:08     at org.eclipse.digitaltwin.basyx.databridge.core.component.DataBridgeComponent.startRoutes(DataBridgeComponent.java:86)
2024-10-29 16:18:08     at org.eclipse.digitaltwin.basyx.databridge.core.component.DataBridgeComponent.startComponent(DataBridgeComponent.java:80)
2024-10-29 16:18:08     at org.eclipse.digitaltwin.basyx.databridge.component.DataBridgeExecutable.main(DataBridgeExecutable.java:48)

Does anyone know how to resolve this error?

[aaronzi]

Hi @iamaccosta,

Would you be willing to contribute your implementation of the Kafka producer? If so, please create a pull request so we can take a look at your code and try to help you find the possible cause of the error you are experiencing.

[iamaccosta]

Hi @iamaccosta,

Would you be willing to contribute your implementation of the Kafka producer? If so, please create a pull request so we can take a look at your code and try to help you find the possible cause of the error you are experiencing.

Done. PR submitted.

[aaronzi]

Thank you for creating the PR. We will have a look at this (probably next week) and come back to you.

PS: please sign the Eclipse contributor agreement. If you don't already have an eclipse account, you will need to create one. You can also link the account to your GitHub account. Please make sure that the email that you used to sign your commits is the same mail that you use for your eclipse account.