Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compatability check: Secret Provider 0.0.2 with confluent platform 7.5.3 and Java 8 #71

Open
YaminiNallappagari opened this issue Mar 26, 2024 · 3 comments

Comments

@YaminiNallappagari
Copy link

We are getting below error for the connectors using lenses.io , Post platform upgradation to 7.5.3
Earlier its good with 5.5.2 version. I want to know whether 0.0.2 is compatible with confluent platform 7.5.3 and java 8

When we try to add the lenses path to the executable file of the connect worker node as below, All the connecter will be failed state

CLASSPATH="/usr/share/java/kafka-connect-replicator/replicator-rest-extension-7.5.3.jar:/usr/share/java/secret-provider/secret-provider-0.0.2-all.jar:/usr/share/java/confluent-security/connect/kafka-client-plugins-7.5.3-ce.jar"

current Classpath path we have is:
CLASSPATH="/usr/share/java/kafka-connect-replicator/replicator-rest-extension-7.5.3.jar:/usr/share/java/confluent-security/connect/kafka-client-plugins-7.5.3-ce.jar"

Below is the status of one of the failed connectors which is using lenses io:

{"name":"EMPSTG.stg.transportation.train-reservation.raw.v1.Replicator","connector":{"state":"FAILED","worker_id":"zcc1kfconvml01d.cn.ca:8083","trace":"java.lang.NoClassDefFoundError: Could not initialize class com.azure.core.implementation.http.HttpClientProviders\n\tat com.azure.core.http.HttpClient.createDefault(HttpClient.java:27)\n\tat com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:60)\n\tat com.azure.security.keyvault.secrets.SecretClientBuilder.buildAsyncClient(SecretClientBuilder.java:162)\n\tat com.azure.security.keyvault.secrets.SecretClientBuilder.buildClient(SecretClientBuilder.java:104)\n\tat io.lenses.connect.secrets.providers.AzureSecretProvider.$anonfun$get$1(AzureSecretProvider.scala:61)\n\tat scala.collection.MapLike.getOrElse(MapLike.scala:131)\n\tat scala.collection.MapLike.getOrElse$(MapLike.scala:129)\n\tat scala.collection.AbstractMap.getOrElse(Map.scala:63)\n\tat io.lenses.connect.secrets.providers.AzureSecretProvider.get(AzureSecretProvider.scala:62)\n\tat org.apache.kafka.common.config.ConfigTransformer.transform(ConfigTransformer.java:103)\n\tat org.apache.kafka.connect.runtime.WorkerConfigTransformer.transform(WorkerConfigTransformer.java:58)\n\tat org.apache.kafka.connect.storage.ClusterConfigState.connectorConfig(ClusterConfigState.java:152)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.startConnector(DistributedHerder.java:1894)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$getConnectorStartingCallable$38(DistributedHerder.java:1926)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:750)\n"},"tasks":[{"id":0,"state":"FAILED","worker_id":"zcc1kfconvml01d.cn.ca:8083","trace":"java.lang.NoClassDefFoundError: Could not initialize class com.azure.core.implementation.http.HttpClientProviders\n\tat com.azure.core.http.HttpClient.createDefault(HttpClient.java:27)\n\tat com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:60)\n\tat com.azure.security.keyvault.secrets.SecretClientBuilder.buildAsyncClient(SecretClientBuilder.java:162)\n\tat com.azure.security.keyvault.secrets.SecretClientBuilder.buildClient(SecretClientBuilder.java:104)\n\tat io.lenses.connect.secrets.providers.AzureSecretProvider.$anonfun$get$1(AzureSecretProvider.scala:61)\n\tat scala.collection.MapLike.getOrElse(MapLike.scala:131)\n\tat scala.collection.MapLike.getOrElse$(MapLike.scala:129)\n\tat scala.collection.AbstractMap.getOrElse(Map.scala:63)\n\tat io.lenses.connect.secrets.providers.AzureSecretProvider.get(AzureSecretProvider.scala:62)\n\tat org.apache.kafka.common.config.ConfigTransformer.transform(ConfigTransformer.java:103)\n\tat org.apache.kafka.connect.runtime.WorkerConfigTransformer.transform(WorkerConfigTransformer.java:58)\n\tat org.apache.kafka.connect.storage.ClusterConfigState.connectorConfig(ClusterConfigState.java:152)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:1819)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$getTaskStartingCallable$33(DistributedHerder.java:1872)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:750)\n"},{"id":1,"state":"FAILED","worker_id":"zcc1kfconvml01d.cn.ca:8083","trace":"java.lang.NoClassDefFoundError: Could not initialize class com.azure.core.implementation.http.HttpClientProviders\n\tat com.azure.core.http.HttpClient.createDefault(HttpClient.java:27)\n\tat com.azure.core.http.HttpPipelineBuilder.build(HttpPipelineBuilder.java:60)\n\tat com.azure.security.keyvault.secrets.SecretClientBuilder.buildAsyncClient(SecretClientBuilder.java:162)\n\tat com.azure.security.keyvault.secrets.SecretClientBuilder.buildClient(SecretClientBuilder.java:104)\n\tat io.lenses.connect.secrets.providers.AzureSecretProvider.$anonfun$get$1(AzureSecretProvider.scala:61)\n\tat scala.collection.MapLike.getOrElse(MapLike.scala:131)\n\tat scala.collection.MapLike.getOrElse$(MapLike.scala:129)\n\tat scala.collection.AbstractMap.getOrElse(Map.scala:63)\n\tat io.lenses.connect.secrets.providers.AzureSecretProvider.get(AzureSecretProvider.scala:62)\n\tat org.apache.kafka.common.config.ConfigTransformer.transform(ConfigTransformer.java:103)\n\tat org.apache.kafka.connect.runtime.WorkerConfigTransformer.transform(WorkerConfigTransformer.java:58)\n\tat org.apache.kafka.connect.storage.ClusterConfigState.connectorConfig(ClusterConfigState.java:152)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:1819)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$getTaskStartingCallable$33(DistributedHerder.java:1872)\n\tat java.util.concurrent.FutureTask.run(FutureTask.java:266)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\tat java.lang.Thread.run(Thread.java:750)\n"}],"type":"source"}

@stheppi
Copy link
Contributor

stheppi commented Mar 26, 2024

Is there a specific rationale behind your decision to use version 0.0.2, which was released in 2020, rather than opting for the latest release? The exception appears to be originating from the Azure library. It's possible that Confluent version 7.5.3 includes certain library versions that are incompatible with the version of Azure being utilized.

@YaminiNallappagari
Copy link
Author

We have been using 0.0.2 version in 5.5.2 confluent version, but it's not working post platform upgradation to 7.5.3. Can you please let us know if our classpath setting looks good . and will it work if we install latest secret-provider version with current platform 7.5.3?

@YaminiNallappagari
Copy link
Author

And can you suggest , like I see the latest lenses version is 2.3.0, will it be compatible with confluent platform version of 7.5.3 and java 8? and any dependency on other jars and their versions of confluent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants