Java cannot find certification path

[GergelyKalmar]

TL;DR

It seems that we get a certification path error when we are using workload identity federation in combination with gcs-connector (see issue GoogleCloudDataproc/hadoop-connectors#1106 and actions/runner-images#9354):

Caused by: java.io.IOException: Error getting subject token from metadata server: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target

For the full stacktrace see https://github.com/logikal-io/mindlab/actions/runs/7940823201/job/21853958926.

Expected behavior

We were expecting our test suite to succeed when using workload identity federation.

Observed behavior

Our test suite fails when using workload identity federation. It works when using other auth methods.

Action YAML

See https://github.com/logikal-io/mindlab/blob/update-dependencies/.github/workflows/test-pull-request.yml

Log output

No response

Additional information

No response

[github-actions]

Hi there @GergelyKalmar 👋!

Thank you for opening an issue. Our team will triage this as soon as we can. Please take a moment to review the troubleshooting steps which lists common error messages and their resolution steps.

[sethvargo]

Hi @GergelyKalmar - thank you for opening an issue. It looks like the Hadoop connector does not support Workload Identity Federation. If it's using the standard Google Java SDK, it may require a dependency update. Unfortunately there's nothing we can do in this "auth" action to fix that. Upstream clients must add support for Workload Identity.

You could try using the less-secure Service Account Key Export, which will probably work.

[GergelyKalmar]

I don't think that's quite right, it is supposed to have support since version 3.0, it was one of the new features that was introduced. See GoogleCloudDataproc/hadoop-connectors#671 and also https://github.com/GoogleCloudDataproc/hadoop-connectors/releases/tag/v3.0.0, point 25: "Add support for WORKLOAD_IDENTITY_FEDERATION_CREDENTIAL_CONFIG_FILE authentication type that retrieves a refresh token using workload identity federation configuraiton defined in: fs.gs.auth.workload.identity.federation.credential.config.file".

I am really not sure if the issue is with gcs-connector at this point. Of course, it might be that this feature is broken, or we are using it wrong, but it also seems like the issue is related to the environment that we are using instead (given it complains about this certificate path problem only when using workload identity federation).

[sethvargo]

WORKLOAD_IDENTITY_FEDERATION_CREDENTIAL_CONFIG_FILE is not a standard envvar, so I'm not sure what's expected by that. The project would need to properly handle GOOGLE_APPLICATION_CREDENTIALS pointing to a WIF file (instead of a Service Account Key).

[elvin-sadigov-db]

Hi @sethvargo, Sorry I missed above conversation.
I went through the source code. There is a unit test which reads WIF file, but as you mentioned how the code handle the credential json file, that is the question.
https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v3.0.0/util-hadoop/src/test/java/com/google/cloud/hadoop/util/HadoopCredentialsConfigurationTest.java#L174

They use below google auth versions and had a release for 3.0.0 version: https://github.com/GoogleCloudDataproc/hadoop-connectors/releases/tag/v3.0.0

I believe google auth 1.14.0 supports WIF.
https://github.com/GoogleCloudDataproc/hadoop-connectors/blob/v3.0.0/pom.xml

@GergelyKalmar FYI, thanks!