Add support for MSK IAM authentication in rdkafka2 output type

[madebydna]

Is your feature request related to a problem? Please describe.

One of the supported authentication & authorization methods for AWS MSK is IAM based authentication with OAUTHBEARER sending in a signed token. This is currently not supported by the plugin so there is no way to authenticate FluentD to send messages to an MSK cluster that only allows that method of authn/z.

Previously, a similar suggestion was made but in the context of the ruby-kafka library which is deprecated. This feature could, however, be supported using the rdkafka2 output type.

Describe the solution you'd like

I created a fork that adds basic support for MSK IAM authentication with the help of the aws-msk-iam-sasl-signer-ruby library. I essentially adapted the example code from the signer library for the token generation when using rdkafka as the underlying Kafka library.

However, I'm not sure if this approach is solid enough, so I was wondering if someone could rework the code (or suggest improvements) so that it could be merged into the official plugin. For instance, my solution requires using the shared producer (config share_producer true) which is not the default. I wasn't able to use the multi-threaded producer feature as it led to a ThreadError due to recursive locking.

Describe alternatives you've considered

Since I need this feature for use in our production environment, I'm planning on using my own fork for the time being. However, I would be more comfortable if this feature was officially supported and did not require using a modified fork of this plugin.

Additional context

No response

[Watson1978]

#521 is the PR for this issue?

[daipom]

If you could leave it open, it would be easier for someone familiar with it to advise you.

[madebydna]

Yes, that is the PR. I actually opened it accidentally, as I wanted to ask general advice beforehand, but I can certainly reopen it again if that would make things easier.