Gelf output with multiple hosts for "silent" fail over

[jordansissel]

(This issue was originally filed by @rlanore at elastic/logstash#2460)

---

Given a cluster of gelf nodes, it would be desirable that LS silently fail over in case a gelf node, pointed by LS, dies.

[gergelybosze]

Hi,
I see this issue is still open, it means this logstash gelf output failover feature is not implemented yet?
Is there a plan for it?
Do we have to think about another loadbalancer solution?
Thanks!

[pastukhov]

Any news?

[afreiberger]

I also desire this feature.

While investigating the implementation, I've found the following:

The GELF output plugin calls the GELF::Notifier initializer with a single host and port argument.

https://github.com/logstash-plugins/logstash-output-gelf/blob/main/lib/logstash/outputs/gelf.rb#L86

There is a limitation of the GELF::Notifier module that assumes a single back-end host both in the initiator method as well as the call to the transport layer in the create_sender method.
https://github.com/graylog-labs/gelf-rb/blob/master/lib/gelf/notifier.rb#L25
https://github.com/graylog-labs/gelf-rb/blob/master/lib/gelf/notifier.rb#L137

Looking at the transport modules for TCP, TCP-TLS, and UDP, they all implement an idea of rotation through multiple sockets until a message is sent validly if multiple addresses are provided, but the create_sender method in the GELF::Notifier limits this to a single host and port at: https://github.com/graylog-labs/gelf-rb/blob/master/lib/gelf/notifier.rb#L138.

To resolve this, the upstream GELF::Notifier library would need an update to handle multiple hosts during instantiation and then this logstash plugin would need to have an expanded definition in it's schema and routine that instantiates the GELF::Notifier.