Output Plugin with virtual socket support

[ke55ler]

Feature Request

Adds an output plugin that allows telegraf (running in a guest environment) to write to a virtual socket on the host. The benefit compared to the socket_writer plugin is, that the no network configuration is required.
I reused a lot of codelines from the original socket_writer package. May one could also extend this package instead of writing a new one.

I am still trying to test the vsocket_writer on my local machine. But running into some problems.

• Signed CLA.
• Associated README.md updated.
• Has appropriate unit tests.

[glinton]

It sounds like you are working on implementing this, would you like me to assign you to this so the work isn't doubled?

[ke55ler]

Yes. I've already worked on the code. My problem is to test it. When I run make in $GOPATH/src/github.com/influxdata/telegraf, I dont get any executable in $GOPATH/bin
Who can I contact to get some help with that? I watched also this vidoe from your hompage:

write-your-own-telegraf-plugin

[glinton]

To get it installed into your gopath, use make go-install. make telegraf produces a telegraf binary in the current directory that you can invoke by referencing directly ./telegraf --config ... if you are inside the directory still or $GOPATH/src/github.com/influxdata/telegraf/telegraf elsewhere.

[ke55ler]

./telegraf --config ... this one worked quite well! Thanks.
The code is running. And I get the metrics with stdout on my host, when I am listening on the virtual socket (see picture).

Is there a way to collect this metrics with influxdb? I would like to monitore them. What would approach would you take?

Thanks for your support!

[glinton]

Is there a way to collect this metrics with influxdb?

Sure, you add an influxdb output in your config file.

[ke55ler]

May I should explain my setup. Telegraf is running in a virtual machine, but the influxdb is placed on the host. To send the metrics from the guest environment to the host I don't want to use networking, because it has to be configured on the guest and host and virtual sockets works on the fly.
So when I get it right the influxdb plugin only supports:
urls = ["unix:///var/run/influxdb.sock"]
urls = ["udp://127.0.0.1:8089"]
urls = ["http://127.0.0.1:8086"]

The virtual socket is identified by a CID and a port.

Am I on the wrong way? Hope you could understand my problem.

[danielnelson]

I think you will need Telegraf running a socket_listener input with AF_VSOCK support on the host, and it would write to InfluxDB via the regular influxdb output plugin.

[ke55ler]

Yes that sounds good. So my colleague and me had an idea, we want that multiple VM of the same tenant can write their metrics to the same influxdb. To distinguish from which VM the metric comes I want to append a tag with the vm`s Cid to each metric. I want to do this on the telegraf (forwarder) on the host side because of verification reasons.
I am going set up a input plugin for telegraf which adds the cid tags to each metric message from the guest. And then going to forward it with the regular influxdb output plugin.
Does that sounds plausible for you? What are you thoughts on that?

The approach to enriche the metrics with the cid tag would rudimentary be like, having a vscokets_listener that gets the messages. Parse the messages back to metrics []telegraf.Metric. Then use func (m *metric) AddTag(key, value string) from metric.go.

[danielnelson]

Sounds good, yes AddTag is the way to go.

Are you using the github.com/mdlayher/vsock package? If so, I think it may be possible to get the guests CID from the host using conn.RemoteAddr(). Assuming this works you could have an option on the input plugin like:

[[inputs.socket_listener]]
  ## When set a tag will be created with the specified name and set to the string
  ## representation of the remote CID.
  remote_cid_tag = ""

This would allow a user to choose a tag for the CID or ignore it as desired. We probably want to add something similar in the future to socket_listener for the source aka remote host.

I'd like the vsock support added in the socket_writer and socket_listener directly, but you can continue to work on them separately and we can figure out how to combine them later once we have the proof of concept code working.

[ke55ler]

Exactly I am using that.
conn.RemoteAddr() ... I will try this out.

I'd like the vsock support added in the socket_writer and socket_listener directly

... sounds good. I just wrote it into an own package actually, as I thought it might be easier for testing and experimenting with it for the beginning. But yeah, it makes more sense in the end to add it to the socekt_writer / socket_listener
I keep you updated and will post here when I did some progress on hte socket_listener.

[ke55ler]

Am I right when I think that I've got to use a service_input plugin? Or is it possible with a normal input plugin as well? The service_input plugin seemes to be very complicated. Actually I don't understand how the socker_listener stops, which I have taken as a pattern. I can't stop the process with ctrl + c in the terminal and when I close the window of the process in the terminal, the port stays open and when I try to reuse it i get address already in use.

So how can I call the Stop() function?

[danielnelson]

Yes you would need to use the ServiceInput interface, it is a bit more complicated. If your Stop() function is not being called when you interrupt Telegraf then most likely your plugin doesn't match the interface exactly.

You could use an "interface assertion" and the compiler will tell you why the type doesn't match the interface.

var _ telegraf.ServiceInput = (*MyPlugin)(nil)

[reimda]

Hi @alrob9894 were you able to find a solution to the problem? Did you end up modifying an existing telegraf plugin to get what you need?

[ke55ler]

Hi @alrob9894 were you able to find a solution to the problem? Did you end up modifying an existing telegraf plugin to get what you need?

Yes I found a solution, I just extended the socket plugin for virtual sockets, not that big deal. I just forgot to write some tests for it, since I was busy with my CS master.
I could finish this in the next few weeks.

[reimda]

If you would submit a PR with your changes we'd be happy to review it. Thanks!

[ke55ler]

Do you want to have this beeing integrated to the socket_writer/socket_listener plugins or as a standalone plugin?

[powersj]

Hi,

Do you want to have this beeing integrated to the socket_writer/socket_listener plugins or as a standalone plugin?

Skimming quickly through this feature request, it sounds like this would be good to the extend to the existing socket_listener input plugin and socket_writer output plugin.

[ke55ler]

Hi, quick question regarding how you want the URL for the vsock connection to be represented in the sample.conf, my suggestion would be this:

#address = "vsock://cid:port"

as it follows the represenation of TCP, UDP and so on.

#address = "tcp://127.0.0.1:8094"
#address = "udp://127.0.0.1:8094"

[powersj]

@alrob9894 I think that is worth putting up a PR with. Thanks!

[jeffrey4l]

hey guys, i add a patch to support vsock in prometheus_client output plugin #14108

[powersj]

Going to close this now as both #14108 (prometheus_client) and #14172 (socket listener) PRs have landed.