snmpd: ignore Docker network interfaces #170
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
When setting up ft04 I noticed that Influx doesn't show stats for the ft04_v* interfaces on the vm-host.
In the log I then found:
This is due to Docker containers creating new veth interfaces with different names whenever containers are recreated, and new br interfaces when networks are recreated. Especially the amount of different veth interfaces (i.e. the cardinality of the Influx measurement) over time is huge, it makes up around 95% of the interfaces Influx (and LibreNMS).
As a first countermeasure I removed all series from the
ports
measurement where theifName
starts withveth
before 2024-06-30, and lowered the cardinality from just below the 1005 to ~8k (SHOW TAG VALUES CARDINALITY WITH KEY = "ifName"
).A few minutes afterwards the two ft04_v* interfaces started to show up in Influx.
Solution
Let's exclude the Docker
br-
andveth
interfaces from SNMP measurements, thus not sending their data to LibreNMS, thus not sending the data to Influx.I chose this over increasing the max-values-per-tag-limit, because I think we don't have much use for historical Docker interface stats (especially as you can't really match them to the respective container anymore), and InfluxDB (and LibreNMS) should also be very happy about a much reduced load/database size.
The filter for the
br-
interfaces checks whether their MAC address starts with02:42
, which appears to be the prefix that Docker uses, as not to match our bridges on the gateways.The second line regarding
db_instance=ffmuc_other measurement=net
comes from Telegraf collecting network stats on the VMs directly. The filtering there (if possible) is still TODO, in a separate PR.