Unix poller is unable to monitor pollers when running via docker compose

[cgrinds]

Example

docker ps -a
CONTAINER ID   IMAGE                         COMMAND                  CREATED         STATUS           PORTS                      NAMES
9fd698bf1a07   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13011->13011/tcp   poller-g2
17b5d3a329a3   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:12997->12997/tcp   poller-aff-251
b64804a02d64   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13012->13012/tcp   poller-stack1
1a925223c342   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13017->13017/tcp   poller-stack4
caf97f4b3fee   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 40 weeks     0.0.0.0:13002->13002/tcp   poller-v-7
3fde4c85fa16   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:12999->12999/tcp   poller-aff-251-rest
6a88f29acee4   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13022->13022/tcp   poller-aff400
82c53ab570e2   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13019->13019/tcp   poller-fas2650
91ce0b597d40   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:12993->12993/tcp   poller-cpoc
f7198c3d5d7f   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 80 weeks     0.0.0.0:13021->13021/tcp   poller-play
ac99a6cb2945   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:39000->39000/tcp   poller-simple1
2dbebe63b10f   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13001->13001/tcp   poller-sar
70a85c3a5305   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13000->13000/tcp   poller-fas8060
1ea980864fb9   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13020->13020/tcp   poller-stack7
96dc1331c5be   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:12998->12998/tcp   poller-aff-251-zapi
a042f0b43887   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:12995->12995/tcp   poller-f2240-127-26
fe2dbcc6d4fe   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:12996->12996/tcp   poller-aff-250
ebcbcd7fef57   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13018->13018/tcp   poller-a220
45613f5e2b4f   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13010->13010/tcp   poller-g1
5750328ed164   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 20 weeks     0.0.0.0:12994->12994/tcp   poller-fsx
2fed1d14fdda   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13004->13004/tcp   poller-unix
0e9029025f6f   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13005->13005/tcp   poller-v-72
63747ee805db   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13015->13015/tcp   poller-cie
0cfb4add30e2   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13006->13006/tcp   poller-pauls
0dfbb6be0dde   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 20 weeks     0.0.0.0:13016->13016/tcp   poller-isscale
6ac9a12ce8c5   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13013->13013/tcp   poller-aaron
1c373e80de4b   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:12992->12992/tcp   poller-jeff
60fc352aebfb   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13014->13014/tcp   poller-stack3
805d671b8a5b   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:13023->13023/tcp   poller-yann
a30ec03a3182   cr.netapp.io/harvest:latest   "bin/poller --poller…"   31 weeks ago   Up 30 weeks     0.0.0.0:12990->12990/tcp   poller-u2
14c5255c68f9   grafana/grafana:8.3.4         "/run.sh"                31 weeks ago   Up 30 weeks     0.0.0.0:3000->3000/tcp     grafana
3d28c06e3472   prom/prometheus:v2.33.1       "/bin/prometheus --c…"   51 weeks ago   Up 50 weeks                                prometheus

docker logs -f poller-unix

2023-02-23T19:50:23Z INF collector/collector.go:483 > Collected Poller=unix apiMs=0 calcMs=0 collector=Unix:poller instances=0 metrics=0 parseMs=0 pluginMs=0
2023-02-23T19:50:33Z INF collector/collector.go:483 > Collected Poller=unix apiMs=0 calcMs=0 collector=Unix:poller instances=0 metrics=0 parseMs=0 pluginMs=0
2023-02-23T19:50:43Z INF collector/collector.go:483 > Collected Poller=unix apiMs=0 calcMs=0 collector=Unix:poller instances=0 metrics=0 parseMs=0 pluginMs=0

Note that the number of instances and metrics are zero.

curl -s 'http://localhost:13004/metrics' | rg -o 'poller="\w+"' | sort | uniq -c
  1814398 poller="unix"

[cgrinds]

This happens because the unix poller uses the /proc filesystem to gather metrics and when the unix poller is running in a container, it is unable to see the /proc file system of the other containers.

A potential workaround is to use privileged option in your docker-compose file.

[cgrinds]

For 23.08:

• investigate if privileged will solve this issue and if so, fix it
• otherwise close as wontfix

[rahulguptajss]

verified metadata dashboard is loading with docker workflow.