VM monitoring

5GTANGO/SONATA Metrics Exporter uses the pipeline mechanism provided by openstack ceilometer in order to collect monitoring data for all the VMs inside openstack NFVI and expose them to the Prometheus.io server. The main advantage of this approach is that all VNFs that are hosted in VMs can be monitored without the need of any additional software or configuration. The supported monitoring metrics are defined in ceilometer side:

Dependencies

• python
• configparser
• msgpack
• prometheus-client
• setuptools

Installation

a. Build container

docker build -t son-monitor-ceilexp

b. Run as container

docker run -d --name son-monitor-ceilExporter -p 10000:10000/udp -p 9092:9091 son-monitor-ceilexp

Usage

Monitoring data from Openstack is transmited to metric exporter using the pipeline feature of the ceilometer. In order to enable the pipeline service and define the udp socket you must make the following congiguration in ceilometer side.

a. Set data collecting time interval and the metrics Open the polling.yaml file and make the apropiated configurations

---
sources:
    - name: some_pollsters
      interval: 10
      meters:
        - cpu
        - memory.usage
        - network.incoming.bytes
        - network.incoming.packets
        - network.outgoing.bytes
        - network.outgoing.packets
        - disk.read.bytes
        - disk.read.requests
        - disk.write.bytes
        - disk.write.requests
        - hardware.cpu.util
        - hardware.memory.used
        - hardware.memory.total
        - hardware.memory.buffer
        - hardware.memory.cached
        - hardware.memory.swap.avail
        - hardware.memory.swap.total
        - hardware.system_stats.io.outgoing.blocks
        - hardware.system_stats.io.incoming.blocks
        - hardware.network.ip.incoming.datagrams
        - hardware.network.ip.outgoing.datagrams

b. Set the udp socket in pipeline.yaml

sources:
    - name: meter_source
      meters:
          - "*"
      sinks:
          - meter_sink
    - name: cpu_source
      meters:
          - "cpu"
      sinks:
          - cpu_sink
          - cpu_delta_sink
    - name: disk_source
      meters:
          - "disk.read.bytes"
          - "disk.read.requests"
          - "disk.write.bytes"
          - "disk.write.requests"
          - "disk.device.read.bytes"
          - "disk.device.read.requests"
          - "disk.device.write.bytes"
          - "disk.device.write.requests"
      sinks:
          - disk_sink
    - name: network_source
      meters:
          - "network.incoming.bytes"
          - "network.incoming.packets"
          - "network.outgoing.bytes"
          - "network.outgoing.packets"
      sinks:
          - network_sink
sinks:
    - name: meter_sink
      transformers:
      publishers:
          - gnocchi://10.102.2.240:8041
          - udp://10.102.2.240:10000/
    - name: cpu_sink
      transformers:
          - name: "rate_of_change"
            parameters:
                target:
                    name: "cpu_util"
                    unit: "%"
                    type: "gauge"
                    max: 100
                    scale: "100.0 / (10**9 * (resource_metadata.cpu_number or 1))"
      publishers:
          - gnocchi://10.102.2.240:8041
          - udp://10.102.2.240:10000/
    - name: cpu_delta_sink
      transformers:
          - name: "delta"
            parameters:
                target:
                    name: "cpu.delta"
                growth_only: True
      publishers:
          - gnocchi://10.102.2.240:8041
          - udp://10.102.2.240:10000/
    - name: disk_sink
      transformers:
          - name: "rate_of_change"
            parameters:
                source:
                    map_from:
                        name: "(disk\\.device|disk)\\.(read|write)\\.(bytes|requests)"
                        unit: "(B|request)"
                target:
                    map_to:
                        name: "\\1.\\2.\\3.rate"
                        unit: "\\1/s"
                    type: "gauge"
      publishers:
          - gnocchi://10.102.2.240:8041
          - udp://10.102.2.240:10000/
    - name: network_sink
      transformers:
          - name: "rate_of_change"
            parameters:
                source:
                   map_from:
                       name: "network\\.(incoming|outgoing)\\.(bytes|packets)"
                       unit: "(B|packet)"
                target:
                    map_to:
                        name: "network.\\1.\\2.rate"
                        unit: "\\1/s"
                    type: "gauge"
      publishers:
          - gnocchi://10.102.2.240:8041
          - udp://10.102.2.240:10000/

c. Enable the pipeline feature by adding the following lines in ceilometer.cong

[DEFAULT]
# Configuration file for pipeline definition. (string value)
pipeline_cfg_file = pipeline.yaml

[polling]

# 
# From ceilometer
# 

# Configuration file for pipeline definition. (string value)
cfg_file = polling.yaml

d. Restart openstack ceilometer services

service ceilometer-agent-central restart 
service ceilometer-agent-notification restart 
service ceilometer-api restart
service ceilometer-agent-compute
service ceilometer-collector restart