Debugging perfSONAR

MaDDash debugging

Debugging the maddash-server part of MaDDash usually involves running a Java Management Extensions (JMX) agent to connect to the running process. This then gives you the possibility to have a real-time monitoring of the CPU and memory usage, running threads and detailed informations about some loaded libraries coming from probes (MBean).

The most common tool to do that is JConsole, but more sofisticated ones like VisualVM exists.

If you can run JConsole directly on the MaDDash server, i.e. you have access to a GUI on that machine, then it's the easiest. You can also use a JConsole on your own development environment and connect to the remote JMX agent, but you'll need to have no firewall in between.

Otherwise, you can also tunnel the JMX protocol through an ssh connection. But you'll need to take care of a few different settings. The following worked fine for me:

• Tunneling a local port to the remote host, with ssh -A -L 8484:127.0.0.1:8484 maddash.host.to.be.monitored
• Enabling JMX on MaDDash, edit /usr/lib/maddash/maddash-server/bin/maddash-server.env and add a line at the end like:
  • JAVA_OPTS="$JAVA_OPT -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.port=8484 -Dcom.sun.management.jmxremote.rmi.port=8484 -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=127.0.0.1"
  • Both the jmxremote.port and the jmxremote.rmi.port need to be set, when not set, the RMI port is allocated dynamically and can be blocked by the firewall. The rmi.server.hostname property need to be set to an IP address or JConsole need to access the JMX agent using the same hostname as known by JMX (can be localhost or localhost.localdomain or something else), otherwise the connection will be refused.
• Use JConsole on your development host (from where you run your ssh client) and use 127.0.0.1:8484 to connect to the remote JMX agent.