Profiling memory and CPU usage

Installing requirements

# Install meliae
sudo apt-get install cython
wget https://launchpad.net/meliae/trunk/0.4/+download/meliae-0.4.0.tar.gz
tar -zxpvf meliae-0.4.0.tar.gz
cd meliae-0.4.0/
sudo python setup.py install

# Install yappi + RunSnakeRun
sudo pip install yappi --upgrade
sudo pip install RunSnakeRun --upgrade

Profiling a w3af run

Manual analysis:

W3AF_CPU_PROFILING=1 ./w3af_console -s scripts/test.w3af

Automated profiling information collection can be done using the collector tool.

Profiling a nosetests run

python -m cProfile -o profile.out `which nosetests` -v w3af/plugins/tests/crawl/test_phishtank.py:TestPhishtank.test_xml_parsing
runsnake profile.out

Analysis

Manual analysis:

runsnake /tmp/yappi-w3af.cpu
runsnakemem /tmp/meliae-w3af.memory

Automated analysis using the w3af-performance-analysis tool.

Related source code

The profiling instrumentation is embedded into w3af itself. It was the best way to allow me to collect this information in cases where users wanted to help me debug memory usage issues.

The source code is pretty simple to understand.