Green AI torch tries to create awareness over energy consumption within the Pytorch ML framework. Its goal is to measure energy consumption throughout the complete process of AI engineers and can give overviews and indications for performance gains with respect to energy consumption.
Currently it supports energy measurement of the training passes per layer.
You can find the full API documentation here.
You can install GATorch using pip. To install GATorch, at the command line, run:
pip install GATorch- cuda >=11.7
- cuDNN >=8
Create a virtualenv like virtualenv .venv and activate it using source ./.venv/bin/activate and install the other requirements with pip install requirements.txt.
from GA import GA
# Create the profiler object and attach a model to it
ga_measure = GA()
ga_measure.attach_model(model)
# Let's try to do a single forward pass and a backward pass
x = torch.zeros([1, 1, 28, 28]).to(device)
y = torch.zeros([1, 10]).to(device)
pred = model(x) # forward
loss = loss_fn(pred, y)
optimizer.zero_grad()
loss.backward() # backward
optimizer.step()
# Now lets print the mean measurements
print(ga_measure.get_mean_measurements())or run the example scripts in the examples directory.
Some older hardware might not support energy consumption measurements:
- NVML requires Tesla Architecture NVIDIA GPUs or newer to work.
- RAPLs DRAM measurements are only available for XENON CPUs.
In case you get compatibility errors due to older hardware you can disable the failing measurement application, use the disable_measurements parameter in the attach_model function. This parameter accepts a list of disabled measurements out of ['cpu', 'ram', 'gpu'], default is []. You need to use at least one measurement that is not disabled. The program will indicate that the disabled devices are unavailable.
Due to Platypus attack Intel RAPL requires root permission for energy readings. In order to run this program with the correct permissions, do NOT make Intel RAPL readable for any user as this introduces vulnerability. Instead use Python with sudo instead:
sudo ./.venv/bin/python <script_name>.pyThis tool can automatically generate energy consumption reports and display these in Tensorboard. To use Tensorboard run tensorboard --logdir=runs and open the browser to view the graphs. This tool further allows for custom graph generation and tensorboard integration, but is not complete and needs to be extended.
The current architecture of this tool uses the integrated hooks of the PyTorch library, which restricts the current implementation towards the final goal of complete coverage including data loading, pre-processing, saving and loading a model etc. To give a more thorough analysis of the impact of energy consumption in ML development, this still needs to be developed.
This tool differs from other tools by measuring in-depth layers and system components and could be expanded to provide energy consumption data that can lead to recommendations for eliminating certain layers due to high energy consumption compared to accuracy gain.
PyJoules measures the energy consumption per individual hardware components and this data could be separated in order to provide a relative component view. Another improvement could be to measure the system component utilization over time, which can be an indicator of wasted energy.