Extend EC measuring approach

[rvandernoort]

Summary

Currently, this work supports measuring the energy consumption of the forward and backward passes of the model for each layer and the loss function. To get a complete overview of the energy usage for ML development, we recommend for future work that this tool is extended such that it includes measurements for data loading, pre-processing, loading and saving the model, optimizers etc. The current implementation of PyTorch 2.0.0 allowed for the implementation of the current method using hooks. The lack of hooks in PyTorch for the other functionalities blocks the implementation of the other functionalities.

Roadmap

• Introduce hooks for data loaders, preprocessors, loading and saving the model, and optimizers in PyTorch
• Introduce measurements of these hooks in GATorch
• Implement reporting of the measurements in a seamless and understandable way.