Skip to content

Latest commit

 

History

History
124 lines (86 loc) · 3.81 KB

README.md

File metadata and controls

124 lines (86 loc) · 3.81 KB
Raw

Hydra Callbacks

style framework codecov CD CI Release

A collection of usefulls and simple to use callbacks for the hydra configuration framework.

Installation

pip install hydra-callbacks

Development version

pip install git+https://github.com/paquiteau/hydra-callbacks

Usage

In your hydra root config file add the following, or analoguous:

hydra:
  callbacks:
    git_infos:
      _target_: hydra_callbacks.GitInfo
      clean: true
    latest_run:
      _target_: hydra_callbacks.LatestRunLink
    resource_monitor:
      _target_: hydra_callbacks.ResourceMonitor
      sample_interval: 0.5
    runtime_perf:
      _target_: hydra_callbacks.RuntimePerformance

This will enrich your script output with:

paquiteau@laptop$ python my_app.py
[hydra] Git sha: 844b9ca1a74d8307ef5331351897cebb18f71b88, dirty: False

## All your app log and outputs ##

[hydra][INFO] - Total runtime: 0.51 seconds
[hydra][INFO] - Writing monitoring data to [...]/outputs/2023-04-06/16-02-46/resource_monitoring.csv
[hydra][INFO] - Latest run is at: [...]/outputs/latest

Detailled configuration for each callback is available in the tests/test_app/ folder.

Available Callbacks

Name Action
GitInfo Check status of Repository
LatestRunLink Get a link to the latest run
MultiRunGatherer Gather results json file in a single table
RuntimePerformance Get Execution time for each run
ResourceMonitor Monitor resources of running jobs (CPU and Memory)

And more to come !

Also Available

  • PerfLogger : A simple to use performance logger
  
from hydra_callbacks import PerfLogger 
import logging

log = logging.getLogger(__name__)
def main_app(cfg):
    with PerfLogger(log, "step1"):
        sleep(1)

    with PerfLogger(log, "step2"):
        sleep(2)
    PerfLogger.recap(log)
  • RessourceMonitorService : A simple CPU and GPU usage and memory sampler. It launches an extra process to monitor everything.
from hydra_callbacks.monitor import RessourceMonitorService
import os 

monitor = RessourceMonitorService(interval=0.2, gpu_monit=True)

monitor.start()
# Launch heavy stuff 
metrics = monitor.stop()

# Or use it as a context manager
with RessourceMonitorService(interval=0.2, gpu_monit=True) as monitor: 
   # launch heavy stuff
   
metrics_values = monitor.get_values()

You too, have cool Callbacks, or idea for one ?

Open a PR or an issue !

Possible Ideas

  • A callback that summarize log from multiple runs
  • Monitoring of GPU using nvitop

🌟 If you like this work, don't forget to star it and share it 🌟