Skip to content

Commit

Permalink
Create RFC for a logging system and list some required features
Browse files Browse the repository at this point in the history
  • Loading branch information
kurtamohler committed Jun 23, 2022
1 parent b379b19 commit ff90402
Showing 1 changed file with 103 additions and 0 deletions.
103 changes: 103 additions & 0 deletions RFC-0026-logging-system.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# PyTorch Logging System

## **Summary**
Create a message logging system for PyTorch with the following requirements:

* All errors, warnings, and other messages generated by PyTorch should be
emitted using the the logging system API

* The APIs for emitting messages and changing settings should all be consistent
between C++ and Python

* Offer different message severity levels, including at least the following:

- **Info**: Emits a message without creating a warning or error. By default,
this gets printed to stdout

- **Warning**: Emits a message as a warning. By default, this will turn into
a Python warning

- **Error**: Emits a message as an error. By default, this will turn into
a Python error

- TODO: Should we also have a **Fatal** severity for integration with
Meta's internal logging system? A fatal message terminates the program

* Offer different classes of messages, including at least the following:

- **Default**: A catch-all message class

- **Nondeterministic**: Emitted when `torch.use_deterministic_algorithms(True)`
is set and a nondeterministic operation is called

- **Deprecated**: Emitted when a deprecated function is called

- **Beta**: Emitted when a beta feature is called. See
[PyTorch feature classifications](https://pytorch.org/blog/pytorch-feature-classification-changes/)

- **Prototype**: Emitted when a prototype feature is called. See
[PyTorch feature classifications](https://pytorch.org/blog/pytorch-feature-classification-changes/)

* Continue using warning/error APIs that currently exist in PyTorch wherever
possible. For instance, `TORCH_CHECK`, `TORCH_WARN`, and `TORCH_WARN_ONCE`
should continue to be used in C++

- NOTE: These existing APIs don't currently have a concept of message classes,
so that will need to be added

* Creating new message classes and severity levels should be easy

* Settings to turn warnings for a specific message class into errors

* Settings to disable specific message classes and severity levels

- TODO: However, most errors should not be disableable, right? Perhaps only
some message classes should allow disabling or downgrading errors. For
instance, currently in PyTorch, we can downgrade a nondeterministic error
to a warning, but we wouldn't want to downgrade an error from invalid
arguments given to an operation.

* Settings to avoid emitting duplicate messages generated by multiple
`torch.distribted` ranks (related to issue
[#68768](https://github.com/pytorch/pytorch/issues/68768))

* Ability to make a particular warning only warn once

- NOTE: Currently `TORCH_WARN_ONCE` does this in C++, but there is no Python
equivalent

- TODO: Should there be a setting to turn a warn-always into a warn-once for
a given message class and vice versa?

- TODO: Should warn-once be its own separate severity level?

* Settings can be changed from Python, C++, or environment variables

- Filtering warnings with Python command line arguments should
remain possible. For instance, the following turns a `DeprecationWarning`
into an error: `python -W error::DeprecationWarning your_script.py`

* Should integrate with Meta's internal logging system, which is
[glog](https://github.com/google/glog)

- TODO: What are all the requirements that define "integrating with glog"

* Must be OSS-friendly, so it shouldn't require libraries (like glog) which may
cause incompatibility issues for projects that use PyTorch

* TODO: Determine the requirements for the following concepts:

- Log files (default behavior and any settings)


## **Motivation**
Original issue: [link](https://github.com/pytorch/pytorch/issues/72948)

Currently, it is challenging for PyTorch developers to provide messages that
act consistently between Python and C++.

It is also challenging for PyTorch users to manage the messages that PyTorch
emits. For instance, if a PyTorch user happens to be calling PyTorch functions
that emit lots of warnings, it can be difficult for them to filter out those
warnings so that their project's users don't get bombarded with warnings that
they don't need to see.

0 comments on commit ff90402

Please sign in to comment.