From ff90402faa27bf04779af61607e4f0602a045b9b Mon Sep 17 00:00:00 2001 From: Kurt Mohler Date: Wed, 22 Jun 2022 15:08:08 -0500 Subject: [PATCH] Create RFC for a logging system and list some required features --- RFC-0026-logging-system.md | 103 +++++++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+) create mode 100644 RFC-0026-logging-system.md diff --git a/RFC-0026-logging-system.md b/RFC-0026-logging-system.md new file mode 100644 index 0000000..24b691a --- /dev/null +++ b/RFC-0026-logging-system.md @@ -0,0 +1,103 @@ +# PyTorch Logging System + +## **Summary** +Create a message logging system for PyTorch with the following requirements: + +* All errors, warnings, and other messages generated by PyTorch should be + emitted using the the logging system API + +* The APIs for emitting messages and changing settings should all be consistent + between C++ and Python + +* Offer different message severity levels, including at least the following: + + - **Info**: Emits a message without creating a warning or error. By default, + this gets printed to stdout + + - **Warning**: Emits a message as a warning. By default, this will turn into + a Python warning + + - **Error**: Emits a message as an error. By default, this will turn into + a Python error + + - TODO: Should we also have a **Fatal** severity for integration with + Meta's internal logging system? A fatal message terminates the program + +* Offer different classes of messages, including at least the following: + + - **Default**: A catch-all message class + + - **Nondeterministic**: Emitted when `torch.use_deterministic_algorithms(True)` + is set and a nondeterministic operation is called + + - **Deprecated**: Emitted when a deprecated function is called + + - **Beta**: Emitted when a beta feature is called. See + [PyTorch feature classifications](https://pytorch.org/blog/pytorch-feature-classification-changes/) + + - **Prototype**: Emitted when a prototype feature is called. See + [PyTorch feature classifications](https://pytorch.org/blog/pytorch-feature-classification-changes/) + +* Continue using warning/error APIs that currently exist in PyTorch wherever + possible. For instance, `TORCH_CHECK`, `TORCH_WARN`, and `TORCH_WARN_ONCE` + should continue to be used in C++ + + - NOTE: These existing APIs don't currently have a concept of message classes, + so that will need to be added + +* Creating new message classes and severity levels should be easy + +* Settings to turn warnings for a specific message class into errors + +* Settings to disable specific message classes and severity levels + + - TODO: However, most errors should not be disableable, right? Perhaps only + some message classes should allow disabling or downgrading errors. For + instance, currently in PyTorch, we can downgrade a nondeterministic error + to a warning, but we wouldn't want to downgrade an error from invalid + arguments given to an operation. + +* Settings to avoid emitting duplicate messages generated by multiple + `torch.distribted` ranks (related to issue + [#68768](https://github.com/pytorch/pytorch/issues/68768)) + +* Ability to make a particular warning only warn once + + - NOTE: Currently `TORCH_WARN_ONCE` does this in C++, but there is no Python + equivalent + + - TODO: Should there be a setting to turn a warn-always into a warn-once for + a given message class and vice versa? + + - TODO: Should warn-once be its own separate severity level? + +* Settings can be changed from Python, C++, or environment variables + + - Filtering warnings with Python command line arguments should + remain possible. For instance, the following turns a `DeprecationWarning` + into an error: `python -W error::DeprecationWarning your_script.py` + +* Should integrate with Meta's internal logging system, which is + [glog](https://github.com/google/glog) + + - TODO: What are all the requirements that define "integrating with glog" + +* Must be OSS-friendly, so it shouldn't require libraries (like glog) which may + cause incompatibility issues for projects that use PyTorch + +* TODO: Determine the requirements for the following concepts: + + - Log files (default behavior and any settings) + + +## **Motivation** +Original issue: [link](https://github.com/pytorch/pytorch/issues/72948) + +Currently, it is challenging for PyTorch developers to provide messages that +act consistently between Python and C++. + +It is also challenging for PyTorch users to manage the messages that PyTorch +emits. For instance, if a PyTorch user happens to be calling PyTorch functions +that emit lots of warnings, it can be difficult for them to filter out those +warnings so that their project's users don't get bombarded with warnings that +they don't need to see.