Skip to content

Commit

Permalink
Create RFC for a logging system and list some required features
Browse files Browse the repository at this point in the history
  • Loading branch information
kurtamohler committed Jun 22, 2022
1 parent b379b19 commit 9dd373a
Showing 1 changed file with 160 additions and 0 deletions.
160 changes: 160 additions & 0 deletions RFC-0026-logging-system.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# PyTorch Logging System

## **Summary**
Create a new message logging system for PyTorch with the following features:

* The Python and C++ APIs for emitting messages and changing settings should
all be consistent

* Offer different log levels (info, warning, error, ...)

* Offer different classes of messages

- TODO: What are some of the message classes that we'll need?

* Creating new message classes and log levels should be easy

* Settings to turn warnings for a specific message class into errors

* Settings to disable specific message classes and log levels

- TODO: However, I assume most errors should not be disableable

* Settings to avoid emitting duplicate messages generated by multiple
`torch.distribted` ranks (related to issue
[#68768](https://github.com/pytorch/pytorch/issues/68768))

* Ability to make a particular warning class or log level only warn once

- NOTE: Currently `TORCH_WARN_ONCE` does this in C++, but there is no Python
equivalent

- TODO: Should there be a setting to turn a warn-always into a warn-once for
a given message class?

* Settings can be changed from Python, C++, or environment variables

* Should integrate with Meta's internal logging system, which is
[glog](https://github.com/google/glog)

- TODO: What are all the requirements that definine "integrating with glog"

* Must be OSS-friendly, so it shouldn't require libraries (like glog) which may
cause incompatibility issues for projects that use PyTorch


## **Motivation**
Original issue: [link](https://github.com/pytorch/pytorch/issues/72948)

Currently, it is challenging for PyTorch developers to provide messages that
act consistently between Python and C++.

It is also challenging for PyTorch users to manage the messages that PyTorch
emits. For instance, if a PyTorch user happens to be calling PyTorch functions
that emit lots of warnings, it can be difficult for them to filter out those
warnings so that their project's users don't get bombarded with warnings that
they don't need to see.

---

<details>
<summary>Remaining template not yet filled in - click to expand</summary>

## **Proposed Implementation**
This is the bulk of the RFC. Explain the design in enough detail for somebody familiar with PyTorch to understand, and for somebody familiar with the implementation to implement.
This should get into specifics and corner-cases, and include examples of how the feature is used, and how it will interact with other features. Any new terminology should be defined here.
Consider:
* using examples and diagrams to help illustrate your ideas.
* including code examples, if you're proposing an interface or system contract.
* linking to project briefs or wireframes that are relevant.


## **Metrics **
What are the main metrics to measure the value of this feature?


## **Drawbacks**
Are there any reasons why we should not do this? Here we aim to evaluate risk and check ourselves.

Please consider:
* is it a breaking change?
* Impact on UX
* implementation cost, both in terms of code size and complexity
* integration of this feature with other existing and planned features


## **Alternatives**
What other designs have been considered? What is the impact of not doing this?


## **Prior Art**
Discuss prior art (both good and bad) in relation to this proposal:
* Does this feature exist in other libraries? What experience has their community had?
* What lessons can be learned from other implementations of this feature?
* Published papers or great posts that discuss this


## **How we teach this**
* What names and terminology work best for these concepts and why? How is this idea best presented?
* Would the acceptance of this proposal mean the PyTorch documentation must be re-organized or altered?
* How should this feature be taught to existing PyTorch users?


## **Unresolved questions**
* What parts of the design do you expect to resolve through the RFC process before this gets merged?
* What parts of the design do you expect to resolve through the implementation of this feature before stabilization?
* What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC?


## Resolution
We decided to do it. X% of the engineering team actively approved of this change.

### Level of Support
Choose one of the following:
* 1: Overwhelming positive feedback.
* 2: Positive feedback.
* 3: Majority Acceptance, with conflicting Feedback.
* 4: Acceptance, with Little Feedback.
* 5: Unclear Resolution.
* 6: RFC Rejected.
* 7: RFC Rejected, with Conflicting Feedback.


#### Additional Context
Some people were in favor of it, but some people didn’t want it for project X.


### Next Steps
Will implement it.


#### Tracking issue
<github issue URL>


#### Exceptions
Not implementing on project X now. Will revisit the decision in 1 year.

</details>

<details>
<summary>Instructions - click to expand</summary>

- Fork the rfcs repo: https://github.com/pytorch/rfcs
- Copy `RFC-0000-template.md` to `RFC-00xx-my-feature.md`, or write your own open-ended proposal. Put care into the details.
- Submit a pull request titled `RFC-00xx-my-feature`.
- Assign the `draft` label while composing the RFC. You may find it easier to use a WYSIWYG editor (like Google Docs) when working with a few close collaborators; feel free to use whatever platform you like. Ideally this document is publicly visible and is linked to from the PR.
- When opening the RFC for general discussion, copy your document into the `RFC-00xx-my-feature.md` file on the PR and assign the `commenting` label.
- Build consensus for your proposal, integrate feedback and revise it as needed, and summarize the outcome of the discussion via a [resolution template](https://github.com/pytorch/rfcs/blob/rfc-process/RFC-0000-template.md#resolution).
- If the RFC is idle here (no activity for 2 weeks), assign the label `stalled` to the PR.
- Once the discussion has settled, assign a new label based on the level of support:
- `accepted` if a decision has been made in the RFC
- `draft` if the author needs to rework the RFC’s proposal
- `shelved` if there are no plans to move ahead with the current RFC’s proposal. We want neither to think about evaluating the proposal
nor about implementing the described feature until some time in the future.
- A state of `accepted` means that the core team has agreed in principle to the proposal, and it is ready for implementation.
- The author (or any interested developer) should next open a tracking issue on Github corresponding to the RFC.
- This tracking issue should contain the implementation next steps. Link to this tracking issue on the RFC (in the Resolution > Next Steps section)
- Once all relevant PRs are merged, the RFC’s status label can be finally updated to `closed`.

</details>

0 comments on commit 9dd373a

Please sign in to comment.