Fixed NeptuneLogger when using DDP

[Raalsky]

What does this PR do?

Makes Neptune Experiment (Run in the scope of Neptune) initialization to be called only on rank == 0 in distributed environment. Should not introduce any breaking changes and was tested with previously pickled NeptuneLogger.

Fixes #10604

And could resolve a few similar issues in external repositories:

• BUG: Error msg during training - Timestamp must be non-decreasing for series attribute neptune-ai/neptune-client#733
• BUG: "Timestamp must be non-decreasing for series attribute:" error while running with accelerator=ddp neptune-ai/neptune-pytorch-lightning#12
• Running Pytorch Lightning with accelerator="ddp" creates a run for every GPU core neptune-ai/neptune-pytorch-lightning#11

Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

• Was this discussed/approved via a GitHub issue? (not for typos and docs)
• Did you read the contributor guideline, Pull Request section?
• Did you make sure your PR does only one thing, instead of bundling different changes together?
• Did you make sure to update the documentation with your changes? (if necessary)
• Did you write any new necessary tests? (not for typos and docs)
• Did you verify new and existing tests pass locally with your changes?
• Did you list all the breaking changes introduced by this pull request?
• Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review

Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

• Is this pull request ready for review? (if not, please submit in draft mode)
• Check that all items from Before submitting are resolved
• Make sure the title is self-explanatory and the description concisely explains the PR
• Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?

Make sure you had fun coding 🙃
Definitely ❤️

[awaelchli]

Great! Thanks for the quick fix!

One could consider adding Neptune to the parametrization of
tests/loggers/test_all.py::test_logger_created_on_rank_zero_only

[Raalsky]

@awaelchli

Great! Thanks for the quick fix!

One could consider adding Neptune to the parametrization of tests/loggers/test_all.py::test_logger_created_on_rank_zero_only

Added 😉. Sorry, am I broke something (don't know about an active auto merge before committing)? :D

[Raalsky]

It looks like a final state now 😉. @rohitgr7 @awaelchli 😉 Could you re-enable auto-merge?

[codecov]

Codecov Report

Merging #11030 (6c819c4) into master (62f1e82) will increase coverage by 0%.
The diff coverage is 87%.

@@           Coverage Diff           @@
##           master   #11030   +/-   ##
=======================================
  Coverage      88%      88%           
=======================================
  Files         177      177           
  Lines       16526    16551   +25     
=======================================
+ Hits        14580    14603   +23     
- Misses       1946     1948    +2