float8nocompile: add e2e fsdp test #1523
Open
+97
−3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Stack from ghstack (oldest at bottom): |
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1523
Note: Links to docs will display an error until the docs builds have been completed. ⏳ No Failures, 1 PendingAs of commit a0544db with merge base 4738377 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This was referenced Jan 8, 2025
facebook-github-bot
added
the
CLA Signed
This was referenced Jan 8, 2025
danielvegamyhre
added a commit
that referenced
this pull request
Jan 8, 2025
ghstack-source-id: 5baf685343844750b542f98880ef3b8174141ec6 ghstack-comment-id: 2576459235 Pull Request resolved: #1523
danielvegamyhre
added
the
topic: not user facing
danielvegamyhre
added a commit
that referenced
this pull request
Jan 8, 2025
ghstack-source-id: da38b4a141de7dfee4cec9132967ad76d7d6dc20 ghstack-comment-id: 2576459235 Pull Request resolved: #1523
Closed
danielvegamyhre
added a commit
that referenced
this pull request
Jan 8, 2025
ghstack-source-id: 619ae59ae76acfda5492b56241f74878a4d1b04f ghstack-comment-id: 2576459235 Pull Request resolved: #1523
danielvegamyhre
added a commit
that referenced
this pull request
Jan 8, 2025
ghstack-source-id: 95876811b0aa1096b2bbf1151279ea7f65b9fd47 ghstack-comment-id: 2576459235 Pull Request resolved: #1523
danielvegamyhre
added a commit
that referenced
this pull request
Jan 8, 2025
ghstack-source-id: bcd7f53ea269223ec5d6f5bff1e169f426f6c474 ghstack-comment-id: 2576459235 Pull Request resolved: #1523
danielvegamyhre
added a commit
that referenced
this pull request
Jan 8, 2025
ghstack-source-id: 0525d57fe1e188dd6d8790c8d0b39e44d60b2f7b ghstack-comment-id: 2576459235 Pull Request resolved: #1523
danielvegamyhre
added a commit
that referenced
this pull request
Jan 8, 2025
ghstack-source-id: 0a8863393a11404f12d3f82173175fc37c100656 ghstack-comment-id: 2576459235 Pull Request resolved: #1523
vkuzo
reviewed
Jan 8, 2025
| import torch | ||
| import torch.distributed as dist | ||
| import torch.nn as nn | ||
| from torch.distributed.fsdp import FullyShardedDataParallel as FSDP |
There was a problem hiding this comment.
this is FSDP1. Can we test FSDP2, which is the new, recently released version of FSDP with support for float8 all-gather?
https://github.com/pytorch/torchtitan/blob/main/docs/fsdp.md
IMO testing just FSDP2 is fine
There was a problem hiding this comment.
Updated to test with fsdp2
danielvegamyhre
added a commit
that referenced
this pull request
Jan 8, 2025
ghstack-source-id: f888979319102ea666dcf1325d17d6e7e51aeb72 ghstack-comment-id: 2576459235 Pull Request resolved: #1523
vkuzo
approved these changes
Jan 8, 2025
danielvegamyhre
added a commit
that referenced
this pull request
Jan 8, 2025
ghstack-source-id: fb4ef48e6aa1e98ecd36cb2843fb83f74efe3c74 ghstack-comment-id: 2576459235 Pull Request resolved: #1523
danielvegamyhre
added a commit
that referenced
this pull request
Jan 9, 2025
ghstack-source-id: fb4ef48e6aa1e98ecd36cb2843fb83f74efe3c74 ghstack-comment-id: 2576459235 Pull Request resolved: #1523
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.