Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix transformers test skipping all tests. #4501

Closed
wants to merge 3 commits into from

Conversation

loadams
Copy link
Contributor

@loadams loadams commented Oct 11, 2023

Around the release of torch 2.1, something changed that began skipping all of the tests that should have been running in nv-transformers.

Sample passing build before: https://github.com/microsoft/DeepSpeed/actions/runs/6399875223/job/17372640478#step:8:1486
Sample passing build after (skips all tests): https://github.com/microsoft/DeepSpeed/actions/runs/6476276918/job/17584733821#step:8:1355

@loadams loadams changed the title Transformers skips all tests currently Fix transformers test skipping all tests. Oct 11, 2023
@@ -27,7 +27,7 @@ jobs:
- name: Install pytorch
run: |
# use the same pytorch version as transformers CI
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stas00 - could you point me to the place in transformers CI where this is set? It looks like these are failing due to using torch 2.1.0 in place of the 2.0.1 we specify, but I wanted to make sure we were still on the same torch version.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you have sorted it out already, Logan.

Also for future similar requests please tag @pacman100 who has taken over the maintenance of the Deepspeed integration and I'm not longer with HF. Thank you!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did - and will update who I tag in the future. My apologies for the incorrect tag!

Copy link
Collaborator

@stas00 stas00 Oct 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no problem at all, Logan - you couldn't have known ;)

But please feel free to tag me on other non-HF-specific issues - I still care very much for Deepspeed and hope to start using it again soon (the current internode speed is too slow to permit ZeRO).

@loadams
Copy link
Contributor Author

loadams commented Oct 11, 2023

Closing in favor of #4388

@loadams loadams closed this Oct 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants