Fix crash when creating Torch tensor on NPU with device=get_accelerator().current_device() #5464
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@microsoft-github-policy-service agree |
|
[like] Olatunji Ruwase reacted to your message:
…________________________________
From: minchao ***@***.***>
Sent: Friday, April 26, 2024 9:44:56 AM
To: microsoft/DeepSpeed ***@***.***>
Cc: Olatunji Ruwase ***@***.***>; Mention ***@***.***>
Subject: Re: [microsoft/DeepSpeed] Fix crash when creating Torch tensor on NPU with device=get_accelerator().current_device() (PR #5464)
Hi, @tjruwase<https://github.com/tjruwase> . Would you please review this PR?
BTW, this issue happened before. See #3933<#3933>.
—
Reply to this email directly, view it on GitHub<#5464 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABAS34D4BV2RJ5HFDQT2YNDY7IOZRAVCNFSM6AAAAABGYZW3G2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANZZGAZDMMJYGU>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
tjruwase
approved these changes
Apr 26, 2024
fix the formatting issue for commit 2f382d8
|
Hi, @tjruwase. Just fix a formatting issue in new commit. Would you please approve the workflows to run? |
|
Hi, @tjruwase, the nv-accelerate-v100 and nv-lightening-v100 CI workflows seems to be down. Could you please take a look? Thx. |
|
I did a search for |
github-merge-queue
bot
removed this pull request from the merge queue due to no response for status checks
|
Hi, @tjruwase, this PR was removed from merge queue by the bot, not sure why it was removed, could you please check it out? |
umchand
pushed a commit
to umchand/DeepSpeed
that referenced
this pull request
May 20, 2024
…or().current_device() (microsoft#5464) Creating a Torch tensor with the parameter `device=get_accelerator().current_device()` can result in a crash when using an NPU. This issue arises because the `current_device` API across all accelerators is expected to return a device id as an integer, according to the [interface docs.](https://github.com/microsoft/DeepSpeed/blob/fa8458b1a80d6ba55091b17f092de19bbf95eb3d/docs/_tutorials/accelerator-abstraction-interface.md?plain=1#L52C1-L56C103) However, specifying `device` as an interger when creating tensors by default directs Torch to use the CUDA backend, which leads to crash on NPUs (and potentially other accelerators as well). To resolve this, we should use `get_accelerator().current_device_name()` instead, which returns the correct device identifier strings such as `"npu:0", "cuda:0", or "xpu:0"`. This API provides the appropriate context needed for creating tensors on specific hardware accelerators. I also notice that `device=get_accelerator().current_device()` is used across several files under deepspeed/inference, and may also lead to crash on other accelerators. --------- Co-authored-by: Olatunji Ruwase <[email protected]> Co-authored-by: Logan Adams <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Creating a Torch tensor with the parameter
device=get_accelerator().current_device()can result in a crash when using an NPU.This issue arises because the
current_deviceAPI across all accelerators is expected to return a device id as an integer, according to the interface docs.However, specifying
deviceas an interger when creating tensors by default directs Torch to use the CUDA backend, which leads to crash on NPUs (and potentially other accelerators as well).To resolve this, we should use
get_accelerator().current_device_name()instead, which returns the correct device identifier strings such as"npu:0", "cuda:0", or "xpu:0". This API provides the appropriate context needed for creating tensors on specific hardware accelerators.I also notice that
device=get_accelerator().current_device()is used across several files under deepspeed/inference, and may also lead to crash on other accelerators.