Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

embedding层开启sparse=True后,无法查看该层权重的梯度。 #69854

Open
sealoongleft opened this issue Dec 1, 2024 · 0 comments
Open

Comments

@sealoongleft
Copy link

bug描述 Describe the Bug

description:paddle.nn.embedding层开启sparse=True后无法查看该层权重的梯度了,显示张量未初始化。但是我使用embedding.weight.gradient()可以勉强显示,但是paddle框架的输出结果提示了“该.gradient()马上就要在未来被移除了,尽快改为使用.grad用于查看参数”。而我使用.grad查看该层权重梯度的话就会提示我下面输出结果所示的内容,也就是张量未初始化。
我该如何查看开启sparse=True时的embedding层权重梯度呢?

Running code:

import paddle
import paddle.nn as nn

embedding=nn.Embedding(3,2,sparse=True)
embedding.weight.set_value(paddle.arange(6).reshape((3,2)).astype('float32'))
data=paddle.to_tensor((0,1))
out=embedding(data)
y=out.sum()*2
y.backward()
print(embedding.weight.grad) #这行还可以改为print(embedding.weight.gradient())

output result:

W1201 12:24:43.693264  4996 gpu_resources.cc:119] Please NOTE: device: 0, GPU Compute Capability: 7.5, Driver API Version: 12.6, Runtime API Version: 12.3
W1201 12:24:43.694262  4996 gpu_resources.cc:164] device: 0, cuDNN Version: 9.0.
Tensor(Not initialized)
(.venv)

其他补充信息 Additional Supplementary Information

感谢技术大佬花费解决问题我的issue,不胜感激。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant