Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问 worker 发送push后, server端是如何合并最新的参数的? #142

Open
lrjocean opened this issue Dec 17, 2018 · 4 comments

Comments

@lrjocean
Copy link

lrjocean commented Dec 17, 2018

我所理解的方案,模型参数合并的方法主要有:
1)在worker端更新响应的参数w后,push到server覆盖旧的参数w;
2)server接收的是delta w, 再更新参数w;
不清楚ps-lite是什么策略更新参数的?
在代码中没找到参数更新的代码。。。(可能是我没理解)
@mli @tqchen

@lrjocean lrjocean changed the title 请问 server端在接收到worker's参数更新采用的是什么策略? 请问 worker 发送push后, server端是如何合并最新的参数的? Dec 17, 2018
@szha
Copy link
Member

szha commented Dec 17, 2018

push的是gradient的话会加到weight上

@lrjocean
Copy link
Author

lrjocean commented Dec 18, 2018

push的是gradient的话会加到weight上

请问ps-lite代码在哪里?还是说要自己实现?
(只找到 kv_app.h中 KVServerDefaultHandle,store[key]+= req_data.vals[i];)
如果是将push的gradient直接加到weight上,那么这个gradient 应该是乘以步长之后的gradient吧?
@szha

@Ethan199111
Copy link

ps架构下,只能接受asgd的更新策略,实际上所有分布式机器学习梯度更新策略都是这样子。可以去看mxnet中DCASGD

@zmxdream
Copy link

对,默认的是直接加,本地应该要乘以步长

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants