Implement fluid API using python with guard. #6508
Description
- change current implement to
listen_and_serv,send,recvop implementation. - implement python API with guard for
listen_and_serv - build sample program using python with statement APIs.
- update document about this.
According to https://github.com/PaddlePaddle/Paddle/blob/develop/doc/design/concurrent_programming.md#the-worker-program we need to implement similar API looks like:
Server side:
loss = define_model() server = fluid.listen_and_serv() with server.do(): opt = fluid.optimizer.Adam() opt.minimize(loss)Worker side:
loss = define_model() params, grads = fluid.append_backward(loss) splited = layers.split(params, grads) with fluid.parallel_for(len(splited)) as iter: layers.send(splited["grad"][iter.idx]) with fluid.parallel_for(len(splited)) as iter: layers.recv(splited["param"][iter.idx]) layers.concat(splited["param"])If we are using CSP model, the server side may look like:
loss = define_model() params, grads = fluid.append_backward(loss) param_ch = fluid.make_chan() param_recved_ch = fluid.make_chan() grad_ch = fluid.make_chan() layers.split_to_chan(params, param_ch) layers.split_to_chan(grads, grad_ch) with fluid.go(): layers.send(grad_ch) with fluid.go(): updated_param = layers.recv(param_ch) param_recved_ch.push(updated_param) layers.concat(param_recved_ch)Metadata
Metadata
Assignees
Labels
No labels