[ProcessGroup]add new comm primitive

[Baibaifan]

PR types

New features

PR changes

Others

Describe

1. Add new comm primitive (barrier\send\recv)

with _test_eager_guard(): paddle.set_device('gpu:%d' % paddle.distributed.ParallelEnv().dev_id) pg = init_process_group() # test barrier # rank 0 if pg.rank() == 0: task = pg.barrier() # test send/recv # rank 0 x = np.random.random(self.shape).astype(self.dtype) tensor_x = paddle.to_tensor(x) if pg.rank() == 0: task = pg.send(tensor_x, dst=1) task.wait() paddle.device.cuda.synchronize() # rank 1 else: y = np.random.random(self.shape).astype(self.dtype) tensor_y = paddle.to_tensor(y) task = pg.recv(tensor_y, src=0) task.wait() paddle.device.cuda.synchronize() 

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[sandyhouse]

LGTM