[Zero-Dim] distributed all_gather/scatter/all_to_all support input 0D tensor #53186
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| 你的PR提交成功，感谢你对开源项目的贡献! |
| ❌ The PR is not created using PR's template. You can refer to this Demo. |
zhwesky2010 changed the title 
zhwesky2010 force-pushed the 0d_dist_scatter branch 3 times, most recently from 4d58d16 to 69ee1cf Compare 
zhwesky2010 force-pushed the 0d_dist_scatter branch from 69ee1cf to 33f009c Compare zhwesky2010 changed the title 
LiYuRio reviewed Apr 26, 2023
| int64_t send_numel = in->numel(); | ||
| const T* send_buff = in->data<T>(); | ||
| T* recv_buff = out->data<T>(); | ||
| T* recv_buff = out->mutable_data<T>(place); |
Contributor
There was a problem hiding this comment.
这样out的大小对吗，allgather的out是in的nranks倍
Contributor Author
There was a problem hiding this comment.
是的，原来的nranks倍，在infershape里已经做了out的dim设置。目前是op计算时又重复设置了一次，所以去掉了
From00 approved these changes Apr 26, 2023
Contributor
From00 left a comment
From00 left a comment There was a problem hiding this comment.
LGTM for use mutable_data
| 你的PR已合入Paddle库，请关注后续测试结果。 |
zhwesky2010 added a commit to zhwesky2010/Paddle that referenced this pull request May 8, 2023
XiaoguangHu01 pushed a commit that referenced this pull request May 9, 2023
#53601) * [Zero-Dim] fix functool.reduce more safe with intial value, to support empty list (#53182) * [Zero-Dim] support 0d tensor for shape and squeeze onednn kernel (#52832) * support 0d tensor for shape and squeeze onednn kernel * set python api for shape op ut * [Zero-Dim] distributed scatter/all_to_all support input 0D tensor (#53186) * [Zero-Dim] Support paddle.sum/mean/loss api output 0D,test=allcase (#52739) * [CINN Support 0D-Tensor] CINN supports 0D-Tensor with trick temporarily (#53382) * [CINN Support 0D-Tensor] CINN supports 0D-Tensor with trick temporarily * Add unittest * [CINN Support 0D-Tensor] CINN hack squeeze2 with trick temporarily (#53454) * fix test_autograd_dynamic (#53473) Co-authored-by: zhwesky2010 <zhouwei25@baidu.com> --------- Co-authored-by: YangQun <qun.yang@intel.com> Co-authored-by: HongyuJia <jiahongyu@baidu.com> Co-authored-by: HydrogenSulfate <490868991@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
APIs
Description
Pcard-66984
为distributed all_gather/scatter/all_to_all 支持0D Tennsor输入
相当于 #49279 的静态图分支实现。