Support multi-outputs feature for broadcast ops #38329
Conversation
| ✅ This PR's description meets the template requirements! |
| Thanks for your contribution! |
JamesLim-sy changed the title JamesLim-sy changed the title JamesLim-sy force-pushed the support_multi-output_for_broadcast branch from 3d77a4c to 474b13c Compare 
JamesLim-sy force-pushed the support_multi-output_for_broadcast branch from 474b13c to ffccae7 Compare 
| Functor func) { | ||
| InT args[Arity][VecSize]; | ||
| OutT result[VecSize]; | ||
| ScalarType<OutT> vec_result[NumOuts][VecSize]; |
There was a problem hiding this comment.
这个vec_result可以挪到L199 WriteData函数里面定义
JamesLim-sy changed the title 
| // Vector type | ||
| const int kVectorSize = (NX % 4 == 0) ? 4 : (NX % 2 == 0) ? 2 : 1; | ||
| const int kVectorsPerThread = NX / kVectorSize; | ||
| constexpr int kVectorSize = (NX % 4 == 0) ? 4 : (NX % 2 == 0) ? 2 : 1; |
There was a problem hiding this comment.
constexpr 有性能影响吗 (c++14 支持) ？理论上 const 编译器也会在编译时计算
There was a problem hiding this comment.
这个修改是从读代码的角度出发的，切换成constexpr的话，可以显式认定是编译期计算
| }; | ||
| | ||
| template <typename T, int VecSize, int Rank, bool IsBoundary = false> | ||
| template <typename T, int VecSize, int Rank, bool IsBoundary> |
There was a problem hiding this comment.
修改的时候看到LoadData只接受了ElementwiseBroadcastKernelImpl 函数的调用，而ElementwiseBroadcastKernelImpl 已经设定了bool IsBoundary = false，所以取消了这里的默认模板参数设置；
修改回原始状态
| T *dst, | ||
| const T *__restrict__ src, | ||
| uint32_t block_offset, | ||
| int block_offset, |
There was a problem hiding this comment.
和上一条相同，修改的时候看到LoadData只接受了ElementwiseBroadcastKernelImpl函数的调用，而ElementwiseBroadcastKernelImpl的参数列表中是int block_offset，所以就把这里改成了，不过看到了KP中采用的是uint32_t ，会再提个commit把这里改回来
修改回原始状态uint32_t
| * size: The current block needs to load size elements continuously. | ||
| */ | ||
| | ||
| #if defined(__NVCC__) |
There was a problem hiding this comment.
功能上已经完成了，此处修改的目的是想从编译阶段区分IsBoundary的两种情况所走入的计算分支，本质是想实现c++ 17中的逻辑
if constexpr (condition) { } There was a problem hiding this comment.
经过检查，这里采用偏特化的修改导致了性能下降问题，故复原回原始写法
… support_multi-output_for_broadcast
… support_multi-output_for_broadcast
4 participants
PR types
New features
PR changes
OPs
Describe
LaunchBroadcastElementwiseCudaKernel所产生的启动overhead开销out_1和out_2的维度应相同，且2个functor的axis设置须要相同paddle::framework::Array作为输入的数据类型，并设计如下所示的functorLaunchBroadcastElementwiseCudaKernel时，模板参数由于<InT, OutT, functor>转变为<InT, OutT, functor， NumOuts>，其中NumOuts用于表达functor的输出元素数量（本例为2），默认值为1故兼容现有的单个functor计算的写法支持多输出前后，
elementwise计算性能几无影响：Fig. 1 支持多输出功能前
Fig. 2 支持多输出功能后
NumOuts用于实现多输出情形，但面对多输出情形Function_traits中的ReturnType模板参数本身也是paddle::framework::Array<OutT, NumOuts>的类型，一直想从ReturnType中尝试获取NumOuts，暂时未能有效实现