Skip to content

[Phi] Move softmax with cross entropy kernel into phi #40832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 20 commits into from
Apr 1, 2022
Merged

[Phi] Move softmax with cross entropy kernel into phi #40832

merged 20 commits into from
Apr 1, 2022

Conversation

@chenwhql
Copy link
Contributor

@chenwhql chenwhql commented Mar 22, 2022 •
edited
Loading

PR types

Function optimization

PR changes

OPs

Describe

[Phi] Move softmax with cross entropy kernel into phi

  1. softmax_with_cross_entropy op现在对应的Python API是cross_entropy，因此kernel文件命名为cross_entropy_kernel.*
  2. softmax_with_cross_entropy op名不符实，现在它的行为是一个可以选择是否先进行softmax计算，再进行cross_entropy计算，cross_entropy计算是一定有的，而softmax不一定计算，因此将kernel更名为cross_entropy_with_softmax，实际上这也只是一种特殊的cross_entropy运算而已
  3. 前向CPU kernel有存在显而易见的冗余代码，整合了重复代码，进行了简化封装
  4. 内部调用函数保持原命名，未进行更改，有必要后续再进行，仅kernel名进行了调整
chenwhql added 20 commits March 22, 2022 04:09
@chenwhql
add cross_entropy_with_softmax phi kernel
2dbabfa
@chenwhql
remove softmax_with_cross_entropy kernel
9341232
@chenwhql
add softmax_with_cross_entropy grad kernel
e50770e
@chenwhql
remove original op kernel
cb35956
@chenwhql
refine cross entropy impl
020aa09
@chenwhql
resolve conflict
de90324
@chenwhql
fix pointer error
e4390a9
@chenwhql
revert kernel cu change
11c835f
@chenwhql
fix xpu failed
3595b97
@chenwhql
fix cinn failed
e01d16c
@chenwhql
Merge branch 'develop' into phi/move_softmax_with_cross_entropy_kernel
e1654a0
@chenwhql
fix npu failed
7e2459b
@chenwhql
add forward sig
4c14795
@chenwhql
Merge branch 'phi/move_softmax_with_cross_entropy_kernel' of https://…
9fdeee3 
…github.com/chenwhql/Paddle into phi/move_softmax_with_cross_entropy_kernel
@chenwhql
resolve conflict
8c81f1e
@chenwhql
add check_nan_inf for pt kernel
6155dce
@chenwhql
resolve conflict
307b5eb
@chenwhql
remove repeat cmake item
37c0ec5
@chenwhql
Merge branch 'develop' into phi/move_softmax_with_cross_entropy_kernel
129bc13
@chenwhql
fix unittest error
2d55ed7
XiaoguangHu01
XiaoguangHu01 approved these changes Apr 1, 2022
View reviewed changes
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
@chenwhql chenwhql merged commit e6ec98f into PaddlePaddle:develop Apr 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants

@chenwhql @zyfncg @XiaoguangHu01