optimize cross entropy kernel by using reduce. #4237
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
lcy-seso force-pushed the optimize_cross_entropy_kernel branch from c6a995f to a3a8a09 Compare 
reyoung reviewed Sep 20, 2017
| self.inputs = {'X': X, 'Label': label} | ||
| self.outputs = {'Y': cross_entropy} | ||
| self.attrs = {'soft_label': 1} | ||
| self.inputs = {"X": X, "Label": label} |
There was a problem hiding this comment.
https://www.python.org/dev/peps/pep-0008/#string-quotes
We following PEP8 as our style guide. In python, both '' and "" are fine.
There was a problem hiding this comment.
Yes, I know, only personally, I prefer to keep consistent in one file.
lcy-seso force-pushed the optimize_cross_entropy_kernel branch 2 times, most recently from 4b55fd2 to 1904577 Compare 
qingqing01 reviewed Sep 22, 2017
paddle/operators/cross_entropy_op.cu Outdated
There was a problem hiding this comment.
这样分配二维block，相比一维的block空的线程更多，这个计算实际一维block就够了, 可以是使用非soft-label的方式：
int block = 512; int grid = (n * d + block - 1) / block;kernel 改成：
for (int i = blockIdx.x * blockDim.x + threadIdx.x; i < N * D; i += blockDim.x * gridDim.x) { int id_dy = ids / D; dX[i] = -label[i] * dY[id_dy] / X[i]; }另外如果方便的话，可以帮忙一起修复下:
// TODO(qingqing): launch kernel on specified stream
// base on ExecutionContext.
谢谢了，当时写的时候ExecutionContext里还没有stream.
paddle/operators/cross_entropy_op.cu Outdated
There was a problem hiding this comment.
去掉line 123， 写成：
SoftCrossEntropyKernel<T, 512><<<d, block>>> lcy-seso force-pushed the optimize_cross_entropy_kernel branch 4 times, most recently from a637d86 to 031d7eb Compare 
|
|
lcy-seso force-pushed the optimize_cross_entropy_kernel branch 15 times, most recently from e8ed548 to fa3e373 Compare | All finished, please review this, thanks, everyone. @qingqing01 @Xreki |
lcy-seso force-pushed the optimize_cross_entropy_kernel branch 4 times, most recently from 11516af to 058dfbb Compare lcy-seso force-pushed the optimize_cross_entropy_kernel branch 3 times, most recently from 8f776c8 to f5be351 Compare lcy-seso force-pushed the optimize_cross_entropy_kernel branch 5 times, most recently from 1a4eff4 to ae59260 Compare qingqing01 previously approved these changes Sep 26, 2017
paddle/operators/cross_entropy_op.cc Outdated
paddle/operators/cross_entropy_op.cc Outdated
There was a problem hiding this comment.
set to 1 -> set to true.
lcy-seso force-pushed the optimize_cross_entropy_kernel branch from ae59260 to c80489a Compare lcy-seso force-pushed the optimize_cross_entropy_kernel branch from c80489a to 000d751 Compare qingqing01 approved these changes Sep 26, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
fixes #4236