add paddle functions for Matrix ContextProjection APIs #1009
Conversation
There was a problem hiding this comment.
generateSequenceStartPositions是一个用于paddle test的通用接口，还是把它单独写一个文件，放到utils编成一个单独的lib，add_library(paddle_test ....)。这样add_unittest(context_projection_op_test里面就不需要显示的增加../gserver/tests/TestUtil.cpp。
There was a problem hiding this comment.
will do it in a separate pull request.
There was a problem hiding this comment.
@hedaoyuan , done, please refer to #1058
tianbingsz force-pushed the paddle_func_mat branch from 491edf8 to 8d55be4 Compare tianbingsz changed the title tianbingsz force-pushed the paddle_func_mat branch from 8d55be4 to 61c1edb Compare 
| 后续可以一起看一下 @hedaoyuan ,sure, will do it later. |
There was a problem hiding this comment.
Function ContextProjectionForward可以去掉is_padding参数，直接根据weight是否为空判断是否padding即可。这样这里也不需要再init一次了。对于is_padding == false的情况，可以构造空的weight作为参数传递（或者通过检查参数个数来判断）。
by going through Daoyuan's excellent paddle function design.
…ProjectionBackwardWeightw
resolve merge conflicts
tianbingsz force-pushed the paddle_func_mat branch from ef47517 to ea4d08d Compare 
| | ||
| void calc(const Arguments& inputs, | ||
| const Arguments& outputs, | ||
| const Arguments& inouts) override { |
There was a problem hiding this comment.
Need clear comment about what the function does.
inputs are readonly. outputs are writeonly. inouts are for both read and write. For this function, we should use inouts instead of outputs. Because the result is added to output value.
| * \param inputs[1] weight grad. | ||
| * \param inputs[2] input sequence. | ||
| * \param outputs[0] output value. | ||
| */ |
There was a problem hiding this comment.
comment what the function does
| | ||
| void calc(const Arguments& inputs, | ||
| const Arguments& outputs, | ||
| const Arguments& inouts) override { |
There was a problem hiding this comment.
should use inouts because gradient are accumulated.
| * \param inputs[0] input grad. | ||
| * \param inputs[1] input sequence. | ||
| * \param outputs[0] output grad. | ||
| */ |
There was a problem hiding this comment.
comment what the function does
| | ||
| void calc(const Arguments& inputs, | ||
| const Arguments& outputs, | ||
| const Arguments& inouts) override { |
There was a problem hiding this comment.
use inouts for accumulate gradient.
| | ||
| void calc(const Arguments& inputs, | ||
| const Arguments& outputs, | ||
| const Arguments& inouts) override { |
fix dead chain at op_notes fix Sharelod() to ShareLoD()
3 participants
added paddle functions for Matrix ContextProjection (#977)
contextProjectionForward
contextProjectionBackwardData
contextProjectionBackwardWeight
contextProjectionBackward