Fix for split op in BF16 inference #39548
Conversation
| Thanks for your contribution! |
wozna left a comment
wozna left a comment There was a problem hiding this comment.
You did a good job. Could you please add a unit test to these changes in cpu_bfloat16_pass._tester.cc?
| @sfraczek Please help review, thanks ! |
sfraczek left a comment
sfraczek left a comment There was a problem hiding this comment.
It looks good, however I had some comments.
| @baoachun could you please review this PR? |
lidanqing-vv requested review from Aganlengzi and baoachun and removed request for Aganlengzi 
| void CPUBFloat16Pass::SetOutputDataType(ir::Graph* graph) const { | ||
| void AddDequantize(Graph* g, ir::Node* op, ir::Node* op_out, | ||
| int& dequantize_counter) { | ||
| if (op->Op()->Type() == "prior_box") return; |
There was a problem hiding this comment.
Why does prior_box return directly? Could you add descriptions or can we use set to maintain operators that require special handling?
There was a problem hiding this comment.
@wozna could you please advise us on this prior_box scenario?
There was a problem hiding this comment.
Prior_box output always produces floating-point results because these are prior boxes generated. Therefore, we do not need dequantization. And so far only this operator is behaving this way.
There was a problem hiding this comment.
Jakub may add comments in next PR, but this PR has passed all CIs and hopefully it could be merged
5 participants
PR types
Bug fixes
PR changes
Others
Describe
Fix for
splitop in BF16 inference. Bug was related to incorrect handling of multi-output which was resolved in similar way to handling multi-input in ops such asconcat.