Update passes in quant2_int8_mkldnn_pass

[wozna]

PR types

Bug fixes

PR changes

Others

Describe

This PR updates passes that are applied to the graph during quant2_int8_mkldnn_pass. It turns out that many passes were added to CpuPassStrategy and EnableMkldnn passes, but they weren't added to mentioned python script. I used the same order of the passes that is presented in paddle_pass_builder.cc.

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[lidanqing-vv]

@wozna @pmajchrzak Please verify this PR will not cause our existing daily CI int8 models accuracy and performance drop. Thanks!

[sfraczek]

LGTM

[lidanqing-vv]

@wozna Hi could you or ask Pior to upload spreadsheet showing this PR does not cause any accuracy drop or performance drop on exisiting int8 modes CI

[wozna]

I can confirm that all recent changes don't cause any accuracy change or performance change in any model from our CI. Before I had a problem with scale_matmul_fuse_pass. I couldn't understand why this pass has to be in _quantize_fp32_graph not in the _optimize_fp32_graph. It turned out that we use a scale operator to gather scale for matmul and when I changed the place of scale_matmul_fuse_pass, 12 matmuls weren't quantized because the scale for input was missing. This change improved a little accuracy on Ernie from 0.791165 to 0.8000 but caused a performance drop of 1.5%. That's why I returned to the previous order, but I moved cpu_quantize_placement_pass, after scale_matmul and reshape_transpose_matmul/V2 passes.

[wozna]

@lidanqing-intel @sfraczek Could you please repeat your review?

[sfraczek]

LGTM.

[lidanqing-vv]

LGTM

[lidanqing-vv]

@baoachun Could you please review this PR. Thanks

[lidanqing-vv]

@baoachun Hi could you please approve this PR? Some models from BML team still need to use save_quant_model.py solution for now. So we still need to maintain this save_quant_model.py. Thanks

[baoachun]

LGTM