Skip to content

Conversation

@Vvsmile
Copy link
Contributor

@Vvsmile Vvsmile commented Jun 14, 2023

PR types

Others

PR changes

OPs

Description

  • Modify the accuracy checking framework of bf16 datatype in OpTest, including both of forward and backward.
  • The forward result of bf16 is compared with the result of fp32.
  • The backward result of bf16 is compared with the result of fp32.
  • This modification does not need any fix in unit test file.
  • This accuracy checking mechanism is same as it of fp16 now, and this mechanism uses the bf16 or fp16 datatype as the input's datatype, converts to fp32 in the OpTest framework, gets the computation results of each datatype and compares them with each other. Hence, it removes the error caused by rounding from high-precision to low-precision, which is the primary cause resulting in the huge accuracy error in bf16 computation checking in some cases, such as cumsum.
    card-72073
@paddle-bot
Copy link

paddle-bot bot commented Jun 14, 2023

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@Vvsmile Vvsmile changed the title modify the bf16 accuracy checking framework in OpTest [Cherry-Pick] Modify the bf16 accuracy checking framework in OpTest Jun 14, 2023
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@XiaoguangHu01 XiaoguangHu01 merged commit 8077d79 into PaddlePaddle:release/2.5 Jun 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants