Skip to content

Conversation

@helinwang
Copy link
Contributor

Fix: #8168

@helinwang helinwang changed the title long running training tests: fail when got NaN loss Terminate and fail long running training tests when got NaN loss Feb 5, 2018
Copy link
Contributor

@abhinavarora abhinavarora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! This makes sense. We had similar checks when training with SpeechDL

@abhinavarora abhinavarora merged commit be7fcc0 into PaddlePaddle:develop Feb 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants