Register sdp lower precision autocast #7299
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
ghost mentioned this pull request 
qihqi approved these changes Jun 17, 2024
| Most of the failure caused by numpy, seem no related about this PR. How can I rerun this PR after the fix of build/test enviroment? |
| hmm if you rebase the issue should be gone |
| I think this PR need re-approve to run the tests. |
| I can not reproduce the test_train_mp_mnist_amp.py failure in my machine. And it is weird that the training loss has decreased to 0.00066 which is same as I see in my machine but the accuracy is 0. I really don’t know why sdp affects the mnist. It’s obvious that this op is not used at all. |
| Let me retrigger, it might be using the fake data so accuracy doesn't matter. It seems like CI just exited with an error code but I don't know why. |
| The CI has passed, this PR can be merged. |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
Details can be seen in Issue 7177.