Skip to content

perf(activation): add fast, memory efficient mish #46

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ClashLuke wants to merge 1 commit into from
Closed

perf(activation): add fast, memory efficient mish #46

ClashLuke wants to merge 1 commit into from

Conversation

@ClashLuke
Copy link

@ClashLuke ClashLuke commented Aug 31, 2020

No description provided.

@ClashLuke ClashLuke changed the title perf(activation): add fast, memory efficient swish perf(activation): add fast, memory efficient mish Aug 31, 2020
@rwightman
Copy link
Owner

rwightman commented Aug 31, 2020

There is already an autograd.fn mish in https://github.com/rwightman/gen-efficientnet-pytorch/blob/master/geffnet/activations/activations_me.py
@rwightman rwightman closed this Aug 31, 2020
@ClashLuke
Copy link
Author

ClashLuke commented Aug 31, 2020

One is efficient, the other recomputes everything in the backward pass. Hence the perf label.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants

@ClashLuke @rwightman