Update attention / self-attn based models from a series of experiments#821

Merged

rwightman merged 22 commits intomasterfrom

Sep 14, 2021

Commits on Aug 20, 2021

Update attention / self-attn based models from a series of experiments:
rwightman
committed

Commits on Aug 21, 2021

Commits on Sep 2, 2021

Update HaloAttn comment
rwightman
committed
LR scheduler update:
rwightman
committed
Add a BCE loss impl that converts dense targets to sparse /w smoothing as an alternate to CE w/ smoothing. For training experiments.
rwightman
committed
Add RepeatAugSampler as per DeiT RASampler impl, showing promise for current (distributed) training experiments.
rwightman
committed
Update training script and loader factory to allow use of scheduler updates, repeat augment, and bce loss
rwightman
committed
Fix misnamed arg, tweak other train script args for better defaults.
rwightman
committed
Fix updated validation_batch_size fallback
rwightman
committed

Commits on Sep 4, 2021

Commits on Sep 5, 2021

Commits on Sep 9, 2021

Add resnet33ts weights, update resnext26ts baseline weights
rwightman
committed

Commits on Sep 13, 2021

Merge branch 'master' into attn_update
rwightman
committed

Commits on Sep 14, 2021

BotNet models were still off, remove weights for bad configs. Add good SE-HaloNet33-TS weights.
rwightman
committed