Merge upstream to local (#4)

* Implement the rotation trick. * release the rotation trick for VQ from @cfifty https://arxiv.org/abs/2410.06424 * cleanup rotation trick * address lucidrains#165 * address lucidrains#166 * 1.18.5 * it had its chance * address lucidrains/audiolm-pytorch#279 * attempt to address lucidrains/audiolm-pytorch#279 again * try to fix sync seed again * add compatibility for the residual VQ proposed in TIGER https://arxiv.org/abs/2305.05065, for building recommendation systems * misread * address lucidrains#171 * only sync seed during training * take care of a small pain point * move the rotation operation from rotation trick to a function for potential reuse for other research * simvq * update init * uncomment st * fix commit loss * add SimVQ with or without rotation trick https://arxiv.org/abs/2411.02038 * still need comit loss from quantize to input to optimize linear projection in SimVQ, best combination is rotation trick without straight though and without commit loss from input to quantize * allow for an MLP implicitly parameterized codebook instead of just a single Linear, for SimVQ. seems to converge just fine with rotation trick * cleanup * seems to work even better with a one layer mlp * add .indices_to_codes for SimVQ * more efficient to select out the frozen codes and then do projection in `indices_to_codes` for SimVQ * complete residual sim vq * add ResidualSimVQ to the special Sequential module * document the new SimVQ and ResidualSimVQ commit loss weighting for sim vq * allow for arbitrary dimensions into SimVQ and ResidualSimVQ (video and beyond) * fix sim vq autoencoder example * update * Changed implementation of lfq's frac_per_sample_entropy * Update test_lfq.py * 1.20.10 * also run tests on pr * bring back the other half of the commit loss even in the presence of rotation trick, addressing lucidrains#177 * 1.20.11 * Add symmetry-preserving and noise-approximated quantization for FSQ from arxiv:2411.19842. * release the additional research findings for FSQ, out of stability ai, added by @lucasnewman * always practice good credit assignment * do not do straight through nor rotation trick if input does not require grad, to make Genie2 cleaner * Use element-wise selection for noise dropout. * 1.21.2 --------- Co-authored-by: Christopher Fifty <artorius@Christophers-MacBook-Pro-6.local> Co-authored-by: Phil Wang <lucidrains@gmail.com> Co-authored-by: emmettbicker <emmettbicker@gmail.com> Co-authored-by: Lucas Newman <lucas@future.fit>

Author: 5 people