Feature request

Support H100 training with FP8 in Trainer and Deepspeed

Motivation

FP8 should be much faster than FP16 on supported Hopper hardware. Particularly with Deepspeed integration @stas00

Your contribution

Happy to help in any way that I can.