Bug description

hi, I implemented a customer precision with torchao.quantization.Int8WeightOnlyConfig, but unfortunately it looks like customer precision plugin integrated is not allowed in the lightning since the function _convert_precision_to_unified_args in lightning/fabric/connector.py check only the offical implement precision. Does anyone know, how can I implement it?

What version are you seeing the problem on?

v2.5

Reproduced in studio

No response

How to reproduce the bug

Error messages and logs

# Error messages and logs here please 

Environment

#- PyTorch Lightning Version (e.g., 2.5.0): #- PyTorch Version (e.g., 2.5): #- Python version (e.g., 3.12): #- OS (e.g., Linux): #- CUDA/cuDNN version: #- GPU models and configuration: #- How you installed Lightning(`conda`, `pip`, source): 

More info

No response

cc @ethanwharris