a simple pipline of int8 quantization based on tensorrt.
classification detection
cd classification wget https://hanlab.mit.edu/files/OnceForAll/ofa_cvpr_tutorial/imagenet_1k.zip unzip 'imagenet_1k.zip' mkdir model python test_torch.py python torch2onnx.py python quantization.py python test_int8trt.py or run a pipline including the above steps.
python tensorrt_PTA_classification_pipline.py 
| model | accuracy | time | size |
|---|---|---|---|
| float32(pth) | 0.759 | 0.0799 | 171M |
| int8(trt) | 0.738 | 0.0013 | 44M |
You can replace resnet101 with your network. If your dataset structure is different, you need to modify some code about dataset.
# test_torch.py torch2onnx.py quantization.py if __name__ == "__main__": net = models.resnet101(pretrained=True).to('cpu') or
# tensorrt_PTA_classification_pipline.py if __name__ == "__main__": net = models.resnet101(pretrained=True).to('cpu') cd detection wget wget https://github.com/Megvii-BaseDetection/storage/releases/download/0.0.1/yolox_s.onnx python demo_onnx.py --model_path yolox_s.onnx --label_name_path coco.label --image_path dog.jpg --output_path output_onnx.jpg mkdir calibration python sample.py --traing_data_path your_path/coco/images/train2017/ --count 2000 --calibration_path ./calibration/ python3 -m onnxsim yolox_s.onnx yolox_s.onnx python quantization.py python demo_trt.py --model_path modelInt8.engine --label_name_path coco.label --image_path dog.jpg --output_path output_trt.jpg | model | time | size |
|---|---|---|
| float32(pth) | 0.0064 | 35M |
| int8(trt) | 0.0025 | 9.2M |
| float32 onnx | int8 tensorrt |
|---|---|
 |  |
- example for detection.
- example for segmention.