onnx model problem - cudaActivationRunner.cpp (96)

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I try to infere an ONNX model via tensorrt-6 c-api but I get this error:

[GIE] …/rtSafe/cuda/cudaActivationRunner.cpp (96) - Cudnn Error in execute: 3 (CUDNN_STATUS_BAD_PARAM)
[GIE] FAILED_EXECUTION: std::exception
[GIE] TensorRTHandle::apply() – failed to execute tensorRT context

Any idea to resolve this problem?
model0.onnx.zip (7.61 MB)

2

trtexec of 5.1.5 can run the given onnx model but trtexec of 6.0.1.5 fails.

3

Here is the output of trtexec on trt6:

trtexec --onnx=model0.onnx
&&&& RUNNING TensorRT.trtexec # trtexec --onnx=model0.onnx
[00/13/2020-11:41:34] [I] === Model Options ===
[00/13/2020-11:41:34] [I] Format: ONNX
[00/13/2020-11:41:34] [I] Model: model0.onnx
[00/13/2020-11:41:34] [I] Output:
[00/13/2020-11:41:34] [I] === Build Options ===
[00/13/2020-11:41:34] [I] Max batch: 1
[00/13/2020-11:41:34] [I] Workspace: 16 MB
[00/13/2020-11:41:34] [I] minTiming: 1
[00/13/2020-11:41:34] [I] avgTiming: 8
[00/13/2020-11:41:34] [I] Precision: FP32
[00/13/2020-11:41:34] [I] Calibration:
[00/13/2020-11:41:34] [I] Safe mode: Disabled
[00/13/2020-11:41:34] [I] Save engine:
[00/13/2020-11:41:34] [I] Load engine:
[00/13/2020-11:41:34] [I] Inputs format: fp32:CHW
[00/13/2020-11:41:34] [I] Outputs format: fp32:CHW
[00/13/2020-11:41:34] [I] Input build shapes: model
[00/13/2020-11:41:34] [I] === System Options ===
[00/13/2020-11:41:34] [I] Device: 0
[00/13/2020-11:41:34] [I] DLACore:
[00/13/2020-11:41:34] [I] Plugins:
[00/13/2020-11:41:34] [I] === Inference Options ===
[00/13/2020-11:41:34] [I] Batch: 1
[00/13/2020-11:41:34] [I] Iterations: 10 (200 ms warm up)
[00/13/2020-11:41:34] [I] Inputs:
[00/13/2020-11:41:34] [I] Duration: 3s
[00/13/2020-11:41:34] [I] Sleep time: 0ms
[00/13/2020-11:41:34] [I] Streams: 1
[00/13/2020-11:41:34] [I] Spin-wait: Disabled
[00/13/2020-11:41:34] [I] Multithreading: Disabled
[00/13/2020-11:41:34] [I] CUDA Graph: Disabled
[00/13/2020-11:41:34] [I] Skip inference: Disabled
[00/13/2020-11:41:34] [I] Input inference shapes: model
[00/13/2020-11:41:34] [I] === Reporting Options ===
[00/13/2020-11:41:34] [I] Verbose: Disabled
[00/13/2020-11:41:34] [I] Averages: 10 inferences
[00/13/2020-11:41:34] [I] Percentile: 99
[00/13/2020-11:41:34] [I] Dump output: Disabled
[00/13/2020-11:41:34] [I] Profile: Disabled
[00/13/2020-11:41:34] [I] Export timing to JSON file:
[00/13/2020-11:41:34] [I] Export output to JSON file:
[00/13/2020-11:41:34] [I] Export profile to JSON file:
[00/13/2020-11:41:34] [I]

Input filename: model0.onnx
ONNX IR version: 0.0.4
Opset version: 9
Producer name: pytorch
Producer version: 1.2
Domain:
Model version: 0
Doc string:

WARNING: ONNX model has a newer ir_version (0.0.4) than this parser was built against (0.0.3).
[00/13/2020-11:41:35] [I] [TRT] 566:Conv → (64, 128, 64)
[00/13/2020-11:41:35] [I] [TRT] 567:BatchNormalization → (64, 128, 64)
[00/13/2020-11:41:35] [I] [TRT] 568:Relu → (64, 128, 64)
[00/13/2020-11:41:35] [I] [TRT] 569:MaxPool → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 570:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 571:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 572:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 573:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 574:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 575:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 576:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 577:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 578:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 579:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 580:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 581:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 582:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 583:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 584:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 585:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 586:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 587:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 588:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 589:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 590:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 591:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 592:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 593:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 594:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 595:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 596:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 597:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 598:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 599:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 600:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 601:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 602:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 603:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 604:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 605:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 606:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 607:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 608:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 609:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 610:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 611:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 612:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 613:GlobalAveragePool → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 614:Conv → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 615:Relu → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 616:Conv → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 617:Sigmoid → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 618:Mul → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 619:GlobalAveragePool → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 620:Conv → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 621:Relu → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 622:Conv → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 623:Sigmoid → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 624:Mul → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 625:Add → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 626:GlobalAveragePool → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 627:Conv → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 628:Relu → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 629:Conv → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 630:Sigmoid → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 631:Mul → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 632:Add → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 633:GlobalAveragePool → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 634:Conv → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 635:Relu → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 636:Conv → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 637:Sigmoid → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 638:Mul → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 639:Add → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 640:Conv → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 641:BatchNormalization → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 642:Conv → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 643:BatchNormalization → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 644:Add → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 645:Relu → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 646:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 647:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 648:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 649:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 650:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 651:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 652:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 653:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 654:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 655:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 656:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 657:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 658:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 659:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 660:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 661:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 662:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 663:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 664:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 665:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 666:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 667:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 668:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 669:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 670:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 671:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 672:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 673:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 674:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 675:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 676:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 677:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 678:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 679:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 680:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 681:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 682:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 683:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 684:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 685:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 686:Conv → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 687:BatchNormalization → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 688:Relu → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 689:GlobalAveragePool → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 690:Conv → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 691:Relu → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 692:Conv → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 693:Sigmoid → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 694:Mul → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 695:GlobalAveragePool → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 696:Conv → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 697:Relu → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 698:Conv → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 699:Sigmoid → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 700:Mul → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 701:Add → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 702:GlobalAveragePool → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 703:Conv → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 704:Relu → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 705:Conv → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 706:Sigmoid → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 707:Mul → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 708:Add → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 709:GlobalAveragePool → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 710:Conv → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 711:Relu → (4, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 712:Conv → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 713:Sigmoid → (64, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 714:Mul → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 715:Add → (64, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 716:Conv → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 717:BatchNormalization → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 718:Add → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 719:Relu → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 720:Conv → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 721:BatchNormalization → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 722:Relu → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 723:Pad → (256, 64, 32)
[00/13/2020-11:41:35] [I] [TRT] 724:AveragePool → (256, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 725:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 726:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 727:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 728:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 729:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 730:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 731:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 732:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 733:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 734:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 735:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 736:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 737:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 738:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 739:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 740:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 741:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 742:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 743:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 744:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 745:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 746:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 747:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 748:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 749:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 750:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 751:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 752:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 753:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 754:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 755:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 756:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 757:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 758:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 759:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 760:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 761:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 762:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 763:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 764:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 765:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 766:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 767:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 768:GlobalAveragePool → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 769:Conv → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 770:Relu → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 771:Conv → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 772:Sigmoid → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 773:Mul → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 774:GlobalAveragePool → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 775:Conv → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 776:Relu → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 777:Conv → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 778:Sigmoid → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 779:Mul → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 780:Add → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 781:GlobalAveragePool → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 782:Conv → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 783:Relu → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 784:Conv → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 785:Sigmoid → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 786:Mul → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 787:Add → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 788:GlobalAveragePool → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 789:Conv → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 790:Relu → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 791:Conv → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 792:Sigmoid → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 793:Mul → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 794:Add → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 795:Conv → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 796:BatchNormalization → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 797:Conv → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 798:BatchNormalization → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 799:Add → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 800:Relu → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 801:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 802:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 803:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 804:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 805:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 806:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 807:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 808:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 809:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 810:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 811:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 812:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 813:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 814:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 815:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 816:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 817:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 818:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 819:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 820:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 821:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 822:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 823:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 824:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 825:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 826:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 827:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 828:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 829:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 830:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 831:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 832:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 833:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 834:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 835:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 836:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 837:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 838:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 839:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 840:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 841:Conv → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 842:BatchNormalization → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 843:Relu → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 844:GlobalAveragePool → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 845:Conv → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 846:Relu → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 847:Conv → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 848:Sigmoid → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 849:Mul → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 850:GlobalAveragePool → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 851:Conv → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 852:Relu → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 853:Conv → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 854:Sigmoid → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 855:Mul → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 856:Add → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 857:GlobalAveragePool → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 858:Conv → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 859:Relu → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 860:Conv → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 861:Sigmoid → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 862:Mul → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 863:Add → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 864:GlobalAveragePool → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 865:Conv → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 866:Relu → (6, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 867:Conv → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 868:Sigmoid → (96, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 869:Mul → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 870:Add → (96, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 871:Conv → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 872:BatchNormalization → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 873:Add → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 874:Relu → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 875:Conv → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 876:BatchNormalization → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 877:Relu → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 878:Pad → (384, 32, 16)
[00/13/2020-11:41:35] [I] [TRT] 879:AveragePool → (384, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 880:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 881:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 882:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 883:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 884:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 885:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 886:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 887:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 888:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 889:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 890:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 891:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 892:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 893:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 894:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 895:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 896:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 897:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 898:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 899:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 900:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 901:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 902:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 903:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 904:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 905:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 906:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 907:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 908:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 909:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 910:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 911:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 912:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 913:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 914:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 915:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 916:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 917:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 918:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 919:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 920:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 921:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 922:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 923:GlobalAveragePool → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 924:Conv → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 925:Relu → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 926:Conv → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 927:Sigmoid → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 928:Mul → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 929:GlobalAveragePool → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 930:Conv → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 931:Relu → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 932:Conv → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 933:Sigmoid → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 934:Mul → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 935:Add → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 936:GlobalAveragePool → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 937:Conv → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 938:Relu → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 939:Conv → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 940:Sigmoid → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 941:Mul → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 942:Add → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 943:GlobalAveragePool → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 944:Conv → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 945:Relu → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 946:Conv → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 947:Sigmoid → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 948:Mul → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 949:Add → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 950:Conv → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 951:BatchNormalization → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 952:Conv → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 953:BatchNormalization → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 954:Add → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 955:Relu → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 956:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 957:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 958:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 959:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 960:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 961:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 962:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 963:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 964:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 965:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 966:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 967:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 968:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 969:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 970:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 971:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 972:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 973:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 974:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 975:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 976:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 977:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 978:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 979:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 980:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 981:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 982:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 983:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 984:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 985:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 986:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 987:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 988:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 989:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 990:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 991:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 992:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 993:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 994:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 995:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 996:Conv → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 997:BatchNormalization → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 998:Relu → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 999:GlobalAveragePool → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1000:Conv → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1001:Relu → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1002:Conv → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1003:Sigmoid → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1004:Mul → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1005:GlobalAveragePool → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1006:Conv → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1007:Relu → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1008:Conv → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1009:Sigmoid → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1010:Mul → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1011:Add → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1012:GlobalAveragePool → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1013:Conv → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1014:Relu → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1015:Conv → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1016:Sigmoid → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1017:Mul → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1018:Add → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1019:GlobalAveragePool → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1020:Conv → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1021:Relu → (8, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1022:Conv → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1023:Sigmoid → (128, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1024:Mul → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1025:Add → (128, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1026:Conv → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1027:BatchNormalization → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1028:Add → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1029:Relu → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1030:Conv → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1031:BatchNormalization → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1032:Relu → (512, 16, 8)
[00/13/2020-11:41:35] [I] [TRT] 1033:GlobalAveragePool → (512, 1, 1)
[00/13/2020-11:41:35] [I] [TRT] 1034:Flatten → (512)
[00/13/2020-11:41:35] [I] [TRT] 1035:Gemm → (512)
[00/13/2020-11:41:35] [E] [TRT] (Unnamed Layer* 473) [Shuffle]: uninferred dimensions are not an exact divisor of input dimensions, so inferred dimension cannot be calculated
While parsing node number 470 [Unsqueeze → “1036”]:
3
— Begin node —
input: “1035”
output: “1036”
op_type: “Unsqueeze”
attribute {
name: “axes”
ints: 2
type: INTS
}

— End node —
ERROR: /workspace/TensorRT/parsers/onnx/builtin_op_importers.cpp:2055 In function importUnsqueeze:
[8] Assertion failed: get_shape_size(layer->getOutput(0)->getDimensions()) == get_shape_size(old_shape)
[00/13/2020-11:41:35] [E] Failed to parse onnx file
[00/13/2020-11:41:35] [E] Parsing model failed
[00/13/2020-11:41:35] [E] Engine creation failed
[00/13/2020-11:41:35] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # trtexec --onnx=model0.onnx

4

Hi bozkalayci,

Just tried your model with TensorRT 7.0 and it worked for me:

$ dpkg -l | grep nvinfer ... ii libnvinfer7 7.0.0-1+cuda10.2 amd64 TensorRT runtime libraries $ trtexec --onnx=model0.onnx --explicitBatch ... &&&& PASSED TensorRT.trtexec # trtexec --onnx=model0.onnx --explicitBatch

But it failed for TensorrRT 6.0:

$ nvidia-docker run -it -v ${PWD}:/mnt --workdir=/mnt nvcr.io/nvidia/tensorrt:19.12-py3 $ trtexec --onnx=model0.onnx --explicitBatch ... [00/13/2020-21:46:09] [E] [TRT] (Unnamed Layer* 0) [Convolution]: at least 4 dimensions are required for input While parsing node number 1 [BatchNormalization]: ERROR: builtin_op_importers.cpp:695 In function importBatchNormalization: [6] Assertion failed: scale_weights.shape == weights_shape [00/13/2020-21:46:09] [E] Failed to parse onnx file [00/13/2020-21:46:09] [E] Parsing model failed [00/13/2020-21:46:09] [E] Engine could not be created &&&& FAILED TensorRT.trtexec # trtexec --onnx=model0.onnx --explicitBatch

But if you build the OSS ONNX parser (19.12 tag for TRT6) on top of TensorRT 6, it works, so this was likely found and fixed in upstream ONNX parser after the release:

$ nvidia-docker run -it -v ${PWD}:/mnt --workdir=/mnt nvcr.io/nvidia/tensorrt:19.12-py3 $ bash /usr/src/tensorrt/install_opensource.sh ... Done! Removing outdated ONNX python samples... Done! Copying over prerequisite files for building OSS samples to /workspace/tensorrt... Done! Replacing samples with open source versions and copying back python samples... Done! Cleaning up OSS directories... Done! $ trtexec --onnx=model0.onnx --explicitBatch ... &&&& PASSED TensorRT.trtexec # trtexec --onnx=model0.onnx --explicitBatch

I believe this script (/usr/src/tensorrt/install_opensource.sh) comes with the NGC containers as of 19.12-py3, I don’t think it comes with TRT6 release code.

You can copy the script from the container, or if you don’t have docker installed then you can find a similar script here: tensorrt-utils/OSS at master · rmccorm4/tensorrt-utils · GitHub