chore: moving away from tensorrt_bindings

[narendasan]

Description

Moves the trtp feature away from tensorrt_bindings

Fixes # (issue)

Type of change

Please delete options that are not relevant and/or add your own.

• Bug fix (non-breaking change which fixes an issue)

Checklist:

• My code follows the style guidelines of this project (You can use the linters)
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas and hacks
• I have made corresponding changes to the documentation
• I have added tests to verify my fix or my feature
• New and existing unit tests pass locally with my changes
• I have added the relevant labels to my PR in so that relevant reviewers are notified

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/docs/_downloads/0e30a6276601af7e5fc4d5166e2e3d37/torch_compile_advanced_usage.py	2025-01-29 18:03:27.608729+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/0e30a6276601af7e5fc4d5166e2e3d37/torch_compile_advanced_usage.py	2025-01-29 18:03:48.471207+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_advanced_usage: Torch Compile Advanced Usage ====================================================== -This interactive script is intended as an overview of the process by which `torch_tensorrt.compile(..., ir="torch_compile", ...)` works, and how it integrates with the `torch.compile` API.""" +This interactive script is intended as an overview of the process by which `torch_tensorrt.compile(..., ir="torch_compile", ...)` works, and how it integrates with the `torch.compile` API. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/2a9ac10f2667047a7f398d1593b7ca33/torch_export_gpt2.py	2025-01-29 18:03:27.609729+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/2a9ac10f2667047a7f398d1593b7ca33/torch_export_gpt2.py	2025-01-29 18:03:48.487310+00:00 @@ -2,11 +2,12 @@ .. _torch_export_gpt2: Compiling GPT2 using the dynamo backend ========================================================== -This script illustrates Torch-TensorRT workflow with dynamo backend on popular GPT2 model.""" +This script illustrates Torch-TensorRT workflow with dynamo backend on popular GPT2 model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ import torch --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/418941399c146271a7b7728ba3059960/dynamo_compile_resnet_example.py	2025-01-29 18:03:27.609729+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/418941399c146271a7b7728ba3059960/dynamo_compile_resnet_example.py	2025-01-29 18:03:48.524574+00:00 @@ -2,11 +2,12 @@ .. _dynamo_compile_resnet: Compiling ResNet using the Torch-TensorRT Dyanmo Frontend ========================================================== -This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a ResNet model.""" +This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a ResNet model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/3d4d74f6636d986f33167154f6553961/torch_export_cudagraphs.py	2025-01-29 18:03:27.609729+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/3d4d74f6636d986f33167154f6553961/torch_export_cudagraphs.py	2025-01-29 18:03:48.546499+00:00 @@ -2,11 +2,12 @@ .. _torch_export_cudagraphs: Torch Export with Cudagraphs ====================================================== -This interactive script is intended as an overview of the process by which the Torch-TensorRT Cudagraphs integration can be used in the `ir="dynamo"` path. The functionality works similarly in the `torch.compile` path as well.""" +This interactive script is intended as an overview of the process by which the Torch-TensorRT Cudagraphs integration can be used in the `ir="dynamo"` path. The functionality works similarly in the `torch.compile` path as well. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/7b7004dc2ea6f839be532665e16e0426/torch_export_llama2.py	2025-01-29 18:03:27.612729+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/7b7004dc2ea6f839be532665e16e0426/torch_export_llama2.py	2025-01-29 18:03:48.569004+00:00 @@ -2,11 +2,12 @@ .. _torch_export_llama2: Compiling Llama2 using the dynamo backend ========================================================== -This script illustrates Torch-TensorRT workflow with dynamo backend on popular Llama2 model.""" +This script illustrates Torch-TensorRT workflow with dynamo backend on popular Llama2 model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ import torch --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/e1ef5a42560a98a132f56a79d0b66f79/dynamo_compile_advanced_usage.py	2025-01-29 18:03:27.614729+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/e1ef5a42560a98a132f56a79d0b66f79/dynamo_compile_advanced_usage.py	2025-01-29 18:03:48.645966+00:00 @@ -2,11 +2,12 @@ .. _dynamo_compile_advanced_usage: Dynamo Compile Advanced Usage ====================================================== -This interactive script is intended as an overview of the process by which `torch_tensorrt.dynamo.compile` works, and how it integrates with the new `torch.compile` API.""" +This interactive script is intended as an overview of the process by which `torch_tensorrt.dynamo.compile` works, and how it integrates with the new `torch.compile` API. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/dfa60e8f9850fd7761f3e7da81304d32/torch_compile_transformers_example.py	2025-01-29 18:03:27.614729+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/dfa60e8f9850fd7761f3e7da81304d32/torch_compile_transformers_example.py	2025-01-29 18:03:48.649451+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_transformer: Compiling BERT using the `torch.compile` backend ============================================================== -This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a BERT model.""" +This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a BERT model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/d6e1bb6ec5f884994554d9d12e37a0f6/torch_compile_resnet_example.py	2025-01-29 18:03:27.614729+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/d6e1bb6ec5f884994554d9d12e37a0f6/torch_compile_resnet_example.py	2025-01-29 18:03:48.662085+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_resnet: Compiling ResNet with dynamic shapes using the `torch.compile` backend ========================================================== -This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a ResNet model.""" +This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a ResNet model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/e550c5f53cc43e11aa6da8cfb79b54df/dynamo_compile_transformers_example.py	2025-01-29 18:03:27.614729+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/e550c5f53cc43e11aa6da8cfb79b54df/dynamo_compile_transformers_example.py	2025-01-29 18:03:48.692621+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_transformer: Compiling a Transformer using torch.compile and TensorRT ============================================================== -This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a transformer-based model.""" +This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a transformer-based model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/418941399c146271a7b7728ba3059960/dynamo_compile_resnet_example.py	2025-01-29 18:03:28.067727+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/418941399c146271a7b7728ba3059960/dynamo_compile_resnet_example.py	2025-01-29 18:03:48.703456+00:00 @@ -2,11 +2,12 @@ .. _dynamo_compile_resnet: Compiling ResNet using the Torch-TensorRT Dyanmo Frontend ========================================================== -This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a ResNet model.""" +This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a ResNet model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/e1ef5a42560a98a132f56a79d0b66f79/dynamo_compile_advanced_usage.py	2025-01-29 18:03:28.068727+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/e1ef5a42560a98a132f56a79d0b66f79/dynamo_compile_advanced_usage.py	2025-01-29 18:03:48.719402+00:00 @@ -2,11 +2,12 @@ .. _dynamo_compile_advanced_usage: Dynamo Compile Advanced Usage ====================================================== -This interactive script is intended as an overview of the process by which `torch_tensorrt.dynamo.compile` works, and how it integrates with the new `torch.compile` API.""" +This interactive script is intended as an overview of the process by which `torch_tensorrt.dynamo.compile` works, and how it integrates with the new `torch.compile` API. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/e550c5f53cc43e11aa6da8cfb79b54df/dynamo_compile_transformers_example.py	2025-01-29 18:03:28.068727+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/e550c5f53cc43e11aa6da8cfb79b54df/dynamo_compile_transformers_example.py	2025-01-29 18:03:48.739744+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_transformer: Compiling a Transformer using torch.compile and TensorRT ============================================================== -This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a transformer-based model.""" +This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a transformer-based model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_advanced_usage.py	2025-01-29 18:03:28.099727+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_advanced_usage.py	2025-01-29 18:03:48.883707+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_advanced_usage: Torch Compile Advanced Usage ====================================================== -This interactive script is intended as an overview of the process by which `torch_tensorrt.compile(..., ir="torch_compile", ...)` works, and how it integrates with the `torch.compile` API.""" +This interactive script is intended as an overview of the process by which `torch_tensorrt.compile(..., ir="torch_compile", ...)` works, and how it integrates with the `torch.compile` API. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_resnet_example.py	2025-01-29 18:03:28.099727+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_resnet_example.py	2025-01-29 18:03:48.921287+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_resnet: Compiling ResNet with dynamic shapes using the `torch.compile` backend ========================================================== -This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a ResNet model.""" +This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a ResNet model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_cudagraphs.py	2025-01-29 18:03:28.099727+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_cudagraphs.py	2025-01-29 18:03:48.942125+00:00 @@ -2,11 +2,12 @@ .. _torch_export_cudagraphs: Torch Export with Cudagraphs ====================================================== -This interactive script is intended as an overview of the process by which the Torch-TensorRT Cudagraphs integration can be used in the `ir="dynamo"` path. The functionality works similarly in the `torch.compile` path as well.""" +This interactive script is intended as an overview of the process by which the Torch-TensorRT Cudagraphs integration can be used in the `ir="dynamo"` path. The functionality works similarly in the `torch.compile` path as well. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_gpt2.py	2025-01-29 18:03:28.099727+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_gpt2.py	2025-01-29 18:03:48.945614+00:00 @@ -2,11 +2,12 @@ .. _torch_export_gpt2: Compiling GPT2 using the dynamo backend ========================================================== -This script illustrates Torch-TensorRT workflow with dynamo backend on popular GPT2 model.""" +This script illustrates Torch-TensorRT workflow with dynamo backend on popular GPT2 model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ import torch --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_transformers_example.py	2025-01-29 18:03:28.099727+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_transformers_example.py	2025-01-29 18:03:48.957628+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_transformer: Compiling BERT using the `torch.compile` backend ============================================================== -This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a BERT model.""" +This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a BERT model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_llama2.py	2025-01-29 18:03:28.099727+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_llama2.py	2025-01-29 18:03:48.970382+00:00 @@ -2,11 +2,12 @@ .. _torch_export_llama2: Compiling Llama2 using the dynamo backend ========================================================== -This script illustrates Torch-TensorRT workflow with dynamo backend on popular Llama2 model.""" +This script illustrates Torch-TensorRT workflow with dynamo backend on popular Llama2 model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ import torch --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_Input.py	2025-01-29 18:03:28.108727+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_Input.py	2025-01-29 18:03:49.474994+00:00 @@ -259,11 +259,11 @@ else: return False @staticmethod def _parse_tensor_domain( - domain: Optional[Tuple[float, float]] + domain: Optional[Tuple[float, float]], ) -> Tuple[float, float]: """ Produce a tuple of integers which specifies a tensor domain in the interval format: [lo, hi) Args: --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/_TRTBuilderMonitor.py	2025-01-29 18:03:28.110727+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/_TRTBuilderMonitor.py	2025-01-29 18:03:49.705061+00:00 @@ -51,17 +51,17 @@ def _redraw(self, *, blank_lines: int = 0) -> None: if self._render: def clear_line() -> None: - print("\x1B[2K", end="") + print("\x1b[2K", end="") def move_to_start_of_line() -> None: - print("\x1B[0G", end="") + print("\x1b[0G", end="") def move_cursor_up(lines: int) -> None: - print("\x1B[{}A".format(lines), end="") + print("\x1b[{}A".format(lines), end="") def progress_bar(steps: int, num_steps: int) -> str: INNER_WIDTH = 10 completed_bar_chars = int(INNER_WIDTH * steps / float(num_steps)) return "[{}{}]".format( --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_enums.py	2025-01-29 18:03:28.108727+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_enums.py	2025-01-29 18:03:49.842630+00:00 @@ -1198,11 +1198,11 @@ "Provided unsupported source type for EngineCapability conversion" ) @classmethod def try_from( - c: Union[trt.EngineCapability, EngineCapability] + c: Union[trt.EngineCapability, EngineCapability], ) -> Optional[EngineCapability]: """Create a Torch-TensorRT engine capability enum from a TensorRT engine capability enum. Takes a device type enum from tensorrt and create a ``torch_tensorrt.EngineCapability``. If the source is not supported or the engine capability level is not supported in Torch-TensorRT, --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/impl/activation/ops.py	2025-01-29 18:03:28.111727+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/impl/activation/ops.py	2025-01-29 18:03:50.129773+00:00 @@ -245,11 +245,11 @@ beta: float, ) -> TRTTensor: operation_type = trt.ActivationType.HARD_SIGMOID def hard_sigmoid_dyn_range_fn( - dyn_range: Tuple[float, float] + dyn_range: Tuple[float, float], ) -> Tuple[float, float]: def hard_sigmoid_fn(x: float) -> float: return max(0, min(1, alpha * x + beta)) return hard_sigmoid_fn(dyn_range[0]), hard_sigmoid_fn(dyn_range[1]) @@ -308,11 +308,11 @@ alpha: float, ) -> TRTTensor: operation_type = trt.ActivationType.THRESHOLDED_RELU def thresholded_relu_dyn_range_fn( - dyn_range: Tuple[float, float] + dyn_range: Tuple[float, float], ) -> Tuple[float, float]: def thresholded_relu_fn(x: float) -> float: return x if x > alpha else 0 return thresholded_relu_fn(dyn_range[0]), thresholded_relu_fn(dyn_range[1]) --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/utils.py	2025-01-29 18:03:28.115727+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/utils.py	2025-01-29 18:03:51.555347+00:00 @@ -463,11 +463,11 @@ else: return torch.device(device) def to_torch_tensorrt_device( - device: Optional[Union[Device, torch.device, str]] + device: Optional[Union[Device, torch.device, str]], ) -> Device: """Cast a device-type to torch_tensorrt.Device Returns the corresponding torch_tensorrt.Device """ --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/fx/test/converters/acc_op/test_where.py	2025-01-29 18:03:28.120727+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/fx/test/converters/acc_op/test_where.py	2025-01-29 18:03:52.537650+00:00 @@ -99,11 +99,11 @@ self.y = torch.ones(y_shape) def forward(self, condition): return torch.where(condition, self.x, self.y) - inputs = [(torch.randn(condition_shape) > 0)] + inputs = [torch.randn(condition_shape) > 0] self.run_test( Where(x_shape, y_shape), inputs, expected_ops={acc_ops.where}, test_implicit_batch_dim=False, --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/fx/tracer/acc_tracer/acc_tracer.py	2025-01-29 18:03:28.124727+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/fx/tracer/acc_tracer/acc_tracer.py	2025-01-29 18:03:53.632668+00:00 @@ -515,11 +515,11 @@ dim0 = cast(int, transpose_node.args[1]) dim1 = cast(int, transpose_node.args[2]) changed = False def _calculate_dim( - transpose_dim: Union[torch.fx.Node, int] + transpose_dim: Union[torch.fx.Node, int], ) -> Union[torch.fx.Node, int]: nonlocal transpose_input_node nonlocal changed if isinstance(transpose_dim, torch.fx.Node): # Transpose dim is sub node

[github-actions]

There are some changes that do not conform to Python style guidelines:

--- /home/runner/work/TensorRT/TensorRT/docs/_downloads/0e30a6276601af7e5fc4d5166e2e3d37/torch_compile_advanced_usage.py	2025-01-29 23:31:29.161675+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/0e30a6276601af7e5fc4d5166e2e3d37/torch_compile_advanced_usage.py	2025-01-29 23:31:49.060139+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_advanced_usage: Torch Compile Advanced Usage ====================================================== -This interactive script is intended as an overview of the process by which `torch_tensorrt.compile(..., ir="torch_compile", ...)` works, and how it integrates with the `torch.compile` API.""" +This interactive script is intended as an overview of the process by which `torch_tensorrt.compile(..., ir="torch_compile", ...)` works, and how it integrates with the `torch.compile` API. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/2a9ac10f2667047a7f398d1593b7ca33/torch_export_gpt2.py	2025-01-29 23:31:29.161675+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/2a9ac10f2667047a7f398d1593b7ca33/torch_export_gpt2.py	2025-01-29 23:31:49.075480+00:00 @@ -2,11 +2,12 @@ .. _torch_export_gpt2: Compiling GPT2 using the dynamo backend ========================================================== -This script illustrates Torch-TensorRT workflow with dynamo backend on popular GPT2 model.""" +This script illustrates Torch-TensorRT workflow with dynamo backend on popular GPT2 model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ import torch --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/418941399c146271a7b7728ba3059960/dynamo_compile_resnet_example.py	2025-01-29 23:31:29.162675+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/418941399c146271a7b7728ba3059960/dynamo_compile_resnet_example.py	2025-01-29 23:31:49.114114+00:00 @@ -2,11 +2,12 @@ .. _dynamo_compile_resnet: Compiling ResNet using the Torch-TensorRT Dyanmo Frontend ========================================================== -This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a ResNet model.""" +This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a ResNet model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/3d4d74f6636d986f33167154f6553961/torch_export_cudagraphs.py	2025-01-29 23:31:29.162675+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/3d4d74f6636d986f33167154f6553961/torch_export_cudagraphs.py	2025-01-29 23:31:49.131386+00:00 @@ -2,11 +2,12 @@ .. _torch_export_cudagraphs: Torch Export with Cudagraphs ====================================================== -This interactive script is intended as an overview of the process by which the Torch-TensorRT Cudagraphs integration can be used in the `ir="dynamo"` path. The functionality works similarly in the `torch.compile` path as well.""" +This interactive script is intended as an overview of the process by which the Torch-TensorRT Cudagraphs integration can be used in the `ir="dynamo"` path. The functionality works similarly in the `torch.compile` path as well. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/7b7004dc2ea6f839be532665e16e0426/torch_export_llama2.py	2025-01-29 23:31:29.164675+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/7b7004dc2ea6f839be532665e16e0426/torch_export_llama2.py	2025-01-29 23:31:49.160175+00:00 @@ -2,11 +2,12 @@ .. _torch_export_llama2: Compiling Llama2 using the dynamo backend ========================================================== -This script illustrates Torch-TensorRT workflow with dynamo backend on popular Llama2 model.""" +This script illustrates Torch-TensorRT workflow with dynamo backend on popular Llama2 model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ import torch --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/e1ef5a42560a98a132f56a79d0b66f79/dynamo_compile_advanced_usage.py	2025-01-29 23:31:29.166675+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/e1ef5a42560a98a132f56a79d0b66f79/dynamo_compile_advanced_usage.py	2025-01-29 23:31:49.238267+00:00 @@ -2,11 +2,12 @@ .. _dynamo_compile_advanced_usage: Dynamo Compile Advanced Usage ====================================================== -This interactive script is intended as an overview of the process by which `torch_tensorrt.dynamo.compile` works, and how it integrates with the new `torch.compile` API.""" +This interactive script is intended as an overview of the process by which `torch_tensorrt.dynamo.compile` works, and how it integrates with the new `torch.compile` API. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/dfa60e8f9850fd7761f3e7da81304d32/torch_compile_transformers_example.py	2025-01-29 23:31:29.166675+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/dfa60e8f9850fd7761f3e7da81304d32/torch_compile_transformers_example.py	2025-01-29 23:31:49.242381+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_transformer: Compiling BERT using the `torch.compile` backend ============================================================== -This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a BERT model.""" +This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a BERT model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/d6e1bb6ec5f884994554d9d12e37a0f6/torch_compile_resnet_example.py	2025-01-29 23:31:29.166675+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/d6e1bb6ec5f884994554d9d12e37a0f6/torch_compile_resnet_example.py	2025-01-29 23:31:49.246477+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_resnet: Compiling ResNet with dynamic shapes using the `torch.compile` backend ========================================================== -This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a ResNet model.""" +This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a ResNet model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/418941399c146271a7b7728ba3059960/dynamo_compile_resnet_example.py	2025-01-29 23:31:29.614678+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/418941399c146271a7b7728ba3059960/dynamo_compile_resnet_example.py	2025-01-29 23:31:49.285262+00:00 @@ -2,11 +2,12 @@ .. _dynamo_compile_resnet: Compiling ResNet using the Torch-TensorRT Dyanmo Frontend ========================================================== -This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a ResNet model.""" +This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a ResNet model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/_downloads/e550c5f53cc43e11aa6da8cfb79b54df/dynamo_compile_transformers_example.py	2025-01-29 23:31:29.167675+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/_downloads/e550c5f53cc43e11aa6da8cfb79b54df/dynamo_compile_transformers_example.py	2025-01-29 23:31:49.290465+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_transformer: Compiling a Transformer using torch.compile and TensorRT ============================================================== -This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a transformer-based model.""" +This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a transformer-based model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/e1ef5a42560a98a132f56a79d0b66f79/dynamo_compile_advanced_usage.py	2025-01-29 23:31:29.614678+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/e1ef5a42560a98a132f56a79d0b66f79/dynamo_compile_advanced_usage.py	2025-01-29 23:31:49.311202+00:00 @@ -2,11 +2,12 @@ .. _dynamo_compile_advanced_usage: Dynamo Compile Advanced Usage ====================================================== -This interactive script is intended as an overview of the process by which `torch_tensorrt.dynamo.compile` works, and how it integrates with the new `torch.compile` API.""" +This interactive script is intended as an overview of the process by which `torch_tensorrt.dynamo.compile` works, and how it integrates with the new `torch.compile` API. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/e550c5f53cc43e11aa6da8cfb79b54df/dynamo_compile_transformers_example.py	2025-01-29 23:31:29.614678+00:00 +++ /home/runner/work/TensorRT/TensorRT/docs/v1.4.0/_downloads/e550c5f53cc43e11aa6da8cfb79b54df/dynamo_compile_transformers_example.py	2025-01-29 23:31:49.329939+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_transformer: Compiling a Transformer using torch.compile and TensorRT ============================================================== -This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a transformer-based model.""" +This interactive script is intended as a sample of the `torch_tensorrt.dynamo.compile` workflow on a transformer-based model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_advanced_usage.py	2025-01-29 23:31:29.644679+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_advanced_usage.py	2025-01-29 23:31:49.486229+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_advanced_usage: Torch Compile Advanced Usage ====================================================== -This interactive script is intended as an overview of the process by which `torch_tensorrt.compile(..., ir="torch_compile", ...)` works, and how it integrates with the `torch.compile` API.""" +This interactive script is intended as an overview of the process by which `torch_tensorrt.compile(..., ir="torch_compile", ...)` works, and how it integrates with the `torch.compile` API. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_transformers_example.py	2025-01-29 23:31:29.644679+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_transformers_example.py	2025-01-29 23:31:49.514666+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_transformer: Compiling BERT using the `torch.compile` backend ============================================================== -This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a BERT model.""" +This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a BERT model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_resnet_example.py	2025-01-29 23:31:29.644679+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_compile_resnet_example.py	2025-01-29 23:31:49.526415+00:00 @@ -2,11 +2,12 @@ .. _torch_compile_resnet: Compiling ResNet with dynamic shapes using the `torch.compile` backend ========================================================== -This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a ResNet model.""" +This interactive script is intended as a sample of the Torch-TensorRT workflow with `torch.compile` on a ResNet model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_cudagraphs.py	2025-01-29 23:31:29.644679+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_cudagraphs.py	2025-01-29 23:31:49.536797+00:00 @@ -2,11 +2,12 @@ .. _torch_export_cudagraphs: Torch Export with Cudagraphs ====================================================== -This interactive script is intended as an overview of the process by which the Torch-TensorRT Cudagraphs integration can be used in the `ir="dynamo"` path. The functionality works similarly in the `torch.compile` path as well.""" +This interactive script is intended as an overview of the process by which the Torch-TensorRT Cudagraphs integration can be used in the `ir="dynamo"` path. The functionality works similarly in the `torch.compile` path as well. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_gpt2.py	2025-01-29 23:31:29.644679+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_gpt2.py	2025-01-29 23:31:49.536932+00:00 @@ -2,11 +2,12 @@ .. _torch_export_gpt2: Compiling GPT2 using the dynamo backend ========================================================== -This script illustrates Torch-TensorRT workflow with dynamo backend on popular GPT2 model.""" +This script illustrates Torch-TensorRT workflow with dynamo backend on popular GPT2 model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ import torch --- /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_llama2.py	2025-01-29 23:31:29.644679+00:00 +++ /home/runner/work/TensorRT/TensorRT/examples/dynamo/torch_export_llama2.py	2025-01-29 23:31:49.555620+00:00 @@ -2,11 +2,12 @@ .. _torch_export_llama2: Compiling Llama2 using the dynamo backend ========================================================== -This script illustrates Torch-TensorRT workflow with dynamo backend on popular Llama2 model.""" +This script illustrates Torch-TensorRT workflow with dynamo backend on popular Llama2 model. +""" # %% # Imports and Model Definition # ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ import torch --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_Input.py	2025-01-29 23:31:29.653679+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_Input.py	2025-01-29 23:31:50.025028+00:00 @@ -259,11 +259,11 @@ else: return False @staticmethod def _parse_tensor_domain( - domain: Optional[Tuple[float, float]] + domain: Optional[Tuple[float, float]], ) -> Tuple[float, float]: """ Produce a tuple of integers which specifies a tensor domain in the interval format: [lo, hi) Args: --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/_TRTBuilderMonitor.py	2025-01-29 23:31:29.655679+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/_TRTBuilderMonitor.py	2025-01-29 23:31:50.269215+00:00 @@ -51,17 +51,17 @@ def _redraw(self, *, blank_lines: int = 0) -> None: if self._render: def clear_line() -> None: - print("\x1B[2K", end="") + print("\x1b[2K", end="") def move_to_start_of_line() -> None: - print("\x1B[0G", end="") + print("\x1b[0G", end="") def move_cursor_up(lines: int) -> None: - print("\x1B[{}A".format(lines), end="") + print("\x1b[{}A".format(lines), end="") def progress_bar(steps: int, num_steps: int) -> str: INNER_WIDTH = 10 completed_bar_chars = int(INNER_WIDTH * steps / float(num_steps)) return "[{}{}]".format( --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_enums.py	2025-01-29 23:31:29.654679+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/_enums.py	2025-01-29 23:31:50.495607+00:00 @@ -1198,11 +1198,11 @@ "Provided unsupported source type for EngineCapability conversion" ) @classmethod def try_from( - c: Union[trt.EngineCapability, EngineCapability] + c: Union[trt.EngineCapability, EngineCapability], ) -> Optional[EngineCapability]: """Create a Torch-TensorRT engine capability enum from a TensorRT engine capability enum. Takes a device type enum from tensorrt and create a ``torch_tensorrt.EngineCapability``. If the source is not supported or the engine capability level is not supported in Torch-TensorRT, --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/impl/activation/ops.py	2025-01-29 23:31:29.656679+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/conversion/impl/activation/ops.py	2025-01-29 23:31:50.670244+00:00 @@ -245,11 +245,11 @@ beta: float, ) -> TRTTensor: operation_type = trt.ActivationType.HARD_SIGMOID def hard_sigmoid_dyn_range_fn( - dyn_range: Tuple[float, float] + dyn_range: Tuple[float, float], ) -> Tuple[float, float]: def hard_sigmoid_fn(x: float) -> float: return max(0, min(1, alpha * x + beta)) return hard_sigmoid_fn(dyn_range[0]), hard_sigmoid_fn(dyn_range[1]) @@ -308,11 +308,11 @@ alpha: float, ) -> TRTTensor: operation_type = trt.ActivationType.THRESHOLDED_RELU def thresholded_relu_dyn_range_fn( - dyn_range: Tuple[float, float] + dyn_range: Tuple[float, float], ) -> Tuple[float, float]: def thresholded_relu_fn(x: float) -> float: return x if x > alpha else 0 return thresholded_relu_fn(dyn_range[0]), thresholded_relu_fn(dyn_range[1]) --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/utils.py	2025-01-29 23:31:29.660679+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/dynamo/utils.py	2025-01-29 23:31:52.131275+00:00 @@ -463,11 +463,11 @@ else: return torch.device(device) def to_torch_tensorrt_device( - device: Optional[Union[Device, torch.device, str]] + device: Optional[Union[Device, torch.device, str]], ) -> Device: """Cast a device-type to torch_tensorrt.Device Returns the corresponding torch_tensorrt.Device """ --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/fx/test/converters/acc_op/test_where.py	2025-01-29 23:31:29.665679+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/fx/test/converters/acc_op/test_where.py	2025-01-29 23:31:53.080883+00:00 @@ -99,11 +99,11 @@ self.y = torch.ones(y_shape) def forward(self, condition): return torch.where(condition, self.x, self.y) - inputs = [(torch.randn(condition_shape) > 0)] + inputs = [torch.randn(condition_shape) > 0] self.run_test( Where(x_shape, y_shape), inputs, expected_ops={acc_ops.where}, test_implicit_batch_dim=False, --- /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/fx/tracer/acc_tracer/acc_tracer.py	2025-01-29 23:31:29.669679+00:00 +++ /home/runner/work/TensorRT/TensorRT/py/torch_tensorrt/fx/tracer/acc_tracer/acc_tracer.py	2025-01-29 23:31:54.067037+00:00 @@ -515,11 +515,11 @@ dim0 = cast(int, transpose_node.args[1]) dim1 = cast(int, transpose_node.args[2]) changed = False def _calculate_dim( - transpose_dim: Union[torch.fx.Node, int] + transpose_dim: Union[torch.fx.Node, int], ) -> Union[torch.fx.Node, int]: nonlocal transpose_input_node nonlocal changed if isinstance(transpose_dim, torch.fx.Node): # Transpose dim is sub node