What is default optimization profile of an engine?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I have created an engine file using the command

../TensorRT-10.3.0.26/bin/trtexec --onnx=<onnx_name>.onnx --saveEngine=<engine_name>.engine --tempdir=./temp

When I am inferencing using this engine file in a c++ program, I am getting the following error.

After building the engine I checked the number of optimization profiles in it, everything seems okay

image

But before execution, i am getting this error from TensorRT v10.3. I am using executeV2() api
mExecutionContext->executeV2(mModelBindings.data());

image

2

*** Please note that this reply is generated by LLM automatically ***

The error you’re encountering is likely due to the fact that the executeV2 API requires an array of buffers, and the engine bindings map from tensor names to indices in this array. However, the binding indices are assigned at ICudaEngine build time and may not match the indices in your mModelBindings array.

To resolve this issue, you need to ensure that the binding indices in your mModelBindings array match the binding indices assigned by the ICudaEngine. You can do this by using the get_binding_index method of the ICudaEngine class to retrieve the binding index for each tensor name.

Here’s an example of how you can do this:

// Assume that mEngine is a valid ICudaEngine object // and mModelBindings is an array of buffers // Get the number of bindings int nbBindings = mEngine->getNbBindings(); // Create an array to store the binding indices int* bindingIndices = new int[nbBindings]; // Iterate over the bindings and retrieve the binding index for each tensor name for (int i = 0; i < nbBindings; i++) { const char* tensorName = mEngine->getBindingName(i); int bindingIndex = mEngine->getBindingIndex(tensorName); bindingIndices[i] = bindingIndex; } // Use the bindingIndices array to access the correct buffers in mModelBindings mExecutionContext->executeV2(mModelBindings, bindingIndices);

Alternatively, you can use the get_binding_name method of the ICudaEngine class to retrieve the tensor name for each binding index, and then use the get_binding_index method to retrieve the binding index for each tensor name.

It’s also worth noting that the executeV2 API requires the mModelBindings array to be in the correct order, which is the order of the binding indices assigned by the ICudaEngine. If the mModelBindings array is not in the correct order, you may need to reorder it to match the binding indices.

In comparison to the execute API, the executeV2 API provides more flexibility and control over the execution process, but it also requires more manual management of the binding indices and buffers.

In terms of the error message you’re seeing, it’s likely due to the fact that the executeV2 API is trying to access a buffer at an invalid index, which is causing the error. By ensuring that the binding indices and buffers are correctly aligned, you should be able to resolve the error and successfully execute the engine using the executeV2 API.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***