Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate TensorRT Version 10 into TensorRT exporter, importer, and forward pass #1799

Merged
merged 2 commits into from
Jan 24, 2025

Conversation

asakhare-zdata
Copy link
Contributor

Added code to do the following:

  1. Handle creation of TensorRT engine using version 10 while maintaining compatibility with previous versions
  2. Handle loading of TensorRT engine using version 10
  3. Handle forward pass of engine created using TensorRT version 10

Largely followed Ultralytics code for handling different TensorRT versions.

Specifically:
https://github.com/ultralytics/ultralytics/blob/3d4a86d2ee35e52b99a65f35126f758078fcccac/ultralytics/nn/autobackend.py#L306
https://github.com/ultralytics/ultralytics/blob/3d4a86d2ee35e52b99a65f35126f758078fcccac/ultralytics/engine/exporter.py#L940

@asakhare-zdata asakhare-zdata changed the title Integrate TensorRT Version 10 with TensoRT exporter, importer, and forward pass Integrate TensorRT Version 10 into TensorRT exporter, importer, and forward pass Jan 23, 2025
@mikel-brostrom
Copy link
Owner

mikel-brostrom commented Jan 24, 2025

Nice @asakhare-zdata! Thank you for your contribution. I do not have a laptop with an nvidia gpu at the moment, so cannot try this out. I will assume you have tried it out and it works?

@asakhare-zdata
Copy link
Contributor Author

Yep, I tried it out on a Jetson AGX Orin and was able to successfully export, load, and call the forward pass on the model.

@mikel-brostrom
Copy link
Owner

Ok, merging. Just to make this available 😊

@mikel-brostrom mikel-brostrom merged commit 000cb0b into mikel-brostrom:master Jan 24, 2025
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants