TensorRT8 support (Plugin Migration to IPluginV2 & IPluginV2Ext ) #273

2021-11-10T10:12:22+01:00

perseusdg commented

2021-11-10 10:12:22 +01:00

(Migrated from github.com)

New Features:

Added support for TensorRT 8 through IPluginV2(activation Layers) && IPluginV2Ext(all other plugins) along the lines of how they are in NVIDIA's open source TensorRT repository ,Updated C++ version to 14 and Tested it on both Windows 10(MSVC 16) and Ubuntu-20.04(gcc-9.3 and clang-11) with CUDA-11.3(CUDNN 8.2.1 and TENSORRT 8.0.3) and CUDA-10.2(CUDNN 8.1.1 and TENSORRT 7.2.3) with almost all the networks in the test folder with both INT8 and FP32 (unfortunately my hardware doesn't support FP16)
Updated Docker to CUDA-11.3
Added CUDA arch auto-detect to CMakeLists.txt and option for OpenCV cuda contrib from CMakeLists.txt instead of uncommenting it from DetectionNN.h
Shelfnet now works on cuda 11.x and on windows thanks to @mive93

API Changes:

Modified DetectionNN::init to include
config and name files for darknet based networks (null for centernet and mobilenet)
Plugins are created using a combination of pluginRegistry and directly depending on the type of plugin

Known Issues:

Severe slump in fps ,if opencv is built with QT enabled both on windows(410 native vs 192 qt) and linux(440 gtk vs 330 qt) for yolo4tiny on a gtx 1070
Centernet doesn't work properly on WSL2 (with gpu access)

New Features: - Added support for TensorRT 8 through IPluginV2(activation Layers) && IPluginV2Ext(all other plugins) along the lines of how they are in NVIDIA's open source TensorRT repository ,Updated C++ version to 14 and Tested it on both Windows 10(MSVC 16) and Ubuntu-20.04(gcc-9.3 and clang-11) with CUDA-11.3(CUDNN 8.2.1 and TENSORRT 8.0.3) and CUDA-10.2(CUDNN 8.1.1 and TENSORRT 7.2.3) with almost all the networks in the test folder with both INT8 and FP32 (unfortunately my hardware doesn't support FP16) - Updated Docker to CUDA-11.3 - Added CUDA arch auto-detect to CMakeLists.txt and option for OpenCV cuda contrib from CMakeLists.txt instead of uncommenting it from DetectionNN.h - Shelfnet now works on cuda 11.x and on windows thanks to @mive93 API Changes: - Modified [DetectionNN::init ](https://github.com/perseusdg/tkDNN/blob/tensorrt8/include/tkDNN/DetectionNN.h#L90) to include config and name files for darknet based networks (null for centernet and mobilenet) - Plugins are created using a combination of pluginRegistry and directly depending on the type of plugin Known Issues: - Severe slump in fps ,if opencv is built with QT enabled both on windows(410 native vs 192 qt) and linux(440 gtk vs 330 qt) for yolo4tiny on a gtx 1070 - Centernet doesn't work properly on WSL2 (with gpu access)

👍 1

mive93 commented

2021-11-24 18:30:08 +01:00

(Migrated from github.com)

TensorRT8 is now supported on tensorrt8 branch thanks to this pull request.
It will soon be merged on the master.
Thank you for your work @perseusdg.

TensorRT8 is now supported on [tensorrt8 branch](https://github.com/ceccocats/tkDNN/tree/tensorrt8) thanks to this pull request. It will soon be merged on the master. Thank you for your work @perseusdg.

This repo is archived. You cannot comment on pull requests.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mmr/tkDNN#273