From d6c28c5ba2654c4e80276bc83b7cfe1da9237d20 Mon Sep 17 00:00:00 2001 From: Francesco Gatti Date: Wed, 29 Apr 2020 18:09:50 +0200 Subject: [PATCH 1/3] Update README.md --- README.md | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index a06a2b8..0f6b607 100644 --- a/README.md +++ b/README.md @@ -114,9 +114,9 @@ python run_ssd_live_demo.py mb2-ssd-lite To run the an object detection demo follow these steps (example with yolov3): ``` -rm yolo3_FP32.rt # be sure to delete(or move) old tensorRT files +rm yolo3_fp32.rt # be sure to delete(or move) old tensorRT files ./test_yolo3 # run the yolo test (is slow) -./demo yolo3_FP32.rt ../demo/yolo_test.mp4 y +./demo yolo3_fp32.rt ../demo/yolo_test.mp4 y ``` In general the demo program takes 4 parameters: ``` @@ -136,9 +136,9 @@ N.b. By default it is used FP32 inference To run the an object detection demo with FP16 inference follow these steps (example with yolov3): ``` export TKDNN_MODE=FP16 # set the half floating point optimization -rm yolo3_FP16.rt # be sure to delete(or move) old tensorRT files +rm yolo3_fp16.rt # be sure to delete(or move) old tensorRT files ./test_yolo3 # run the yolo test (is slow) -./demo yolo3_FP16.rt ../demo/yolo_test.mp4 y +./demo yolo3_fp16.rt ../demo/yolo_test.mp4 y ``` N.b. Using FP16 inference will lead to some errors in the results (first or second decimal). @@ -153,9 +153,9 @@ export TKDNN_CALIB_IMG_PATH=/path/to/calibration/image_list.txt # label_list.txt contains the list of the absolute paths to the calibration labels export TKDNN_CALIB_LABEL_PATH=/path/to/calibration/label_list.txt -rm yolo3_INT8.rt # be sure to delete(or move) old tensorRT files +rm yolo3_int8.rt # be sure to delete(or move) old tensorRT files ./test_yolo3 # run the yolo test (is slow) -./demo yolo3_INT8.rt ../demo/yolo_test.mp4 y +./demo yolo3_int8.rt ../demo/yolo_test.mp4 y ``` N.b. Using INT8 inference will lead to some errors in the results. @@ -166,6 +166,21 @@ N.b. INT8 calibration requires TensorRT version greater than or equal to 6.0 ### BatchSize bigger than 1 ``` export TKDNN_BATCHSIZE=2 +# build tensorRT files +``` +This will create a TensorRT file with the desidered **max** batch size. +The test will still run with a batch of 1, but the created tensorRT can manage the desidered batch size. + +### Test batch Inference +``` +./test_rtinference +# should be less or equal to the max batch size of the + +# example +export TKDNN_BATCHSIZE=4 # set max batch size +rm yolo3_fp32.rt # be sure to delete(or move) old tensorRT files +./test_yolo3 # build RT file +./test_rtinference yolo3_fp32.rt 4 # test with a batch size of 4 ``` ## mAP demo From d8f034a7adc577428e492105ef453c8304c7cf63 Mon Sep 17 00:00:00 2001 From: Francesco Gatti Date: Wed, 29 Apr 2020 18:17:32 +0200 Subject: [PATCH 2/3] Update README.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 0f6b607..8431079 100644 --- a/README.md +++ b/README.md @@ -172,6 +172,7 @@ This will create a TensorRT file with the desidered **max** batch size. The test will still run with a batch of 1, but the created tensorRT can manage the desidered batch size. ### Test batch Inference +This will test the network with random input and check if the output of each batch is the same. ``` ./test_rtinference # should be less or equal to the max batch size of the From 4fd84b1876feafc701016c64614a30903c53e7f1 Mon Sep 17 00:00:00 2001 From: Micaela Verucchi Date: Wed, 29 Apr 2020 18:34:06 +0200 Subject: [PATCH 3/3] Update README.md --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 8431079..7f1016d 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,9 @@ tkDNN is a Deep Neural Network library built with cuDNN and tensorRT primitives, specifically thought to work on NVIDIA Jetson Boards. It has been tested on TK1(branch cudnn2), TX1, TX2, AGX Xavier and several discrete GPU. The main goal of this project is to exploit NVIDIA boards as much as possible to obtain the best inference performance. It does not allow training. +Accepted paper @ IRC 2020, will soon been published. +M. Verucchi, L. Bartoli, F. Bagni, F. Gatti, P. Burgio and M. Bertogna, "Real-Time clustering and LiDAR-camera fusion on embedded platforms for self-driving cars", in proceedings in IEEE Robotic Computing (2020) + ## Index - [tkDNN](#tkdnn) - [Index](#index)