diff --git a/README.md b/README.md index a06a2b8..7f1016d 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,9 @@ tkDNN is a Deep Neural Network library built with cuDNN and tensorRT primitives, specifically thought to work on NVIDIA Jetson Boards. It has been tested on TK1(branch cudnn2), TX1, TX2, AGX Xavier and several discrete GPU. The main goal of this project is to exploit NVIDIA boards as much as possible to obtain the best inference performance. It does not allow training. +Accepted paper @ IRC 2020, will soon been published. +M. Verucchi, L. Bartoli, F. Bagni, F. Gatti, P. Burgio and M. Bertogna, "Real-Time clustering and LiDAR-camera fusion on embedded platforms for self-driving cars", in proceedings in IEEE Robotic Computing (2020) + ## Index - [tkDNN](#tkdnn) - [Index](#index) @@ -114,9 +117,9 @@ python run_ssd_live_demo.py mb2-ssd-lite To run the an object detection demo follow these steps (example with yolov3): ``` -rm yolo3_FP32.rt # be sure to delete(or move) old tensorRT files +rm yolo3_fp32.rt # be sure to delete(or move) old tensorRT files ./test_yolo3 # run the yolo test (is slow) -./demo yolo3_FP32.rt ../demo/yolo_test.mp4 y +./demo yolo3_fp32.rt ../demo/yolo_test.mp4 y ``` In general the demo program takes 4 parameters: ``` @@ -136,9 +139,9 @@ N.b. By default it is used FP32 inference To run the an object detection demo with FP16 inference follow these steps (example with yolov3): ``` export TKDNN_MODE=FP16 # set the half floating point optimization -rm yolo3_FP16.rt # be sure to delete(or move) old tensorRT files +rm yolo3_fp16.rt # be sure to delete(or move) old tensorRT files ./test_yolo3 # run the yolo test (is slow) -./demo yolo3_FP16.rt ../demo/yolo_test.mp4 y +./demo yolo3_fp16.rt ../demo/yolo_test.mp4 y ``` N.b. Using FP16 inference will lead to some errors in the results (first or second decimal). @@ -153,9 +156,9 @@ export TKDNN_CALIB_IMG_PATH=/path/to/calibration/image_list.txt # label_list.txt contains the list of the absolute paths to the calibration labels export TKDNN_CALIB_LABEL_PATH=/path/to/calibration/label_list.txt -rm yolo3_INT8.rt # be sure to delete(or move) old tensorRT files +rm yolo3_int8.rt # be sure to delete(or move) old tensorRT files ./test_yolo3 # run the yolo test (is slow) -./demo yolo3_INT8.rt ../demo/yolo_test.mp4 y +./demo yolo3_int8.rt ../demo/yolo_test.mp4 y ``` N.b. Using INT8 inference will lead to some errors in the results. @@ -166,6 +169,22 @@ N.b. INT8 calibration requires TensorRT version greater than or equal to 6.0 ### BatchSize bigger than 1 ``` export TKDNN_BATCHSIZE=2 +# build tensorRT files +``` +This will create a TensorRT file with the desidered **max** batch size. +The test will still run with a batch of 1, but the created tensorRT can manage the desidered batch size. + +### Test batch Inference +This will test the network with random input and check if the output of each batch is the same. +``` +./test_rtinference +# should be less or equal to the max batch size of the + +# example +export TKDNN_BATCHSIZE=4 # set max batch size +rm yolo3_fp32.rt # be sure to delete(or move) old tensorRT files +./test_yolo3 # build RT file +./test_rtinference yolo3_fp32.rt 4 # test with a batch size of 4 ``` ## mAP demo