Update READMEs
Signed-off-by: Micaela Verucchi <micaelaverucchi@gmail.com>
This commit is contained in:
@@ -19,6 +19,7 @@ If you use tkDNN in your research, please cite the [following paper](https://iee
|
||||
|
||||
### What's new (20 July 2021)
|
||||
- [x] Support to sematic segmentation [README](docs/README_seg.md)
|
||||
- [x] Support 2D/3D Object Detection and Tracking [README](docs/README_2d3dtracking.md)
|
||||
- [ ] Support to TensorRT8 (WIP)
|
||||
|
||||
## FPS Results
|
||||
@@ -70,15 +71,13 @@ Results for COCO val 2017 (5k images), on RTX 2080Ti, with conf threshold=0.001
|
||||
- [tkDNN](#tkdnn)
|
||||
- [Index](#index)
|
||||
- [Dependencies](#dependencies)
|
||||
- [About OpenCV](#about-opencv)
|
||||
- [How to compile this repo](#how-to-compile-this-repo)
|
||||
- [Workflow](#workflow)
|
||||
- [How to export weights](#how-to-export-weights)
|
||||
- [Run the demo](#run-the-demo)
|
||||
- [mAP demo](#map-demo)
|
||||
- [Exporting weights](#exporting-weights)
|
||||
- [Run the demos](#run-the-demos)
|
||||
- [tkDNN on Windows 10 (experimental)](#tkdnn-on-windows-10-experimental)
|
||||
- [Existing tests and supported networks](#existing-tests-and-supported-networks)
|
||||
- [References](#references)
|
||||
- [tkDNN on Windows 10 (experimental)](#tkdnn-on-windows-10-experimental)
|
||||
|
||||
|
||||
## Dependencies
|
||||
@@ -97,7 +96,7 @@ sudo apt install libyaml-cpp-dev curl libeigen3-dev
|
||||
|
||||
```
|
||||
|
||||
## About OpenCV
|
||||
#### About OpenCV
|
||||
To compile and install OpenCV4 with contrib us the script ```install_OpenCV4.sh```. It will download and compile OpenCV in Download folder.
|
||||
```
|
||||
bash scripts/install_OpenCV4.sh
|
||||
@@ -125,15 +124,21 @@ Steps needed to do inference on tkDNN with a custom neural network.
|
||||
|
||||
## Exporting weights
|
||||
|
||||
For specific details on how to export weights see [here](./docs/exporting_weights.md)
|
||||
For specific details on how to export weights see [HERE](./docs/exporting_weights.md).
|
||||
|
||||
## Run the demo
|
||||
## Run the demos
|
||||
|
||||
For specific details on how to run the demos see [here](./docs/demo.md)
|
||||
For specific details on how to run:
|
||||
- 2D object detection demos, details on FP16, INT8 and batching see [HERE](./docs/demo.md).
|
||||
- segmentation demos see [HERE](./docs/README_seg.md).
|
||||
- 2D/3D object detection and tracking demos see [HERE](./docs/README_2d3dtracking.md).
|
||||
- mAP demo to evaluate 2D object detectors see [HERE](./docs/mAP_demo.md).
|
||||
|
||||
## mAP demo
|
||||

|
||||
|
||||
For specific details on how to run the mAP demo see [here](./docs/mAP_demo.md)
|
||||
## tkDNN on Windows 10 (experimental)
|
||||
|
||||
For specific details on how to run tkDNN on Windows 10 see [HERE](./docs/windows.md).
|
||||
|
||||
## Existing tests and supported networks
|
||||
|
||||
@@ -160,14 +165,20 @@ For specific details on how to run the mAP demo see [here](./docs/mAP_demo.md)
|
||||
| resnet101_cnet | Centernet (Resnet101 backend)<sup>4</sup> | [COCO 2017](http://cocodataset.org/) | 80 | 512x512 | [weights](https://cloud.hipert.unimore.it/s/5BTjHMWBcJk8g3i/download) |
|
||||
| csresnext50-panet-spp | Cross Stage Partial Network <sup>7</sup> | [COCO 2014](http://cocodataset.org/) | 80 | 416x416 | [weights](https://cloud.hipert.unimore.it/s/Kcs4xBozwY4wFx8/download) |
|
||||
| yolo4 | Yolov4 <sup>8</sup> | [COCO 2017](http://cocodataset.org/) | 80 | 416x416 | [weights](https://cloud.hipert.unimore.it/s/d97CFzYqCPCp5Hg/download) |
|
||||
| yolo4_320 | Yolov4 <sup>8</sup> | [COCO 2017](http://cocodataset.org/) | 80 | 320x320 | [weights](https://cloud.hipert.unimore.it/s/d97CFzYqCPCp5Hg/download) |
|
||||
| yolo4_512 | Yolov4 <sup>8</sup> | [COCO 2017](http://cocodataset.org/) | 80 | 512x512 | [weights](https://cloud.hipert.unimore.it/s/d97CFzYqCPCp5Hg/download) |
|
||||
| yolo4_608 | Yolov4 <sup>8</sup> | [COCO 2017](http://cocodataset.org/) | 80 | 608x608 | [weights](https://cloud.hipert.unimore.it/s/d97CFzYqCPCp5Hg/download) |
|
||||
| yolo4_berkeley | Yolov4 <sup>8</sup> | [BDD100K ](https://bair.berkeley.edu/blog/2018/05/30/bdd/) | 10 | 540x320 | [weights](https://cloud.hipert.unimore.it/s/nkWFa5fgb4NTdnB/download) |
|
||||
| yolo4tiny | Yolov4 tiny <sup>9</sup> | [COCO 2017](http://cocodataset.org/) | 80 | 416x416 | [weights](https://cloud.hipert.unimore.it/s/iRnc4pSqmx78gJs/download) |
|
||||
| yolo4x | Yolov4x-mish <sup>9</sup> | [COCO 2017](http://cocodataset.org/) | 80 | 640x640 | [weights](https://cloud.hipert.unimore.it/s/5MFjtNtgbDGdJEo/download) |
|
||||
| yolo4x | Yolov4x-mish <sup>9</sup> | [COCO 2017](http://cocodataset.org/) |
|
||||
| yolo4tiny_512 | Yolov4 tiny <sup>9</sup> | [COCO 2017](http://cocodataset.org/) | 80 | 512x512 | [weights](https://cloud.hipert.unimore.it/s/iRnc4pSqmx78gJs/download) |
|
||||
80 | 640x640 | [weights](https://cloud.hipert.unimore.it/s/5MFjtNtgbDGdJEo/download) |
|
||||
| yolo4x-cps | Scaled Yolov4 <sup>10</sup> | [COCO 2017](http://cocodataset.org/) | 80 | 512x512 | [weights](https://cloud.hipert.unimore.it/s/AfzHE4BfTeEm2gH/download) |
|
||||
| shelfnet | ShelfNet18_realtime<sup>11</sup> | [Cityscapes](https://www.cityscapes-dataset.com/) | 19 | 1024x1024 | [weights](https://cloud.hipert.unimore.it/s/mEDZMRJaGCFWSJF/download) |
|
||||
| shelfnet_berkeley | ShelfNet18_realtime<sup>11</sup> | [DeepDrive](https://bdd-data.berkeley.edu/) | 20 | 1024x1024 | [weights](https://cloud.hipert.unimore.it/s/m92e7QdD9gYMF7f/download) |
|
||||
| dla34_cnet3d | Centernet3D (DLA34 backend)<sup>4</sup> | [KITTI 2017](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d) | 1 | 512x512 | [weights](https://cloud.hipert.unimore.it/s/2MDyWGzQsTKMjmR/download) |
|
||||
| dla34_ctrack | CenterTrack (DLA34 backend)<sup>12</sup> | [NuScenes 3D](https://www.nuscenes.org/) | 7 | 512x512 | [weights](https://cloud.hipert.unimore.it/s/rjNfgGL9FtAXLHp/download) |
|
||||
|
||||
### tkDNN on Windows 10 (experimental)
|
||||
|
||||
For specific details on how to run the demos on Windows 10 see [here](./docs/windows.md)
|
||||
|
||||
## References
|
||||
|
||||
@@ -181,3 +192,5 @@ For specific details on how to run the demos on Windows 10 see [here](./docs/win
|
||||
8. Bochkovskiy, Alexey, Chien-Yao Wang, and Hong-Yuan Mark Liao. "YOLOv4: Optimal Speed and Accuracy of Object Detection." arXiv preprint arXiv:2004.10934 (2020).
|
||||
9. Bochkovskiy, Alexey, "Yolo v4, v3 and v2 for Windows and Linux" (https://github.com/AlexeyAB/darknet)
|
||||
10. Wang, Chien-Yao, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. "Scaled-YOLOv4: Scaling Cross Stage Partial Network." arXiv preprint arXiv:2011.08036 (2020).
|
||||
11. Zhuang, Juntang, et al. "ShelfNet for fast semantic segmentation." Proceedings of the IEEE International Conference on Computer Vision Workshops. 2019.
|
||||
12. Zhou, Xingyi, Vladlen Koltun, and Philipp Krähenbühl. "Tracking objects as points." European Conference on Computer Vision. Springer, Cham, 2020.
|
||||
|
||||
@@ -1,6 +1,42 @@
|
||||
# 2D/3D Object Detection and Tracking
|
||||
|
||||
Currently tkDNN supports only CenterTrack as OD & Tracker network.
|
||||
Currently tkDNN supports only CenterTrack as 3DOD & 2D/3D Tracker network.
|
||||
|
||||
## 3D Object Detection
|
||||
|
||||
To run the 3D object detection demo follow these steps (example with CenterNet based on DLA34):
|
||||
```
|
||||
rm dla34_cnet3d_fp32.rt # be sure to delete(or move) old tensorRT files
|
||||
./test_dla34_cnet3d # run the yolo test (is slow)
|
||||
./demo3D dla34_cnet3d_fp32.rt ../demo/yolo_test.mp4 NULL c
|
||||
```
|
||||
The demo3D program takes the same parameters of the demo program:
|
||||
```
|
||||
./demo3D <network-rt-file> <path-to-video> <calibration-file> <kind-of-network> <number-of-classes> <n-batches> <show-flag> <conf-thresh>
|
||||
```
|
||||
where
|
||||
|
||||
* ```<calibration-file>``` is the camera calibration file (opencv format). It is important that the file contains entry "camera_matrix" with sub-entry "rows", "cols", "data". If you do not want to pass the calibration file, pass "NULL" instead.
|
||||
|
||||
## Object Detection and Tracking
|
||||
|
||||
To run the 3D object detection & tracking demo follow these steps (example with CenterTrack based on DLA34):
|
||||
```
|
||||
rm dla34_ctrack_fp32.rt # be sure to delete(or move) old tensorRT files
|
||||
./test_dla34_ctrack # run the yolo test (is slow)
|
||||
./demoTracker dla34_ctrack_fp32.rt ../demo/yolo_test.mp4 NULL c
|
||||
```
|
||||
|
||||
The demoTracker program takes the same parameters of the demo program:
|
||||
```
|
||||
./demoTracker <network-rt-file> <path-to-video> <calibration-file> <kind-of-network> <number-of-classes> <n-batches> <show-flag> <conf-thresh> <2D/3D-flag>
|
||||
```
|
||||
|
||||
where
|
||||
|
||||
* ```<calibration-file>``` is the camera calibration file (opencv format). It is important that the file contains entry "camera_matrix" with sub-entry "rows", "cols", "data". If you do not want to pass the calibration file, pass "NULL" instead.
|
||||
* ```<2D/3D-flag>``` if set to 0 the demo will be in the 2D mode, while if set to 1 the demo will be in the 3D mode (Default is 1 - 3D mode).
|
||||
|
||||
|
||||
## FPS Results
|
||||
|
||||
+1
-22
@@ -2,19 +2,6 @@
|
||||
|
||||
Currently tkDNN supports only ShelfNet as semantic segmentation network.
|
||||
|
||||
## Export weights from Shelfnet
|
||||
To get the weights needed to run Shelfnet tests use [this](https://git.hipert.unimore.it/mverucchi/shelfnet) fork of a Pytorch implementation of Shelfnet network.
|
||||
|
||||
```
|
||||
git clone https://git.hipert.unimore.it/mverucchi/shelfnet
|
||||
cd shelfnet
|
||||
cd ShelfNet18_realtime
|
||||
conda env create --file shelfnet_env.yml
|
||||
conda activate shelfnet
|
||||
mkdir layer debug
|
||||
python export.py
|
||||
```
|
||||
|
||||
|
||||
## Run the demo
|
||||
|
||||
@@ -46,15 +33,7 @@ NB) The batching is not used to work on more streams, rather to work on more til
|
||||
|
||||
For other demo videos refer to [this playlist](https://www.youtube.com/playlist?list=PLv0nEQYDD45y5EdSiywwCGPBmJVUzIWwe).
|
||||
|
||||
|
||||
## Existing tests and supported networks
|
||||
|
||||
| Test Name | Network | Dataset | N Classes | Input size | Weights |
|
||||
| :---------------- | :-------------------------------------------- | :-----------------------------------------------------------: | :-------: | :-----------: | :------------------------------------------------------------------------ |
|
||||
| shelfnet | ShelfNet18_realtime<sup>1</sup> | [Cityscapes](https://www.cityscapes-dataset.com/) | 19 | 1024x1024 | [weights](https://cloud.hipert.unimore.it/s/mEDZMRJaGCFWSJF/download) |
|
||||
| shelfnet_berkeley | ShelfNet18_realtime<sup>1</sup> | [DeepDrive](https://bdd-data.berkeley.edu/) | 20 | 1024x1024 | [weights](https://cloud.hipert.unimore.it/s/m92e7QdD9gYMF7f/download) |
|
||||
|
||||
1. Zhuang, Juntang, et al. "ShelfNet for fast semantic segmentation." Proceedings of the IEEE International Conference on Computer Vision Workshops. 2019.
|
||||
NB) The gif and the videos are obtained with Mapillary Vistas weights, that we cannot publicly share due to its license restrictions. However, you can train Shelfnet using Mapillary and [this](https://git.hipert.unimore.it/mverucchi/shelfnet) fork of the original repo.
|
||||
|
||||
|
||||
## FPS Results
|
||||
|
||||
+8
-106
@@ -1,8 +1,6 @@
|
||||
# tkDNN Demo
|
||||
# 2D Object Detection with tkDNN
|
||||
|
||||
## Supported Network
|
||||
|
||||
2D Object Detection:
|
||||
## Supported Networks
|
||||
|
||||
* Yolo4, Yolo4-csp, Yolo4x, Yolo4_berkeley, Yolo4tiny
|
||||
* Yolo3, Yolo3_berkeley, Yolo3_coco4, Yolo3_flir, Yolo3_512, Yolo3tiny, Yolo3tiny_512
|
||||
@@ -11,28 +9,12 @@
|
||||
* Resnet101_cnet, Dla34_cnet
|
||||
* Mobilenetv2ssd, Mobilenetv2ssd512, Bdd-mobilenetv2ssd
|
||||
|
||||
3D Object Detection:
|
||||
|
||||
* Dla34_cnet3d
|
||||
|
||||
2D/3D Object Detection and Tracking:
|
||||
|
||||
* Dla34_cnet3d_track
|
||||
|
||||
## Index
|
||||
|
||||
- [Run the demo](#run-the-demo)
|
||||
- [2D Object Detection](#2d-object-detection)
|
||||
- [3D Object Detection](#3d-object-detection)
|
||||
- [Object Detection and Tracking](#object-detection-and-tracking)
|
||||
- [FP16 inference](#fp16-inference)
|
||||
- [INT8 inference](#int8-inference)
|
||||
- [Batching](#batching)
|
||||
- [Run the demo on Windows](#run-the-demo-on-windows)
|
||||
|
||||
## Run the demo
|
||||
|
||||
N.b. By default it is used FP32 inference
|
||||
- [2D Object Detection](#2d-object-detection)
|
||||
- [FP16 inference](#fp16-inference)
|
||||
- [INT8 inference](#int8-inference)
|
||||
- [Batching](#batching)
|
||||
|
||||
### 2D Object Detection
|
||||
This is an example using yolov4.
|
||||
@@ -66,45 +48,11 @@ where
|
||||
* ```<show-flag>``` if set to 0 the demo will not show the visualization but save the video into result.mp4 (if n-batches ==1)
|
||||
* ```<conf-thresh>``` confidence threshold for the detector. Only bounding boxes with threshold greater than conf-thresh will be displayed.
|
||||
|
||||
|
||||
N.B. By default it is used FP32 inference
|
||||
|
||||
|
||||

|
||||
|
||||
### 3D Object Detection
|
||||
|
||||
To run the 3D object detection demo follow these steps (example with CenterNet based on DLA34):
|
||||
```
|
||||
rm dla34_cnet3d_fp32.rt # be sure to delete(or move) old tensorRT files
|
||||
./test_dla34_cnet3d # run the yolo test (is slow)
|
||||
./demo3D dla34_cnet3d_fp32.rt ../demo/yolo_test.mp4 NULL c
|
||||
```
|
||||
The demo3D program takes the same parameters of the demo program:
|
||||
```
|
||||
./demo3D <network-rt-file> <path-to-video> <calibration-file> <kind-of-network> <number-of-classes> <n-batches> <show-flag> <conf-thresh>
|
||||
```
|
||||
where
|
||||
|
||||
* ```calibration-file``` is the camera calibration file (opencv format). It is important that the file contains entry "camera_matrix" with sub-entry "rows", "cols", "data". If you do not want to pass the calibration file, pass "NULL" instead.
|
||||
|
||||
### Object Detection and Tracking
|
||||
|
||||
To run the 3D object detection & tracking demo follow these steps (example with CenterTrack based on DLA34):
|
||||
```
|
||||
rm dla34_ctrack_fp32.rt # be sure to delete(or move) old tensorRT files
|
||||
./test_dla34_ctrack # run the yolo test (is slow)
|
||||
./demoTracker dla34_ctrack_fp32.rt ../demo/yolo_test.mp4 NULL c
|
||||
```
|
||||
|
||||
The demoTracker program takes the same parameters of the demo program:
|
||||
```
|
||||
./demoTracker <network-rt-file> <path-to-video> <calibration-file> <kind-of-network> <number-of-classes> <n-batches> <show-flag> <conf-thresh> <2D/3D-flag>
|
||||
```
|
||||
|
||||
where
|
||||
|
||||
* ```calibration-file``` is the camera calibration file (opencv format). It is important that the file contains entry "camera_matrix" with sub-entry "rows", "cols", "data". If you do not want to pass the calibration file, pass "NULL" instead.
|
||||
* ```<2D/3D-flag>``` if set to 0 the demo will be in the 2D mode, while if set to 1 the demo will be in the 3D mode (Default is 1 - 3D mode).
|
||||
|
||||
### FP16 inference
|
||||
|
||||
@@ -115,7 +63,7 @@ rm yolo3_fp16.rt # be sure to delete(or move) old tensorRT files
|
||||
./test_yolo3 # run the yolo test (is slow)
|
||||
./demo yolo3_fp16.rt ../demo/yolo_test.mp4 y
|
||||
```
|
||||
N.b. Using FP16 inference will lead to some errors in the results (first or second decimal).
|
||||
N.B. Using FP16 inference will lead to some errors in the results (first or second decimal).
|
||||
|
||||
### INT8 inference
|
||||
|
||||
@@ -169,49 +117,3 @@ rm yolo3_fp32.rt # be sure to delete(or move) old tensorRT fil
|
||||
./test_yolo3 # build RT file
|
||||
./test_rtinference yolo3_fp32.rt 4 # test with a batch size of 4
|
||||
```
|
||||
|
||||
### Run the demo on Windows
|
||||
|
||||
This example uses yolo4_tiny.\
|
||||
To run the object detection file create .rt file bu running:
|
||||
```
|
||||
.\test_yolo4tiny.exe
|
||||
```
|
||||
|
||||
Once the rt file has been successfully create,run the demo using the following command:
|
||||
```
|
||||
.\demo.exe yolo4tiny_fp32.rt ..\demo\yolo_test.mp4 y
|
||||
```
|
||||
For general info on more demo paramters,check Run the demo section on top
|
||||
To run the test_all_tests.sh on windows,use git bash or msys2
|
||||
|
||||
### FP16 inference windows
|
||||
|
||||
This is an untested feature on windows.To run the object detection demo with FP16 interference follow the below steps(example with yolo4tiny):
|
||||
```
|
||||
set TKDNN_MODE=FP16
|
||||
del /f yolo4tiny_fp16.rt
|
||||
.\test_yolo4tiny.exe
|
||||
.\demo.exe yolo4tiny_fp16.rt ..\demo\yolo_test.mp4
|
||||
```
|
||||
|
||||
### INT8 inference windows
|
||||
To run object detection demo with INT8 (example with yolo4tiny):
|
||||
```
|
||||
set TKDNN_MODE=INT8
|
||||
set TKDNN_CALIB_LABEL_PATH=..\demo\COCO_val2017\all_labels.txt
|
||||
set TKDNN_CALIB_IMG_PATH=..\demo\COCO_val2017\all_images.txt
|
||||
del /f yolo4tiny_int8.rt # be sure to delete(or move) old tensorRT files
|
||||
.\test_yolo4tiny.exe # run the yolo test (is slow)
|
||||
.\demo.exe yolo4tiny_int8.rt ..\demo\yolo_test.mp4 y
|
||||
|
||||
```
|
||||
|
||||
### Known issues with tkDNN on Windows
|
||||
|
||||
Mobilenet and Centernet demos work properly only when built with msvc 16.7 in Release Mode,when built in debug mode for the mentioned networks one might encounter opencv assert errors
|
||||
|
||||
All Darknet models work properly with demo using MSVC version(16.7-16.9)
|
||||
|
||||
It is recommended to use Nvidia Driver(465+),Cuda unknown errors have been observed when using older drivers on pascal(SM 61) devices.
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
- [3)Export weights for CenterNet](#3export-weights-for-centernet)
|
||||
- [4)Export weights for MobileNetSSD](#4export-weights-for-mobilenetssd)
|
||||
- [5)Export weights for CenterTrack](#5export-weights-for-centertrack)
|
||||
- [6)Export weights for ShelfNet](#6export-weights-for-shelfnet)
|
||||
- [Darknet Parser](#darknet-parser)
|
||||
|
||||
## How to export weights
|
||||
@@ -30,7 +31,7 @@ make
|
||||
mkdir layers debug
|
||||
./darknet export <path-to-cfg-file> <path-to-weights> layers
|
||||
```
|
||||
N.b. Use compilation with CPU (leave GPU=0 in Makefile) if you also want debug.
|
||||
N.B. Use compilation with CPU (leave GPU=0 in Makefile) if you also want debug.
|
||||
|
||||
### 2)Export weights for DLA34 and ResNet101
|
||||
To get weights and outputs needed to run the tests dla34 and resnet101 use the Python script and the Anaconda environment included in the repository.
|
||||
@@ -72,6 +73,19 @@ git clone https://github.com/sapienzadavide/CenterTrack.git
|
||||
python demo.py tracking,ddd --load_model ../models/nuScenes_3Dtracking.pth --dataset nuscenes --pre_hm --track_thresh 0.1 --demo /path/to/image/or/folder/or/video/or/webcam --test_focal_length 633 --exp_wo --exp_wo_dim 512 --input_h 512 --input_w 512
|
||||
```
|
||||
|
||||
### 6)Export weights for ShelfNet
|
||||
To get the weights needed to run Shelfnet tests use [this](https://git.hipert.unimore.it/mverucchi/shelfnet) fork of a Pytorch implementation of Shelfnet network.
|
||||
|
||||
```
|
||||
git clone https://git.hipert.unimore.it/mverucchi/shelfnet
|
||||
cd shelfnet
|
||||
cd ShelfNet18_realtime
|
||||
conda env create --file shelfnet_env.yml
|
||||
conda activate shelfnet
|
||||
mkdir layer debug
|
||||
python export.py
|
||||
```
|
||||
|
||||
## Darknet Parser
|
||||
tkDNN implement and easy parser for darknet cfg files, a network can be converted with *tk::dnn::darknetParser*:
|
||||
```
|
||||
|
||||
+2
-4
@@ -1,8 +1,6 @@
|
||||
# mAP demo
|
||||
# Run the mAP demo
|
||||
|
||||
## Run the mAP demo
|
||||
|
||||
To compute mAP, precision, recall and f1score, run the map_demo.
|
||||
To compute mAP, precision, recall and f1score to evaluate 2D object detectors, run the map_demo.
|
||||
|
||||
A validation set is needed.
|
||||
To download COCO_val2017 (80 classes) run (form the root folder):
|
||||
|
||||
Reference in New Issue
Block a user