Update READMEs, add README_depth, minors
Signed-off-by: Micaela Verucchi <micaelaverucchi@gmail.com>
This commit is contained in:
@@ -23,7 +23,10 @@ If you use tkDNN in your research, please cite the [following paper](https://iee
|
|||||||
- [x] Support 2D/3D Object Detection and Tracking [README](docs/README_2d3dtracking.md)
|
- [x] Support 2D/3D Object Detection and Tracking [README](docs/README_2d3dtracking.md)
|
||||||
#### 24 November 2021
|
#### 24 November 2021
|
||||||
- [x] Support to sematic segmentation on cuda 11
|
- [x] Support to sematic segmentation on cuda 11
|
||||||
- [x] Support to TensorRT8.
|
- [x] Support to TensorRT8 (tensort8 branch).
|
||||||
|
|
||||||
|
#### 30 March 2022
|
||||||
|
- [x] Support to monocular depth esitmation (tensort8 branch) [README](docs/README_depth.md)
|
||||||
|
|
||||||
TensorRT8 (and therefore Jetpack 4.6) is currently supported only on the branch tensort8 due to [performance issue with TensorRT8](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html)). We will merge it to the master as soon as those issues are fixed (probably in future minor releases).
|
TensorRT8 (and therefore Jetpack 4.6) is currently supported only on the branch tensort8 due to [performance issue with TensorRT8](https://docs.nvidia.com/deeplearning/tensorrt/release-notes/tensorrt-8.html)). We will merge it to the master as soon as those issues are fixed (probably in future minor releases).
|
||||||
|
|
||||||
@@ -138,6 +141,7 @@ For specific details on how to export weights see [HERE](./docs/exporting_weight
|
|||||||
For specific details on how to run:
|
For specific details on how to run:
|
||||||
- 2D object detection demos, details on FP16, INT8 and batching see [HERE](./docs/demo.md).
|
- 2D object detection demos, details on FP16, INT8 and batching see [HERE](./docs/demo.md).
|
||||||
- segmentation demos see [HERE](./docs/README_seg.md).
|
- segmentation demos see [HERE](./docs/README_seg.md).
|
||||||
|
- monocular depth estimation see [HERE](./docs/README_depth.md).
|
||||||
- 2D/3D object detection and tracking demos see [HERE](./docs/README_2d3dtracking.md).
|
- 2D/3D object detection and tracking demos see [HERE](./docs/README_2d3dtracking.md).
|
||||||
- mAP demo to evaluate 2D object detectors see [HERE](./docs/mAP_demo.md).
|
- mAP demo to evaluate 2D object detectors see [HERE](./docs/mAP_demo.md).
|
||||||
|
|
||||||
@@ -185,6 +189,8 @@ For specific details on how to run tkDNN on Windows 10/11 see [HERE](./docs/wind
|
|||||||
| shelfnet_berkeley | ShelfNet18_realtime<sup>11</sup> | [DeepDrive](https://bdd-data.berkeley.edu/) | 20 | 1024x1024 | [weights](https://cloud.hipert.unimore.it/s/m92e7QdD9gYMF7f/download) |
|
| shelfnet_berkeley | ShelfNet18_realtime<sup>11</sup> | [DeepDrive](https://bdd-data.berkeley.edu/) | 20 | 1024x1024 | [weights](https://cloud.hipert.unimore.it/s/m92e7QdD9gYMF7f/download) |
|
||||||
| dla34_cnet3d | Centernet3D (DLA34 backend)<sup>4</sup> | [KITTI 2017](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d) | 1 | 512x512 | [weights](https://cloud.hipert.unimore.it/s/2MDyWGzQsTKMjmR/download) |
|
| dla34_cnet3d | Centernet3D (DLA34 backend)<sup>4</sup> | [KITTI 2017](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d) | 1 | 512x512 | [weights](https://cloud.hipert.unimore.it/s/2MDyWGzQsTKMjmR/download) |
|
||||||
| dla34_ctrack | CenterTrack (DLA34 backend)<sup>12</sup> | [NuScenes 3D](https://www.nuscenes.org/) | 7 | 512x512 | [weights](https://cloud.hipert.unimore.it/s/rjNfgGL9FtAXLHp/download) |
|
| dla34_ctrack | CenterTrack (DLA34 backend)<sup>12</sup> | [NuScenes 3D](https://www.nuscenes.org/) | 7 | 512x512 | [weights](https://cloud.hipert.unimore.it/s/rjNfgGL9FtAXLHp/download) |
|
||||||
|
| monodepth2 | Monodepth2 <sup>13</sup> | [KITTI DEPTH](http://www.cvlibs.net/datasets/kitti/raw_data.php) | - | 640x192 | [weights-mono](https://cloud.hipert.unimore.it/s/iYw9QwgP6CsqxLR/download) |
|
||||||
|
| monodepth2 | Monodepth2 <sup>13</sup> | [KITTI DEPTH](http://www.cvlibs.net/datasets/kitti/raw_data.php) | - | 640x192 | [weights-stereo](https://cloud.hipert.unimore.it/s/XmwbWNXDfqyQ4EL/download) |
|
||||||
|
|
||||||
|
|
||||||
## References
|
## References
|
||||||
@@ -201,3 +207,11 @@ For specific details on how to run tkDNN on Windows 10/11 see [HERE](./docs/wind
|
|||||||
10. Wang, Chien-Yao, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. "Scaled-YOLOv4: Scaling Cross Stage Partial Network." arXiv preprint arXiv:2011.08036 (2020).
|
10. Wang, Chien-Yao, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. "Scaled-YOLOv4: Scaling Cross Stage Partial Network." arXiv preprint arXiv:2011.08036 (2020).
|
||||||
11. Zhuang, Juntang, et al. "ShelfNet for fast semantic segmentation." Proceedings of the IEEE International Conference on Computer Vision Workshops. 2019.
|
11. Zhuang, Juntang, et al. "ShelfNet for fast semantic segmentation." Proceedings of the IEEE International Conference on Computer Vision Workshops. 2019.
|
||||||
12. Zhou, Xingyi, Vladlen Koltun, and Philipp Krähenbühl. "Tracking objects as points." European Conference on Computer Vision. Springer, Cham, 2020.
|
12. Zhou, Xingyi, Vladlen Koltun, and Philipp Krähenbühl. "Tracking objects as points." European Conference on Computer Vision. Springer, Cham, 2020.
|
||||||
|
13. Godard, Clément, et al. "Digging into self-supervised monocular depth estimation." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.
|
||||||
|
|
||||||
|
## Contributors
|
||||||
|
The main contibutors, in chronological order, are:
|
||||||
|
- [Francesco Gatti](https://github.com/ceccocats), francesco.gatti@hipert.it
|
||||||
|
- [Micaela Verucchi](https://github.com/mive93), micaela.verucchi@unimore.it
|
||||||
|
- [Davide Sapienza](https://github.com/sapienzadavide), davide.sapienza@unimore.it
|
||||||
|
- [Harshvardhan Chandirasekar](https://github.com/perseusdg), f20180523@goa.bits-pilani.ac.in
|
||||||
|
|||||||
@@ -57,7 +57,7 @@ int main(int argc, char *argv[]) {
|
|||||||
if(save) {
|
if(save) {
|
||||||
int w = depthNN.output_w;
|
int w = depthNN.output_w;
|
||||||
int h = depthNN.output_h;
|
int h = depthNN.output_h;
|
||||||
resultVideo.open("result.mp4", cv::VideoWriter::fourcc('M','J','P','G'), 30, cv::Size(w, h));
|
resultVideo.open("result.mp4", cv::VideoWriter::fourcc('M','P','4','V'), 30, cv::Size(w, h));
|
||||||
}
|
}
|
||||||
|
|
||||||
if(show)
|
if(show)
|
||||||
|
|||||||
@@ -0,0 +1,54 @@
|
|||||||
|
# Monocular depth estimation with tkDNN
|
||||||
|
|
||||||
|
Currently tkDNN supports only Monodepth2 as monocular depth esitmation network.
|
||||||
|
|
||||||
|
|
||||||
|
## Run the demo
|
||||||
|
|
||||||
|
To run the depth estimation demo follow these steps (example with monodepth2):
|
||||||
|
```
|
||||||
|
rm monodepth2_fp32.rt # be sure to delete(or move) old tensorRT files
|
||||||
|
./test_monodepth2 # run the yolo test (is slow)
|
||||||
|
./demoDepth monodepth2_fp32.rt ../demo/yolo_test.mp4
|
||||||
|
```
|
||||||
|
In general the demo program takes the following parameters:
|
||||||
|
```
|
||||||
|
./demoDepth <network-rt-file> <path-to-video> <show-flag> <save-flag>
|
||||||
|
```
|
||||||
|
where
|
||||||
|
* ```<network-rt-file>``` is the rt file generated by a test
|
||||||
|
* ```<<path-to-video>``` is the path to a video file or a camera input
|
||||||
|
* ```<show-flag>``` if set to 0 the demo will not show the visualization, it will otherwise (default=1)
|
||||||
|
* ```<save-flag>``` if set to 1 the demo will save the video into result.mp4, it won't otherwise (default=1)
|
||||||
|
|
||||||
|
NB) By default it is used FP32 inference
|
||||||
|
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
|
||||||
|
<!-- ## FPS Results
|
||||||
|
|
||||||
|
Inference FPS of shelfnet with tkDNN, average of 1200 images on:
|
||||||
|
* RTX 2080Ti (CUDA 10.2, TensorRT 7.0.0, Cudnn 7.6.5);
|
||||||
|
* Xavier AGX, Jetpack 4.3 (CUDA 10.0, CUDNN 7.6.3, tensorrt 6.0.1 );
|
||||||
|
|
||||||
|
| Platform | Test | Phase | FP32, ms | FP32, FPS | FP16, ms | FP16, FPS | INT8, ms | INT8, FPS |
|
||||||
|
| :------: | :-----: | :-----: | :-----: | :-----: | :-----: | :-----: | :-----: | :-----: |
|
||||||
|
| RTX 2080Ti | shelfnet 1024x1024 (B=1) | pre | 6.11863 | 163.435 | 5.81465 | 171.979 | 5.88699 | 169.866 |
|
||||||
|
| RTX 2080Ti | shelfnet 1024x1024 (B=1) | inf | 11.5464 | 86.6074 | 7.35396 | 135.981 | 6.37623 | 156.832 |
|
||||||
|
| RTX 2080Ti | shelfnet 1024x1024 (B=1) | post | 4.09058 | 244.464 | 3.91961 | 255.128 | 4.07343 | 245.493 |
|
||||||
|
| RTX 2080Ti | shelfnet 1024x1024 (B=1) | tot | 21.7556 | 45.9652 | 17.0882 | 58.5199 | 16.3366 | 61.2121 |
|
||||||
|
| RTX 2080Ti | shelfnet 2048x2048 (B=4) | pre | 25.435 | 39.3158 | 25.2953 | 39.5331 | 25.9303 | 38.565 |
|
||||||
|
| RTX 2080Ti | shelfnet 2048x2048 (B=4) | inf | 36.5015 | 27.3961 | 17.0534 | 58.6395 | 15.6061 | 64.0773 |
|
||||||
|
| RTX 2080Ti | shelfnet 2048x2048 (B=4) | post | 17.3917 | 57.4985 | 17.1649 | 58.2583 | 17.5539 | 56.9675 |
|
||||||
|
| RTX 2080Ti | shelfnet 2048x2048 (B=4) | tot | 79.3283 | 12.6058 | 59.5136 | 16.8029 | 59.0903 | 16.9233 |
|
||||||
|
| AGX Xavier | shelfnet 1024x1024 (B=1) | pre | 8.0174 | 124.729 | 7.5117 | 133.126 | 7.47333 | 133.809 |
|
||||||
|
| AGX Xavier | shelfnet 1024x1024 (B=1) | inf | 72.4173 | 13.8089 | 37.505 | 26.6631 | 31.3286 | 31.9197 |
|
||||||
|
| AGX Xavier | shelfnet 1024x1024 (B=1) | post | 8.89958 | 112.365 | 8.83576 | 113.176 | 9.42655 | 106.083 |
|
||||||
|
| AGX Xavier | shelfnet 1024x1024 (B=1) | tot | 89.3342 | 11.1939 | 53.8525 | 18.5692 | 48.2285 | 20.7346 |
|
||||||
|
| AGX Xavier | shelfnet 2048x2048 (B=4) | pre | 47.1454 | 21.211 | 21.6475 | 46.1947 | 21.4201 | 46.6851 |
|
||||||
|
| AGX Xavier | shelfnet 2048x2048 (B=4) | inf | 266.537 | 3.75183 | 128.321 | 7.79293 | 107.621 | 9.29185 |
|
||||||
|
| AGX Xavier | shelfnet 2048x2048 (B=4) | post | 44.0711 | 22.6906 | 40.1732 | 24.8922 | 39.873 | 25.0796 |
|
||||||
|
| AGX Xavier | shelfnet 2048x2048 (B=4) | tot | 357.753 | 2.79522 | 190.142 | 5.25922 | 168.914 | 5.92016 | -->
|
||||||
|
|
||||||
@@ -86,6 +86,18 @@ mkdir layer debug
|
|||||||
python export.py
|
python export.py
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### 6)Export weights for monodepth2
|
||||||
|
To get the weights needed to run Shelfnet tests use [this](https://github.com/perseusdg/monodepth2) fork of a Pytorch implementation of monodepth2 network.
|
||||||
|
|
||||||
|
```
|
||||||
|
git clone https://github.com/perseusdg/monodepth2
|
||||||
|
cd monodepth2
|
||||||
|
mkdir models # Download the official weights and put depth.pth and encorder.pth inside this new folder
|
||||||
|
conda env create --file monodepth.yaml
|
||||||
|
conda activate monodepth2
|
||||||
|
python exporter.py # you will find the weights inside the tkDNN_bin folder
|
||||||
|
```
|
||||||
|
|
||||||
## Darknet Parser
|
## Darknet Parser
|
||||||
tkDNN implement and easy parser for darknet cfg files, a network can be converted with *tk::dnn::darknetParser*:
|
tkDNN implement and easy parser for darknet cfg files, a network can be converted with *tk::dnn::darknetParser*:
|
||||||
```
|
```
|
||||||
|
|||||||
Reference in New Issue
Block a user