Merge with master, all tests passed

Signed-off-by: Micaela Verucchi <micaelaverucchi@gmail.com>
2021-07-20 12:48:41 +02:00
parent 4ca69836e9 c306b36860
commit 84ff978ccb
73 changed files with 5207 additions and 305 deletions
@@ -0,0 +1,66 @@
+# Semantic Segmentation with tkDNN
+
+Currently tkDNN supports only ShelfNet as semantic segmentation network.
+
+## Export weights from Shelfnet
+To get the weights needed to run Shelfnet tests use [this](https://git.hipert.unimore.it/mverucchi/shelfnet) fork of a Pytorch implementation of Shelfnet network. 
+
+```
+git clone https://git.hipert.unimore.it/mverucchi/shelfnet
+cd shelfnet 
+cd ShelfNet18_realtime
+conda env create --file shelfnet_env.yml
+conda activate shelfnet
+mkdir layer debug
+python export.py
+```
+
+
+## Run the demo
+
+To run the semantic segmentation demo follow these steps (example with shelfnet):
+```
+rm shelfnet_fp32.rt        # be sure to delete(or move) old tensorRT files
+export TKDNN_BATCHSIZE=4   # be sure you have batch size > than 1 if you want to run inference on images bigger than 1024
+./test_shelfnet            # run the yolo test (is slow)
+./demo shelfnet_fp32.rt ../demo/yolo_test.mp4 1 19
+```
+In general the demo program takes the following parameters:
+```
+./seg_demo <network-rt-file> <path-to-video> <n-batches> <number-of-classes> <resize-flag> <baseline-resize> <show-flag> <write-pred>
+```
+where
+*  ```<network-rt-file>``` is the rt file generated by a test
+*  ```<<path-to-video>``` is the path to a video file or a camera input  
+*  ```<n-batches>``` number of batches to use in inference (N.B. you should first export TKDNN_BATCHSIZE to the required n_batches and create again the rt file for the network).
+*  ```<number-of-classes>```is the number of classes the network is trained on
+*  ```<resize-flag>``` if set to 0 the demo will not resize the input frames, but use it as it is, otherwise it will resize it.
+*  ```<baseline-resize>``` is ```<resize-flag>``` is set to 1, then the input frames will be proportionally resized using ```<baseline-resize>``` as width baseline.
+*  ```<show-flag>``` if set to 0 the demo will not show the visualization but save the video into result.mp4 (if n-batches ==1)
+*  ```<write-pred>``` if set to 0 (default) the demo will run, otherwise the evaluation of a dataset will run and the output of the segmentation will be saved. Attention: this is under development and paths are embedded, so change them in the code in advance.
+
+N.b. By default it is used FP32 inference
+
+<center>
+
+![gif](output.gif "Results on yolo_test.mp4")  
+
+</center>
+
+For other demo videos refer to [this playlist](https://www.youtube.com/playlist?list=PLv0nEQYDD45y5EdSiywwCGPBmJVUzIWwe).
+
+
+## Existing tests and supported networks
+
+| Test Name         | Network                                       | Dataset                                                       | N Classes | Input size    | Weights                                                                   |
+| :---------------- | :-------------------------------------------- | :-----------------------------------------------------------: | :-------: | :-----------: | :------------------------------------------------------------------------ |
+| shelfnet              | ShelfNet18_realtime<sup>1</sup>                           | [Cityscapes](https://www.cityscapes-dataset.com/)                          | 19        | 1024x1024       | [weights](https://cloud.hipert.unimore.it/s/mEDZMRJaGCFWSJF/download)                                                                   |
+| shelfnet_berkeley              | ShelfNet18_realtime<sup>1</sup>                           | [DeepDrive](https://bdd-data.berkeley.edu/)                          | 20        | 1024x1024       | [weights](https://cloud.hipert.unimore.it/s/m92e7QdD9gYMF7f/download)                                                                   |
+
+1. Zhuang, Juntang, et al. "ShelfNet for fast semantic segmentation." Proceedings of the IEEE International Conference on Computer Vision Workshops. 2019.
+
+
+## Known issues
+
+When creating the rt file all the checks returns errors. It is due to a different resize function and handling of the original ShelfNet outputs.
+However, the network is supposed to work.