mmr/tkDNN
Archived
1
0
Fork 0
Code Issues 46 Pull Requests 12 Releases 4 Activity

Problem in adding dilated convolution #37

New Issue
Closed
opened 2020-06-14 16:31:52 +02:00 by mrhosseini · 3 comments
mrhosseini commented 2020-06-14 16:31:52 +02:00 (Migrated from github.com)
Copy Link
Copy Source

Hi
The AlexeyAB/darknet repository, supports dilated convolution. I tried to add the dilation support to tkDNN but it results in error.

Changes can be seen in my fork here.
This is the output of a test program (test_yolo3_tiny_dilation) created based on test_yolo3tiny, using this cfg file:

Not supported field: batch=128
Not supported field: subdivisions=16
Not supported field: momentum=0.9
Not supported field: decay=0.0005
Not supported field: angle=0
Not supported field: saturation = 1.5
Not supported field: exposure = 1.15
Not supported field: hue=0
Not supported field: learning_rate=0.001
Not supported field: burn_in=1000
Not supported field: max_batches = 15000
Not supported field: policy=steps
Not supported field: steps=13000,14000
Not supported field: scales=.1,.1
New NETWORK (tkDNN v0.5, CUDNN v8)
Reading weights: I=1 O=16 KERNEL=3x3x1
Reading weights: I=16 O=32 KERNEL=3x3x1
Reading weights: I=32 O=16 KERNEL=1x1x1
Reading weights: I=16 O=32 KERNEL=3x3x1
Reading weights: I=32 O=64 KERNEL=3x3x1
Reading weights: I=64 O=32 KERNEL=1x1x1
Reading weights: I=32 O=64 KERNEL=3x3x1
Reading weights: I=64 O=32 KERNEL=1x1x1
Reading weights: I=32 O=64 KERNEL=3x3x1
Reading weights: I=64 O=32 KERNEL=1x1x1
Reading weights: I=32 O=64 KERNEL=3x3x1
Reading weights: I=64 O=32 KERNEL=1x1x1
Reading weights: I=64 O=32 KERNEL=1x1x1
Reading weights: I=32 O=64 KERNEL=3x3x1
Reading weights: I=64 O=32 KERNEL=1x1x1
Reading weights: I=64 O=32 KERNEL=1x1x1
Reading weights: I=64 O=32 KERNEL=1x1x1
Reading weights: I=64 O=128 KERNEL=3x3x1
Reading weights: I=128 O=64 KERNEL=1x1x1
Reading weights: I=64 O=128 KERNEL=3x3x1
Reading weights: I=128 O=64 KERNEL=1x1x1
Reading weights: I=64 O=128 KERNEL=3x3x1
Reading weights: I=64 O=16 KERNEL=1x1x1
Reading weights: I=192 O=128 KERNEL=1x1x1
Reading weights: I=128 O=256 KERNEL=3x3x1
Reading weights: I=256 O=18 KERNEL=1x1x1
Not supported field: anchors = 25, 21,  36, 23,  37, 31,  49, 35,  63, 48,  98, 61
Not supported field: jitter=.3
Not supported field: ignore_thresh = .7
Not supported field: truth_thresh = 1
Not supported field: random=1
Reading weights: I=32 O=64 KERNEL=3x3x1
Reading weights: I=64 O=128 KERNEL=3x3x1
Reading weights: I=128 O=256 KERNEL=3x3x1
Reading weights: I=256 O=512 KERNEL=3x3x1
Reading weights: I=512 O=18 KERNEL=1x1x1
Not supported field: anchors = 25, 21,  36, 23,  37, 31,  49, 35,  63, 48,  98, 61
Not supported field: jitter=.3
Not supported field: ignore_thresh = .7
Not supported field: truth_thresh = 1
Not supported field: random=1

====================== NETWORK MODEL ======================
N.  Layer type       input (H*W,CH)        output (H*W,CH) 
  0 Conv2d           416 x  416,    1  ->  416 x  416,   16
  1 ActivationLeaky  416 x  416,   16  ->  416 x  416,   16
  2 Conv2d           416 x  416,   16  ->  208 x  208,   32
  3 ActivationLeaky  208 x  208,   32  ->  208 x  208,   32
  4 Conv2d           208 x  208,   32  ->  208 x  208,   16
  5 ActivationLeaky  208 x  208,   16  ->  208 x  208,   16
  6 Conv2d           208 x  208,   16  ->  208 x  208,   32
  7 ActivationLeaky  208 x  208,   32  ->  208 x  208,   32
  8 Conv2d           208 x  208,   32  ->  104 x  104,   64
  9 ActivationLeaky  104 x  104,   64  ->  104 x  104,   64
 10 Conv2d           104 x  104,   64  ->  104 x  104,   32
 11 ActivationLeaky  104 x  104,   32  ->  104 x  104,   32
 12 Conv2d           104 x  104,   32  ->  104 x  104,   64
 13 ActivationLeaky  104 x  104,   64  ->  104 x  104,   64
 14 Conv2d           104 x  104,   64  ->  104 x  104,   32
 15 ActivationLeaky  104 x  104,   32  ->  104 x  104,   32
 16 Conv2d           104 x  104,   32  ->  104 x  104,   64
 17 ActivationLeaky  104 x  104,   64  ->  104 x  104,   64
 18 Conv2d           104 x  104,   64  ->  104 x  104,   32
 19 ActivationLeaky  104 x  104,   32  ->  104 x  104,   32
 20 Conv2d           104 x  104,   32  ->  104 x  104,   64
 21 ActivationLeaky  104 x  104,   64  ->  104 x  104,   64
 22 Conv2d           104 x  104,   64  ->  104 x  104,   32
 23 ActivationLeaky  104 x  104,   32  ->  104 x  104,   32
 24 Route            104 x  104,   64  ->  104 x  104,   64
 25 Conv2d           104 x  104,   64  ->  104 x  104,   32
 26 ActivationLeaky  104 x  104,   32  ->  104 x  104,   32
 27 Conv2d           104 x  104,   32  ->  104 x  104,   64
 28 ActivationLeaky  104 x  104,   64  ->  104 x  104,   64
 29 Conv2d           104 x  104,   64  ->  104 x  104,   32
 30 ActivationLeaky  104 x  104,   32  ->  104 x  104,   32
 31 Route            104 x  104,   64  ->  104 x  104,   64
 32 Conv2d           104 x  104,   64  ->  104 x  104,   32
 33 ActivationLeaky  104 x  104,   32  ->  104 x  104,   32
 34 Route            104 x  104,   64  ->  104 x  104,   64
 35 Conv2d           104 x  104,   64  ->  104 x  104,   32
 36 ActivationLeaky  104 x  104,   32  ->  104 x  104,   32
 37 Route            104 x  104,   64  ->  104 x  104,   64
 38 Conv2d           104 x  104,   64  ->   52 x   52,  128
 39 ActivationLeaky   52 x   52,  128  ->   52 x   52,  128
 40 Conv2d            52 x   52,  128  ->   52 x   52,   64
 41 ActivationLeaky   52 x   52,   64  ->   52 x   52,   64
 42 Conv2d            52 x   52,   64  ->   52 x   52,  128
 43 ActivationLeaky   52 x   52,  128  ->   52 x   52,  128
 44 Conv2d            52 x   52,  128  ->   52 x   52,   64
 45 ActivationLeaky   52 x   52,   64  ->   52 x   52,   64
 46 Conv2d            52 x   52,   64  ->   52 x   52,  128
 47 ActivationLeaky   52 x   52,  128  ->   52 x   52,  128
 48 Route            104 x  104,   64  ->  104 x  104,   64
 49 Conv2d           104 x  104,   64  ->  104 x  104,   16
 50 ActivationLeaky  104 x  104,   16  ->  104 x  104,   16
 51 Reorg            104 x  104,   16  ->   52 x   52,   64
 52 Route             52 x   52,  192  ->   52 x   52,  192
 53 Conv2d            52 x   52,  192  ->   52 x   52,  128
 54 ActivationLeaky   52 x   52,  128  ->   52 x   52,  128
 55 Conv2d            52 x   52,  128  ->   52 x   52,  256
 56 ActivationLeaky   52 x   52,  256  ->   52 x   52,  256
 57 Conv2d            52 x   52,  256  ->   52 x   52,   18
 58 Yolo              52 x   52,   18  ->   52 x   52,   18
 59 Route            208 x  208,   32  ->  208 x  208,   32
 60 Pooling          208 x  208,   32  ->  104 x  104,   32
 61 Conv2d           104 x  104,   32  ->  104 x  104,   64
 62 ActivationLeaky  104 x  104,   64  ->  104 x  104,   64
 63 Pooling          104 x  104,   64  ->   52 x   52,   64
 64 Conv2d            52 x   52,   64  ->   52 x   52,  128
 65 ActivationLeaky   52 x   52,  128  ->   52 x   52,  128
 66 Pooling           52 x   52,  128  ->   26 x   26,  128
 67 Conv2d            26 x   26,  128  ->   26 x   26,  256
 68 ActivationLeaky   26 x   26,  256  ->   26 x   26,  256
 69 Pooling           26 x   26,  256  ->   13 x   13,  256
 70 Conv2d            13 x   13,  256  ->   13 x   13,  512
 71 ActivationLeaky   13 x   13,  512  ->   13 x   13,  512
 72 Pooling           13 x   13,  512  ->   13 x   13,  512
 73 Conv2d            13 x   13,  512  ->   13 x   13,   18
 74 Yolo              13 x   13,   18  ->   13 x   13,   18
===========================================================

GPU free memory: 324.542 mb.
New NetworkRT (TensorRT v7.1)
Float16 support: 1
Int8 support: 0
DLAs: 0
Selected maxBatchSize: 1
GPU free memory: 79.02 mb.
Building tensorRT cuda engine...
cloud not build cuda engine
.../tkDNN/src/NetworkRT.cpp:145
Aborting...

Any help?

Hi The [AlexeyAB/darknet](https://github.com/AlexeyAB/darknet) repository, supports dilated convolution. I tried to add the dilation support to tkDNN but it results in error. Changes can be seen in [my fork here](https://github.com/mrhosseini/tkDNN). This is the output of a test program (`test_yolo3_tiny_dilation`) created based on `test_yolo3tiny`, using [this cfg file](https://github.com/ceccocats/tkDNN/files/4776580/config.cfg.txt): ``` Not supported field: batch=128 Not supported field: subdivisions=16 Not supported field: momentum=0.9 Not supported field: decay=0.0005 Not supported field: angle=0 Not supported field: saturation = 1.5 Not supported field: exposure = 1.15 Not supported field: hue=0 Not supported field: learning_rate=0.001 Not supported field: burn_in=1000 Not supported field: max_batches = 15000 Not supported field: policy=steps Not supported field: steps=13000,14000 Not supported field: scales=.1,.1 New NETWORK (tkDNN v0.5, CUDNN v8) Reading weights: I=1 O=16 KERNEL=3x3x1 Reading weights: I=16 O=32 KERNEL=3x3x1 Reading weights: I=32 O=16 KERNEL=1x1x1 Reading weights: I=16 O=32 KERNEL=3x3x1 Reading weights: I=32 O=64 KERNEL=3x3x1 Reading weights: I=64 O=32 KERNEL=1x1x1 Reading weights: I=32 O=64 KERNEL=3x3x1 Reading weights: I=64 O=32 KERNEL=1x1x1 Reading weights: I=32 O=64 KERNEL=3x3x1 Reading weights: I=64 O=32 KERNEL=1x1x1 Reading weights: I=32 O=64 KERNEL=3x3x1 Reading weights: I=64 O=32 KERNEL=1x1x1 Reading weights: I=64 O=32 KERNEL=1x1x1 Reading weights: I=32 O=64 KERNEL=3x3x1 Reading weights: I=64 O=32 KERNEL=1x1x1 Reading weights: I=64 O=32 KERNEL=1x1x1 Reading weights: I=64 O=32 KERNEL=1x1x1 Reading weights: I=64 O=128 KERNEL=3x3x1 Reading weights: I=128 O=64 KERNEL=1x1x1 Reading weights: I=64 O=128 KERNEL=3x3x1 Reading weights: I=128 O=64 KERNEL=1x1x1 Reading weights: I=64 O=128 KERNEL=3x3x1 Reading weights: I=64 O=16 KERNEL=1x1x1 Reading weights: I=192 O=128 KERNEL=1x1x1 Reading weights: I=128 O=256 KERNEL=3x3x1 Reading weights: I=256 O=18 KERNEL=1x1x1 Not supported field: anchors = 25, 21, 36, 23, 37, 31, 49, 35, 63, 48, 98, 61 Not supported field: jitter=.3 Not supported field: ignore_thresh = .7 Not supported field: truth_thresh = 1 Not supported field: random=1 Reading weights: I=32 O=64 KERNEL=3x3x1 Reading weights: I=64 O=128 KERNEL=3x3x1 Reading weights: I=128 O=256 KERNEL=3x3x1 Reading weights: I=256 O=512 KERNEL=3x3x1 Reading weights: I=512 O=18 KERNEL=1x1x1 Not supported field: anchors = 25, 21, 36, 23, 37, 31, 49, 35, 63, 48, 98, 61 Not supported field: jitter=.3 Not supported field: ignore_thresh = .7 Not supported field: truth_thresh = 1 Not supported field: random=1 ====================== NETWORK MODEL ====================== N. Layer type input (H*W,CH) output (H*W,CH) 0 Conv2d 416 x 416, 1 -> 416 x 416, 16 1 ActivationLeaky 416 x 416, 16 -> 416 x 416, 16 2 Conv2d 416 x 416, 16 -> 208 x 208, 32 3 ActivationLeaky 208 x 208, 32 -> 208 x 208, 32 4 Conv2d 208 x 208, 32 -> 208 x 208, 16 5 ActivationLeaky 208 x 208, 16 -> 208 x 208, 16 6 Conv2d 208 x 208, 16 -> 208 x 208, 32 7 ActivationLeaky 208 x 208, 32 -> 208 x 208, 32 8 Conv2d 208 x 208, 32 -> 104 x 104, 64 9 ActivationLeaky 104 x 104, 64 -> 104 x 104, 64 10 Conv2d 104 x 104, 64 -> 104 x 104, 32 11 ActivationLeaky 104 x 104, 32 -> 104 x 104, 32 12 Conv2d 104 x 104, 32 -> 104 x 104, 64 13 ActivationLeaky 104 x 104, 64 -> 104 x 104, 64 14 Conv2d 104 x 104, 64 -> 104 x 104, 32 15 ActivationLeaky 104 x 104, 32 -> 104 x 104, 32 16 Conv2d 104 x 104, 32 -> 104 x 104, 64 17 ActivationLeaky 104 x 104, 64 -> 104 x 104, 64 18 Conv2d 104 x 104, 64 -> 104 x 104, 32 19 ActivationLeaky 104 x 104, 32 -> 104 x 104, 32 20 Conv2d 104 x 104, 32 -> 104 x 104, 64 21 ActivationLeaky 104 x 104, 64 -> 104 x 104, 64 22 Conv2d 104 x 104, 64 -> 104 x 104, 32 23 ActivationLeaky 104 x 104, 32 -> 104 x 104, 32 24 Route 104 x 104, 64 -> 104 x 104, 64 25 Conv2d 104 x 104, 64 -> 104 x 104, 32 26 ActivationLeaky 104 x 104, 32 -> 104 x 104, 32 27 Conv2d 104 x 104, 32 -> 104 x 104, 64 28 ActivationLeaky 104 x 104, 64 -> 104 x 104, 64 29 Conv2d 104 x 104, 64 -> 104 x 104, 32 30 ActivationLeaky 104 x 104, 32 -> 104 x 104, 32 31 Route 104 x 104, 64 -> 104 x 104, 64 32 Conv2d 104 x 104, 64 -> 104 x 104, 32 33 ActivationLeaky 104 x 104, 32 -> 104 x 104, 32 34 Route 104 x 104, 64 -> 104 x 104, 64 35 Conv2d 104 x 104, 64 -> 104 x 104, 32 36 ActivationLeaky 104 x 104, 32 -> 104 x 104, 32 37 Route 104 x 104, 64 -> 104 x 104, 64 38 Conv2d 104 x 104, 64 -> 52 x 52, 128 39 ActivationLeaky 52 x 52, 128 -> 52 x 52, 128 40 Conv2d 52 x 52, 128 -> 52 x 52, 64 41 ActivationLeaky 52 x 52, 64 -> 52 x 52, 64 42 Conv2d 52 x 52, 64 -> 52 x 52, 128 43 ActivationLeaky 52 x 52, 128 -> 52 x 52, 128 44 Conv2d 52 x 52, 128 -> 52 x 52, 64 45 ActivationLeaky 52 x 52, 64 -> 52 x 52, 64 46 Conv2d 52 x 52, 64 -> 52 x 52, 128 47 ActivationLeaky 52 x 52, 128 -> 52 x 52, 128 48 Route 104 x 104, 64 -> 104 x 104, 64 49 Conv2d 104 x 104, 64 -> 104 x 104, 16 50 ActivationLeaky 104 x 104, 16 -> 104 x 104, 16 51 Reorg 104 x 104, 16 -> 52 x 52, 64 52 Route 52 x 52, 192 -> 52 x 52, 192 53 Conv2d 52 x 52, 192 -> 52 x 52, 128 54 ActivationLeaky 52 x 52, 128 -> 52 x 52, 128 55 Conv2d 52 x 52, 128 -> 52 x 52, 256 56 ActivationLeaky 52 x 52, 256 -> 52 x 52, 256 57 Conv2d 52 x 52, 256 -> 52 x 52, 18 58 Yolo 52 x 52, 18 -> 52 x 52, 18 59 Route 208 x 208, 32 -> 208 x 208, 32 60 Pooling 208 x 208, 32 -> 104 x 104, 32 61 Conv2d 104 x 104, 32 -> 104 x 104, 64 62 ActivationLeaky 104 x 104, 64 -> 104 x 104, 64 63 Pooling 104 x 104, 64 -> 52 x 52, 64 64 Conv2d 52 x 52, 64 -> 52 x 52, 128 65 ActivationLeaky 52 x 52, 128 -> 52 x 52, 128 66 Pooling 52 x 52, 128 -> 26 x 26, 128 67 Conv2d 26 x 26, 128 -> 26 x 26, 256 68 ActivationLeaky 26 x 26, 256 -> 26 x 26, 256 69 Pooling 26 x 26, 256 -> 13 x 13, 256 70 Conv2d 13 x 13, 256 -> 13 x 13, 512 71 ActivationLeaky 13 x 13, 512 -> 13 x 13, 512 72 Pooling 13 x 13, 512 -> 13 x 13, 512 73 Conv2d 13 x 13, 512 -> 13 x 13, 18 74 Yolo 13 x 13, 18 -> 13 x 13, 18 =========================================================== GPU free memory: 324.542 mb. New NetworkRT (TensorRT v7.1) Float16 support: 1 Int8 support: 0 DLAs: 0 Selected maxBatchSize: 1 GPU free memory: 79.02 mb. Building tensorRT cuda engine... cloud not build cuda engine .../tkDNN/src/NetworkRT.cpp:145 Aborting... ``` Any help?
ceccocats commented 2020-06-14 17:06:12 +02:00 (Migrated from github.com)
Copy Link
Copy Source

Have you tried without tensorrt? Does it work?
Enable tensorrt debug to get more info on the reason why tensorrt stopped:
cmake .. -DDEBUG=True

In Conv2d.cpp you multiply padding*dilation, maybe you should do the same in NetworkRT

Have you tried without tensorrt? Does it work? Enable tensorrt debug to get more info on the reason why tensorrt stopped: `cmake .. -DDEBUG=True` In Conv2d.cpp you multiply padding*dilation, maybe you should do the same in NetworkRT
mrhosseini commented 2020-06-15 07:47:29 +02:00 (Migrated from github.com)
Copy Link
Copy Source

Thank you Francesco.

Have you tried without tensorrt? Does it work?

No, I only tested with tensorrt.

In Conv2d.cpp you multiply padding*dilation, maybe you should do the same in NetworkRT

Yes. You are probably right.

I will check the above with DEBUG flag and report the results.

Thank you Francesco. > Have you tried without tensorrt? Does it work? No, I only tested with tensorrt. > In Conv2d.cpp you multiply padding*dilation, maybe you should do the same in NetworkRT Yes. You are probably right. I will check the above with `DEBUG` flag and report the results.
mrhosseini commented 2020-06-18 14:25:40 +02:00 (Migrated from github.com)
Copy Link
Copy Source

In Conv2d.cpp you multiply padding*dilation, maybe you should do the same in NetworkRT

This solved the problem. Thanks for your great help.

> In Conv2d.cpp you multiply padding*dilation, maybe you should do the same in NetworkRT This solved the problem. Thanks for your great help.
This repo is archived. You cannot comment on issues.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
duplicate
enhancement
explanation
help wanted
invalid
question
wontfix
No Label
Milestone
No items
No Milestone
Assignees
Clear assignees
eeeck (Erick)
No Assignees
1 Participants
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: mmr/tkDNN#37
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?