Convert yolov4 model to tensorrt get "Wrong" error #10

New Issue

Closed

opened 2020-05-11 13:03:35 +02:00 by lthbk5919 · 13 comments

lthbk5919 commented 2020-05-11 13:03:35 +02:00 (Migrated from github.com)

Copy Link

Copy Source

Hi, thank for this awesome project.
I have a question about converting yolov4 model.
I try run test_yolo4 with yolo-608 config and get red line Wrongs in log as shown below:

Line Wrong what dose it mean?
Thanks for the repo again.

Hi, thank for this awesome project. I have a question about converting yolov4 model. I try run `test_yolo4` with yolo-608 config and get red line `Wrongs` in log as shown below:  Line `Wrong` what dose it mean? Thanks for the repo again.

ceccocats commented 2020-05-11 13:58:14 +02:00 (Migrated from github.com)

Copy Link

Copy Source

Hi,
It means that the first yolo output layer have 102 wrong values respect to the output of darknet.
We have the same errors, so I need to investigate, the strange thing is that the "correct" values from darknet are NaN.

Hi, It means that the first yolo output layer have 102 wrong values respect to the output of darknet. We have the same errors, so I need to investigate, the strange thing is that the "correct" values from darknet are NaN.

ceccocats commented 2020-05-11 14:23:32 +02:00 (Migrated from github.com)

Copy Link

Copy Source

Since the wrong values are all NaN In the correct version is more a problem of darknet. The final bboxs should not be affected anyway.
The difference in result could be due to different implementation. The CUDNN version should not be to much different from darknet, the main difference is that that they uses a custom implementation of batchnorm and not the one from CUDNN.

Since the wrong values are all NaN In the correct version is more a problem of darknet. The final bboxs should not be affected anyway. The difference in result could be due to different implementation. The CUDNN version should not be to much different from darknet, the main difference is that that they uses a custom implementation of batchnorm and not the one from CUDNN.

lthbk5919 commented 2020-05-12 04:24:18 +02:00 (Migrated from github.com)

Copy Link

Copy Source

Hi, thank you.

The difference in result could be due to different implementation. The CUDNN version should not be to much different from darknet, the main difference is that that they uses a custom implementation of batchnorm and not the one from CUDNN

I will check it.

Hi, thank you. > The difference in result could be due to different implementation. The CUDNN version should not be to much different from darknet, the main difference is that that they uses a custom implementation of batchnorm and not the one from CUDNN I will check it.

vasanktt commented 2020-06-12 05:19:14 +02:00 (Migrated from github.com)

Copy Link

Copy Source

Hi, Thank you for your great repo.

I also meet the same error while converting Yolov4 model got trained from AlexeyAB/Darknet repo. I trying to test TensorRT model that converted but it seem I didn't get the correct result but on AlexeyAB I able to get correct result.

Hi, Thank you for your great repo. I also meet the same error while converting Yolov4 model got trained from AlexeyAB/Darknet repo. I trying to test TensorRT model that converted but it seem I didn't get the correct result but on AlexeyAB I able to get correct result.

ceccocats commented 2020-06-12 10:10:49 +02:00 (Migrated from github.com)

Copy Link

Copy Source

What you mean for correct result?

What you mean for correct result?

vasanktt commented 2020-06-12 10:42:39 +02:00 (Migrated from github.com)

Copy Link

Copy Source

I'm sorry for confused message. I mean I test my own Yolov4 model via ./darknet detector test ..... command and I got all correct object detected but when I export that model to TensorRT format and test it by ./demo yolo4_fp32.rt video.mp4 y I didn't get any object.

I'm sorry for confused message. I mean I test my own Yolov4 model via `./darknet detector test .....` command and I got all correct object detected but when I export that model to TensorRT format and test it by `./demo yolo4_fp32.rt video.mp4 y` I didn't get any object.

ceccocats commented 2020-06-12 14:31:15 +02:00 (Migrated from github.com)

Copy Link

Copy Source

It could be my reasons for this problem
#29 is trying to do the same

It could be my reasons for this problem #29 is trying to do the same

vasanktt commented 2020-06-15 12:08:03 +02:00 (Migrated from github.com)

Copy Link

Copy Source

My issue have been solved because of I forgot to change the path of cfg file in tests/darknet/yolo4.cpp.

std::string cfg_path  = "plateocr_yolov4.cnf";
std::string name_path = "plateocr_yolov4.name";

@ceccocats Thank you for your support.

My issue have been solved because of I forgot to change the path of cfg file in `tests/darknet/yolo4.cpp`. ``` std::string cfg_path = "plateocr_yolov4.cnf"; std::string name_path = "plateocr_yolov4.name"; ``` @ceccocats Thank you for your support.

👍 1

beizhengren commented 2020-07-15 04:37:10 +02:00 (Migrated from github.com)

Copy Link

Copy Source

Hi@ceccocats

Since the wrong values are all NaN In the correct version is more a problem of darknet. The final bboxs should not be affected anyway.
The difference in result could be due to different implementation. The CUDNN version should not be to much different from darknet, the main difference is that that they uses a custom implementation of batchnorm and not the one from CUDNN.

By your comment, I understand that Nans is the bug of darknet, and Nans don't affect the inference result by ./demo yolo4_fp32.rt video.mp4 y ....
Is that right?

Hi@ceccocats > Since the wrong values are all NaN In the correct version is more a problem of darknet. The final bboxs should not be affected anyway. > The difference in result could be due to different implementation. The CUDNN version should not be to much different from darknet, the main difference is that that they uses a custom implementation of batchnorm and not the one from CUDNN. By your comment, I understand that `Nans` is the bug of darknet, and `Nans` don't affect the inference result by `./demo yolo4_fp32.rt video.mp4 y ...`. Is that right?

mive93 commented 2020-07-15 18:53:41 +02:00 (Migrated from github.com)

Copy Link

Copy Source

Hi @beizhengren, that is right.

Hi @beizhengren, that is right.

beizhengren commented 2020-07-17 09:14:55 +02:00 (Migrated from github.com)

Copy Link

Copy Source

@mive93 Hi, thank you so much!

@mive93 Hi, thank you so much!

👍 1

teplofizik commented 2020-08-03 14:43:03 +02:00 (Migrated from github.com)

Copy Link

Copy Source

Have such problem at jetson xavier on converting fp16/int8 mode (fp32 ok). It can run converted on another machine models (at any mode: fp32, fp16), but cannot convert models to reduced precision. Maybe it caused by tensorrt 7. It was fixed by updating tkDNN from git.

 Have such problem at jetson xavier on converting fp16/int8 mode (fp32 ok). It can run converted on another machine models (at any mode: fp32, fp16), but cannot convert models to reduced precision. Maybe it caused by tensorrt 7. It was fixed by updating tkDNN from git.

Crystalxd commented 2021-03-12 06:22:36 +01:00 (Migrated from github.com)

Copy Link

Copy Source

@teplofizik Now I convert the yolo4 model to fp16, I still got this problem, but I can run the demo currently.

@teplofizik Now I convert the yolo4 model to fp16, I still got this problem, but I can run the demo currently.

This repo is archived. You cannot comment on issues.

No Branch/Tag Specified

Branches Tags

master

dev_rtls

mbzirc

tensorrt7

gpu-prepocess

resize

yolo4tiny

cudnn-8.0.1

class

cudnn5

cudnn2

v0.7

v0.6

v0.5

v0.4

testData

Labels

Clear labels

bug

duplicate

enhancement

explanation

help wanted

invalid

question

wontfix

No Label bug

Milestone

No items

No Milestone

Assignees

Clear assignees

eeeck (Erick)

No Assignees

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: mmr/tkDNN#10