outputs are different for torch and trt

[pkraison]

i'm getting a weird output with tensorrt, for the same image torch is giving good results. any help would be appreciated

[TrojanXu]

FP16 or FP32? How about the max_diff printed by main.py?

[pkraison]

Traceback (most recent call last):
File "/media/ssd/yolov5N/yolov5-tensorrt/main.py", line 222, in
diff = abs(a-b)
ValueError: operands could not be broadcast together with shapes (1,214200,7) (1,17640,7)
Segmentation fault (core dumped)
I'm using default FP32

[TrojanXu]

Can you provide the image for reproduction?

[pkraison]

okay, I'm getting a max difference of 0.0007 between torch and tensorrt. But when I'm passing the array to nms, it's giving none

[pkraison]

how can I visualize the results

[TrojanXu]

That's strange. You can create a torch tensor from the numpy array and feed the tensor to nms module. I'm planning to implement it in following days.

[pkraison]

I did that, but something is wrong with it Sent from LG.

…

On Sun, 28 Jun, 2020, 4:51 PM autoregister, ***@***.***> wrote: That's strange. You can create a torch tensor from the numpy array and feed the tensor to nms module. I'm planning to implement it in following days. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#15 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE4JGPTZGCVE7KBDIBPCJRDRY4RUTANCNFSM4OKFI3LA> .

[imprashr]

it's working fine but when i try to infer over a directory of images, it gives pycuda error after 8 images

[TrojanXu]

I've implemented NMS now. For previous shape mismatch problem, I think it's caused by variant shapes of images. Torch model can adapt to that, but TRT only supports fixed input size.

[TrojanXu]

it's working fine but when i try to infer over a directory of images, it gives pycuda error after 8 images

Did you call allocate buffers for multiple times? Maybe you run into OOM error on GPU.

[wudashuo]

Hello, I've met exactly the same problem.
This is the inference from the original .pt file(yolov5l), it looks fine：

And after I convert pt to onnx, the inference became wired, some output numbers are smaller than 0.
I tried the original pt file replacing nn.Upsample with Upsample, it also became wired, the inference is almost exactly the same with the onnx:

Is replacing nn.Upsample that caused the problem?
Should I replace nn.Upsample with your Upsample and retrain yolov5 from the beginning?

[orangeccc]

same problem as @wudashuo

[wudashuo]

same problem as @wudashuo

yolov5现在release2.0版了，然后Tutorials里也有TensorRT Deployment了，你可以去看看，我已经成功了，检测也正常了，但是必须用最新的代码重新训练模型，因为2.0的模型结构变了。

[orangeccc]

yolov5现在release2.0版了，然后Tutorials里也有TensorRT Deployment了，你可以去看看，我已经成功了，检测也正常了，但是必须用最新的代码重新训练模型，因为2.0的模型结构变了。

好的，谢谢。

[TrojanXu]

Updated to v3.0. Upsample is no longer needed to be replaced. Note that, the GPU NMS is different from the default NMS in pytorch.