Failed caps renegotiation when using NVIDIA nvv4l2decoder with interpipes

[rrcarlosrodriguez]

PC Test

1. Start GstD
   GST_DEBUG=2,*perf*:6,*EVENT*:9 gstd
2. Create a pipeline with SW encoder, SW decoder and interpipes, at a framerate of 30fps:
   gstd-client pipeline_create p0 videotestsrc is-live=true ! "video/x-raw,width=640,height=480,framerate=30/1" ! videorate name=rate drop-only=true ! perf ! x264enc ! avdec_h264 ! interpipesink name=frames sync=false forward-events=true drop=false interpipesrc allow-renegotiation=true listen-to=frames format=time ! perf ! fakesink sync=false
3. Play the pipeline
   gstd-client pipeline_play p0
4. Change framerate in the videorate to 10fps:
   gstd-client element_set p0 rate max-rate 10

Result

And the behavior is as expected, there is a new caps event triggered, the renegotiation happens, and the framerate changes from 30 to 10 in the perf element, which means that everything went well.

TX2 Test

1. Start GstD
   GST_DEBUG=2,*perf*:6,*EVENT*:9 gstd
2. Create a pipeline with NVIDIA encoder, NVIDIA decoder and interpipes, at a framerate of 30fps:
   gstd-client pipeline_create p0 videotestsrc is-live=true ! "video/x-raw,width=640,height=480,framerate=30/1" ! videorate name=rate drop-only=true ! nvvidconv ! nvv4l2h264enc ! nvv4l2decoder ! nvvidconv ! interpipesink name=frames sync=false forward-events=true drop=false interpipesrc allow-renegotiation=true listen-to=frames format=time ! perf ! fakesink sync=false
3. Play the pipeline
   gstd-client pipeline_play p0
4. Change framerate in the videorate to 10fps:
   gstd-client element_set p0 rate max-rate 10

Result

The pipeline fails to renegotiate and it gets stuck, no more buffers flow through interpipes, because the interpipesrc is disconnected from the interpipesink.

Analysis

Based on the 2 results, it looks like the problem is not entirely caused by interpipes, but more related to a combination between interpipes and the NVIDIA decoder when renegotiation caps. Maybe the decoder is not handling properly the caps renegotiation when asked, as the SW decoder does, but that is just a theory.

[hungxiu1995]

Hi, any update on this issue?

[dzhang28]

I met this issue recently.
It seems that once caps is negotiated, the interpipesink->caps_negotiated would be fixed.
So if upstream triggers caps renegotiation with different framerate, the caps query would be failed.
It's because the new caps filter passed to gst_inter_pipe_sink_get_caps would carry new framerate.
But the intersection of listeners would lead to same caps result as interpipesink->caps_negotiated with old framerate.
So no intersection can be made between the old interpipesink->caps_negotiated and the given filter.
Then it would cause the listener being removed and no further buffer could be pushed to the listener since then.

It could be solved by https://github.com/RidgeRun/gst-interpipe/compare/master...dzhang28:gst-interpipe:hotfix/allow-caps-renegotiation-in-interpipesink
But I didn't test it enough yet.