gst-shark with custom application (OpenCV)

[ll-CrYo-ll]

Hey guys,

Thanks for the great work on gst-shark, it is a great tool to use! Similar to Issue #59 I am trying to build tracing capability into a custom application for review at a later stage. The issue I am facing seems to be a bit different though. For reference, gstreamer version is 1.16.2, OpenCV version is 3.4.10.

I have not yet had the need to create any loops using the gst_init or g_main_loop_new methods, it seems as though gst-shark does create the output files in the correct locations and has tracing capability. However, there seems to be small issue with the generation of the files themselves (specifically the metadata and datastream files).

When running the traditional gst-launch-1.0 approach, the files seem to be correctly created and traced according to the selected items that are exported. I have attached a copy of this labelled as metadata_correct (just renamed from metadata). Plotting works perfectly with this approach.

When running the wrapped version of the call, for example an OpenCV wrapped gstreamer pipeline using appsrc it can be seen that the files are still created, and in some cases certain items are traced and can be plotted afterwards. However, there seems to be an issue with the creation of the metadata file specifically, it seems to get chopped off before it is finished, more so at 4096 bytes exactly, which seems dubious. I have attached a copy of this example labelled as metadata_broken.

From metadata_broken it can be seen that cpuusage is not fully defined, and as such the datastream file is hindered during run-time it seems and plotting afterwards is no longer possible.

I hope that you could maybe help to spot what could be wrong here, and I really appreciate the time!

Best regards.
metadata_versions.zip

[michaelgruner]

Hey! Thanks for the kind words. I'm glad you're finding it useful, that's the main purpose.

To minimize IO overhead we do not write every single tracer event, but instead we leverage the OS caching mechanisms so that data is written until a certain size threshold is reached or the application is closed. I think you are getting the later.

Can you try setting the following environment variable to zero?

GST_SHARK_FILE_BUFFERING=0

[ll-CrYo-ll]

Hi Michael,

Thanks for the super quick response, much appreciated! :D

I tried setting the above-mentioned variable but so far no luck (no change in result). I have however noticed that with the traditional gst-launch-1.0 method the metadata file is created but not finished, only when the pipeline is manually closed does it finish the definition of the file. And as per your mention above I can now confirm that through the OpenCV wrapped gstreamer pipeline (via VideoCapture), when the application is closed it does not finish the definition of the file.

Below is an output message from the application when closed:

GStreamer-CRITICAL **: 09:27:44.995: Trying to dispose element videoconvert0, but it is in READY instead of the NULL state. You need to explicitly set elements to the NULL state before dropping the final reference, to allow them to clean up. This problem may also be caused by a refcounting bug in the application or some element.

The same message is presented for each element within the gstreamer pipeline defined, for example the source, sink and in-between elements such as the videoconvert element above.

This led me to believe that when the application is closed manually, the OpenCV VideoCapture object does not fully unref and close the wrapped gstreamer pipeline in the background for safe shutdown of the pipeline. And if this pipeline is not safely closed by OpenCV then the gst-shark metadata file will not be fully defined. I have tried to explicitly close the OpenCV VideoCapture but still no luck just yet.

Do you think I should perhaps rather consider just using the base gstreamer C++ interface instead of wrapping the pipeline through OpenCV?

Thanks again for the time and consideration!

Best regards.

[michaelgruner]

Sorry for the delay! Thanks for clarifying that this was done via VideoCapture. Actually, I think the metadata can be made to be written at the very beginning. Let me see if I can draft something real quick.

[michaelgruner]

Hey @CrYo-Dyne I just pushed the hotfix/flush-metadata-file-at-startup branch. It should get rid of the truncated metadata file problem. Do you mind confirming?

Having said that, there are still a few details that don't fully match to me:

• You metadata is truncated at 4096, but it normally weights around 1.3K
• GST_SHARK_FILE_BUFFERING=0 did flush the metadata for me

I'm investigating possible causes.

[michaelgruner]

Update:

Found that OpenCV Video Capture actually doesn't start a main loop. As such, some tracers won't work properly. Specifically the periodic ones, since they install a periodic timeout that depends on such event loop. I've created a branch with an improvement I plan on pushing to OpenCV on our fork.

https://github.com/RidgeRun/opencv/tree/optional-gstreamer-mainloop

Here are instructions on how to build our fork if you'd like to give it a try (remember to switch to the respective branch).

[ll-CrYo-ll]

Hi @michaelgruner

Apologies for the late response on my behalf! Thank you so much for the info and for the investigation as well, I really appreciate it :)

I have not yet been able to test this on my end, hopefully I will be able to do so in the next few days after which I can let you know how it goes.

Once again thanks for the time and consideration, much appreciated!