native memory still exists after Pipeline.close

[JamesTangChengDu]

[neilcsmith-net]

Please provide contextual information and a code example that demonstrates this problem, or this issue will be closed. This is not a general problem, so is being triggered by something specific to your usage or code. Thanks.

[JamesTangChengDu]

Thanks for your reply, I will upload a demo code later.

[JamesTangChengDu]

1, I execute the following command to run my demo, The heap capacity of the jvm is 3M.
java -Xmx3M -verbose:gc -jar /mnt/tangjun/demo/vdm-0.0.1.jar

2, The source code of my program is as follows: (where the keepAlive function is just to prevent the program from exiting)
/**

• the main entry of GStreamer Demo.
  */
  public class App {
  private static final int GST_VERSION_MAJOR = 1;
  private static final int GST_VERSION_MINOR = 10;

  /**

  • CM App instance.
    */
    public App() {
    Gst.init(new Version(GST_VERSION_MAJOR, GST_VERSION_MINOR));
    //testGst();

    keepAlive();
    Gst.deinit();
    }

  private void testGst() {
  final Pipeline mPipeline = new Pipeline();

   String url = "rtsp://admin:[email protected]/onvif-media/media.amp?profile=profile_1_h264&sessiontimeout=60&streamtype=unicast";
  
   Element src = ElementFactory.make("rtspsrc", "src");
   src.set("location", url);
   src.set("latency", 0);
   src.set("ntp-time-source", 3);
  
   final Element sink = ElementFactory.make("fakesink", "sink");
  
  
   mPipeline.addMany(src, sink);
  
   src.connect(new Element.PAD_ADDED() {
       @Override
       public void padAdded(Element element, Pad pad) {
           Pad sinkPad = sink.getStaticPad("sink");
           try {
               pad.link(sinkPad);
  
               System.out.println("Pad " + pad.getName() + " link to Pad"
                       + sinkPad.getName());
           } catch (PadLinkException ex) {
               System.out.println("Pad link failed : " + ex.getLinkResult());
           }
       }
   });
  
   final String gstCmd = "rtspsrc location="+url+" latency=0 ntp-time-source=3 ! faksink";
  
   System.out.println("demo gstreamer play: " + gstCmd);
   mPipeline.play();
  
   final int buffer_cacher_waiting_time1 = 1000 * 10; // in milliseconds
   new java.util.Timer().schedule(new java.util.TimerTask() {
       @Override
       public void run() {
           System.out.println("demo gstreamer stop: "+ gstCmd);
           mPipeline.stop();
  
           System.out.println("demo gstreamer close: "+ gstCmd);
           mPipeline.close();
       }
   }, buffer_cacher_waiting_time1);
  

  }

  /**

  • keep main thread alive.
    */
    public void keepAlive() {
    synchronized (this) {
    try {
    this.wait();
    } catch (InterruptedException ex) {
    ex.printStackTrace();
    }
    }
    }

  /**

  • The main function for CM.

  */
  public static void main(String[] args) {
  // keep main thread alive
  App main = new App();
  }
  }

3, Below I have 3 test results:
3.1, when the testGst function is commented out in the code, top RES=32.9m. ("top RES=32.9m" means that after executing top command, the value of RES is 32.9M)
3.2, When the testGst function is executed in the code and mPipeline.play() is run, but it has not run to mPipeline.stop, top RES=44.9m
3.3, When 3.2 continues to run, and after mPipeline.stop and mPipeline.close are executed, top RES=46.9m, and the memory does not decrease but increases.
I'm guessing the memory should go back to around "top RES=32.9m" after step 3, right?

[JamesTangChengDu]

The code typesetting is a bit unfriendly, I rearranged it as follows for your convenience:

public class App {

private static final int GST_VERSION_MAJOR = 1;
private static final int GST_VERSION_MINOR = 10;

public App() {
    Gst.init(new Version(GST_VERSION_MAJOR, GST_VERSION_MINOR));
    testGst();

    keepAlive();
    Gst.deinit();
}

private void testGst() {
    final Pipeline mPipeline = new Pipeline();

    String url = "rtsp://admin:[email protected]/onvif-media/media.amp?profile=profile_1_h264&sessiontimeout=60&streamtype=unicast";

    Element src = ElementFactory.make("rtspsrc", "src");
    src.set("location", url);
    src.set("latency", 0);
    src.set("ntp-time-source", 3);

    final Element sink = ElementFactory.make("fakesink", "sink");


    mPipeline.addMany(src, sink);

    src.connect(new Element.PAD_ADDED() {
        @Override
        public void padAdded(Element element, Pad pad) {
            Pad sinkPad = sink.getStaticPad("sink");
            try {
                pad.link(sinkPad);

                System.out.println("Pad " + pad.getName() + " link to Pad"
                        + sinkPad.getName());
            } catch (PadLinkException ex) {
                System.out.println("Pad link failed : " + ex.getLinkResult());
            }
        }
    });

    final String gstCmd = "rtspsrc location="+url+" latency=0 ntp-time-source=3 ! faksink";

    System.out.println("demo gstreamer play: " + gstCmd);
    mPipeline.play();

    final int buffer_cacher_waiting_time1 = 1000 * 10; // in milliseconds
    new java.util.Timer().schedule(new java.util.TimerTask() {
        @Override
        public void run() {
            System.out.println("demo gstreamer stop: "+ gstCmd);
            mPipeline.stop();

            System.out.println("demo gstreamer close: "+ gstCmd);
            mPipeline.close();
        }
    }, buffer_cacher_waiting_time1);
}

public void keepAlive() {
    synchronized (this) {
        try {
            this.wait();
        } catch (InterruptedException ex) {
            ex.printStackTrace();
        }
    }
}

public static void main(String[] args) {
    App main = new App();
}

}

[JamesTangChengDu]

Looking forward to your reply, thank you very much.

[neilcsmith-net]

There are a number of problems in your code, I'm not sure your assumption that all memory will be released is necessarily correct, and it's possible you're seeing some interaction with #244 caused by calling close() off the GStreamer event thread or directly after stop().

A few pointers -

• Always store the pipeline in a field.
• Use Gst.main() and Gst.quit() to keep alive.
• Try to always interact with GStreamer, particularly after starting a pipeline, using Gst.executor() - that will correctly interleave with bus messages, and supports scheduling already (no need for Timer).
• Don't call close() immediately after stop(). Use Gst.invokeLater() if you want to do this.
• You don't need to explicitly dispose of anything, but if you do you need to make sure that other things that might interact with the same element have completed.
• Never call Gst.deinit()

Try looking at the structure of the various examples in https://github.com/gstreamer-java/gst1-java-examples

Moving this into discussions. I don't believe it's a sign of a native memory leak.

[JamesTangChengDu]

Thanks for your reply.
Always store the pipeline in a field.
====>My app needs to start several pipelines at the same time to obtain the streams of different cameras. There are many cameras, and the user decides which camera to pull.
Sometimes users just pull the stream and watch it for a while and then don't need it. At this time, I hope this pipeline completely disappears from memory.

Use Gst.main() and Gst.quit() to keep alive.
====>Gst.main() and Gst.quit() seems to be global, as mentioned above, I have several pipelines that need to work at the same time, can Gst.main() and Gst.quit() do it ? Gst.main() and Gst.quit() can work on any thread, right ?

[JamesTangChengDu]

From gst1-java-examples, I noticed a phenomenon, the process is dying after Gst.main(), and the memory is released by the process death ?

[JamesTangChengDu]

Below is the new code, the situation has not improved.
Below I have 3 test results:
1, when the code between "// test gst begin" and "// test gst end" is commented out in the code, top RES=37.4m. ("top RES=37.4m" means that after executing top command, the value of RES is 37.4M)
2, When the code between "// test gst begin" and "// test gst end" is executed in the code and mPipeline.play() is run, but it has not run to mPipeline.stop, top RES=46.6m
3, When case 2 continues to run, and after mPipeline.stop and mPipeline.close are executed, top RES=47.4m, and the memory does not decrease but increases.

public class App {

private static final int GST_VERSION_MAJOR = 1;
private static final int GST_VERSION_MINOR = 10;

private Pipeline mPipeline = null;
private Semaphore mGotEosPipeline = new Semaphore(1);

public App() {
    Gst.init(new Version(GST_VERSION_MAJOR, GST_VERSION_MINOR));
    mPipeline = new Pipeline();

    // test gst begin
    Gst.invokeLater(() -> testGst());
    Gst.getExecutor().schedule(new Runnable() {
        @Override
        public void run() {
            System.out.println("demo gstreamer eos");
            mPipeline.sendEvent(new EOSEvent());
            try {
                mGotEosPipeline.acquire(1);
            } catch (Exception ex) {
                ex.printStackTrace();
            }

            System.out.println("demo gstreamer stop");
            mPipeline.stop();

            Gst.invokeLater(new Runnable() {
                @Override
                public void run() {
                    System.out.println("demo gstreamer close");
                    mPipeline.close();

                    Gst.quit();
                }
            });
        }
    }, 10, TimeUnit.SECONDS);

    System.out.println("main");
    Gst.main();
    // test gst end

    System.out.println("keepAlive");
    keepAlive();
    //Gst.deinit();
}

private void testGst() {
    String url = "rtsp://admin:[email protected]/onvif-media/media.amp?profile=profile_1_h264&sessiontimeout=60&streamtype=unicast";

    Element src = ElementFactory.make("rtspsrc", "src");
    src.set("location", url);
    src.set("latency", 0);
    src.set("ntp-time-source", 3);

    final Element sink = ElementFactory.make("fakesink", "sink");


    mPipeline.addMany(src, sink);

    src.connect(new Element.PAD_ADDED() {
        @Override
        public void padAdded(Element element, Pad pad) {
            Pad sinkPad = sink.getStaticPad("sink");
            try {
                pad.link(sinkPad);

                System.out.println("Pad " + pad.getName() + " link to Pad"
                        + sinkPad.getName());
            } catch (PadLinkException ex) {
                System.out.println("Pad link failed : " + ex.getLinkResult());
            }
        }
    });

    final String gstCmd = "rtspsrc location="+url+" latency=0 ntp-time-source=3 ! faksink";

    mPipeline.getBus().connect((Bus.EOS) (source) -> {
        System.out.println("demo gstreamer EOS1: "+ gstCmd);
        mGotEosPipeline.release();
    });

    System.out.println("demo gstreamer play: " + gstCmd);
    mPipeline.play();
}

public void keepAlive() {
    synchronized (this) {
        try {
            this.wait();
        } catch (InterruptedException ex) {
            ex.printStackTrace();
        }
    }
}

public static void main(String[] args) {
    // keep main thread alive
    App main = new App();
}

}

[neilcsmith-net]

If you need multiple pipelines, store them in multiple fields or a list, etc. The important thing is the top level pipeline should have a reference outside any method.

The role of Gst.main() is solely to keep the main thread alive if you need it, as threads inside GStreamer don't count from a JVM perspective. It's often not needed except in simple examples. Use the scheduled executor and you shouldn't need to block threads.

Relying on the garbage collector is usually enough by itself, but if you're going to do manual memory management you need to know you need to do it, and be careful how you use it. Here you're disposing of pipeline explicitly, but relying on the GC to dispose of the extra references you create such as src and sink. They might hold references that keep native memory in use, and GStreamer might just be caching something.

Try calling System.gc() a couple of times after disposing of the pipeline (or better use VisualVM, etc. to trigger GC and inspect the memory usage and references) and see if the memory "leak" remains. That will tell you if it's caused by a live Java reference.

Best of all, write the app you want and monitor it in usage for signs of memory leaking.

[JamesTangChengDu]

I found that org.freedesktop.gstreamer.Buffer can close() to release native memory as long as it does not call disown. It seems that when calling close, it goes to org.freedesktop.gstreamer.glib.NativeObject.Handle.dispose -> disposeNativeHandle. So it can release native memory.

When I just only new Pipeline() (no other operations such as addMany, play, etc.), the native memory grows, but why does calling Pipeline.close not release the native memory ? Can you have any code level guidance about release native memory ?

In addition, No matter how many times System.gc() fails to release the native memory corresponding to the pipeline

[neilcsmith-net]

I've got various battletest programs, some for client projects, checking for such memory leaks with multiple pipelines, and don't see this on the system or hardware they've run on. Without going in to the specifics of your project and hardware, it's not something I can really help further with unless you have a fully reproducible test case.

System.gc() will have no effect on a Pipeline that has been disposed. However, the usual issue is people forgetting about other native wrappers they've created or are received in callbacks (eg. bus listeners). This is where System.gc() might show a coding problem. It should never be required in production code.

Native references for any NativeObject ges released by garbage collection or explicit disposal. Both should never be required by any individual object, but objects can reference each other and so a Java reference to a native object that holds a GObject reference eg. to the pipeline can also stop resources being cleared. Note that GStreamer uses reference counting - disposing on the Java side only releases Java's reference.

This is managed by the NativeObject.NativeRef and NativeObject.Handle - eg. at https://github.com/gstreamer-java/gst1-java-core/blob/master/src/org/freedesktop/gstreamer/glib/NativeObject.java#L235 And also by use of the toggle reference at https://github.com/gstreamer-java/gst1-java-core/blob/master/src/org/freedesktop/gstreamer/glib/GObject.java#L724 which ensures Java keeps a strong reference to any GObject referenced elsewhere in GStreamer.

We can help with specifics of architecture and implementation for commercial projects via www.codelerity.com Otherwise, I can provide generic advice only.