Push Server seems to consume memory continually. Memory Leak?

[dk7td]

I installed volkszaehler last week on my RasPi 3B+ and I’m using one USB probe for my electricity meter.

Today my RasPi stopped working for the second time since then and I realized the push server php process had consumed more than 80% of the memory. There were only 19MB left in the system.

The browser frontend did not react anymore and also the ssh login was extremely slow but still possible.
After a reboot everything looked fine but some days later same effect.

Today I observed the memory consumption and found out that it started at 7%. After six hours it‘s now at 15% already. If this will go on this way, it will consume 32% per day and probably stop running again after three days.

I assume there is sort of memory leak in the push server. There was also another finding of this issue mentioned here:

https://www.photovoltaikforum.com/thread/192176-raspberry-pi-4-2gb-volkszähler-push-server-verbraucht-kompletten-ram-und-swap/

Thanks!

[dk7td]

If there is no other solution, I will need to set up a cron job and restart push server every night.

Since yesterday again it allocated 35% of memory.

[dk7td]

Short info: the cron job prevents the crash of the system caused by push server. But that’s not a very nice solution.

I still hope that someone will check and perhaps optimize the memory usage of push server.

[jahir]

the push server is merely some php code (using a bunch of modules). php does its memory management by itself, so as long as we are not doing nasty stuff (like keeping references to stuff not used any more), it should work just fine.

System details, like php version?
push server configuration?
What is your push server usage pattern? I.e. how many clients sending data, how many clients consuming data, using WAMP or or plain web socket, request/rate (e.g. per second/hour), etc.

For the time being, as the server has no restarting the server once in a while should not do much harm as it keeps no state. You could even try to limit its memory usage, see e.g. (https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html), and set up the systemd service to restart in case of failure.

[dk7td]

I‘m using the Raspberry Pi image from here https://wiki.volkszaehler.org/software/releases/start on a RasPi 3B+. I did not check what versions of php etc. are used in this image.

As mentioned above I‘m using only one USB probe for receiving data from my power meter via vzlogger.

I don’t know any internal details of the installation. The only thing I observed is the critical memory consumption of the push server php process that stops the system from working after some time.

I‘m sorry, but I’m not so deeply involved in linux system tasks so I will not be able to dig into this very deep. I’m a simple user of an officially provided image and I‘m struggling with issues caused by flaws I don’t understand completely.

[J-A-U]

How the (up-to-date) Image is configured can be looked up in Wiki: https://wiki.volkszaehler.org/howto/building_raspberry_pi_image_for_vz

Everything not specifically mentioned is 1:1 like git (or OS).

• PHP 7.4
• out of box (Port 8082)
• Frontend only does WAMP
• Request rate? I don't understand. It's push...

One thing i'm interested: Does the memory always fill or only when Frontend is open somewhere?

[dk7td]

Thanks for clarification!

My impression is that it always fills up, also without open frontend.
I‘m using mainly my iPad for the frontend and the browser is in the foreground only if I‘m viewing the frontend actively. I don’t know if there is any activity if it’s in the background or in standby mode.

[dk7td]

If I understand the push server correctly it refreshes the chart in the frontend continuously while the browser is open?
There are no other targets defined the push server sends data to.

[J-A-U]

I'm also not sure whether or not a browser tab in background will continuously hold the push-server active to store data.

Would be nice if you can test this by actively close the tab (or the whole browser).
I think there is no need to deactivate the workaround, just check memory consumption before the service is forced to restart.

Edit: push-server doesn't have defined targets, it works on subscription basis.

[dk7td]

Ok. I closed all browser tabs and restarted the push server. Now it‘s consuming 2.8% of system memory. See attached screenshot.
Swap seems also to be used about 70%. I don’t know where this results from. Is this critical in any way?

Thanks!

[dk7td]

Eight hours later: 13.2%

I will not do the restart tonight, let’s wait for tomorrow.

[dk7td]

39.1% today - and only 11.5 % of swap left.

[J-A-U]

I tried to replicate the failure over here. Push-server consumes 7% of memory, even after 24h of runtime.

Do you have some unusual notifications under systemctl status push-server ?

[dk7td]

Hi,
thanks for testing! There is nothing special configured.

Are there any other configurations I should check?

[J-A-U]

I'm out of ideas.
And also not familiar how to hunt for memory leaks... 😓

[Sn4k3E4t3r]

I have the Same Problem. Fresh Install an 4 USB IR Reader on an RPI 4 2GB an daily the RAM is full.

[YARD2]

I also have the same issue.
Use a RaspiZero and everyday 2x memory is full.
1 USB IR sensor with 1 SML meter, 2 channels.

Use a cronjob which restarts the Push-server every 6 hours

#!/bin/bash
#
# Restart push-server because of memory leak
#
sudo systemctl stop push-server
sleep 2
sudo systemctl start push-server
exit 0

[Gitsaibot]

For me, the problem described here started with the update to Bullseye and the current version of the middleware. The memory fills up regardless of whether the frontend is open or not.
With buster and php7.3 there was no problem. Now with bullseye and php7.4 unfortunately it does.

[timonue]

For me, the problem described here started with the update to Bullseye and the current version of the middleware. The memory fills up regardless of whether the frontend is open or not. With buster and php7.3 there was no problem. Now with bullseye and php7.4 unfortunately it does.

exact same issue for me.
Raspberry Pi Zero 2, used pre build Image and updated.

[r00t-]

anybody up to trying something like https://github.com/arnaud-lb/php-memory-profiler ?