Page 2 of 4
Re: High Ram usage
Posted: Mon Mar 01, 2021 8:43 am
by phgerin
Does anyone have the last 3.15.16 focal amd64 version.
I can't find it in the archive.
Thanks !
Re: High Ram usage
Posted: Mon Mar 01, 2021 9:00 am
by Magic919
Re: High Ram usage
Posted: Mon Mar 01, 2021 10:46 am
by phgerin
hmm ? I can't see any built file there.
Re: High Ram usage
Posted: Mon Mar 01, 2021 1:46 pm
by Magic919
Oops. I presume they do some cleaning up. Was there when I grabbed it a while back.
Try this
www.magic919.co.uk/zoneminder_1.35.16~2 ... _amd64.deb
Re: High Ram usage
Posted: Mon Mar 01, 2021 3:20 pm
by phgerin
Many thanks !
Re: High Ram usage
Posted: Tue Mar 09, 2021 7:31 pm
by dashie
I'm also having memory issues, which seems caused mostly by recording, memory usage grow quickly when recording, and memory is never released (unless kill of zmc processes).
already bumped the memory of the server to 55G which even at that size ends up exhausted fast. I'm at ~6G RES per zmc process right now.
I've already changed the "Image Buffer Size (frames)" to 2 on all monitors but that doesn't seemed to help.
for informations my monitors are pretty much all:
source type: ffmpeg in modect and analysis enabled
analysis/max/alarm FPS empty
source rtsp/TCP/32bit/1920x1080
save frames+analysis if available, writer H264 passthrough, outputcodec and encoder auto and container mp4 (or it saves mkv and the browser doesn't know what to do with that in auto)
buffers: 2, 25, 25, 25, 500, 1
and misc is default
Re: High Ram usage
Posted: Tue Mar 09, 2021 7:42 pm
by Magic919
Last suggestion I saw was 5 for the buffers.
Reducing the interval between keyframes (on the cameras) will help.
Re: High Ram usage
Posted: Wed Mar 10, 2021 5:49 am
by dashie
I think the buffers was a suggestion of around 2 for the image buffer size one.
I've left it at two, and changed warmup/pre/post/stream to 5 and alarm frame count to 1.
as for the keyframe, cannot be changed on any of my cameras, so no workaround there.
Re: High Ram usage
Posted: Mon Mar 22, 2021 8:47 am
by topminttwo
I answered before I had done much testing. I see it now. Some zmc processes are using enormous amounts of memory, even in Monitor mode.
Re: High Ram usage
Posted: Mon Mar 22, 2021 3:12 pm
by iconnor
I havn't been able to reproduce the high ram usage in monitor mode here. Or conceive of how it would happen.
If doing passthrough we have to keep everything from the last keyframe. We only have to keep the encoded data though. So this shouldn't be too much. My 1080p h265 cam is using 74Mb of ram.
If doing motion detection, we decode to raw buffers and these take up a lot of ram. However if motion detection is keeping up, it frees that ram as it goes, so we still don't use much ram.
The only issue is that if the analysis thread gets backed up for some reason (slow db or slow disk or bug in code) the capture process will keep capturing and stuffing data into the queue.
So you guys who are seeing this ram use, I need debug level 4 logs for zmc.
Re: High Ram usage
Posted: Mon Mar 22, 2021 4:16 pm
by iconnor
I just pushed a change that makes ZM only buffer since the last keyframe when using passthrough. Otherwise we don't need to do that, so people using Encode or saving jpegs might see a ram reduction.
Re: High Ram usage
Posted: Tue Mar 23, 2021 7:10 am
by phgerin
I have an i7 Ubuntu Focal with 16GB of ram and 16 cams (all the same, same config).
If I start Zoneminder with all the cams set to "none" and then I turn them to "modect" one by one, I have 4,4BG ram usage and zmc process are max 114mb of ram.
If I reboot the computer with all the cam enabled, the total ram usage is 15 Gb with 2Gb of swap.
zmc process are between 80Mb and 1,6Gb of ram.
The problem seems to occur when CPU usage is very high.
Re: High Ram usage
Posted: Wed Mar 24, 2021 6:20 pm
by dmidthun
I'm noticing that the memory issue is only happening when the cameras are set to record of any kind, and then drops back off when the event(s) are complete and the process starts over again (~10 mins) . I also noticed that there are 4 zmc processes per camera enabled. Could this also be attributing to the resource hogging? I have my install running with 24G of ram, 12G of swap and can literally watch the usage increase even though there is little to no change in the image being captured (an empty room in my house that gets no foot traffic -- ever). 10 rtsp cameras set to monitor == 4.5~5.5G RAM; change 1 to Mocord, system will again start hoarding memory.
Isaac, is there possibly a memory leak in the zmc process when in record mode?
Thanks,
Dan
Setup:
Ubuntu 18.04
ZM 1.35/master (latest)
8 Merkury 720p 'smartcams; hacked'
Ovirt 4.4.4 KVM Env - 2CPU, 24G RAM, 72G vdrive; 60sda 12sdb(swap) - Nimble backend storage @ 10G FC
Re: High Ram usage
Posted: Wed Mar 24, 2021 6:46 pm
by dmidthun
iconnor wrote: ↑Mon Mar 22, 2021 3:12 pm
So you guys who are seeing this ram use, I need debug level 4 logs for zmc.
Forgive my ignorance - where/how can i get this information to give you?
Dan
Re: High Ram usage
Posted: Thu Mar 25, 2021 5:13 pm
by iconnor
Possible? Yes.
You didn't say whether you are doing passthrough or encode. Encoding takes a ton of ram too (default settings will buffer 20 frames in the encoder).
Passthrough can buffer some stuff but it is encoded data and should just be ref counted so shouldn't amount to much.
dmidthun wrote: ↑Wed Mar 24, 2021 6:20 pm
I'm noticing that the memory issue is only happening when the cameras are set to record of any kind, and then drops back off when the event(s) are complete and the process starts over again (~10 mins) . I also noticed that there are 4 zmc processes per camera enabled. Could this also be attributing to the resource hogging? I have my install running with 24G of ram, 12G of swap and can literally watch the usage increase even though there is little to no change in the image being captured (an empty room in my house that gets no foot traffic -- ever). 10 rtsp cameras set to monitor == 4.5~5.5G RAM; change 1 to Mocord, system will again start hoarding memory.
Isaac, is there possibly a memory leak in the zmc process when in record mode?
Thanks,
Dan
Setup:
Ubuntu 18.04
ZM 1.35/master (latest)
8 Merkury 720p 'smartcams; hacked'
Ovirt 4.4.4 KVM Env - 2CPU, 24G RAM, 72G vdrive; 60sda 12sdb(swap) - Nimble backend storage @ 10G FC