Page 1 of 1

Strange Memory consuption pattern Decode Ondemand vs KeyFrames

Posted: Mon May 09, 2022 9:48 pm
by MarsRover
Hi again!
1.37.15_20220507113926-focal
I've noticed strange zmc behavior when Decoding is changed between KeyFrames and Ondemand, and when there's no Decoding demand (Record always, passthrough, no Analysis and no cameras are watched Live - "pure" DVR setup).

KeyFrames -> Ondemand:
CPU goes DOWN for most zmc processes >x2 this is understandable. Stays the same after. But for some znc processes it goes up, this is strange..
Memory for most zmc processes goes UP about x2 - this is strange. Stays the same after. Goes down for some zmc processes, this is also strange, although it's what I was expecting to acheive

The same, but mirrored, happens on the swithcback.

Why this is happening?

Zabbix Graph for zmc processes attached (system with 12 cameras connected. Bulk change set Decoding='Ondemand' in zm.Monitiors table and then systemctl restart zoneminder at ~ 23:03:00. Bold lines are the sum of %CPU or %MEM for all zmc processes (right axis). All other lines reflect CPU and Mem metrics for individual zmc processes (left axis). Recording SectionLength=600.
All metrics are collected once per minute by parsing ps output

Code: Select all

 ~$ ps -C zmc -o %mem -o "|%C|" -o cmd
%MEM|%CPU|CMD
 6.7| 1.6|/usr/bin/zmc -m 1
 7.8| 1.8|/usr/bin/zmc -m 2
 6.8| 1.7|/usr/bin/zmc -m 3
 6.6| 2.1|/usr/bin/zmc -m 4
 7.9| 2.1|/usr/bin/zmc -m 5
 7.8| 2.1|/usr/bin/zmc -m 6
 8.2| 1.9|/usr/bin/zmc -m 8
 7.0| 1.8|/usr/bin/zmc -m 9
11.1| 2.3|/usr/bin/zmc -m 11
11.0| 2.7|/usr/bin/zmc -m 14
 0.1| 1.4|/usr/bin/zmc -m 15
 0.1| 1.3|/usr/bin/zmc -m 16

Re: Strange Memory consuption pattern Decode Ondemand vs KeyFrames

Posted: Tue May 10, 2022 5:31 pm
by iconnor
Do you have rtsp server turned on for any of the monitors? It counts as a viewer.

I'm not seeing this here.

Turn on debug at level 3 and send me the log. isaac@zoneminder.com

Re: Strange Memory consuption pattern Decode Ondemand vs KeyFrames

Posted: Tue May 10, 2022 9:03 pm
by MarsRover
No rtsp server, no Janus Live Stream for all cameras.

Code: Select all

"JanusEnabled":false
"RTSPServer":false
Even thumbnails are off on the server level.

debug log for _zmc_mX component?

offtopic:
how do I query events with EndDateTime IS NULL via API?
/api/events/index/MonitorId:4/EndDateTime%20IS:NULL.json gives me:
"check the manual that corresponds to your MySQL server version for the right syntax to use near '= 'NULL'"

Re: Strange Memory consuption pattern Decode Ondemand vs KeyFrames

Posted: Wed May 11, 2022 2:24 pm
by MarsRover
OK. More pictures. I've created a new Dashboard to illustrate different Decoding settings. No other setting are changed.
I have 6 servers with 8-12 cameras of different type, FPS and resolution on each, and I'm able to reproduce this behavior on any of them.
segment 1: Ondemand
segment 2: KeyFrames + Ondemand
segment 3: KeyFrames Only
more meaningful than thousand words. CPU is Red, Memory is Yellow
Debug Log:
tail | grep 'Last Viewed'

Code: Select all

05/11/22 20:16:40.869666 zmc_m4[437322].DB1-zm_monitor.h/636 [Last viewed 1652282200 seconds ago]
05/11/22 20:16:40.879522 zmc_m4[437322].DB1-zm_monitor.h/636 [Last viewed 1652282200 seconds ago]
05/11/22 20:16:40.930528 zmc_m4[437322].DB1-zm_monitor.h/636 [Last viewed 1652282200 seconds ago]
05/11/22 20:16:40.930567 zmc_m4[437322].DB1-zm_monitor.h/636 [Last viewed 1652282200 seconds ago]
05/11/22 20:16:40.949591 zmc_m4[437322].DB1-zm_monitor.h/636 [Last viewed 1652282200 seconds ago]
tail | grep -i 'no Viewers' - nothing

Re: Strange Memory consuption pattern Decode Ondemand vs KeyFrames

Posted: Wed May 11, 2022 4:08 pm
by MarsRover
Playing around with decoding for some more time, this is what I've found:
Thumbnails Enabled
Ondemand eats more memory than KeyFrames Only.
KeyFrames + Ondemand eats even more, because (my guess) decoded keyframes aren't used for thumbnail generation and zmc does the same as with Decoding='Ondemand' for thumbnails.

Thumbnails Disabled
See no point in KeyFrames + Ondemand if Analysis and thumbnails are off.

Debug=3 log
https://drive.google.com/file/d/18671l2 ... sp=sharing