CPU load/performance problem on new install Ubuntu 22.04 + ZM 1.36.12

[bloomer]

Following a catastrophic server upgrade failure, I've had to start afresh with my home server. I have installed Ubuntu 22.04 (server) on my server (AMD Turion(tm) II Neo N40L Dual-Core Processor, 2 cores, 1.5GHz, 1.8G RAM), and then installed ZM 1.36.12. (I was previously running 18.04 LTS and 1.32.)

I have one 1980x1080 camera, and left ZM settings untouched, meaning it defaulted to Mocord, ffmpeg, with a full-frame zone for analysis.

Pretty much as soon as it was all up and running, ZM was rendering the server inaccessible via remote access (Webmin) and inoperable from local CLI (slow or unresponsive to keystrokes). I could hear the drive chugging away constantly too. A hard reboot only brought a few minutes' joy.

I changed analysis FPS to 5, changed to Modect, and created a single much smaller test zone with best-high-sensitivity. Based on a comment on Github, I disabled decoding in other options (Nodect, etc).

For most of the afternoon ZM was then using about 50% of CPU, based on Webmin's Running Processes view, but it climbed slowly and this morning it was at 92%, and I can hear more disk activity. The disk has heaps of space. Back home this evening and it is the same. ZM's own load indicator varies between about 0.9 and 1.5. Viewing an alarm takes it up to 2.5.

I just restarted the server and the output from top is:

Code: Select all

USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
www-data  20   0  704732 316516  51628 S  57.9  16.7   5:26.08 zmc  

I've been using ZM for many years on my server without problems with the same hardware and five analysis zones, but am only a lightweight Linux user when it comes to troubleshooting this sort of stuff. I hope I've provided useful info above. Would appreciate your suggestions.

[iconnor]

In the past we didn't care about keyframes. Now we do. We have to queue everything since the last keyframe (if using passthrough) because an mp4 has to start with a full frame.

I suspect your camera has a large keyframe interval. So set a MaxImageBuffer setting to something reasonable like 2*keyframe interval. Go into your camera see what it's set to and set it more appropriately.

Upgrade to latest 1.36.25. I have fixed a lot of things.

If you want even more options for reducing ram/cpu use, upgrade to latest master.

[bloomer]

Thank you for this! As a first step, I upgraded to 1.36.25 as suggested. I have now lost the camera stream in ZM, although it is still accessible using the same address in VLC (rtsp://[user]:[pass]@[...]:554/11).

Code: Select all

9/9/22, 7:17:37 PM GMT+10	zmdc		3427	INF	'zmc -m 1' crashed, signal 134	zmdc.pl	-
9/9/22, 7:17:36 PM GMT+10	zmdc		6228	INF	'zmc -m 1' started at 22/09/09 19:17:36	zmdc.pl	-
9/9/22, 7:17:36 PM GMT+10	zmdc		3427	INF	'zmc -m 1' starting at 22/09/09 19:17:36, pid = 6228	zmdc.pl	-
9/9/22, 7:17:36 PM GMT+10	zmdc		3427	INF	Command 'zmc -m 1' removed from pending list at 22/09/09 19:17:36	zmdc.pl	-
9/9/22, 7:17:36 PM GMT+10	zmwatch		3475	INF	Restarting capture daemon for Monitor-1, shared data not valid	zmwatch.pl	-
9/9/22, 7:17:36 PM GMT+10	zmwatch		3475	ERR	Memory map file '/dev/shm/zm.mmap.1' does not exist in zmMemAttach. zmc might not be running.	zmwatch.pl	-

[bloomer]

Ok, so I've tried moving to the master version with no luck. The video stream doesn't work. I also wanted to try downgrading to the last working version (1.36.12) but it seems that package no longer exists.

'zmc -m 1' crashed, signal 134 zmdc.pl seems to be the key error since the upgrade to 1.36.25 and master (the other errs appear to have to do with event alarms but no stream viewtopic.php?t=32038 ).

The stream is, as mentioned before, viewable in VLC. Nothing is viewable in ZM. I've tried alternative ways to find a working stream in ZM by using the various ONVIF discoveries, but it makes no difference.

[Magic919]

If you’ve come from such an old version originally, it would be worth checking your monitor settings in ZM. Make sure you are using ffmpeg source, for example.

Regarding 1.36.12 vs later, I’ve seen many upgrade and it’s not normal to lose the ability to capture video.

[bloomer]

Thanks for the suggestion. Yep, confirmed ffmpeg setting in the monitor settings (and TCP, correct image size, etc).

I forgot to include in my last post that the debug log shows Restarting capture daemon shared data not valid

[Magic919]

If it can’t grab the stream, you’ll get all sorts of errors. I wouldn’t focus on those.

Are you running VLC and testing on the ZM machine?

[bloomer]

Are you running VLC and testing on the ZM machine?

That is a very good point. Sigh. I was using Webmin to control the server remotely, and of course using VLC on my non-server. So it seems that the upgrade to 1.35.25 stuffed ffmpeg or something related to it, and that didn't get repaired in the uninstall/master/reinstall processes I tried. I just did another complete uninstall/reinstall of x.25 and now I have a viewable stream, thank you!

[Magic919]

I’m glad it’s worked out.

I’m staying off Ubuntu 22.04 at moment, whilst some things get ironed out. Things such as php-intl, date formats and some other bits might have gone awry during your journey.

There’s quite a bit of info that comes up in the terminal during the installs, that can give clues.

[bbunge]

For what it's worth, I've been trying Ubuntu again and have tested 22.04 with MySQL and Mariadb. I install the minimal Ubuntu server the add nano, ntpdate and Webmin 2.0. I also removed the cloud init stuff.
I have found that MySQL uses more resources (almost double the processor load) with my three cams running at 720P/10 FPS with a key frame interval of 10. I also had problems with MySQL setting the root password.
My server is an i3 quad core with 8 GB RAM and a 1 TB WD Green drive. Have also tested with a WD 1 TB Blue drive and it seems to perform as well as the Green.
I had been running Debian 11 "server" install with the same ZM config and cams and that ran really well. In time I will likely go back to Debian 11 with a 500 GB SSD. That will be on a Dell Optiplex 9020 mini that has an i3 and 8 GB RAM. I had far fewer issues with Debian 11 than Ubuntu 22.04. Of course there is Ubuntu 20.04...