Zoneminder 1.36 not functional

[ocascante]

Hello,
This is the infrastructure where I have version 1.36 of zoneminder installed:

DELL PowerEdge 7120
32 Gb Ram
4 Tb Hard Disk
4 Cores
8 Mb L3 cache
Ubuntu Server 20.04 Lts
8 cameras of the brand Hikvision IP wired utp
( all 8 cameras 640x480 resolution, 15 fps, function "record", alarm frame count 100 )

This server is dedicated to zoneminder.

Problem:
Zoneminder is consuming 51% of cpu's, 29 Gb Ram memory and mp4 video recording files have cuts in time and frames.

does anyone know how to solve these problems?
Thanks

[bbunge]

What instructions did you use to install Zoneminder?
You mention Ubuntu Server but are you running a desktop? Are you running Zoneminder in a virtual machine? Have you removed/disabled the cloud services in Ubuntu (shouldn't matter but when I use Ubuntu I install a minimal version from a net bood mini iso)?
MySQL or Mariadb?
On the surface it looks like you shouldn't be having those issues. I ran an old Dell i7 on 20.04 with 19 cams and it barely tasked the processor.

[ocascante]

Hello,
As indicated, it is ubuntu server so it does not have a desktop, it is not virtual, and it is dedicated to zoneminder so it does not have cloud service activated, and I use mariadb as a database.

[ocascante]

Update:
Only changing function "record" to "monitor" in the 8 cameras, memory usage changes to:
Total: 32 Gb, Used:1199
apparently the problem is in the permanent recording.

[bbunge]

Hello,
As indicated, it is ubuntu server so it does not have a desktop, it is not virtual, and it is dedicated to zoneminder so it does not have cloud service activated, and I use mariadb as a database.

OK, but what instructions did you use to install Zoneminder. I ask this because some of the install instructions are not complete. If you installed Ubuntu Server it does have cloud services running. Debian is better because it does not automatically add cloud services and other features that are not needed.

[parky]

As many other posts say, "do you really need 15fps ?" Knock it down to 6 fps which should allow you to run them at 720P

Your system is similar to mine which is a 2010 HP Z400 Quad with 16GB. It also runs 8 cameras at 720P with passthrough and ticks over with a load of 2. The load has run the same when it was on Record and now over to Mocord. RAM usage is usually down to buffer settings which need to be set lower in 1.36.x

My system only consumes about 4GB which allows the tin to run KVM with other test versions of ZM to try things out

[ocascante]

Hello,
I have been a Linux consultant for 36 years, and have installed Ubuntu Server many times, disabling cloud-init. The installation is correct following the procedures mentioned in the Zoneminder wiki, I don't think it is necessary to repeat the steps since zoneminder is running correctly. As I indicated, obviously the problem is in the record mode. On the other hand, if a person needs to put 10, 15 or 20 fps the software should respond correctly, I think.
In fact I have installed zoneminder many times, but this is the first time that I need to activate the record mode permanently.
I have observed that I am not the only one who has reported this problem.

[ocascante]

Update:
I have found many posts on the internet about the memory consumption of ffmpeg in different software, and I think that there is the problem.
ej: Ffmpeg eats all the memory and crash within a minute - recording or streaming

I will review more about this problem to know if it is the reason for the problem

[SteveGilvarry]

Try on the buffer tab Maximum Image Buffer Size (frames) (?) if set to 0 I have seen it going out of control on Ram usage. Play around with values that don't start giving warnings about dropping frames. Also some posts about it already on here that may help to know what values to try (2xIFrame Int + 1) etc

[ocascante]

Hello,
I try with a lot of different parameters in maximun buffer size option without any positive result.
Continue without having an answer to this problem!

[lbm]

Post some more info, like logs, and/or debug logs.
Sounds defiantly like something is crashing. I have had similar issues where the recording had holes in the recordings, and it was because of already mentioned misconfigured buffers.

[SteveGilvarry]

ej: Ffmpeg eats all the memory and crash within a minute - recording or streaming

Do you mean zmc? ffmpeg would not be a running process.

As mentioned logs and then probably debug logs, if you need a hand how to do them let me know

[bbunge]

Hello,
I have been a Linux consultant for 36 years, and have installed Ubuntu Server many times, disabling cloud-init. The installation is correct following the procedures mentioned in the Zoneminder wiki, I don't think it is necessary to repeat the steps since zoneminder is running correctly. As I indicated, obviously the problem is in the record mode. On the other hand, if a person needs to put 10, 15 or 20 fps the software should respond correctly, I think.
In fact I have installed zoneminder many times, but this is the first time that I need to activate the record mode permanently.
I have observed that I am not the only one who has reported this problem.

I can not match the years of consulting for Linux and am rather pleased that I'm self taught with linux. The skills I have brought to the program were gained in years of doing installs and writing procedures for a rather large government operation. Many of the WIKI install procedures for Ubuntu and Debian are my fault. I ask about the procedure you used to check up on my product. If something fails I want to know so I can research and fix it. You know...continuous improvement!
As Zoneminder has a lot of settings variables it is sometimes a challenge to get it working the way some think it should work. Knowledge of a system can lead one to skip a step they think is unimportant or do something another way. I likely have run the Zoneminder install on Ubuntu and Debian many more times than you. I do not ask questions lightly.

[kitkat]

I have found many posts on the internet about the memory consumption of ffmpeg in different software, and I think that there is the problem.
ej: Ffmpeg eats all the memory and crash within a minute - recording or streaming

Have you tried setting the Video Writer option on each camera's Storage tab to Camera Passthrough?

[dhmsjs]

I'll just add my situation and details here for the record.

I am seeing a very fragile ZM 1.36.5 on Ubuntu Server 20.04 64bit , on a RPi 3b (2G ram). Installation was a clean install of Ubuntu followed by ZM as per the wiki procedure https://wiki.zoneminder.com/Ubuntu_Serv ... der_1.36.x.

As a full disclosure note: I originally installed this system on my RPi 4 8G ram. Seemed to run okay but only tested lightly, and only with one camera attached. Then as an experiment swapped that SD card into the Pi3b. Seems to boot and run identically, but these issues might be related to that swap -- not likely but I cannot rule it out. I will retest on the Pi4 next week when I am at that location. One big difference between the two systems is 2G ram on the RPi3b vs 8G ram on the 4.

On the RPi3b with Focal and ZM 1.36.5, it seems stable with only one cam (H264 streaming, FFMPEG, 704x480 10 fps), modect). Memory use is low (1%) and CPU load is typically around 0.5 max.

As soon as I start modect on the second (identical) cam, the system becomes very fragile. As long as it is not doing anything (no motion, no user activity) it runs fine. But any activity tends to spike the CPU load way way up (to 8 often, even 25). Very quickly the system freezes on all interfaces including ssh. Power cycle seems to be the only way to recover.

I have limited Max Image Buffer Size to 2x+1 key frame interval (101 frames in this case) which seemed to help but not eliminate the crash. I have not done the math to make sure 101 is still not too large for 2 cams on these settings with 2G ram.

Logs are filled with the "Frame Buffer Full. Waiting" error so it is pretty clear that (as iconnor describes) the basic problem here is that the system can't process frames fast enough. What seems significant to me here is that any action that accesses the database seems to trigger the crash -- be it motion events or even just listing events or simply viewing the weblog.

Finally, a key observation is that if I run the RPi Debian (Buster) Lite OS, with ZM 1.36.0 on exactly the same RPi3 2G ram hardware and cams, it seems to run rock solid in full operation; motion detecting, storing events, viewing events, viewing the weblog etc.

ZM 1.36.5 on Focal is acting like any database access drags the system down in a fatal way, whereas similar database access is not a problem on Debian 10. Or it might be a difference in ZM 1.36.0 vs 1.36.5 (which waits on the buffer full condition instead of dropping frames). Or a combo of the two.

I am not posting this to ask for help in making this work for me. I'm just providing this information as more real-world data to help debug the issue.