ZoneMinder going defunct?

hp1 · Post by **hp1** » Thu Feb 23, 2006 5:34 am

Since 1.21.4 (and now also in 1.22.0) the zm processes seem to go defunct after a few hours. I have no idea why, sometimes zm runs for days, other times only a couple of hours.

This is what a process listing looks like:

apache 9016 1 0 02:00 ? 00:00:00 /usr/bin/perl -wT /usr/lib/zm/bin/zmdc.pl startup
apache 9049 9016 3 02:00 ? 00:47:43 [zmc] <defunct>
apache 9055 9016 1 02:00 ? 00:19:51 /usr/lib/zm/bin/zma -m 7
apache 9062 9016 0 02:00 ? 00:00:01 [zmfilter.pl] <defunct>
apache 9071 9016 0 02:00 ? 00:00:01 [zmaudit.pl] <defunct>
apache 1674 22771 0 22:23 ? 00:00:00 /usr/bin/perl -wT /usr/lib/zm/bin/zmdc.pl status

In looking at the logs, only zmaudit.log and zmdc.log have recent timestamps (at what I assume is the time of the problem).

zmaudit.log snippet

...
02/22/2006 19:30:53.063151 zmaudit[9071].DBG [Found filesystem monitor '7']
02/22/2006 19:30:53.065462 zmaudit[9071].DBG [Got 274 events]
02/22/2006 19:45:53.129540 zmaudit[9071].DBG [Found database monitor '7']
02/22/2006 19:45:53.142507 zmaudit[9071].DBG [Got 274 events]
02/22/2006 19:45:53.142838 zmaudit[9071].DBG [Found filesystem monitor '7']
02/22/2006 19:45:53.146040 zmaudit[9071].DBG [Got 274 events]
02/22/2006 20:00:53.194189 zmaudit[9071].DBG [Found database monitor '7']
02/22/2006 20:00:53.210510 zmaudit[9071].DBG [Got 275 events]
02/22/2006 20:00:53.210786 zmaudit[9071].DBG [Found filesystem monitor '7']
02/22/2006 20:00:53.212915 zmaudit[9071].DBG [Got 275 events]

zmdc.log snippet (the reason for the additional restarts is that I've now cron'd a job at 2:00am to restart zm)

02/22/2006 02:00:46.750062 zmdc[9016].INF ['zma -m 7' starting at 06/02/22 02:00:46, pid = 9055]
02/22/2006 02:00:46.750888 zmdc[9055].INF ['zma -m 7' started at 06/02/22 02:00:46]
02/22/2006 02:00:47.000471 zmdc[9062].INF ['zmfilter.pl' started at 06/02/22 02:00:47]
02/22/2006 02:00:47.000470 zmdc[9016].INF ['zmfilter.pl' starting at 06/02/22 02:00:46, pid = 9062]
02/22/2006 02:00:47.408185 zmdc[9071].INF ['zmaudit.pl -c' started at 06/02/22 02:00:47]
02/22/2006 02:00:47.410532 zmdc[9016].INF ['zmaudit.pl -c' starting at 06/02/22 02:00:47, pid = 9071]
02/22/2006 02:00:47.788195 zmdc[9076].INF ['zmwatch.pl' started at 06/02/22 02:00:47]
02/22/2006 02:00:47.789870 zmdc[9016].INF ['zmwatch.pl' starting at 06/02/22 02:00:47, pid = 9076]
02/22/2006 20:06:45.280142 zmdc[9016].INF ['zmwatch.pl ' died, signal 14]
02/22/2006 20:06:45.289374 zmdc[9016].INF ['zmc -m 7' stopping at 06/02/22 20:06:45]

I only have the one camera

Any suggestions or ideas?

Post by **zoneminder** » Fri Feb 24, 2006 11:13 pm

The zmdc.pl process should catch the deaths of it's child processes. If they are goign defunct it implies this is not happening. Do you have anything in older versions of zmdc.log?

hp1 · Post by **hp1** » Sun Feb 26, 2006 5:19 am

Thanks for the response. I don't know quite what you're asking, what types of things should I be looking for?

thanks for your help.

yg1976 · Post by **yg1976** » Mon Feb 27, 2006 9:49 am

I am having a very similar problem. After a day or two the computer it is running on seems very slow and preoccupied. I have to reboot the pc.

Also I have noticed that occasionally ZM goes a little crazy (none technical term) and records a few events with thousands of images. The largest I have had is 297124 captured images for 1 event at around 3am in the morning. Nothing was happening so I dont understand why it started recording.
My 40gb hd was filled up withing 4 days.

I am running version 1.22.0 on a pentium 2 which is running Ubuntu 5.10 with 2 Logitech quickcams on a USB2 pci card.

hp1 · Post by **hp1** » Mon Apr 17, 2006 4:33 pm

I did a fresh FC5 install, and compiled/installed zoneminder from source and I still get processes going defunct. It takes about 1.5 days now (from 1 day before), but I'm getting the similar behaviour.

I am, however, using the same zm database from my previous installation. As there are no events that I wish to keep, I could reinit the database if it would help solve the problem. Is there any way to export the monitor settings as a reference to recreate them?

Any assistance is greatly appreciated as I really want this to stablize.

thanks in advance!!!

Post by **zoneminder** » Mon Apr 17, 2006 7:31 pm

A defunct process is one who's death has not been noticed by it's parent. The zmdc.pl script _should_ always notice when a child dies unless there is something odd going on. The only way we might track this down is if you can find entries in zmdc.log from around when the processes turn into zombies.

hp1 · Post by **hp1** » Mon Apr 17, 2006 10:35 pm

Thanks for the reply. It looks a little different than last time, as I've added an additional monitor, and the zma process for the second monitor has also gone defunct. Not much in zmdc.log.

I have cron'd a job that runs every five minutes and runs wget against zm. If it dies, it will copy the logs, do a process listing, etc.. I'm also running debug for zma_m8, so hopefully this will shed some light. Any other suggestions on how I can track this down?

thanks again

--

[root@hx1a /]# ps -ef |grep zm
apache 22666 1 0 Apr16 ? 00:00:07 /usr/bin/perl -wT /usr/local/bin/zmdc.pl startup
apache 22701 22666 10 Apr16 ? 01:46:29 /usr/local/bin/zmc -m 8
apache 22709 22666 0 Apr16 ? 00:00:00 [zmfilter.pl] <defunct>
apache 22717 22666 0 Apr16 ? 00:00:00 [zmaudit.pl] <defunct>
apache 23352 22666 9 Apr16 ? 01:40:25 [zma] <defunct>
apache 29672 22666 5 Apr16 ? 00:56:45 /usr/local/bin/zmc -m 7
apache 29674 22666 0 Apr16 ? 00:03:25 /usr/local/bin/zma -m 7
apache 21052 20426 1 10:05 ? 00:00:00 /usr/bin/perl -wT /usr/local/bin/zmdc.pl status

[root@hx1a] cat /var/log/zm/* |grep \ 01:05
zmdc.log:04/17/06 01:05:10.704873 zmdc[22666].INF ['zmwatch.pl ' died, signal 14]
zmdc.log:04/17/06 01:05:10.720385 zmdc[22666].INF ['zmfilter.pl ' stopping at 06/04/17 01:05:10]

Post by **zoneminder** » Tue Apr 18, 2006 3:21 pm

This definately looks strange, even perl scripts are going doolally. You don't have anythign that might be sending odd signals to these processes do you? The snippet of log appears to show normal behaviour.

Can you post a bit more information about your system and distro etc, and also what version of perl you have installed?

hp1 · Post by **hp1** » Wed Apr 19, 2006 4:20 am

I don't think I have anything odd being sent to the scripts -- I wouldn't know how to send something even if I wanted!

at 8:15 this morrning only zmfilter.pl went defunct. In reviewing the debug logs, I got two interesting events:

04/18/06 06:49:56.718953 zma_m8[16898].ERR-zm_event.cpp/58 [Can't insert event: MySQL server has gone away]

04/18/06 08:15:42.369750 zma_m8[9726].WAR-zm_monitor.cpp/737 [Approaching buffer overrun, consider slowing capture, simplyfying analysis or increasing ring buffer size]

the MySql is curious as I also use the same server for mythtv and didn't see any problems, nor was anything logged my mysql.

zoneminder also became unresponsive at 18:55, but didn't see any defunct processes.

Distro: Fedora Core 5 (running 2.6.16-1.2080_FC5smp kernel)
Machine: Pentium 4 2.3 w/ HyperThreading. 1.5 GB Ram.
Using it for mythtv (2 capture cards), file server, misterhouse and asterisk (1000xp card)
Perl: 5.8.8

hp1 · Post by **hp1** » Fri Jun 09, 2006 2:49 am

Sorry for resurrecting a dead thread, but to put closure to my issue: After fully uninstalling the rpms and re-creating the databases, my processes no longer go defunct.

Post by **cordel** » Fri Jun 09, 2006 5:43 am

Just to better close this thread, can you elaborate more on what rpms you removed and why?

Regards,
Corey

hp1 · Post by **hp1** » Sat Jun 10, 2006 2:20 am

Sorry, it was late and I was mistaken. I did a make uninstall, and then dropped the databases. (The databases were originally installed through a manual build on FC2, then rpm upgrades in FC3, and I used the same db with a fresh zoneminder install in FC5).

I did this, because the database was the only part of my system that was common from my freshly installed FC5 system. Didn't want to lose my old events, and it was a PITA to recreate the monitors, but I wanted to solve the defunct issue.

thanks for the clarifying question Cordel.