Zoneminder keeps stopping

[kuehnem]

Ubuntu Server 64bit 12.04.1 LTS, Dell Power Edge Dual Xeon with 16GB of RAM. HD is separated into 200GB for / and 2TB for /var. I have 5 servers setup this way. Each is recording mjpeg video from 20 Sony snc-ds60 and snc-ds10 ip cams at 640x480 5-10fps depending on location.

This configuration runs fine 98% of the time. We only do record 24/7 mode on each monitor. Not fooling with motion detection on IP cams means my CPU utilization hovers around 11-12%, and it makes reviewing video fast.

My problem is a random bug that stops Zoneminder. Without warning at random times (about 5-12 days apart) I will log in to find Zoneminder is setting at state Stopped. The log only shows: "zmaudit 1515 FAT Can't execute: MySQL server has gone away" Changing the state to "Run" works fine and the box will run for another 5-12 days. I should note that I actually have a scheduled reboot every 72 hours. Plenty of RAM, plenty of HD space (only filtering after 90% full on /var). Having / and /var on separate partitions and drives seems to keep performance nice too.

Ok, so any ideas?

Secondly, what does everyone use for remote monitoring the state of their ZM boxes? I thought about just using a snmp app and a mysql monitor web app, but thought I'd ask the pros first.

Thanks.

[Flasheart]

The obvious question - and you've probably checked but don't mention - is has the mysql server gone away?

Are there any issues with the tables, does it crash and restart - what are its logs?

How do you restart zoneminder when it dies?

Not something I've found BTW, although I do recall many years ago having to add "zmpkg.pl restart" to cron once a day for a server that crashed at a remote site, but I can't remember why and I don't think it was sql related.

[PatMcLJr]

my rig is only 4 cams and a 750GB drive with 2GB of memory and 3Ghz Celeron
I had some trouble letting the drive fill up to 90% setting it to 75% seemed to help keep it up

I have had issues with MySQL and a table corruption. There is a fix in here that worked once but another time I had to reinstall the whole system, mostly because I don't know what I'm doing.

From my POV, ZM 1.24.4 was much more stable then 1.25

I have also had trouble with ubuntu and its grub2 so I used puppy and put grub legacy(?) in the mbr to boot the system, I tried to use lilo but I could get it to install properly.

[kuehnem]

The obvious question - and you've probably checked but don't mention - is has the mysql server gone away?

Are there any issues with the tables, does it crash and restart - what are its logs?

How do you restart zoneminder when it dies?

Not something I've found BTW, although I do recall many years ago having to add "zmpkg.pl restart" to cron once a day for a server that crashed at a remote site, but I can't remember why and I don't think it was sql related.

Yes, SQL is still up. The second place I checked is mysql.err too. Nothing. I did have some JPEG related errors in the ZM logs, so I switched from simple on my cams to the second (regexp or something). I'm going to watch it closely for a week or so and see what happens. I have restarted it several ways, all with the same outcome. I can just click on the word "stopped" at the top and set it to "run" and apply, or I can reboot the server from the command line. Neither seems to have an affect on future viability.

As far as filling up the HD goes, I'm running the filter at 88% of 1TB. Keep in mind that / is on it's own volume and /var is on it's own. This really has helped performance. 20 cams, 640x480, 5fps, 24/7 and I can get about 10 days. If I upgrade to 2TB this summer I can expect to double that. And the reviewing is fast.

My goal is to make these boxes as reliable without user intervention as possible. I hate going back to review an incident and and logging in only to find that ZM stopped 6 hours ago.