Random lockups of network camera after upgrade 1.28->1.29
Posted: Sun Sep 04, 2016 2:03 pm
Hello all,
I recently updated my box with Wheezy Debian to Jessie (stable) and upgraded a working zoneminder install from 1.28.0 -> 1.29.0 (debian backports)
This worked before but I had to upgrade due to EOL on Wheezy. Since I will be adding a new camera with audio, I wanted to go with later version for better features.
I followed the instructions from the wiki and got my existing setup working. However, I’m getting very regular hangs for from my network camera. I get the blue fullscreen no signal image on the camera after anywhere from 1 hour to 12 hours. My USB camera seems unaffected.
The lockup seems to happen more often when I am also viewing the stream via the browser. Although it does happen on it’s own. In all cases, I can connect to the camera directly or other software and see no problems with image. Restarting zoneminder resolves the problem but only for a few hours. Again, this setup worked before upgrading without these issues.
From what I read of other posters, they seem to run out of /dev/shm memory or similar. In my case that doesn’t appear to be the problem.
Thanks for any help.
My hardware:
Software:
Network camera config:
System Info at time of lockup:
Debug Logs:
I recently updated my box with Wheezy Debian to Jessie (stable) and upgraded a working zoneminder install from 1.28.0 -> 1.29.0 (debian backports)
This worked before but I had to upgrade due to EOL on Wheezy. Since I will be adding a new camera with audio, I wanted to go with later version for better features.
I followed the instructions from the wiki and got my existing setup working. However, I’m getting very regular hangs for from my network camera. I get the blue fullscreen no signal image on the camera after anywhere from 1 hour to 12 hours. My USB camera seems unaffected.
The lockup seems to happen more often when I am also viewing the stream via the browser. Although it does happen on it’s own. In all cases, I can connect to the camera directly or other software and see no problems with image. Restarting zoneminder resolves the problem but only for a few hours. Again, this setup worked before upgrading without these issues.
From what I read of other posters, they seem to run out of /dev/shm memory or similar. In my case that doesn’t appear to be the problem.
Thanks for any help.
My hardware:
Code: Select all
i5-4590 @ 3.3Ghz
6TB RAID-10 (mdm)
16GB RAM
Camera 1: usb 1280x720p 24Bit @ 5FPS Logitech HD
Camera 2: HOSAFE 2MB2W ONVIF 1920x1080 32Bit @ 12-24FPS (POE/Wired)Code: Select all
Debian Jessie 64bit
Zoneminder 1.29.0+dfsg-1~bpo8+
Code: Select all
ffmpeg, modect, 1920x1080 @ 32bit (camera ranges from 12FPS night to 25FPS day)
all fps limits left blank
rtsp://admin:admin@10.0.50.100/live0.264
ffmpeg buffer size: 200
warmup frames: 50
Pre event image count: 50
Post event image count: 100
Stream replay image buffer: 1000
Alarm frame count: 3Code: Select all
Filesystem Size Used Avail Use% Mounted on
/dev/dm-2 5.4T 4.8T 612G 89% /
udev 10M 0 10M 0% /dev
tmpfs 3.2G 313M 2.9G 10% /run
tmpfs 7.8G 1.9G 6.0G 24% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/sda3 233M 76M 145M 35% /boot
/dev/sda2 122M 120K 121M 1% /boot/efi
tmpfs 1.6G 0 1.6G 0% /run/user/1000
total used free shared buffers cached
Mem: 15956 14703 1252 2165 6548 4715
-/+ buffers/cache: 3439 12517
Swap: 15255 62 15193
PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command
26651 www-data 20 0 2282M 1661M 1597M S 2.4 10.4 1h29:34 /usr/bin/zmc -m 3
26634 www-data 20 0 659M 282M 275M S 1.9 1.8 11:50.65 /usr/bin/zmc -d /dev/webcam0
26687 www-data 20 0 655M 295M 275M S 1.9 1.9 11:46.00 /usr/bin/zma -m 1
26686 www-data 20 0 2011M 1652M 1593M S 0.0 10.4 1h04:34 /usr/bin/zma -m 3
26603 www-data 20 0 153M 15968 4788 S 0.0 0.1 0:13.30 /usr/bin/perl -wT /usr/bin/zmdc.pl startup
26681 www-data 20 0 2282M 1661M 1597M S 0.0 10.4 15:54.12 /usr/bin/zmc -m 3
26682 www-data 20 0 2282M 1661M 1597M S 0.0 10.4 10:54.86 /usr/bin/zmc -m 3
26683 www-data 20 0 2282M 1661M 1597M S 0.0 10.4 10:02.38 /usr/bin/zmc -m 3
26684 www-data 20 0 2282M 1661M 1597M S 0.0 10.4 9:59.55 /usr/bin/zmc -m 3
26685 www-data 20 0 2282M 1661M 1597M S 0.0 10.4 9:59.12 /usr/bin/zmc -m 3
26660 www-data 20 0 177M 28632 6784 S 0.0 0.2 0:00.70 /usr/bin/perl -wT /usr/bin/zmfilter.pl
26668 www-data 20 0 164M 27064 6452 S 0.0 0.2 0:08.48 /usr/bin/perl -wT /usr/bin/zmaudit.pl -c
26676 www-data 20 0 151M 16216 6664 S 0.0 0.1 0:02.24 /usr/bin/perl -wT /usr/bin/zmwatch.pl
Code: Select all
09/04/16 20:33:47.183944 web_js[27145].ERR [getStreamCmdResponse stream error: socket_bind( /var/run/zm/zms-869858w.sock ) failed: Address already in use - checkSt
reamForErrors()] at ?view=watch line
09/04/16 20:33:50.186261 web_php[27145].DBG [LogOpts: level=DBG/DBG, screen=OFF, database=WAR, logfile=DBG->/var/log/zm/zm_debug.log.27145, weblog=INF, syslog=INF] at /usr/share/zoneminder/www/includes/logger.php line 168
09/04/16 20:33:50.186358 web_php[27145].ERR [socket_sendto( /var/run/zm/zms-869858s.sock ) failed: No such file or directory] at /usr/share/zoneminder/www/includes/functions.php line 2371
09/04/16 20:33:50.227050 web_php[27145].DBG [LogOpts: level=DBG/DBG, screen=OFF, database=WAR, logfile=DBG->/var/log/zm/zm_debug.log.27145, weblog=INF, syslog=INF] at /usr/share/zoneminder/www/includes/logger.php line 168
09/04/16 20:33:50.227153 web_js[27145].DBG [LogOpts: level=DBG/DBG, screen=OFF, database=WAR, logfile=DBG->/var/log/zm/zm_debug.log.27145, weblog=OFF, syslog=ERR] at /usr/share/zoneminder/www/includes/logger.php line 168
09/04/16 20:33:50.227195 web_js[27145].ERR [getStreamCmdResponse stream error: socket_sendto( /var/run/zm/zms-869858s.sock ) failed: No such file or directory - checkStreamForErrors()] at ?view=watch line
09/04/16 20:33:53.228232 web_php[27145].DBG [LogOpts: level=DBG/DBG, screen=OFF, database=WAR, logfile=DBG->/var/log/zm/zm_debug.log.27145, weblog=INF, syslog=INF] at /usr/share/zoneminder/www/includes/logger.php line 168
09/04/16 20:33:53.228328 web_php[27145].ERR [socket_sendto( /var/run/zm/zms-869858s.sock ) failed: No such file or directory] at /usr/share/zoneminder/www/includes/functions.php line 2371
09/04/16 22:03:21.122717 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 634000 - Analysing at 12.50 fps]
09/04/16 22:04:41.108375 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 635000 - Analysing at 12.50 fps]
09/04/16 22:06:01.068296 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 636000 - Analysing at 12.50 fps]
09/04/16 22:07:21.000237 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 637000 - Analysing at 12.50 fps]
09/04/16 22:08:40.997983 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 638000 - Analysing at 12.66 fps]
09/04/16 22:10:00.930121 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 639000 - Analysing at 12.50 fps]
09/04/16 22:11:20.861992 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 640000 - Analysing at 12.50 fps]
09/04/16 22:12:40.854133 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 641000 - Analysing at 12.50 fps]
09/04/16 22:14:00.792846 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 642000 - Analysing at 12.50 fps]
09/04/16 22:15:20.757180 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 643000 - Analysing at 12.50 fps]
09/04/16 22:15:52.121952 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Lost]
09/04/16 22:16:02.221715 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Reacquired]
09/04/16 22:16:02.314231 zma_m3[26686].INF-zm_monitor.cpp/1680 [Driveway: 643626 - Gone into prealarm state]
09/04/16 22:16:02.315838 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Lost]
09/04/16 22:16:18.679437 zma_m3[26686].INF-zm_monitor.cpp/1287 [Driveway: 644000 - Analysing at 17.24 fps]
09/04/16 22:16:27.431679 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Reacquired]
09/04/16 22:16:27.456906 zma_m3[26686].INF-zm_monitor.cpp/1680 [Driveway: 644187 - Gone into prealarm state]
09/04/16 22:16:27.458570 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Lost]
09/04/16 22:16:30.491104 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Reacquired]
09/04/16 22:16:30.880958 zma_m3[26686].INF-zm_monitor.cpp/1680 [Driveway: 644253 - Gone into prealarm state]
09/04/16 22:16:31.068620 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Lost]
09/04/16 22:16:37.426716 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Reacquired]
09/04/16 22:16:37.541635 zma_m3[26686].INF-zm_monitor.cpp/1680 [Driveway: 644372 - Gone into prealarm state]
09/04/16 22:16:37.599039 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Lost]
09/04/16 22:16:42.176752 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Reacquired]
09/04/16 22:16:42.239750 zma_m3[26686].INF-zm_monitor.cpp/1680 [Driveway: 644489 - Gone into prealarm state]
09/04/16 22:16:42.242449 zma_m3[26686].WAR-zm_monitor.cpp/1420 [Signal: Lost]
09/04/16 23:57:58.166000 zms[746].DB1-zm_event.cpp/1088 [Got QUERY command, sending STATUS]
09/04/16 23:57:59.268307 zms[746].DB1-zm_event.cpp/1088 [Got QUERY command, sending STATUS]
09/04/16 23:58:00.370700 zms[746].DB1-zm_event.cpp/1088 [Got QUERY command, sending STATUS]
09/04/16 23:58:09.590310 zms[746].INF-zm_signal.cpp/40 [Got signal 15 (Terminated), exiting]
09/04/16 23:58:09.590339 zms[746].ERR-zm_event.cpp/1329 [Unable to send raw frame 154: Broken pipe]
09/04/16 23:58:09.782833 zms[746].DB1-zm_logger.cpp/239 [Terminating Logger]
09/04/16 22:08:00.319116 zmc_m3[26651].INF-zm_monitor.cpp/3131 [Driveway: 638000 - Capturing at 12.50 fps]
09/04/16 22:09:20.271517 zmc_m3[26651].INF-zm_monitor.cpp/3131 [Driveway: 639000 - Capturing at 12.50 fps]
09/04/16 22:10:40.268201 zmc_m3[26651].INF-zm_monitor.cpp/3131 [Driveway: 640000 - Capturing at 12.50 fps]
09/04/16 22:12:00.227458 zmc_m3[26651].INF-zm_monitor.cpp/3131 [Driveway: 641000 - Capturing at 12.50 fps]
09/04/16 22:13:20.156581 zmc_m3[26651].INF-zm_monitor.cpp/3131 [Driveway: 642000 - Capturing at 12.50 fps]
09/04/16 22:14:40.137003 zmc_m3[26651].INF-zm_monitor.cpp/3131 [Driveway: 643000 - Capturing at 12.50 fps]
09/04/16 22:15:52.025777 zmc_m3[26651].ERR-zm_ffmpeg_camera.cpp/167 [Unable to read packet from stream 0: error -541478725 "End of file".]
09/04/16 22:15:52.093131 zmc_m3[26651].ERR-zm_ffmpeg_camera.cpp/167 [Unable to read packet from stream 0: error -541478725 "End of file".]
09/04/16 22:15:52.172840 zmc_m3[26651].ERR-zm_ffmpeg_camera.cpp/167 [Unable to read packet from stream 0: error -541478725 "End of file".]
09/04/16 22:15:52.264833 zmc_m3[26651].ERR-zm_ffmpeg_camera.cpp/167 [Unable to read packet from stream 0: error -541478725 "End of file".]
09/04/16 22:15:52.310446 zmc_m3[26651].ERR-zm_ffmpeg_camera.cpp/167 [Unable to read packet from stream 0: error -541478725 "End of file".]
09/04/16 20:35:19.034448 zma_m3[26686].DB1-zm_event.cpp/560 [Writing capture frame 87]
09/04/16 20:35:19.046091 zma_m3[26686].DB1-zm_event.cpp/574 [Adding frame 87 of type "Alarm" to DB]
09/04/16 20:35:19.059618 zma_m3[26686].DB1-zm_event.cpp/611 [Writing analysis frame 87]
09/04/16 20:35:19.092753 zma_m3[26686].DB1-zm_zone.cpp/816 [All: Pixel Diff: 36, Alarm Pixels: 123244, Filter Pixels: 95403, Blob Pixels: 86216, Blobs: 1, Score: 5
]
09/04/16 20:35:19.134924 zma_m3[26686].DB1-zm_event.cpp/560 [Writing capture frame 88]
09/04/16 20:35:19.147969 zma_m3[26686].DB1-zm_event.cpp/574 [Adding frame 88 of type "Alarm" to DB]
09/04/16 20:35:19.168385 zma_m3[26686].DB1-zm_event.cpp/611 [Writing analysis frame 88]
09/04/16 20:35:19.195772 zma_m3[26686].INF-zm_monitor.cpp/1695 [Driveway: 567921 - Gone into alert state]
09/04/16 20:35:19.195785 zma_m3[26686].DB1-zm_event.cpp/560 [Writing capture frame 89]
09/04/16 20:35:19.206868 zma_m3[26686].DB1-zm_event.cpp/574 [Adding frame 89 of type "Normal" to DB]
09/04/16 20:35:19.242879 zma_m3[26686].DB1-zm_event.cpp/560 [Writing capture frame 90]
09/04/16 20:35:19.253950 zma_m3[26686].DB1-zm_event.cpp/574 [Adding frame 90 of type "Normal" to DB]
09/04/16 20:35:19.279574 zma_m3[26686].DB1-zm_event.cpp/560 [Writing capture frame 91]
09/04/16 20:35:19.290832 zma_m3[26686].DB1-zm_event.cpp/574 [Adding frame 91 of type "Normal" to DB]