Zoneminder stops responding on LAN after viewing

[Beofres]

I just compiled and installed ZM 1.23.3 onto Debian Etch with ffmpeg streaming and video generation working perfectly. I'm not using MPEG streaming, but instead Cambozola / MooTools. Everything appears to work fine, but as soon as I start streaming one or more cameras...can't say how long it takes specifically, I start getting errors in my logs (see below) and then streaming/SSH/web interface stops responding. I have a dedicated machine in our store hooked to a television screen displaying two cameras through Viseon Viewer and a manager that views the cameras through that application on his desktop as well. Console works fine and I can issue a restart command and everything is cherry afterwards, till I start streaming again. This is what I'm getting in my syslog:

Code: Select all

Feb  2 08:54:21 zoneminder zma_m10[3042]: ERR [Incomplete frame write: 80960 of 93883 bytes written]
Feb  2 08:54:21 zoneminder zmf_m4[3021]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:21 zoneminder zmf_m10[3039]: ERR [Incomplete read of frame image data, 80940 bytes only]
Feb  2 08:54:21 zoneminder zmf_m3[3014]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:21 zoneminder zmf_m3[3014]: ERR [Incomplete read of frame image data, 80940 bytes only]
Feb  2 08:54:21 zoneminder zma_m3[3016]: ERR [Can't write frame: Broken pipe]
Feb  2 08:54:21 zoneminder zmf_m10[3039]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:21 zoneminder zmf_m3[3014]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:23 zoneminder zmf_m11[3048]: ERR [Incomplete read of frame image data, 32364 bytes only]
Feb  2 08:54:23 zoneminder zma_m11[3051]: ERR [Can't write frame: Broken pipe]
Feb  2 08:54:23 zoneminder zmf_m11[3048]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:23 zoneminder zmf_m11[3048]: ERR [Incomplete read of frame image data, 32364 bytes only]
Feb  2 08:54:23 zoneminder zma_m11[3051]: ERR [Can't write frame: Broken pipe]
Feb  2 08:54:23 zoneminder zmf_m11[3048]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:26 zoneminder zma_m11[3051]: ERR [Incomplete frame write: 32384 of 57368 bytes written]
Feb  2 08:54:26 zoneminder zma_m3[3016]: ERR [Incomplete frame write: 64768 of 101714 bytes written]
Feb  2 08:54:26 zoneminder zma_m10[3042]: ERR [Incomplete frame write: 80960 of 94282 bytes written]
Feb  2 08:54:26 zoneminder zma_m4[3024]: ERR [Incomplete frame write: 80960 of 95040 bytes written]
Feb  2 08:54:26 zoneminder zma_m16[3069]: ERR [Incomplete frame write: 16192 of 81156 bytes written]
Feb  2 08:54:26 zoneminder zma_m13[3060]: ERR [Incomplete frame write: 48576 of 72629 bytes written]
Feb  2 08:54:26 zoneminder zma_m7[3033]: ERR [Incomplete frame write: 16192 of 83096 bytes written]
Feb  2 08:54:26 zoneminder zmf_m16[3066]: ERR [Incomplete read of frame image data, 16172 bytes only]
Feb  2 08:54:26 zoneminder zmf_m11[3048]: ERR [Incomplete read of frame image data, 32364 bytes only]
Feb  2 08:54:26 zoneminder zmf_m13[3057]: ERR [Incomplete read of frame image data, 48556 bytes only]
Feb  2 08:54:26 zoneminder zmf_m3[3014]: ERR [Incomplete read of frame image data, 64748 bytes only]
Feb  2 08:54:26 zoneminder zmf_m7[3030]: ERR [Incomplete read of frame image data, 16172 bytes only]
Feb  2 08:54:26 zoneminder zmf_m10[3039]: ERR [Incomplete read of frame image data, 80940 bytes only]
Feb  2 08:54:26 zoneminder zmf_m13[3057]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:26 zoneminder zmf_m3[3014]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:26 zoneminder zmf_m16[3066]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:26 zoneminder zmf_m4[3021]: ERR [Incomplete read of frame image data, 80940 bytes only]
Feb  2 08:54:26 zoneminder zmf_m11[3048]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:26 zoneminder zmf_m10[3039]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:26 zoneminder zmf_m4[3021]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:26 zoneminder zmf_m7[3030]: INF [Frame server socket open, awaiting images]
Feb  2 08:54:27 zoneminder zmf_m11[3048]: ERR [Incomplete read of frame image data, 16172 bytes only]
Feb  2 08:54:27 zoneminder zma_m11[3051]: ERR [Can't write frame: Broken pipe]
Feb  2 08:54:27 zoneminder zmf_m11[3048]: INF [Frame server socket open, awaiting images]

I have my kernel.shmmax and shmall set to 536674304 (my system is a core2 with 2GB of RAM)

I also have:
ZM_FRAME_SOCKET_SIZE = 1310710
ZM_SHM_KEY = 0x7a6d0000

Edit: I'm still getting the error messages despite the fact that noone is viewing the streams.

Thanks in advance...

[nuck]

I have my kernel.shmmax and shmall set to 536674304 (my system is a core2 with 2GB of RAM)

Don't know if it would make a bit of difference but try using 536870912. I'm running on a 2 gig ram machine with 5 cameras on a bt878 card.

[Paranoid]

I used to have a similar problem, cant remember what the log entries looked like though.

Turns out that mine was due to a problem with the firefox browser I was using.
It seems that firefox NEVER closed any of the open image streams so the number of streams kept on building up until everything hung.
You can find out if its the same problem by switching firefox to work offline (File->WorkOffline) and then switch it back online. If its caused by the browser keeping the streams open then everything should be working without any restarts.

The latest version of firefox fixes this problem.

[cordel]

I have seen this error recently some where else.
I'm starting to wonder if it has something to do with a particular distro.
Try disabling zmf in the system tab. The other system worked on would leave shared memory segments while zmf was in use so if a monitor thought it needed to restart, it would fall on it's face until you restarted the whole system or go in and manually remove the segments.

I just hope that this don't show up in 1.24.x

[phowarth]

I can confirm the same problem with Slackware 12.2 when I use Firefox 3.0.4 so I don't think it is limited to one distro.

I haven't confirmed the root cause, but if I use IE I don't get the issue.

What version of Firefox are you running that fixes the problem?

[Paranoid]

The Firefox problem was fixed in 3.0.5.

[cordel]

I can confirm the same problem with Slackware 12.2 when I use Firefox 3.0.4 so I don't think it is limited to one distro.

I haven't confirmed the root cause, but if I use IE I don't get the issue.

What version of Firefox are you running that fixes the problem?

Firefox <=3.0.4 has a serious bug, upgrade it or don't use it. It don't close media sockets until the browser is completely closed.

[Beofres]

Update time!

I was able to fix the issue in the first post I made (the incomplete images) by unchecking ZM_OPT_FRAME_SERVER. I was still having the timeout issues. I thought it might have had to have been something with the network card / chipset overheating, so I disabled the machine till I could find a new network card to put in there and fix the cooling issue. I replaced the card, got it configured etc, and watched the stream. It lasted for about 10 minutes streaming 4 cams (320x240 5fps, refresh rate in zm4ms @ 100ms; these are settings I used with my Ubuntu box that had previously worked fine on same hardware) I managed to get to the console and noticed some errors. I rebooted the machine, SSH'd in and found the following in the syslog at about the point where the stream locked up on me:

Code: Select all

Feb 17 00:43:25 zoneminder zms[1506]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:25 zoneminder zms[1507]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:25 zoneminder zms[1508]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:25 zoneminder zms[1510]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:25 zoneminder zms[1509]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:25 zoneminder zms[1511]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:25 zoneminder zms[1512]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:25 zoneminder zms[1513]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:25 zoneminder zms[1514]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:25 zoneminder zms[1515]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:26 zoneminder kernel: bttv3: timeout: drop=0 irq=237714/1045607, risc=33f0601c, bits: VSYNC HSYNC OFLOW RISCI
Feb 17 00:43:26 zoneminder zmc_dvideo3[2956]: ERR [Sync failure for frame 5 buffer 6(1): Input/output error]
Feb 17 00:43:26 zoneminder zmc_dvideo3[2956]: ERR [Failed to post-capture monitor 0]
Feb 17 00:43:26 zoneminder zmdc[2924]: ERR ['zmc -d /dev/video3' exited abnormally, exit status 255] 
Feb 17 00:43:26 zoneminder zmdc[2924]: INF [Starting pending process, zmc -d /dev/video3] 
Feb 17 00:43:26 zoneminder zmdc[2924]: INF ['zmc -d /dev/video3' starting at 09/02/17 00:43:26, pid = 1516] 
Feb 17 00:43:26 zoneminder zmdc[1516]: INF ['zmc -d /dev/video3' started at 09/02/17 00:43:26] 
Feb 17 00:43:26 zoneminder zmc_dvideo3[1516]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:26 zoneminder kernel: bttv3: reset, reinitialize
Feb 17 00:43:26 zoneminder zmc_dvideo3[1516]: INF [Starting Capture]
Feb 17 00:43:26 zoneminder kernel: bttv7: timeout: drop=0 irq=237673/1045603, risc=33e5501c, bits: VSYNC HSYNC RISCI
Feb 17 00:43:26 zoneminder zmc_dvideo7[2980]: ERR [Sync failure for frame 1 buffer 2(1): Input/output error]
Feb 17 00:43:26 zoneminder zmc_dvideo7[2980]: ERR [Failed to post-capture monitor 0]
Feb 17 00:43:26 zoneminder zmdc[2924]: ERR ['zmc -d /dev/video7' exited abnormally, exit status 255] 
Feb 17 00:43:26 zoneminder zmdc[2924]: INF [Starting pending process, zmc -d /dev/video7] 
Feb 17 00:43:26 zoneminder zmdc[1517]: INF ['zmc -d /dev/video7' started at 09/02/17 00:43:26] 
Feb 17 00:43:26 zoneminder zmdc[2924]: INF ['zmc -d /dev/video7' starting at 09/02/17 00:43:26, pid = 1517] 
Feb 17 00:43:26 zoneminder zmc_dvideo7[1517]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:26 zoneminder kernel: bttv7: reset, reinitialize
Feb 17 00:43:26 zoneminder zmc_dvideo7[1517]: INF [Starting Capture]
Feb 17 00:43:27 zoneminder kernel: bttv3: timeout: drop=0 irq=237714/1045607, risc=33f0601c, bits: VSYNC HSYNC OFLOW RISCI
Feb 17 00:43:27 zoneminder zmc_dvideo3[1516]: ERR [Sync failure for frame 0 buffer 1(1): Input/output error]
Feb 17 00:43:27 zoneminder zmc_dvideo3[1516]: ERR [Failed to post-capture monitor 0]
Feb 17 00:43:27 zoneminder zmdc[2924]: ERR ['zmc -d /dev/video3' exited abnormally, exit status 255] 
Feb 17 00:43:27 zoneminder kernel: bttv7: timeout: drop=0 irq=237673/1045603, risc=33e5501c, bits: VSYNC HSYNC RISCI
Feb 17 00:43:27 zoneminder zmc_dvideo7[1517]: ERR [Sync failure for frame 0 buffer 1(1): Input/output error]
Feb 17 00:43:27 zoneminder zmc_dvideo7[1517]: ERR [Failed to post-capture monitor 0]
Feb 17 00:43:27 zoneminder zmdc[2924]: ERR ['zmc -d /dev/video7' exited abnormally, exit status 255] 
Feb 17 00:43:32 zoneminder zmdc[2924]: INF [Starting pending process, zmc -d /dev/video3] 
Feb 17 00:43:32 zoneminder zmdc[1518]: INF ['zmc -d /dev/video3' started at 09/02/17 00:43:32] 
Feb 17 00:43:32 zoneminder zmdc[2924]: INF ['zmc -d /dev/video3' starting at 09/02/17 00:43:32, pid = 1518] 
Feb 17 00:43:32 zoneminder zmdc[2924]: INF [Starting pending process, zmc -d /dev/video7] 
Feb 17 00:43:32 zoneminder zmdc[1519]: INF ['zmc -d /dev/video7' started at 09/02/17 00:43:32] 
Feb 17 00:43:32 zoneminder zmdc[2924]: INF ['zmc -d /dev/video7' starting at 09/02/17 00:43:32, pid = 1519] 
Feb 17 00:43:32 zoneminder zmc_dvideo3[1518]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:32 zoneminder kernel: bttv3: reset, reinitialize
Feb 17 00:43:32 zoneminder zmc_dvideo3[1518]: INF [Starting Capture]
Feb 17 00:43:32 zoneminder zmc_dvideo7[1519]: INF [Debug Level = 0, Debug Log = <none>]
Feb 17 00:43:32 zoneminder kernel: bttv7: reset, reinitialize
Feb 17 00:43:32 zoneminder zmc_dvideo7[1519]: INF [Starting Capture]
Feb 17 00:43:32 zoneminder kernel: bttv3: timeout: drop=0 irq=237714/1045607, risc=33f0603c, bits: VSYNC HSYNC OFLOW RISCI
Feb 17 00:43:32 zoneminder zmc_dvideo3[1518]: ERR [Sync failure for frame 0 buffer 1(1): Input/output error]
Feb 17 00:43:32 zoneminder zmc_dvideo3[1518]: ERR [Failed to post-capture monitor 0]
Feb 17 00:43:32 zoneminder zmdc[2924]: ERR ['zmc -d /dev/video3' exited abnormally, exit status 255] 
Feb 17 00:43:32 zoneminder kernel: bttv7: timeout: drop=0 irq=237673/1045603, risc=33e5503c, bits: VSYNC HSYNC RISCI
Feb 17 00:43:32 zoneminder zmc_dvideo7[1519]: ERR [Sync failure for frame 0 buffer 1(1): Input/output error]
Feb 17 00:43:32 zoneminder zmc_dvideo7[1519]: ERR [Failed to post-capture monitor 0]
Feb 17 00:43:32 zoneminder zmdc[2924]: ERR ['zmc -d /dev/video7' exited abnormally, exit status 255] 
Feb 17 00:43:34 zoneminder kernel: NETDEV WATCHDOG: eth2: transmit timed out
Feb 17 00:43:37 zoneminder kernel: eth2: Transmit timeout, status 0c 0005 c07f media 10.
Feb 17 00:43:37 zoneminder kernel: eth2: Tx queue start entry 394428  dirty entry 394424.
Feb 17 00:43:37 zoneminder kernel: eth2:  Tx descriptor 0 is 0008a5ea. (queue head)
Feb 17 00:43:37 zoneminder kernel: eth2:  Tx descriptor 1 is 0008a5ea.
Feb 17 00:43:37 zoneminder kernel: eth2:  Tx descriptor 2 is 0008a5ea.
Feb 17 00:43:37 zoneminder kernel: eth2:  Tx descriptor 3 is 0008a5ea.
Feb 17 00:43:37 zoneminder kernel: eth2: link up, 100Mbps, full-duplex, lpa 0x45E1

The debug level messages repeat quite a bit throughout the logs.

Any ideas? Does Debian have support for the bttv cards offered by BlueCherry?