Thousands of Blue Screen events on remote IP cameras

[mjsameth]

Hi Gentlemen,

This problem has developed since I have upgraded an Ubuntu Zoneminder instance running 1.25.0. Originally, I had 8 IP Panasonic BB-HCM515a cameras running flawlessly on 1.22.0. This camera server provided security motion detection for a Data Center. Since the upgrade to Ubuntu 12.04 and Zoneminder 1.25.0 - each camera view indicates a period loss of signal (blue screen) and then records a 5-7 second image. Their were no changes on the source path or the configuration of each camera. The hardware is identical. Does anybody have any recommendations for eliminating the periodic loss of signal? (I have confirmed for a network perspective that the cameras are completely available and have never been disconnected.) Interestingly, I have 3 other ZM servers with similar cameras all running 1.24.0 and lower and this problem has never developed. I am happy to post any additional logs or info that could help in identifying the cause of this problem.

Sincerely,

Micah J. Sameth
Data Center Manager

[mjsameth]

Their are thousands of these events in /var/log/syslog:

Apr 30 08:46:20 ubuntu-zoneminder-DCT zmc_m9[14136]: WAR [Select timed out]
Apr 30 08:46:20 ubuntu-zoneminder-DCT zmc_m9[14136]: ERR [Connection dropped by remote end]
Apr 30 08:46:20 ubuntu-zoneminder-DCT zmc_m9[14136]: WAR [Unable to capture image, retrying]
Apr 30 08:46:20 ubuntu-zoneminder-DCT zma_m9[14151]: WAR [Signal: Lost]
Apr 30 08:46:20 ubuntu-zoneminder-DCT zma_m9[14151]: WAR [Signal: Reacquired]
Apr 30 08:46:20 ubuntu-zoneminder-DCT zma_m9[14151]: INF [PAT_SCH_3RR: 2970909 - Gone into alarm state]
Apr 30 08:46:20 ubuntu-zoneminder-DCT zma_m9[14151]: INF [PAT_SCH_3RR: 2970909 - Opening new event 64177, alarm start]
Apr 30 08:46:20 ubuntu-zoneminder-DCT zma_m9[14151]: INF [PAT_SCH_3RR: 2970910 - Gone into alert state]
Apr 30 08:46:21 ubuntu-zoneminder-DCT zmc_m6[14087]: INF [PAT_DC_Bay_1: 2960000 - Capturing at 5.43 fps]

[netwrkr]

Similar issue here - I resolved it by removing any settings in "Maximum FPS" for each camera. HTH

[mjsameth]

Thanks for your response. I just checked my source settings and currently "Maximum FPS" is blank on all 8 cameras. So I don't think that is it. All other settings have been set at default with the exception of the resolution being set at 640X480. Any other ideas?

[Briago]

I've used wired cameras for years. I recently got an IP wireless camera to test them out for friends and family that want a similar setup to mine now, but can't (or won't) run the wires.

I was also getting many zero byte (blue) images from the camera (seemingly random). I tried many options with frame rates, etc. Some things helped the situation, but nothing resolved it. On the wireless box I enabled and disabled the QOS function. My father had success in disabling this, but I found much better throughput on my wife's laptop with it turned on (maybe this varies by brand). However, what seems to have completely solved this for me was to set my wireless N router to not broadcast multiple types. I limited it to G only (no B and no N). My datarate on my wife's laptop went from 2Mbps (with N enabled) to 11Mbps (set only to G) testing with speakeasy.org. I think if I tried to use N only it would be much faster, but I have several non-N devices so that is not an option for me.

So set your wireless router to use only wireless technology, not the mixture of B/G/N that it is normally defaulted to. Let us know if this helps you. Also, keep in mind what equipment your firends may bring over when deciding if you will leave it on N or G. You will have to choose between compatability and throughput.

[mjsameth]

Hi Briago,

Thanks for your reply. All of these cameras are wired on an isolated VLAN. No wireless routers - any other ideas?

Micah J

[cambot]

Are you using http, rtsp, or ffmpeg to grab your images?

[mjsameth]

they are all http....

[Briago]

Hi Briago,

Thanks for your reply. All of these cameras are wired on an isolated VLAN. No wireless routers - any other ideas?

Micah J

I don't have any direct ideas, but from what I saw in my situation the issue was related to the network having intermittent data transfer loss. When the pakets were delayed it would result in the blue image. My guess is that when the http request is made, it is allowing a very short timeout for the data, and when the network delay occurs, it just closes out the session with a zero byte image.

Since you mention you are using an isolated VLAN, I would first try unisolating it. Put it all on one network and see if the issue goes away (for testing, I'm sure you have a reason for the isolation). If it works, then it seems that something in the VLAN configuration (I have no experience in VLANs) is causing intermittent packet delays.

When I was having the trouble on the wireless, I charted the data transfer rate over time, and saw with a single very lage file download the data rate would go up for x seconds, go to zero for y seconds, and oscillate like this continuously. You may not have a wireless issue, but maybe the cause is still something causing these kinds of symptoms on your wired network.

[mjsameth]

Your comments make me think that their might be a "HTTP timeout setting" that could be adjusted. Maybe in 1.25.0 it is set too low. Does anybody know if this is such a setting and where it could be adjusted? Is this an Apache issue? I will go ahead and move one of the cameras to a different vlan to see if that has any effect. Will post results....

[stealthboy]

I am also having this issue, but I'm over on the 1.24.4 line version, so it's not 1.25.x specific. Ever since upgrading to 1.24.4 I have had hundreds of blue flash screens each day (usually ~700 a day on *each* cam). This renders the system useless. I'm using wireless DLink cameras that, until the upgrade, worked flawlessly. Even more annoying, is all the blue flash screens get logged as events, but actual movement doesn't even trigger an alarm anymore.

Anyone have any ideas yet?

(also posted on the 1.24.x forums section)

[Briago]

Stealtboy: I am also using the dlink wireless cameras. Please be sure to try what I wrote about the wireless settings on the wireless router our access point. Try forcing it to only support G. You may not want to leave it, but test it.

[mjsameth]

I went ahead and changed the vlan on all of these wired POE ethernet cameras and had no change. Intermittent blue screen still appears and is recorded. At least for me this strange "loss of signal" after upgrade does not seem related to a network issue.

[stealthboy]

Stealtboy: I am also using the dlink wireless cameras. Please be sure to try what I wrote about the wireless settings on the wireless router our access point. Try forcing it to only support G. You may not want to leave it, but test it.

I will certainly experiment, and I'll used wired connections as well, but it strikes me as odd to start focusing on something that did not change (network configuration). The only variable was the software change to 1.24.4, so my empirical data so far points toward a ZoneMinder software issue (hence my posting on this forum).

I'll eliminate the network as a possible source of problems just to be thorough, however. Naturally, in my case keeping the wireless on 'G' is a no-go since I have many older devices that do not support that standard.

Thanks for your input! I'm trying really hard to not get completely frustrated at this. I had my system working flawlessly until I upgraded. I really, really cannot stand "upgrades" that break things.

[DarkBeer]

Sorry to dig up an old thread, but this is exactly the issue I am having. Was running 1.24.2 on an older single core cpu server with older 10.x Ubuntu. Setup 1.25.0 on a newer dual core with Mint Linux and same 5 cameras, but now I get numerous "signal" recordings with blue frames. Has anyone found a solution to this, or is the solution to go back to 1.24.2? I don't see how hit is a wifi issue since same cameras and wireless worked fine on the older setup