Timeline is "stuck"

[brentl99]

I have ZM 1.25.0 / CentOS 6.2 running with one camera on modect, it is basically working and the log is clean (no errors, no warnings, etc).

Every few days the timeline view gets "stuck", it will show the timeline ending anywhere from a few hours to a day in the past. Yet I know from the Log that the camera monitor is continuing to run and I know that the camera task is recording and saving to disk.

If I restart the "zm" service, the timeline corrects, shows the missing frames and is "up to date".

I'm not really sure where to start with debugging this, I don't know what information to provide to help with this thread and it would helpful if someone could give me a idea of where to start looking to solve this issue.

I have also tried the current version of ZM from the SVN repository and it has the same problem.

Thanks.

[knight-of-ni]

I'm running zm 1.25 on CentOs 6.2 as well, but I do not appear to have any issues with the timeline.

I know you said your log file is clean, but have you checked your log files for any errors or warnings within a couple +/- minutes when the timeline stops working?

Can you post the hardware you are using, including the camera? What frame rate and resolution are you recording at?

[brentl99]

I'm using Foscam FI8918W's and "snapshot.cgi". I could not get "videostream.cgi" to work and I saw a post that recommending shapshot for Foscam cameras. I have a max and alarm frame rate of 3. I have not looked at the time it's freezing because I'm not seeing any errors in the log at all. On the contrary, I'm seeing regular "Processing at 3.00 fps" and "Capturing at 3.00fps" for the two cameras I'm currently testing with. If I knew what script looked for the frames I could probably debug it.

What is most strange is that when I restart zm, "service zm stop; service zm start", the timeline shows all the "missing" frames.

Just out of curiousity, are you running with selinux disabled? I've addressed all my avc issues, but it's possible something is going unreported. Since my system is connected to the Internet and I'm planning on using squid to reverse proxy to zoneminder, I want selinux enabled.

[brentl99]

Something that doesn't add up for me is the "Linked Monitors" list under the camera configuration. When I bring up the monitor configuration, it always lists the name of the "other" camera. But if I shuffle through the tabs and back to the "General" tab the list then changes to show both camera names. Because I don't know what this means, it makes me wonder if that is causing some problem.

[knight-of-ni]

I'm running with selinux completely disabled, and I've also got modsecurity set to ignore zoneminder.

For testing purposes, try turning it off. If it turns out to be the cause then we can get into re-configuring some of the paths zoneminder uses to be more selinux friendly. My instance of Zoneminder is based on a Fedora rpm spec file, and there is a comment inside that spec file that talks about this.

Recommend you leave the max & alarm fps fields empty and manually set the frame rate inside the camera. Those fields were not intended for ip cameras (zoneminder cannot directly control frame rate of ip cameras).

Just want to verify that you are testing this directly and not through squid, right? Sounds like you are, but I thought I'd ask anyway. I've never used Squid, but I imagine it caches webpages for performance.

I'm using Foscam FI8918W's and "snapshot.cgi". I could not get "videostream.cgi" to work and I saw a post that recommending shapshot for Foscam cameras. I have a max and alarm frame rate of 3. I have not looked at the time it's freezing because I'm not seeing any errors in the log at all. On the contrary, I'm seeing regular "Processing at 3.00 fps" and "Capturing at 3.00fps" for the two cameras I'm currently testing with. If I knew what script looked for the frames I could probably debug it.

What is most strange is that when I restart zm, "service zm stop; service zm start", the timeline shows all the "missing" frames.

Just out of curiousity, are you running with selinux disabled? I've addressed all my avc issues, but it's possible something is going unreported. Since my system is connected to the Internet and I'm planning on using squid to reverse proxy to zoneminder, I want selinux enabled.

[knight-of-ni]

Linked Monitors are a way to cause a camera to start recording based on activity from another camera. It's in the documentation if you care to learn more. For now, this should be safe to ignore.

Something that doesn't add up for me is the "Linked Monitors" list under the camera configuration. When I bring up the monitor configuration, it always lists the name of the "other" camera. But if I shuffle through the tabs and back to the "General" tab the list then changes to show both camera names. Because I don't know what this means, it makes me wonder if that is causing some problem.

[brentl99]

I have changed the default paths and it does make for easier/few selinux issues. Curiously ZoneMinder does have some path references to /tmp/zm hard coded. I'm very familiar with selinux, it really is not that hard to configure to work with ZoneMinder. Anyway, that is a difference. I have built a zoneminder selinux policy and currently get no avc audit messages I do not believe this is my issue. I have not looked at modsecurity, that could be a factor.

I'm Not currently testing with squid. You are correct, there can be squid caching issues, more recent versions of squid allow for caching to be easily disabled.

I'll blank the max and alarm frame rates and try videostream.cgi again with the parameters to set the frame rate. I thought ZM automatically built the parameter list (other that user id/password) for videostream.cgi and snapshot.cgi based on these configuration values. I actually did not realized until now that videostream.cgi and snapshot.cgi were part of the camera firmware not ZoneMonder. I feel some what dumb for not cluing into this sooner.

Thanks for the information on link monitors. I actually did look through the user manual for it, but I could never find anything other than the most elementary description of it. I guess I needed to look harder.

I appreciate your help.

[knight-of-ni]

Just had a random thought...
Have you tried viewing the timeline from a different pc/browser and then compared the results? Just wondering if the browser itself is not caching the page? Don't know if this affects the timeline, but you might want to make sure the following is enabled: Options -> Images -> RAND_STREAM

Also, do the pan left, pan right, & zoom buttons still work after the timeline gets stuck?

[brentl99]

It's not a browser caching issue because, where possible, I have my browsers set to clear cache on close. I have tested it with IE, FF, Opera, Chrome and Safari. I can still navigate, left, right pan works and I can "view" the timeline that is shown. RAND_STREAM is checked, it must install that way.

Today it stopped at about 1PM, it about 3PM and I see no timeline after 1PM. The last event says:
932 New Event Monitor-1 Signal, Linked 04/16 12:59:02 0.00

I scanned the log:
2012-04-16 13:13:16 zma_m1 16079 INF Monitor-1: 76000 - Processing at 0.98 fps zm_monitor.cpp 1055
2012-04-16 13:13:15 zmc_m1 16076 INF Monitor-1: 76000 - Capturing at 0.98 fps zm_monitor.cpp 2598
2012-04-16 12:59:30 zmaudit 16085 INF Deleted 19 log table entries by time zmaudit.pl
2012-04-16 12:59:17 zma_m1 16079 WAR Signal: Reacquired zm_monitor.cpp 1190
2012-04-16 12:59:15 zma_m1 16079 INF Monitor-1: 75173 - Opening new event 932, alarm start zm_monitor.cpp 1346

That looks like what I would expect.

Then I restarted zm and event 932 says:
932 Event-932 Monitor-1 Signal, Linked 04/16 12:59:02 6367.77 6268 6258 315584 50 150

Now I can see the remainder of the timeline. So I think what I'm seeing is that some events are running a really long time. Is there a way to set a maximum event duration?

[knight-of-ni]

Under the buffers tab there is "Post Event Image Count" which is the number of additional frames (not time) to record following an "event". The defualt on my system seems to be 10, which would correspond to 10 seconds if recording at 1fps.

It also looks like you've still got a linked monitor.

Try deleting all the monitors you have created thus far and create a new one with the following:
Name: Test
Function: Modect
Enabled: checked
Linked Monitors: {nothing should be highlighted here}
Max FPS: {empty}
Alarm Max FPS: {empty}

Set up the Source tab like you normally would and leave everything else at the defaults.

Once you have that set up, open the live monitor. Verify the monitor does not immediately go into Alert or Alarm. If it does then you need to adjust your motion sensitivity.

The idea here is to get the motion sensitivity so that, you can walk in front of camera, generate an event, let the event process, and then see it on the timeline. Too sensitive and it will generate many events and possibly be creating your problem.
Now the boring part... keep an eye on your camera over time to verify that it is not going into alarm too often or too little.

The idea behind this is to verify that we are not dealing with a camera configuration issue.

[brentl99]

I haven't attempted to change any of the default parameters other than those that seem needed to connect to each camera. The pre/post image counts are 10.

I deleted the two cameras that I have setup.

I recreated the first and gave it a unique zone. I then recreated the second and gave it a unique zone. I set the zone to "Fast, High Sensitivity" as that was what I found in the documentation as the recommended starting setting. After watching the live image for about 10 minutes, neither is in alarm.

However, each camera still shows the other camera's name under the "linked monitors". The Linked Monitors section seems to have a mind of it's own.

I'll let this configuration run for a while and see what I find.

Thank you.

[knight-of-ni]

You should see the other cameras listed, but not highlighted.
I've attached a screenshot of what I'm talking about.
None of the cameras in the screenshot are linked.

For this test, I recommend you run with just one camera. It'll make things easier to figure out what is going on.

[brentl99]

Well, truth be told, I actually have two systems I'm working on, both running CentOS 6.2, ZM 1.25.0 and the same Foscam cameras, the first system has one camera and the second system has two cameras.

On the system with one camera it was in "alarm" state. I changed the zone preset to "Fast High Sensitivity", it appears that it defaulted to "Best High Sensitivity". Even though that camera is focused on a static setting, the best/high preset was apparently resulting in long, continuous, alarm events. At first glance, it appears to be working better on "Fast High Sensitivity".

I'll have to check in and look at the other system with two cameras (I changed the zone sensitivity preset on it last night) to see how it is working.

Thank you.

[brentl99]

I believe I now do know what my issues are.

One problem was that there is a ZM post that identifies the Foscam videostream.cgi parameters incorrectly. Based on this I thought I was passing a parameter to the camera asking for 3fps when it was 1fps. For the benefit of anyone else using Foscam cameras here is what I would consider to be the definitive resource on Foscam cgi:
http://www.foscam.es/descarga/ipcam_cgi_sdk.pdf

A frame rate of 1fps was not allowing modect to do its job.

Having said, I have run the gamut of the available ZM zone presets and tweaking those preset values and I am getting one of two results. The alarm state is always idle (modect doesn't detect any motion). Or, the alarm state begins idle, validation motion occurs, the camera goes into alarm and begins recording and never stops. The camera view that I am focused on trying to get working is an interior office view that is static unless someone walks by. If I watch the video stream there is no obvious reason to me why modect will not stop recording after going into alarm.

[knight-of-ni]

The presets won't always work. Sounds like you will need to manually edit the parameters yourself.

What I like to do is take a close look at the "score" of the frames that are detected as motion. Clicking the number in the Frame Id column should open an image with what the software saw as motion (it will be outlined in red). I find it very helpful when I can actually see what the software identified as motion. Based on these frames, you should be able to estimate a score, below which is just noise and above which is legitimate motion.

Now, enter that number into the Min Pixel Threshold field. Now test.

Chances are you will still need to tweak the other motion detection parameters. For that, I encourage you to read the documentation in the Wiki as I would just be repeating what it already says.

NOTE: Every camera will be different, which is why no one can simply tell you what to set the motion parameters to. You have to figure that out for each camera you have.