Better motion detection?

[mor]

I note that the current motion detection in Zoneminder is pretty simple, and it's performance on my (relatively noisy ) cameras leaves a lot to be desired.

A number of fairly simple improvements could be made:

#1. Use a 16-bit fixed point number to hold the reference image. This is very quick to update and holds much better precision, particularly in darker areas. i.e.

ref_pixel = ref_pixel - (ref_pixel>>7) +( image_pixel<<1)

where ref_pixel is a array of shorts, not chars. This gives a 1/128 blend, with all shifts and adds.

#2. Use smart filtering. This is slightly more complicated. You store 2nd reference image for the delta image, with very slow blending, and then subtract this from the delta image before you check for motion.

This has the effect of subtracting out areas in the image that are always changing a little bit. For example, waving branches, moving shadows, camera noise, and JPEG artifacts.

This allows you to get rid of the 'min_threshold' stuff as the filter will automatically adjust to the level of background noise the camera generates.

#3. Use the 4th power of the difference instead of the absolute difference. Generally, a small part of the image changing intensity a lot is much more important than the entire image changing a little bit. The current scoring treats then the same, but summing over the 4th power gives much bigger numbers for a object moving, and small numbers for the entire image getting a little brighter (as the camera auto-adjusts).

This lets you disregard the min_threshold as small changes don't do anything anyway.

#4. Fix the color space! argh! Zoneminder assumes that the input JPEG is RGB, which gives wierd image colouring when the image is really YUV.



So the question is: If I get off my arse and implement all this, is there any interest in merging the result into ZoneMinder?

[zoneminder]

Thanks for this. You have some good suggestions. The motion detection methods in ZM have been around for a while and could do with an update. I have plans to make the motion detection use plugin type functions so that different methods can be swapped in and out as desired. This would make it easier for you to implement your suggestions.

At this stage it is just a question of working out what the functional interface should be that would not prevent useful methods being applied. In other words what should be passed into the motion detection function and what should come out. Also what should persist between calls.

[mor]

I started writing this on the weekend. The biggest problem for me is the interaction between Image, Image::Delta and Zone.

Zone does a lot of fiddling with the internals of the Image object which makes it difficult to cleanly separate it out.

At the moment, Image::Delta calculates a delta, but Zone interprets it to build a score.

However, for more powerful scoring functions, there's a lot more state that's held than just the ref image on it's own, and there's a lot more calculation that's done then just counting pixels in an image delta. (it's not actually slow on modern CPUs are far as I can tell, as the speed is mostly dictated by the memory accesses).

So ideally, there's be an ref_image->Detect(Image, Poly[]) function that returned true if there was motion in the area defined by the Poly's. That could usefully use the state in the ref_image. Plug-in's could then inherit from Image and override the function?

Maybe. I'll play around with this some more.

Ps: I have a patch mostly finished that take seems to be much improved motion detection. My test is a 5 hour video of a sunlight room that people occasionally walk through. It nicely ignore the clouds going past, the shadows from the waving branches, the image going lighter and darker as the camera adjusts, but reliably picks up all the people walking past. I'm pretty happy with it!

The algorithm is to essentially calculate the standard deviation per pixel, the average per pixel, and the difference between the ref image average and the new image average, and munch all that together.

[cordel]

I think the calculate center is mostly just for referance for PTZ tracking (zmtrack.pl) so idealy you still want to calculate centers if enabled.

[zoneminder]

Having been reminded by your ealier post this is an area I will be looking at again in more detail in the next few weeks as I agree it is in need of a revamp. So any contributions you have will be welcomed and factored into that. Feel free to post patches for people to try anyway in the meantime.

[mor]

Ok, work in progress available at http://www.dgmo.org/diffs/zm-3.diff (note that it's unlikely to compile. The interest parts are the math in ::Blend and the calculations in ::Delta).

Essentially what I'm doing is computing the average variance per pixel, and the average per pixel.

Then when looking for motion, I take the new pixel value, subtract off the average, and compare the square of the difference with the variance. If it's less than twice the variance, then I ignore the pixel. (This is essentially saying that I ignore pixels that unless there's a better than 95% chance that the value is 'unusual'. i.e. within 2 standard deviations).

Then I compute the Z-score^2 for each 'unusual' pixel, and pass that into the zone detector. The zone detector can than compute the average Z-score^s for a zone, and use a threshold detector against it.

The advantages of this are that it automatically compensates for any noise in the picture (thermal, JPEG artifacts, MPEG artifacts, small, constantly moving shadows) as these will grow the variance for the affected pixel(s) over time.

I also compute the average pixel value for both the reference and new images, and use this to apply a level correction. This is used to ignore cameras changing their white balance or exposure settings.

The disadvantage of all this is that it's slightly computationally more expensive. There's a number of adds, shifts and multiplies per pixel. One divide per pixel is done only when there's motion. On semi-modern CPUs, the calculations should run fairly close to the memory bandwidth limit so little or no actual slowdown should be seen in practice. I think.

In day-to-day usage, this all results in MUCH more accurate motion detection, with a huge decrease in the number of false positives and false negatives. My disaster case is a room with sunlight in one section, shaded by branches. The new detector successfully ignores clouds going past, shadows of branches moved by the wind, the sky getting brighter and darker with time of day, but picks up people walking through the room 100% of the time.

Any comments on this approach?

[zoneminder]

This is very interesting work. I am a bit stuck for time to go into this in any depth at the moment but please keep it up and I will have a proper look once I get time.

[SyRenity]

Hi.

This sounds interesting and may finally fix the motion detection issues I have personally with ZM .

Any chance of having the path in compilable state, so I could test it as well?

Also, any special settings need to be set in the zone configuration, or it will work out of box?

Thanks.

P.S.: The link seems down?

[mor]

Updated patch available at http://www.dgmo.org/motion-1.diff

This against 1.23.2 and it's what I'm running on my system at the moment. It's still slightly CPU heavy, but not terrible. My system is watching 4 x MJPEG cameras at 640x480 at 20fps each.

I find this to be _much_ improved on the current algorithm. Of the 306 events today, it caught all of the actual motion events, and got about 20 false positives (i.e. Thought there was motion, but it was actually the sun coming out from behind a cloud). So I'm pretty happy with it.

The algorithm is a little changed from the above. I'm trying the average value for each pixel (i.e. a blended image), and the average square-of-the-difference for each pixel. The later is used to estimate the noise values for each pixel, and motion on a pixel must exceed the noise value to be considered.

I'm also then using the sum of square of the square of the differences to consider against a threshold value (the idea here is that a smaller number of pixels that changed a lot is much more important than a large number that changed slightly).

Patch is still pretty rough; a fair number of stray debugging statements et al are littered through it, but it should work.[/url]

[jameswilson]

Impressive stuff. Sounds like your doing well

[SyRenity]

Thanks, I will try it.

One question though again - do I need to change anything in the zone settings? Or I can use the default values for new monitors?

[moOd]

Hi there! This is really interesting stuff!

I tried your patch on version 1.23.2, with some slight modifications on your diff-file (some entries seemed to be incorrect).. but I am expericing problems with the "zma".

Here is what my /var/log/messages says:

Code: Select all

May 13 23:48:43 yin zma_m1[7058]: ERR [Got signal (Segmentation fault), crashing]
May 13 23:48:43 yin zma_m1[7058]: ERR [Signal address is (nil), from 0x8050445#012]
May 13 23:48:43 yin zma_m1[7058]: ERR [Backtrace: /usr/local/bin/zma [0x8050445]]
May 13 23:48:43 yin zma_m1[7058]: ERR [Backtrace: /usr/local/bin/zma [0x8050445]]
May 13 23:48:43 yin zma_m1[7058]: ERR [Backtrace: /usr/local/bin/zma [0x806419a]]
May 13 23:48:43 yin zma_m1[7058]: ERR [Backtrace: /usr/local/bin/zma [0x806dd47]]
May 13 23:48:43 yin zma_m1[7058]: ERR [Backtrace: /usr/local/bin/zma [0x804a671]]
May 13 23:48:43 yin zma_m1[7058]: ERR [Backtrace: /lib/libc.so.6(__libc_start_main+0xe0) [0x160390]]
May 13 23:48:43 yin zma_m1[7058]: ERR [Backtrace: /usr/local/bin/zma(__gxx_personality_v0+0x211) [0x804a321]]
May 13 23:48:43 yin zma_m1[7058]: INF [Backtrace complete]
May 13 23:48:43 yin zmdc[6994]: ERR ['zma -m 1' exited abnormally, exit status 11]

ZM worked fine with no errors before patching.

I changed all "/* double pts = */" entries in your diff to "double pts =" instead, then I also removed "static struct timezone..." from zm_monitor.h and leaved the "extern struct timezone" line there instead.

Did I something wrong?

[cordel]

ZM worked fine with no errors before patching.

I changed all "/* double pts = */" entries in your diff to "double pts =" instead, then I also removed "static struct timezone..." from zm_monitor.h and leaved the "extern struct timezone" line there instead.

Did I something wrong?

All the "/* double pts = */" entries are in the existing code, if the lines does not match the existing in the patch, then the patch will not apply properly. Changing any line that has a minus in front of it, breaks the patch.

[moOd]

ZM worked fine with no errors before patching.

I changed all "/* double pts = */" entries in your diff to "double pts =" instead, then I also removed "static struct timezone..." from zm_monitor.h and leaved the "extern struct timezone" line there instead.

Did I something wrong?

All the "/* double pts = */" entries are in the existing code, if the lines does not match the existing in the patch, then the patch will not apply properly. Changing any line that has a minus in front of it, breaks the patch.

Are you sure that they exist? I don't think so.. double checked the original code for those entries, but can't find any (ZoneMinder-1.23.2).
For me the patch did not apply properly without changing it, which makes me believe that may be my problem with crashing zma.

[mor]

Try using http://www.dgmo.org-a.googlepages.com/z ... ocal.patch instead. That's a diff generated versus a virgin 1.23.2 source code tree. It should apply without any rejects at all.

Note that this version it will write out a few debugging images that I'm using to watch how success the system is.

I take it you are trying to apply the patch to 1.23.2?

If that still fails, I can upload some x86_64 RPMS for Fedora-8 that I'm using.