Better motion detection?

mor · Post by **mor** » Wed May 14, 2008 10:25 pm

SyRenity wrote:Hi.
Indeed, I even posted about this a year ago here in forums. His 3rd method (http://www.codeproject.com/KB/audio-vid ... tion/3.jpg about half page) said to bring great detection quality, while significantly lowering CPU usage.

What he does, is basically pixelating the image before comparing it, thus lowering the number of pixels needed to be checked, and subsequently lowering the required CPU.

Mor, I wonder whether you can use it with your algorithm, and this way solve the increased CPU usage you mentioned.

(Btw, how exactly higher this usage then standard ZM method?)

Yes, decimation is a standard technique to reduce the CPU cost. At the moment, I'm not fussed about it as the difference in CPU is ('only') in the order of 20 or 30% on my system. I've got plenty of CPU to handle this. Decimation will also reduce the accuracy somewhat.

Having said that, the next step is to use a median filter to further reduce image noise which is significantly more expensive which is where decimation will become needed. (this is akin to the 'Erosion' filter that's mentioned in the article. Erosion is a significantly more expensive operation though).

It would probably be worth just adding decimation as a setting (2x, 4x or 8x decimation). I have a look at this when I start doing code optimization.

SyRenity · Post by **SyRenity** » Thu May 15, 2008 11:01 am

Hi.

mor wrote: It would probably be worth just adding decimation as a setting (2x, 4x or 8x decimation). I have a look at this when I start doing code optimization.

It's a good idea, and would allow people to decide for themselves whether they prefer speed or accuracy.

moOd · Post by **moOd** » Thu May 15, 2008 12:52 pm

mor wrote:In zm_zone.cpp, line 201, there's a line that reads
Code: Select all
        if (pixel_avg > 2.50 && 0) {
It should read
Code: Select all
        if (pixel_avg > 2.50) {

I'm sorry.. but, I think this correction was already implemented in your previous patch, that I was testing out. I doublechecked and there was no "&& 0".

This is what's shown in my log:

Code: Select all

May 15 22:49:42 yin zma_m1[16079]: INF [In mode 3/1, warming up]
May 15 22:49:53 yin zma_m1[16079]: INF [Average 47.034281, pixels 125625 (10, 10, 10)]
May 15 22:49:53 yin zma_m1[16079]: INF [IPCamera: 026 - Gone into alarm state]
May 15 22:49:53 yin zma_m1[16079]: INF [IPCamera: 026 - Creating new event 87]
May 15 22:49:53 yin zma_m1[16079]: INF [Average 52.207340, pixels 125664 (10, 10, 10)]
May 15 22:49:54 yin zma_m1[16079]: INF [Average 46.839957, pixels 125584 (10, 10, 10)]
May 15 22:49:55 yin zma_m1[16079]: INF [Average 48.454124, pixels 125638 (10, 10, 10)]
May 15 22:49:56 yin zma_m1[16079]: INF [Average 48.623677, pixels 125671 (10, 10, 10)]
May 15 22:49:56 yin zma_m1[16079]: INF [Average 49.818443, pixels 125728 (10, 10, 10)]
May 15 22:49:57 yin zma_m1[16079]: INF [Average 51.278156, pixels 125755 (10, 10, 10)]
.
.

It just goes like this, forever, in an endless loop.

Do you think my zoning is incorrectly setup, or do I need to manually change the "pixel_avg" value in the sourcecode?

mor · Post by **mor** » Thu May 15, 2008 11:09 pm

Hmm. Are the numbers trending down slowly over time?

If you're only running 2 fps, it may take up to 15 mins to train the noise filter. On my 20 fps noisy camera, it takes about 60 seconds to go quiet, which would be about 10 mins at 2 fps.

In /tmp, there should be a *.jpeg file which is the current noise filter. It should start black, and then white bits should appear over time as it works out which parts of the image are noisy. If you can, could you leave it run for about 30 mins and this maybe post this image?

Also, in /var/lib/zoneminder/events/1 should be files named diag-*.jpeg which are the delta images. if you look at these, they'll be white where there are differences from the reference image.

Hmm. and the obvious question: Are you running an RGB camera, or a black and white camera?

moOd · Post by **moOd** » Fri May 16, 2008 4:56 am

Thanks for your kind help with this..

I left it on over the night and it's still recording (~8hrs later).

The numbers does not seem to trend down, as they look like this now:

Code: Select all

May 16 06:42:20 yin zma_m1[16079]: INF [Average 220.483776, pixels 125961 (10, 10, 10)]
May 16 06:42:20 yin zma_m1[16079]: INF [Average 220.336514, pixels 125961 (10, 10, 10)]
May 16 06:42:21 yin zma_m1[16079]: INF [Average 220.705546, pixels 125960 (10, 10, 10)]
May 16 06:42:22 yin zma_m1[16079]: INF [Average 220.842371, pixels 125961 (10, 10, 10)]
May 16 06:42:22 yin zma_m1[16079]: INF [Average 220.827454, pixels 125961 (10, 10, 10)]

They seem to go up instead.

I could not find any images in /tmp, just only a bunch of zm* logfiles.. maybe this has something to do with the problem?
My diag files are normally created in: /usr/share/zoneminder/www/events/1/<event> folder, but diag-logging is not turned on right now. I tested diag-logging before, and those pics looks like you describe.. b&w with white pixels where there's differences from the reference. Should I leave diag-logging on?

I'm running a RGB camera.

mor · Post by **mor** » Fri May 16, 2008 10:35 pm

Now that is a puzzlement.

In zm_image.cpp, around line 1888 is there a chunk of code that looks like

Code: Select all

                {
                        char b[1024];
                        sprintf(b,"/tmp/sq.%d.jpeg", getpid());
                        result->WriteJpeg(b);
                        delete result;
                }

This should be writing the noise mask out to /tmp/sq.*.jpeg

Hmm. Just to check, you are using zoneminder-1.23.2-local.patch with crash-fix.diff applied on top, against 1.23.3, right?

Trying to think of what else could be different; Hmm. What resolution is your camera?

The "10, 10, 10" in the log line indicates that the level correction is clamping. Which is itself odd. Maybe try applying

Code: Select all

--- a/src/zm_image.cpp
+++ b/src/zm_image.cpp
@@ -1073,11 +1073,11 @@ Image *Image::Delta( const Image &image ) const
                b_correct = (int)(avg_b - b_correct) / (int)(width * height);
 #endif
                if (r_correct > 20)
-                       r_correct = 10;
+                       r_correct = 0;
                if (g_correct > 20)
-                       g_correct = 10;
+                       g_correct = 0;
                if (b_correct > 20)
-                       b_correct = 10;
+                       b_correct = 0;
 
                result->r_correct = r_correct;
                result->g_correct = g_correct;

to clamp to zero. I'm trying to imagine why the average RGB values for your blending image would be so radically different than your new images tho.

If nothing else, could you possibly tar up a few hundred of the images it logged and put them somewhere? I'll manually run them through to see where things are going wrong.

What Blend percentage are you using BTW? is it 1%?
I take it you're running on a i386 architecuture? or x86_64?
(I'm running x86_64, but I don't think I've relied on 64 bit values anywhere).

Edit: Ahhh. Do you have ZM_BLEND_ALARMED_IMAGES ticked? It definitely should be ticked! (Options/Config/ZM_BLEND_ALARMED_IMAGES)

I should just change the code to enforce that.

mor · Post by **mor** » Fri May 16, 2008 11:22 pm

Revised patch:

http://www.dgmo.org/zoneminder-1.23.3-motion-2.diff

This enforces the 'blend during motion' setting, allows the noise filter to ramp up a little faster, and clamps to zero when the level adjuster fails.

moOd · Post by **moOd** » Sat May 17, 2008 1:06 am

Now it seems to work correctly!

I took a look at zm_image.cpp and found this:

Code: Select all

if ( config.fast_image_blends )
.
.
} else {
.
.

..and realized that "ZM_FAST_IMAGE_BLENDS" needs to be turned off. I turned it off and the Average counter started ticking down and an image was also created in /tmp.

It's currently too dark outside, so I need to get some sleep til' tomorrow to see the result.

Thanks for your help!

mor · Post by **mor** » Sat May 17, 2008 2:43 am

Ahh. Yet another option I hadn't considered!

Good to hear it's going in the right direction. I'll be interested to hear about your results tomorrow.

Note that if you set the image blend percentage higher, it should compensate for your low frame. You may want to experiment with as high as 8%.

moOd · Post by **moOd** » Sat May 17, 2008 11:17 am

I've been testing various settings of image blend now. My monitor is currently updating around 1.5-2fps.

1% was way too slow in compensating lightning changes. 10% was pretty OK, but I was still receiving a bunch of false alerts when my cameras built-in light compensator was working. Now I't running with 12% blend and it works pretty well.

Original image:

Delta image from /tmp:

As you can see from the pictures, it's pretty windy and raining today, and it's impressively not picking up any false alarms at all from all the waving leaves. Birds flying across the picture and people going behind the building at the right is picked up without problem.

Is there any way to create analysed alarm images with motion outlined (ZM_CREATE_ANALYSIS_IMAGES)? ..or is your method not working with blobs at all?

mor · Post by **mor** » Sat May 17, 2008 10:56 pm

Yay! Looks like it's working just the way it's supposed to.

It's interested to see on your fairly good camera it stil picks out all the JPEG noise (you can see all the 8x8 squares in the noise filter).

All the normal things are desensitised: The moving branches, the straight diagonals, shiny surfaces, and translucent surfaces. Ok. I'm pretty happy

Re blob outlining: I've never tried it. Not been interested.

Maybe try applying:

Code: Select all

diff --git a/src/zm_zone.cpp b/src/zm_zone.cpp
index f2386b6..8fa5e4e 100644
--- a/src/zm_zone.cpp
+++ b/src/zm_zone.cpp
@@ -212,6 +212,7 @@ bool Zone::CheckAlarms( const Image *delta_image )
                return( true );
        }
 
+#if 0
        return false;
 
        if ( !alarm_pixels )
@@ -227,6 +228,7 @@ bool Zone::CheckAlarms( const Image *delta_image )
                overload_count = overload_frames;
                return( false );
        }
+#endif
        score = (100*alarm_pixels)/polygon.Area();
        Debug( 5, ( "Current score is %d", score ));

and give it a go

moOd · Post by **moOd** » Mon May 19, 2008 8:00 am

As I understand it, it learns to ignore areas that are constantly appearing in the image and filter out that "noise".

What happens if I temporarily put a chair or any other object in front of the camera, for say.. 5 minutes and then remove it. Will that area still be ignored until I restart ZM, or will it "forget" that object after some amount of time?

I will give your new patch a try later..

Thanks!

mor · Post by **mor** » Mon May 19, 2008 8:58 am

The short answer to that is 'no'.

The longer answer is: It's not change that's important, so much as constant change. If an area is always different from the blended image, than that difference will be reduced in importance over time until it's ignored.

For example, if a pixel is constantly changing between 90 and 110, then eventually those changes will be ignored. But that pixel changing to be 120 will still be picked up.

Also, a pixel changing once from 90 to 110 which temporarily result in a small increase in the noise filter (say to +2 maybe, for that pixel), but it will slowly fall back to zero, because there's no more change coming. And then, when you "take the chair away", the pixel will revert back, the noise filter will bump up slightly from the change, and then fade back to zero.

To directly answer your question; the noise filter decays over time back to zero. If you look at /tmp/sq.*.jpeg before dawn, you should see it be fairly uniformly back (as it's decayed back to zero).

Does that make sense?

The maths for this are implemented mostly in the ::Blend() routine. First, the new blended image is calculated. Then the difference between that blend image and the current image is created, and then that's used to create the 'average difference' (well, really the average of the square of the difference).

Then in the ::Delta() routine, that square of the average difference is compared with the square of the difference, and the change is ignored if it's less than the average difference squared.

Hmm. It's easier to write the code for this than explain it!

There are also some magic numbers. I clamp the growth in the noise filter to be less than (24/256 * the transparency) per frame. Differences less than 8 times the noise mask are ignored. And the threshold is 2.5 (these are all trial and error. I should work out real values at some point).

If it's still not learning fast enough, you could increase the 24 (should be stable at values up to about 128). You can play with the 2.5 somewhat. I've contemplated raising it to as much as 4.0 but have run out of time to really test it properly.

SyRenity · Post by **SyRenity** » Mon May 19, 2008 10:05 am

Hi.

This image was impressive!

I presume, some more advanced functions like object removal and placement detections could be developed with this approach? Because you already have the ongoing "state" of the background, and can detect whether something was added or removed?

overly · Post by **overly** » Mon May 19, 2008 4:35 pm

mor,

Thanks for your work on this. I've been battling with tree shadows for a while now and I'm excited to try out your work.

I'm working on patching now and will post some results soon.