I wrote an object-detection add-on for ZM

[tpcarmen]

Right now, it's running on the latest ZM release on CentOS 7.3.

It's a stand-alone C app that populates a table with matched objects (people, cars, etc), so besides "something moved" now ZM has the data for "a person" is in this frame. Or "a car" or any of a number of other things it recognises.

If anybody wants to take a crack at integrating it into ZM, it would be appreciated.

https://github.com/tpcarmen/darknet

Terry

[rockedge]

nice!.... what is required to build it? how does it work?

I get this error trying to compile. Is it related to Cuda? I have opencv.

Code: Select all

gcc  -I/usr/include/mysql -DGPU -I/usr/local/cuda/include/ -Wall -Wfatal-errors -Ofast -DGPU -c ./src/regressor.c -o obj/regressor.o
In file included from ./src/activations.h:3:0,
                 from ./src/layer.h:4,
                 from ./src/network.h:6,
                 from ./src/regressor.c:1:
./src/cuda.h:10:26: fatal error: cuda_runtime.h: No such file or directory
 #include "cuda_runtime.h"
                          ^
compilation terminated.
make: *** [obj/regressor.o] Error 1

[tpcarmen]

Yes, I beleive so.

Try setting GPU=0 in the Makefile. (it's the first line).

To get the GPU stuff working and get really high performance, you'll need an nVidia CUDA compatible graphics card and the CUDA tooolkit from https://developer.nvidia.com/cuda-downloads

However just for trying it out, you can set GPU = 0 and it does it with the normal CPU only a lot slower.

> I have opencv

I've never tried opencv.

For help with the actual image detection stuff, check out https://pjreddie.com/darknet/yolo/

Terry

[rockedge]

I have modified the Frames interface to send the individual frame image selected, to darknet/YOLO which writes the predictions.png which is symlinked to the webserver root and is opened and displayed next to the Frame image.
not perfect but fun to experiment with.

[tpcarmen]

Cool!

That's how I got started, it is a lot of fun, but it takes time for it to create the new image and draw the boxes and write them out, so I ended up removing all the actual image output code and just pop the names of the objects into the new ZM.FrameMatches table.

However it would be nice to have the images available, maybe as a debug switch.

In any case, this all looks very promising for making ZM the best software available. I don't think anything in the "costs less than a fortune" class does any actual recognition".

With the new data, it would be possible to have alerts like "Someone is at the back door", but ignore "The trash can blew over by the back door".

I need to learn a whole lot about YOLO/Darknet and suspect that it could be even faster with a smaller set of objects it can recognize.

[rockedge]

The training data to modify the weights file I think needs to be refined. I have not tried Tiny YOLO yet.
I do have some lag waiting for the image to be finished but looked at this also as an option that can be selected. Just having the detection data for further Filter actions would be a powerful addition. I am looking into an experiment with darknet-yolo detection on a ZM stream. Just learning how to drive this stuff...agreed this as an addition to ZM would place it right up there with most commercially available systems.

I do not have a good graphics card as of yet to use GPU.... but I am looking into your code and maybe using it modified so I can integrate into ZM and look at the possibilities.

[asker]

Good work.

Yes, using a CNN approach without a GPU is extremely slow. Here is another approach if you want to tinker - this approach directly reads from ZM's shared memory buffer and does face/object/or whatever recognition. You can use CNNs, or HOG (which is much much faster for face recognition, but less accurate). Tapping from memory makes this approach faster and eliminates DB dips

https://medium.com/ozone-security/dont- ... ab2ebd007d

Notes:

• This is not the right forum for an extended oZone discussion - the only reason I posted it here is for folks who want to download and tinker along with ZM for their personal use
• oZone is not maintained by the current ZM team, so don't ask them for help. oZone also is not GPL licensed, its free only for personal use.
• Disclaimer: I am the co-maintainer for oZone

[tpcarmen]

The training data to modify the weights file I think needs to be refined. I have not tried Tiny YOLO yet.

The training data is the key, but I don't know how to get access to the training images, so I used it as-is. Tiny YOLO worked, but guessed wrong a lot. If you edit the weights file, maybe you could post some info on them?

I bought a low end GPU card, and it makes a tremendous difference.

> I do not have a good graphics card as of yet to use GPU.... but I am looking into your code and maybe using it modified so I can integrate into ZM and look at the possibilities.

Feel free. If you fix anything please feel free to update git.

I do have some lag waiting for the image to be finished but looked at this also as an option that can be selected. Just having the detection data for further Filter actions would be a powerful addition. I am looking into an experiment with darknet-yolo detection on a ZM stream. Just learning how to drive this stuff...agreed this as an addition to ZM would place it right up there with most commercially available systems.

I do not have a good graphics card as of yet to use GPU.... but I am looking into your code and maybe using it modified so I can integrate into ZM and look at the possibilities.

FWIW, one of the reasons I wrote it as a detached add-on is that the GPU can't keep up with the images if there's a lot of movement (meaning a lot of images to go through). Using the Frames table DB as a processing queue means that the GPU can chug through any number of images at it's own speed instead of losing stuff when overloaded.

> I do not have a good graphics card as of yet to use GPU....

It's quite usable for testing even without a graphics card. It only takes a few seconds to do an image.

Thanks!

Terry

[tpcarmen]

Yes, using a CNN approach without a GPU is extremely slow. Here is another approach if you want to tinker - this approach directly reads from ZM's shared memory buffer and does face/object/or whatever recognition.

That won't hold up when the system starts tossing out a lot of frames to analyze, which is why I use the Frames table as a queue.

A wind storm for example, will flag a lot of frames as "something moved", but the GPU won't be able to keep up and might lose interesting data.

By processing from the DB, everything gets looked at, even if it takes while to chug through the queue.

[asker]

Yes, using a CNN approach without a GPU is extremely slow. Here is another approach if you want to tinker - this approach directly reads from ZM's shared memory buffer and does face/object/or whatever recognition.

That won't hold up when the system starts tossing out a lot of frames to analyze, which is why I use the Frames table as a queue.

A wind storm for example, will flag a lot of frames as "something moved", but the GPU won't be able to keep up and might lose interesting data.

By processing from the DB, everything gets looked at, even if it takes while to chug through the queue.

Depends on your need. If you are looking at real-time analysis (and more importantly, real time notification due to a detection - example 'send me an alarm if a person is in front of my door'), dipping on DB is not a good idea. If you are looking at post event analysis it works. Picking up frames from memory doesn't mean you process _every_ frame. You decide how many frames you pick.

[rockedge]

Both looking at image data via the Frame table, or processing live streams should be options. A daemon process that can be started to achieve this along the path tpcarmen is suggesting, something like zmaudit.pl? The real-time analysis need only be applied maybe in the alarm frames and then at first by working with a sampling of some frames and then some logic to determine if that analysis frame rate should increase. So let ZM detect the initial motion then trigger real time analysis?

[tpcarmen]

Both looking at image data via the Frame table, or processing live streams should be options. A daemon process that can be started to achieve this along the path tpcarmen is suggesting, something like zmaudit.pl? The real-time analysis need only be applied maybe in the alarm frames and then at first by working with a sampling of some frames and then some logic to determine if that analysis frame rate should increase. So let ZM detect the initial motion then trigger real time analysis?

That sounds like it would work well, although I'm not sure how many users would have the hardware to do realtime in HD.

My card was a few hundred $ ($250 IIRC), and it can almost make 4fps. A more expensive card could keep up with one camera.

The C app can easily be a daemon. It's just a couple of lines of code.

It's a foreground now because I wanted to watch it.

[rockedge]

I almost successfully compiled the code but I ended up with the build stopping with this error. (GPU=0)

Code: Select all

obj/zmdn.o: In function `insert_detections':
zmdn.c:(.text+0x2bc): undefined reference to `mysql_query'
zmdn.c:(.text+0x2d0): undefined reference to `mysql_errno'
zmdn.c:(.text+0x2e7): undefined reference to `mysql_error'
zmdn.c:(.text+0x2f1): undefined reference to `mysql_errno'
obj/zmdn.o: In function `CreateTables':
zmdn.c:(.text+0xbe8): undefined reference to `mysql_query'
zmdn.c:(.text+0xbf6): undefined reference to `mysql_error'
obj/zmdn.o: In function `main':
zmdn.c:(.text.startup+0x200): undefined reference to `cfg_init'
zmdn.c:(.text.startup+0x228): undefined reference to `cfg_parse'
zmdn.c:(.text.startup+0x35d): undefined reference to `mysql_init'
zmdn.c:(.text.startup+0x3ca): undefined reference to `mysql_real_connect'
zmdn.c:(.text.startup+0x8b6): undefined reference to `mysql_query'
zmdn.c:(.text.startup+0x8ec): undefined reference to `mysql_query'
zmdn.c:(.text.startup+0x918): undefined reference to `mysql_store_result'
zmdn.c:(.text.startup+0x92e): undefined reference to `mysql_num_rows'
zmdn.c:(.text.startup+0x95a): undefined reference to `mysql_fetch_row'
zmdn.c:(.text.startup+0xd5d): undefined reference to `mysql_query'
zmdn.c:(.text.startup+0xe35): undefined reference to `mysql_free_result'
zmdn.c:(.text.startup+0xe60): undefined reference to `mysql_close'
zmdn.c:(.text.startup+0xe86): undefined reference to `mysql_close'
zmdn.c:(.text.startup+0xe8e): undefined reference to `mysql_error'
zmdn.c:(.text.startup+0xeda): undefined reference to `mysql_close'
zmdn.c:(.text.startup+0xee2): undefined reference to `mysql_error'
zmdn.c:(.text.startup+0xf2e): undefined reference to `mysql_error'
zmdn.c:(.text.startup+0xf5e): undefined reference to `mysql_close'
zmdn.c:(.text.startup+0xf76): undefined reference to `mysql_error'
collect2: error: ld returned 1 exit status
make: *** [zmdn] Error 1
root#

[tpcarmen]

You're almost there! You need the mysql libs.

Ubuntu/Debian: apt-get install libmysqlclient libmysqlclient-dev

Centos/Redhat/Fedora: yum install mysql-devel

[rockedge]

I am doing this on Puppy Linux Tahr 6.0.6 (Ubuntu 14.04 base) and I have libmysqlclient libmysqlclient-dev installed v5.5.55 but can not get past this error. Yet.