Autodelete events without objects detected to dramatically reduce event count

[russell_i_brown]

I've been using Asker's most excellent motion detection stuff for ZM (https://github.com/pliablepixels/zmeventnotification) to 'declutter' ZM with dramatic effects. Of course, Asker's zmeventnotification extension's prime purpose is to filter events for actual objects of interest before sending you a notification; I'm not sending notifications but leveraging it to sanitise events on my ZM server.

Here is the console line for the same camera running on two ZM systems - one with Object Detection & Autodelete and one without. Same ZM motion detection settings for both.





As you can see, the system running Object Detection & Autodelete has only saved 20 events for the past week whereas the normal ZM system has 864. The storage is only halved because I'm saving analysis images and the .mp4 on the object-detection server.

The 20 events on the Object Detection ZM all feature 'interesting' things (people) whereas the 'traditional' Motion Detection system is full of events triggered by birds, bats and bugs.

All this means that I can glance at my ZM console (or see the event counters on my Homeassistant main page) and look at events of interest with some reasonable confidence that they will actually be... interesting


Now... the motion detection isn't perfect (I still get the occasional event where it's mis-identified something or, less often, missed something that I would be interested in) but it's very good and, let's face it, you're never going to review 864 events but 20 in a week is much more managable. I should also state at this point that I'm also recording the hi-res feeds from all my cameras 24/7 so if the Motion Detection Autodelete did get rid of something I can refer to those (I've been doing this for 'normal' motion detection since 2018 - viewtopic.php?f=9&t=27537).

Another useful side effect is that the balance of motion detection sensitivity can be tipped firmly towards the very sensitive (which would normally drive you mad with zillions of events) and the Autodelete will simply get rid of events that are not 'interesting'. You will see events appear in the ZM console and then disappear a few seconds later but I can live with that.

Asker's got all the instructions needed to install his EventNotification/Motion Detection stuff at the link show at the top of this post. Once that's installed and working, simply download the attached script (zm_event_end_autodelete.sh.txt - the .txt extension is only there to pacify the forum gods, get rid of it once downloaded) and follow the simple instructions contained therein.

Other notes.

Asker's Object detection doesn't need a GPU (although having one helps as it does for zmc). What does impact the overall load of doing Motion Detection is not having to reload the Yolov3 model for each event (this can take a couple of seconds). Asker's written an API gateway (MLAPI) to a standalone Object Detection routine (https://github.com/pliablepixels/mlapi) and running this, even on the same server as ZM, means that it loads the Yolov3 model once when it fires up and thereafter 'just' runs the images against the model each time it's called (milliseconds not seconds). You don't need to run the mlapi gateway but it smooths things out loadwise.

I fire up the MLAPI routine with a simple script and supervisord entry:

/usr/local/bin/run_mlapi

Code: Select all

#!/bin/bash
cd /home/russell/mlapi
python3 api.py -c config.ini

/etc/supervisor/conf.d/mlapi.conf

Code: Select all

#
#       MLAPI
[program:MLAPI]
command=/usr/local/bin/run_mlapi
autostart=yes
autorestart=yes
startretries=100
stderr_logfile=/var/log/zm/mlapi.err.log
stdout_logfile=/var/log/zm/mlapi.log

I hope this is useful and huge kudos to Asker (and indeed Isaac and all the other ZM contributors).

[ejpb]

Hello.
I have used this script for a while, it's great! Thanks.

I have, like many other it seems, a low-res monitor with Modect and a linked HD monitor with Nodect, which works good. I added a delete section for the linked events also to the script. It really deletes the latest event from the linked monitor, I couldn't find any good way to find the exact event, if there are difference in the recording lengths between the two, etc (see below). Best to keep their buffer length settings synced, I guess.
I made a python adaptation of it, the main benefits are being able to have OPT_USE_AUTH enabled and you get the logs within the zoneminder ecosystem. I had to "go low" and use the "private" function _make_request and also do a modification in the return handling. Perhaps that change is something Asker can verify (does it mess up other uses?). It works for this, as I don't use the return value. I also don't use the notifications, so the script exit values perhaps need to be verified by someone using them.

One issue regarding pyzm:
I have seen some weird bits in the event filtering in pyzm. I had som extra debug logging with the five input arguments to the script. The filter 'e_filter' (line 126) has m_id set to the linked monitor but the returned event is from the lowres monitor... I can't see how that could be, so I'm at a loss here. It usually works as intended. I had 0 post-event frames on the lowres and 12 seconds post-event on the HD, so I had two events versus one long, but I can't see how that would affect this, results from wrong monitor...

Code: Select all

$ journalctl -g hook_del_events -o cat
INF [hook_del_events] [args: 63594 - 8 - Office - Motion: door - /var/lib/zoneminder/events/8/2020-07-23/63594]
INF [hook_del_events] [Delete linked event 63596 @ m_24 (event from filter reports: monitor 8)]
INF [hook_del_events] [Delete event 63594 @ 8-Office: "Motion: door"]
INF [hook_del_events] [args: 63596 - 8 - Office - Motion: door - /var/lib/zoneminder/events/8/2020-07-23/63596]
INF [hook_del_events] [Delete linked event 63596 @ m_24 (event from filter reports: monitor 8)]
INF [hook_del_events] [Delete event 63596 @ 8-Office -: "Motion: door"]

And as intended:

Code: Select all

INF [hook_del_events] [args: 63616 - 8 - Office - Motion: door - /var/lib/zoneminder/events/8/2020-07-23/63616]
INF [hook_del_events] [Delete linked event 63617 @ m_24 (event from filter reports: monitor 24)]
INF [hook_del_events] [Delete event 63616 @ 8-Office: "Motion: door"]

Hope it helps someone.

[asker]

pyzm is work in progress. If there are bugs, please open an issue in the pyzm github repo. If you have fixed it, please do a PR.
https://github.com/pliablepixels/pyzm

[ejpb]

I saw that pyzm had been updated to include Event.delete(), so I updated the python-script to use it. The old version still works with the new pyzm-version and may be more concise, as you can delete an event directly with the id in the _make_request call, whereas here you need to use a filter to get the event first, resulting in more calls, but that's no big deal.

As I had another look at it, I saw my mistake in the previous version, I had defined the filter's key as 'm_id' instead of 'mid', resulting in no monitor specified. You could just correct that if you'll keep using the old one. Sorry about that.

[timf]

EDIT

Seems it is actually working ok, just had to do a full power down / up rather than just stop and starting

##################

Resurecting an old thread which looks as though it could be useful.

I've installed this script and edited the username, password, groups and user where required

I've also edited the secrets.ini and objectconfig.ini as required but noticed in my objectconfg.ini the end_hook part was already commented out

When I create movement with no objects the event should be deleted but looking in the zmesdetectlog I see no evidence of it being called.

Is this script still ok for ZM1.36 or is there a better way to delete false positives ? Any specific logs I should be looking at to figure what's going on?

Regards Tim

[trumee]

The script is not working very well for me. The lowres videos are getting deleted correctly. But sometimes highres videos deletion throws up errors in zoneminder log.

I suspect the timing in the linked_event_filter is the problem,

Code: Select all

        linked_event_filter = {
            'mid'        : int(LINKED_MON[m_id]),
            'from'       : '2 minutes ago', #best value here depends on the buffer settings etc... This applies to every linked monitor's events
            'object_only': False,
            'max_events' : 1,
        }

How does one calculate the right amount of time e.g. '2 minutes'. My hires camera records at 15fps and has 5 frames in the buffer,



With this information what should be the time value?