ZMES upgrades, GIF annotated for android pushover notifications, MQTT, etc.

[tsp84]

Just seeing if anyone would be interested? I tried doing it with the mp4 as well but for some reason the color palette is off for the bounding box frames no matter what color scheme i convert to using openCV, I'll keep working on that though.[completed]

There is Past/Live event logic and some things are optimized/skipped/forced based on if its a LIVE or PAST/DEBUG event, also takes into consideration if this is an event START or event END.

I've been busy adding all sorts of stuff to zmeventnotification, pyzm, mlapi. I have made a python mqtt add on (TLS,mTLS) for zmes to send the gif/jpg to HA by mqtt camera and am currently working on configurable topics with configurable data and attributes.
Added PushOver logic to the hooks scripts for better control. I also have a HA add on that integrates sensors to control sending PushOver notifications at all and also controls the cool down period between PushOver notifications, per monitor (no AppDaemon scripts, all ZMES side, just need to activate HA API and create a token). You could use a Bayesian sensor for presence detection (working on adding presence logic to pushover send), I currently use an input boolean for sending on/off and an input.number for cooldown time in seconds. I am also adding a counter so if you are home/away and only want to get notified x in a set amount of time, per monitor.

Currently working on a Zone python module to control zones to make it easy to have 'Night' or 'Day' zones switched based on time/sunset/sunrise. Before the method was to create duplicate monitors and setup the Day zones on 1 set , save as a run state and then same for Night on the other duplicate set. Then you needed to make API calls to switch run states........ I did this and made a bash script to manage auth and refresh tokens and also used sunwait to do all the sunrise/sunset stuff with date wrapping, While it does work, it was much clunkier as it restarts all streams, I noticed a few issues cropping from changing run states so decided to go this route.
My way will use Zones and switch them using API (not just Active/Inactive either), no more duplicate monitors that zmNinja has issues displaying the events list for. It will be expandable for not just Night/Day but certain times and you can make groups of Zones to control, utilizes the same config structure as zmes with !secrets and {{ template_replacement }}. It accepts strings like 'tomorrow at noon", "07/19/2021 14:15:05.123456". I haven't tested the human readability string input to the extreme but it's fairly good so far. You setup your LAT, LONG, timezone, city/region data in config file and all the time zone stuff is taken care of in background (DST aware).

I noticed Linked monitors stop triggering after awhile and the only cure is to restart the linked and linked from streams, so there is another add on for zmes to restart specified monitors every x sec/min/hour to avoid missing linked events or just let it run and it will automatically find the linked/linked from monitors and restart them every 3 hours.

I upgraded the animation logic for better error handling, it will drop the anchor frame by fps if certain conditions are met on failure to get the correct frame buffer. To completely annotate the gif would increase the notification time even more, so I only added timestamp (based on event API calls - So when ZM recorded them) and the first few frames of the gif are the objdet.jpg frame that is annotated (draw zone/polygon and filtered objects configurable).

Changed almost ALL requests to ZM API class calls (except for get started and backup calls; if API class isn't working correctly somehow). Did a deep dive on error handling and fallback from remote to local, added several new data points for correlation/graphing in future.

I am looking into feasibility of adding host/client ability to zmes so 1 zmes can be the central notification authority for several other zmes/ZM installs and they communicate amongst themselves, this isn't a small thing so I am just in the research part of it right now. I am heavily leaning towards MQTT (TCP and or WS - SSL/noSSL) or just websockets for protocol (SSL/noSSL) with auth. Also looking into feasibility of TOTP integration with some things (would also like to see ZM add this and am researching PHP and how ZM stores user data to add TOTP secret to user DB)

Includes alias and bash functions for debugging and easier control, If you pass it only event ID (even if you pass it a monitor ID it confirms it with an API call) it will immediately figure out the monitor ID and apply all masking options from config file. Syntax for some of the alias' and functions ->

• es.start.debug = start zmeventnotification.pl manually with --debug and set to background
  es.start = calls zmdc.pl to start
  es.restart = calls zmdc to restart
  es.stop = calls zmdc to stop
  es.status = calls zmdc to get status
  FUNC: es.objdet.debug <event ID - required> <monitorID - optional> = run zm_detect.sh on a PAST event for debugging or gif/jpeg recreation/force
  FUNC: es.debug.event_start <flags> = same as above but starts it off with the event_start.sh script

• mlapi.debug = start mlapi with --debug and --config in background
  mlapi.init.start/stop/restart/status = alias linked to mlapi user to control systemctl events
  mlapi.start = start mlapi manually in foreground
  mlapi.stop/restart = manual controls

Completely reworked the filtering logic and added several filtering options for tighter control of detection filtering; per object confidence filtering, per object AREA inside of zone/polygon threshold filtering, most filtering options are per object now instead of global. I am working on the logic for picking the best match, tighter control using sum of all confidences or mean average, etc.
Currently working on buffer for match_past_detections that is time aware, this is actually tricky for me, I have a basic buffer implemented but it needs more work.

Adding most of the ZM API calls to pyzm and classes to interface with/functions to call. Haven't figured out how the index.php?view=video option works yet, everytime ive tried it, it brings me to a video encoding page I've never seen to encode the video into .avi or .3gp or .xxx instead of feeding me a video. Then again I haven't learned much HTML so maybe this is some sort of HTML interface you call strictly from HTML? IDK.

zmMagik has also been updated to use pyzm API calls (meaning mp4 and jpg support) while still retaining ability to feed it urls or a file. It's still a work in progress though (converting ML from in-house to pyzm but also leaving in-house functionality available to call) and also trying to find additional template matching algo's for the find/search function (only added 1 extra so far with roughly same results as original function, honestly the better solution is to train a yolo model to recognize certain things in the frame and then setup zmes to alert on it missing/moved/interacted with).
I am adding the zmMagik functionality and calls into pyzm and thinking of ways to add the blend or annotation features into a daily task that won't affect performance or be intrusive. Also means adding an interface to call from a script to run batches of blend or annotations based on a config file.
The blend logic has been upgraded for tighter control over how blending 2 different monitors should work (if you have duplicate monitors and want to blend them together based on start time or event ID, or make a blended video of monitor x events for the first bit of video and then move onto the next monitor for rest of video). Added support for more then 1 mask.

Added an option to store darknet 'training' images and also 'compare' frames. The training image is the base/raw frame that the object was detected in, no bounding boxes,zones/polygons,timestamp or annotation that you can label to train/test a model. The 'compare' image is a debug frame to compare with that has all the bounding boxes. You have the option to remove/add zone/polygon outline all together and error box's (filtered out objects) outlined in red.

The mqtt and HA add on can be extended for all sorts of other things if needed.

Adding ability to mlapi to reload config file/ feed it a config file via API and also to preload 1st sequence into memory on start (means being able to receive data from ZM host with info about its ML-overrides and sequences)[completed], adding Error handling to better catch weird Errors -> OverFlowError: cannot convert float * infinite to int() - is a weird one i've noticed using yolo and gpu, sometimes it takes 10 days of detections, sometimes its 2 hours before it starts hitting that error and then it needs a restart. Also the USB tpu cable error, I haven't experienced it yet so I haven't been able to setup error catching for that. Basically it catches those weird errors logs them and restarts mlapi, then it has logic for notifications based on severity/occurrence of errors.

I haven't published anything yet, just seeing if there's any outside interest at all? Ill keep this going for my own uses anyways but maybe someone would like some of these add ons. Next big add on is a better HA ZM integration then HA currently uses. I would like to have better history/state data to correlate and graph better then just current events or events in last hour/day/week.

Also hoping to add more data points to ZMES/HA for something to read like grafana or maybe make mqtt sensors in HA vor visualization, but I haven't moved onto actively setting things up for that yet. Would be nice to see data on hits/misses between tpu/yolo detections and correlate it, among several other data points. HA could send daily stats via email/push/whatever. Options are endless!

I'll start working on python 2.7 compatibility and get it somewhat ready for testing if anyone's interested. I will post repos when available and see if anything is PR worthy.

Code: Select all

07/17/21 19:52:03.739366 zmesdetect_m1[1156656] INF zm_detect:661 [zm_detect:prediction: '[70] person(75%),']
07/17/21 19:52:03.740479 zmesdetect_m1[1156656] DBG1 zm_detect:664 [zm_detect:prediction:JSON: {"labels": ["person", "person"], "boxes": [[277, 98, 317, 195], [430, 181, 479, 277]], "frame_id": "70", "confidences": [0.75390625, 0.71484375], "image_dimensions": {"original": [1080, 1920], "resized": [450, 800]}}]
07/17/21 19:52:03.741415 zmesdetect_m1[1156656] DBG2 api:303 [pyzm:api:make_req: 'get'-><hidden>/api/events/17685.json query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciO...'}]
07/17/21 19:52:04.264553 zmesdetect_m1[1156656] DBG1 zm_detect:697 [zm_detect: old_notes='Linked: Front - Sub' new_notes='[70] detected:person(75%),  Linked:Front - Sub' notes_zone=' Front - Sub' old_notes_split=['Linked', ' Front - Sub'] notes_cause='Linked' g.config.get('api_cause') = 'Linked']
07/17/21 19:52:04.27475 zmesdetect_m1[1156656] DBG1 zm_detect:752 [zm_detect: writing objects.json and objdetect.jpg to /nvr/1/2021-07-17/17685]
07/17/21 19:52:04.315504 zmesdetect_m1[1156656] DBG1 zm_detect:828 [zm_detect:pushover:hass: input_boolean.front_pushover state is {'entity_id': 'input_boolean.front_pushover', 'state': 'on', 'attributes': {'editable': True, 'friendly_name': 'Front Pushover', 'icon': 'mdi:motion'}, 'last_changed': '2021-07-18T01:15:41.314310+00:00', 'last_updated': '2021-07-18T01:15:41.314310+00:00', 'context': {'id': 'f6020a5ad0aa8c9ef3280478c7d05972', 'parent_id': None, 'user_id': None}}]
07/17/21 19:52:04.341597 zmesdetect_m1[1156656] DBG1 zm_detect:842 [zm_detect:pushover:hass: input_number.pushover_cooldown_front state is {'entity_id': 'input_number.pushover_cooldown_front', 'state': '1.0', 'attributes': {'initial': None, 'editable': True, 'min': 0.0, 'max': 3600.0, 'step': 1.0, 'mode': 'box', 'unit_of_measurement': 'seconds', 'friendly_name': 'pushover_cooldown_front', 'icon': 'mdi:clock'}, 'last_changed': '2021-07-18T01:15:41.311723+00:00', 'last_updated': '2021-07-18T01:15:41.311723+00:00', 'context': {'id': 'b993d676124eaa4d72365b103a4a6819', 'parent_id': None, 'user_id': None}}]
07/17/21 19:52:04.342629 zmesdetect_m1[1156656] DBG2 pyzm_utils:794 [push:pkl:trying to load /var/lib/zmeventnotification/push/mon-1-pushover.pkl]
07/17/21 19:52:04.343596 zmesdetect_m1[1156656] DBG2 zm_detect:880 [going to save model training and compare images to: /nas/images/zm-ml/1/17685-training-frame-70.jpg /nas/images/zm-ml/1/17685-compare-frame-70.jpg]
07/17/21 19:52:04.359816 zmesdetect_m1[1156656] DBG1 zm_detect:898 [zm_detect: replacing old note: 'Linked: Front - Sub' with new note: '[70] detected:person(75%),  Linked:Front - Sub']
07/17/21 19:52:04.360785 zmesdetect_m1[1156656] DBG2 api:303 [pyzm:api:make_req: 'put'-><hidden>/api/events/17685.json payload={'Event[Notes]': '[70] detected:person(75%),  Linked:Front - Sub'} query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciO...'}]
07/17/21 19:52:04.392479 zmesdetect_m1[1156656] DBG1 zm_detect:910 [zm_detect:animation: gathering data...]
07/17/21 19:52:04.41378 zmesdetect_m1[1156656] DBG1 pyzm_utils:128 [r_frame_len=213 | fid+fps*buffer_seconds=120 | fid=70 | fps=10 | buffer_seconds=5 | total_time=21 | target_fps=2 ]
07/17/21 19:52:04.414769 zmesdetect_m1[1156656] DBG1 pyzm_utils:171 [animation: 213 frames to work with->anchor_frame=70 start_frame=20 end_frame=120 skip_frames=5 fps=10]
07/17/21 19:52:04.415696 zmesdetect_m1[1156656] DBG1 pyzm_utils:177 [animation: grabbing frames.....]
07/17/21 19:52:13.973416 zmesdetect_m1[1156656] DBG2 pyzm_utils:214 [animation: grabbed frames: [20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100, 105, 110, 115, 120]]
07/17/21 19:52:13.976537 zmesdetect_m1[1156656] DBG1 pyzm_utils:238 [animation: fast GIF requested....]
07/17/21 19:52:13.977841 zmesdetect_m1[1156656] DBG1 pyzm_utils:258 [gif_buffer_seconds=3.0 | target_fps=4 | gif_start_frame=40 | gif_end_frame=100 | slicing from s1=4 | negative s2=4]
07/17/21 19:52:13.978994 zmesdetect_m1[1156656] DBG1 pyzm_utils:260 [animation:gif: fast gif slicing 4 to -4 from a total of 21]
07/17/21 19:52:18.633982 zmesdetect_m1[1156656] DBG1 pyzm_utils:262 [animation:gif: optimizing...]
07/17/21 19:52:18.930596 zmesdetect_m1[1156656] DBG1 pyzm_utils:265 [animation:gif: saved to /nvr/1/2021-07-17/17685/objdetect.gif, size: 1.37MB, frames: 15]
07/17/21 19:52:18.931662 zmesdetect_m1[1156656] DBG1 zm_detect:935 [zm_detect:pushover:gif: data={'token': '<hidden>', 'user': '<hidden>', 'title': '(17685) Front:Linked->Front - Sub', 'message': '[70] detected:person(75%),  at 07:52 PM, Jul-17', 'url': '<hidden>/zm/cgi-bin/nph-zms?mode=jpeg&scale=50maxfps=5&buffer=1000&replay=single&monitor=1&event=17685&connkey=1288728048174780&token=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJab25lTWluZGVyIiwiaWF0IjoxNjI2NTczMTE1LCJleHAiOjE2MjY1ODAzMTUsInVXIiOiJ6bWVzIiwidHlwZSI6ImFjY2VzcyJ9.hIWXdh-L0_KgAX-FZA-HLQzQcOeEYTE01Q7LR3WA3ug', 'url_title': 'View event in browser'} files={'attachment': ('1-objdet.gif', <_io.BufferedReader name='/nvr/1/2021-07-17/17685/objdetect.gif'>, 'image/jpeg')} ]
07/17/21 19:52:22.601827 zmesdetect_m1[1156656] DBG4 pyzm_utils:816 [push:pkl:time since sent:2021-07-17 19:52:22.600034 to /var/lib/zmeventnotification/push/mon-1-pushover.pkl]
07/17/21 19:52:22.618846 zmesdetect_m1[1156656] DBG1 zm_detect:962 [zm_detect:mqtt: enabled and initilising...]
07/17/21 19:52:22.634508 zmesdetect_m1[1156656] DBG1 mqtt:167 [mqtt:connect: <hidden>:8883 trying mTLS -> tls_ca: /nas/mqtt_certs/ca.crt tls_client_key: /nas/mqtt_certs/client-zm.key tls_client_cert: /nas/mqtt_certs/client-zm.crt]
07/17/21 19:52:22.637064 zmesdetect_m1[1156656] DBG2 mqtt:203 [mqtt:connect: waiting for broker to reply (timeout: 2]
07/17/21 19:52:23.232285 zmesdetect_m1[1156656] DBG1 mqtt:54 [mqtt:paho_log: Sending CONNECT (u1, p1, wr0, wq0, wf0, c1, k60) client_id=zmes-mTLS-WPyDvQ0lJw8LwCvi']
07/17/21 19:52:23.718461 zmesdetect_m1[1156656] DBG1 mqtt:54 [mqtt:paho_log: Received CONNACK (0, 0)]
07/17/21 19:52:24.136076 zmesdetect_m1[1156656] DBG1 mqtt:58 [mqtt:connect: connected to broker with flags-> {'session present': 0} result code-> 0]
07/17/21 19:52:24.142466 zmesdetect_m1[1156656] DBG1 mqtt:128 [mqtt:grab_image: gif to be used is: /nvr/1/2021-07-17/17685/objdetect.gif, converting to byte array]
07/17/21 19:52:24.146044 zmesdetect_m1[1156656] DBG2 mqtt:232 [mqtt:publish: sending -> topic: zmes/picture/1  data: <serialized bytearray>  size=1.37 MB]
07/17/21 19:52:24.147679 zmesdetect_m1[1156656] DBG1 mqtt:54 [mqtt:paho_log: Sending PUBLISH (d0, q0, r0, m1), 'b'zmes/picture/1'', ... (1439998 bytes)]
07/17/21 19:52:24.150125 zmesdetect_m1[1156656] DBG2 mqtt:230 [mqtt:publish: sending -> topic: 'zmes/data/1' data: '{"mid": "1", "name": "Front", "reason": "[70] detected:person(75%),", "eid": "17685", "zone": "Front - Sub", "cause": "Linked"}']
07/17/21 19:52:24.151541 zmesdetect_m1[1156656] DBG1 mqtt:54 [mqtt:paho_log: Sending PUBLISH (d0, q0, r0, m2), 'b'zmes/data/1'', ... (127 bytes)]
07/17/21 19:52:24.152438 zmesdetect_m1[1156656] DBG2 mqtt:230 [mqtt:publish: sending -> topic: 'zmes/detection/1' data: '{"labels": ["person", "person"], "boxes": [[277, 98, 317, 195], [430, 181, 479, 277]], "frame_id": "70", "confidences": [0.75390625, 0.71484375], "image_dimensions": {"original": [1080, 1920], "resized": [450, 800]}}']
07/17/21 19:52:24.154036 zmesdetect_m1[1156656] DBG1 mqtt:54 [mqtt:paho_log: Sending PUBLISH (d0, q0, r0, m3), 'b'zmes/detection/1'', ... (216 bytes)]
07/17/21 19:52:24.155096 zmesdetect_m1[1156656] DBG2 mqtt:244 [mqtt:close: disconnecting from mqtt broker: <hidden>:8883]
07/17/21 19:52:24.156431 zmesdetect_m1[1156656] DBG1 mqtt:54 [mqtt:paho_log: Sending DISCONNECT]
07/17/21 19:52:24.157586 zmesdetect_m1[1156656] DBG2 zm_detect:988 [perf: 1-->17685: [total:27.534651 sec] [detection:7.081833 sec] [processing objdetect.jpg and .gif:20.452842 sec] [mqtt:1.554616 sec]]
07/17/21 19:52:24.159012 zmesdetect_m1[1156656] DBG1 zm_detect:999 [zm_detect: closing logs]

[SteveGilvarry]

Nice.
I am sure Asker/PliablePixels, who is still supporting at the moment will be happy to accept PR's to the repos that we just moved under ZoneMinder organisation in Github. And we are always happy to welcome people willing to develop new features, clean up existing or just generally help out.
It is safe to assume that if you found it worthwhile to develop others will also value it.

[tsp84]

I think in the future I would also like to add remote backup to mlapi and ability to smartly queue events amongst different processors based on queue/load. This is only an idea at the moment though, nothing more. I haven't experienced the TPU overload i've seen mentioned places online but I think that also has a lot to do with zone tuning if you don't have a large amount of monitors creating events. If you do have a lot of monitors making events IDK how mlapi/zmes responds and utilizes its Semaphore locks and how the memory/model is passed around as I haven't set up an environment to test like that yet but it sure sounds fun.

[yannlieb]

This is going too deep technical for me to understand everything, but it sounds promising. Thanks for taking the time to detail your plans!

[tsp84]

The linked script is basically ready, there is more polishing to be done but at the moment it works.

Its python3, and needs pyzm to work at the moment, also only logs to console (I will add logging and fall-back not using pyzm soon). Currently only automatically finds linked to and linked from monitors but am planning to also let you over ride that if you want in the config file (idk if this is really needed though). Config file is just path to secrets file, portal/portal_api, user, password, allow self signed cert options at the moment.

Script needs to be setup to run as a cron job every x mins/hours/etc.

So, example being, you have a monitor with id: 2 set to Mocord and a monitor id:1 set to Nodect. Both of these monitors need to be 'enabled'. Monitor 1 is the 'linked to' and 2 is the 'linked from'.
The script checks both of those monitors current function and saves it, sets the monitors stream to None waits 5 seconds and then sets the monitors back to whatever function they were before setting to None.

Code: Select all

07/26/21 13:19:36.974676 CNSL[DBG1] linked_reset:34->[util: secret filename: ./secrets.ini ]
07/26/21 13:19:36.975688 CNSL[DBG2] linked_reset:69->[util: secret token found in config: !ZMUSER ]
07/26/21 13:19:36.976247 CNSL[DBG2] linked_reset:69->[util: secret token found in config: !ZMPASS ]
07/26/21 13:19:36.976766 CNSL[DBG2] linked_reset:69->[util: secret token found in config: !PORTAL ]
07/26/21 13:19:36.977291 CNSL[DBG2] linked_reset:69->[util: secret token found in config: !API_PORTAL ]
07/26/21 13:19:36.977955 CNSL[DBG2] linked_reset:94->[util: parameter substitution ({{ <var> }}) ]
07/26/21 13:19:36.979292 CNSL[DBG2] api:83->[pyzm:api: SSL certificate verification disabled (encryption still works, vuln to spoofing) ]
07/26/21 13:19:36.980008 CNSL[DBG1] api:199->[pyzm:api: no token found, using user/pass to login ]
07/26/21 13:19:37.133449 CNSL[DBG2] api:228->[pyzm:api: detected API ver 2.0+, using token system ]
07/26/21 13:19:37.134183 CNSL[DBG1] api:236->[pyzm:api: access token expires on: 2021-07-26 19:19:37.133546 (21600s) ]
07/26/21 13:19:37.134847 CNSL[DBG1] api:243->[pyzm:api: refresh token expires on: 2021-07-27 13:19:37.134228 (86400s) ]
07/26/21 13:19:37.135726 CNSL[DBG2] Monitors:22->[Retrieving monitors via API ]
07/26/21 13:19:37.13656 CNSL[DBG2] api:303->[pyzm:api:make_req: 'get'->https://<hidden>/zm/api/monitors.json query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciO...'} ]
mid: 1 name: Front function: Nodect linked monitors: 2 enabled:True
mid: 2 name: Front - Sub function: Mocord linked monitors: None enabled:True
mid: 3 name: Back Alley function: Modect linked monitors: None enabled:True
mid: 4 name: Back Yard function: Monitor linked monitors: None enabled:True
inside linked_from loop x=2 mon_ = <pyzm.helpers.Monitor.Monitor object at 0x7fe58c7b79d0>
has_linked = [1]  ----  linked_from = [2] ---- tot_mons = [2, 1] --- tot = {1: 'Nodect', 2: 'Mocord'}
07/26/21 13:19:37.156598 CNSL[DBG2] api:303->[pyzm:api:make_req: 'post'->https://<hidden>/zm/api/monitors/1.json payload={'Monitor[Function]': 'None'} query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciO...'} ]
07/26/21 13:19:37.413014 CNSL[DBG2] api:303->[pyzm:api:make_req: 'post'->https://<hidden>/zm/api/monitors/2.json payload={'Monitor[Function]': 'None'} query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciO...'} ]
sleeping for 5 seconds and then returning monitors to previous function
07/26/21 13:19:47.725932 CNSL[DBG2] api:303->[pyzm:api:make_req: 'post'->https://<hidden>/zm/api/monitors/1.json payload={'Monitor[Function]': 'Nodect'} query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciO...'} ]
07/26/21 13:19:48.186538 CNSL[DBG2] api:303->[pyzm:api:make_req: 'post'->https://<hidden>/zm/api/monitors/2.json payload={'Monitor[Function]': 'Mocord'} query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciO...'} ]

Process finished with exit code 0

[tsp84]

The zone switcher script (module if daemonized) is almost done as well I just don't have a use case for it at the moment So i can't test it out thoroughly. I am unsure if it should be a standalone script for a cron job or to daemonize it and let it compare times every x seconds/minutes.

Most of the other stuff is ready as well i'm just figuring out how to roll it all out and get it installed. It is very different from the core code and I added and moved a lot of things into pyzm. So it would be a forked pyzm install, forked mlapi and forkled zmeventnotification needed to run in a venv or however you set up your python env. For now its going to be a python3 and zm 1.34+ minimum and there may be a cpl more dependencies to download.

I have added a python based USB port resetting tool for the USB TPU delegate error but haven't been able to test it out or automate a USB port reset when its actually needed yet.

Some log output ->

Code: Select all

07/26/21 13:51:28.377207 CNSL[DBG1] zsone:38->[util: secret filename: ./secrets.ini ]
07/26/21 13:51:28.378226 CNSL[DBG2] zsone:73->[util: secret token found in config: !ZMUSER ]
07/26/21 13:51:28.378775 CNSL[DBG2] zsone:73->[util: secret token found in config: !ZMPASS ]
07/26/21 13:51:28.379298 CNSL[DBG2] zsone:73->[util: secret token found in config: !PORTAL ]
07/26/21 13:51:28.379818 CNSL[DBG2] zsone:73->[util: secret token found in config: !API_PORTAL ]
07/26/21 13:51:28.380369 CNSL[DBG2] zsone:73->[util: secret token found in config: !LAT ]
07/26/21 13:51:28.38089 CNSL[DBG2] zsone:73->[util: secret token found in config: !LONG ]
07/26/21 13:51:28.381842 CNSL[DBG2] zsone:98->[util: parameter substitution ({{ <var> }}) ]
07/26/21 13:51:28.383026 CNSL[DBG2] api:83->[pyzm:api: SSL certificate verification disabled (encryption still works, vuln to spoofing) ]
07/26/21 13:51:28.383715 CNSL[DBG1] api:199->[pyzm:api: no token found, using user/pass to login ]
07/26/21 13:51:28.54551 CNSL[DBG2] api:228->[pyzm:api: detected API ver 2.0+, using token system ]
07/26/21 13:51:28.546252 CNSL[DBG1] api:236->[pyzm:api: access token expires on: 2021-07-26 19:51:28.545611 (21600s) ]
07/26/21 13:51:28.546924 CNSL[DBG1] api:243->[pyzm:api: refresh token expires on: 2021-07-27 13:51:28.546298 (86400s) ]
07/26/21 13:51:28.547953 CNSL[DBG2] Zones:48->[Retrieving zones via API ]
07/26/21 13:51:28.548806 CNSL[DBG2] api:303->[pyzm:api:make_req: 'get'->https://<hidden>/zm/api/zones/index.json query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciO...'} ]

Information for <CITY>/<REGION>
Timezone: America/<TZ NAME>
Latitude: xx.xx; Longitude: -xxx.xx

just after midnight -> 2021-07-26 00:00:00.000001-06:00
now -> 2021-07-26 13:51:28.599612-06:00
sunrise -> 2021-07-26 05:40:45.884823-06:00
noon -> 2021-07-26 13:40:30-06:00
sunset -> 2021-07-26 21:39:12.639746-06:00
just before midnight -> 2021-07-26 23:59:59.999999-06:00
midnight -> 2021-07-26 00:00:00-06:00

currently between noon and sunset

trigger = {'trigger_1': {'name': 'Shift Change (Closing) - 25 mins before', 'offset': '25', 'time': 'today at 10:30PM', 'zones': {'1': {'name': 'cash room door', 'type': 'Active'}, '2': {'name': 'cash room door window', 'type': 'Active'}, '3': {'name': 'cash room door handle', 'type': 'Active'}, '4': {'name': 'staff room', 'type': 'Privacy'}, '5': {'name': 'staff room', 'type': 'Privacy'}}}}

converting: trigger_1:time 'today at 10:30PM' -> 2021-07-26 22:30:00-06:00
offset for trigger_1 -> 25
offset applied! 2021-07-26 22:05:00-06:00
currently before trigger_1:time

trigger = {'trigger_2': {'name': 'Shift Change (Opening) - 30 mins After', 'offset': '-30', 'time': 'today at 10:30PM', 'zones': {'1': {'name': 'staff room', 'type': 'Active'}, '2': {'name': 'staff room', 'type': 'Active'}}}}

converting: trigger_2:time 'today at 10:30PM' -> 2021-07-26 22:30:00-06:00
offset for trigger_2 -> -30
offset applied! 2021-07-26 23:00:00-06:00
currently before trigger_2:time

trigger = {'trigger_3': {'name': 'Management Meeting', 'time': 'today at noon', 'zones': {'1': {'name': 'walkway', 'type': 'Active'}, '2': {'name': 'cash room door window', 'type': 'Inactive'}, '3': {'name': 'cash room door handle', 'type': 'Inactive'}}}}

converting: trigger_3:time 'today at noon' -> 2021-07-26 12:00:00-06:00
currently after trigger_3:time, TRIGGERING!

match has been found!
No need to switch Zone 'type', already set as specified -> Active

Process finished with exit code 0

[tsp84]

Repos are ready to go, test away.

The current zmes_hook_helpers and pyzm from python library will be removed during new installs or start a new virtual env for this setup. This new ZMES requires my forked version of pyzm, I also updated mlapi. I updated config files as well so take a look in there to see the new options to configure.

Order to install:

optional: remove pyzm and zmes_hook_helpers
clone pyzm and then run ./install, it will ask you to uninstall old pyzm first (do not install pyzm using pip or it will grab the original)
clone zmeventnotification and run install, it will ask you to remove zmes_hook_helpers
clone mlapi, install like original


ZMES:
https://github.com/baudneo/zmeventnotification

PYZM:
https://github.com/baudneo/pyzm

MLAPI:
https://github.com/baudneo/mlapi

Some things to note are:
I provided some shell aliases in TERM_helpers.txt and a bash function called es.debug.objdet that I put somewhere that is in the $PATH env variable.
es.debug.objdet <EventID> is all that is needed the script will figure out the Monitor ID and apply per monitor settings with that.

use_sequence is deprecated and ml_sequence / stream_sequence are mandatory.
It will always write objdetect.jpg/.gif/.mp4 to the event storage folder even if turned off in objconfig.ini
debug images is removed and delete after analyzed, instead i've added save_training_image to save the raw frame that the object was detected in and also a 'compare' frame that is basically the objdetect.jpg frame
It assumes you will always pass it the 'cause' or 'notes' from api if available otherwise it assumes this is a PAST event (can remove that logic, es.debug.objdet passes a parameter to let the script know its processing a PAST event.
The pushover notification title always has the Event ID in it.
Unexpected things may happen, let me know so I can fix the issue.
Requests for features accepted.

[tsp84]

Noteable upgrades are

• gif/mp4 -> timestamp and first few frames are annotated, this means the thumbnail for pushover is the same as objdetect.jpg then it plays the animation!

• mlapi-> preloading weights on first run (mainly helps TPU)

• mlapi -> fallback to remote, you can specify another mlapi host to fallback to if errors happen and it will try that before falling back to local (if configured)

• Past/Live event aware to optimize and/or skip some things

• MQTT publish objdet.jpg or objdet.gif to topic -> zmes/picture/<monitor ID> - MQTT camera in Home Assistant works great

• MQTT publish detection data to -> zmes/data/<monitor ID>

• Added pushover into main python script to make the jpeg pushover notification the fastest option of all (see objconfig.ini for breakdown of sending JPG and GIF to same/different pushover apps

• Pushover notification has a clickable url that lets you view the event in your browser, very quick and convenient.

• Home Assistant 'Helper' input_boolean sensor control of pushover notification sending

• Home Assistant 'Helper' input_number sensor control for 'cool down' period between sending pushover notifications

• Updated almost all web requests to internal pyzm calls

• Convert snapshot to fid has improved logic. Also the [s], [a], [x] indicator in notifications has changed, [s] and [a] are the same but instead of [x] it displays the frame ID so if frame 145 is the frame that is returned by detections then the indicator is [145]. convert_snap_to_fid also changes this, if it was a snapshot frame that was converted to a frame ID 101 it will now say [s101], meaning snapshot converted to frame ID 101, [s] only if convert snapshot to fid is off

This is my own personal blend that I use and alter daily, I won't push much once its stable until new add ons are working decently.

Here is a screenshot of HA for the general idea on the pushover sensors and also the MQTT camera entities. They accept JPG and GIF.

Screenshot from 2021-07-29 19-48-55.png (966.87 KiB) Viewed 37094 times

[juan11perez]

First thank you very much. I want to try it on a new installation.
To confirm: I just clone your zmes and pyzm repos and run install.sh?

[tsp84]

Clone pyzm first, cd into pyzm dir and run install.sh

Then clone zmeventnotification cd into its dir and then run install.sh

They have their own separate install files. Mlapi you install with original instructions if you want to install it at all.

[juan11perez]

Noted.
Thank you for the clarification

[tsp84]

Let me know if there are any issues and I'll fix them ASAP. I tested by spinning up a new VM and cloning then installing the repos in there but I could only test mlapi in the new environment so there might be some errors for zmeventnotification. I didnt want to install a whole new instance of ZM to install zmeventnotification but it all *should* work.

[alabamatoy]

Be gentle on the newb, what does "clone" mean? I know how to install from a repo, is that same as cloning it?

Do you expect this to work on Ubuntu?

[tsp84]

Yes, it will work on ubuntu or any properly configured linux install. I tested it by running an ubuntu 20.04 LXC, installed git and virtualenv. created a venv for this install, cloned pyzm repo and ran ./install.sh, same for zmeventnotification and mlapi. I didn't want to install a whole new ZM instance so I setup mlapi as a new remote host, sent it a detection and everything worked as expected. So i'm assuming everything should work fairly well, there will be errors because I didn't account for all the different setups out there but I will work them out as they happen.

Install git by

Code: Select all

sudo apt install git

Code: Select all

git clone https://github.com/baudneo/pyzm.git

It will make a pyzm dir for pyzm

Code: Select all

cd pyzm

and run

Code: Select all

./install.sh

using either sudo (for system wide install), your user if a local install or activate a venv and install into that. pyzm is a python module so it gets installed using pip but from a local package (the cloned pyzm dir)

Same method for zmeventnotification but the dir is zmeventnotification

Code: Select all

git clone https://github.com/baudneo/zmeventnotification.git

Code: Select all

cd zmeventnotification

You want to use sudo for zmeventnotification, the script will install using your www-data or whatever webuser your system uses for apache.

Code: Select all

sudo ./install.sh

zmeventnotification is not a python module so the script handles installing things where they need to be with the proper permissions. The only thing I would add is the CRON job for linked_reset.py if you have any linked monitor, linked_reset.py is in /etc/zm

[juan11perez]

Evening. Trying to build an mlapi docker and getting this error:

Code: Select all

  File "/var/lib/zmeventnotification/modules/mlapi_utils.py", line 2, in <module>

    from future import standard_library
    
ModuleNotFoundError: No module named 'future'

Does future need to be added to requirements.txt? thank you.