neo-ZMES - Public Beta

[juan11perez]

Thank you. I do use mlapi precisely for the reason you mention. I get inference times of "Total remote detection detection took: 0:00:00.811376" with face recognition. I use a TPU and nvidia card.

for gotify I use the ZM TOKEN and I renew it every 3 hours. So I would use that.

[tsp84]

for gotify I use the ZM TOKEN and I renew it every 3 hours. So I would use that.

Have you gotten the clickable notification URL to work? No kidding I have spent the last day trying to get it to work. Better option I guess is to just have a link in the message of the body to click.

Im pushing last few things right away and then ill be testing install. I added a pyzm install script to mlapi repos, I added effeiceintdetv3 lite and mobilenet v2 TF2 models when the install scripts grab the tpu models.

The custom push script option I am adding will send the ZM_TOKEN along with the usual args to the bash script, so you wont have to patch together something to get a token yourself. Its configurable in the objectconfig file. The push_user and push_pass options will be used to grab a token if they are configured, if not, the ZM_TOKEN will be the ZM_USER token.

Code: Select all

# Arguments passed
# ARG1 = event ID
# ARG2 = monitor ID
# ARG3 = monitor name
# ARG4 = Alarm cause
# ARG5 = type of event (event_start or event_end)
# ARG6 is the AUTH token (the token starts with 'token=', so dont put token= before ${ZM_TOKEN})
# ARG7 is the path to the image (Optional)
# If you configured push_user and push_pass in objectconfig then the token will be for that user.
# If you did not configure push_user it will pass you the configured ZM_USER token instead.
# I reccomend making a ZM API user with VIEW privileges and using that for push_user and push_pass.

This allows it to be modified to send to whatever push notification service. I am really impressed with the speed of gotify notifications. The web app plays GIFS, the android app isn't so great but it gets the job done.

The performance I'm talking about was hardcoded into the logic of how zmes handled things and the metrics I'm going for are total time for detections (using most as the strategy for both frame and same model sequence strategies and with image creation/labeling, push and mqtt notifications).

Logging into the ZM API and grabbing an AUTH JWT took 0.8-1.2 seconds for me on average. Before, the script would sit and wait until the API returned the token and then it would go on about doing other things. Now, it creates the API and logs in, in the background and does as much as it can before waiting for the API to return some requested data it needs before it can do anything further. I am optimizing this system as we speak to make things even faster (which means testing all sorts of scenarios and ways to optimize them).

Animations are still slow, I threaded the function that creates them but I can only do so much due to needing to wait for the matching image to be identified.

Edit: With the gotify web app you can embed the events right into the notification!!! It does not work on the android app but the web app works great, web app also plays GIF and I imagine mp4 as well (browser dependant?)

Code: Select all

      \"message\": \"${MESSAGE^}\n![Embed the event right into the notification](${ZM_PORTAL}/cgi-bin/nph-zms?mode=jpeg&frame=1&replay=none&source=event&event=${EVENT_ID}&connkey=77493&${ZM_TOKEN})\n ![Camera Image](${ZM_PORTAL}/index.php?view=image&eid=${EVENT_ID}&fid=objdetect&popup=1&${ZM_TOKEN})\",

I have instead created a clickable link inside of the notification above the image. It is like the Pushover notification now! Pushover has the custom notification sounds though which I've come to really enjoy. I get 'motion detected in the front yard' when anything but a person was detected in the front yard. When a person is detected in the front yard I get a 'person detected in the front yard' sound along with my pushover notification. Super handy!

Screenshot_20211109-233934_Gotify.jpg (436.63 KiB) Viewed 91745 times

[juan11perez]

For gotify yes (clickable notification URL) works. but dont click on the expand arrow on the right. when you get the notification, touch the tittle section. For me it opens a webrowser with the zm url to the video/event and it just plays it.

Yes gotify is faster than any other. You're hosting yourself and there's no throttling. It has some limitations, but it more than suffices for this purpose.

I dont really understand the benefit of the animation, if I can click on the notification and see the event video. Maybe I'm missing something.

Performance upgrades sound very promising (although I understood half of what you said ).

So to install mlapi now i do ....clone.... pip3 install -r requirements.txt &&
yes | INSTALL_CORAL_EDGETPU=yes INSTALL_YOLOV3=yes INSTALL_YOLOV4=yes INSTALL_TINYYOLOV3=no INSTALL_TINYYOLOV4=yes ./install_pyzm.sh ?

But zmevent also installs the models?

[tsp84]

Zmevent is a seperate system that can run all the object detection itself. So when zmeventnotification installs, it downloads and installs models into /var/lib/zmeventnotification/models (or a different dir if you configure it in the install script)

Mlapi is a seperate system that can be installed on a host without ZM or zmeventnotification. So mlapi will download models into its own /models directory. If you don't want that behaviour you can edit the install scripts so all the model download variables say no or when the prompt asks you if you want to install the models answer with no.

If you have mlapi and zmes on same host you can just symlink one model directory to the other.

[juan11perez]

Thank you I understand.
In my current setup I have zmevent and mlapi in 1 docker container. I only downloaded the files once into /var/lib/zmeventnotification/models via the zmevents install script and either zmevent or mlapi use them.

So for neo I guess I would do this for zmeventnotifications
yes | INSTALL_CORAL_EDGETPU=no INSTALL_YOLOV3=no INSTALL_YOLOV4=no INSTALL_TINYYOLOV3=no INSTALL_TINYYOLOV4=no ./install.sh
and this for mlapi
yes | INSTALL_CORAL_EDGETPU=yes INSTALL_YOLOV3=yes INSTALL_YOLOV4=yes INSTALL_TINYYOLOV3=no INSTALL_TINYYOLOV4=yes ./install_pyzm.sh

ok I tried building and getting this error with install_pyzm

Code: Select all

./install_pyzm.sh: line 72: local: `=': not a valid identifier
  Skipping removal of pyzm

./install_pyzm.sh: line 72: local: `=': not a valid identifier
  Skipping neo pyzm installation


./install_pyzm.sh: line 72: local: `=': not a valid identifier
  Skipping model downloads

[tsp84]

Yes that is a viable solution. Have mlapi download the models into its models folder and then remove zmeventnotifications model folder and create a symbolic link to the mlapi models folder. Example ->

Code: Select all

# mlapi is in ~/mlapi
# zmes is /var/lib/zmeventnotification
# Only download models for mlapi

# BEFORE rm-rf MAKE SURE THAT THERE IS NOTHING in the models directory that you don't want deleted, there shouldn't be but....
# Dont be a Linutz Tech Tips...... lulz
sudo -u www-data rm -rf /var/lib/zmeventnotification/models
sudo -u www-data ln -s ~/mlapi/models /var/lib/zmeventnotification

# To unlink
sudo -u www-data unlink /var/lib/zmeventnotification/models

This way zmes and mlapi can share the same directory for models. I am push right away to all 3 repos so you should be able to download and use the new system. there may be some strange debugging messages in logs, just ignore as I am timing things and looking for data everywhere to optimize.

The 'custom push script' option has been added to the top of the objectconfig.yml file to configure. Review the gotify_zmes.sh file, its basically your script with a few tweaks and a wrapper to return 0 or 1. ZM_TOKEN is passed to the custom push script as an argument so you don't have to do any thing to retrieve a JWT. You can control which user the ZM_TOKEN is created for by changing 'psuh_user' and 'push_password' If you do not configure push_user then the ZM_TOKEN that is passed is the one that ZMES itself has been configured top use. This means it has all the same privs as ZMES, I recommend creating a new ZM API user with view privs of stream and event only for the 'push_user' as that users token is only used to create clickable URLS for notifications.

Everything should work, I admit I haven't tested a new install form scratch recently though, Im on a roll optimizing.

[juan11perez]

Thank you. I'm having issues with the installation scripts, so I'm trying to only install zmeventnotif but the script completes without installing anything.
It seems that there's a number of pieces that are "disconnected".
For instance neo repo doesn't have zmeventnotification.ini file but zmeventnotification.pl refers to it.
At this point I can't install zmevent

[tsp84]

Yeah I haven't installed from scratch and have my source configs on different disks. Using a new ide and for some reason it keeps doing strange things. Adding dependencies to the requirements file when I didn't configure it too and all sorts of goodies.

Ill stop optimizing and take a moment to make the install go good and create a different branch to optimize in. It's all new to me so the beginning will be tough. The gains are worth it though. Once it's installed it just works, I had been trying to test all the different setups to catch weird errors but optimizing took over once I started getting obsessed with performance. Give me roughly 2-3 hours and I'll have a properly working installable repo ready for all 3 components.


Zmeventnotification.pl does rely on zmeventnotification.ini as that's the config file for the perl script.

[tsp84]

Alright, I created a new virtualenv, deleted everything and installed from scratch. I fixed whatever errors I found along the way and got a detection using GPU without mlapi. I then ran mlapi and got a detection on the GPU so from what I can tell things are working. Check it out and let me know.

One big optimization is that an auth token is sent to mlapi. The password and user are sent as well but they don't really matter. This is so the ZM API is only logged into once, by zmes who keeps a copy of the token and encrypts it to send it to mlapi. This saves mlapi having to log in, it just grabs and decrypts the token and starts using it (this saves 0.8-1.2 seconds).

I highly reccomend having your storage method for monitors set to jpegs. It takes up more space but it is way faster for doing object detection. I was using the MP4 video passthrough writer for storage and I noticed delays while trying to grab frames for detection and grabbing frames to create animations. My detections with animation creation went from 20-40 seconds all the way down to 13-14 seconds..... That's massive!

[juan11perez]

Thank you. I installed and everything seems to be in its place, but i cant created the mlapi user.

Code: Select all

root@7576d4ab5890:/var/lib/zmeventnotification# python3 mlapi_dbuser.py -u mlapitest -p test123
Traceback (most recent call last):
  File "mlapi_dbuser.py", line 3, in <module>
    import pyzm.helpers.globals as g
  File "/usr/local/lib/python3.8/dist-packages/pyzm/helpers/globals.py", line 3, in <module>
    from yaml import safe_load
ModuleNotFoundError: No module named 'yaml'

Also in your zmevenotification installation scrip the gotify.sh is called different to the file name (line 380) in tools folder

[tsp84]

You need to sudo pip3 install pyyaml, that should of been in requirements (I'll add that today, I swear it was in the requirements though). The gotify.sh script is configurable in the objectconfig.yml file. The script can be named whatever you want, the main point is that whatever the script is named it gets sent the data needed to create notifications. It's up to the user to create a script that takes that data and sends it to a notification service. The gotify stuff is just an example of how to use the arguments that the script will receive.

Code: Select all

custom_push_script: '{{base_data_path}}/bin/gotify_zmes.sh'

If you mean that gotify_zmes.sh is in the tools dir, its there in the repo but when you run ./install.sh gotify_zmes.sh gets installed into the bin directory with zm_detect.py.

[juan11perez]

Thank you . I did. Still not working

Code: Select all

root@7576d4ab5890:/var/lib/zmeventnotification# sudo pip3 install pyyaml
Collecting pyyaml
  Downloading PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
     |████████████████████████████████| 701 kB 8.9 MB/s 
Installing collected packages: pyyaml
Successfully installed pyyaml-6.0
root@7576d4ab5890:/var/lib/zmeventnotification# python3 mlapi_dbuser.py -u mlapitest -p test123
Traceback (most recent call last):
  File "mlapi_dbuser.py", line 20, in <module>
    mlc, g = proc_conf(args, conf_globals=g, type_='mlapi')
  File "/usr/local/lib/python3.8/dist-packages/pyzm/helpers/new_yaml.py", line 727, in process_config
    config_obj.process_config()
  File "/usr/local/lib/python3.8/dist-packages/pyzm/helpers/new_yaml.py", line 178, in process_config
    self.config = self.parse_secrets(config=self.config, filename=self.secret_file_name)
  File "/usr/local/lib/python3.8/dist-packages/pyzm/helpers/new_yaml.py", line 317, in parse_secrets
    new_config = compile(pattern=pattern).sub(secrets_pool[sec], new_config)
  File "/usr/lib/python3.8/re.py", line 327, in _subx
    template = _compile_repl(template, pattern)
  File "/usr/lib/python3.8/re.py", line 318, in _compile_repl
    return sre_parse.parse_template(repl, pattern)
  File "/usr/lib/python3.8/sre_parse.py", line 972, in parse_template
    s = Tokenizer(source)
  File "/usr/lib/python3.8/sre_parse.py", line 228, in __init__
    string = str(string, 'latin1')
TypeError: decoding to str: need a bytes-like object, NoneType found
root@7576d4ab5890:/var/lib/zmeventnotification# 

[tsp84]

I ran into hat error last night.... Thought it was fixed, one moment I'll hop on and take a look.

[tsp84]

I cant get that error to reproduce on my end. Did you delete and re clone the repos and reinstall? Can you post more log output so I can see if there's any odd errors before this one. Some of the errors are just a one line message (in order to keep the detections going instead of failing outright).

Edit: I even purposely malformed some of the keys in the secrets and config file, my detections are still going through.

Code: Select all

❯ ./mlapi_dbuser.py -u test - p test
--------------- ML API DB User Creation ------------

user name (Ctrl+C to exit):

(I'll fix that its not just creating the user instead of prompting, but it seems to work for me?)

[juan11perez]

i'm trying to get zmevent to start, but it's giving this error

Code: Select all

Nov 12 19:16:27 2869521fd12c zmeventnotification[1219]: FAT [Encountered errors while reading /etc/zm/zm_secrets.yml:
Line 1 in file /etc/zm/zm_secrets.yml is malformed:

	---
Line 17 in file /etc/zm/zm_secrets.yml is malformed:

	ZM_USER: "juan"

Line 18 in file /etc/zm/zm_secrets.yml is malformed:

	ZM_PASSWORD: "password"