ZoneMinder Forums

ZoneMinder Forums
https://forums.zoneminder.com/

neo-ZMES - Public Beta

https://forums.zoneminder.com/viewtopic.php?t=31218

Page 5 of 8

Re: neo-ZMES - Public Beta

Posted: Wed Nov 17, 2021 1:54 am
by tsp84
The problem is that when it goes to parse the JSON there seems to still be a byte encoded string. Which shouldn't be happening because I decode and decrypt the byte stream strings into normal strings.

There's the debug branch that will print out the keys that it is complaining about and should hopefully give some insight. If you want I can post the 2 changed files instead so you don't have to reinstall.

I would be posting mlapi.py and zm-detect.py. which means all you do is delete those 2 files wherever they are installed and replace them with the 2 provided files.

Until I see what those print statements are saying I can't really debug further.

Re: neo-ZMES - Public Beta

Posted: Wed Nov 17, 2021 2:46 pm
by juan11perez
Good day. Thank you for looking into this. I replaced the below mentioned files from the respective DEBUG branches and run test

Code: Select all

root@e4221043b14e:/var/lib/zmeventnotification# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 76
encrypting kickstart key 'access_token'
encrypting kickstart key 'refresh_token'
encrypting kickstart key 'access_token_expires'
encrypting kickstart key 'refresh_token_expires'
encrypting kickstart key 'api_version'
encrypting kickstart key 'zm_version'
encrypting kickstart key 'timezone'

11/17/21 19:15:02.534926 zmesdetect_m6[6629] DBG1 new_yaml.py:242->[config:init: the supplied config file exists -> '/etc/zm/objectconfig.yml']

11/17/21 19:15:02.615138 zmesdetect_m6[6629] DBG1 new_yaml.py:265->[config::init: default configuration built (no secrets or substitution vars replaced, yet!)]

11/17/21 19:15:02.616008 zmesdetect_m6[6629] DBG1 new_yaml.py:170->[config:zmes:proc: 39 built in default keys added to the 'base' config -> ['force_mpd', 'same_model_high_conf', 'skip_mons', 'force_live', 'sanitize_str', 'version', 'basic_user', 'basic_password', 'image_path', 'max_detection_size', 'delete_after_analyze', 'write_debug_image', 'write_image_to_zm', 'hass_enabled', 'hass_people', 'hass_notify', 'hass_cooldown', 'push_err_device', 'push_jpg', 'push_jpg_key', 'push_gif', 'push_gif_key', 'push_debug_device', 'push_cooldown', 'mqtt_topic', 'mqtt_broker', 'mqtt_port', 'tls_cert', 'tls_key', 'fp16_target', 'alpr_api_type', 'alpr_url', 'openalpr_recognize_vehicle', 'openalpr_country', 'openalpr_state', 'openalpr_min_confidence', 'smart_fs_thresh', 'frame_strategy', 'same_model_sequence_strategy']]

11/17/21 19:15:02.616313 zmesdetect_m6[6629] DBG1 new_yaml.py:114->[config:zmes:proc: substituting '{{variables}}' for the 'base' config keys]

11/17/21 19:15:02.617213 zmesdetect_m6[6629] DBG1 new_yaml.py:136->[config:zmes: successfully replaced 3 default sub vars in the base config -> ['coral_models', 'yolo4_models', 'base_data_path']]

11/17/21 19:15:02.617581 zmesdetect_m6[6629] DBG1 new_yaml.py:285->[config:zmes: the configured secrets file exists and is a file -> '/etc/zm/zm_secrets.yml']

11/17/21 19:15:02.617907 zmesdetect_m6[6629] DBG1 new_yaml.py:291->[config:zmes: starting '{[secrets]}' substitution]

11/17/21 19:15:02.625071 zmesdetect_m6[6629] DBG1 new_yaml.py:324->[config:zmes: successfully replaced 17 secrets in the base config -> ['HA_SERVER', 'MQTT_PASSWORD', 'ML_USER', 'PUSHOVER_APP_TOKEN', 'PUSHOVER_USER_KEY', 'MQTT_USERNAME', 'ML_PASSWORD', 'PUSHOVER_ERR_TOKEN', 'HA_TOKEN', 'mlapi_one_key', 'ZM_API_PORTAL', 'PUSHOVER_PASS', 'ZM_PASSWORD', 'ZM_PORTAL', 'ZM_USER', 'PUSHOVER_USER', 'PLATEREC_ALPR_KEY']]

11/17/21 19:15:02.625511 zmesdetect_m6[6629] DBG1 new_yaml.py:329->[config:zmes: there are 2 secrets configured that have no substitution candidates in the base config or the secrets file -> ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY']]

11/17/21 19:15:02.627794 zmesdetect_m6[6629] DBG1 new_yaml.py:358->[config:zmes: starting '{{variable}}' substitution]

11/17/21 19:15:02.632168 zmesdetect_m6[6629] DBG1 new_yaml.py:375->[config:zmes: successfully replaced 51 sub vars in the base config -> ['max_detection_size', 'same_model_sequence_strategy', 'platerec_min_score', 'yolo4_object_framework', 'tpu_face_weights_mobilenetv2', 'alpr_key', 'save_unknown_faces', 'openalpr_cmdline_min_confidence', 'face_detection_pattern', 'gpu_max_processes', 'show_models', 'yolo4_object_processor', 'cpu_max_lock_wait', 'tpu_min_confidence', 'face_num_jitters', 'tpu_max_lock_wait', 'tpu_object_weights_mobiledet', 'tpu_object_framework', 'face_detection_framework', 'frame_set', 'frame_strategy', 'alpr_detection_pattern', 'gpu_max_lock_wait', 'model_sequence', 'face_upsample_times', 'tpu_efficientdet_lite3', 'yolo4_object_labels', 'platerec_stats', 'tpu_object_labels', 'face_recog_dist_threshold', 'openalpr_cmdline_params', 'face_model', 'face_train_model', 'object_detection_pattern', 'tpu_max_processes', 'smart_fps_thresh', 'yolo4_object_weights', 'save_unknown_faces_leeway_pixels', 'cpu_max_processes', 'platerec_min_dscore', 'past_det_max_diff_area', 'yolo4_object_config', 'contained_area', 'tpu_object_weights_mobilenetv2', 'tpu_tf2_mobilenetv2', 'known_images_path', 'disable_locks', 'object_min_confidence', 'unknown_images_path', 'openalpr_cmdline_binary', 'match_past_detections']]

11/17/21 19:15:02.634035 zmesdetect_m6[6629] DBG1 new_yaml.py:187->[config:zmes:proc: Base config has been built, all properly configured {[secrets]} and {{vars}} have been replaced! Remember you must manually build per monitor overrode configurations]

11/17/21 19:15:02.634362 zmesdetect_m6[6629] DBG1 zm_detect.py:598->[perf:zmes: building the intial config took 0.099493 ]

11/17/21 19:15:02.634695 zmesdetect_m6[6629] DBG1 new_yaml.py:747->[zmes:api create: building ZM API Session]

11/17/21 19:15:02.639091 zmesdetect_m6[6629] DBG1 api.py:112->[api: SSL certificate verification disabled (encryption enabled, vulnerable to MITM attacks)]

11/17/21 19:15:02.639751 zmesdetect_m6[6629] DBG1 api.py:281->[api: no token found, trying user/pass for login]

11/17/21 19:15:03.068694 zmesdetect_m6[6629] DBG1 ZMLog:336->[------|  FORKED NEO --- app->Hooks: 7.0.2 - pyzm: 1.0.2 - ES: 7.0.0 - OpenCV:4.5.4 |------]

11/17/21 19:15:02.760818 zmesdetect_m6[6629] DBG1 zm_detect.py:629->[zmes: waiting for the monitor ID to be verified! This happens on PAST events because the api double checks to make sure its the correct monitor ID]

11/17/21 19:15:02.775543 zmesdetect_m6[6629] DBG1 api.py:318->[api: detected API ver 2.0+, using token system]

11/17/21 19:15:02.776201 zmesdetect_m6[6629] DBG1 api.py:333->[api: access token expires on: 2021-11-17 21:15:02.776168 (7200s)]

11/17/21 19:15:02.776551 zmesdetect_m6[6629] DBG1 api.py:345->[api: refresh token expires on: 2021-11-18 19:15:02.776542 (86400s)]

11/17/21 19:15:02.791458 zmesdetect_m6[6629] DBG1 new_yaml.py:780->[zmes:api create: ZM API created]

11/17/21 19:15:02.794211 zmesdetect_m6[6629] DBG1 zm_detect.py:643->[perf:zmes: Monitor ID (6) verified! pausing to wait for verification took 0.033396 seconds -=- api creation took 0.15961 seconds]

11/17/21 19:15:03.07324 zmesdetect_m6[6629] DBG1 ZMLog:336->[zmes: Setting up signal handlers for log 'rotation' and log 'interrupt']

11/17/21 19:15:02.866748 zmesdetect_m6[6629] DBG1 zm_detect.py:710->[zmes: this is a 'PAST' (debugging?) event!]

11/17/21 19:15:03.074283 zmesdetect_m6[6629] DBG1 ZMLog:336->[|----------= Encrypted Route Name: 'mlapi_one' | Gateway URL: 'http://192.168.1.100:5003/api/v1' | Weight: 0 =----------|]

11/17/21 19:15:02.868897 zmesdetect_m6[6629] DBG1 zm_detect.py:103->[zmes:mlapi:login: No need to login, access token is valid for 2044.5638103485107 sec]

11/17/21 19:15:03.029203 zmesdetect_m6[6629] DBG1 zm_detect.py:377->[
** Gateway URL: 'http://192.168.1.100:5003/api/v1/detect/object?type=object' using auth_header={'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGc......'} 
**** params={'delete': True, 'response_format': 'zm_detect'}
****** JSON: stream: 76 - mid: 6 - reason: None - stream options: {'frame_strategy': 'first', 'frame_set': 'snapshot, alarm, snapshot', 'contig_frames_before_error': 2, 'delay_between_attempts': 2.143256, 'max_attempts': 3, 'delay_between_snapshots': 1, 'smart_fps_thresh': '5', 'save_frames': 'no', 'save_frames_dir': None, 'PAST_EVENT': True} - files: {}
]

11/17/21 19:15:03.075885 zmesdetect_m6[6629] DBG1 ZMLog:336->[Connected to ZoneMinder Logging system with user 'www-data' -> '/var/log/zm/zmesdetect_m6.log']

11/17/21 19:15:03.047558 zmesdetect_m6[6629] DBG1 zm_detect.py:422->[zmes:mlapi: error during post to mlapi host-> keys must be str, int, float, bool or None, not bytes]

11/17/21 19:15:03.04857 zmesdetect_m6[6629] DBG1 zm_detect.py:821->[perf:zmes:mlapi: Monitor: office (6)->'Event': 76 mlapi detection took: 0.180621]

11/17/21 19:15:03.049742 zmesdetect_m6[6629] DBG1 zm_detect.py:1590->[zmes: no predictions returned from detections]

11/17/21 19:15:03.051276 zmesdetect_m6[6629] DBG1 zm_detect.py:1658->[perf:zmes:FINAL: 'Monitor': office (6)->'Event': 76 [PAST event] [total:0.520373] [detection:0.180621] [after core detection: 0.000952] ]
root@e4221043b14e:/var/lib/zmeventnotification#
This is the contect of zm_mlapi.log

Code: Select all

root@e4221043b14e:/var/lib/zmeventnotification# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 76
encrypting kickstart key 'access_token'
encrypting kickstart key 'refresh_token'
encrypting kickstart key 'access_token_expires'
encrypting kickstart key 'refresh_token_expires'
encrypting kickstart key 'api_version'
encrypting kickstart key 'zm_version'
encrypting kickstart key 'timezone'

11/17/21 19:15:02.534926 zmesdetect_m6[6629] DBG1 new_yaml.py:242->[config:init: the supplied config file exists -> '/etc/zm/objectconfig.yml']

11/17/21 19:15:02.615138 zmesdetect_m6[6629] DBG1 new_yaml.py:265->[config::init: default configuration built (no secrets or substitution vars replaced, yet!)]

11/17/21 19:15:02.616008 zmesdetect_m6[6629] DBG1 new_yaml.py:170->[config:zmes:proc: 39 built in default keys added to the 'base' config -> ['force_mpd', 'same_model_high_conf', 'skip_mons', 'force_live', 'sanitize_str', 'version', 'basic_user', 'basic_password', 'image_path', 'max_detection_size', 'delete_after_analyze', 'write_debug_image', 'write_image_to_zm', 'hass_enabled', 'hass_people', 'hass_notify', 'hass_cooldown', 'push_err_device', 'push_jpg', 'push_jpg_key', 'push_gif', 'push_gif_key', 'push_debug_device', 'push_cooldown', 'mqtt_topic', 'mqtt_broker', 'mqtt_port', 'tls_cert', 'tls_key', 'fp16_target', 'alpr_api_type', 'alpr_url', 'openalpr_recognize_vehicle', 'openalpr_country', 'openalpr_state', 'openalpr_min_confidence', 'smart_fs_thresh', 'frame_strategy', 'same_model_sequence_strategy']]

11/17/21 19:15:02.616313 zmesdetect_m6[6629] DBG1 new_yaml.py:114->[config:zmes:proc: substituting '{{variables}}' for the 'base' config keys]

11/17/21 19:15:02.617213 zmesdetect_m6[6629] DBG1 new_yaml.py:136->[config:zmes: successfully replaced 3 default sub vars in the base config -> ['coral_models', 'yolo4_models', 'base_data_path']]

11/17/21 19:15:02.617581 zmesdetect_m6[6629] DBG1 new_yaml.py:285->[config:zmes: the configured secrets file exists and is a file -> '/etc/zm/zm_secrets.yml']

11/17/21 19:15:02.617907 zmesdetect_m6[6629] DBG1 new_yaml.py:291->[config:zmes: starting '{[secrets]}' substitution]

11/17/21 19:15:02.625071 zmesdetect_m6[6629] DBG1 new_yaml.py:324->[config:zmes: successfully replaced 17 secrets in the base config -> ['HA_SERVER', 'MQTT_PASSWORD', 'ML_USER', 'PUSHOVER_APP_TOKEN', 'PUSHOVER_USER_KEY', 'MQTT_USERNAME', 'ML_PASSWORD', 'PUSHOVER_ERR_TOKEN', 'HA_TOKEN', 'mlapi_one_key', 'ZM_API_PORTAL', 'PUSHOVER_PASS', 'ZM_PASSWORD', 'ZM_PORTAL', 'ZM_USER', 'PUSHOVER_USER', 'PLATEREC_ALPR_KEY']]

11/17/21 19:15:02.625511 zmesdetect_m6[6629] DBG1 new_yaml.py:329->[config:zmes: there are 2 secrets configured that have no substitution candidates in the base config or the secrets file -> ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY']]

11/17/21 19:15:02.627794 zmesdetect_m6[6629] DBG1 new_yaml.py:358->[config:zmes: starting '{{variable}}' substitution]

11/17/21 19:15:02.632168 zmesdetect_m6[6629] DBG1 new_yaml.py:375->[config:zmes: successfully replaced 51 sub vars in the base config -> ['max_detection_size', 'same_model_sequence_strategy', 'platerec_min_score', 'yolo4_object_framework', 'tpu_face_weights_mobilenetv2', 'alpr_key', 'save_unknown_faces', 'openalpr_cmdline_min_confidence', 'face_detection_pattern', 'gpu_max_processes', 'show_models', 'yolo4_object_processor', 'cpu_max_lock_wait', 'tpu_min_confidence', 'face_num_jitters', 'tpu_max_lock_wait', 'tpu_object_weights_mobiledet', 'tpu_object_framework', 'face_detection_framework', 'frame_set', 'frame_strategy', 'alpr_detection_pattern', 'gpu_max_lock_wait', 'model_sequence', 'face_upsample_times', 'tpu_efficientdet_lite3', 'yolo4_object_labels', 'platerec_stats', 'tpu_object_labels', 'face_recog_dist_threshold', 'openalpr_cmdline_params', 'face_model', 'face_train_model', 'object_detection_pattern', 'tpu_max_processes', 'smart_fps_thresh', 'yolo4_object_weights', 'save_unknown_faces_leeway_pixels', 'cpu_max_processes', 'platerec_min_dscore', 'past_det_max_diff_area', 'yolo4_object_config', 'contained_area', 'tpu_object_weights_mobilenetv2', 'tpu_tf2_mobilenetv2', 'known_images_path', 'disable_locks', 'object_min_confidence', 'unknown_images_path', 'openalpr_cmdline_binary', 'match_past_detections']]

11/17/21 19:15:02.634035 zmesdetect_m6[6629] DBG1 new_yaml.py:187->[config:zmes:proc: Base config has been built, all properly configured {[secrets]} and {{vars}} have been replaced! Remember you must manually build per monitor overrode configurations]

11/17/21 19:15:02.634362 zmesdetect_m6[6629] DBG1 zm_detect.py:598->[perf:zmes: building the intial config took 0.099493 ]

11/17/21 19:15:02.634695 zmesdetect_m6[6629] DBG1 new_yaml.py:747->[zmes:api create: building ZM API Session]

11/17/21 19:15:02.639091 zmesdetect_m6[6629] DBG1 api.py:112->[api: SSL certificate verification disabled (encryption enabled, vulnerable to MITM attacks)]

11/17/21 19:15:02.639751 zmesdetect_m6[6629] DBG1 api.py:281->[api: no token found, trying user/pass for login]

11/17/21 19:15:03.068694 zmesdetect_m6[6629] DBG1 ZMLog:336->[------|  FORKED NEO --- app->Hooks: 7.0.2 - pyzm: 1.0.2 - ES: 7.0.0 - OpenCV:4.5.4 |------]

11/17/21 19:15:02.760818 zmesdetect_m6[6629] DBG1 zm_detect.py:629->[zmes: waiting for the monitor ID to be verified! This happens on PAST events because the api double checks to make sure its the correct monitor ID]

11/17/21 19:15:02.775543 zmesdetect_m6[6629] DBG1 api.py:318->[api: detected API ver 2.0+, using token system]

11/17/21 19:15:02.776201 zmesdetect_m6[6629] DBG1 api.py:333->[api: access token expires on: 2021-11-17 21:15:02.776168 (7200s)]

11/17/21 19:15:02.776551 zmesdetect_m6[6629] DBG1 api.py:345->[api: refresh token expires on: 2021-11-18 19:15:02.776542 (86400s)]

11/17/21 19:15:02.791458 zmesdetect_m6[6629] DBG1 new_yaml.py:780->[zmes:api create: ZM API created]

11/17/21 19:15:02.794211 zmesdetect_m6[6629] DBG1 zm_detect.py:643->[perf:zmes: Monitor ID (6) verified! pausing to wait for verification took 0.033396 seconds -=- api creation took 0.15961 seconds]

11/17/21 19:15:03.07324 zmesdetect_m6[6629] DBG1 ZMLog:336->[zmes: Setting up signal handlers for log 'rotation' and log 'interrupt']

11/17/21 19:15:02.866748 zmesdetect_m6[6629] DBG1 zm_detect.py:710->[zmes: this is a 'PAST' (debugging?) event!]

11/17/21 19:15:03.074283 zmesdetect_m6[6629] DBG1 ZMLog:336->[|----------= Encrypted Route Name: 'mlapi_one' | Gateway URL: 'http://192.168.1.100:5003/api/v1' | Weight: 0 =----------|]

11/17/21 19:15:02.868897 zmesdetect_m6[6629] DBG1 zm_detect.py:103->[zmes:mlapi:login: No need to login, access token is valid for 2044.5638103485107 sec]

11/17/21 19:15:03.029203 zmesdetect_m6[6629] DBG1 zm_detect.py:377->[
** Gateway URL: 'http://192.168.1.100:5003/api/v1/detect/object?type=object' using auth_header={'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGc......'} 
**** params={'delete': True, 'response_format': 'zm_detect'}
****** JSON: stream: 76 - mid: 6 - reason: None - stream options: {'frame_strategy': 'first', 'frame_set': 'snapshot, alarm, snapshot', 'contig_frames_before_error': 2, 'delay_between_attempts': 2.143256, 'max_attempts': 3, 'delay_between_snapshots': 1, 'smart_fps_thresh': '5', 'save_frames': 'no', 'save_frames_dir': None, 'PAST_EVENT': True} - files: {}
]

11/17/21 19:15:03.075885 zmesdetect_m6[6629] DBG1 ZMLog:336->[Connected to ZoneMinder Logging system with user 'www-data' -> '/var/log/zm/zmesdetect_m6.log']

11/17/21 19:15:03.047558 zmesdetect_m6[6629] DBG1 zm_detect.py:422->[zmes:mlapi: error during post to mlapi host-> keys must be str, int, float, bool or None, not bytes]

11/17/21 19:15:03.04857 zmesdetect_m6[6629] DBG1 zm_detect.py:821->[perf:zmes:mlapi: Monitor: office (6)->'Event': 76 mlapi detection took: 0.180621]

11/17/21 19:15:03.049742 zmesdetect_m6[6629] DBG1 zm_detect.py:1590->[zmes: no predictions returned from detections]

11/17/21 19:15:03.051276 zmesdetect_m6[6629] DBG1 zm_detect.py:1658->[perf:zmes:FINAL: 'Monitor': office (6)->'Event': 76 [PAST event] [total:0.520373] [detection:0.180621] [after core detection: 0.000952] ]
root@e4221043b14e:/var/lib/zmeventnotification# cd logs/
root@e4221043b14e:/var/lib/zmeventnotification/logs# ls
README.md  zm_mlapi.log
root@e4221043b14e:/var/lib/zmeventnotification/logs# cat zm_mlapi.log 
11/17/21 18:39:21.774432 zm_mlapi[1413] DBG1 new_yaml.py:242 [config:init: the supplied config file exists -> '/var/lib/zmeventnotification/mlapiconfig.yml']
11/17/21 18:39:21.803156 zm_mlapi[1413] DBG1 new_yaml.py:68 [config:hash: the SHA-256 hex digest for file '/var/lib/zmeventnotification/mlapiconfig.yml' -> fe9bd93395f8909afa82261dc0f78c2c82f84f1c634e820057b21d4c52aa4fc8]
11/17/21 18:39:21.84567 zm_mlapi[1413] DBG1 new_yaml.py:265 [config::init: default configuration built (no secrets or substitution vars replaced, yet!)]
11/17/21 18:39:21.846361 zm_mlapi[1413] DBG1 new_yaml.py:170 [config:mlapi:proc: 21 built in default keys added to the 'base' config -> ['host', 'wsgi_server', 'same_model_high_conf', 'sanitize_str', 'log_user', 'log_group', 'log_name', 'force_mpd', 'auth_enabled', 'wait', 'max_detection_size', 'detection_sequence', 'smart_fs_thresh', 'object_framework', 'fp16_target', 'alpr_api_type', 'alpr_url', 'openalpr_recognize_vehicle', 'openalpr_country', 'openalpr_state', 'openalpr_min_confidence']]
11/17/21 18:39:21.846644 zm_mlapi[1413] DBG1 new_yaml.py:114 [config:mlapi:proc: substituting '{{variables}}' for the 'base' config keys]
11/17/21 18:39:21.847377 zm_mlapi[1413] DBG1 new_yaml.py:136 [config:mlapi: successfully replaced 3 default sub vars in the base config -> ['coral_models', 'yolo4_models', 'base_data_path']]
11/17/21 18:39:21.847743 zm_mlapi[1413] DBG1 new_yaml.py:285 [config:mlapi: the configured secrets file exists and is a file -> '/etc/zm/zm_secrets.yml']
11/17/21 18:39:21.848117 zm_mlapi[1413] DBG1 new_yaml.py:68 [config:hash: the SHA-256 hex digest for file '/etc/zm/zm_secrets.yml' -> d2f65b3c56cb91a8e77cf830897bb115259b71320e9f6ee10a41a6120ca0568b]
11/17/21 18:39:21.848494 zm_mlapi[1413] DBG1 new_yaml.py:291 [config:mlapi: starting '{[secrets]}' substitution]
11/17/21 18:39:21.853711 zm_mlapi[1413] DBG1 new_yaml.py:324 [config:mlapi: successfully replaced 3 secrets in the base config -> ['PLATEREC_ALPR_KEY', 'MLAPI_SECRET_KEY', 'mlapi_one_key']]
11/17/21 18:39:21.854035 zm_mlapi[1413] DBG1 new_yaml.py:329 [config:mlapi: there are 2 secrets configured that have no substitution candidates in the base config or the secrets file -> ['AWS_SECRET_ACCESS_KEY', 'AWS_ACCESS_KEY_ID']]
11/17/21 18:39:21.855233 zm_mlapi[1413] DBG1 new_yaml.py:358 [config:mlapi: starting '{{variable}}' substitution]
11/17/21 18:39:21.859442 zm_mlapi[1413] DBG1 new_yaml.py:375 [config:mlapi: successfully replaced 51 sub vars in the base config -> ['platerec_stats', 'save_unknown_faces_leeway_pixels', 'tpu_object_framework', 'known_images_path', 'platerec_min_dscore', 'match_past_detections', 'save_unknown_faces', 'face_model', 'tpu_object_labels', 'yolo4_object_framework', 'face_detection_framework', 'contained_area', 'cpu_max_processes', 'model_sequence', 'smart_fps_thresh', 'tpu_min_confidence', 'tpu_object_weights_mobiledet', 'yolo4_object_processor', 'tpu_max_processes', 'alpr_key', 'face_recog_dist_threshold', 'tpu_object_weights_mobilenetv2', 'gpu_max_processes', 'gpu_max_lock_wait', 'alpr_detection_pattern', 'openalpr_cmdline_params', 'face_num_jitters', 'object_min_confidence', 'tpu_tf2_mobilenetv2', 'show_models', 'unknown_images_path', 'face_upsample_times', 'tpu_max_lock_wait', 'same_model_sequence_strategy', 'openalpr_cmdline_binary', 'yolo4_object_weights', 'yolo4_object_config', 'frame_set', 'frame_strategy', 'tpu_face_weights_mobilenetv2', 'face_train_model', 'cpu_max_lock_wait', 'yolo4_object_labels', 'object_detection_pattern', 'openalpr_cmdline_min_confidence', 'max_detection_size', 'past_det_max_diff_area', 'platerec_min_score', 'disable_locks', 'tpu_efficientdet_lite3', 'face_detection_pattern']]
11/17/21 18:39:21.860493 zm_mlapi[1413] DBG1 new_yaml.py:187 [config:mlapi:proc: Base config has been built, all properly configured {[secrets]} and {{vars}} have been replaced! Remember you must manually build per monitor overrode configurations]
11/17/21 18:39:21.860784 zm_mlapi[1413] DBG1 new_yaml.py:416 [config:mlapi:6942069: attempting to build an overrode config from monitor '6942069' overrides]
11/17/21 18:39:21.861372 zm_mlapi[1413] DBG1 new_yaml.py:432 [config:mlapi:6942069: monitor '6942069' overrides, starting '{[secrets]}' substitution]
11/17/21 18:39:21.861655 zm_mlapi[1413] DBG1 new_yaml.py:474 [config:mlapi:6942069: no secrets were replace during overrode config build]
11/17/21 18:39:21.861923 zm_mlapi[1413] DBG1 new_yaml.py:512 [config:mlapi:6942069: no vars were replace during overrode config build]
11/17/21 18:39:21.862253 zm_mlapi[1413] DBG1 new_yaml.py:547 [config:mlapi:6942069: polygon specified -> 'front_yard_polygonzone' for monitor 6942069, validating polygon shape...]
11/17/21 18:39:21.862624 zm_mlapi[1413] DBG1 new_yaml.py:575 [config:mlapi:6942069: creating new entry in polygons!]
11/17/21 18:39:21.862896 zm_mlapi[1413] DBG1 new_yaml.py:583 [config:mlapi:6942069: successfully validated polygon for defined zone 'front_yard' -> [(0, 877), (2170, 553), (3822, 1131), (3822, 2141), (0, 2159)]]
11/17/21 18:39:21.863162 zm_mlapi[1413] DBG1 new_yaml.py:542 [config:mlapi:6942069: detection pattern for monitor '6942069' defined zone 'front_yard' -> (person|dog|cat)]
11/17/21 18:39:21.863711 zm_mlapi[1413] DBG1 new_yaml.py:598 [config:mlapi:6942069: 3 keys overridden in the 'base' config -> ['object_detection_pattern', 'frame_set', 'model_sequence']]
11/17/21 18:39:21.863975 zm_mlapi[1413] DBG1 new_yaml.py:603 [config:mlapi:6942069: 5 keys that did not have a 'base' value to override that are now in the 'base' config -> ['front_yard_polygonzone', 'front_yard_zone_detection_pattern', 'person_max_detection_size', 'person_min_confidence', 'defined zones']]
11/17/21 18:39:22.195206 zm_mlapi[1413] DBG1 ZMLog:336 [mlapi:signal handlers: Setting up for log 'rotation' and log 'interrupt']
11/17/21 18:39:21.864665 zm_mlapi[1413] DBG1 mlapi.py:161 [perf:mlapi:init: total time to build initial config -> 0.090283]
11/17/21 18:39:21.864509 zm_mlapi[1413] DBG1 new_yaml.py:632 [config:init:log: Zoneminder is installed]
11/17/21 18:39:22.222548 zm_mlapi[1413] DBG1 ZMLog:336 [|*** FORKED NEO - Machine Learning API (mlapi) version: 3.0.2 - pyzm version: 1.0.2 - OpenCV version: 4.5.4 ***|]
11/17/21 18:39:22.231194 zm_mlapi[1413] DBG1 ZMLog:336 [mlapi: using 'Flask' with a maximum of '1' processes as WSGI server @ 0.0.0.0:5000]
11/17/21 18:39:22.262385 zm_mlapi[1413] DBG1 ZMLog:336 [Connected to ZoneMinder Logging system with user 'root' -> '/var/lib/zmeventnotification/logs/zm_mlapi.log']
11/17/21 18:40:58.273612 zm_mlapi[1413] DBG1 mlapi_db:84 [mlapi db: login SUCCEEDED for user -> 'juan11perez' (172.17.0.1) [correct password]]
Also the latest pyzm is not installing correctly. I installed with sudo pip3 install -e . and it says successfull, but throws the error

Code: Select all

root@e4221043b14e:~/pyzm# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 76
Traceback (most recent call last):
  File "/var/lib/zmeventnotification/bin/zm_detect.py", line 28, in <module>
    import pyzm.api
ModuleNotFoundError: No module named 'pyzm'
If I install it using the previous install.sh script it works.

Re: neo-ZMES - Public Beta

Posted: Wed Nov 17, 2021 7:38 pm
by tsp84
Are you installing from your forked version of my forked
version?

There are print statements in your logs that are no where in the current codebase.

You would need to delete your forked repo and refork my new repo. I think you are running old code from the deleted repo.

The print statements for "kickstart key" were removed before I deleted and reforked. I just double checked my current repos and for sure those print statements containing "kickstart key" are nowhere to be found in master or DEBUG branch

Re: neo-ZMES - Public Beta

Posted: Wed Nov 17, 2021 11:59 pm
by tsp84
Are you using the Nvidia docker tool to passthrough a GPU? I just started reading up on what a docker setup would take and that seems like the goto solution for GPU use in docker containers.

I might take a look at the docker setup as then at least it would be a controllable environment that's the same everywhere.

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 1:33 am
by juan11perez
I'm running your latest master branch. All I did was go to the mlapi repo select DEBUG branch and copy mlapi.py. then I did the same in zmeventnotification for the file zm-detect.py.

I wouldn't know where to add those print statements. So I didn't modify those files.

I run an unraid server and it has a plugin that effectively manages nvidia passtthrough. Equivalent to the docker tool.

Once the zm+es+ml docker Is running I build opencv and dlib with nvidia with 1 script that does it all. It's simple and works perfect.

I will rebuild the docker from scratch using your repo. Which branch should I use? DEBUG ?

As I mentioned earlier I've been running the zmes docker from original repo for over a year with no problems. Docker is very convenient for a number of reasons including the controllable environment as well as resource use. I can't pass 1 gpu to more than 1 vm (well its possible but definitely way beyond the skill level of even an intermediate user) where as in docker I use the same card for zm, plex, unmanic ( video converter)

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 2:37 am
by tsp84
Reinstall pyzm without the -e flag It wasn't installing to the system wide dir. There is still the issue of those weird kickstart print outs though but reinstall pyzm without the -e flag first and see if that does anything.

Code: Select all

git clone https://github.com/baudneo/pyzm.git
cd pyzm
sudo pip3 install .
I use LXC for my env, not as portable as docker but I also passthrough 1 GPU to multiple containers. At one point I had 1 LXC mining ETH or RVN, the ZM LXC using GPU for video streams and the mlapi LXC using it for detections.

In my test LXC I had pyzm, zmeventnotification and mlapi all in the git dir, so when mlapi went to go call the pyzm modules the directory structure somehow matched up to give me half assed detections until it gave me that weird OpenCV error. Once I installed without the -e flag everything went as it should.

My dev env has all the files symlinked so I can 'hot reload', due to that I don't need to install anything in the dev env. The really weird part, is in the test lxc, how it could find some of the pyzm modules by searching the local git directory but not all of them, therefore giving me the false sense that it was installed properly. Once I moved the pyzm dir out of the git dir so mlapi and zm_detect couldn't get to it locally I got the 'couldnt import pyzm' error.

About that mlapi_dbuser.py not accepting -u and -p, I just tried myself and it seems to work alright for me?

Code: Select all

❯ ./mlapi_dbuser.py -u test -p test
❯ ./mlapi_dbuser.py -l
----- Configured users ---------------
User: Somebody
User: test
Hopefully all of it just clears up after reinstalling neo-pyzm without the -e flag.

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 3:59 pm
by juan11perez
Thank you again. rebuilt entirely from your repo and followed pyzm installation instruction.
pyzm seems ok now. However, the mlapi login problem persists.

Code: Select all

root@98ee1fddfdeb:/var/lib/zmeventnotification# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 6

11/18/21 19:56:13.23832 zmesdetect_m6[1423] DBG1 new_yaml.py:242->[config:init: the supplied config file exists -> '/etc/zm/objectconfig.yml']

11/18/21 19:56:13.326445 zmesdetect_m6[1423] DBG1 new_yaml.py:265->[config::init: default configuration built (no secrets or substitution vars replaced, yet!)]

11/18/21 19:56:13.32759 zmesdetect_m6[1423] DBG1 new_yaml.py:170->[config:zmes:proc: 39 built in default keys added to the 'base' config -> ['force_mpd', 'same_model_high_conf', 'skip_mons', 'force_live', 'sanitize_str', 'version', 'basic_user', 'basic_password', 'image_path', 'max_detection_size', 'delete_after_analyze', 'write_debug_image', 'write_image_to_zm', 'hass_enabled', 'hass_people', 'hass_notify', 'hass_cooldown', 'push_err_device', 'push_jpg', 'push_jpg_key', 'push_gif', 'push_gif_key', 'push_debug_device', 'push_cooldown', 'mqtt_topic', 'mqtt_broker', 'mqtt_port', 'tls_cert', 'tls_key', 'fp16_target', 'alpr_api_type', 'alpr_url', 'openalpr_recognize_vehicle', 'openalpr_country', 'openalpr_state', 'openalpr_min_confidence', 'smart_fs_thresh', 'frame_strategy', 'same_model_sequence_strategy']]

11/18/21 19:56:13.327907 zmesdetect_m6[1423] DBG1 new_yaml.py:114->[config:zmes:proc: substituting '{{variables}}' for the 'base' config keys]

11/18/21 19:56:13.328839 zmesdetect_m6[1423] DBG1 new_yaml.py:136->[config:zmes: successfully replaced 3 default sub vars in the base config -> ['yolo4_models', 'coral_models', 'base_data_path']]

11/18/21 19:56:13.329205 zmesdetect_m6[1423] DBG1 new_yaml.py:285->[config:zmes: the configured secrets file exists and is a file -> '/etc/zm/zm_secrets.yml']

11/18/21 19:56:13.329522 zmesdetect_m6[1423] DBG1 new_yaml.py:291->[config:zmes: starting '{[secrets]}' substitution]

11/18/21 19:56:13.335482 zmesdetect_m6[1423] DBG1 new_yaml.py:324->[config:zmes: successfully replaced 17 secrets in the base config -> ['PUSHOVER_ERR_TOKEN', 'PUSHOVER_USER_KEY', 'PUSHOVER_APP_TOKEN', 'PLATEREC_ALPR_KEY', 'mlapi_one_key', 'HA_TOKEN', 'HA_SERVER', 'MQTT_PASSWORD', 'ML_USER', 'PUSHOVER_PASS', 'MQTT_USERNAME', 'ML_PASSWORD', 'ZM_API_PORTAL', 'ZM_PORTAL', 'PUSHOVER_USER', 'ZM_USER', 'ZM_PASSWORD']]

11/18/21 19:56:13.335865 zmesdetect_m6[1423] DBG1 new_yaml.py:329->[config:zmes: there are 2 secrets configured that have no substitution candidates in the base config or the secrets file -> ['AWS_SECRET_ACCESS_KEY', 'AWS_ACCESS_KEY_ID']]

11/18/21 19:56:13.337972 zmesdetect_m6[1423] DBG1 new_yaml.py:358->[config:zmes: starting '{{variable}}' substitution]

11/18/21 19:56:13.342996 zmesdetect_m6[1423] DBG1 new_yaml.py:375->[config:zmes: successfully replaced 51 sub vars in the base config -> ['gpu_max_processes', 'disable_locks', 'tpu_object_framework', 'yolo4_object_labels', 'past_det_max_diff_area', 'alpr_key', 'match_past_detections', 'tpu_tf2_mobilenetv2', 'gpu_max_lock_wait', 'tpu_efficientdet_lite3', 'alpr_detection_pattern', 'platerec_min_dscore', 'smart_fps_thresh', 'known_images_path', 'yolo4_object_config', 'tpu_object_weights_mobilenetv2', 'yolo4_object_processor', 'tpu_object_labels', 'openalpr_cmdline_binary', 'tpu_max_lock_wait', 'show_models', 'save_unknown_faces_leeway_pixels', 'contained_area', 'yolo4_object_weights', 'cpu_max_lock_wait', 'openalpr_cmdline_params', 'yolo4_object_framework', 'face_num_jitters', 'object_min_confidence', 'cpu_max_processes', 'max_detection_size', 'same_model_sequence_strategy', 'object_detection_pattern', 'unknown_images_path', 'face_detection_pattern', 'tpu_face_weights_mobilenetv2', 'model_sequence', 'save_unknown_faces', 'frame_strategy', 'platerec_min_score', 'tpu_min_confidence', 'tpu_object_weights_mobiledet', 'face_recog_dist_threshold', 'openalpr_cmdline_min_confidence', 'face_model', 'frame_set', 'face_train_model', 'face_detection_framework', 'face_upsample_times', 'tpu_max_processes', 'platerec_stats']]

11/18/21 19:56:13.345858 zmesdetect_m6[1423] DBG1 new_yaml.py:187->[config:zmes:proc: Base config has been built, all properly configured {[secrets]} and {{vars}} have been replaced! Remember you must manually build per monitor overrode configurations]

11/18/21 19:56:13.346309 zmesdetect_m6[1423] DBG1 zm_detect.py:542->[perf:zmes: building the intial config took 0.108051 ]

11/18/21 19:56:13.346842 zmesdetect_m6[1423] DBG1 new_yaml.py:762->[zmes:api create: building ZM API Session]

11/18/21 19:56:13.355495 zmesdetect_m6[1423] DBG1 api.py:114->[api: SSL certificate verification disabled (encryption enabled, vulnerable to MITM attacks)]

11/18/21 19:56:13.356366 zmesdetect_m6[1423] DBG1 api.py:283->[api: no token found, trying user/pass for login]

11/18/21 19:56:13.436144 zmesdetect_m6[1423] DBG1 api.py:320->[api: detected API ver 2.0+, using token system]

11/18/21 19:56:13.436853 zmesdetect_m6[1423] DBG1 api.py:335->[api: access token expires on: 2021-11-18 21:56:13.436821 (7200s)]

11/18/21 19:56:13.437399 zmesdetect_m6[1423] DBG1 api.py:347->[api: refresh token expires on: 2021-11-19 19:56:13.437386 (86400s)]

11/18/21 19:56:13.459052 zmesdetect_m6[1423] DBG1 new_yaml.py:795->[zmes:api create: ZM API created]

11/18/21 19:56:13.714888 zmesdetect_m6[1423] DBG1 ZMLog:336->[------|  FORKED NEO --- app->Hooks: 7.0.3 - pyzm: 1.0.3 - ES: 7.0.3 - OpenCV:4.5.4 |------]

11/18/21 19:56:13.715384 zmesdetect_m6[1423] DBG1 ZMLog:336->[zmes: Setting up signal handlers for log 'rotation' and log 'interrupt']

11/18/21 19:56:13.533965 zmesdetect_m6[1423] DBG1 zm_detect.py:654->[zmes: this is a 'PAST' (debugging?) event!]

11/18/21 19:56:13.716582 zmesdetect_m6[1423] DBG1 ZMLog:336->[|----------= Encrypted Route Name: 'mlapi_one' | Gateway URL: 'http://192.168.1.100:5003/api/v1' | Weight: 0 =----------|]

11/18/21 19:56:13.534562 zmesdetect_m6[1423] DBG1 zm_detect.py:103->[zmes:mlapi:login: No need to login, access token is valid for 339.20241832733154 sec]

11/18/21 19:56:13.666285 zmesdetect_m6[1423] DBG1 zm_detect.py:318->[
** Gateway URL: 'http://192.168.1.100:5003/api/v1/detect/object?type=object' using auth_header={'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGc......'} 
**** params={'delete': True, 'response_format': 'zm_detect'}
****** JSON: stream: 6 - mid: 6 - reason: None - stream options: {'frame_strategy': 'first', 'frame_set': 'snapshot, alarm, snapshot', 'contig_frames_before_error': 2, 'delay_between_attempts': 2.143256, 'max_attempts': 3, 'delay_between_snapshots': 1, 'smart_fps_thresh': '5', 'save_frames': 'no', 'save_frames_dir': None, 'PAST_EVENT': True} - files: {}
]

11/18/21 19:56:13.718506 zmesdetect_m6[1423] DBG1 ZMLog:336->[Connected to ZoneMinder Logging system with user 'www-data' -> '/var/log/zm/zmesdetect_m6.log']

11/18/21 19:56:13.685881 zmesdetect_m6[1423] DBG1 zm_detect.py:365->[zmes:mlapi: error during post to mlapi host-> keys must be str, int, float, bool or None, not bytes]

11/18/21 19:56:13.68787 zmesdetect_m6[1423] DBG1 zm_detect.py:368->[traceback-> Traceback (most recent call last):
  File "/var/lib/zmeventnotification/bin/zm_detect.py", line 328, in remote_detect
    r = requests.post(
  File "/usr/lib/python3/dist-packages/requests/api.py", line 116, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/api.py", line 60, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 519, in request
    prep = self.prepare_request(req)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 452, in prepare_request
    p.prepare(
  File "/usr/lib/python3/dist-packages/requests/models.py", line 316, in prepare
    self.prepare_body(data, files, json)
  File "/usr/lib/python3/dist-packages/requests/models.py", line 466, in prepare_body
    body = complexjson.dumps(json)
  File "/usr/lib/python3.8/json/__init__.py", line 231, in dumps
    return _default_encoder.encode(obj)
  File "/usr/lib/python3.8/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.8/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
TypeError: keys must be str, int, float, bool or None, not bytes
]

11/18/21 19:56:13.688581 zmesdetect_m6[1423] DBG1 zm_detect.py:765->[perf:zmes:mlapi: Monitor: office (6)->'Event': 6 mlapi detection took: 0.154184]

11/18/21 19:56:13.689235 zmesdetect_m6[1423] DBG1 zm_detect.py:1533->[zmes: no predictions returned from detections]

11/18/21 19:56:13.69055 zmesdetect_m6[1423] DBG1 zm_detect.py:1601->[perf:zmes:FINAL: 'Monitor': office (6)->'Event': 6 [PAST event] [total:0.455474] [detection:0.154184] [after core detection: 0.000632] ]
root@98ee1fddfdeb:/var/lib/zmeventnotification#
with zmes only detection is working fine

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 7:28 pm
by tsp84
I'm wondering if the JSON parser is different in python3.8 than in 3.9, what it's sounding like to me is it doesn't want to JSON parse the encrypted data because it's in byte form (I'm using 3.9 and I don't get that error). I'm going to try something quick and ill push a new zm_detect to the DEBUG branch. If you reclone the repo just copy the zm_detect.py file and place it in /var/lib/zmeventnotification/bin to overwrite the current zm_detect.py.

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 7:47 pm
by tsp84
Actually just pull from master branch. The only file you need to change is zm_detect.py. I am pushing right now.

Edit: Or if you are comfortable editing your zm_detect.py in place find line 284 ->

Code: Select all

                f.encrypt(str(k).encode('utf-8')): f.encrypt(str(v).encode('utf-8'))
Change that line to this ->

Code: Select all

                f.encrypt(str(k).encode('utf-8')).decode(): f.encrypt(str(v).encode('utf-8')).decode()

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 8:02 pm
by juan11perez
that seems to complete the login and object detection. It throws another error at the end though.

Code: Select all

root@98ee1fddfdeb:/var/lib/zmeventnotification# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 6

11/19/21 00:01:29.961189 zmesdetect_m6[1355] DBG1 new_yaml.py:242->[config:init: the supplied config file exists -> '/etc/zm/objectconfig.yml']

11/19/21 00:01:30.102578 zmesdetect_m6[1355] DBG1 new_yaml.py:265->[config::init: default configuration built (no secrets or substitution vars replaced, yet!)]

11/19/21 00:01:30.103975 zmesdetect_m6[1355] DBG1 new_yaml.py:170->[config:zmes:proc: 39 built in default keys added to the 'base' config -> ['force_mpd', 'same_model_high_conf', 'skip_mons', 'force_live', 'sanitize_str', 'version', 'basic_user', 'basic_password', 'image_path', 'max_detection_size', 'delete_after_analyze', 'write_debug_image', 'write_image_to_zm', 'hass_enabled', 'hass_people', 'hass_notify', 'hass_cooldown', 'push_err_device', 'push_jpg', 'push_jpg_key', 'push_gif', 'push_gif_key', 'push_debug_device', 'push_cooldown', 'mqtt_topic', 'mqtt_broker', 'mqtt_port', 'tls_cert', 'tls_key', 'fp16_target', 'alpr_api_type', 'alpr_url', 'openalpr_recognize_vehicle', 'openalpr_country', 'openalpr_state', 'openalpr_min_confidence', 'smart_fs_thresh', 'frame_strategy', 'same_model_sequence_strategy']]

11/19/21 00:01:30.104389 zmesdetect_m6[1355] DBG1 new_yaml.py:114->[config:zmes:proc: substituting '{{variables}}' for the 'base' config keys]

11/19/21 00:01:30.105742 zmesdetect_m6[1355] DBG1 new_yaml.py:136->[config:zmes: successfully replaced 3 default sub vars in the base config -> ['base_data_path', 'yolo4_models', 'coral_models']]

11/19/21 00:01:30.106251 zmesdetect_m6[1355] DBG1 new_yaml.py:285->[config:zmes: the configured secrets file exists and is a file -> '/etc/zm/zm_secrets.yml']

11/19/21 00:01:30.106738 zmesdetect_m6[1355] DBG1 new_yaml.py:291->[config:zmes: starting '{[secrets]}' substitution]

11/19/21 00:01:30.115616 zmesdetect_m6[1355] DBG1 new_yaml.py:324->[config:zmes: successfully replaced 17 secrets in the base config -> ['PUSHOVER_APP_TOKEN', 'ZM_PORTAL', 'ZM_USER', 'PUSHOVER_USER_KEY', 'ML_PASSWORD', 'ZM_PASSWORD', 'mlapi_one_key', 'MQTT_USERNAME', 'PUSHOVER_ERR_TOKEN', 'HA_TOKEN', 'ML_USER', 'PUSHOVER_USER', 'MQTT_PASSWORD', 'ZM_API_PORTAL', 'PLATEREC_ALPR_KEY', 'PUSHOVER_PASS', 'HA_SERVER']]

11/19/21 00:01:30.11648 zmesdetect_m6[1355] DBG1 new_yaml.py:329->[config:zmes: there are 2 secrets configured that have no substitution candidates in the base config or the secrets file -> ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY']]

11/19/21 00:01:30.121315 zmesdetect_m6[1355] DBG1 new_yaml.py:358->[config:zmes: starting '{{variable}}' substitution]

11/19/21 00:01:30.129141 zmesdetect_m6[1355] DBG1 new_yaml.py:375->[config:zmes: successfully replaced 51 sub vars in the base config -> ['alpr_detection_pattern', 'show_models', 'openalpr_cmdline_params', 'openalpr_cmdline_min_confidence', 'cpu_max_lock_wait', 'match_past_detections', 'tpu_max_lock_wait', 'gpu_max_lock_wait', 'object_min_confidence', 'gpu_max_processes', 'known_images_path', 'save_unknown_faces', 'tpu_object_weights_mobilenetv2', 'platerec_min_score', 'contained_area', 'tpu_efficientdet_lite3', 'model_sequence', 'tpu_min_confidence', 'face_model', 'platerec_min_dscore', 'openalpr_cmdline_binary', 'frame_strategy', 'tpu_object_weights_mobiledet', 'alpr_key', 'face_detection_framework', 'face_upsample_times', 'save_unknown_faces_leeway_pixels', 'unknown_images_path', 'yolo4_object_processor', 'cpu_max_processes', 'object_detection_pattern', 'tpu_tf2_mobilenetv2', 'tpu_max_processes', 'disable_locks', 'smart_fps_thresh', 'frame_set', 'face_num_jitters', 'tpu_object_framework', 'yolo4_object_config', 'yolo4_object_weights', 'max_detection_size', 'yolo4_object_labels', 'tpu_face_weights_mobilenetv2', 'platerec_stats', 'same_model_sequence_strategy', 'yolo4_object_framework', 'face_recog_dist_threshold', 'past_det_max_diff_area', 'face_train_model', 'tpu_object_labels', 'face_detection_pattern']]

11/19/21 00:01:30.132835 zmesdetect_m6[1355] DBG1 new_yaml.py:187->[config:zmes:proc: Base config has been built, all properly configured {[secrets]} and {{vars}} have been replaced! Remember you must manually build per monitor overrode configurations]

11/19/21 00:01:30.133305 zmesdetect_m6[1355] DBG1 zm_detect.py:542->[perf:zmes: building the intial config took 0.172153 ]

11/19/21 00:01:30.135178 zmesdetect_m6[1355] DBG1 new_yaml.py:762->[zmes:api create: building ZM API Session]

11/19/21 00:01:30.136481 zmesdetect_m6[1355] DBG1 api.py:114->[api: SSL certificate verification disabled (encryption enabled, vulnerable to MITM attacks)]

11/19/21 00:01:30.137217 zmesdetect_m6[1355] DBG1 api.py:283->[api: no token found, trying user/pass for login]

11/19/21 00:01:30.241623 zmesdetect_m6[1355] DBG1 api.py:320->[api: detected API ver 2.0+, using token system]

11/19/21 00:01:30.242584 zmesdetect_m6[1355] DBG1 api.py:335->[api: access token expires on: 2021-11-19 02:01:30.242546 (7200s)]

11/19/21 00:01:30.243029 zmesdetect_m6[1355] DBG1 api.py:347->[api: refresh token expires on: 2021-11-20 00:01:30.243018 (86400s)]

11/19/21 00:01:30.265278 zmesdetect_m6[1355] DBG1 new_yaml.py:795->[zmes:api create: ZM API created]

11/19/21 00:01:30.821085 zmesdetect_m6[1355] DBG1 ZMLog:336->[------|  FORKED NEO --- app->Hooks: 7.0.3 - pyzm: 1.0.3 - ES: 7.0.3 - OpenCV:4.5.4 |------]

11/19/21 00:01:30.821548 zmesdetect_m6[1355] DBG1 ZMLog:336->[zmes: Setting up signal handlers for log 'rotation' and log 'interrupt']

11/19/21 00:01:30.489889 zmesdetect_m6[1355] DBG1 zm_detect.py:654->[zmes: this is a 'PAST' (debugging?) event!]

11/19/21 00:01:30.822523 zmesdetect_m6[1355] DBG1 ZMLog:336->[|----------= Encrypted Route Name: 'mlapi_one' | Gateway URL: 'http://192.168.1.100:5003/api/v1' | Weight: 0 =----------|]

11/19/21 00:01:30.503128 zmesdetect_m6[1355] DBG1 zm_detect.py:103->[zmes:mlapi:login: No need to login, access token is valid for 54.41167736053467 sec]

11/19/21 00:01:30.642301 zmesdetect_m6[1355] DBG1 zm_detect.py:318->[
** Gateway URL: 'http://192.168.1.100:5003/api/v1/detect/object?type=object' using auth_header={'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGc......'} 
**** params={'delete': True, 'response_format': 'zm_detect'}
****** JSON: stream: 6 - mid: 6 - reason: None - stream options: {'frame_strategy': 'first', 'frame_set': 'snapshot, alarm, snapshot', 'contig_frames_before_error': 2, 'delay_between_attempts': 2.143256, 'max_attempts': 3, 'delay_between_snapshots': 1, 'smart_fps_thresh': '5', 'save_frames': 'no', 'save_frames_dir': None, 'PAST_EVENT': True} - files: {}
]

11/19/21 00:01:30.828048 zmesdetect_m6[1355] DBG1 ZMLog:336->[Connected to ZoneMinder Logging system with user 'www-data' -> '/var/log/zm/zmesdetect_m6.log']

11/19/21 00:01:31.960913 zmesdetect_m6[1355] DBG1 zm_detect:383->[zmes:mlapi: parsed JSON detection data in multipart response]

11/19/21 00:01:31.961402 zmesdetect_m6[1355] DBG1 zm_detect:379->[zmes:mlapi: parsed the matched image from bytes to a jpeg encoded array in multipart response]

11/19/21 00:01:31.981064 zmesdetect_m6[1355] DBG1 zm_detect:765->[perf:zmes:mlapi: Monitor: office (6)->'Event': 6 mlapi detection took: 1.482059]

11/19/21 00:01:31.98146 zmesdetect_m6[1355] DBG1 pyzm_utils:1336->['success'->  True  ]

11/19/21 00:01:31.981761 zmesdetect_m6[1355] DBG1 pyzm_utils:1297->[--- --- ---]
11/19/21 00:01:31.982022 zmesdetect_m6[1355] DBG1 pyzm_utils:1298->['matched_data'->  labels-->['person']  ]
11/19/21 00:01:31.982301 zmesdetect_m6[1355] DBG1 pyzm_utils:1298->['matched_data'->  model_names-->['yolo[GPU]']  ]
11/19/21 00:01:31.982574 zmesdetect_m6[1355] DBG1 pyzm_utils:1298->['matched_data'->  confidences-->[0.8503115773200989]  ]
11/19/21 00:01:31.982872 zmesdetect_m6[1355] DBG1 pyzm_utils:1298->['matched_data'->  frame_id-->s-77  ]
11/19/21 00:01:31.983127 zmesdetect_m6[1355] DBG1 pyzm_utils:1298->['matched_data'->  type-->['object']  ]
11/19/21 00:01:31.983375 zmesdetect_m6[1355] DBG1 pyzm_utils:1298->['matched_data'->  boxes-->[[-34, 290, 1936, 1074]]  ]
11/19/21 00:01:31.98362 zmesdetect_m6[1355] DBG1 pyzm_utils:1298->['matched_data'->  image_dimensions-->{'original': [1080, 1920], 'resized': None}  ]
11/19/21 00:01:31.983877 zmesdetect_m6[1355] DBG1 pyzm_utils:1298->['matched_data'->  polygons-->[{'name': 'full_image', 'value': [[0, 0], [1920, 0], [1920, 1080], [0, 1080]], 'pattern': None}]  ]
11/19/21 00:01:31.984117 zmesdetect_m6[1355] DBG1 pyzm_utils:1298->['matched_data'->  image--><uint-8 encoded jpg>  ]

11/19/21 00:01:31.984182 zmesdetect_m6[1355] INF zm_detect:864->[zmes:prediction: '[s-77] person(85%)']

11/19/21 00:01:31.984651 zmesdetect_m6[1355] DBG1 zm_detect:866->[zmes:prediction:JSON: {"frame_id": "77", "labels": ["person"], "confidences": [0.8503115773200989], "boxes": [[-34, 290, 1936, 1074]], "image_dimensions": {"original": [1080, 1920], "resized": null}}]
zmes: err_msg->'mon_name'
zmes: traceback: Traceback (most recent call last):
  File "/var/lib/zmeventnotification/bin/zm_detect.py", line 1591, in <module>
    main_handler()
  File "/var/lib/zmeventnotification/bin/zm_detect.py", line 1234, in main_handler
    f"{lp} not writing objdetect.jpg or objects.json as monitor {g.mid}->'{args['mon_name']}' "
KeyError: 'mon_name'

root@98ee1fddfdeb:/var/lib/zmeventnotification#

edit: more testing. seems its working. only throws that last error if I rescan an event that it already detected.

Code: Select all

oot@98ee1fddfdeb:/var/lib/zmeventnotification# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 13

11/19/21 00:07:56.686415 zmesdetect_m6[1927] DBG1 new_yaml.py:242->[config:init: the supplied config file exists -> '/etc/zm/objectconfig.yml']

11/19/21 00:07:56.768462 zmesdetect_m6[1927] DBG1 new_yaml.py:265->[config::init: default configuration built (no secrets or substitution vars replaced, yet!)]

11/19/21 00:07:56.769282 zmesdetect_m6[1927] DBG1 new_yaml.py:170->[config:zmes:proc: 39 built in default keys added to the 'base' config -> ['force_mpd', 'same_model_high_conf', 'skip_mons', 'force_live', 'sanitize_str', 'version', 'basic_user', 'basic_password', 'image_path', 'max_detection_size', 'delete_after_analyze', 'write_debug_image', 'write_image_to_zm', 'hass_enabled', 'hass_people', 'hass_notify', 'hass_cooldown', 'push_err_device', 'push_jpg', 'push_jpg_key', 'push_gif', 'push_gif_key', 'push_debug_device', 'push_cooldown', 'mqtt_topic', 'mqtt_broker', 'mqtt_port', 'tls_cert', 'tls_key', 'fp16_target', 'alpr_api_type', 'alpr_url', 'openalpr_recognize_vehicle', 'openalpr_country', 'openalpr_state', 'openalpr_min_confidence', 'smart_fs_thresh', 'frame_strategy', 'same_model_sequence_strategy']]

11/19/21 00:07:56.769594 zmesdetect_m6[1927] DBG1 new_yaml.py:114->[config:zmes:proc: substituting '{{variables}}' for the 'base' config keys]

11/19/21 00:07:56.770416 zmesdetect_m6[1927] DBG1 new_yaml.py:136->[config:zmes: successfully replaced 3 default sub vars in the base config -> ['yolo4_models', 'coral_models', 'base_data_path']]

11/19/21 00:07:56.770797 zmesdetect_m6[1927] DBG1 new_yaml.py:285->[config:zmes: the configured secrets file exists and is a file -> '/etc/zm/zm_secrets.yml']

11/19/21 00:07:56.771118 zmesdetect_m6[1927] DBG1 new_yaml.py:291->[config:zmes: starting '{[secrets]}' substitution]

11/19/21 00:07:56.777215 zmesdetect_m6[1927] DBG1 new_yaml.py:324->[config:zmes: successfully replaced 17 secrets in the base config -> ['HA_SERVER', 'PUSHOVER_APP_TOKEN', 'mlapi_one_key', 'ZM_PASSWORD', 'ZM_PORTAL', 'PUSHOVER_USER_KEY', 'PUSHOVER_USER', 'ML_PASSWORD', 'PUSHOVER_PASS', 'ZM_USER', 'PLATEREC_ALPR_KEY', 'ZM_API_PORTAL', 'MQTT_USERNAME', 'HA_TOKEN', 'ML_USER', 'PUSHOVER_ERR_TOKEN', 'MQTT_PASSWORD']]

11/19/21 00:07:56.777617 zmesdetect_m6[1927] DBG1 new_yaml.py:329->[config:zmes: there are 2 secrets configured that have no substitution candidates in the base config or the secrets file -> ['AWS_ACCESS_KEY_ID', 'AWS_SECRET_ACCESS_KEY']]

11/19/21 00:07:56.779442 zmesdetect_m6[1927] DBG1 new_yaml.py:358->[config:zmes: starting '{{variable}}' substitution]

11/19/21 00:07:56.784135 zmesdetect_m6[1927] DBG1 new_yaml.py:375->[config:zmes: successfully replaced 51 sub vars in the base config -> ['yolo4_object_config', 'face_detection_pattern', 'face_detection_framework', 'face_model', 'object_min_confidence', 'tpu_efficientdet_lite3', 'platerec_min_dscore', 'same_model_sequence_strategy', 'smart_fps_thresh', 'tpu_max_lock_wait', 'max_detection_size', 'cpu_max_processes', 'face_train_model', 'past_det_max_diff_area', 'tpu_object_weights_mobiledet', 'platerec_min_score', 'object_detection_pattern', 'tpu_object_labels', 'face_num_jitters', 'tpu_object_framework', 'tpu_max_processes', 'openalpr_cmdline_min_confidence', 'tpu_face_weights_mobilenetv2', 'match_past_detections', 'yolo4_object_labels', 'save_unknown_faces', 'tpu_min_confidence', 'openalpr_cmdline_binary', 'known_images_path', 'frame_strategy', 'openalpr_cmdline_params', 'alpr_detection_pattern', 'gpu_max_lock_wait', 'yolo4_object_framework', 'face_recog_dist_threshold', 'gpu_max_processes', 'disable_locks', 'show_models', 'tpu_tf2_mobilenetv2', 'cpu_max_lock_wait', 'save_unknown_faces_leeway_pixels', 'contained_area', 'tpu_object_weights_mobilenetv2', 'unknown_images_path', 'face_upsample_times', 'model_sequence', 'platerec_stats', 'alpr_key', 'yolo4_object_weights', 'frame_set', 'yolo4_object_processor']]

11/19/21 00:07:56.786425 zmesdetect_m6[1927] DBG1 new_yaml.py:187->[config:zmes:proc: Base config has been built, all properly configured {[secrets]} and {{vars}} have been replaced! Remember you must manually build per monitor overrode configurations]

11/19/21 00:07:56.786789 zmesdetect_m6[1927] DBG1 zm_detect.py:542->[perf:zmes: building the intial config took 0.100445 ]

11/19/21 00:07:56.787277 zmesdetect_m6[1927] DBG1 new_yaml.py:762->[zmes:api create: building ZM API Session]

11/19/21 00:07:56.795668 zmesdetect_m6[1927] DBG1 api.py:114->[api: SSL certificate verification disabled (encryption enabled, vulnerable to MITM attacks)]

11/19/21 00:07:56.796523 zmesdetect_m6[1927] DBG1 api.py:283->[api: no token found, trying user/pass for login]

11/19/21 00:07:56.865564 zmesdetect_m6[1927] DBG1 api.py:320->[api: detected API ver 2.0+, using token system]

11/19/21 00:07:56.866066 zmesdetect_m6[1927] DBG1 api.py:335->[api: access token expires on: 2021-11-19 02:07:56.866049 (7200s)]

11/19/21 00:07:56.866428 zmesdetect_m6[1927] DBG1 api.py:347->[api: refresh token expires on: 2021-11-20 00:07:56.866420 (86400s)]

11/19/21 00:07:56.883361 zmesdetect_m6[1927] DBG1 new_yaml.py:795->[zmes:api create: ZM API created]

11/19/21 00:07:57.155119 zmesdetect_m6[1927] DBG1 ZMLog:336->[------|  FORKED NEO --- app->Hooks: 7.0.3 - pyzm: 1.0.3 - ES: 7.0.3 - OpenCV:4.5.4 |------]

11/19/21 00:07:57.1556 zmesdetect_m6[1927] DBG1 ZMLog:336->[zmes: Setting up signal handlers for log 'rotation' and log 'interrupt']

11/19/21 00:07:56.97689 zmesdetect_m6[1927] DBG1 zm_detect.py:654->[zmes: this is a 'PAST' (debugging?) event!]

11/19/21 00:07:57.156744 zmesdetect_m6[1927] DBG1 ZMLog:336->[|----------= Encrypted Route Name: 'mlapi_one' | Gateway URL: 'http://192.168.1.100:5003/api/v1' | Weight: 0 =----------|]

11/19/21 00:07:56.979678 zmesdetect_m6[1927] DBG1 zm_detect.py:103->[zmes:mlapi:login: No need to login, access token is valid for 440.88838744163513 sec]

11/19/21 00:07:57.106068 zmesdetect_m6[1927] DBG1 zm_detect.py:318->[
** Gateway URL: 'http://192.168.1.100:5003/api/v1/detect/object?type=object' using auth_header={'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGc......'} 
**** params={'delete': True, 'response_format': 'zm_detect'}
****** JSON: stream: 13 - mid: 6 - reason: None - stream options: {'frame_strategy': 'first', 'frame_set': 'snapshot, alarm, snapshot', 'contig_frames_before_error': 2, 'delay_between_attempts': 2.143256, 'max_attempts': 3, 'delay_between_snapshots': 1, 'smart_fps_thresh': '5', 'save_frames': 'no', 'save_frames_dir': None, 'PAST_EVENT': True} - files: {}
]

11/19/21 00:07:57.158577 zmesdetect_m6[1927] DBG1 ZMLog:336->[Connected to ZoneMinder Logging system with user 'www-data' -> '/var/log/zm/zmesdetect_m6.log']

11/19/21 00:07:58.03875 zmesdetect_m6[1927] DBG1 zm_detect:383->[zmes:mlapi: parsed JSON detection data in multipart response]

11/19/21 00:07:58.039119 zmesdetect_m6[1927] DBG1 zm_detect:379->[zmes:mlapi: parsed the matched image from bytes to a jpeg encoded array in multipart response]

11/19/21 00:07:58.057862 zmesdetect_m6[1927] DBG1 zm_detect:765->[perf:zmes:mlapi: Monitor: office (6)->'Event': 13 mlapi detection took: 1.074231]

11/19/21 00:07:58.058308 zmesdetect_m6[1927] DBG1 pyzm_utils:1336->['success'->  True  ]

11/19/21 00:07:58.05864 zmesdetect_m6[1927] DBG1 pyzm_utils:1297->[--- --- ---]
11/19/21 00:07:58.058949 zmesdetect_m6[1927] DBG1 pyzm_utils:1298->['matched_data'->  labels-->['person']  ]
11/19/21 00:07:58.059246 zmesdetect_m6[1927] DBG1 pyzm_utils:1298->['matched_data'->  model_names-->['yolo[GPU]']  ]
11/19/21 00:07:58.059544 zmesdetect_m6[1927] DBG1 pyzm_utils:1298->['matched_data'->  confidences-->[0.6476893424987793]  ]
11/19/21 00:07:58.059854 zmesdetect_m6[1927] DBG1 pyzm_utils:1298->['matched_data'->  frame_id-->s-77  ]
11/19/21 00:07:58.060183 zmesdetect_m6[1927] DBG1 pyzm_utils:1298->['matched_data'->  type-->['object']  ]
11/19/21 00:07:58.060488 zmesdetect_m6[1927] DBG1 pyzm_utils:1298->['matched_data'->  boxes-->[[-20, 295, 1658, 1077]]  ]
11/19/21 00:07:58.060792 zmesdetect_m6[1927] DBG1 pyzm_utils:1298->['matched_data'->  image_dimensions-->{'original': [1080, 1920], 'resized': None}  ]
11/19/21 00:07:58.061098 zmesdetect_m6[1927] DBG1 pyzm_utils:1298->['matched_data'->  polygons-->[{'name': 'full_image', 'value': [[0, 0], [1920, 0], [1920, 1080], [0, 1080]], 'pattern': None}]  ]
11/19/21 00:07:58.061393 zmesdetect_m6[1927] DBG1 pyzm_utils:1298->['matched_data'->  image--><uint-8 encoded jpg>  ]

11/19/21 00:07:58.061465 zmesdetect_m6[1927] INF zm_detect:864->[zmes:prediction: '[s-77] person(65%)']

11/19/21 00:07:58.062142 zmesdetect_m6[1927] DBG1 zm_detect:866->[zmes:prediction:JSON: {"frame_id": "77", "labels": ["person"], "confidences": [0.6476893424987793], "boxes": [[-20, 295, 1658, 1077]], "image_dimensions": {"original": [1080, 1920], "resized": null}}]

11/19/21 00:07:58.0666 zmesdetect_m6[1927] DBG1 zm_detect:1218->[zmes: writing objects.json and objdetect.jpg to '/var/cache/zoneminder/events/6/2021-11-18/13']

11/19/21 00:07:58.075301 zmesdetect_m6[1927] DBG2 zm_detect:1390->[api:make_req: 'put'->http://192.168.1.100:8089/zm/api/events/13.json payload={'Event[Notes]': '[s-77] :detected:person(65%) Motion: All'} query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJab25lTWluZGVyIiwiaWF0IjoxNjM3MjY2MDc2LCJleHAiOjE2MzcyNzMyNzYsInVzZXIiOiJqdWFuIiwidHlwZSI6ImFjY2VzcyJ9.FXSBp6m83tc18tIltbuHdyyZplhA7fH0NU0cuSS5zdM...'}]

11/19/21 00:07:58.087407 zmesdetect_m6[1927] DBG1 zm_detect:1401->[zmes: replaced old note -> 'Motion: All' with new note -> '[s-77] :detected:person(65%) Motion: All']

11/19/21 00:07:58.088672 zmesdetect_m6[1927] DBG1 zm_detect:1601->[perf:zmes:FINAL: 'Monitor': office (6)->'Event': 13-->'Frame ID':77 [PAST event] [total:1.406142] [detection:1.074231] [after core detection: 0.029513] ]

11/19/21 00:07:58.089233 zmesdetect_m6[1927] DBG1 zm_detect:1602->[ZM Log: Closing all log handlers NOW]
root@98ee1fddfdeb:/var/lib/zmeventnotification#

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 8:15 pm
by tsp84
Just pushed, line 1234 ->

Code: Select all

                    f"{lp} not writing objdetect.jpg or objects.json as monitor {g.mid}->'{args['mon_name']}' "
to ->

Code: Select all

                    f"{lp} not writing objdetect.jpg or objects.json as monitor {g.mid}->'{g.config.get('mon_name')}' "

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 8:19 pm
by juan11perez
yes. that seems to correct it all

Code: Select all

root@98ee1fddfdeb:/var/lib/zmeventnotification# sudo -u www-data /var/lib/zmeventnotification/bin/zm_detect.py --config /etc/zm/objectconfig.yml --debug --eventid 9

11/19/21 00:19:07.248036 zmesdetect_m6[1320] DBG1 new_yaml.py:242->[config:init: the supplied config file exists -> '/etc/zm/objectconfig.yml']

11/19/21 00:19:07.332261 zmesdetect_m6[1320] DBG1 new_yaml.py:265->[config::init: default configuration built (no secrets or substitution vars replaced, yet!)]

11/19/21 00:19:07.333083 zmesdetect_m6[1320] DBG1 new_yaml.py:170->[config:zmes:proc: 39 built in default keys added to the 'base' config -> ['force_mpd', 'same_model_high_conf', 'skip_mons', 'force_live', 'sanitize_str', 'version', 'basic_user', 'basic_password', 'image_path', 'max_detection_size', 'delete_after_analyze', 'write_debug_image', 'write_image_to_zm', 'hass_enabled', 'hass_people', 'hass_notify', 'hass_cooldown', 'push_err_device', 'push_jpg', 'push_jpg_key', 'push_gif', 'push_gif_key', 'push_debug_device', 'push_cooldown', 'mqtt_topic', 'mqtt_broker', 'mqtt_port', 'tls_cert', 'tls_key', 'fp16_target', 'alpr_api_type', 'alpr_url', 'openalpr_recognize_vehicle', 'openalpr_country', 'openalpr_state', 'openalpr_min_confidence', 'smart_fs_thresh', 'frame_strategy', 'same_model_sequence_strategy']]

11/19/21 00:19:07.333413 zmesdetect_m6[1320] DBG1 new_yaml.py:114->[config:zmes:proc: substituting '{{variables}}' for the 'base' config keys]

11/19/21 00:19:07.334256 zmesdetect_m6[1320] DBG1 new_yaml.py:136->[config:zmes: successfully replaced 3 default sub vars in the base config -> ['coral_models', 'base_data_path', 'yolo4_models']]

11/19/21 00:19:07.334679 zmesdetect_m6[1320] DBG1 new_yaml.py:285->[config:zmes: the configured secrets file exists and is a file -> '/etc/zm/zm_secrets.yml']

11/19/21 00:19:07.33504 zmesdetect_m6[1320] DBG1 new_yaml.py:291->[config:zmes: starting '{[secrets]}' substitution]

11/19/21 00:19:07.341818 zmesdetect_m6[1320] DBG1 new_yaml.py:324->[config:zmes: successfully replaced 17 secrets in the base config -> ['PLATEREC_ALPR_KEY', 'ZM_API_PORTAL', 'ML_PASSWORD', 'mlapi_one_key', 'ZM_PASSWORD', 'ML_USER', 'PUSHOVER_APP_TOKEN', 'HA_TOKEN', 'ZM_USER', 'HA_SERVER', 'PUSHOVER_USER_KEY', 'PUSHOVER_PASS', 'MQTT_USERNAME', 'PUSHOVER_ERR_TOKEN', 'MQTT_PASSWORD', 'ZM_PORTAL', 'PUSHOVER_USER']]

11/19/21 00:19:07.342357 zmesdetect_m6[1320] DBG1 new_yaml.py:329->[config:zmes: there are 2 secrets configured that have no substitution candidates in the base config or the secrets file -> ['AWS_SECRET_ACCESS_KEY', 'AWS_ACCESS_KEY_ID']]

11/19/21 00:19:07.344155 zmesdetect_m6[1320] DBG1 new_yaml.py:358->[config:zmes: starting '{{variable}}' substitution]

11/19/21 00:19:07.349416 zmesdetect_m6[1320] DBG1 new_yaml.py:375->[config:zmes: successfully replaced 51 sub vars in the base config -> ['tpu_efficientdet_lite3', 'gpu_max_processes', 'face_recog_dist_threshold', 'openalpr_cmdline_binary', 'past_det_max_diff_area', 'unknown_images_path', 'tpu_object_weights_mobiledet', 'platerec_min_score', 'tpu_object_labels', 'frame_strategy', 'alpr_detection_pattern', 'platerec_stats', 'openalpr_cmdline_min_confidence', 'yolo4_object_processor', 'cpu_max_processes', 'show_models', 'tpu_face_weights_mobilenetv2', 'face_model', 'gpu_max_lock_wait', 'tpu_max_processes', 'platerec_min_dscore', 'face_detection_pattern', 'yolo4_object_framework', 'object_detection_pattern', 'save_unknown_faces', 'face_num_jitters', 'known_images_path', 'contained_area', 'object_min_confidence', 'openalpr_cmdline_params', 'face_detection_framework', 'tpu_min_confidence', 'tpu_max_lock_wait', 'tpu_object_weights_mobilenetv2', 'disable_locks', 'model_sequence', 'face_upsample_times', 'tpu_tf2_mobilenetv2', 'match_past_detections', 'face_train_model', 'cpu_max_lock_wait', 'max_detection_size', 'yolo4_object_weights', 'smart_fps_thresh', 'tpu_object_framework', 'yolo4_object_config', 'same_model_sequence_strategy', 'save_unknown_faces_leeway_pixels', 'alpr_key', 'frame_set', 'yolo4_object_labels']]

11/19/21 00:19:07.352713 zmesdetect_m6[1320] DBG1 new_yaml.py:187->[config:zmes:proc: Base config has been built, all properly configured {[secrets]} and {{vars}} have been replaced! Remember you must manually build per monitor overrode configurations]

11/19/21 00:19:07.353266 zmesdetect_m6[1320] DBG1 zm_detect.py:542->[perf:zmes: building the intial config took 0.105242 ]

11/19/21 00:19:07.353795 zmesdetect_m6[1320] DBG1 new_yaml.py:762->[zmes:api create: building ZM API Session]

11/19/21 00:19:07.36206 zmesdetect_m6[1320] DBG1 api.py:114->[api: SSL certificate verification disabled (encryption enabled, vulnerable to MITM attacks)]

11/19/21 00:19:07.362869 zmesdetect_m6[1320] DBG1 api.py:283->[api: no token found, trying user/pass for login]

11/19/21 00:19:07.43918 zmesdetect_m6[1320] DBG1 api.py:320->[api: detected API ver 2.0+, using token system]

11/19/21 00:19:07.439701 zmesdetect_m6[1320] DBG1 api.py:335->[api: access token expires on: 2021-11-19 02:19:07.439677 (7200s)]

11/19/21 00:19:07.440054 zmesdetect_m6[1320] DBG1 api.py:347->[api: refresh token expires on: 2021-11-20 00:19:07.440045 (86400s)]

11/19/21 00:19:07.452768 zmesdetect_m6[1320] DBG1 new_yaml.py:795->[zmes:api create: ZM API created]

11/19/21 00:19:07.696452 zmesdetect_m6[1320] DBG1 ZMLog:336->[------|  FORKED NEO --- app->Hooks: 7.0.3 - pyzm: 1.0.3 - ES: 7.0.3 - OpenCV:4.5.4 |------]

11/19/21 00:19:07.696905 zmesdetect_m6[1320] DBG1 ZMLog:336->[zmes: Setting up signal handlers for log 'rotation' and log 'interrupt']

11/19/21 00:19:07.533195 zmesdetect_m6[1320] DBG1 zm_detect.py:654->[zmes: this is a 'PAST' (debugging?) event!]

11/19/21 00:19:07.697927 zmesdetect_m6[1320] DBG1 ZMLog:336->[|----------= Encrypted Route Name: 'mlapi_one' | Gateway URL: 'http://192.168.1.100:5003/api/v1' | Weight: 0 =----------|]

11/19/21 00:19:07.535745 zmesdetect_m6[1320] DBG1 zm_detect.py:103->[zmes:mlapi:login: No need to login, access token is valid for 1111.4444606304169 sec]

11/19/21 00:19:07.645837 zmesdetect_m6[1320] DBG1 zm_detect.py:318->[
** Gateway URL: 'http://192.168.1.100:5003/api/v1/detect/object?type=object' using auth_header={'Authorization': 'Bearer eyJ0eXAiOiJKV1QiLCJhbGc......'} 
**** params={'delete': True, 'response_format': 'zm_detect'}
****** JSON: stream: 9 - mid: 6 - reason: None - stream options: {'frame_strategy': 'first', 'frame_set': 'snapshot, alarm, snapshot', 'contig_frames_before_error': 2, 'delay_between_attempts': 2.143256, 'max_attempts': 3, 'delay_between_snapshots': 1, 'smart_fps_thresh': '5', 'save_frames': 'no', 'save_frames_dir': None, 'PAST_EVENT': True} - files: {}
]

11/19/21 00:19:07.699787 zmesdetect_m6[1320] DBG1 ZMLog:336->[Connected to ZoneMinder Logging system with user 'www-data' -> '/var/log/zm/zmesdetect_m6.log']

11/19/21 00:19:09.445331 zmesdetect_m6[1320] DBG1 zm_detect:383->[zmes:mlapi: parsed JSON detection data in multipart response]

11/19/21 00:19:09.445757 zmesdetect_m6[1320] DBG1 zm_detect:379->[zmes:mlapi: parsed the matched image from bytes to a jpeg encoded array in multipart response]

11/19/21 00:19:09.477776 zmesdetect_m6[1320] DBG1 zm_detect:765->[perf:zmes:mlapi: Monitor: office (6)->'Event': 9 mlapi detection took: 1.937886]

11/19/21 00:19:09.478149 zmesdetect_m6[1320] DBG1 pyzm_utils:1336->['success'->  True  ]

11/19/21 00:19:09.478386 zmesdetect_m6[1320] DBG1 pyzm_utils:1297->[--- --- ---]
11/19/21 00:19:09.47862 zmesdetect_m6[1320] DBG1 pyzm_utils:1298->['matched_data'->  labels-->['person']  ]
11/19/21 00:19:09.478868 zmesdetect_m6[1320] DBG1 pyzm_utils:1298->['matched_data'->  model_names-->['yolo[GPU]']  ]
11/19/21 00:19:09.479101 zmesdetect_m6[1320] DBG1 pyzm_utils:1298->['matched_data'->  confidences-->[0.7246050238609314]  ]
11/19/21 00:19:09.47933 zmesdetect_m6[1320] DBG1 pyzm_utils:1298->['matched_data'->  frame_id-->s-190  ]
11/19/21 00:19:09.479557 zmesdetect_m6[1320] DBG1 pyzm_utils:1298->['matched_data'->  type-->['object']  ]
11/19/21 00:19:09.479791 zmesdetect_m6[1320] DBG1 pyzm_utils:1298->['matched_data'->  boxes-->[[-28, 294, 1342, 1076]]  ]
11/19/21 00:19:09.480028 zmesdetect_m6[1320] DBG1 pyzm_utils:1298->['matched_data'->  image_dimensions-->{'original': [1080, 1920], 'resized': None}  ]
11/19/21 00:19:09.480262 zmesdetect_m6[1320] DBG1 pyzm_utils:1298->['matched_data'->  polygons-->[{'name': 'full_image', 'value': [[0, 0], [1920, 0], [1920, 1080], [0, 1080]], 'pattern': None}]  ]
11/19/21 00:19:09.480491 zmesdetect_m6[1320] DBG1 pyzm_utils:1298->['matched_data'->  image--><uint-8 encoded jpg>  ]

11/19/21 00:19:09.480547 zmesdetect_m6[1320] INF zm_detect:864->[zmes:prediction: '[s-190] person(72%)']

11/19/21 00:19:09.480981 zmesdetect_m6[1320] DBG1 zm_detect:866->[zmes:prediction:JSON: {"frame_id": "190", "labels": ["person"], "confidences": [0.7246050238609314], "boxes": [[-28, 294, 1342, 1076]], "image_dimensions": {"original": [1080, 1920], "resized": null}}]

11/19/21 00:19:09.483525 zmesdetect_m6[1320] DBG1 zm_detect:1218->[zmes: writing objects.json and objdetect.jpg to '/var/cache/zoneminder/events/6/2021-11-18/9']

11/19/21 00:19:09.491508 zmesdetect_m6[1320] DBG2 zm_detect:1390->[api:make_req: 'put'->http://192.168.1.100:8089/zm/api/events/9.json payload={'Event[Notes]': '[s-190] :detected:person(72%) Motion: All'} query={'token': 'eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJab25lTWluZGVyIiwiaWF0IjoxNjM3MjY2NzQ3LCJleHAiOjE2MzcyNzM5NDcsInVzZXIiOiJqdWFuIiwidHlwZSI6ImFjY2VzcyJ9.cI91Fc6v-kU_joD8SfdRxrSUMe3Gh4eRpf9jd94vPn8...'}]

11/19/21 00:19:09.509041 zmesdetect_m6[1320] DBG1 zm_detect:1401->[zmes: replaced old note -> '[a-5] :detected:person(82%) Motion: All' with new note -> '[s-190] :detected:person(72%) Motion: All']

11/19/21 00:19:09.511193 zmesdetect_m6[1320] DBG1 zm_detect:1601->[perf:zmes:FINAL: 'Monitor': office (6)->'Event': 9-->'Frame ID':190 [PAST event] [total:2.264011] [detection:1.937886] [after core detection: 0.031334] ]

11/19/21 00:19:09.512506 zmesdetect_m6[1320] DBG1 zm_detect:1602->[ZM Log: Closing all log handlers NOW]
root@98ee1fddfdeb:/var/lib/zmeventnotification#

edit:
sorry spoke to fast
no error
but not detecting person. it provides %
this event already detected a person before

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 8:29 pm
by tsp84
I wont touch anything in this master branch since its stable-ish, I will just do bug fixes and stuff.

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 8:33 pm
by juan11perez
ok. tested more and now it''s working
it's detecting and no errors
will you push that fix to master?

Re: neo-ZMES - Public Beta

Posted: Thu Nov 18, 2021 8:37 pm
by tsp84
Pushed a min ago, sorry I was pushing to my private repo when I thought I was pushing to the public one
All times are UTC
Page 5 of 8

Powered by phpBB® Forum Software © phpBB Limited