ZoneMinder Forums

I have been unable to get face training to work correctly on Ubuntu 20.04. I have tried creating the system twice now and still have the same error with training faces. If I enable face detection in objectconfig.ini I see the same errors everytime a hook is run. Everything else for detection works fine using the hooks. I have been unable to find a solution here or on google. Any help with narrowing down the cause of this would be greatly appreciated. Thank you,

I am running Cuda 11.1 with CuDNN-11.1-linux-x64-v8.0.4.30
OpenCV 4.4.0

Console Output

$ sudo -u www-data /var/lib/zmeventnotification/bin/zm_train_faces.py

CONSOLE:Face Recognition library load time took: 0.004 milliseconds
CONSOLE:secret filename: /etc/zm/secrets.ini
CONSOLE:Secret token found in config: !ZM_PORTAL
CONSOLE:Secret token found in config: !ZM_API_PORTAL
CONSOLE:Secret token found in config: !ZM_USER
CONSOLE:Secret token found in config: !ZM_PASSWORD
CONSOLE:Secret token found in config: !ML_USER
CONSOLE:Secret token found in config: !ML_PASSWORD
CONSOLE:Secret token found in config: !PLATEREC_ALPR_KEY
CONSOLE:allowing self-signed certs to work...
CONSOLE:Ignoring monitor specific settings, as you did not provide a monitor id
CONSOLE:key [config] is '/var/lib/zmeventnotification/models/yolov3/yolov3.cfg' after substitution
CONSOLE:key [weights] is '/var/lib/zmeventnotification/models/yolov3/yolov3.weig hts' after substitution
CONSOLE:key [labels] is '/var/lib/zmeventnotification/models/yolov3/coco.names' after substitution
CONSOLE:key [tiny_config] is '/var/lib/zmeventnotification/models/tinyyolo/yolov 3-tiny.cfg' after substitution
CONSOLE:key [tiny_weights] is '/var/lib/zmeventnotification/models/tinyyolo/yolo v3-tiny.weights' after substitution
CONSOLE:key [tiny_labels] is '/var/lib/zmeventnotification/models/tinyyolo/yolov 3-tiny.txt' after substitution
CONSOLE:key [known_images_path] is '/var/lib/zmeventnotification/known_faces' af ter substitution
CONSOLE:key [unknown_images_path] is '/var/lib/zmeventnotification/unknown_faces ' after substitution
CONSOLE:josh_k is a directory. Processing all images inside it
CONSOLE:loading face from josh_k/1.jpg
CONSOLE:Error initializing face recognition: Error while calling cudaOccupancyMa xPotentialBlockSize(&num_blocks,&num_threads,K) in file /tmp/pip-install-scqomot r/dlib/dlib/cuda/cuda_utils.h:186. code: 98, reason: invalid device function
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/zmes_hook_helpers/face_train.py", line 77, in train
face_encodings = face_recognition.face_encodings(
File "/usr/local/lib/python3.8/dist-packages/face_recognition/api.py", line 21 4, in face_encodings
return [np.array(face_encoder.compute_face_descriptor(face_image, raw_landma rk_set, num_jitters)) for raw_landmark_set in raw_landmarks]
File "/usr/local/lib/python3.8/dist-packages/face_recognition/api.py", line 21 4, in <listcomp>
return [np.array(face_encoder.compute_face_descriptor(face_image, raw_landma rk_set, num_jitters)) for raw_landmark_set in raw_landmarks]
RuntimeError: Error while calling cudaOccupancyMaxPotentialBlockSize(&num_blocks ,&num_threads,K) in file /tmp/pip-install-scqomotr/dlib/dlib/cuda/cuda_utils.h:1 86. code: 98, reason: invalid device function

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/var/lib/zmeventnotification/bin/zm_train_faces.py", line 26, in <module >
train.train()
File "/usr/local/lib/python3.8/dist-packages/zmes_hook_helpers/face_train.py", line 109, in train
raise ValueError(
ValueError: Error opening known faces directory. Is the path correct?


nvidia-smi:

Mon Oct 12 14:11:35 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.23.05 Driver Version: 455.23.05 CUDA Version: 11.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 GeForce GT 710 On | 00000000:03:00.0 N/A | N/A |
| 50% 46C P0 N/A / N/A | 93MiB / 2002MiB | N/A Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+

$ ls -l /var/lib/zmeventnotification/known_faces
total 4
drwxrwxr-x 2 www-data www-data 4096 Oct 11 14:46 josh_k

$ ls -l /var/lib/zmeventnotification/known_faces/josh_k
total 36
-rw-rw-r-- 1 www-data www-data 35437 Oct 11 14:43 1.jpg


zmesdectect logs:

2020-10-12 14:17:18 zmesdetect_m2 102072 ERR Error initializing face recognition: Error while calling cudaOccupancyMaxPotentialBlockSize(&num_blocks,&num_threads,K) in file /tmp/pip-install-scqomotr/dlib/dlib/cuda/cuda_utils.h:164. code: 98, reason: invalid device function face_train.py 108
2020-10-12 14:17:17 zmesdetect_m2 102072 INF Importing local classes for Yolo/Face zm_detect.py 210
2020-10-12 14:17:17 zmesdetect_m2 102072 INF ---------| hook version: 5.15.6, ES version: 5.15 , OpenCV version: 4.4.0|------------ zm_detect.py 181
2020-10-12 14:16:54 zmesdetect_m2 101935 ERR Error initializing face recognition: Error while calling cudaOccupancyMaxPotentialBlockSize(&num_blocks,&num_threads,K) in file /tmp/pip-install-scqomotr/dlib/dlib/cuda/cuda_utils.h:164. code: 98, reason: invalid device function face_train.py 108
2020-10-12 14:16:52 zmesdetect_m2 101935 INF Importing local classes for Yolo/Face zm_detect.py 210
2020-10-12 14:16:52 zmesdetect_m2 101935 INF ---------| hook version: 5.15.6, ES version: 5.15 , OpenCV version: 4.4.0|------------ zm_detect.py 181


Thank you.

As it’s not a ZM matter, you’ll get more response in Mobile Apps section. ZMES is mostly covered there.

There’s been quite a bit of work on the event server and it might be worth trying the newest version first.

Thank you, I will try the mobile forum if I don't get much traction here it just didn't seem like a mobile question to me. I was running v5.15.6 which was the latest stable version when I reinstalled a couple days ago. I took your advice and tried updating to v6.0.1 which was just released 13 minutes before I updated. Still the same result: cudaOccupancyMaxPotentialBlockSize invalid device function error.

The author of ZMES and ZMNinja is more likely to see it in that section.