How do I enable FFMPEG cuda acceleration?

[DonZalmrol]

Hi,

I have an Nvidia NVS510 Quadro card which support CUDA. The drivers are installed and CUDA has been enabled.

• NVIDIA-SMI 460.32.03

• Driver Version: 460.32.03

• CUDA Version: 11.2

NVCC output:

Code: Select all

/usr/local/cuda-11.2/bin/nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Jan_28_19:32:09_PST_2021
Cuda compilation tools, release 11.2, V11.2.142
Build cuda_11.2.r11.2/compiler.29558016_0

FFMPEG output:

Code: Select all

/usr/bin/ffmpeg -hwaccels
ffmpeg version 4.1.6-1~deb10u1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --prefix=/usr --extra-version='1~deb10u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Hardware acceleration methods:
vdpau
vaapi
drm

As you can see CUDA is not listed in my FFMPEG acceleration output. How do I enable this?
When I do google searches you come out on various sources saying that you need to recompile it to include the cuda support.

However they do not inform you on how you can do it and test it...

[lightguy48]

I followed the instructions on this page, skipping the notes regarding a VMWare environment -

https://wiki.zoneminder.com/GPU_passthrough_in_VMWare

So basically starting with the NVIDIA drivers and CUDA section

I did notice a slight improvement in Load values but it wasn't a night and day difference.

[DonZalmrol]

Does this also applies to none VMWare ESX servers?
I have ZM running dedicated on a custom server with latest Debian installed (no virtualization).

I've tried this source, but to no avail.
https://www.reddit.com/r/ZoneMinder/com ... _with_gpu/

[DonZalmrol]

I was able to install libnvidia-encode1 on my Debian as an alternative to the libnvidia-decode-440 which includes libcuda and libcuvid.
https://packages.debian.org/buster/libnvidia-encode1

After a reboot, ffmpeg still has no cuda/cuvid acceleration. Probably I need to install or configure something additionally for ffmpeg.

Code: Select all

ffmpeg -hwaccels

ffmpeg version 4.1.6-1~deb10u1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --prefix=/usr --extra-version='1~deb10u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
  
  Hardware acceleration methods:
vdpau
vaapi
drm

I also checked my /dev/dri devices:

Code: Select all

ls -l /dev/dri/
total 0
drwxr-xr-x 2 root root        140 Mar  9 18:04 by-path
crw-rw---- 1 root video  226,   0 Mar  9 18:04 card0
crw-rw---- 1 root video  226,   1 Mar  9 18:04 card1
crw-rw---- 1 root video  226,   2 Mar  9 18:04 card2
crw-rw---- 1 root render 226, 128 Mar  9 18:04 renderD128
crw-rw---- 1 root render 226, 129 Mar  9 18:04 renderD129

By that output I believe that my Nvidia (Quadro) NVS510 is card1
As the lspci grepped output is: 01:00:00

Code: Select all

lspci | grep VGA
01:00.0 VGA compatible controller: NVIDIA Corporation GK107 [NVS 510] (rev a1)
03:00.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200e [Pilot] ServerEngines (SEP1) (rev 05)

Nvidia card details:

Code: Select all

sudo lspci -v -s 01:00.0
01:00.0 VGA compatible controller: NVIDIA Corporation GK107 [NVS 510] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: NVIDIA Corporation GK107 [NVS 510]
        Flags: bus master, fast devsel, latency 0, IRQ 141
        Memory at b2000000 (32-bit, non-prefetchable) [size=16M]
        Memory at 90000000 (64-bit, prefetchable) [size=256M]
        Memory at a0000000 (64-bit, prefetchable) [size=32M]
        I/O ports at 6000 [size=128]
        [virtual] Expansion ROM at b3080000 [disabled] [size=512K]
        Capabilities: [60] Power Management version 3
        Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [78] Express Endpoint, MSI 00
        Capabilities: [b4] Vendor Specific Information: Len=14 <?>
        Capabilities: [100] Virtual Channel
        Capabilities: [128] Power Budgeting <?>
        Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
        Kernel driver in use: nvidia
        Kernel modules: nvidia

And nvidia-detect

Code: Select all

nvidia-detect
Detected NVIDIA GPUs:
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107 [NVS 510] [10de:0ffd] (rev a1)

Checking card:  NVIDIA Corporation GK107 [NVS 510] (rev a1)
Your card is supported by all driver versions.
It is recommended to install the
    nvidia-driver

Then set the following in my ZM (v1.34.23) one camera to the following:
DecoderHWAccelName = cuda
DecoderHWAccelDevice = tried empty, tried /dev/dri/card0,1 and 2

Nvidia-smi and nvtop aren't reporting any services that use the card (which makes sense since ffmpeg has no cuda/ cuvid enabled).

Code: Select all

nvidia-smi
Tue Mar  9 18:47:11 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVS 510             On   | 00000000:01:00.0 N/A |                  N/A |
| 28%   41C    P8    N/A /  N/A |      1MiB /  1999MiB |     N/A      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

[BlyatBlazer91]

Im only commenting because I also have an nvidia quadro, and was able to get it up and running on 1.37 by adding the cuda option to each cameras source with a blank device path. and then i went into options>image>ffmpeg_path and added the extra argument -hwaccel cuda. Im running in a docker container and nvidia-smi doesnt show me any processes using the GPU but in the host, nvidia-smi does show me a process for each camera.

[openman]

I noticed that if cuvid is used in camera's source acceleration method, nvidia-smi reports the connection
A usefull link is also here: https://developer.nvidia.com/blog/nvidi ... ing-guide/

[CocodriloDundee]

From what I know, and I wish the good God send a soul to correct me, zoneminder does not use the operative system ffmpeg binary; so that option PATH_FFMPEG in the options>images is not being used at all. At least in the version v1.36.26, I even pointed that path to a bash script that writes some string in a txt file just to see if the option is being used and I have not notice any call at all from the zoneminder app to that binary path.

I think we have to recompile the hole zoneminder in an environment with a cuda toolkit and an ffmpeg binary with cuda support to have a cuda support in the zoneminder app.

From what I have read, cuda only works in some zoneminder versions and depending on the operating systems. The option that seems more reliable is using the docker version with an ESX or a gpu hardware passthrough.

Currently I am saving only jpgs and no video at all, and when I want a video for an event I manually ssh into the zoneminder directory and I generate manually the video using the cuda cores with this command:

Code: Select all

$ ffmpeg -framerate 25 -pattern_type glob -i '*capture.jpg' -c:v hevc_nvenc out.mp4

But the drawback is that I generate 25GB of images per hour and per camera, which is crazy inefficient. Furthermore, I cannot reduce the fps because those options in zoneminder do not work very well. So right now I am stucked in having a few hours of history events per camera.

Cheers,
CD

[dougmccrary]

First, if using 1.36, you should post there.
Second, you should upgrade to 1.36.34
Third, do you really need 25FPS?
Finally, what OS are you on?

[CocodriloDundee]

First, if using 1.36, you should post there.

Well that is my mistake by using the upper right search box instead of the search box inside the 1.36 section. But having said that, wondering if there is any point in duplicating post entries when the issues are the same? Should I start a new thread in 1.36 copying and pasting things? Isn't util my comments for the original thread creator? that was my idea for the record, I am not properly raising a bug here but sharing what I understand about the zoneminder features and have a talk about the content is published in this thread.

Second, you should upgrade to 1.36.34

That is a common answer I read in the forum, but a few can properly say if this suggested procedure fixes the ffmpeg issue. So although I appreciate this advice I also encourage to think about the people's time doing the update and at the end not having any result. Can you properly mention points in the new release notes that are related to the issues described in this topic?

Third, do you really need 25FPS?

Not at all, that is what I mentioned in my post this

... I cannot reduce the fps because those options in zoneminder do not work very well.

Do you have a procedure in order to reduce the fps when capturing images different than the "Maximum FPS" and "Alarm Maximum FPS"? When I try to use them I have the well known issues regarding them.

Finally, what OS are you on?

Debian 12, no vm, a phisical one

[burger]

See this link which will answer some of your questions regarding where FFMPEG is used.
https://wiki.zoneminder.com/Ffmpeg
There is also this:
https://wiki.zoneminder.com/GPU_passthrough_in_VMWare

[dougmccrary]

Do you have a procedure in order to reduce the fps when capturing images different than the "Maximum FPS" and "Alarm Maximum FPS"? When I try to use them I have the well known issues regarding them.

I seems to work for me in 1.36 and 1.37.

And where you do the manual ffmpeg, I meant you could probably put 4 or 5 instead of 25.

[iconnor]

ZM does not need to be recompiled to use cuda. It depends on the installed system libraries. So they need to be ones that have cuda installed. If the ffmpeg command line works, then zm will too.

The ffmpeg binary is used in some scripts for generating videos from the jpgs. That is all. We use the libraries that come with ffmpeg for everything else.

[CocodriloDundee]

Hi everybody,
This week I reinstalled everything from zero updating all versions, I finally could get the cuda working (running zm in native host, no docker). This are the installed software and versions in order in case anyone needs it.

1) Debian-12.5.0-amd64 (Bookworm)
2) Zoneminder 1.36.33 installation guide for Debian 12 (Bookworm) (https://zoneminder.readthedocs.io/en/st ... 2-bookworm)
3) The zoneminder installation installed between dependencies the ffmpeg. At this point I have to say that after running "ffmpeg -hwaccels" I saw "cuda" in the "Hardware acceleration methods", which was new as in previous version this option was not available.

Code: Select all

$ ffmpeg -hwaccels
ffmpeg version 5.1.4-0+deb12u1 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 12 (Debian 12.2.0-14)
...
...
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Hardware acceleration methods:
vdpau
cuda
vaapi
qsv
drm
opencl
vulkan

4) I tested the ffmpeg with the cuda option but did not work due to the missing library 'libcuda.so.1'. So a this point I proceeded installing the nvidia cuda toolkit 'cuda_12.5.0_555.42.02_linux.run'. (https://developer.nvidia.com/cuda-downloads)
5) I updated the file /etc/ld.so.conf with the cuda library path and then I did run "ldconfig" as root (This is step is mentioned by the cuda installer at the end)

Code: Select all

$ sudo su -
# echo "/usr/local/cuda-12.5/lib64" > /etc/ld.so.conf
# cat  /etc/ld.so.conf
include /etc/ld.so.conf.d/*.conf
/usr/local/cuda-12.5/lib64
# ldconfig
# exit
$ exit

6) After the installation of the cuda toolkit I tested again the ffmpeg cuda option and it started working, so I did not needed to compile ffmpeg or anything like that.
7) I updated the file "/etc/zm/conf.d/03-ffmpeg.conf" with the value ZM_PATH_FFMPEG="/usr/local/bin/ffmpeg"
8 ) I updated the file "/etc/zm/conf.d/01-system-paths.conf" with the value ZM_PATH_FFMPEG="/usr/local/bin/ffmpeg"
9) In zoneminder "options/images" I enabled the option OPT_FFMPEG=yes
10) In zoneminder, for each configured monitor, under the option "source", I configured both attributes "DecoderHWAccelName" and "DecoderHWAccelDevice" to use cuda. I just entered the text "cuda" in the text box of each option.
11) I restarted the pc and I could see cuda was being used by the executable "/usr/bin/zmc".

Code: Select all

$ nvidia-smi
Sun Jun  9 21:40:40 2024       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 555.42.02              Driver Version: 555.42.02      CUDA Version: 12.5     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3060 Ti     Off |   00000000:01:00.0 Off |                  N/A |
| 30%   41C    P0             55W /  200W |     406MiB /   8192MiB |      1%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
                                                                                         
+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|    0   N/A  N/A    115586      C   /usr/bin/zmc                                  212MiB |
|    0   N/A  N/A    128327      C   /usr/bin/zmc                                  180MiB |
+-----------------------------------------------------------------------------------------+

I hope this can be util

Thank you all for the help,
Regards.