Does ZM 1.30.x use hardware acceleration?

[dingbatca]

I have been hunting around the web looking for any information on ZoneMinder and hardware acceleration. Does ZM use hardware acceleration?

I am using ffmpeg as my Source Type. I know ffmpeg can leverage some hardware acceleration, if compiled for it. But I am not sure of the relationship between ZM and ffmpeg.

Does anyone have some good info on this?

[PacoLM]

Then, the question is if the installed ffmpeg version is using it, not if ZM is... isn´t it?

Regards,
PacoLM

[dingbatca]

Kinda. Need to make sure that the options for ffmpeg include acceleration....

Passing option to ffmpeg Edit Camera --> Source tab --> Options, and there is even a hand help (?)

Code: Select all

OPTIONS_FFMPEG
Parameters in this field are passed on to FFmpeg. Multiple parameters can be separated by ,
Examples (do not enter quotes)

"allowed_media_types=video" Set datatype to request fromcam (audio, video, data)

"reorder_queue_size=nnn" Set number of packets to buffer for handling of reordered packets

"loglevel=debug" Set verbosity of FFmpeg (quiet, panic, fatal, error, warning, info, verbose, debug)

So here is the "FFmpeg provides a subsystem for hardware acceleration" https://trac.ffmpeg.org/wiki/HWAccelIntro

Now I just need to scrounge up a graphics card that supports both hardware acceleration and PCI passthrough, as my ZM install is a VM.

[dingbatca]

To make a very long story short, I got FFMpeg hardware acceleration running but had to compile my own version of FFMpeg to get cuvid support.

Code: Select all

root@zoneminder:~# ffmpeg -hwaccels
ffmpeg version 3.2.4 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --enable-nonfree --disable-shared --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags=-Ilocal/include
  libavutil      55. 34.101 / 55. 34.101
.......
  libswresample   2.  3.100 /  2.  3.100
Hardware acceleration methods:
vdpau
vaapi
cuvid

CPU usage before/after.

Code: Select all

120%  ffmpeg -i rtsp://admin:password@192.168.1.18:554/Streaming/Channels/1 -an -f rawvideo -y /dev/null
30%    ffmpeg -c:v h264_cuvid -i rtsp://admin:password@192.168.1.18:554/Streaming/Channels/1 -an -f rawvideo -y /dev/null

But when I try this to apply this to ZoneMinder, I get stuck. If I edit a camera, source, "Options" I get an "Unable to parse ffmpeg option 0 '-c:v h264_cuvid', expecting key=value" in the logs, which makes some sense. I think the Option under the camera only apply to the RTSP portion of FFMpeg.

When I apply this option to ZoneMinder, Options, Images, FFMPEG_INPUT_OPTIONS. I don't see any kind of effect.

At this point I am not sure where to go. I need help. Any ideas from those who know the deep magic that makes ZoneMinder work?

[SteveGilvarry]

You were close to what the options under camera are controlling, it is setting AVFormatContext so options for AVFormat lib only.
You would need to pass the codec into avcodec, but all the examples I have seen seem to be code based solutions to use it not as simple as setting the option. I think the fact that your need to compile ffmpeg to enable it is a barrier to most people using it. I share your interest in getting it to work, just finding the time to play with it, and would really like to see the distros enable it in their packages.

[pgrunwald]

What happens for you if you put "-hwaccel auto" before your input (-i) ?

[iconnor]

I would like to engage your assistance with all this as well. I got as far as generating an ffmpeg build that had some hwaccel support built in, but then had to move on to other things. Perhaps together we can solve it.

I think what needs to be done is that we will have to package ffmpeg in the zoneminder ppa with as much hwaccell stuff enabled as possible. And then figure out how to use it in zoneminder. I have been stumped by that as well. Perhaps I will find some time this week to look at ffmpeg code and see how it does it.

[dingbatca]

Iconnor. Thanks for running the Ubuntu Zoneminder PPA!

I just got my tech bench computer re-built with an Nvidia card in it (GeForce GT 730). I will document the install/compile process of FFMpeg with CUDA support and post it in this thread (Monday~ish). If anyone could get me a zoneminder style ffmpeg test line that would be great.

This line is just me guessing at what zoneminder does. Something derived from the code inside of zoneminder would be a MUCH better test case.

Code: Select all

ffmpeg -i rtsp://admin:password@192.168.1.18:554/Streaming/Channels/1 -an -f rawvideo -y /dev/null

[dingbatca]

This took a bit of doing... Here are my tech notes for those who follow:

This is all done on my tech bench computer:

Code: Select all

Ubuntu 16.04 X86_64
4.4.0-66-generic #87-Ubuntu SMP Fri Mar 3 15:29:05 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz
4GB RAM
16GB SSD
NVIDIA Corporation GK208 [GeForce GT 730] (rev a1)

Just my normal opening moves

Code: Select all

#Clean install of Ubuntu 16.04
apt-get update
apt-get dist-upgrade
reboot

Some pro tips
https://developer.nvidia.com/ffmpeg

Install the Nvidia drivers as you see fit. Use nvidia-smi to confirm functionality

Code: Select all

nvidia-smi 
Mon Mar 27 10:28:53 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 378.13                 Driver Version: 378.13                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GT 730      Off  | 0000:01:00.0     N/A |                  N/A |
| 40%   16C    P8    N/A /  N/A |     50MiB /  1995MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0                  Not Supported                                         |
+-----------------------------------------------------------------------------+

Install all the stuff needed to support the ffmpeg install, configure, compile, install

Code: Select all

apt-get install cmake g++ libqt4-dev libasound2-dev libass-dev libcdio-dev libcddb2-dev libsidplayfp-dev libgme-dev libxv-dev libtag1-dev libpulse-dev libva-dev libvdpau-dev libavcodec-dev libavformat-dev libavutil-dev libswscale-dev libswresample-dev libavdevice-dev yasm libssh-dev nvidia-cuda-dev nvidia-cuda-toolkit git

git clone https://github.com/FFmpeg/FFmpeg.git
cd FFmpeg/

#WARNING, note the hard coded in path.  Please update for your enviroment: /usr/lib/nvidia-375
LIBRARY_PATH=/usr/lib/nvidia-378:$LIBRARY_PATH ./configure --enable-nonfree --disable-shared --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags=-Ilocal/include

#Tweak -jX for the number of CPUs +1
LIBRARY_PATH=/usr/lib/nvidia-378:$LIBRARY_PATH make -j5

make install

ok, now that I have this installed. Lets check both version:

Code: Select all

#Stock
/usr/bin/ffmpeg -hwaccels
ffmpeg version 2.8.11-0ubuntu0.16.04.1 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Hardware acceleration methods:
vdpau


#My new compile
/usr/local/bin/ffmpeg -hwaccels
ffmpeg version N-84683-gd7896e9 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --enable-nonfree --disable-shared --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags=-Ilocal/include
  libavutil      55. 51.100 / 55. 51.100
  libavcodec     57. 86.103 / 57. 86.103
  libavformat    57. 68.100 / 57. 68.100
  libavdevice    57.  3.101 / 57.  3.101
  libavfilter     6. 78.100 /  6. 78.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
Hardware acceleration methods:
vdpau
vaapi
cuvid

Now the test. I like the 50% savings!

Code: Select all

46~50% CPU usage: /usr/bin/ffmpeg -i rtsp://admin:password@192.168.1.18:554/Streaming/Channels/1 -an -f rawvideo -y /dev/null
46~50% CPU usage: /usr/local/bin/ffmpeg -i rtsp://admin:password@192.168.1.18:554/Streaming/Channels/1 -an -f rawvideo -y /dev/null
21~22% CPU usage: /usr/local/bin/ffmpeg -c:v h264_cuvid -i rtsp://admin:password@192.168.1.18:554/Streaming/Channels/1 -an -f rawvideo -y /dev/null

-hwaccel auto, 46~50% CPU usage

Code: Select all

/usr/local/bin/ffmpeg -hwaccel auto -i rtsp://admin:password@192.168.1.18:554/Streaming/Channels/1 -an -f rawvideo -y /dev/null
ffmpeg version N-84683-gd7896e9 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --enable-nonfree --disable-shared --enable-nvenc --enable-cuda --enable-cuvid --enable-libnpp --extra-cflags=-Ilocal/include
  libavutil      55. 51.100 / 55. 51.100
  libavcodec     57. 86.103 / 57. 86.103
  libavformat    57. 68.100 / 57. 68.100
  libavdevice    57.  3.101 / 57.  3.101
  libavfilter     6. 78.100 /  6. 78.100
  libswscale      4.  3.101 /  4.  3.101
  libswresample   2.  4.100 /  2.  4.100
[rtsp @ 0x33c3be0] max delay reached. need to consume packet
[rtsp @ 0x33c3be0] RTP: missed 20 packets
[h264 @ 0x33c7180] cabac decode of qscale diff failed at 18 92
[h264 @ 0x33c7180] error while decoding MB 18 92, bytestream -5
[h264 @ 0x33c7180] concealing 535 DC, 535 AC, 535 MV errors in P frame
[h264 @ 0x33c7180] Increasing reorder buffer to 1
Input #0, rtsp, from 'rtsp://admin:password@192.168.1.18:554/Streaming/Channels/1':
  Metadata:
    title           : Media Presentation
  Duration: N/A, start: 0.350000, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 2688x1520 [SAR 1:1 DAR 168:95], 20 fps, 20 tbr, 90k tbn, 40 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[AVHWDeviceContext @ 0x7f9ebc311660] Cannot open the X11 display .
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
[AVHWDeviceContext @ 0x7f9ebc3113e0] Failed to initialise VAAPI connection: -1 (unknown libva error).
[vaapi @ 0x16a3ae0] Failed to create a VAAPI device
Output #0, rawvideo, to '/dev/null':
  Metadata:
    title           : Media Presentation
    encoder         : Lavf57.68.100
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuvj420p, 2688x1520 [SAR 1:1 DAR 168:95], q=2-31, 980582 kb/s, 20 fps, 20 tbn, 20 tbc
    Metadata:
      encoder         : Lavc57.86.103 rawvideo
[h264 @ 0x35f0360] cabac decode of qscale diff failed at 18 92
[h264 @ 0x35f0360] error while decoding MB 18 92, bytestream -5
[h264 @ 0x35f0360] concealing 535 DC, 535 AC, 535 MV errors in P frame
[rtsp @ 0x33c3be0] max delay reached. need to consume packet
[rtsp @ 0x33c3be0] RTP: missed 53 packets
[h264 @ 0x3681280] error while decoding MB 3 79, bytestream -11
[h264 @ 0x3681280] concealing 2734 DC, 2734 AC, 2734 MV errors in P frame
frame=  279 fps= 21 q=-0.0 Lsize= 1669815kB time=00:00:13.95 bitrate=980582.4kbits/s dup=1 drop=4 speed=1.07x  

I hope that this info can help some one connect the dots and give us hwaccel in ZoneMinder.

[nirv199]

Was searching this topic and bumped into this post. Just curious if there's any changes being done in zm to support ffmpeg using GPU acceleration?

thanks

[iconnor]

My code will use omx gpu accel on the raspberry pi. Havn't had any success with anything else yet.

[dinskuns]

Have any one succeeded to manage video stream on zm through ffmpeg with hardware acceleration to offload cpu?

[dinskuns]

In a context of https://github.com/ZoneMinder/ZoneMinder/issues/209 which was closed after gathering $50 plus - what do you think will be a valuable bounty amount for the knowing person to implement this feature?
2. While so many requesting this beneficiary feature dont you think we are able to collect that amount?
I am new to this "github" way of open source contracting, so just wonder on how difficult and worthly could it be.

p.s. Can add $100 to bounty

[snake]

Have any one succeeded to manage video stream on zm through ffmpeg with hardware acceleration to offload cpu?

If by video stream, you mean record, and by ffmpeg you mean source type: ffmpeg then yes. With 1.31 or 1.32 and an RPI. See the post here: viewtopic.php?f=34&t=26112&p=106795&hilit=mmal#p106795

It's recommended to use 1.32.

[dinskuns]

I meant the use of pc platform. RPI has a significient memory limit on the number of cams served.