I'm thinking about going back to 1.34

[rilny]

I'm a longtime ZM user, running three sites now. All with Foscam cameras (mostly wired), two on Fedora and one CentOS. I'm having a whole lot of video issues since upgrading to 1.36:

• Short segments due to monitors crashing frequently
• "Unable to free up older packets" error (even with analysis frames turned off)
• Smudging/noise artifacts
• Various other error messages when capturing video, including "Error writing trailer," "Error occurred when writing out file header," "Failed to capture image from monitor," "Invalid data found when processing input"

The worst ZM instance is a lightly-loaded system with only one camera and plenty of CPU and RAM. I've tried tuning various buffer settings and source parameters (movflags) but nothing makes a difference.

Looking through the forums, there are several threads describing similar issues, and nobody seems to have any luck getting them resolved. I don't think I'd be the only person with a stable 1.34 setup who's been unhappy since upgrading.

Personally, I love the UI improvements in 1.36 but it doesn't matter if the video capture is broken. Are we at all close to figuring out what happened between 1.34 and 1.36 to cause this instability?

[dougmccrary]

Did you reinstall or upgrade?
If reinstall did you import the DB or do it manually?
Is the one with one can centos?

[rilny]

I just replaced that server last night and I started from scratch, clean install of F34, created a new database and didn't import anything. It's running 1.36.8 clean out of the box. I have the same issues on all three servers but the monitor crashes most frequently on this one.

If you browse this forum, you can find a ton of similar reports but no resolution. It looks like ffmpeg in pass-through mode is something they all have in common.

[Magic919]

I don’t think the devs have enough to go on. There are quite possibly things they could fix, but they’ll need debug logs and a bit of help. Grab some logs and post on the Slack channel would be my suggestion.

[guynew]

I am using a Reolink 520 camera and had lots of Smudging/noise artifacts using 1.36 (running on Ubuntu). I solved this by changing the monitor source->method from TCP to UDP.

[iconnor]

There is an issue where if analysis falls behind, and maximagebuffer is set too low, that none of the processes can continue. So zmwatch kills it and restarts.

In 1.34 there was no concept of locking so frames would just be lost.

The quick fix is to set MaxImagebuffer = 2*keyframe interval+1 or larger.

I also have a fix in master that fixes the lockup situation by simply removing frames from the beginning of the queue but keeping the needed keyframe.

If you are having lockups... increase MaxImageBuffer. Please note: in the past ImageBUffer was about raw frames, 4*width*height. MaxImageBuffer is about encoded frames.. and should take not that much ram.

However if for some reason analysis can't keep up, you are going to run into problems. The consumer HAS to keep up with capture.

[rilny]

Earlier today I wiped my database and reinstalled 1.36.8 from scratch. This is with just one camera, on a quad-core machine with 8GB doing almost nothing else. The camera is a Foscam bullet cam with these settings:

Stream Type: FHD Mode
Resolution: 720P
Bit Rate: 1M
Frame Rate: 5
Key Frame Interval: 20
Variable bitrate: Yes

My camera is set for Mocord with the following:

Source: Ffmpeg/TCP (analysis enabled)
Capture resolution: 720p
Save JPEGs: Analysis images only
Video writer: Camera passthrough (crf=23, no other encoder options)
Image Buffer Size: 10
Maximum Image Buffer Size: 200
Warmup Frames: 0
Pre Event Image Count: 10
Post Event Image Count: 60
Stream Replay Image Buffer: 0
Alarm Frame Count: 2
Event Section length: 120 seconds

EVERY event recording, even those that go the full 120 seconds, ends with this error:

Code: Select all

10/25/21 17:36:32.266660 zmc_m1[3962].INF-zm_videostore.cpp/514 [some options not used, turn on debugging for a list.]
10/25/21 17:36:32.338046 zmc_m1[3962].INF-zm_monitor.cpp/2023 [cam-front: 20155 - Opened new event 53, section start]
10/25/21 17:36:38.647248 zmc_m1[3593].INF-zm_monitor.cpp/1663 [cam-front: 20400 - Capturing at 5.01 fps, capturing bandwidth 111885bytes/sec]
10/25/21 17:36:40.865358 zmc_m1[3962].INF-zm_monitor.cpp/1704 [cam-front: 20200 - Analysing at 5.12 fps from 19909 - 19809=100 / 1635197800.865354 - 1635197781.324776 = 19.540578]
10/25/21 17:36:58.640712 zmc_m1[3593].INF-zm_monitor.cpp/1663 [cam-front: 20500 - Capturing at 5.00 fps, capturing bandwidth 112750bytes/sec]
10/25/21 17:37:00.866756 zmc_m1[3962].INF-zm_monitor.cpp/1704 [cam-front: 20300 - Analysing at 5.00 fps from 20009 - 19909=100 / 1635197820.866752 - 1635197800.865354 = 20.001398]
10/25/21 17:37:18.638790 zmc_m1[3593].INF-zm_monitor.cpp/1663 [cam-front: 20600 - Capturing at 5.00 fps, capturing bandwidth 111900bytes/sec]
10/25/21 17:37:21.244682 zmc_m1[3962].INF-zm_monitor.cpp/1704 [cam-front: 20400 - Analysing at 4.91 fps from 20109 - 20009=100 / 1635197841.244678 - 1635197820.866752 = 20.377926]
10/25/21 17:37:38.627380 zmc_m1[3593].INF-zm_monitor.cpp/1663 [cam-front: 20700 - Capturing at 5.00 fps, capturing bandwidth 111299bytes/sec]
10/25/21 17:37:40.849316 zmc_m1[3962].INF-zm_monitor.cpp/1704 [cam-front: 20500 - Analysing at 5.10 fps from 20209 - 20109=100 / 1635197860.849312 - 1635197841.244678 = 19.604634]
10/25/21 17:37:58.624613 zmc_m1[3593].INF-zm_monitor.cpp/1663 [cam-front: 20800 - Capturing at 5.00 fps, capturing bandwidth 111331bytes/sec]
10/25/21 17:38:00.854275 zmc_m1[3962].INF-zm_monitor.cpp/1704 [cam-front: 20600 - Analysing at 5.00 fps from 20309 - 20209=100 / 1635197880.854271 - 1635197860.849312 = 20.004959]
10/25/21 17:38:18.626395 zmc_m1[3593].INF-zm_monitor.cpp/1663 [cam-front: 20900 - Capturing at 5.00 fps, capturing bandwidth 111494bytes/sec]
10/25/21 17:38:20.848442 zmc_m1[3962].INF-zm_monitor.cpp/1704 [cam-front: 20700 - Analysing at 5.00 fps from 20409 - 20309=100 / 1635197900.848438 - 1635197880.854271 = 19.994167]
10/25/21 17:38:28.059733 zmc_m1[3962].INF-zm_monitor.cpp/1949 [cam-front: 20947 - Closing event 53, section end forced 1635197908 - 1635197788 = 120 >= 120]
10/25/21 17:38:28.060774 zmc_m1[4020].ERR-zm_videostore.cpp/642 [Error writing trailer Error number 618 occurred]

But I also get frequent short segments that look like this:

Code: Select all

10/25/21 17:20:20.980765 zmc_m1[3926].INF-zm_videostore.cpp/514 [some options not used, turn on debugging for a list.]
10/25/21 17:20:21.148139 zmc_m1[3926].INF-zm_monitor.cpp/2023 [cam-front: 15440 - Opened new event 44, section start]
10/25/21 17:20:54.025753 zmc_m1[3593].INF-zm_ffmpeg_camera.cpp/226 [Unable to read packet from stream 0: error -541478725 "End of file".]
10/25/21 17:20:54.028756 zmc_m1[3593].ERR-zmc.cpp/305 [Failed to capture image from monitor 1 cam-front (1/1)]
10/25/21 17:20:54.029206 zmc_m1[3593].INF-zm_monitor.cpp/3111 [cam-front: image_count:15677 - Closing event 44, shutting down]
10/25/21 17:20:54.030418 zmc_m1[3960].ERR-zm_videostore.cpp/642 [Error writing trailer Error number 86 occurred]

I'm not able to replay the short events typically. I see the first frame and then an error message: The video playback was aborted due to a corruption problem or because the video used features your browser did not support.

Also I'm getting occasional smearing of the kind pictured here, even though I'm already using TCP and all of the other suggestions on this page. I think this usually happens on short events. I tried switching to libvlc but couldn't get an image at all.

Finally... when viewing frames, I will sometimes get this error in the logfile:

Code: Select all

web_php[5402]: 10/25/21 23:17:33.094302 web_php[5402].FAT [10.0.0.228] [Can't create frame images from video for this event 32-video.mp4
web_php[5402]:
web_php[5402]: Command was: /usr/bin/ffmpeg -ss 64.45 -i /home/zoneminder/1/2021-10-25/32/32-video.mp4 -frames:v 1 /home/zoneminder/1/2021-10-25/32/00109-capture.jpg 2>&1
web_php[5402]:
web_php[5402]: Output was: ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers
web_php[5402]:  built with gcc 11 (GCC)
web_php[5402]:  configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --docdir=/usr/share/doc/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -flto=auto -ffat-l
to-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection' --extra-ldflags='-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld ' --extra-cflags=' -I/usr/include/rav1e' --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvo-amrwbenc --enable-version3 --enable-bzlib --disable-crystalhd --enable-fontconfig --enable-frei0r --enable-gcrypt --enable-gnutls --enable-ladspa --enable-libaom --enable-libdav1d --enable-libass --enable-libbluray --enable-libcdio --enable-libdrm --enable-libjack --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-nvenc --enable-openal --enable-opencl --enable-opengl --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librav1e --enable-libsmbclient --enable-version3 --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-version3 --enable-vapoursynth --enable-libvpx --enable-vulkan --enable-libglslang --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzvbi --enable-lv2 --enable-avfilter --enable-avresample --enable-libmodplug --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-lto --enable-libmfx --enable-runtime-cpudetect
web_php[5402]:  libavutil      56. 70.100 / 56. 70.100
web_php[5402]:  libavcodec     58.134.100 / 58.134.100
web_php[5402]:  libavformat    58. 76.100 / 58. 76.100
web_php[5402]:  libavdevice    58. 13.100 / 58. 13.100
web_php[5402]:  libavfilter     7.110.100 /  7.110.100
web_php[5402]:  libavresample   4.  0.  0 /  4.  0.  0
web_php[5402]:  libswscale      5.  9.100 /  5.  9.100
web_php[5402]:  libswresample   3.  9.100 /  3.  9.100
web_php[5402]:  libpostproc    55.  9.100 / 55.  9.100
web_php[5402]: Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/zoneminder/1/2021-10-25/32/32-video.mp4':
web_php[5402]:  Metadata:
web_php[5402]:    major_brand     : isom
web_php[5402]:    minor_version   : 512
web_php[5402]:    compatible_brands: isomiso6iso2avc1mp41
web_php[5402]:    title           : Zoneminder Security Recording
web_php[5402]:    encoder         : Lavf58.76.100
web_php[5402]:  Duration: 00:01:01.21, start: 0.000000, bitrate: 343 kb/s
web_php[5402]:  Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuvj420p(pc), 1280x720, 342 kb/s, 1.88 fps, 5 tbr, 90k tbn, 180k tbc (default)
web_php[5402]:    Metadata:
web_php[5402]:      handler_name    : VideoHandler
web_php[5402]:      vendor_id       : [0][0][0][0]
web_php[5402]: Stream mapping:
web_php[5402]:  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
web_php[5402]: Press [q] to stop, [?] for help
web_php[5402]: [h264 @ 0x55ff9d8eef80] out of range intra chroma pred mode
web_php[5402]: [h264 @ 0x55ff9d8eef80] error while decoding MB 27 12
web_php[5402]: [h264 @ 0x55ff9d8eef80] concealing 2662 DC, 2662 AC, 2662 MV errors in I frame
web_php[5402]: /home/zoneminder/1/2021-10-25/32/32-video.mp4: corrupt decoded frame in stream 0
web_php[5402]: [h264 @ 0x55ff9dbfbb80] dquant out of range (5794) at 15 5
web_php[5402]: [h264 @ 0x55ff9dbfbb80] error while decoding MB 15 5
web_php[5402]: [h264 @ 0x55ff9dbfbb80] concealing 3234 DC, 3234 AC, 3234 MV errors in P frame
web_php[5402]: /home/zoneminder/1/2021-10-25/32/32-video.mp4: corrupt decoded frame in stream 0
web_php[5402]: Output #0, image2, to '/home/zoneminder/1/2021-10-25/32/00109-capture.jpg':
web_php[5402]:  Metadata:
web_php[5402]:    major_brand     : isom
web_php[5402]:    minor_version   : 512
web_php[5402]:    compatible_brands: isomiso6iso2avc1mp41
web_php[5402]:    title           : Zoneminder Security Recording
web_php[5402]:    encoder         : Lavf58.76.100
web_php[5402]:  Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 1280x720, q=2-31, 200 kb/s, 5 fps, 5 tbn (default)
web_php[5402]:    Metadata:
web_php[5402]:      handler_name    : VideoHandler
web_php[5402]:      vendor_id       : [0][0][0][0]
web_php[5402]:      encoder         : Lavc58.134.100 mjpeg
web_php[5402]:    Side data:
web_php[5402]:      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
web_php[5402]: frame=    0 fps=0.0 q=0.0 size=N/A time=00:00:00.00 bitrate=N/A speed=   0x    \x0dframe=    0 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.00 bitrate=N/A speed=   0x
web_php[5402]: video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
web_php[5402]: Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)] at /usr/share/zoneminder/www/views/image.php line 282

[iconnor]

Need debug logs. With FFMPEG logging on.

[rilny]

I already have LOG_LEVEL_FILE = Debug and LOG_FFMPEG turned on and the logs above are all that I have. Can you tell me which other options should be set?

[iconnor]

there should be a file in /var/log/zm/zmc_m[monitor id].log