High IO & Tons of errors in log

Forum for questions and support relating to the 1.30.x releases only.
Locked
Tuxxin
Posts: 14
Joined: Fri Jan 06, 2017 3:09 pm

High IO & Tons of errors in log

Post by Tuxxin »

I finally got around to configuring my POE system using ZoneMinder. I currently have 14 POE cameras online with zoneminder running at 1920x1080 32bit color @ 5FPS. The problem I'm running into is I'm getting about 70+ log entries every second in ZoneMinder (see below), montage is not loading more than 5-6 cameras and the ones that do, are snowy (almost like the pepper static from old tv's). When I view a monitor directly, it looks perfectly fine and works without problem. The system is jumping between 30% and 60% iowait (Using a WD Purple 4Tb).

I am new to Zoneminder and camera systems in general so I don't know all these little tricks/tweaks to get things working properly.

Anyone have any suggestions on what I could try?

Much appreciated!

  • 2017-02-28 16:36:13.819064 zmc_m1 19469 WAR Packet in sequence, gap 4 zm_rtp_source.cpp 127
    2017-02-28 16:36:13.819055 zmc_m10 22381 WAR Discarding partial frame 52, 24538 bytes zm_rtp_source.cpp 345
    2017-02-28 16:36:13.818987 zmc_m4 20573 WAR Discarding incomplete frame 7795, 0 bytes zm_rtp_source.cpp 336
    2017-02-28 16:36:13.760685 zmc_m5 21780 WAR Packet in sequence, gap 4 zm_rtp_source.cpp 127
    2017-02-28 16:36:13.752135 zmc_m9 22035 WAR Discarding incomplete frame 3186, 0 bytes zm_rtp_source.cpp 336
    2017-02-28 16:36:13.748443 zmc_m10 22381 WAR Packet in sequence, gap 3 zm_rtp_source.cpp 127
    2017-02-28 16:36:13.685360 zmc_m1 19469 WAR Discarding frame 12543 zm_rtp_source.cpp 349
    2017-02-28 16:36:13.685327 zmc_m4 20573 WAR Discarding partial frame 7795, 81784 bytes zm_rtp_source.cpp 345
    2017-02-28 16:36:13.618599 zmc_m9 22035 WAR Discarding frame 3186 zm_rtp_source.cpp 349
    2017-02-28 16:36:13.565080 zmc_m4 20573 WAR Packet in sequence, gap 3 zm_rtp_source.cpp 127
    2017-02-28 16:36:13.551907 zmc_m1 19469 WAR Packet in sequence, gap 4 zm_rtp_source.cpp 127
    2017-02-28 16:36:13.485042 zmc_m9 22035 WAR Packet in sequence, gap 4 zm_rtp_source.cpp 127
    2017-02-28 16:36:13.418426 zmc_m1 19469 WAR Discarding partial frame 12543, 32716 bytes zm_rtp_source.cpp 345
    2017-02-28 16:36:13.351594 zmc_m9 22035 WAR Discarding partial frame 3186, 24538 bytes zm_rtp_source.cpp 345
    2017-02-28 16:36:13.285063 zmc_m1 19469 WAR Packet in sequence, gap 2 zm_rtp_source.cpp 127
    2017-02-28 16:36:13.284820 zmc_m14 21479 WAR Discarding incomplete frame 5519, 0 bytes zm_rtp_source.cpp 336
    2017-02-28 16:36:13.218299 zmc_m9 22035 WAR Packet in sequence, gap 3 zm_rtp_source.cpp 127
    2017-02-28 16:36:13.162462 zmc_m14 21479 WAR Discarding partial frame 5519, 89962 bytes zm_rtp_source.cpp 345
    2017-02-28 16:36:13.162397 zmc_m1 19469 WAR Discarding incomplete frame 12542, 0 bytes zm_rtp_source.cpp 336
    2017-02-28 16:36:13.106746 zmc_m6 22350 WAR Discarding incomplete frame 88, 0 bytes zm_rtp_source.cpp 336
    2017-02-28 16:36:13.106680 zmc_m9 22035 WAR Discarding incomplete frame 3185, 0 bytes zm_rtp_source.cpp 336
    2017-02-28 16:36:13.059560 zmc_m14 21479 WAR Packet in sequence, gap 54 zm_rtp_source.cpp 127
    2017-02-28 16:36:13.051106 zmc_m1 19469 WAR Discarding frame 12542 zm_rtp_source.cpp 349
    2017-02-28 16:36:12.995556 zmc_m6 22350 WAR Discarding partial frame 88, 32716 bytes zm_rtp_source.cpp 345
    2017-02-28 16:36:12.995366 zmc_m9 22035 WAR Discarding frame 3185 zm_rtp_source.cpp 349
    2017-02-28 16:36:12.939812 zmc_m1 19469 WAR Packet in sequence, gap 4 zm_rtp_source.cpp 127
    2017-02-28 16:36:12.939754 zmc_m10 22381 WAR Discarding incomplete frame 46, 0 bytes zm_rtp_source.cpp 336
    2017-02-28 16:36:12.932226 zmc_m6 22350 WAR Packet in sequence, gap 4 zm_rtp_source.cpp 127
    2017-02-28 16:36:12.884124 zmc_m12 21578 WAR Discarding incomplete frame 5547, 0 bytes zm_rtp_source.cpp 336
    2017-02-28 16:36:12.883993 zmc_m9 22035 WAR Packet in sequence, gap 3 zm_rtp_source.cpp 127
Attachments
IOTop output
IOTop output
2017-02-28 16_43_24.jpg (152.42 KiB) Viewed 1982 times
Top output
Top output
2017-02-28 16_41_37.jpg (130.26 KiB) Viewed 1983 times
Last edited by Tuxxin on Tue Feb 28, 2017 9:45 pm, edited 2 times in total.
bbunge
Posts: 2956
Joined: Mon Mar 26, 2012 11:40 am
Location: Pennsylvania

Re: High IO & Tons of errors in log

Post by bbunge »

Start by lowering the resolution by half. Better yet by 1/4.

Use tmpfs for PATH_SWAP. (/dev/shm if Ubuntu 16.x) And make sure you have some free space in tmpfs!

If you are using Firefox look up the montage fix in the WIKI.
User avatar
knight-of-ni
Posts: 2406
Joined: Thu Oct 18, 2007 1:55 pm
Location: Shiloh, IL

Re: High IO & Tons of errors in log

Post by knight-of-ni »

What your log shows is exactly why we tell everyone to always use the ffmpeg source type.

Our latest documentation has been updated to reflect that:
http://zoneminder.readthedocs.io/en/lat ... source-tab

The remote source type is no longer being maintained because it attempts to re-invent what ffmpeg & libvlc already do much better.

See the ZoneMinder FAQ for Montage > 6 cameras
http://zoneminder.readthedocs.io/en/sta ... in-firefox
Visit my blog for ZoneMinder related projects using the Raspberry Pi, Orange Pi, Odroid, and the ESP8266
All of these can be found at https://zoneminder.blogspot.com/
Tuxxin
Posts: 14
Joined: Fri Jan 06, 2017 3:09 pm

Re: High IO & Tons of errors in log

Post by Tuxxin »

knnniggett wrote:What your log shows is exactly why we tell everyone to always use the ffmpeg source type.

Our latest documentation has been updated to reflect that:
http://zoneminder.readthedocs.io/en/lat ... source-tab

The remote source type is no longer being maintained because it attempts to re-invent what ffmpeg & libvlc already do much better.

See the ZoneMinder FAQ for Montage > 6 cameras
http://zoneminder.readthedocs.io/en/sta ... in-firefox

You nailed it. I updated all of the cameras to FFMPEG and the usage is like night and day. I'm still getting some bugs in the images, but it mostly seems to be coming from the montage. I will just probably use a different browser for viewing in the future. I appreciate your help.
Attachments
top after switch to ffmpeg
top after switch to ffmpeg
2017-02-28 17_20_43.jpg (151.64 KiB) Viewed 1974 times
Locked