RTP vs Ffmpeg

[Daylights]

I have a couple of Xiaomi Xiaofang camera's flashed with custom firmware (Dafang-hack). They support RTSP streams, works well in VLC. In Zoneminder I have 1 configured as:
- Remote
- RTSP
- RTP/Unicast
- <IP address>
- 8554
- Response Media URL enabled

The logging for this camera gives me loads of warnings 'discarding incomplete frame XXXX' and 'packet in sequence, gap XX', though de camera works fine.

I have another configured as:
- Ffmpeg
- rtsp://<IP address>:8554/unicast
- TCP
- (no options)

The logging for this camera gives no warnings, but over time the stream goes blanc (usually ends with a ? as a view). Resetting the stream on the camera or disabling/enabling the camera in ZM makes it work again. It could be stable for days, or it can fail every hour.

The camera outputs a 720P 15fps stream (which I already lowered from 25fps). Is there a way how I can make Ffmpeg stable, of get rid of the many warnings with RTP? Or perhaps some other setting I can check?

[jwarfin]

First things I do when struggling with a new IP cam include:

- Log into the camera and check all the settings for sanity. I'll set the frame rate low, to 3 or 5 FPS. I'll make sure the network settings are perfect.

-Set the cam to rtsp via TCP rather than UDP. Set ZM monitor for the cam to use rtsp via TCP rather than UDP. I believe I've had more luck running my rtsp cams via multicast. With some cams it makes no real difference (multicast of unicast) but some cams prefer multicast.

- I'll also set I Frames (or key frames) to the same number as FPS - so I get a key frame at least every second. Super high I Frame settings can lead to stream corruption - especially if network connection is quirky.

- I'll always review the health of the network connection, looking for dropped packets or other errors for RX or TX. If your ZM server is on linux then "ifconfig -a" is a good place to start.

- If the cam is being used via wireless, but has an RJ45 for a wired connection, I'll switch it to the wired connection. If the cam becomes stable it's a clear indication something about the wireless connection (router/AP, signal stability, camera wireless implementation, etc) is funky.

- Some cameras effectively lie about their FPS setting. That is, I'll set them at 5 FPS but they'll still try and stream via rtsp at 30 or 60 FPS. I confirm the actual streaming FPS via VLC. I won't use cameras like this with ZM. Just too problematic to waste time on.

[Daylights]

First things I do when struggling with a new IP cam include:

- Log into the camera and check all the settings for sanity. I'll set the frame rate low, to 3 or 5 FPS. I'll make sure the network settings are perfect.

-Set the cam to rtsp via TCP rather than UDP. Set ZM monitor for the cam to use rtsp via TCP rather than UDP. I believe I've had more luck running my rtsp cams via multicast. With some cams it makes no real difference (multicast of unicast) but some cams prefer multicast.

- I'll also set I Frames (or key frames) to the same number as FPS - so I get a key frame at least every second. Super high I Frame settings can lead to stream corruption - especially if network connection is quirky.

- I'll always review the health of the network connection, looking for dropped packets or other errors for RX or TX. If your ZM server is on linux then "ifconfig -a" is a good place to start.

- If the cam is being used via wireless, but has an RJ45 for a wired connection, I'll switch it to the wired connection. If the cam becomes stable it's a clear indication something about the wireless connection (router/AP, signal stability, camera wireless implementation, etc) is funky.

- Some cameras effectively lie about their FPS setting. That is, I'll set them at 5 FPS but they'll still try and stream via rtsp at 30 or 60 FPS. I confirm the actual streaming FPS via VLC. I won't use cameras like this with ZM. Just too problematic to waste time on.

Thanks a lot for your extensive reply The camera's only support WiFi, so UTP is no option. They are running Linux themselves and I can SSH to them. The camera's have a few dropped receive frames, as well as the Zoneminder server (about 450 frames on 600GB of data received, so not too bad I'd say). The average transmit data transfer on the camera's is 370Kbit/s, which is about 12% of the capacity of the connection. Setting the fps to 5 is shown in ZM, as it receives only 5fps after that.

I do have a couple of questions:
- How do I set an RSTP stream to TCP in stead of UDP in Zoneminder?
- What do you mean by setting the (key) frames, where do I set that?

[jwarfin]

...

I do have a couple of questions:
- How do I set an RSTP stream to TCP in stead of UDP in Zoneminder?
- What do you mean by setting the (key) frames, where do I set that?

In ZM, the monitor option to set the stream to TCP/UDP appears in the "source" tab, if you've set the camera to ffmpeg in the "general" tab. If you've actually set the cam to something like "remote" in the general tab then you won't see any options for TCP/UDP in the "source" tab.

"I frames" are reference frames - which can also be named I frames, B frames or P frames, depending on the codec employed. The I frame setting is a camera setting - done by logging into the camera and using its settings UI. All my h.264 cams have "I frame" settings. I've seen some h.264 cams that have no I frame setting capability.

Many codecs (mpeg 1 or 2, h,264, etc) employ reference frames (aka: "I frames"). In effect, most of the time the cam is streaming frames such that each frame is just the pixels that "changed" since the last reference frame (yes, it's actually more complicated than that for h.264). A reference frame - or I frame - is the full encoded image. A better explanation is here: https://www.dipolnet.com/h_264_compress ... bib312.htm

So, if you set I frames to 5, every 5th frame will be an I frame that contains the complete encoded picture information. There's pros and cons to using super low or high I frame settings. Too low and you're probably using more network/storage bandwidth than needed. Too high and you may get glitches in the stream and/or poorer quality video.

If an h264 streaming device regularly drops frames on the network then stream corruption can result. In this scenario, streaming via UDP is often problematic as UDP itself does not facilitate error correction or re-transmission like TCP does.

[Daylights]

Thanks again! With the 'ffmpeg' setting, I am not able to get a stable connection and after some time the screen goes blanc. Using UDP in the setting creates smearing, so I already used TCP for that.

Also, my camera has JPEG or H264 RSTP, but apart from the resolution and the fps, I cannot change anything to the settings (like I frames).

Over all, it looks like 'something' cannot keep up with the data transfer; in UDP smearing in shown, in TCP the screen goes blanc after some time, in RSTP I get warnings of incomplete frames. But at least it's stable with that last settings. I just cannot imagine what would be the bottleneck here, since my hardware (camera - cpu/memory/network, server - cpu/memory/network) is mostly idle and just not doing much.

[edit]
I can see the result of the incomplete frames in the recorded events, so that is not ideal either

[jwarfin]

If you are using ZM 1.30.4, have you tried setting the monitor to Source Type libvlc to see how well that works?

What version of operating system are you using? What version of ffmpeg?

I'm running ZM 1.30.4 of Ubuntu 18.04. FFmpeg version 3.4. Several Trendnet/Hikvision IP cams (h264 via rtsp over TCP, 6 fps, I Frames set to 6), along with a BTTV analog capture card and some old analog cams. Very stable setup.

Also, you said you're running custom firmware on your camera(s). Considering the cams don't have the ability to set a fairly common tunable feature (I frames), I'd be suspicious the camera/firmware is buggy & that's what is causing you problems. Have you tested the cams with the standard (not custom) firmware?

[jwarfin]

Curious, I googled a bit on the Xiaomi Dafang cams. Looks like Xiaomi and Wyze make the same models. From what I could tell, the cams don't officially support rtsp. They're really optimized for iOS/Android phone use and a proprietary cloud service.

To get rtsp, you have to install firmware hacks. The hacks vary, depending on the hardware version. For example, there are different hardware versions of the Dafang and Wyze cams. The hacks I found are on git hub:

https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks . This one appears to be for a more recent version of the hardware and even includes a section about integrating with zoneminder.

https://github.com/samtap/fang-hacks . This one is for the older ARM based version of the Dafang cam. No mention of zoneminder. The hacks seem to provide some rtsp capability but the docs seem pretty clear that memory limitations and performance could be a problem. If you look at the version history, you'll see an older hack version could cause overheating problems.

Looking over reviews on the Wyze cams on Amazon, I saw a number of negative comments related to problems. Seems like quality is kind of hit-or-miss with these cams.

All this seems pretty funky to me. Good luck.

[Daylights]

Curious, I googled a bit on the Xiaomi Dafang cams. Looks like Xiaomi and Wyze make the same models. From what I could tell, the cams don't officially support rtsp. They're really optimized for iOS/Android phone use and a proprietary cloud service.

To get rtsp, you have to install firmware hacks. The hacks vary, depending on the hardware version. For example, there are different hardware versions of the Dafang and Wyze cams. The hacks I found are on git hub:

https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks . This one appears to be for a more recent version of the hardware and even includes a section about integrating with zoneminder.

https://github.com/samtap/fang-hacks . This one is for the older ARM based version of the Dafang cam. No mention of zoneminder. The hacks seem to provide some rtsp capability but the docs seem pretty clear that memory limitations and performance could be a problem. If you look at the version history, you'll see an older hack version could cause overheating problems.

Looking over reviews on the Wyze cams on Amazon, I saw a number of negative comments related to problems. Seems like quality is kind of hit-or-miss with these cams.

All this seems pretty funky to me. Good luck.

Yes, indeed I use the Dafang-Hacks firmware to use these camera's. They are tiny, cheap and the hacked firmware is being developed better over time as well. I am aware that I cannot expect perfect performance of this, but I was interested how I could tweak them a little better.
Right now I am back to

- Ffmpeg
- rtsp://<IP address>:8554/unicast
- TCP
- (no options)

At 10fps the camera's seem to be stable this way. I am still interested to see what actually is the bottleneck when they become more unstable at higher fps, as the hardware has no trouble coping at all.

[garbia]

I was wondering if there is any recent feedback on this camera, just got 2 and I will integrate into ZM 1.32.3