zm_videostore.cpp warning on 1.36.18, was also present on 1.36.17

[iconnor]

Yeah just try a larger size. I was only guessing at it being related to keyframe interval

[Andyrh]

No new errors in over 24 hours with the value set to 2x the keyframe interval, 16.
I have one set at the interval (10) and one at 2 times (16).
If anyone thinks there is value I can experiment with lowering the 16 to see when the error returns.

[iconnor]

I don't think there is any value. It simply depends on the output from the encoder in the camera, and how ffmpeg decodes it. Hopefully over time a good default value will emerge.

[Magic919]

I'd have thought analysing the video to get some frame and timing information would be the starting place. As Isaac suggests, it'd be specific to that type of camera.

[zSprawl]

I upgraded to 1.36.20 and I've been seeing this recently too. I can't say which version it started, but it's with two cameras where I can't control or throttle the fps at the source.

The data comes in at 15 fps, so I set "reorder_queue_size=30", and it's been steady for a few hours.

[iconnor]

I honestly don't know what the right value is. I was guessing that any predictive frames would only reference a single GOP. I would be surprised if it needed to be > than 2xkeyframe interval.

Once we get this figured out, I will make it all automatic. Today I added support for detecting the presence of out of order frames, and the counting of keyframe interval so that we can set these automatically.

[Magic919]

Would be ok if it’s a closed GOP.

[zSprawl]

Unfortunately, I let it run overnight, and the issue persists.

It seems to be camera related for sure, because it's only the two cameras I can't control or throttle the fps at the source.

The issue seems to happen roughly once per hour, but it not exactly on the hour or anything precise like that.

If the camera is left as TCP, when you go to review the camera footage, the 10 minute video will be cut short. If the camera is put at UDP, the video will be outright non-existent.

I've tried everything from every value from 15, 30, 45, 90, 180, 360.... and then I googled, and saw people using larger numbers, so I went with 1000, and 9000, also without luck. In fact, it seems regardless of what I put, the issue persists hourly.

How does one change the keyframe interval?

[zSprawl]

I've messed with this a bunch today and it appears related to the two cameras where I can't control the fps and the seconds slowly drift off by up to a minute per week. The camera has really bad NTP that only syncs weekly, so I can't even control how accurate the time is.

In the end, I'm going to just suck it up to these two crappy cameras and move on.

Thanks for the time though!

[xeroiv]

Having this issue clog up my logs on 1.36.32. I assume its b/c I went from ffmpeg 4.3.3 to 4.3.5 during the update process, any good fixes available?

[bitflip10]

Was seeing the same log issues (dts err, zm_videostore err) after the ffmpeg-5.0.2-1.fc36.x86_64 "upgrade" during a FC35 to FC36 upgrade. My problem of trying to sort out was it the fedora upgrade or the ffmpeg upgrade? System load was going crazy, probably due to ZM trying to fix the out of order video packets. The old ffmpeg version was 4.4.3.

Was using rtsp:// with cameras, adding reorder_queue_size produced no joy. Log problems went away with rtmp:// camera access. Using RLC-410-5MP cameras:

rtmp://{IP}/bcs/channel0_main.bcs?channel=0&stream=0&user={USER}&password={PASSWORD}

have not seen the visual defects others have seen. Based on some googling, rtsp potentially contains UDP control information in addition to the TCP based video. rtmp is just TCP based video. https://www.zype.com/blog/rtmp-vs.-rtsp ... g-protocol

Event playback is showing the video upside down from all other streaming displays (real time, recent tile mouse overs, etc) which is new behaviour with ZM 1.36.32. Did not see this in ZM 1.36.31-1.fc35.x86_64 and the older ffmpeg.

[xeroiv]

So I added a reorder_queue_size=146 (which is what I have my max buffer set to) and log entries for 3 out of 4 of my monitors went away. Despite setting the reorder queue size to super large values I still get the same log entry on one of my monitors (Doorbird 2101KV) around non increasing DTS and that the reorder queue size = 0. Any idea on why the reorder_queue_size parameter might not be working on just the one monitor?

[iconnor]

check for typos.

[Andyrh]

This problem had gone away, now it is back.
I added the lowres streams to record, on one of the 6 low res streams I get these errors:

Code: Select all

11/13/23, 7:39:59 PM CST	zmc_m9		2854411	WAR	pkt.dts(36144090) must be <= pkt.pts(-2297039656).Decompression must happen before presentation.	zm_videostore.cpp	1409
11/13/23, 7:39:59 PM CST	zmc_m9		2854411	WAR	non increasing dts, fixing. our dts -2297039656 stream 0 last_dts 36144090. reorder_queue_size=5	zm_videostore.cpp	1398

I have adjusted the reorder_queue_size a few times with no success.

The stream is 5FPS @ 640x480. The hires stream on the same camera does not have the error.
I see a burst of more the 1800 of these errors in just a few minutes, then it stops for less than 30 minutes.

[iconnor]

I have a couple of cameras that do this. For some reason they send a very out of order packet and it makes it complain about all subsequent packets. I may have to simply drops these way out of order packets or something.