Storage Optimization

[ameterasu]

Hello everyone!

I've been strugling to achieve stable zoneminder setup for a while. All of my problems were from overshooting stuff, like going to h265 or H264+, using very high iframe or high frame rate. Thanks to your support, I've solved almost all of my problems.

Now that these have been solved, I would like to investigate how I can achieve lower Data/day.

My current camera settings are H264 codec, TCP RTSP, 8fps, ffmpeg camera passthrough (save jpegs disabled), all record, decode enabled, analysis disabled. Image buffer size = 4, Max Image buffer size = 0.

Right now the load is about 7, ram usage is around 10-15gb and its writing about 10mb/s to a 6.5tb storage. This config gives me a bout 7-8days of recordings. I would like to keep recordings min. 30 days.

What would you suggest to improve? When I went to H264+, h265 or h265+, warnings and errors were being spit. I dont think this is a stable option.

Will encoding help? Should I invest in a GPU? If so what kind of GPU would be sufficient for this kind of load?

Or are there further optimization I can do to get over 30days of recording?

I will increase the camera count in the future as well, due to the server restrictions, max. storage I can have is 16tb. So I must find a way to lower mb/s.

[Ralphred]

I really depends on each cameras scene as to how to optimise for storage. It also depends on the degree of control you have over the codec.
Using CBR/ABR is nice for storage and bandwidth predictions, but using quality based codec parameters* lets you eek out the best "image per byte".
A couple of standard non-codec related space saving tips are make sure that you switch to IR mode (night mode) on the cameras ASAP**, dark scenes get very noisy very quickly, this eats bandwidth until the camera switches. Also use the "blanking" option on any preset switching PTZ's, or limit the codec so it doesn't try to (or can't) accurately reproduce a camera swinging through a 90 degree arc.
**It's totally acceptable to force this with external light detection.

Will encoding help?

Yes, if you are doing passthrough the only errors it's spitting are about detection, it shouldn't affect your recorded data. If you are looking to recode on computer hardware, well that's a whole other thing.

Should I invest in a GPU?

Meh, if the machine is struggling to encode/decode the streams then yes it may help, as long as the GPU can de/encode the stream in question. Either way, split it up, just because you can do hardware encoding/decoding doesn't mean you should offload all that onto a GPU and have CPU cycles going to waste.
There are plenty of manufacturers that support the building of your own server/client hardware, and they'll have "hardware requirements" regarding GPU's that are a good guide. Just look up the benchmarks of what they are recommending, and buy a gaming version of the card at 1/4 of the price of the "professional" version though, I've been doing this for years and it works just fine - be aware these recommendations are for decoding.

Now, assuming you haven't over done your codec settings by 300%, making 7 days into 30 is going to involve some pruning, this can be done a number of ways but the only ways to do it without modifying zoneminder itself will be:

• Have shorter video segments, enable motion based events, and "prune" the video segments without any movement in them after a set period.
• Record both high quality and low quality streams, delete the HQ streams after a set period.

**Tuning quality based codec settings is more of an art and experience thing than something that can just be written down. But there are two states you want to try and avoid, and reproducing these can give you some insight into what the settings for that scene should be. I find it easiest to set the I frame interval to 2 seconds to do this.

• I frame quality too low
  You'll get a bad quality image that gets progressively better for the 2 seconds, until the next I frame is inserted.
• P frame quality too low
  You'll get a good image, that progressively gets worse for the 2 seconds, until the next I frame is inserted.

If you can make these two things observable, you'll get an idea of where the sweet spot is for that scene with that camera, then revert to your normal I frame interval. Depending on the degree of control your cameras give you over the codec, YMMV.

[bbunge]

Use modect instead of record or mocord. Increase pre and post frame count and tweak the motion settings to get the desired motion capture.