Memory or RamDrive Buffer?

[Briago]

My zoneminder system is also my data storage system for the whole house. I use a raid-1 drive setup and cycle out a thrid drive about every two weeks to ensure my data is safe and secure.

However, after having two drives fail a couple weeks ago (not exactly at the same time, so no data was lost). Also, since they are all seagate and only two years old, the replacement is under warranty (one already replaced). I was thinking about how zoneminder causes my drives to be written to 24/7 constantly. This is extreme wear and tear on drives.

Would it be possible to make an option for a "buffer" drive. The idea would be that all live images are written either to memory or to a ram drive (the latter seems it would be easier to implement). Then when the buffer gets to ~80%, all the images would be transferred to your normal recording hard drive. This way instead of having continuous tiny writes to the drive, you only have an occasional long write to the drive.

I would try to write something for this, but I am only a very lightweight programmer by hobby. But it seems that the biggest challenge would be keeping the database in sync with where the images are located as they will move from the ram drive and later be on the hard drive.

Any thoughts?

[Briago]

Is there any other interest in this? Reply if you have had this concern/issue.

[theforce]

I dont know exactly how ZM works but doesn't it keep all incoming frames in memory until there is an event? When the alarm is triggered the event gets recorded to the hard drive.

Or does ZM use swap space for all incoming frames and processing?

[iconnor]

I don't think writing small files 24/7 will wreck your drives.

If you want the drives to spin down... well then the spinup/down puts more wear and tear on your drives.

Currently ZM deals only in jpgs. Future versions will write videos streams. How this affects the qty of writes remains to be seen.

You can probably tune linux's write buffering to wait longer before writing to disk. This is something that laptops do to let the drive spin down more. You might look into that.

[fastolfe]

Most operating systems already use such a buffer for disk writes. Writing a thousand tiny files in a short period of time does not result in a thousand tiny write IOPS. For Linux specifically (the site won't let me paste links since I'm a "new user", but you can probably figure these out):

www
thomas-krenn
com
/en/wiki/Linux_Page_Cache_Basics

And some more practical information and tuning controls:
www
westnet
com
/~gsmith/content/linux-pdflush.htm

The drawback to heavily caching data in RAM is that it's all lost if there's a power failure.

The frames in the image buffer (in Modect mode) only exist in memory and don't hit the filesystem until a motion event is detected.

David

[Nighthowl]

I really doubt too many writes were the cause of the malfunction. Either you got a bad batch (did you buy the drives at the same time?) or there is something wrong with your computer (lousy PSU?), or - most likely - you were just unlucky...

I have a couple of embedded linux DVR's working 24/7 always writing to HDD for more than 5 years with no problems.

Anyway with the decrease of RAM prices more control of how zoneminder writes files to disk/ram buffer would be welcome, but not a priority IMO.

Running a ZM system entirely from a SD card or USB pen drive and writing only when absolutely necessary (ie upload to ftp, if buffer full (internet down or ftp server problems?) write to disk until successful ftp upload) would benefit the most from this feature.