Page 1 of 1
Question on purging
Posted: Thu Jun 25, 2015 1:45 am
by Frosty
In my tests of ZoneMinder, I set the program to purge data when the disk fills to a certain level. Unfortunately, instead of erasing the oldest data first and continuing on, it deleted everything that had been recorded up to that point for all the cameras! Is there any way to get around this?
Re: Question on purging
Posted: Thu Jun 25, 2015 6:15 pm
by mikb
Frosty wrote:Is there any way to get around this?
Yes. Set the filter up correctly next time!
Sometimes it's better to
submit the filter and see what it does before allowing it to
execute. That way you find out what it's about to do before it does nothing/too much.
Do you still have the filter settings you used? If so, post them here and someone will probably spot the deliberate mistake ...
Configuration Screenshot
Posted: Thu Jun 25, 2015 9:08 pm
by Frosty
mikb wrote:Frosty wrote:Is there any way to get around this?
Yes. Set the filter up correctly next time!
Sometimes it's better to
submit the filter and see what it does before allowing it to
execute. That way you find out what it's about to do before it does nothing/too much.
Do you still have the filter settings you used? If so, post them here and someone will probably spot the deliberate mistake ...
Whew, I was worried that this was just how ZoneMinder handled purging. Here's a screenshot of my filter configuration (note that the percentage is only set to 8 because this is a small-scale test; in regular circulation, it would obviously be set much higher):
Re: Question on purging
Posted: Fri Jun 26, 2015 6:09 pm
by mikb
OK that looks good, but maybe you are not on the same page as ZoneMinder regarding "used" and "free"
If you are trying to tell ZoneMinder to only use up to 8% of the disk -- then that would seem sensible for testing. But ... big but ... only if Zoneminder events are the ONLY THING on that disk device.
How much space /usage is there on the disk where your events (jpgs) are being stored?
If you have a disk with 90% of other stuff (let's say your OS and all your apps!) -- that rule will fire, and fire, and fire until there's nothing of ZM's events left. I suspect that might be what happened.
It's "delete on > 8% of the total disk space in-use, as reported by the OS", so what does "df -h /mnt/point/for/your/images" say? If it's anything over 8% used, that's why it went rampant.
Either: Set 95% from now on and don't worry about testing, or give ZoneMinder a dedicated disk/mounted file system for its events, which will be 0% full at the start, and slowly climb -- and if it does overflow by some quirk, it will only bring down ZoneMinder, not your OS/other apps.
clarification
Posted: Fri Jun 26, 2015 10:05 pm
by Frosty
mikb wrote:OK that looks good, but maybe you are not on the same page as ZoneMinder regarding "used" and "free"
If you are trying to tell ZoneMinder to only use up to 8% of the disk -- then that would seem sensible for testing. But ... big but ... only if Zoneminder events are the ONLY THING on that disk device.
How much space /usage is there on the disk where your events (jpgs) are being stored?
If you have a disk with 90% of other stuff (let's say your OS and all your apps!) -- that rule will fire, and fire, and fire until there's nothing of ZM's events left. I suspect that might be what happened.
It's "delete on > 8% of the total disk space in-use, as reported by the OS", so what does "df -h /mnt/point/for/your/images" say? If it's anything over 8% used, that's why it went rampant.
Either: Set 95% from now on and don't worry about testing, or give ZoneMinder a dedicated disk/mounted file system for its events, which will be 0% full at the start, and slowly climb -- and if it does overflow by some quirk, it will only bring down ZoneMinder, not your OS/other apps.
That's the thing - ZoneMinder recordings are the only things on there. This machine was a clean install of Ubuntu Server and disk usage was only at 2% when I started recording. In fact, it took quite a while to fill it up to 8% because I had the framerates set very low (Mocord; 1FPS idle, 5FPS motion).
Re: Question on purging
Posted: Sat Jun 27, 2015 7:35 pm
by mikb
In which case you've got me stumped!
If there is only one file system, with everything in it (All of your OS, files, and zoneminder's events) and it's only 8% full, then you have one (1) HUGE file system
Simple, but not optimal setup.
Or you have more than one file system, with an (unspecified) % full OS section, any others, and finally a file system (disk, partition, volume) that is mostly empty just for Zoneminder, ranging 2-8% that got emptied totally when there was no need. Which shouldn't happen[tm].
Can you post the full output of "df -h" to see how simple->complex your file systems actually are?
Another thought: Is there another rule running, something about "Purge When Old" that may have caught you out?
I'm not sure how you work out exactly which rule did the deleting, as all I can notice in my logs is it saying "Deleting event 123456" -- but not *why*.
more data
Posted: Sat Aug 29, 2015 10:19 pm
by Frosty
mikb wrote:In which case you've got me stumped!
If there is only one file system, with everything in it (All of your OS, files, and zoneminder's events) and it's only 8% full, then you have one (1) HUGE file system
Simple, but not optimal setup.
Yes. If this were put into a production environment, I'd probably set it up with the OS on a smaller disk and the DVR data on a separate disk (or RAID array) or partition it with the OS on a small volume and the DVR data on a separate, larger volume. We'll get there when we get there though - first I need to make sure I can get everything to work properly on this test rig before we dive into those more advanced setups.
mikb wrote:
Or you have more than one file system, with an (unspecified) % full OS section, any others, and finally a file system (disk, partition, volume) that is mostly empty just for Zoneminder, ranging 2-8% that got emptied totally when there was no need. Which shouldn't happen[tm].
Can you post the full output of "df -h" to see how simple->complex your file systems actually are?
Yup. This is what I've got:
Code: Select all
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ZoneMinderTest--vg-root 228G 2.9G 214G 2% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
udev 973M 4.0K 973M 1% /dev
tmpfs 197M 584K 197M 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 984M 0 984M 0% /run/shm
none 100M 0 100M 0% /run/user
/dev/sda1 236M 66M 158M 30% /boot
Another thought: Is there another rule running, something about "Purge When Old" that may have caught you out?
I'm not sure how you work out exactly which rule did the deleting, as all I can notice in my logs is it saying "Deleting event 123456" -- but not *why*.
I don't know for sure. I know that "PurgeWhenFull" is the only filter on the dropdown menu in the filters dialog box though.
Re: Question on purging
Posted: Mon Aug 31, 2015 6:15 am
by Nerre
What that rule says is:
If disk is occupied more than 8%, delete the last 100 events.
So, if you have less than 100 events, everything will be deleted.
Set the number of event to 5 or something like that, then zm will delete 5 events at a time until it reaches below 8% full.
Re: Question on purging
Posted: Mon Aug 31, 2015 6:04 pm
by mikb
Yes, but look at the output of "df" ... it shouldn't be deleting ANY events, as the usage is 2% !
Re: Question on purging
Posted: Tue Sep 01, 2015 11:51 am
by Nerre
I compared to my working filter.
I have mine set to 51%, sorting by Date/Time (instead of Id as you have) and limit to 5 events at a time. Other that I see no difference.
But you are right that it seems strange that events get deleted even though the disk is just 2% full. What if you change from 8% to 95%? Does it still delete events? Trying to find out if the wrong space is checked or if the filter just doesn't work at all.