Script to delete events when 32000 file system limit.

nickcol · Post by **nickcol** » Mon Oct 24, 2005 1:41 pm

Hi,

I had help a few weeks ago as our cameras would stop recording when events reached 32,000. This turned out to be a filesystem limit.

I have set the purge when full when disk >90%
but I need some sort of filter or script to delete say the oldest 1000 events on a camera when events >31000. as some cameras would reach 32,000 a long time before the 800gb drive filled up.

What is the best way.?

If I did a cron job to run a script every few hours what sort of line would I need.

?
some way to detect if events >31000 on camera 1

delete * from Events where Id=1 order by datetime limit 1000;
?

how often does the audit run as would this clean the frames,event folders and jpg files automaticaly later?.

Thanks
Nick

jameswilson · Post by **jameswilson** » Mon Oct 24, 2005 1:56 pm

im sure you can do that but i dont know how, how about you delete footage after 31 days?

nickcol · Post by **nickcol** » Mon Oct 24, 2005 1:58 pm

I only want around 1 month of stuff but some cameras hit the 32,000 events limit of the file system after around 2.5 weeks.

jameswilson · Post by **jameswilson** » Mon Oct 24, 2005 1:59 pm

christ
ok another way as i dont use modect mode change to mocord mode and have a filter to remove all event with a max score of 0

Or wait till you get your solution

Ruler · Post by **Ruler** » Mon Oct 24, 2005 5:40 pm

What file system are you using on the partition you're storing your events on? Reiserfs is what I use, it's fast, and I've had no problems storing events for as long as there's space for - I've got systems with 1TB and have just finished building two more with 1.6TB each. I'd bet you're using an older file system (ext2 possibly?) and that's where the limit is being imposed.

If you type mount with no options, you can see what file system type is used on each partition in your system.

nickcol · Post by **nickcol** » Tue Oct 25, 2005 9:19 am

The filesystem is ext3 and using lvm to span the / partition over
3 drives (128gb 400gb 400gb)

Is it possible to change it to Reiserfs without a total reinstall ?.

matador · Post by **matador** » Tue Oct 25, 2005 10:45 am

Hi,

Heres something you can play with.. Its a bit rough, as I just scrubbed it together when I saw your post.

Its a perl script which calls a shell script

You would have to run the script using absolute path names, but you can change it later if it works for you.

ok heres the perl script 1st;
This would run from the monitor ID directory, where all the event directories are;

#!/usr/bin/perl

use strict;

my @list;
my $file;

@list = (`/opt/zm/scripts/prt_list.scr`);

foreach $file ( @list ) {
chomp $file;
print "Deleting Event $file\n";
`/bin/rm -rf $file `;
print "Finished deleting event $file ! \n";
}
See how the @list calls another script from /opt/zm/scripts/prt_list.scr

in the prt_list.scr file is a simple 1 line written as follows

ls -lt | tail -2 | awk '{ printf $9 "\n" }'

Now all you have to do is change the tail -2 to tail -1000 to list only the oldest 1000 events. Also you may have to change the printf $9 to the correct column number which lists the directory event number. In my case it was the ninth column, counting from the left to right.

jameswilson · Post by **jameswilson** » Tue Oct 25, 2005 12:19 pm

good call
Matador you seem to be a bit of a demon at this scripting
great stuff

Ruler · Post by **Ruler** » Tue Oct 25, 2005 3:23 pm

nickcol wrote:The filesystem is ext3 and using lvm to span the / partition over
3 drives (128gb 400gb 400gb)

Is it possible to change it to Reiserfs without a total reinstall ?.

I don't believe it's possible to change it without a reinstall of the partition (backup data, umount partition, mkreiserfs on the partition, mount it, restore data), but I'm not certain of this at all. A linux expert may be able to tell you for certain. I did considerable research on the file systems before deciding on which to use and discovered that reiserfs really does win out for this application.

What I've done on my systems is have a smallish (80 gig) boot drive and then the extra HDs (4 x 400 gig) set up in a RAID-0 array so that they appear to be one drive mounted as /video, then serve the hypertext and cgi documents from there.

jameswilson · Post by **jameswilson** » Tue Oct 25, 2005 4:32 pm

i use ext3 would reiserfs be better then as the var partition?

Ruler · Post by **Ruler** » Tue Oct 25, 2005 8:42 pm

It would be my preference. It's like ext3 but faster and more robust, especially when using lots of small files (such as each frame of an alarm).

Post by **cordel** » Tue Oct 25, 2005 9:38 pm

Any one played with ReiserFS version 4 yet???
Last I heard it is stable these days, but I haven't tried myself.
I'm curious if anyone has had experiance with it at all.

IMHO, ReiserFS is much better than ext3. It's a transactional file system that runs much faster than ext3. Just my opinon though

Regards,
Cordel

jameswilson · Post by **jameswilson** » Tue Oct 25, 2005 9:50 pm

looks like im converted then. Thanks all

Post by **cordel** » Wed Oct 26, 2005 1:24 am

FYI, for those that aren't aware:
With the CTU/ZM Core distro you can enable RieserFS at install by passing the option at boot time linux selinux=0 reiserfs

Same for all Fedora/RH based Distros as far as I know.
Regards,
Cordel

SyRenity · Post by **SyRenity** » Wed Oct 26, 2005 8:09 am

Hi.

I had used ReiserFS on Vector Linux some time ago - an excellent and a reliable FS. It would be interested to know how it performs for the ZM users.

Also, are there any recommended articles of how to safely convert the the ext3 to ReiserFS?