Page 1 of 1
Broken archive upload function
Posted: Sat Feb 28, 2009 1:26 am
by rdmelin
Since 1.23.3 there have been many posts regarding ftp uploads not working. If using zip the symptom is empty zip files and no error messages, if using tar the error message is "tar cowardly refusing to create empty archive".
http://www.zoneminder.com/forums/viewtopic.php?t=12616
http://www.zoneminder.com/forums/viewtopic.php?t=12348
http://www.zoneminder.com/forums/viewtopic.php?t=11546
http://www.zoneminder.com/forums/viewtopic.php?t=11414
If zmfilter.pl line 713:
Code: Select all
my $arch_image_path = getEventPath( $event )."/".((ZM_UPLOAD_ARCH_ANALYSE)?'{*analyse,*capture}':'*capture').".jpg";
is changed to to 1.22.2 version:
Code: Select all
my $arch_image_path = "$event->{MonitorId}/$event->{Id}/".((ZM_UPLOAD_ARCH_ANALYSE)?'{*analyse,*capture}':'*capture').".jpg";
then ftp upload works with both zip and tar.
I suspect this would not work with USE_DEEP_STORAGE
Sorry, my perl is not sharp enough to figure out the fix for that line. From the debug I added to my zmfilter.pl it looks like it should work.
Posted: Sun Mar 01, 2009 2:28 pm
by rdmelin
So the current line returns an $arch_image_path that is the full path like this:
/var/lib/zm/www/events/1/133/{*analyse,*capture}.jpg
This doesn't work with USE_DEEP_STORAGE = 0
Maybe someone using USE_DEEP_STORAGE can tell us if ftp uploads are working.
Using the 1.22.2 version for line 713 returns a relative path like this:
1/135/{*analyse,*capture}.jpg
This works at least with USE_DEEP_STORAGE = 0
Posted: Sun Mar 01, 2009 2:48 pm
by rdmelin
Tried USE_DEEP_STORAGE = 1 here and still get empty zip files.
Posted: Sun Mar 01, 2009 5:13 pm
by rdmelin
OK this is a hack but it works with zip or tar, USE_DEEP_STORAGE = 1 or 0
Add "use Cwd;" at line 56 of zmfilter.pl
Add these after line 713 (now 714)
Code: Select all
my $cwd = cwd();
$arch_image_path =~ s/$cwd\///;
Posted: Sun Mar 01, 2009 9:44 pm
by zoneminder
This is a bit strange as the path returned is correct. It just seems to be the <> construct that objects to the path. Try this patch which uses a slightly different method and has a couple of other defensive and diagnostic tweaks.
Code: Select all
Index: scripts/zmfilter.pl
===================================================================
--- scripts/zmfilter.pl (revision 2785)
+++ scripts/zmfilter.pl (working copy)
@@ -709,18 +709,25 @@
my $filter = shift;
my $event = shift;
+ if ( !ZM_UPLOAD_FTP_HOST )
+ {
+ Error( "Cannot upload archive as no FTP host defined" );
+ return( 0 );
+ }
+
my $arch_file = ZM_UPLOAD_FTP_LOC_DIR.'/'.$event->{MonitorName}.'-'.$event->{Id};
my $arch_image_path = getEventPath( $event )."/".((ZM_UPLOAD_ARCH_ANALYSE)?'{*analyse,*capture}':'*capture').".jpg";
+ my @arch_image_files = glob($arch_image_path);
+
my $arch_error;
-
if ( ZM_UPLOAD_ARCH_FORMAT eq "zip" )
{
$arch_file .= '.zip';
my $zip = Archive::Zip->new();
- Info( "Creating upload file '$arch_file'\n" );
+ Info( "Creating upload file '$arch_file', ".int(@arch_image_files)." files\n" );
my $status = &AZ_OK;
- foreach my $image_file ( <*$arch_image_path> )
+ foreach my $image_file ( @arch_image_files )
{
Info( "Adding $image_file\n" );
my $member = $zip->addFile( $image_file );
@@ -744,9 +751,9 @@
{
$arch_file .= '.tar';
}
- Info( "Creating upload file '$arch_file'\n" );
+ Info( "Creating upload file '$arch_file', ".int(@arch_image_files)." files\n" );
- if ( $arch_error = !Archive::Tar->create_archive( $arch_file, ZM_UPLOAD_ARCH_COMPRESS, <*$arch_image_path> ) )
+ if ( $arch_error = !Archive::Tar->create_archive( $arch_file, ZM_UPLOAD_ARCH_COMPRESS, @arch_image_files ) )
{
Error( "Tar error: ".Archive::Tar->error()."\n " );
}
Posted: Sun Mar 01, 2009 10:48 pm
by rdmelin
I cant seem to get that patch to work against 1.24.1.
I pulled zmfilter.pl from svn revision 2791 and tried it but still get empty zip archives.
I'm thinking it's Archive::Zip and Archive::Tar that expect a relative path.
Posted: Sun Mar 01, 2009 11:19 pm
by rdmelin
I hand applied the changes and they fix the problem. Works with zip and tar, USE_DEEP_STORAGE = 1 and 0
Added Passive Line
Posted: Sat Mar 07, 2009 5:07 am
by rdmapes
All,
Still having problems. The FTP server I go to wants passive before sending the file.
I also modified the CWD command to CD. The server appears to be lost when it sees it. So I added. To line 790 after making all the changes.
$ftp->passive() or warn( "FTP - Can't go passive" );
I connect and watched on Wireshark the process. I see the good login, but no file is delivered. My filter log repeats the lines below over and over:
03/06/09 23:57:08.769341 zmfilter[23421].INF [Scanning for events]
03/06/09 23:57:13.772299 zmfilter[23421].INF [Archiving event 1208]
03/06/09 23:57:13.773744 zmfilter[23421].INF [Creating upload file '/tmp/Living_Room-1208.tar.gz', 27 files]
03/06/09 23:57:13.844900 zmfilter[23421].INF [Uploading to x.x.x.x]
03/07/09 00:02:34.686070 zmfilter[26536].INF [Scanning for events]
03/07/09 00:02:39.689177 zmfilter[26536].INF [Archiving event 1208]
03/07/09 00:02:39.690589 zmfilter[26536].INF [Creating upload file '/tmp/Living_Room-1208.tar.gz', 27 files]
03/07/09 00:02:39.845548 zmfilter[26536].INF [Uploading to x.x.x.x]
Any insight would be helpful?
Regards,
Ron
Posted: Sun Mar 08, 2009 1:18 pm
by zoneminder
Are the files that zmfilter says it is creating actually there, and good?