Broken archive upload function

[rdmelin]

Since 1.23.3 there have been many posts regarding ftp uploads not working. If using zip the symptom is empty zip files and no error messages, if using tar the error message is "tar cowardly refusing to create empty archive".

http://www.zoneminder.com/forums/viewtopic.php?t=12616
http://www.zoneminder.com/forums/viewtopic.php?t=12348
http://www.zoneminder.com/forums/viewtopic.php?t=11546
http://www.zoneminder.com/forums/viewtopic.php?t=11414


If zmfilter.pl line 713:

Code: Select all

my $arch_image_path = getEventPath( $event )."/".((ZM_UPLOAD_ARCH_ANALYSE)?'{*analyse,*capture}':'*capture').".jpg";

is changed to to 1.22.2 version:

Code: Select all

my $arch_image_path = "$event->{MonitorId}/$event->{Id}/".((ZM_UPLOAD_ARCH_ANALYSE)?'{*analyse,*capture}':'*capture').".jpg";

then ftp upload works with both zip and tar.
I suspect this would not work with USE_DEEP_STORAGE

Sorry, my perl is not sharp enough to figure out the fix for that line. From the debug I added to my zmfilter.pl it looks like it should work.

[rdmelin]

So the current line returns an $arch_image_path that is the full path like this:

/var/lib/zm/www/events/1/133/{*analyse,*capture}.jpg

This doesn't work with USE_DEEP_STORAGE = 0
Maybe someone using USE_DEEP_STORAGE can tell us if ftp uploads are working.

Using the 1.22.2 version for line 713 returns a relative path like this:

1/135/{*analyse,*capture}.jpg

This works at least with USE_DEEP_STORAGE = 0

[rdmelin]

Tried USE_DEEP_STORAGE = 1 here and still get empty zip files.

[rdmelin]

OK this is a hack but it works with zip or tar, USE_DEEP_STORAGE = 1 or 0

Add "use Cwd;" at line 56 of zmfilter.pl

Add these after line 713 (now 714)

Code: Select all

my $cwd = cwd();
$arch_image_path =~ s/$cwd\///;

[zoneminder]

This is a bit strange as the path returned is correct. It just seems to be the <> construct that objects to the path. Try this patch which uses a slightly different method and has a couple of other defensive and diagnostic tweaks.

Code: Select all

Index: scripts/zmfilter.pl
===================================================================
--- scripts/zmfilter.pl (revision 2785)
+++ scripts/zmfilter.pl (working copy)
@@ -709,18 +709,25 @@
     my $filter = shift;
     my $event = shift;
 
+    if ( !ZM_UPLOAD_FTP_HOST )
+    {
+        Error( "Cannot upload archive as no FTP host defined" );
+        return( 0 );
+    }
+
     my $arch_file = ZM_UPLOAD_FTP_LOC_DIR.'/'.$event->{MonitorName}.'-'.$event->{Id};
     my $arch_image_path = getEventPath( $event )."/".((ZM_UPLOAD_ARCH_ANALYSE)?'{*analyse,*capture}':'*capture').".jpg";
+    my @arch_image_files =  glob($arch_image_path);
+
     my $arch_error;
-
     if ( ZM_UPLOAD_ARCH_FORMAT eq "zip" )
     {
         $arch_file .= '.zip';
         my $zip = Archive::Zip->new();
-        Info( "Creating upload file '$arch_file'\n" );
+        Info( "Creating upload file '$arch_file', ".int(@arch_image_files)." files\n" );
 
         my $status = &AZ_OK;
-        foreach my $image_file ( <*$arch_image_path> )
+        foreach my $image_file ( @arch_image_files )
         {
             Info( "Adding $image_file\n" );
             my $member = $zip->addFile( $image_file );
@@ -744,9 +751,9 @@
         {
             $arch_file .= '.tar';
         }
-        Info( "Creating upload file '$arch_file'\n" );
+        Info( "Creating upload file '$arch_file', ".int(@arch_image_files)." files\n" );
 
-        if ( $arch_error = !Archive::Tar->create_archive( $arch_file, ZM_UPLOAD_ARCH_COMPRESS, <*$arch_image_path> ) )
+        if ( $arch_error = !Archive::Tar->create_archive( $arch_file, ZM_UPLOAD_ARCH_COMPRESS, @arch_image_files ) )
         {
             Error( "Tar error: ".Archive::Tar->error()."\n " );
         }

[rdmelin]

I cant seem to get that patch to work against 1.24.1.
I pulled zmfilter.pl from svn revision 2791 and tried it but still get empty zip archives.

I'm thinking it's Archive::Zip and Archive::Tar that expect a relative path.

[rdmelin]

I hand applied the changes and they fix the problem. Works with zip and tar, USE_DEEP_STORAGE = 1 and 0

[rdmapes]

All,
Still having problems. The FTP server I go to wants passive before sending the file.
I also modified the CWD command to CD. The server appears to be lost when it sees it. So I added. To line 790 after making all the changes.

$ftp->passive() or warn( "FTP - Can't go passive" );

I connect and watched on Wireshark the process. I see the good login, but no file is delivered. My filter log repeats the lines below over and over:

03/06/09 23:57:08.769341 zmfilter[23421].INF [Scanning for events]
03/06/09 23:57:13.772299 zmfilter[23421].INF [Archiving event 1208]
03/06/09 23:57:13.773744 zmfilter[23421].INF [Creating upload file '/tmp/Living_Room-1208.tar.gz', 27 files]
03/06/09 23:57:13.844900 zmfilter[23421].INF [Uploading to x.x.x.x]
03/07/09 00:02:34.686070 zmfilter[26536].INF [Scanning for events]
03/07/09 00:02:39.689177 zmfilter[26536].INF [Archiving event 1208]
03/07/09 00:02:39.690589 zmfilter[26536].INF [Creating upload file '/tmp/Living_Room-1208.tar.gz', 27 files]
03/07/09 00:02:39.845548 zmfilter[26536].INF [Uploading to x.x.x.x]

Any insight would be helpful?

Regards,
Ron

[zoneminder]

Are the files that zmfilter says it is creating actually there, and good?