Empty zip files uploaded to ftp server

808moshin · Post by **808moshin** » Tue Apr 22, 2008 3:03 am

I have the purgewhenfull filer set to upload events to my ftp server...everything appears to work and but the zip files on the ftp server are empty. Nothing stands out in the logs. I confirmed and reinstalled zip perl module. Any ideas?

kevin_robson · Post by **kevin_robson** » Tue Apr 22, 2008 6:34 am

Have you manually ftp'd a file and checked it? Perhaps there is something wrong at the FTP end i.e. quotas or something.
There should be a temporary .zip file created - in /tmp I think. I you checked the contents of that?

808moshin · Post by **808moshin** » Wed Apr 23, 2008 5:58 am

Thanks for the reply. I manually ftp'd files and they show up on the server. I am well below my alloted disk space on the server. I don't see any zip files in /tmp. But I may be looking when there is no filter activity. I'm going to change the archive to tar.gz and see what happens. The filter has both delete and upload options checked...could the filter be deleting the images prior to compressing them?

kevin_robson · Post by **kevin_robson** » Wed Apr 23, 2008 6:54 am

Have you tried running zmfilter.pl manually from the command line?
Preferably as the apache web user. This should give you some errors if there aren't any in /tmp/zmfilter.log

Note:if you run manually as root remember to remove or change permissions on the log file etc or it wont run from zoneminder next time due to permissions on the log file.

I'm sure I had this before and it was something silly, but cant remember what it was now.

Is it to do with whether ftp is using passive transfer or not? Try playing about with manually ftping files with various options for passive and the other obscure options, and see if you can reproduce a zero file size at the ftp server.

Esky · Post by **Esky** » Fri May 09, 2008 10:40 am

Did you find an answer? I have the same problem.

When I use tar as compression it doesn't upload anything (says: [Tar error: Cowardly refusing to create empty archive!]), when i choose zip it uploads 1k empty zip files and when i choose not to use compression it still uploads 1k empty zip files or gives the tar error. Also switching pasive mode has no effect.

zmfilter.log says:

Code: Select all

05/09/2008 12:31:30.853611 zmfilter[7621].INF [Archiving event 6385]
05/09/2008 12:31:30.854356 zmfilter[7621].INF [Creating upload file '/tmp/Office-cam2-6385.zip']
05/09/2008 12:31:30.854975 zmfilter[7621].INF [Uploading to FTP.SERVER.NAME]

and when I run zmfilter.pm from shell it says:

Code: Select all

Net::FTP>>> Net::FTP(2.75)
Net::FTP>>>   Exporter(5.58)
Net::FTP>>>   Net::Cmd(2.26)
Net::FTP>>>   IO::Socket::INET(1.29)
Net::FTP>>>     IO::Socket(1.29)
Net::FTP>>>       IO::Handle(1.25)
Net::FTP=GLOB(0x8ff3dd8)<<< 220---------- Welcome to Pure-FTPd [TLS] ----------
Net::FTP=GLOB(0x8ff3dd8)<<< 220-You are user number 1 of 50 allowed.
Net::FTP=GLOB(0x8ff3dd8)<<< 220-Local time is now 12:31. Server port: 21.
Net::FTP=GLOB(0x8ff3dd8)<<< 220-This is a private system - No anonymous login
Net::FTP=GLOB(0x8ff3dd8)<<< 220-IPv6 connections are also welcome on this server.
Net::FTP=GLOB(0x8ff3dd8)<<<220>>> user USERNAME
Net::FTP=GLOB(0x8ff3dd8)<<<331>>> PASS ....
Net::FTP=GLOB(0x8ff3dd8)<<< 230-User USERNAME has group access to:  GROUPNAME
Net::FTP=GLOB(0x8ff3dd8)<<<230>>> TYPE I
Net::FTP=GLOB(0x8ff3dd8)<<<200>>> CWD /uploads
Net::FTP=GLOB(0x8ff3dd8)<<<250>>> ALLO 22
Net::FTP=GLOB(0x8ff3dd8)<<<200>>> PASV
Net::FTP=GLOB(0x8ff3dd8)<<<227>>> STOR Office-cam1-6385.zip
Net::FTP=GLOB(0x8ff3dd8)<<< 150 Accepted data connection
Net::FTP=GLOB(0x8ff3dd8)<<< 226-File successfully transferred
Net::FTP=GLOB(0x8ff3dd8)<<<226>>> QUIT
Net::FTP=GLOB(0x8ff3dd8)<<< 221-Goodbye. You uploaded 1 and downloaded 0 kbytes.
Net::FTP=GLOB(0x8ff3dd8)<<< 221 Logout.

This is on Debian Lenny running zoneminder 1.23.2-2+lenny1

808moshin · Post by **808moshin** » Sat May 10, 2008 7:22 am

Nope....yours problem sounds similar to another solution I found when researching mine...something about using an older zmfiler.pl.

Esky · Post by **Esky** » Sat May 10, 2008 8:40 am

just downloaded the latest (1.23.3) version of zmfilter.pl but that doesn't help

kevin_robson · Post by **kevin_robson** » Mon May 12, 2008 8:02 am

There should be an option:

ZM_UPLOAD_FTP_LOC_DIR The local directory in which to create upload files

If you look in this directory, does the file get created, and is it correct?

i.e. is it a problem with creating the zip file, or a problem with ftping it?

Esky · Post by **Esky** » Mon May 12, 2008 8:14 am

the variable ZM_UPLOAD_FTP_LOC_DIR is set to /tmp

in that folder there are only 2 zip files and there have been many many more attempts at uploading stuff.

So I guess the problem is in creating the zip/tar archive. Where can I look to see what's going wrong?

kevin_robson · Post by **kevin_robson** » Mon May 12, 2008 10:01 am

I'm not sure. Its a long time since I've done ftping. I think it leaves the last one in there. Dont ever remember seeing lots of them.

If you generate a big event and ftp it, you should have time to see it before it ftp's then deletes it. In this period make a copy of it and see what it looks like.

kevin_robson · Post by **kevin_robson** » Mon May 12, 2008 7:49 pm

Another option could be to try uploading to your own local linux box on localhost. Pointless I know, but it would rule in or out ISP upload problems if you could upload to your local PC, and you may get some diagnostics in your ftp server log.

Just a thought

mitch421 · Post by **mitch421** » Tue May 13, 2008 9:37 pm

I'm having the same problem. Zip files are very small, but upload to the ftp site. Tar files get the "Cowardly refusing to create empty archive!" error, so of course ftp doesn't send anything for that file type.

This is without a doubt an incorrect passed path problem.

Is there any way to get a debug file to output the full command line of the tar and/or zip file creation parameters?

Messed with the path options within the options for the ftp page with no success or change in the limited error messages seen.

Thanks!

mitch421 · Post by **mitch421** » Tue May 13, 2008 9:44 pm

BTW, I tried the other suggestion about running zmfilter.pl from the commandline manually. That shell doesn't return any error message to the screen, and the log zmfilter log file shows no more info either... just like when zoneminder runs it automatically...

I'm running 1.23.3

Thanks

Esky · Post by **Esky** » Wed May 14, 2008 6:39 am

Tried to look up the zip file being made in /tmp but that's impossible. Selected a long recording (3 seconds) but what zmfilter.log says is:

Code: Select all

05/14/08 08:30:07.341948 zmfilter[1289].INF [Scanning for events using filter '_TempFilter1210746606']
05/14/08 08:30:07.353667 zmfilter[1289].INF [Creating upload file '/tmp/Office-cam1-2466.zip']
05/14/08 08:30:07.354310 zmfilter[1289].INF [Uploading to FTP.SERVER.NAME]

so the file is created, uploaded and deleted in almost no time at all.

So I agree with mitch and think it's a path error.

my perl skills are very limited, but dugg around in zmfilter.pl anyway.

around line 716 the zip process is done I think, the info on line 720 is printed in the zmfilter.log, but the info on line 725 is not:

Code: Select all

 Info( "Adding $image_file\n" );

So I guess that the loop is not run:

Code: Select all

foreach my $image_file ( <arch_image_path> )
        {
            Info( "Adding $image_file\n" );
            my $member = $zip->addFile( $image_file );
            last unless ( $member );
            $member->desiredCompressionMethod( (ZM_UPLOAD_ARCH_COMPRESS)?&COMPRESSION_DEFLATED:&COMPRESSION_STORED );
        }

again my perl skills are very poor so can anyone point out to me if there is an error in those lines?

kevin_robson · Post by **kevin_robson** » Wed May 14, 2008 6:46 am

I'm no expert either, but try putting extra debug statements in to see if you can find out what path it is using:

if ( ZM_UPLOAD_ARCH_FORMAT eq "zip" )
{
$arch_file .= '.zip';
my $zip = Archive::Zip->new();
Info( "Creating upload file '$arch_file'\n" );
#ADD LINE BELOW
Info( "Searching images in '$arch_image_path'\n" );

my $status = &AZ_OK;
foreach my $image_file ( <*$arch_image_path> )
{
Info( "Adding $image_file\n" );
my $member = $zip->addFile( $image_file );
last unless ( $member );
$member->desiredCompressionMethod( (ZM_UPLOAD_ARCH_COMPRESS)?&COMPRESSION_DEFLATED:&COMPRESSION_STORED );
}
$status = $zip->writeToFileNamed( $arch_file );