Upload script in zmfilter.pl cause out of RAM memory

Forum for questions and support relating to the 1.30.x releases only.
Locked
golasm
Posts: 2
Joined: Thu Sep 06, 2018 8:55 am

Upload script in zmfilter.pl cause out of RAM memory

Post by golasm »

The issue can be reproduced when Upload script is packing to tar (without compression) Event, which contains a lot of frames.
In my case it was over 6000 jpeg files in full HD resolution. Unfortunately Tar->create_archive perl function in zmfilter.pl is used, which creates archive in RAM and at the end flush tar file do destination directory (in my case HDD). During packing I observed in htop memory consumption, and zmfilter.pl process used whole RAM till the system failure (out of 8GB memory).

I am not familiar with perl, but in my opinion different function should be used here. Please consider, that a lot of files can be tarred and uploaded.
How about using built in system tar command in zmfilter.pl script?
User avatar
iconnor
Posts: 2880
Joined: Fri Oct 29, 2010 1:43 am
Location: Toronto
Contact:

Re: Upload script in zmfilter.pl cause out of RAM memory

Post by iconnor »

Yes I think using the tar utility is probably the right thing to do.
golasm
Posts: 2
Joined: Thu Sep 06, 2018 8:55 am

Re: Upload script in zmfilter.pl cause out of RAM memory

Post by golasm »

As I wrote before, I am not familiar with perl. Could someone propose code instead of this below:

Code: Select all

$archLocPath = $Config{ZM_UPLOAD_LOC_DIR}.'/'.$archFile;
        Info( "Creating upload file '$archLocPath', ".int(@archImageFiles)." files\n" );

        if ( $archError = !Archive::Tar->create_archive(
                                $archLocPath,
                                $Config{ZM_UPLOAD_ARCH_COMPRESS},
                                @archImageFiles
                          )
           )
        {
            Error( "Tar error: ".Archive::Tar->error()."\n " );
        }
but with the same functionality.
Locked