How to compress old log files with gzip in Perl?

Compressing Old Log Files with gzip in Perl

When managing log files on Unix/Linux systems, it’s common to compress older files to save disk space while keeping recent logs available for quick access. Perl offers several ways to compress files using the gzip format, either by invoking external commands or using built-in modules for more portable and flexible solutions.

Here, we cover practical methods to compress old log files with gzip in Perl, along with considerations specific to Perl such as the use of sigils, context, and CPAN vs core modules.

Common Approaches

• System call to gzip: Use Perl’s system or backticks to invoke the gzip utility directly.
• Using Perl’s core module IO::Compress::Gzip: Compress files in pure Perl without external dependencies.

Key Perl Concepts

• $scalar: scalar variables hold single values, e.g., filenames.
• @array: arrays hold ordered lists, e.g., a list of log files.
• Context: Scalar vs List context matters—for example, glob returns a list of filenames.
• TMTOWTDI (There’s More Than One Way To Do It): Perl allows multiple valid approaches; choose what best fits your environment.

Example: Compress Old Log Files Using IO::Compress::Gzip

This example finds all .log files in a directory older than 7 days, compresses them to .gz, and deletes the original file if compression succeeds.

use strict;
use warnings;
use IO::Compress::Gzip qw(gzip $GzipError);
use File::Find;
use File::Spec;

# Directory containing logs
my $log_dir = './logs';

# Days threshold to consider a log "old"
my $days_old = 7;
my $seconds_old = $days_old * 24 * 60 * 60;

find(\&compress_old_logs, $log_dir);

sub compress_old_logs {
    # Only process .log files
    return unless /\.log$/;

    my $filepath = $File::Find::name;

    # Skip if already compressed or not a file
    return unless -f $filepath;
    return if $filepath =~ /\.gz$/;

    # Check age of file
    my $mtime = (stat($filepath))[9];
    my $age = time - $mtime;
    if ($age < $seconds_old) {
        return;  # too new
    }

    my $gz_file = $filepath . '.gz';

    # Skip if compressed file already exists
    if (-e $gz_file) {
        warn "Compressed file $gz_file already exists, skipping.\n";
        return;
    }

    # Compress
    my $status = gzip $filepath => $gz_file
        or warn "gzip failed for $filepath: $GzipError\n";

    if ($status) {
        unlink $filepath or warn "Could not delete original $filepath: $!\n";
        print "Compressed $filepath → $gz_file\n";
    }
}

How This Works

• File::Find recursively traverses the logs directory.
• For each file ending in .log, it checks if the file is older than 7 days (you can adjust $days_old).
• If older, it compresses the file using IO::Compress::Gzip into a new file with .gz suffix.
• On successful compression, deletes the original log to free space.

Alternative: Using System Gzip Call

If you prefer to rely on the system gzip command (common in many sysadmin scripts), you can do:

my $file = "old.log";
system("gzip", $file) == 0
    or warn "gzip failed: $?";

Note this approach depends on the external gzip utility and your PATH environment.

Common Pitfalls

• Compressing an already compressed file will corrupt it; always check suffixes.
• Deleting the original before successful compression risks data loss.
• Beware of permissions issues when deleting files or writing compressed files.
• Use strict and warnings to catch bugs early.

Summary

Compressing old log files with gzip in Perl can be elegantly handled with IO::Compress::Gzip, which avoids shell dependencies and is fully core since Perl 5.10+. For quick-and-dirty scripts, invoking system gzip directly is also workable.

The example above demonstrates a practical, safe way to implement log compression with flexibility on file age and recursive directory traversal. You can adapt the logic to your environment, such as modifying file age thresholds or directories.