How to download a file from URL in Perl?

Downloading a file from a URL in Perl is a common networking task that can be accomplished using several approaches. The simplest way is to use core Perl modules like LWP::Simple or LWP::UserAgent. Both are part of the popular LWP (Library for WWW in Perl) set of modules, included in most Perl installations. Alternatively, you can use HTTP::Tiny, a minimal HTTP client introduced in Perl 5.14 and later.

Here’s a brief overview of each approach, along with a runnable code example using LWP::Simple, which keeps things concise and easy.

1. Using LWP::Simple

LWP::Simple provides functions like getstore that download and save a file in one step. It is good for simple downloads.

• Use getstore($url, $filename) which returns an HTTP status code (200 on success).
• Requires no manual handling of content or headers.
• Relies on LWP, which ships with many Perl installations.

2. Using LWP::UserAgent

This module gives you full control over HTTP requests and responses, handling headers, redirects, SSL, etc. Use it if you need more flexibility than LWP::Simple.

3. Using HTTP::Tiny

Introduced in Perl 5.14, HTTP::Tiny is a lightweight and core HTTP client with limited features but no external dependencies. Good for quick scripts on modern Perls.

Key Perl Concepts

• Sigils: Scalars start with $, arrays with @. Here, URLs and filenames are scalar values.
• Context: Functions like getstore return different values depending on context (here, HTTP response code).
• TMTOWTDI (There’s More Than One Way To Do It): Demonstrates flexibility in Perl’s network programming—multiple modules can handle downloads.

Common Pitfalls

• Not checking HTTP response status can lead to silent failures.
• Ignoring redirects might cause incomplete downloads.
• Writing binary files requires careful handling of file modes on Windows.
• Older Perls might lack HTTP::Tiny, requiring CPAN or external modules.

Runnable Example Using LWP::Simple

#!/usr/bin/perl
use strict;
use warnings;
use LWP::Simple;

my $url  = 'https://www.example.com/index.html';
my $file = 'downloaded_file.html';

print "Starting download of $url...\n";

# getstore downloads and saves in one step, returns HTTP status code
my $status = getstore($url, $file);

if ($status == 200) {
    print "Download succeeded. File saved as '$file'.\n";
} else {
    print "Failed to download file: HTTP status $status\n";
}

This script downloads the content from the specified URL and saves it locally as downloaded_file.html. It prints progress messages and checks for success using the HTTP status code.

If you want more control (following redirects, setting headers), consider using LWP::UserAgent or HTTP::Tiny. But for most quick downloads, LWP::Simple's getstore is clean and straightforward.