How to use LWP::Simple for quick GET requests in Perl?

LWP::Simple is a lightweight Perl module designed for making quick HTTP GET requests without the overhead of the full LWP::UserAgent. It provides simple functions like get, head, and getprint to fetch web content easily. This makes it ideal for beginners or scripts that just need to retrieve a URL's content rapidly.

Using LWP::Simple for Quick GET Requests

The most common function you'll use from LWP::Simple is get. It takes a URL and returns the content of that URL as a string in scalar context. If the request fails, it returns undef. This simplicity means you can fetch a web page with just a couple of lines.

Here’s a quick rundown of useful LWP::Simple functions:

• get($url): Fetches the URL content as a string.
• getprint($url): Fetches and immediately prints the content.
• head($url): Fetches HTTP headers from the URL.
• getstore($url, $file): Saves the content to a file.

LWP::Simple uses simple scalar context to return content, so understanding Perl's context is useful here, as the return varies by context.

Basic Example: Fetch and Print Web Page Content

use strict;
use warnings;
use LWP::Simple;

my $url = 'http://www.example.com';

# Fetch the content of the URL
my $content = get($url);

if (defined $content) {
    print "Successfully fetched content from $url\n";
    print "---- Start of content ----\n";
    print $content;
    print "\n---- End of content ----\n";
} else {
    warn "Failed to fetch content from $url\n";
}

Save this as a Perl script and run it. You’ll see the HTML content of http://www.example.com printed to STDOUT.

Important Perl Concepts Highlighted

• Sigils: $ is used because get returns a scalar string.
• Context: get behaves differently in scalar vs list context; it returns the entire content as a scalar here.
• TMTOWTDI (“There’s More Than One Way To Do It”): Perl’s philosophy shines with many ways to fetch content; LWP::Simple provides the simplest.

Common Pitfalls and Notes

• LWP::Simple silently returns undef on failure; always check for it before using the content.
• It does not provide advanced HTTP features like headers control or HTTPS certificate options; for more control, use LWP::UserAgent.
• Modules such as HTTP::Tiny (core since 5.14) can also be used for simple HTTP requests in modern Perl.
• Be aware of the URL scheme: LWP::Simple supports HTTP and HTTPS URLs.

Summary

For quick and easy HTTP GET requests in Perl, LWP::Simple is a great first choice. Its get function returns the web page content in only a few lines of code, with minimal setup. Just remember to check for failure with defined, and switch to LWP::UserAgent if you need more control.