How to handle cookies with LWP::UserAgent in Perl?

Handling cookies with LWP::UserAgent in Perl commonly uses the HTTP::Cookies module to maintain a cookie jar that stores and sends cookies automatically across requests. This approach is especially useful for session persistence across multiple HTTP calls.

Key Points

• Sigils: Cookie jars and user agents are scalar objects, so use $.
• Context: Method calls on objects typically return objects or scalars; cookie manipulation methods usually have side effects.
• TMTOWTDI: You can either manually manage cookies with HTTP::Cookies or rely on modules like WWW::Mechanize that integrate cookies automatically, but core LWP::UserAgent + HTTP::Cookies works well too.
• HTTP::Cookies does not provide a cookie_header method; to get cookie headers, use the as_string or scan methods to inspect stored cookies.

Common Pitfall

The original code failed with Can't locate object method "cookie_header" because HTTP::Cookies does not have a cookie_header method. Instead, you must build the cookie header manually or inspect cookies with scan.

Version & Requirements

Works out of the box with Perl 5.10+ and standard core modules LWP::UserAgent and HTTP::Cookies. No external CPAN modules are required.

Runnable Example

use strict;
use warnings;
use LWP::UserAgent;
use HTTP::Cookies;
use URI;

# Create a user agent instance
my $ua = LWP::UserAgent->new;

# Create a cookie jar (in-memory)
my $cookie_jar = HTTP::Cookies->new();

# Attach the cookie jar to user agent
$ua->cookie_jar($cookie_jar);

# Manually set a cookie (simulate)
$cookie_jar->set_cookie(
    0,              # version
    'session_id',   # key
    'abc123',       # value
    '/',            # path
    'example.com',  # domain
    80,             # port
    0,              # secure (false)
    1,              # HTTP only (true)
    time + 3600,    # expires in 1 hour
    undef,          # discard
    undef           # rest
);

print "Simulated setting of cookie:\n";
print "  session_id=abc123 for domain example.com\n\n";

# To get cookies "sent" for a URL, we must find matching cookies manually
my $url = 'http://example.com/';
my $uri = URI->new($url);

my @cookies_to_send;
$cookie_jar->scan(sub {
    my ($version, $key, $val, $path, $domain, $port, $secure, $expires,
        $discard, $rest) = @_;

    # Basic match on domain and path for demonstration
    if ($uri->host eq $domain && index($url, $path) == 0) {
        push @cookies_to_send, "$key=$val";
    }
});

my $cookie_header = @cookies_to_send ? join("; ", @cookies_to_send) : "(none)";

print "Cookies to send with request to $url:\n";
print "$cookie_header\n\n";

# List all stored cookies (using scan)
print "All cookies stored in the jar:\n";
$cookie_jar->scan(sub {
    my ($v, $k, $val, $path, $domain) = @_;
    print "  $k = $val; domain=$domain; path=$path\n";
});

Explanation:

• We create and connect LWP::UserAgent and HTTP::Cookies instances.
• We manually add a cookie using set_cookie with correct parameters (undef for optional fields).
• Since HTTP::Cookies has no cookie_header, we scan cookies manually, matching domain and path roughly.
• This illustrates what cookies would be sent in a header for a given URL.
• Finally, we list all stored cookies for visibility.
• The example runs entirely in memory, no network or filesystem needed, safe for sandbox execution.

Summary

When handling cookies with LWP::UserAgent, use HTTP::Cookies as a cookie jar attached via $ua->cookie_jar($jar). You can manually add cookies using set_cookie, but note that HTTP::Cookies does not provide a method to get cookie headers directly. Instead, use scan to inspect cookies and build headers manually if needed. This approach avoids common errors and works well in isolated environments.