| Subject: | WWW::Mechanize is not automatically sending a "Referer" (sic) header |
WWW::Mechanize does not automatically send the Referer: header for the requests made, unlike most browsers in standard configurations. While it should be well known not to rely on this easily faked header, badly programmed websites do reject downloads or redirect the visitor to the "welcome" or "login" page unless the Referer header is sent.
Thus the proposed change in the behaviour of WWW::Mechanize.
The following change to _do_request would allow automatic creation of the Referer: header from the uri of last request made, and makes it overridable through the %WWW::Mechanize::Headers hash :
sub _do_request {
my $self = shift;
$self->{req}->header( Referer => $self->{last_uri} )
if ($self->{last_uri});
while ( my($key,$value) = each %WWW::Mechanize::headers ) {
$self->{req}->header( $key => $value );
}
$self->{res} = $self->request($self->{req});
# These internal hash elements should be dropped in favor of
# the accessors soon. -- 1/19/03
$self->{status} = $self->{res}->code;
$self->{base} = $self->{res}->base;
$self->{ct} = $self->{res}->content_type || "";
$self->{content} = $self->{res}->content;
$self->{last_uri}= $self->{uri};
if ( $self->is_html ) {
$self->{forms} = [ HTML::Form->parse($self->{content}, $self->{res}->base) ];
$self->{form} = @{$self->{forms}} ? $self->{forms}->[0] : undef;
$self->{links} = $self->extract_links();
}
return $self->{res};
}
Attached are two files, a test and the implicitly started server, to test the changed behaviour in four cases :
1. Initial request (empty uri)
2. Another request (previous uri is sent)
3. Request with Referer specified as empty
4. Request with faked Referer header
Message body not shown because it is not plain text.