Bug #20667 for libwww-perl: LWP::UserAgent::mirror should compare Last-Modified header to timestamp

Tue Jul 25 01:41:55 2006 ANDK [...] cpan.org - Ticket created

Subject:

LWP::UserAgent::mirror should compare Last-Modified header to timestamp

Yesterday I discovered that $ua->mirror does not work the way I expected in the case when the HTTP Server *ignores* the If-Modified-Since header. If the HTTP server sends a 200 response and a Last-Modified header of a time that is older than the local copy, mirror overwrites the local file with the older downloaded content. The documentation says: If the document on the server has not changed since this time, then nothing happens. Clearly the behaviour is not exactly matching the documentation. The question is, if you see it as a bug and how you would like it being fixed. I would think that the downloaded content should be discarded when the Last-Modified header signifies an older file than the local copy.

Tue Aug 01 18:52:51 2006 MARKSTOS [...] cpan.org - Correspondence added

From:

MARKSTOS [...] cpan.org

On Tue Jul 25 01:41:55 2006, ANDK wrote: Show quoted text

> Yesterday I discovered that $ua->mirror does not work the way I expected > in the case when the HTTP Server *ignores* the If-Modified-Since header. > > If the HTTP server sends a 200 response and a Last-Modified header of a > time that is older than the local copy, mirror overwrites the local file > with the older downloaded content. > > The documentation says: > > If the document on the server has not changed since > this time, then nothing happens. > > Clearly the behaviour is not exactly matching the documentation. The > question is, if you see it as a bug and how you would like it being fixed. > > I would think that the downloaded content should be discarded when the > Last-Modified header signifies an older file than the local copy.

For servers that behave like this, it would be more efficient to do just a HEAD, notice the Last-Modified header, and not to a GET at all. However, for well-behaved servers, doing an additional HEAD before the GET to mirror is a waste. Whether or the the document is discarded seems a less important point, since the network traffic (the "hard work") has already happened.

Tue Aug 01 18:52:52 2006 The RT System itself - Status changed from 'new' to 'open'

Wed Aug 02 02:38:51 2006 andreas.koenig.gmwojprw [...] franz.ak.mind.de - Correspondence added

CC:	ANDK [...] cpan.org
Subject:	Re: [rt.cpan.org #20667] LWP::UserAgent::mirror should compare Last-Modified header to timestamp
Date:	Wed, 02 Aug 2006 08:38:17 +0200
To:	bug-libwww-perl [...] rt.cpan.org
From:	andreas.koenig.gmwojprw [...] franz.ak.mind.de (Andreas J. Koenig)

Show quoted text

>>>>> On Tue, 01 Aug 2006 18:52:52 -0400, " via RT" <bug-libwww-perl@rt.cpan.org> said:

Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=20667 >

Show quoted text

> On Tue Jul 25 01:41:55 2006, ANDK wrote:

>> Yesterday I discovered that $ua->mirror does not work the way I expected >> in the case when the HTTP Server *ignores* the If-Modified-Since header. >> >> If the HTTP server sends a 200 response and a Last-Modified header of a >> time that is older than the local copy, mirror overwrites the local file >> with the older downloaded content. >> >> The documentation says: >> >> If the document on the server has not changed since >> this time, then nothing happens. >> >> Clearly the behaviour is not exactly matching the documentation. The >> question is, if you see it as a bug and how you would like it being fixed. >> >> I would think that the downloaded content should be discarded when the >> Last-Modified header signifies an older file than the local copy.

Show quoted text

> For servers that behave like this, it would be more efficient to do just > a HEAD, notice the Last-Modified header, and not to a GET at all. > However, for well-behaved servers, doing an additional HEAD before the > GET to mirror is a waste.

Yes, this is how I re-programmed it, because I know exactly which server I'm talking to (Perlbal). Show quoted text

> Whether or the the document is discarded seems a less important point, > since the network traffic (the "hard work") has already happened.

The way I got bitten was that my local file was overwritten by the older file from the server and that surprised me and does not fit to the description "nothing happens". It's a judgement call which behaviour people would expect in this case and it's not a frequent situation,,,, hmmm. Thanks, -- andreas

Thu Mar 30 20:46:57 2017 olaf [...] wundersolutions.com - Correspondence added

Ticket migrated to github as https://github.com/libwww-perl/libwww-perl/issues/208

Thu Mar 30 20:46:58 2017 olaf [...] wundersolutions.com - Status changed from 'open' to 'resolved'