Skip Menu |

This queue is for tickets about the HTTP-Cache-Transparent CPAN distribution.

Report information
The Basics
Id: 120584
Status: resolved
Priority: 0/
Queue: HTTP-Cache-Transparent

People
Owner: Nobody in particular
Requestors: SREZIC [...] cpan.org
Cc: PARKERM [...] cpan.org
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: t/cache.t fails on FreeBSD systems (1.2, 1.2.1, 1.3)
On my freebsd 9 & 10 smokers I see the following failure: ... # Failed test 'x-cached header should be set when retrieving directly from cache' # at t/cache.t line 55. # got: '' # expected: '1' # Failed test 'x-content-unchanged header should be set when retrieving directly from cache' # at t/cache.t line 56. # got: '' # expected: '1' # Failed test 'x-no-server-contact header should be set when retrieving directly from cache' # at t/cache.t line 57. # got: '' # expected: '1' # Failed test 'x-content-unchanged header should be set when retrieving from cache after HTTP 304 from server' # at t/cache.t line 68. # got: '' # expected: '1' # Looks like you failed 4 tests of 8. t/cache.t ................... Dubious, test returned 4 (wstat 1024, 0x400) Failed 4/8 subtests ...
Thanks for your bug report. As I wrote the failing test, it seems only fair I try and sort it out! What version of HTTP::Message are your BSD boxes using? The failing tests all interrogate the HTTP response headers, looking for the various header fields being set to '1'. It's possible older versions of HTTP::Message didn't set the HTTP response header correctly. Cheers, Mike On Sat Mar 11 09:11:55 2017, SREZIC wrote: Show quoted text
> On my freebsd 9 & 10 smokers I see the following failure: > > ... > # Failed test 'x-cached header should be set when retrieving > directly from cache' > # at t/cache.t line 55. > # got: '' > # expected: '1' > > # Failed test 'x-content-unchanged header should be set when > retrieving directly from cache' > # at t/cache.t line 56. > # got: '' > # expected: '1' > > # Failed test 'x-no-server-contact header should be set when > retrieving directly from cache' > # at t/cache.t line 57. > # got: '' > # expected: '1' > > # Failed test 'x-content-unchanged header should be set when > retrieving from cache after HTTP 304 from server' > # at t/cache.t line 68. > # got: '' > # expected: '1' > # Looks like you failed 4 tests of 8. > t/cache.t ................... > Dubious, test returned 4 (wstat 1024, 0x400) > Failed 4/8 subtests > ...
I've just tried a clean CPAN install of the module on a VM installation of FreeBSD 9. With current versions of all required modules, all tests are passed and the module installs and works fine. It looks like a module version problem with your FreeBSD boxes having an old version of a required module. Although I couldn't reproduce your issue last night by regressing my HTTP::Message install back to a 2012 version, that's still my best bet for the source of your problem Are you able to upgrade your HTTP::Message install without breaking anything else? Cherers, Mike On Fri Mar 17 17:48:50 2017, PARKERM wrote: Show quoted text
> Thanks for your bug report. As I wrote the failing test, it seems only > fair I try and sort it out! > > What version of HTTP::Message are your BSD boxes using? > > The failing tests all interrogate the HTTP response headers, looking > for the various header fields being set to '1'. It's possible older > versions of HTTP::Message didn't set the HTTP response header > correctly. > > Cheers, > > Mike > > On Sat Mar 11 09:11:55 2017, SREZIC wrote:
> > On my freebsd 9 & 10 smokers I see the following failure: > > > > ... > > # Failed test 'x-cached header should be set when retrieving > > directly from cache' > > # at t/cache.t line 55. > > # got: '' > > # expected: '1' > > > > # Failed test 'x-content-unchanged header should be set when > > retrieving directly from cache' > > # at t/cache.t line 56. > > # got: '' > > # expected: '1' > > > > # Failed test 'x-no-server-contact header should be set when > > retrieving directly from cache' > > # at t/cache.t line 57. > > # got: '' > > # expected: '1' > > > > # Failed test 'x-content-unchanged header should be set when > > retrieving from cache after HTTP 304 from server' > > # at t/cache.t line 68. > > # got: '' > > # expected: '1' > > # Looks like you failed 4 tests of 8. > > t/cache.t ................... > > Dubious, test returned 4 (wstat 1024, 0x400) > > Failed 4/8 subtests > > ...
It's a nameserver issue. I have a bind9 nameserver running locally. If I use instead something else (e.g. google's 8.8.8.8) then the tests pass. Now I remember a similar issue: https://rt.cpan.org/Ticket/Display.html?id=107252 In your tests you're using an example.com URL, which seems to be resolved by the default freebsd bind9 to localhost, so the tests are behaving differently here than on other systems. On 2017-03-18 05:42:57, PARKERM wrote: Show quoted text
> I've just tried a clean CPAN install of the module on a VM > installation of FreeBSD 9. With current versions of all required > modules, all tests are passed and the module installs and works fine. > > It looks like a module version problem with your FreeBSD boxes having > an old version of a required module. Although I couldn't reproduce > your issue last night by regressing my HTTP::Message install back to a > 2012 version, that's still my best bet for the source of your problem > > Are you able to upgrade your HTTP::Message install without breaking > anything else? > > Cherers, Mike > > On Fri Mar 17 17:48:50 2017, PARKERM wrote:
> > Thanks for your bug report. As I wrote the failing test, it seems > > only > > fair I try and sort it out! > > > > What version of HTTP::Message are your BSD boxes using? > > > > The failing tests all interrogate the HTTP response headers, looking > > for the various header fields being set to '1'. It's possible older > > versions of HTTP::Message didn't set the HTTP response header > > correctly. > > > > Cheers, > > > > Mike > > > > On Sat Mar 11 09:11:55 2017, SREZIC wrote:
> > > On my freebsd 9 & 10 smokers I see the following failure: > > > > > > ... > > > # Failed test 'x-cached header should be set when retrieving > > > directly from cache' > > > # at t/cache.t line 55. > > > # got: '' > > > # expected: '1' > > > > > > # Failed test 'x-content-unchanged header should be set when > > > retrieving directly from cache' > > > # at t/cache.t line 56. > > > # got: '' > > > # expected: '1' > > > > > > # Failed test 'x-no-server-contact header should be set when > > > retrieving directly from cache' > > > # at t/cache.t line 57. > > > # got: '' > > > # expected: '1' > > > > > > # Failed test 'x-content-unchanged header should be set when > > > retrieving from cache after HTTP 304 from server' > > > # at t/cache.t line 68. > > > # got: '' > > > # expected: '1' > > > # Looks like you failed 4 tests of 8. > > > t/cache.t ................... > > > Dubious, test returned 4 (wstat 1024, 0x400) > > > Failed 4/8 subtests > > > ...
On 2017-03-18 16:14:16, SREZIC wrote: Show quoted text
> It's a nameserver issue. I have a bind9 nameserver running locally. If > I use instead something else (e.g. google's 8.8.8.8) then the tests > pass. > > Now I remember a similar issue: > https://rt.cpan.org/Ticket/Display.html?id=107252 > In your tests you're using an example.com URL, which seems to be > resolved by the default freebsd bind9 to localhost,
More correctly: - example.com resolves to 127.0.0.1 - *.example.com (e.g. www.example.com) does not resolve at all Show quoted text
> so the tests are > behaving differently here than on other systems. > > On 2017-03-18 05:42:57, PARKERM wrote:
> > I've just tried a clean CPAN install of the module on a VM > > installation of FreeBSD 9. With current versions of all required > > modules, all tests are passed and the module installs and works fine. > > > > It looks like a module version problem with your FreeBSD boxes having > > an old version of a required module. Although I couldn't reproduce > > your issue last night by regressing my HTTP::Message install back to > > a > > 2012 version, that's still my best bet for the source of your problem > > > > Are you able to upgrade your HTTP::Message install without breaking > > anything else? > > > > Cherers, Mike > > > > On Fri Mar 17 17:48:50 2017, PARKERM wrote:
> > > Thanks for your bug report. As I wrote the failing test, it seems > > > only > > > fair I try and sort it out! > > > > > > What version of HTTP::Message are your BSD boxes using? > > > > > > The failing tests all interrogate the HTTP response headers, > > > looking > > > for the various header fields being set to '1'. It's possible older > > > versions of HTTP::Message didn't set the HTTP response header > > > correctly. > > > > > > Cheers, > > > > > > Mike > > > > > > On Sat Mar 11 09:11:55 2017, SREZIC wrote:
> > > > On my freebsd 9 & 10 smokers I see the following failure: > > > > > > > > ... > > > > # Failed test 'x-cached header should be set when retrieving > > > > directly from cache' > > > > # at t/cache.t line 55. > > > > # got: '' > > > > # expected: '1' > > > > > > > > # Failed test 'x-content-unchanged header should be set when > > > > retrieving directly from cache' > > > > # at t/cache.t line 56. > > > > # got: '' > > > > # expected: '1' > > > > > > > > # Failed test 'x-no-server-contact header should be set when > > > > retrieving directly from cache' > > > > # at t/cache.t line 57. > > > > # got: '' > > > > # expected: '1' > > > > > > > > # Failed test 'x-content-unchanged header should be set when > > > > retrieving from cache after HTTP 304 from server' > > > > # at t/cache.t line 68. > > > > # got: '' > > > > # expected: '1' > > > > # Looks like you failed 4 tests of 8. > > > > t/cache.t ................... > > > > Dubious, test returned 4 (wstat 1024, 0x400) > > > > Failed 4/8 subtests > > > > ...
After reading old and new RFCs I think that the current behavior should be to not handle example domains in a special way. I opened https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=217915 and changed my nameserver configuration accordingly --- with this change the tests pass now on all my systems. However it means only that my local nameserver configuration is fixed --- still upstream nameservers may have their own special resolution for example.com et al. Regards, Slaven On 2017-03-19 05:03:09, SREZIC wrote: Show quoted text
> On 2017-03-18 16:14:16, SREZIC wrote:
> > It's a nameserver issue. I have a bind9 nameserver running locally. If > > I use instead something else (e.g. google's 8.8.8.8) then the tests > > pass. > > > > Now I remember a similar issue: > > https://rt.cpan.org/Ticket/Display.html?id=107252 > > In your tests you're using an example.com URL, which seems to be > > resolved by the default freebsd bind9 to localhost,
> > More correctly: > - example.com resolves to 127.0.0.1 > - *.example.com (e.g. www.example.com) does not resolve at all >
> > so the tests are > > behaving differently here than on other systems. > > > > On 2017-03-18 05:42:57, PARKERM wrote:
> > > I've just tried a clean CPAN install of the module on a VM > > > installation of FreeBSD 9. With current versions of all required > > > modules, all tests are passed and the module installs and works fine. > > > > > > It looks like a module version problem with your FreeBSD boxes having > > > an old version of a required module. Although I couldn't reproduce > > > your issue last night by regressing my HTTP::Message install back to > > > a > > > 2012 version, that's still my best bet for the source of your problem > > > > > > Are you able to upgrade your HTTP::Message install without breaking > > > anything else? > > > > > > Cherers, Mike > > > > > > On Fri Mar 17 17:48:50 2017, PARKERM wrote:
> > > > Thanks for your bug report. As I wrote the failing test, it seems > > > > only > > > > fair I try and sort it out! > > > > > > > > What version of HTTP::Message are your BSD boxes using? > > > > > > > > The failing tests all interrogate the HTTP response headers, > > > > looking > > > > for the various header fields being set to '1'. It's possible older > > > > versions of HTTP::Message didn't set the HTTP response header > > > > correctly. > > > > > > > > Cheers, > > > > > > > > Mike > > > > > > > > On Sat Mar 11 09:11:55 2017, SREZIC wrote:
> > > > > On my freebsd 9 & 10 smokers I see the following failure: > > > > > > > > > > ... > > > > > # Failed test 'x-cached header should be set when retrieving > > > > > directly from cache' > > > > > # at t/cache.t line 55. > > > > > # got: '' > > > > > # expected: '1' > > > > > > > > > > # Failed test 'x-content-unchanged header should be set when > > > > > retrieving directly from cache' > > > > > # at t/cache.t line 56. > > > > > # got: '' > > > > > # expected: '1' > > > > > > > > > > # Failed test 'x-no-server-contact header should be set when > > > > > retrieving directly from cache' > > > > > # at t/cache.t line 57. > > > > > # got: '' > > > > > # expected: '1' > > > > > > > > > > # Failed test 'x-content-unchanged header should be set when > > > > > retrieving from cache after HTTP 304 from server' > > > > > # at t/cache.t line 68. > > > > > # got: '' > > > > > # expected: '1' > > > > > # Looks like you failed 4 tests of 8. > > > > > t/cache.t ................... > > > > > Dubious, test returned 4 (wstat 1024, 0x400) > > > > > Failed 4/8 subtests > > > > > ...
> >
Slaven, Thanks for tracking down the source of the problem. I've added a trap in the test for the inability to retrieve www.example.com. This gives an informative error rather than just failing all the subsequent tests: PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/cache.t ................... Bailout called. Further testing stopped: Unable to retrieve http://www.example.com FAILED--Further testing stopped: Unable to retrieve http://www.example.com make: *** [test_dynamic] Error 255 I'm not sure if I can do anything else as I need a long-term available, static content URL to test the caching against. www.example.com seemed an obvious choice and, OS-specific, nameserver-config corner-cases excepted, seems a correct choice. Let me know if you have any further thoughts. This is my first exposure to test writing (and supporting a CPAN module) so I'd appreciate any input you have. Regards, Mike On Sun Mar 19 06:07:27 2017, SREZIC wrote: Show quoted text
> After reading old and new RFCs I think that the current behavior > should be to not handle example domains in a special way. I opened > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=217915 and changed > my nameserver configuration accordingly --- with this change the tests > pass now on all my systems. However it means only that my local > nameserver configuration is fixed --- still upstream nameservers may > have their own special resolution for example.com et al. > > Regards, > Slaven > > On 2017-03-19 05:03:09, SREZIC wrote:
> > On 2017-03-18 16:14:16, SREZIC wrote:
> > > It's a nameserver issue. I have a bind9 nameserver running locally. > > > If > > > I use instead something else (e.g. google's 8.8.8.8) then the tests > > > pass. > > > > > > Now I remember a similar issue: > > > https://rt.cpan.org/Ticket/Display.html?id=107252 > > > In your tests you're using an example.com URL, which seems to be > > > resolved by the default freebsd bind9 to localhost,
> > > > More correctly: > > - example.com resolves to 127.0.0.1 > > - *.example.com (e.g. www.example.com) does not resolve at all > >
> > > so the tests are > > > behaving differently here than on other systems. > > > > > > On 2017-03-18 05:42:57, PARKERM wrote:
> > > > I've just tried a clean CPAN install of the module on a VM > > > > installation of FreeBSD 9. With current versions of all required > > > > modules, all tests are passed and the module installs and works > > > > fine. > > > > > > > > It looks like a module version problem with your FreeBSD boxes > > > > having > > > > an old version of a required module. Although I couldn't > > > > reproduce > > > > your issue last night by regressing my HTTP::Message install back > > > > to > > > > a > > > > 2012 version, that's still my best bet for the source of your > > > > problem > > > > > > > > Are you able to upgrade your HTTP::Message install without > > > > breaking > > > > anything else? > > > > > > > > Cherers, Mike > > > > > > > > On Fri Mar 17 17:48:50 2017, PARKERM wrote:
> > > > > Thanks for your bug report. As I wrote the failing test, it > > > > > seems > > > > > only > > > > > fair I try and sort it out! > > > > > > > > > > What version of HTTP::Message are your BSD boxes using? > > > > > > > > > > The failing tests all interrogate the HTTP response headers, > > > > > looking > > > > > for the various header fields being set to '1'. It's possible > > > > > older > > > > > versions of HTTP::Message didn't set the HTTP response header > > > > > correctly. > > > > > > > > > > Cheers, > > > > > > > > > > Mike > > > > > > > > > > On Sat Mar 11 09:11:55 2017, SREZIC wrote:
> > > > > > On my freebsd 9 & 10 smokers I see the following failure: > > > > > > > > > > > > ... > > > > > > # Failed test 'x-cached header should be set when > > > > > > retrieving > > > > > > directly from cache' > > > > > > # at t/cache.t line 55. > > > > > > # got: '' > > > > > > # expected: '1' > > > > > > > > > > > > # Failed test 'x-content-unchanged header should be set > > > > > > when > > > > > > retrieving directly from cache' > > > > > > # at t/cache.t line 56. > > > > > > # got: '' > > > > > > # expected: '1' > > > > > > > > > > > > # Failed test 'x-no-server-contact header should be set > > > > > > when > > > > > > retrieving directly from cache' > > > > > > # at t/cache.t line 57. > > > > > > # got: '' > > > > > > # expected: '1' > > > > > > > > > > > > # Failed test 'x-content-unchanged header should be set > > > > > > when > > > > > > retrieving from cache after HTTP 304 from server' > > > > > > # at t/cache.t line 68. > > > > > > # got: '' > > > > > > # expected: '1' > > > > > > # Looks like you failed 4 tests of 8. > > > > > > t/cache.t ................... > > > > > > Dubious, test returned 4 (wstat 1024, 0x400) > > > > > > Failed 4/8 subtests > > > > > > ...
> > > >
Subject: [rt.cpan.org #120584] t/cache.t fails on FreeBSD systems (1.2, 1.2.1, 1.3)
Date: Mon, 20 Mar 2017 07:51:31 +0100
To: bug-HTTP-Cache-Transparent [...] rt.cpan.org
From: Emmanuel Seyman <emmanuel [...] seyman.fr>
I don't know if this qualifies as a seperate bug or not but Fedora packages are built in a no-network environnement so t/cache.t also fails there. I've worked around the issue by removing the cache.t file before running "make test" but this is less than ideal. URI-Fetch looks for a NO_NETWORK_TESTING env variable before running tests that require network access. Could HTTP-Cache-Transparent do the same thing? Emmanuel
Hi Emmanuel, Thanks for your message. I'll take a look at how URI::Fetch is tested and se if I can apply the same strategy for HTTP::Cache::Transparent. Regards, Mike On Mon Mar 20 03:01:33 2017, emmanuel@seyman.fr wrote: Show quoted text
> > I don't know if this qualifies as a seperate bug or not but Fedora packages > are built in a no-network environnement so t/cache.t also fails there. > > I've worked around the issue by removing the cache.t file before > running "make test" but this is less than ideal. > > URI-Fetch looks for a NO_NETWORK_TESTING env variable before running tests > that require network access. Could HTTP-Cache-Transparent do the same thing? > > Emmanuel
Emmanuel, Slaven, Could you please try the attached, replacement cache.t test? It uses Test::RequiresInternet (same as URI::Fetch) to determine whether www.example.com port 80 is accessible, skipping the tests if not. If your testing confirms a fix, I'll ask Mattias to incorporate it into the next release. Many thanks, Mike On Mon Mar 20 16:55:50 2017, PARKERM wrote: Show quoted text
> Hi Emmanuel, > > Thanks for your message. I'll take a look at how URI::Fetch is tested > and se if I can apply the same strategy for HTTP::Cache::Transparent. > > Regards, > > Mike > > On Mon Mar 20 03:01:33 2017, emmanuel@seyman.fr wrote:
> > > > I don't know if this qualifies as a seperate bug or not but Fedora > > packages > > are built in a no-network environnement so t/cache.t also fails > > there. > > > > I've worked around the issue by removing the cache.t file before > > running "make test" but this is less than ideal. > > > > URI-Fetch looks for a NO_NETWORK_TESTING env variable before running > > tests > > that require network access. Could HTTP-Cache-Transparent do the same > > thing? > > > > Emmanuel
Subject: cache.t
use strict; use warnings; use Test::More; use Test::RequiresInternet ('www.example.com' => 80); use File::Temp 'tempdir'; use LWP::Simple; use HTTP::Cache::Transparent; my $TMPDIR = undef; # URL of static web content to be retrieved. my $url = 'http://www.example.com'; # First locate some suitable tmp-dir. We need an absolute path. # Will be cleaned up once test has completed. for my $dir (tempdir( CLEANUP => 1 )) { if ( open(my $fh, '>', "$dir/test-$$")) { close($fh); unlink("$dir/test-$$"); $TMPDIR = $dir; last; } } if ( $TMPDIR ) { $TMPDIR =~ tr|\\|/|; plan tests => 8; } else { plan skip_all => 'Cannot test without a suitable TMP Directory'; } my $ua = LWP::UserAgent->new; # Create cache in temporary directory with a NoUpdate time of 10 secs HTTP::Cache::Transparent::init( { BasePath => $TMPDIR, NoUpdate => 10 } ); # Cache empty. Fetch the URL directly from the server my $r_init = $ua->get($url); #if (!$r_init->is_success) #{ # BAIL_OUT("Unable to retrieve $url"); #} # Check all headers are undef is (defined $r_init->header('X-Cached'), '', 'x-cached header should be undef when retrieving directly from server'); is (defined $r_init->header('X-Content-Unchanged'), '', 'x-content-unchanged header should be undef when retrieving directly from server'); is (defined $r_init->header('X-No-Server-Contact'), '', 'x-no-server-contact header should be undef when retrieving directly from server'); # URL Cached and within NoUpdate time. Fetching URL again should return directly from cache my $r_cached = $ua->get($url); # Check all headers are set is (defined $r_cached->header('X-Cached'), 1, 'x-cached header should be set when retrieving directly from cache'); is (defined $r_cached->header('X-Content-Unchanged'), 1, 'x-content-unchanged header should be set when retrieving directly from cache'); is (defined $r_cached->header('X-No-Server-Contact'), 1, 'x-no-server-contact header should be set when retrieving directly from cache'); # Wait for NoUpdate time to expire. sleep 10; # URL Cached but outside NoUpdate time. Server should be sent conditional GET before (unchanged) content is returned from cache my $r_server = $ua->get($url); # Check X-Cached & X-Content-Unchanged headers are set but X-No-Server-Contact is undef # Setting of X-Cached header apparently difficult to predict and seems to vary from ISP to ISP. Seems to interact with X-Cache header... #is (defined $r_server->header('X-Cached'), 1, 'x-cached header should be set when retrieving from cache after HTTP 304 from server'); is (defined $r_server->header('X-Content-Unchanged'), 1, 'x-content-unchanged header should be set when retrieving from cache after HTTP 304 from server'); is (defined $r_server->header('X-No-Server-Contact'), '', 'x-no-server-contact header should be undef when retrieving from cache after HTTP 304 from server');
RT-Send-CC: emmanuel [...] seyman.fr
Works for me. I tested the following setups: - normal nameserver setup → test script is not skipped and all tests pass - assign a random IP address to www.example.com in /etc/hosts → the check hangs for a long time (probably 180s) and decides that port 80 cannot be connected → tests are skipped - use the default freebsd bind9 configuration → tests are immediately skipped, message is "no host: www.example.com" Probably it won't work if www.example.com points to an IP address with a working server on port 80 and different behavior than the www.example.com server. On 2017-03-20 17:23:58, PARKERM wrote: Show quoted text
> Emmanuel, Slaven, > > Could you please try the attached, replacement cache.t test? > > It uses Test::RequiresInternet (same as URI::Fetch) to determine > whether www.example.com port 80 is accessible, skipping the tests if > not. > > If your testing confirms a fix, I'll ask Mattias to incorporate it > into the next release. > > Many thanks, > > Mike > > On Mon Mar 20 16:55:50 2017, PARKERM wrote:
> > Hi Emmanuel, > > > > Thanks for your message. I'll take a look at how URI::Fetch is tested > > and se if I can apply the same strategy for HTTP::Cache::Transparent. > > > > Regards, > > > > Mike > > > > On Mon Mar 20 03:01:33 2017, emmanuel@seyman.fr wrote:
> > > > > > I don't know if this qualifies as a seperate bug or not but Fedora > > > packages > > > are built in a no-network environnement so t/cache.t also fails > > > there. > > > > > > I've worked around the issue by removing the cache.t file before > > > running "make test" but this is less than ideal. > > > > > > URI-Fetch looks for a NO_NETWORK_TESTING env variable before > > > running > > > tests > > > that require network access. Could HTTP-Cache-Transparent do the > > > same > > > thing? > > > > > > Emmanuel