Skip Menu |

This queue is for tickets about the URI CPAN distribution.

Report information
The Basics
Id: 36554
Status: resolved
Priority: 0/
Queue: URI

People
Owner: Nobody in particular
Requestors: jkeenan [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: (no value)
Fixed in: (no value)



Subject: t/heuristic.t : failure during 'make test'
I tried to install LWP::Simple today, for which URI is a prerequisite. The following error during URI's 'make test' prevented installation of both URI and LWP::Simple: $ prove -v t/heuristic.t t/heuristic....1..15 ok 1 ok 2 ok 3 ok 4 ok 5 not ok 6 ok 7 ok 8 ok 9 ok 10 ok 11 ok 12 ok 13 ok 14 ok 15 FAILED test 6 Failed 1/15 tests, 93.33% okay Failed Test Stat Wstat Total Fail List of Failed ----------------------------------------------------------------------- -------- t/heuristic.t 15 1 6 Failed 1/1 test scripts. 1/15 subtests failed. Files=1, Tests=15, 0 wallclock secs ( 0.06 cusr + 0.03 csys = 0.09 CPU) Failed 1/1 test programs. 1/15 subtests failed.
I should have added: perl -v This is perl, v5.10.0 built for darwin-2level
On Sat Jun 07 17:42:55 2008, JKEENAN wrote: Show quoted text
> > $ prove -v t/heuristic.t > t/heuristic....1..15 > ok 1 > ok 2 > ok 3 > ok 4 > ok 5 > not ok 6
The test fails because it searches for http://www.perl.com/camel.gif, which is no longer found on that page.
On Sat Jun 07 17:42:55 2008, JKEENAN wrote: Show quoted text
> > $ prove -v t/heuristic.t > t/heuristic....1..15 > ok 1 > ok 2 > ok 3 > ok 4 > ok 5 > not ok 6
The test fails because it searches for http://www.perl.com/camel.gif, which is no longer found on that page.
From: GAAS [...] cpan.org
All that's (normally) needed for this test to pass is that 'www.perl.com' resolves. Doesn't it?
From: jkeenan [...] cpan.org
On Mon Jun 09 02:20:29 2008, GAAS wrote: Show quoted text
> All that's (normally) needed for this test to pass is that > 'www.perl.com' resolves. Doesn't it?
I reported exactly what I got. If it had successfully resolved, the test would have passed and I would never have opened the bug ticket in the first place. Note that it is failing test #6 (MYCOUNTRY = 'bv') but passing in #7 (MYCOUNTRY = 'uk'. (Also, if this test had been written with Test::More descriptions, it would have been easier to diagnose.) Thanks for looking into this. Jim Keenan
On Mon Jun 09 08:23:38 2008, JKEENAN wrote: Show quoted text
> On Mon Jun 09 02:20:29 2008, GAAS wrote:
> > All that's (normally) needed for this test to pass is that > > 'www.perl.com' resolves. Doesn't it?
> > I reported exactly what I got. If it had successfully resolved, the > test would have passed and I > would never have opened the bug ticket in the first place. > > Note that it is failing test #6 (MYCOUNTRY = 'bv') but passing in #7 > (MYCOUNTRY = 'uk'. > > (Also, if this test had been written with Test::More descriptions, it > would have been easier to > diagnose.) > > Thanks for looking into this. >
I also saw a failure in t/heuristics.t today, and these CPAN tester reports seems to have the same problem: http://www.nntp.perl.org/group/perl.cpan.testers/2008/06/msg1758058.html http://www.nntp.perl.org/group/perl.cpan.testers/2008/06/msg1803494.html http://www.nntp.perl.org/group/perl.cpan.testers/2008/07/msg1870857.html Suggestion, without knowing any details of this test: the test is only done if www.netscape.com resolves. Theoretically it is possible due to DNS problems that www.netscape.com resolves, but www.perl.com and/or www.perl.org not. So maybe check whether www.perl.com/org resolves, and otherwise skip the test? Regards, Slaven
On Wed Oct 01 08:04:36 2008, SREZIC wrote: Show quoted text
> On Mon Jun 09 08:23:38 2008, JKEENAN wrote:
> > On Mon Jun 09 02:20:29 2008, GAAS wrote:
> > > All that's (normally) needed for this test to pass is that > > > 'www.perl.com' resolves. Doesn't it?
> > > > I reported exactly what I got. If it had successfully resolved, the > > test would have passed and I > > would never have opened the bug ticket in the first place. > > > > Note that it is failing test #6 (MYCOUNTRY = 'bv') but passing in #7 > > (MYCOUNTRY = 'uk'. > > > > (Also, if this test had been written with Test::More descriptions, it > > would have been easier to > > diagnose.) > > > > Thanks for looking into this. > >
> > I also saw a failure in t/heuristics.t today, and these CPAN tester > reports seems to have the same problem: > > http://www.nntp.perl.org/group/perl.cpan.testers/2008/06/msg1758058.html > http://www.nntp.perl.org/group/perl.cpan.testers/2008/06/msg1803494.html > http://www.nntp.perl.org/group/perl.cpan.testers/2008/07/msg1870857.html > > Suggestion, without knowing any details of this test: the test is only > done if www.netscape.com resolves. Theoretically it is possible due to > DNS problems that www.netscape.com resolves, but www.perl.com and/or > www.perl.org not. So maybe check whether www.perl.com/org resolves, and > otherwise skip the test? >
Right now I can reproduce the failure: $ perl -Mblib -MURI::Heuristic=uf_urlstr -e '$URI::Heuristic::MY_COUNTRY = "bv"; warn uf_urlstr("perl/camel.gif")' http://www.perl.bv/camel.gif at -e line 1. And really, on my machine www.perl.bv seems to exist: $ host www.perl.bv www.perl.bv has address 64.236.172.120 Host www.perl.bv not found: 3(NXDOMAIN) On another machine it does not resolve: $ host www.perl.bv Host www.perl.bv not found: 3(NXDOMAIN) If I type in the URL in my browser, then I get a catchall page from my internet provider. Argh! They are doing nasty things! Regards, Slaven
From: todd.e.rinaldo [...] jpmorgan.com
hi, tests 6-8 require that your DNS server can look them up I believe. As a result, my test on this is failing behind a firewall with a proxy. Do we think these tests are important?
Encountered this bug this morning. Isn't this just a case of setting the country code to blank in heuristic.t or alternatively altering the regex line to check for .bv instead of the .com|.org as it cuurently seems to. Easy fix surely.
On Thu May 28 05:57:47 2009, NYGEL wrote: Show quoted text
> Encountered this bug this morning. Isn't this just a case of setting > the country code to blank in heuristic.t or alternatively altering the > regex line to check for .bv instead of the .com|.org as it cuurently > seems to. Easy fix surely. >
Perhaps it is. But if so, I would have expected the maintainer to do this last year. Instead, the situation is unchanged as of today: $ prove -v t/heuristic.t t/heuristic.... 1..15 ok 1 ok 2 ok 3 ok 4 ok 5 not ok 6 ok 7 ok 8 ok 9 ok 10 ok 11 ok 12 ok 13 ok 14 ok 15 Failed 1/15 subtests Test Summary Report ------------------- t/heuristic (Wstat: 0 Tests: 15 Failed: 1) Failed test: 6 Files=1, Tests=15, 1 wallclock secs ( 0.04 usr 0.01 sys + 0.06 cusr 0.02 csys = 0.13 CPU) Result: FAIL
On Sat Jun 07 17:42:55 2008, JKEENAN wrote: Show quoted text
> I tried to install LWP::Simple today, for which URI is a prerequisite. > The following error during > URI's 'make test' prevented installation of both URI and LWP::Simple: > > $ prove -v t/heuristic.t > t/heuristic....1..15 > ok 1 > ok 2 > ok 3 > ok 4 > ok 5 > not ok 6 > ok 7
Test #7 passes for reasons that will probably cause trouble in the future. $URI::Heuristic::MY_COUNTRY = "uk"; print "not " unless uf_urlstr("perl/camel.gif") =~ m,^http://www\.perl\.(org|co)\.uk/camel\.gif$,; print "ok 7\n"; http://www.perl.co.uk/camel.gif takes you to what appears to be a domain squatter. http://www.perl.org.uk/camel.gif doesn't resolve. But http://www.perl.org.uk takes you to a site being squatted by a tech company called SCINETIQ.
From: todd.e.rinaldo [...] jpmorgan.com
Right now, you're doing the tests if gethostbyname("www.netscape.com") succeeds. For some strange reason, we're behind a firewall, but netscape is resolving. I suggest this patch so it's disabled if proxy is enabled. - if (gethostbyname("www.netscape.com") && !gethostbyname ("example.xyz")) + if (gethostbyname("www.netscape.com") && !gethostbyname ("example.xyz") && !$ENV{http_proxy} & !$ENV{HTTP_PROXY})
On Thu Jul 02 20:35:00 2009, todd_rinaldo wrote: Show quoted text
> Right now, you're doing the tests if gethostbyname("www.netscape.com") > succeeds. > > For some strange reason, we're behind a firewall, but netscape is > resolving. > > I suggest this patch so it's disabled if proxy is enabled. > > - if (gethostbyname("www.netscape.com") && !gethostbyname > ("example.xyz")) > + if (gethostbyname("www.netscape.com") && !gethostbyname > ("example.xyz") && !$ENV{http_proxy} & !$ENV{HTTP_PROXY}) > >
Got this as well today. Strangely it seemed fine the first time or two I installed it, but have been testing a deployment script and wiping it clean after each go. Suddenly I got this error. Now, this particular error appears to be the cause of most of the cpan failures reported, and considering how many other packages depend on URI now would be great if we could do something... I'm not sure exactly what this test is trying to do, but it's reliance of some dubious hosts names is problematic. I guess when this test was written the author assumed 'www.netscape.com' was pretty safe. I'd say change it to www.google.com, but that would mess up since so many countries block google from time to time. I don't completely understand what this test is hoping to achieve, but it seems totally wrong to say: $URI::Heuristic::MY_COUNTRY = "bv"; and then try to match on a .com or something. Maybe .bv was handled differently all those years ago. The docs say something about if the country is two letters, assume is the default country, but that is not clear at all to me what it means. We should probably just skip this test for now since I don't see what is going to get hurt. My guess is all the passes have more to do with netscape.com being unresolvable for most people. Or we can set the dns lookup to an ENV var and skip the test if it's not present. That way the authors can set it for developing if there is some logic there which is important to fix.
Even stranger and stranger. On my work Mac (perl 5.8.8) with latest URI I get: perl -e 'use URI::Heuristic qw(uf_urlstr uf_url); $URI::Heuristic::MY_COUNTRY = "bv"; warn uf_urlstr("perl/camel.gif")' http://www.perl.com/camel.gif at -e line 1. which passes the test. But on my home ubuntu linux with perl 5.10 I get perl -e 'use URI::Heuristic qw(uf_urlstr uf_url); $URI::Heuristic::MY_COUNTRY = "bv"; warn uf_urlstr("perl/camel.gif")' http://www.perl.bv/camel.gif at -e line 1. So there is definitely something going on for the more modern Perls and modules. I'll dig a bit
Sorry for spamming the bug queue, but I don't see any other place to discuss this. So basically if you turn on debugging, the problem becomes clear: ----- john-napiorkowskis-computer:~ jjn1056$ perl -e 'use URI::Heuristic qw(uf_urlstr); $URI::Heuristic::MY_COUNTRY = "bv"; $URI::Heuristic::DEBUG=1; warn uf_urlstr("perl/camel.gif")' uf_uristr: resolving perl/camel.gif uf_uristr: gethostbyname('www.perl.bv.')...no uf_uristr: gethostbyname('www.perl.com.')...yes uf_uristr: ==> http://www.perl.com/camel.gif http://www.perl.com/camel.gif at -e line 1. ----- This is from my work computer, which is passing the test. Now, from my ubuntu box at home I get: ----- uf_uristr: resolving perl/camel.gif uf_uristr: gethostbyname('www.perl.bv.')...yes ----- So, as someone said earlier up in this conversation, sometimes 'www.perl.bv' is gettable. From work it's not, but at home it is. I'm not sure why that is the case but the solution here is that the test needs to skip if "gethostbyname('www.perl.bv')" is true. that would be a minimal fix that should stop the problem. This tests expect gethostbyname('www.perl.bv') to fail since when the author wrote it, that host did not exist. A better long term fix is to allow all these test host names be overridable via %ENV. That way if we choose bad ones we can at least override then via distroprefs while waiting for a fix. john
On Mon Jul 20 10:19:22 2009, JJNAPIORK wrote: Show quoted text
> Sorry for spamming the bug queue, but I don't see any other place to > discuss this. > > So basically if you turn on debugging, the problem becomes clear: > > ----- > john-napiorkowskis-computer:~ jjn1056$ perl -e 'use URI::Heuristic > qw(uf_urlstr); $URI::Heuristic::MY_COUNTRY = "bv"; > $URI::Heuristic::DEBUG=1; warn uf_urlstr("perl/camel.gif")' > uf_uristr: resolving perl/camel.gif > uf_uristr: gethostbyname('www.perl.bv.')...no > uf_uristr: gethostbyname('www.perl.com.')...yes > uf_uristr: ==> http://www.perl.com/camel.gif > http://www.perl.com/camel.gif at -e line 1. > ----- > > This is from my work computer, which is passing the test. Now, from my > ubuntu box at home I get: > > ----- > uf_uristr: resolving perl/camel.gif > uf_uristr: gethostbyname('www.perl.bv.')...yes > ----- > > So, as someone said earlier up in this conversation, sometimes > 'www.perl.bv' is gettable. From work it's not, but at home it is. I'm > not sure why that is the case but the solution here is that the test > needs to skip if "gethostbyname('www.perl.bv')" is true. that would be > a minimal fix that should stop the problem. This tests expect > gethostbyname('www.perl.bv') to fail since when the author wrote it, > that host did not exist. > > A better long term fix is to allow all these test host names be > overridable via %ENV. That way if we choose bad ones we can at least > override then via distroprefs while waiting for a fix. > > john
For anyone interested, there's a git clone with a pending merge request at: http://gitorious.org/~jnapiorkowski/perl-uri/uri-heuristicstest-fix which patches the test to explicit check that www.perl.bv is unreachable, and skips a single test if for some reason it is. I've messaged the maintainers regarding the merge request, but if you are experiencing the error it would be great if you could test the clone to verify it fixes it for you. Fixed it for me, but that's just one box.
On Tue Jul 21 09:09:01 2009, JJNAPIORK wrote: Show quoted text
> On Mon Jul 20 10:19:22 2009, JJNAPIORK wrote:
> > Sorry for spamming the bug queue, but I don't see any other place to > > discuss this. > > > > So basically if you turn on debugging, the problem becomes clear: > > > > ----- > > john-napiorkowskis-computer:~ jjn1056$ perl -e 'use URI::Heuristic > > qw(uf_urlstr); $URI::Heuristic::MY_COUNTRY = "bv"; > > $URI::Heuristic::DEBUG=1; warn uf_urlstr("perl/camel.gif")' > > uf_uristr: resolving perl/camel.gif > > uf_uristr: gethostbyname('www.perl.bv.')...no > > uf_uristr: gethostbyname('www.perl.com.')...yes > > uf_uristr: ==> http://www.perl.com/camel.gif > > http://www.perl.com/camel.gif at -e line 1. > > ----- > > > > This is from my work computer, which is passing the test. Now, from my > > ubuntu box at home I get: > > > > ----- > > uf_uristr: resolving perl/camel.gif > > uf_uristr: gethostbyname('www.perl.bv.')...yes > > ----- > > > > So, as someone said earlier up in this conversation, sometimes > > 'www.perl.bv' is gettable. From work it's not, but at home it is. I'm > > not sure why that is the case but the solution here is that the test > > needs to skip if "gethostbyname('www.perl.bv')" is true. that would be > > a minimal fix that should stop the problem. This tests expect > > gethostbyname('www.perl.bv') to fail since when the author wrote it, > > that host did not exist. > > > > A better long term fix is to allow all these test host names be > > overridable via %ENV. That way if we choose bad ones we can at least > > override then via distroprefs while waiting for a fix. > > > > john
> > For anyone interested, there's a git clone with a pending merge request > at: http://gitorious.org/~jnapiorkowski/perl-uri/uri-heuristicstest-fix > > which patches the test to explicit check that www.perl.bv is > unreachable, and skips a single test if for some reason it is. I've > messaged the maintainers regarding the merge request, but if you are > experiencing the error it would be great if you could test the clone to > verify it fixes it for you. Fixed it for me, but that's just one box.
Sorry, but one more bit of information. This problem is not related to a Perl install or the way hosts or DSN works on a given machine. From my notebook computer when at work the test works properly, but when I am at home the test fails. So exact same computer, exact some perl and network setup, just a different internet provider. I guess you could say the internet providers need to fix whatever they are doing, and that would also be correct. However given the fact this causes 9/11 cpan testers failures for the current version, I'd argue strongly for the presented patch. It definitely won't hurt anything and from what I gather, the hueristics matching ability of URI is one of the more underused features in anycase.
I've now applied http://gitorious.org/perl- uri/mainline/commit/bc44119c3693063e8c72fb47d8146c244ed2c573 commit bc44119c3693063e8c72fb47d8146c244ed2c573 Author: Gisle Aas <gisle@aas.no> Date: Wed Aug 12 21:41:06 2009 +0200 Test exactly the DNS names we care about [RT#36554] diff --git a/t/heuristic.t b/t/heuristic.t index cbaea73..1b21de4 100755 --- a/t/heuristic.t +++ b/t/heuristic.t @@ -41,8 +41,8 @@ print "not " unless uf_urlstr("C:\\CONFIG.SYS") eq "file:C:\\CONFIG.SYS"; } print "ok 5\n"; -if (gethostbyname("www.netscape.com") && !gethostbyname("example.xyz")) { - # DNS probably work, lets run test 6..8 +if (gethostbyname("www.perl.com") && gethostbyname("www.perl.org.uk") && !gethostbyname("perl.bv")) { + # DNS work, lets run test 6..8 $URI::Heuristic::MY_COUNTRY = "bv"; print "not " unless uf_urlstr("perl/camel.gif") =~ m,^http://www\.perl\.(com|org)/camel\.gif$,;
On Wed Aug 12 15:44:02 2009, GAAS wrote: Show quoted text
> I've now applied http://gitorious.org/perl- > uri/mainline/commit/bc44119c3693063e8c72fb47d8146c244ed2c573 > > commit bc44119c3693063e8c72fb47d8146c244ed2c573 > Author: Gisle Aas <gisle@aas.no> > Date: Wed Aug 12 21:41:06 2009 +0200 > > Test exactly the DNS names we care about [RT#36554] > > diff --git a/t/heuristic.t b/t/heuristic.t > index cbaea73..1b21de4 100755 > --- a/t/heuristic.t > +++ b/t/heuristic.t > @@ -41,8 +41,8 @@ print "not " unless uf_urlstr("C:\\CONFIG.SYS") eq > "file:C:\\CONFIG.SYS"; > } > print "ok 5\n"; > > -if (gethostbyname("www.netscape.com") && > !gethostbyname("example.xyz")) { > - # DNS probably work, lets run test 6..8 > +if (gethostbyname("www.perl.com") && gethostbyname("www.perl.org.uk") > && > !gethostbyname("perl.bv")) { > + # DNS work, lets run test 6..8 > > $URI::Heuristic::MY_COUNTRY = "bv"; > print "not " unless uf_urlstr("perl/camel.gif") =~ > m,^http://www\.perl\.(com|org)/camel\.gif$,;
Unfortunately it happened again with 1.39. Maybe it's better to replace example.xyz with www.example.xyz, because my provider (HanseNet Alice) only "helpfully" resolves hostnames beginning with "www.": $ host example.xyz Host example.xyz not found: 3(NXDOMAIN) $ host www.example.xyz www.example.xyz has address 206.222.226.17 Host www.example.xyz not found: 3(NXDOMAIN) Regards, Slaven
On Fri Aug 14 02:50:47 2009, SREZIC wrote: <snip /> Show quoted text
> > Unfortunately it happened again with 1.39. Maybe it's better to replace > example.xyz with www.example.xyz, because my provider (HanseNet Alice) > only "helpfully" resolves hostnames beginning with "www.": > > $ host example.xyz > Host example.xyz not found: 3(NXDOMAIN) > > $ host www.example.xyz > www.example.xyz has address 206.222.226.17 > Host www.example.xyz not found: 3(NXDOMAIN) > > Regards, > Slaven
For whatever it's worth, Comcast.net returns: $ host perl.bv Host perl.bv not found: 3(NXDOMAIN) $ host www.perl.bv www.perl.bv has address 208.68.139.38 Host www.perl.bv not found: 3(NXDOMAIN) $ host example.xyz Host example.xyz not found: 3(NXDOMAIN) $ host www.example.xyz Host www.example.xyz not found: 3(NXDOMAIN) and not surprisingly URI 1.39 t/heuristic.t test 6 fails. If only our ISPs would be less "helpful." Sigh.
2nd attempt and URI-1.40 uploaded to CPAN.