Bug #27168 for Test-WWW-Mechanize: Test::WWW::Mechanize failing tests

Wed May 16 19:57:43 2007 jchimene [...] gmail.com - Ticket created

Subject:	Test::WWW::Mechanize failing tests
Date:	Wed, 16 May 2007 16:55:59 -0700
To:	bug-test-www-mechanize [...] rt.cpan.org
From:	Jeff Chimene <jchimene [...] gmail.com>

Module Name Local CPAN ------------------------------------------------------------------------- Test::WWW::Mechanize 1.1200 1.1400 # cpan CPAN: File::HomeDir loaded ok (v0.65) cpan shell -- CPAN exploration and modules installation (v1.9102) ReadLine support enabled cpan[1]> upgrade CPAN: Storable loaded ok (v2.16) Going to read /root/.cpan/Metadata Database was generated on Wed, 16 May 2007 22:11:34 GMT Package namespace installed latest in CPAN file Test::WWW::Mechanize 1.12 1.14 PETDANCE/Test-WWW-Mechanize-1.14.tar.gz 1328 installed modules have no parseable version number Running install for module 'Test::WWW::Mechanize' Running make for P/PE/PETDANCE/Test-WWW-Mechanize-1.14.tar.gz CPAN: Digest::SHA loaded ok (v5.44) CPAN: Compress::Zlib loaded ok (v2.004) Checksum for /root/.cpan/sources/authors/id/P/PE/PETDANCE/Test-WWW-Mechanize-1.14.tar.gz ok Scanning cache /root/.cpan/build for sizes ...........................-------------------------------------------------DONE Test-WWW-Mechanize-1.14/ Test-WWW-Mechanize-1.14/Changes Test-WWW-Mechanize-1.14/Makefile.PL Test-WWW-Mechanize-1.14/MANIFEST Test-WWW-Mechanize-1.14/Mechanize.pm Test-WWW-Mechanize-1.14/META.yml Test-WWW-Mechanize-1.14/README Test-WWW-Mechanize-1.14/t/ Test-WWW-Mechanize-1.14/t/00load.t Test-WWW-Mechanize-1.14/t/content_contains.t Test-WWW-Mechanize-1.14/t/content_lacks.t Test-WWW-Mechanize-1.14/t/follow_link_ok.t Test-WWW-Mechanize-1.14/t/followable_links.t Test-WWW-Mechanize-1.14/t/get_ok-parms.t Test-WWW-Mechanize-1.14/t/get_ok.t Test-WWW-Mechanize-1.14/t/has_tag.t Test-WWW-Mechanize-1.14/t/html/ Test-WWW-Mechanize-1.14/t/html/badlinks.html Test-WWW-Mechanize-1.14/t/html/form.html Test-WWW-Mechanize-1.14/t/html/goodlinks.html Test-WWW-Mechanize-1.14/t/html/manylinks.html Test-WWW-Mechanize-1.14/t/link_content.t Test-WWW-Mechanize-1.14/t/link_status.t Test-WWW-Mechanize-1.14/t/links_ok.t Test-WWW-Mechanize-1.14/t/new.t Test-WWW-Mechanize-1.14/t/page_links_content.t Test-WWW-Mechanize-1.14/t/page_links_ok.t Test-WWW-Mechanize-1.14/t/pod-coverage.t Test-WWW-Mechanize-1.14/t/pod.t Test-WWW-Mechanize-1.14/t/stuff_inputs.t CPAN: File::Temp loaded ok (v0.18) CPAN.pm: Going to build P/PE/PETDANCE/Test-WWW-Mechanize-1.14.tar.gz Checking if your kit is complete... Looks good Writing Makefile for Test::WWW::Mechanize cp Mechanize.pm blib/lib/Test/WWW/Mechanize.pm Manifying blib/man3/Test::WWW::Mechanize.3pm PETDANCE/Test-WWW-Mechanize-1.14.tar.gz /usr/bin/make CC=/usr/bin/gcc LD=/usr/bin/gcc CC=/usr/bin/gcc LD=/usr/bin/gcc -- OK Running make test PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/00load................# Testing Test::WWW::Mechanize 1.14, Perl 5.008008, /usr/bin/perl t/00load................ok t/content_contains......NOK 4/5 # Failed test 'Finds the contains' # at t/content_contains.t line 31. # STDOUT is: # not ok 1 - Does it say test page? # # not: # ok 1 - Does it say test page? # # as expected # STDERR is: # # Failed test 'Does it say test page?' # # at t/content_contains.t line 30. # # searched: "500 Can't connect to localhost:13432 (connect: Con"... # # can't find: "Test Page" # # not: # # as expected t/content_contains......NOK 5/5 # Failed test 'Handles not finding it' # at t/content_contains.t line 39. # STDERR is: # # Failed test 'Where is Mungo?' # # at t/content_contains.t line 38. # # searched: "500 Can't connect to localhost:13432 (connect: Con"... # # can't find: "Mungo" # # not: # /#\s+Failed\ test.*?\n?.*?at\ t\/content_contains\.t line 38.*\n?/ # # # searched: "<html>\x{0a} <head>\x{0a} <title>Test Page</title>\x{0a} </h"... # # # can't find: "Mungo" # # as expected t/content_contains......FAILED tests 4-5 Failed 2/5 tests, 60.00% okay t/content_lacks.........ok t/follow_link_ok........ok t/followable_links......NOK 4/4 # Failed test 'Got the right links' # at t/followable_links.t line 40. # Structures begin differing at: # $got->[0] = Does not exist # $expected->[0] = 'http://localhost:13432/goodlinks.html' # Looks like you failed 1 test of 4. t/followable_links......dubious Test returned status 1 (wstat 256, 0x100) DIED. FAILED test 4 Failed 1/4 tests, 75.00% okay t/get_ok-parms..........ok t/get_ok................ok t/has_tag...............ok t/link_content..........NOK 6/9 # Failed test 'Handles link content not found' # at t/link_content.t line 48. # STDOUT is: # ok 1 - Checking all page link content failures # # not: # not ok 1 - Checking all page link content failures # # as expected # STDERR is: # # not: # /#\s+Failed\ test.*?\n?.*?at\ t\/link_content\.t line 47.*\n?/ # # # goodlinks.html # # # badlinks.html # # # goodlinks.html # # as expected t/link_content..........NOK 9/9 # Failed test 'Handles link unlike content found' # at t/link_content.t line 68. # STDOUT is: # ok 1 - Checking all page link unlike content failures # # not: # not ok 1 - Checking all page link unlike content failures # # as expected # STDERR is: # # not: # /#\s+Failed\ test.*?\n?.*?at\ t\/link_content\.t line 67.*\n?/ # # # goodlinks.html # # # badlinks.html # # # goodlinks.html # # as expected t/link_content..........FAILED tests 6, 9 Failed 2/9 tests, 77.78% okay t/link_status...........ok t/links_ok..............ok t/new...................ok t/page_links_content....ok t/page_links_ok.........ok t/pod-coverage..........ok t/pod...................ok t/stuff_inputs..........ok Failed Test Stat Wstat Total Fail List of Failed ------------------------------------------------------------------------------- t/content_contains.t 5 2 4-5 t/followable_links.t 1 256 4 1 4 t/link_content.t 9 2 6 9 Failed 3/17 test scripts. 5/106 subtests failed. Files=17, Tests=106, 38 wallclock secs ( 3.96 cusr + 0.42 csys = 4.38 CPU) Failed 3/17 test programs. 5/106 subtests failed. make: *** [test_dynamic] Error 255 PETDANCE/Test-WWW-Mechanize-1.14.tar.gz /usr/bin/make test -- NOT OK Running make install make test had returned bad status, won't install without force Failed during this command: PETDANCE/Test-WWW-Mechanize-1.14.tar.gz : make_test NO

Thu May 17 06:52:14 2007 ANDK [...] cpan.org - Correspondence added

Sorry for a "me too" posting. I'd like to add that the result of the test suite is extremely flaky on my linux box. When I run the test a dozen times, I sometimes get 100% OK and most of the time some random test fails. I can see no pattern. The perls I used were bleadperl and maintperl current state of affairs and stock 5.8.8.

Thu May 17 06:52:26 2007 The RT System itself - Status changed from 'new' to 'open'

Mon May 21 10:59:56 2007 jchimene [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #27168] Test::WWW::Mechanize failing tests
Date:	Mon, 21 May 2007 07:54:13 -0700
To:	bug-Test-WWW-Mechanize [...] rt.cpan.org
From:	Jeff Chimene <jchimene [...] gmail.com>

Andreas Koenig via RT wrote: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=27168 > > > Sorry for a "me too" posting. I'd like to add that the result of the > test suite is extremely flaky on my linux box. When I run the test a > dozen times, I sometimes get 100% OK and most of the time some random > test fails. > I can see no pattern. The perls I used were bleadperl and maintperl > current state of affairs and stock 5.8.8.

The test suite just passed. Sunspots? Perhaps the hinky tests could be enabled only for the developer? ----------------------- Running install for module 'Test::WWW::Mechanize' Running make for P/PE/PETDANCE/Test-WWW-Mechanize-1.14.tar.gz Checksum for /root/.cpan/sources/authors/id/P/PE/PETDANCE/Test-WWW-Mechanize-1.14.tar.gz ok Test-WWW-Mechanize-1.14/ Test-WWW-Mechanize-1.14/Changes Test-WWW-Mechanize-1.14/Makefile.PL Test-WWW-Mechanize-1.14/MANIFEST Test-WWW-Mechanize-1.14/Mechanize.pm Test-WWW-Mechanize-1.14/META.yml Test-WWW-Mechanize-1.14/README Test-WWW-Mechanize-1.14/t/ Test-WWW-Mechanize-1.14/t/00load.t Test-WWW-Mechanize-1.14/t/content_contains.t Test-WWW-Mechanize-1.14/t/content_lacks.t Test-WWW-Mechanize-1.14/t/follow_link_ok.t Test-WWW-Mechanize-1.14/t/followable_links.t Test-WWW-Mechanize-1.14/t/get_ok-parms.t Test-WWW-Mechanize-1.14/t/get_ok.t Test-WWW-Mechanize-1.14/t/has_tag.t Test-WWW-Mechanize-1.14/t/html/ Test-WWW-Mechanize-1.14/t/html/badlinks.html Test-WWW-Mechanize-1.14/t/html/form.html Test-WWW-Mechanize-1.14/t/html/goodlinks.html Test-WWW-Mechanize-1.14/t/html/manylinks.html Test-WWW-Mechanize-1.14/t/link_content.t Test-WWW-Mechanize-1.14/t/link_status.t Test-WWW-Mechanize-1.14/t/links_ok.t Test-WWW-Mechanize-1.14/t/new.t Test-WWW-Mechanize-1.14/t/page_links_content.t Test-WWW-Mechanize-1.14/t/page_links_ok.t Test-WWW-Mechanize-1.14/t/pod-coverage.t Test-WWW-Mechanize-1.14/t/pod.t Test-WWW-Mechanize-1.14/t/stuff_inputs.t CPAN.pm: Going to build P/PE/PETDANCE/Test-WWW-Mechanize-1.14.tar.gz Checking if your kit is complete... Looks good Writing Makefile for Test::WWW::Mechanize cp Mechanize.pm blib/lib/Test/WWW/Mechanize.pm Manifying blib/man3/Test::WWW::Mechanize.3pm PETDANCE/Test-WWW-Mechanize-1.14.tar.gz /usr/bin/make CC=/usr/bin/gcc LD=/usr/bin/gcc CC=/usr/bin/gcc LD=/usr/bin/gcc -- OK Running make test PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/00load................ok 1/1# Testing Test::WWW::Mechanize 1.14, Perl 5.008008, /usr/bin/perl t/00load................ok t/content_contains......ok t/content_lacks.........ok t/follow_link_ok........ok t/followable_links......ok t/get_ok-parms..........ok t/get_ok................ok t/has_tag...............ok t/link_content..........ok t/link_status...........ok t/links_ok..............ok t/new...................ok t/page_links_content....ok t/page_links_ok.........ok t/pod-coverage..........ok t/pod...................ok t/stuff_inputs..........ok All tests successful. Files=17, Tests=106, 43 wallclock secs ( 3.98 cusr + 0.48 csys = 4.46 CPU) PETDANCE/Test-WWW-Mechanize-1.14.tar.gz /usr/bin/make test -- OK Running make install Installing /usr/local/share/perl/5.8.8/Test/WWW/Mechanize.pm Unlinking /usr/share/perl5/Test/WWW/Mechanize.pm (shadowing?) Installing /usr/local/man/man3/Test::WWW::Mechanize.3pm Writing /usr/local/lib/perl/5.8.8/auto/Test/WWW/Mechanize/.packlist Appending installation info to /usr/lib/perl/5.8/perllocal.pod PETDANCE/Test-WWW-Mechanize-1.14.tar.gz /usr/bin/make install UNINST=1 -- OK

Sat Jul 14 03:10:15 2007 ANDK [...] cpan.org - Correspondence added

I'd say this is the same bug that I reported to H:S:S yesterday: http://rt.cpan.org/Ticket/Display.html?id=28122 So Andy, if you can throttle your CPU, give it a try, maybe you can reproduce a few failing tests. My current statistics on T:W:M 1.14 is PASS 80 : FAIL 29 Thanks,

Mon Aug 27 06:52:58 2007 ntyni [...] iki.fi - Correspondence added

From:

ntyni [...] iki.fi

On Sat Jul 14 03:10:15 2007, ANDK wrote: Show quoted text

> I'd say this is the same bug that I reported to H:S:S yesterday: > > http://rt.cpan.org/Ticket/Display.html?id=28122

For the record, we're seeing this on Debian too: http://bugs.debian.org/439470 As I wrote in CPAN #28122, I think the fundamental problem is that HTTP::Server::Simple::background() returns before the server is ready to answer requests. I'm attaching a proposed workaround for test link_content.t in Test::WWW::Mechanize: it always waits one second before issuing the GET request. There's also an additional test for better diagnostics, should the problem resurface. Cheers, -- Niko Tyni (Debian Perl Group) ntyni@iki.fi

--- libtest-www-mechanize-perl-1.14.orig/t/link_content.t +++ libtest-www-mechanize-perl-1.14/t/link_content.t @@ -2,7 +2,7 @@ use strict; use warnings; -use Test::More tests => 9; +use Test::More tests => 10; use Test::Builder::Tester; use URI::file; @@ -24,8 +24,11 @@ my $mech=Test::WWW::Mechanize->new(); isa_ok($mech,'Test::WWW::Mechanize'); +# HTTP::Server::Simple->background() may return prematurely. +sleep 1; $mech->get('http://localhost:'.PORT.'/goodlinks.html'); my @urls=$mech->links(); +ok(@urls, 'Got links from the HTTP server'); # test regex test_out('not ok 1 - link_content_like');

Mon Oct 29 16:25:44 2007 PETDANCE [...] cpan.org - Correspondence added

Done.

Mon Oct 29 16:25:46 2007 PETDANCE [...] cpan.org - Status changed from 'open' to 'resolved'

Bug #27168 for Test-WWW-Mechanize: Test::WWW::Mechanize failing tests

Preferred bug tracker