| Subject: | t/10-view.t: test 7 fails during 'make test' but inconsistent results with 'prove' |
Attempting to install Pod-POM-View-Confluence this evening, I got these results using the 'cpan' shell:
##########
BOOK/Pod-POM-View-Confluence-0.01.tar.gz
/usr/bin/make -- OK
CPAN: YAML loaded ok (v0.66)
Running make test
PERL_DL_NONLAZY=1 /usr/local/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00-load.t ....... 1/1 # Testing Pod::POM::View::Confluence 0.01, Perl 5.018000, /usr/local/bin/perl
t/00-load.t ....... ok
t/10-view.t ....... 1/10
# Failed test 'Confluence output of t/data/05-for.pod'
# at t/10-view.t line 40.
# got: "Some text.\x{0a}Some other text.\x{0a}"
# length: 28
# expected: "Some text.\x{0a}h1. title 1\x{0a}Some other text.\x{0a}"
# length: 40
# strings begin to differ at char 12
# Looks like you failed 1 test of 10.
t/10-view.t ....... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/10 subtests
t/pod-coverage.t .. ok
t/pod.t ........... ok
Test Summary Report
-------------------
t/10-view.t (Wstat: 256 Tests: 10 Failed: 1)
Failed test: 7
Non-zero exit status: 1
Files=4, Tests=13, 2 wallclock secs ( 0.16 usr 0.08 sys + 0.91 cusr 0.29 csys = 1.44 CPU)
Result: FAIL
Failed 1/4 test programs. 1/13 subtests failed.
make: *** [test_dynamic] Error 255
BOOK/Pod-POM-View-Confluence-0.01.tar.gz
/usr/bin/make test -- NOT OK
//hint// to see the cpan-testers results for installing this module, try:
reports BOOK/Pod-POM-View-Confluence-0.01.tar.gz
Failed during this command:
BOOK/Pod-POM-View-Confluence-0.01.tar.gz : make_test NO
##########
However, when I cd-ed to the .cpan/build/ subdirectory and ran a manual test, I sometimes got this failure and sometimes got a PASS:
##########
$ prove -b t/10-view.t
t/10-view.t .. 1/10
# Failed test 'Confluence output of t/data/05-for.pod'
# at t/10-view.t line 40.
# got: "Some text.\x{0a}Some other text.\x{0a}"
# length: 28
# expected: "Some text.\x{0a}h1. title 1\x{0a}Some other text.\x{0a}"
# length: 40
# strings begin to differ at char 12
# Looks like you failed 1 test of 10.
t/10-view.t .. Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/10 subtests
Test Summary Report
-------------------
t/10-view.t (Wstat: 256 Tests: 10 Failed: 1)
Failed test: 7
Non-zero exit status: 1
Files=1, Tests=10, 1 wallclock secs ( 0.15 usr 0.05 sys + 0.26 cusr 0.08 csys = 0.54 CPU)
Result: FAIL
##########
$ prove -vb t/10-view.t
t/10-view.t ..
1..10
ok 1 - Confluence output of t/data/00-heads.pod
ok 2 - Confluence output of t/data/00-para.pod
ok 3 - Confluence output of t/data/01-verbatim.pod
ok 4 - Confluence output of t/data/02-seq.pod
ok 5 - Confluence output of t/data/03-links.pod
not ok 6 - Confluence output of t/data/04-lists-TODO.pod # TODO not fully implemented yet
# Failed (TODO) test 'Confluence output of t/data/04-lists-TODO.pod'
# at t/10-view.t line 36.
# got: ..."# wapiti \x{0a}alces_alces \x{0a}"...
# length: 120
# expected: ..."# wapiti \x{0a}# alces_alces \x{0a}"...
# length: 122
# strings begin to differ at char 108
not ok 7 - Confluence output of t/data/05-for.pod
# Failed test 'Confluence output of t/data/05-for.pod'
# at t/10-view.t line 40.
# got: "Some text.\x{0a}Some other text.\x{0a}"
# length: 28
# expected: "Some text.\x{0a}h1. title 1\x{0a}Some other text.\x{0a}"
# length: 40
# strings begin to differ at char 12
not ok 8 - Confluence output of t/data/06-begin-TODO.pod # TODO not fully implemented yet
# Failed (TODO) test 'Confluence output of t/data/06-begin-TODO.pod'
# at t/10-view.t line 36.
# got: ..."1. title 1 Confluence paragrah.\x{0a}Some other text.\x{0a}"...
# length: 61
# expected: ..."1. title 1\x{0a}Confluence paragrah.\x{0a}Some other text.\x{0a}"...
# length: 61
# strings begin to differ at char 23
not ok 9 - Confluence output of t/data/07-listslists-TODO.pod # TODO not fully implemented yet
# Failed (TODO) test 'Confluence output of t/data/07-listslists-TODO.pod'
# at t/10-view.t line 36.
# got: "A list:\x{0a}* level 1 ** level 2 \x{0a}* back to level 1 \x{0a}"
# length: 49
# expected: "A list:\x{0a}* level 1\x{0a}** level 2\x{0a}* back to level 1\x{0a}"
# length: 47
# strings begin to differ at char 18
not ok 10 - Confluence output of t/data/10-manual-TODO.pod # TODO not fully implemented yet
# Failed (TODO) test 'Confluence output of t/data/10-manual-TODO.pod'
# at t/10-view.t line 36.
# got: ..."nfluence]\x{0a}h1. COPYRIGHT\x{0a}Copyright 2008 Philippe Br"...
# length: 1988
# expected: ..."nfluence]\x{0a}\x{0a}h1. COPYRIGHT\x{0a}Copyright 2008 Philippe B"...
# length: 1989
# strings begin to differ at char 1794
# Looks like you failed 1 test of 10.
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/10 subtests
Test Summary Report
-------------------
t/10-view.t (Wstat: 256 Tests: 10 Failed: 1)
Failed test: 7
Non-zero exit status: 1
Files=1, Tests=10, 1 wallclock secs ( 0.16 usr 0.04 sys + 0.26 cusr 0.06 csys = 0.52 CPU)
Result: FAIL
##########
$ prove -b t/10-view.t
t/10-view.t .. ok
All tests successful.
Files=1, Tests=10, 1 wallclock secs ( 0.14 usr 0.03 sys + 0.24 cusr 0.06 csys = 0.47 CPU)
Result: PASS
##########
Why should running 'prove -b' on this test return different results each time?
Thank you very much.
Jim Keenan