| Subject: | Tests fail as a whole but pass individually |
Here is an example of "make test" failing, but when I run the individual
test files which failed, they pass.
(erebus)(~/.cpan/build/XML-Simple-2.18-lP9yaw# PERL_DL_NONLAZY=1
/usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0,
'blib/lib', 'blib/arch')" t/*.t
# Package Version2/23 0/8 1/7 6/37 ... )===#
# perl 5.10.0
# XML::Simple 2.18
# Storable 2.18
# XML::Parser 2.36
# XML::SAX 0.96
# XML::NamespaceSupport 1.09
# XML::LibXML::SAX::Parser 1.69
# XML::LibXML::SAX 1.69 (default parser)
t/0_Config.t ...... ok
===( 19;2 2/131 0/201 2/23 6/8 1/7 6/37 1/14 ...
)===t/desertnet.xml:1: parser error : Start tag expected, '<' not found
# Looks like you planned 14 tests but ran 2.
# Looks like your test exited with 255 just after 2.
t/7_SaxStuff.t .... Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 12/14 subtests
===( 23;2 3/131 1/201 3/23 6/8 1/7 6/37 0/8 ...
)===t/desertnet.xml:1: parser error : Start tag expected, '<' not found
# Looks like you planned 7 tests but ran 1.
# Looks like your test exited with 255 just after 1.
t/5_MemCopy.t ..... Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 6/7 subtests
t/8_Namespaces.t .. ok
===( 51;3 12/131 11/201 3/23 6/8 7/37 0/40 0/? ... )===
t/6_ObjIntf.t ..... ok
===( 137;3 40/131 36/201 6/23 6/8 0/40 0/? 0/?
)=========# Failed test 'parsed expected data from file (again)'
# at t/3_Storable.t line 162.
# Structures begin differing at:
# $got->{one} = '1'
# $expected->{one} = Does not exist
t/4_MemShare.t .... ok
===( 152;4 43/131 39/201 8/23 3/40 2/14 0/12 )==========
# Failed test 'got expected data from cache'
# at t/3_Storable.t line 168.
# Structures begin differing at:
# $got->{one} = '1'
# $expected->{one} = Does not exist
===( 166;5 46/131 41/201 10/23 6/40 5/14 1/12 )==========
# Failed test 'parsed in expected value again'
# at t/3_Storable.t line 178.
# Structures begin differing at:
# $got->{one} = '1'
# $expected->{one} = Does not exist
t/A_XMLParser.t ... ok
t/B_Hooks.t ....... ok
t/9_Strict.t ...... ok
t/1_XMLin.t ....... ok
t/2_XMLout.t ...... ok
t/3_Storable.t .... 20/23 # Looks like you failed 3 tests of 23.
t/3_Storable.t .... Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/23 subtests
Test Summary Report
-------------------
t/7_SaxStuff.t (Wstat: 65280 Tests: 2 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 14 tests but ran 2.
t/5_MemCopy.t (Wstat: 65280 Tests: 1 Failed: 0)
Non-zero exit status: 255
Parse errors: Bad plan. You planned 7 tests but ran 1.
t/3_Storable.t (Wstat: 768 Tests: 23 Failed: 3)
Failed tests: 5, 7, 9
Non-zero exit status: 3
Files=12, Tests=478, 9 wallclock secs ( 0.55 usr 0.16 sys + 10.83 cusr
1.09 csys = 12.64 CPU)
Result: FAIL
Failed 3/12 test programs. 3/478 subtests failed.
(erebus)(~/.cpan/build/XML-Simple-2.18-lP9yaw# PERL_DL_NONLAZY=1
/usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0,
'blib/lib', 'blib/arch')" t/7_SaxStuff.t
t/7_SaxStuff.t .. ok
All tests successful.
Files=1, Tests=14, 1 wallclock secs ( 0.12 usr 0.03 sys + 0.56 cusr
0.10 csys = 0.81 CPU)
Result: PASS
(erebus)(~/.cpan/build/XML-Simple-2.18-lP9yaw# PERL_DL_NONLAZY=1
/usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0,
'blib/lib', 'blib/arch')" t/5_MemCopy.t
t/5_MemCopy.t .. ok
All tests successful.
Files=1, Tests=7, 3 wallclock secs ( 0.12 usr 0.02 sys + 0.50 cusr
0.04 csys = 0.68 CPU)
Result: PASS
(erebus)(~/.cpan/build/XML-Simple-2.18-lP9yaw# PERL_DL_NONLAZY=1
/usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0,
'blib/lib', 'blib/arch')" t/3_Storable.t
t/3_Storable.t .. ok
All tests successful.
Files=1, Tests=23, 6 wallclock secs ( 0.11 usr 0.05 sys + 0.47 cusr
0.14 csys = 0.77 CPU)
Result: PASS
Here is the output of "perl -V"
perl -V
Summary of my perl5 (revision 5 version 10 subversion 0) configuration:
Platform:
osname=freebsd, osvers=6.2-release-p5-jc1, archname=i386-freebsd
uname='freebsd erebus.leto.net 6.2-release-p5-jc1 freebsd
6.2-release-p5-jc1 #0: tue jun 26 22:13:24 pdt 2007
config_args='-des'
hint=recommended, useposix=true, d_sigaction=define
useithreads=undef, usemultiplicity=undef
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=undef, use64bitall=undef, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='cc', ccflags ='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H
-fno-strict-aliasing -pipe -I/usr/local/include',
optimize='-O',
cppflags='-DHAS_FPSETMASK -DHAS_FLOATINGPOINT_H -fno-strict-aliasing
-pipe -I/usr/local/include'
ccversion='', gccversion='3.4.6 [FreeBSD] 20060305', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=4, prototype=define
Linker and Libraries:
ld='cc', ldflags ='-Wl,-E -L/usr/local/lib'
libpth=/usr/lib /usr/local/lib
libs=-lm -lcrypt -lutil -lc
perllibs=-lm -lcrypt -lutil -lc
libc=, so=so, useshrplib=false, libperl=libperl.a
gnulibc_version=''
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
cccdlflags='-DPIC -fPIC', lddlflags='-shared -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP
USE_LARGE_FILES USE_PERLIO
Built under freebsd
Compiled at Jan 1 2008 08:38:01
%ENV:
PERL5LIB="/opt/local/lib/perl5/site_perl/5.10.0:/Library/Perl/5.8.8"
PERL5_CPANPLUS_IS_RUNNING="96108"
PERL5_CPAN_IS_RUNNING="96108"
@INC:
/opt/local/lib/perl5/site_perl/5.10.0
/Library/Perl/5.8.8
/usr/local/lib/perl5/5.10.0/i386-freebsd
/usr/local/lib/perl5/5.10.0
/usr/local/lib/perl5/site_perl/5.10.0/i386-freebsd
/usr/local/lib/perl5/site_perl/5.10.0
/usr/local/lib/perl5/site_perl/5.8.8
/usr/local/lib/perl5/site_perl
.