| Subject: | Test-Unit 0.25 fails "make test" |
Problem:
On one of our Linux/Intel boxes, Test-Unit fails its "make test"
tests. On another box, which runs Solaris/Sparc, the tests
succceed. (System details and full output are at the end of this
report.)
The specific test which is failing (on the Linux box) is this one:
not ok ERROR test_numericness
tlib/AssertTest.pm:48 - test_numericness(Class::Inner::__A28)
For string '0xF00', expect f but got t
Analysis:
That particular test caselet (at t/tlib/AssertTest.pm line 37) looks
like:
'0xF00' => 'f', # controversial? but if you +=10 then it's == 10
So it would seem that the developers are unsure what the result is
*supposed* to be ... with good reason, as it turns out :-/
On the Linux machine, this Perl program:
$s = '0xF00';
$s += 10;
print "\$s = $s\n";
$i = 0xF00 + 10;
print "\$i = $i\n";
prints:
$s = 3850
$i = 3850
On this box, the assertion in the comment is wrong. The "0x" prefix
is recognized in the value of a string scalar, the same as it is in
a numeric literal.
On the Solaris box, by contrast, that same program does indeed
print:
$s = 10
$i = 3850
suggesting that the "0x" is *not* recognized in $s's value, and
instead the "x" terminates the parsing of the string.
Interestingly, this variant (note the "3x"):
$s = '3xF00';
$s += 10;
print "\$s = $s\n";
prints:
$s = 13
on *both* boxes.
One could argue that is_numeric() should return TRUE iff Perl itself
thinks the string is numeric -- and if different Perls disagree with
each other on that point, is_numeric() should agree with the Perl
binary it's running under. In this case, the "0xF00" test case is
simply meaningless, and should be deleted.
But to me, a more compelling argument is that is_numeric() should
return the same result on all systems. Presumably, only one of the
above string->numeric conversions is correct (though I won't hazard
a guess as to which one), and the other one is a bug in the
corresponding Perl binary.
Such a bug can cause system-dependent problems in my application,
and unit testing should help detect those problems, even if the
underlying bug isn't in my own code.
Feel free to send me private email if you have a candidate fix that you
want me to test on both of these systems.
Thanks.
========== Listings start here =================
On the Linux box, the "make test "output looks like:
$ make test
PERL_DL_NONLAZY=1 /usr/local/bin/perl -Iblib/arch -Iblib/lib -I/usr/lib/perl/5.6.1 -I/usr/share/perl/5.6.1 -e 'use Test::Harness qw(&runtests $verbose); $verbose=0; runtests @ARGV;' t/*.t
t/all_tests.......ok
t/assert..........FAILED test 29
Failed 1/40 tests, 97.50% okay
t/try_examples....ok
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
t/assert.t 40 1 2.50% 29
Failed 1/3 test scripts, 66.67% okay. 1/89 subtests failed, 98.88% okay.
make: *** [test_dynamic] Error 11
$
Digging one level deeper, again on the Linux box:
$ cd t; prove -lbv assert.t
assert....STARTING TEST RUN
1..40
ok PASS 0 == 'foo'
ok PASS 1 == '1.0'
ok PASS '1.0' == 1
ok PASS foo eq foo
ok PASS 0E0 == 0
ok PASS 0 == 'foo'
ok PASS both undef
ok PASS 0 == 0
ok PASS 0 == 0.0
ok PASS 0.0 == 0
ok PASS 0.0 == 0.0
ok PASS '' == ''
ok PASS 1 != 'foo'
ok PASS 'foo' ne 0
ok PASS 'foo' ne 1
ok PASS 0 != 1
ok PASS 'foo' ne 'bar'
ok PASS 'foo' ne undef
ok PASS undef ne 'foo'
ok PASS test_succeed_assert_not_null
ok PASS test_assert_raises
ok PASS test_ok_not_equals
ok PASS test_fail
ok PASS test_ok_equals
ok PASS test_assert_equals
ok PASS test_multi_assert
ok PASS test_success_assert_not_equals
ok PASS test_assert_deep_equals
not ok ERROR test_numericness
tlib/AssertTest.pm:48 - test_numericness(Class::Inner::__A28)
For string '0xF00', expect f but got t
ok PASS test_assert
ok PASS test_assert_matches
ok PASS test_fail_assert_not_equals
ok PASS test_ok_bad_args
ok PASS test_ok_boolean
ok PASS test_fail_assert_null
ok PASS test_assert_str_equals
ok PASS test_fail_assert_not_null
ok PASS test_assert_equals_null
ok PASS test_assert_does_not_match
ok PASS test_succeed_assert_null
FAILED test 29
Failed 1/40 tests, 97.50% okay
Failed Test Stat Wstat Total Fail Failed List of Failed
-------------------------------------------------------------------------------
assert.t 40 1 2.50% 29
Failed 1/1 test scripts, 0.00% okay. 1/40 subtests failed, 97.50% okay.
$
Most of the Solaris output is boring, so I haven't included it. The
only interesting line is:
ok PASS test_numericness
System Info for the Linux box, on which the tests FAIL:
System: Linux Debian 3.0
Kernel: Linux somehostname 2.4.18 #1 SMP Wed May 15 13:46:32 EST 2002 i686 unknown
Test-Unit: 0.25
"perl -v" says:
This is perl, v5.6.1 built for i386-linux
Copyright 1987-2001, Larry Wall
[licensing terms snipped]
"perl -V" says:
Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration:
Platform:
osname=linux, osvers=2.4.13, archname=i386-linux
uname='linux duende 2.4.13 #1 wed oct 31 19:18:07 est 2001 i686 unknown '
config_args='-Dccflags=-DDEBIAN -Dcccdlflags=-fPIC -Darchname=i386-linux -Dprefix=/usr -Dprivlib=/usr/share/perl/5.6.1 -Darchlib=/usr/lib/perl/5.6.1 -Dvendorprefix=/usr -Dvendorlib=/usr/share/perl5 -Dvendorarch=/usr/lib/perl5 -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl/5.6.1 -Dsitearch=/usr/local/lib/perl/5.6.1 -Dman1dir=/usr/share/man/man1 -Dman3dir=/usr/share/man/man3 -Dman1ext=1 -Dman3ext=3perl -Dpager=/usr/bin/sensible-pager -Uafs -Ud_csh -Uusesfio -Duseshrplib -Dlibperl=libperl.so.5.6.1 -Dd_dosuid -des'
hint=recommended, useposix=true, d_sigaction=define
usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef
useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
Compiler:
cc='cc', ccflags ='-DDEBIAN -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O2',
cppflags='-DDEBIAN -fno-strict-aliasing -I/usr/local/include'
ccversion='', gccversion='2.95.4 (Debian prerelease)', gccosandvers=''
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=4, usemymalloc=n, prototype=define
Linker and Libraries:
ld='cc', ldflags =' -L/usr/local/lib'
libpth=/usr/local/lib /lib /usr/lib
libs=-lgdbm -ldb -ldl -lm -lc -lcrypt
perllibs=-ldl -lm -lc -lcrypt
libc=/lib/libc-2.2.4.so, so=so, useshrplib=true, libperl=libperl.so.5.6.1
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-rdynamic'
cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: USE_LARGE_FILES
Built under linux
Compiled at Jan 11 2002 04:09:18
%ENV:
PERL_HOME="/usr/local"
@INC:
/usr/local/lib/perl/5.6.1
/usr/local/share/perl/5.6.1
/usr/lib/perl5
/usr/share/perl5
/usr/lib/perl/5.6.1
/usr/share/perl/5.6.1
/usr/local/lib/site_perl
.
System Info for the Solaris box, on which the tests SUCCEED:
System: Solaris 8 Sparc
Kernel: SunOS anotherhostname 5.8 Generic_108528-12 sun4u sparc SUNW,Ultra-4 Solaris
Test-Unit: 0.25
"perl -v" says:
This is perl, v5.6.1 built for sun4-solaris-thread-multi
(with 1 registered patch, see perl -V for more detail)
Copyright 1987-2001, Larry Wall
Binary build 631 provided by ActiveState Tool Corp. http://www.ActiveState.com
Built 20:33:12 Jan 3 2002
[licensing terms snipped]
"perl -V" says:
Summary of my perl5 (revision 5.0 version 6 subversion 1) configuration:
Platform:
osname=solaris, osvers=2.6, archname=sun4-solaris-thread-multi
uname='sunos sparky 5.6 generic_105181-26 sun4u sparc sunw,ultra-5_10 '
config_args='-des -Dcc=gcc -Dcf_by=ActiveState -Dcf_email=ActivePerl@ActiveState.com -Uinstallusrbinperl -Ud_sigsetjmp -Dusethreads -Duseithreads -Dinc_version_list=5.6.0/$archname 5.6.0 -Dprefix=/usr/local/ActivePerl-5.6'
hint=recommended, useposix=true, d_sigaction=define
usethreads=define use5005threads=undef useithreads=define usemultiplicity=define
useperlio=undef d_sfio=undef uselargefiles=define usesocks=undef
use64bitint=undef use64bitall=undef uselongdouble=undef
Compiler:
cc='gcc', ccflags ='-DUSE_REENTRANT_API -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -fno-strict-aliasing -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64',
optimize='-O',
cppflags='-DUSE_REENTRANT_API -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -fno-strict-aliasing -I/usr/local/include'
ccversion='', gccversion='2.95.2 19991024 (release)', gccosandvers='solaris2.6'
intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
alignbytes=8, usemymalloc=n, prototype=define
Linker and Libraries:
ld='gcc', ldflags =' -L/usr/local/lib '
libpth=/usr/local/lib /usr/lib /usr/ccs/lib
libs=-lsocket -lnsl -ldl -lm -lposix4 -lpthread -lc
perllibs=-lsocket -lnsl -ldl -lm -lposix4 -lpthread -lc
libc=/lib/libc.so, so=so, useshrplib=false, libperl=libperl.a
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' '
cccdlflags='-fPIC', lddlflags='-G -L/usr/local/lib'
Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY USE_ITHREADS USE_LARGE_FILES PERL_IMPLICIT_CONTEXT
Locally applied patches:
ActivePerl Build 631
Built under solaris
Compiled at Jan 3 2002 20:33:12
%ENV:
PERL_HOME="/usr/local/ActivePerl-5.6"
@INC:
/usr/local/ActivePerl-5.6/lib/5.6.1/sun4-solaris-thread-multi
/usr/local/ActivePerl-5.6/lib/5.6.1
/usr/local/ActivePerl-5.6/lib/site_perl/5.6.1/sun4-solaris-thread-multi
/usr/local/ActivePerl-5.6/lib/site_perl/5.6.1
/usr/local/ActivePerl-5.6/lib/site_perl
.