Bug #71064 for Gravatar-URL: t/libravatar.t test failure

Mon Sep 19 02:08:03 2011 DOUGDUDE [...] cpan.org - Ticket created

Subject:

t/libravatar.t test failure

I am currently getting a test failure in t/libravatar.t due to the random number generation. I see the srand(42), but for whatever reason, at least in my perl, I am not getting the values listed in the comments. Here is an example run of the test: t\libravatar.t .. 1/? Use of uninitialized value $target in pattern match (m//) at C:\strawberry\cpan\build\Gravatar-URL-1.05-LveUNU\lib/Libravatar/URL.pm line 203. Use of uninitialized value $port in pattern match (m//) at C:\strawberry\cpan\build\Gravatar-URL-1.05-LveUNU\lib/Libravatar/URL.pm line 206. # Failed test 'avatars1.example.com' # at t\libravatar.t line 237. # Structures begin differing at: # $got->[0] = 'avatars4.example.com' # $expected->[0] = 'avatars1.example.com' # Failed test 'avatars10.example.com' # at t\libravatar.t line 237. # Structures begin differing at: # $got->[0] = 'avatars1.example.com' # $expected->[0] = 'avatars10.example.com' # Looks like you failed 2 tests of 38. t\libravatar.t .. Dubious, test returned 2 (wstat 512, 0x200) Failed 2/38 subtests Test Summary Report ------------------- t\libravatar.t (Wstat: 512 Tests: 38 Failed: 2) Failed tests: 35, 38 Non-zero exit status: 2 Files=1, Tests=38, 0 wallclock secs ( 0.06 usr + 0.00 sys = 0.06 CPU) Result: FAIL I am, however, able to pass the test by changing the seed value from 42 to 127759. I came up with this number using brute force: use v5.10; use strict; use warnings; my @expect = (49, 0, 1, 40); my @max = (67, 2, 11, 97); my $seed = -1; sub rand_pass { srand $_[0]; for (my $i = 0; $i < @expect; $i++) { if ($expect[$i] != int rand $max[$i]) { return 0; } } return 1; } 1 until (rand_pass(++$seed)); say 'Seed: ' . ($seed); Anyhow, I'm not sure what the cause would be. It is possible there is something wrong with my perl. At the end here, I am including my perl version. If other platforms are passing the tests with 42 and others or not, may I suggest you modify the tests to call srand() before each call to srv_hostname() (perhaps having the srand() value in @srv_tests). Summary of my perl5 (revision 5 version 12 subversion 3) configuration: Platform: osname=MSWin32, osvers=6.1, archname=MSWin32-x64-multi-thread uname='Win32 strawberryperl 5.12.3.0 #1 Sun May 15 09:43:50 2011 x64' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags =' -s -O2 -DWIN32 -DHAVE_DES_FCRYPT -DWIN64 -DCONSERVATIVE -DUSE_SITECUSTOMIZE -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -fno-strict-aliasing -mms-bitfields -DPERL_MSVCRT_READFIX', optimize='-s -O2', cppflags='-DWIN32' ccversion='', gccversion='4.4.3', gccosandvers='' intsize=4, longsize=4, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long long', ivsize=8, nvtype='double', nvsize=8, Off_t='long long', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='g++', ldflags ='-s -L"C:\strawberry\perl\lib\CORE" -L"C:\strawberry\c\lib"' libpth=C:\strawberry\c\lib C:\strawberry\c\x86_64-w64-mingw32\lib libs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32 perllibs=-lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 -lcomctl32 libc=, so=dll, useshrplib=true, libperl=libperl512.a gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-mdll -s -L"C:\strawberry\perl\lib\CORE" -L"C:\strawberry\c\lib"' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_PERLIO USE_PERL_ATOF USE_SITECUSTOMIZE Built under MSWin32 Compiled at May 15 2011 09:52:35 %ENV: PERL5_CPANPLUS_IS_RUNNING="5240" PERL5_CPAN_IS_RUNNING="5240" PERL_JSON_BACKEND="JSON::XS" PERL_YAML_BACKEND="YAML" @INC: C:/strawberry/perl/site/lib C:/strawberry/perl/vendor/lib C:/strawberry/perl/lib .

Tue Sep 20 04:29:21 2011 francois [...] fmarier.org - Correspondence added

Subject:	Re: [rt.cpan.org #71064] t/libravatar.t test failure
Date:	Tue, 20 Sep 2011 20:27:25 +1200
To:	Douglas Christopher Wilson via RT <bug-Gravatar-URL [...] rt.cpan.org>
From:	Francois Marier <fmarier [...] gmail.com>

On 2011-09-19 at 02:08:04, Douglas Christopher Wilson via RT wrote: Show quoted text

> I am, however, able to pass the test by changing the seed value from 42 > to 127759. I came up with this number using brute force:

Thanks for the report Douglas. It does look like the random number generator returns different values on Windows than it does on other platforms: http://matrix.cpantesters.org/?dist=Gravatar-URL+1.05 I thought about mocking the call to srand() but then again I'd like the test to be as close as possible to what happens when the code runs for real. So I'm thinking that perhaps I should, inside the test suite, detect the platform that's in use and then seed the random number generator differently. Cheers, Francois -- Francois Marier identi.ca/fmarier http://feeding.cloud.geek.nz twitter.com/fmarier

Tue Sep 20 04:29:22 2011 The RT System itself - Status changed from 'new' to 'open'

Tue Sep 20 19:17:26 2011 DOUGDUDE [...] cpan.org - Correspondence added

On Tue Sep 20 04:29:21 2011, FMARIER wrote: Show quoted text

> Thanks for the report Douglas. It does look like the random number

generator Show quoted text

> returns different values on Windows than it does on other platforms: > > http://matrix.cpantesters.org/?dist=Gravatar-URL+1.05 > > I thought about mocking the call to srand() but then again I'd like

the test Show quoted text

> to be as close as possible to what happens when the code runs for real. So > I'm thinking that perhaps I should, inside the test suite, detect the > platform that's in use and then seed the random number generator > differently. >

Oh, I see. I wanted to look more into the kinds of reports you were getting from CPAN Testers, but since the server crash the results and such have been down ): It is possible you may be able to pick the seed according to $Config{randfunc}. I see on my linux box $Config{randfunc} eq 'drand48' and the seed 42 produces the expected results. The Windows machine has $Config{randfunc} eq 'rand'. Reading the man page of drand48 it sounds like that implementation should work the same across platforms, but rand() and random() (other possible values for $Config{randfunc}) seems to be very generic. So possibly something like: SKIP: { skip qq{Don't know seed for $Config{randfunc} on $^O} scalar(@srv_tests) if $Config{randfunc} ne 'drand48'; for my $test (@srv_tests) { my ($srv_strings, $pair) = @$test; my @srv_records = (); for $str (@$srv_strings) { my $record = Net::DNS::RR->new($str); push @srv_records, $record; } my @result = Libravatar::URL::srv_hostname(@srv_records); is_deeply \@result, $pair; } } Or possibly something even more generic like: my $passes = 0; for my $test (@srv_tests) { my ($srv_strings, $pair) = @$test; my @srv_records = (); for $str (@$srv_strings) { my $record = Net::DNS::RR->new($str); push @srv_records, $record; } my @result = Libravatar::URL::srv_hostname(@srv_records); SKIP: { skip qq{expected $pair->[0]}, 1 unless eq_array(\@result, $pair); pass(); $passes++; } } # NOTE: Below is still subject to possible failure, but greatly reduces the probability ok $passes > 0, 'At least one SRV test passed'; And even other possibility would be to make this an author test, so that the author can see if the function is working correctly in the code, while users won't be subject to the possibility of a false test failure. These are just things to consider.

Wed Sep 21 17:00:01 2011 mschwern [...] cpan.org - Correspondence added

You can't rely that a given random number generation function exists on a given system, or works. Simplest thing to do is override rand() to return a known stream of numbers (I can't find a module for this). BEGIN { *CORE::GLOBAL::rand = sub { ... }; } This isn't entirely robust since changes to the code or submodules might cause rand() to be called when you don't expect it. The most robust thing is to separate the code to get the random piece out into its own function (ie. my $random_number = int(rand($total_weight + 1)); ) and then override it in the test to return something deterministic. Since there's no object, you have to do some surgery... my $rand = 0.123456; local *Libravatar::URL::rand_weight = sub { my $total_weight = shift; return int($rand * ($total_weight + 1)); }; rand_weight() can then be tested to ensure it only returns values inside the accepted range. Finally, if you want to mock as little as possible, you can instead have Libravatar::URL put a box around rand() and override just that. # In Libravatar::URL sub my_rand { return rand( @_ ? $_[0] : () ); } # In srv_whatever my $random_number = int(my_rand($total_weight + 1)); # In the test my $rand = 0.123456; local *Libravatar::URL::my_rand = sub { my $multiplier = shift || 1; return $rand * $multiplier; };

Wed Aug 10 20:45:58 2016 mschwern [...] cpan.org - Correspondence added

See https://github.com/schwern/Gravatar-URL/issues/7

Wed Aug 10 20:45:59 2016 mschwern [...] cpan.org - Status changed from 'open' to 'resolved'