Bug #47760 for Encode: Testing Encode v. 2.34 appears to infinite-loop for hours

Thu Jul 09 20:43:43 2009 csjewell [...] cpan.org - Ticket created

Subject:

Testing Encode v. 2.34 appears to infinite-loop for hours - takes ctrl-C to stop

(I'm reporting this because unless a working version is uploaded within the next few days, 2.33 is the version that will have to end up in Strawberry Perl for the July 2009 release, because I won't use a module to create Strawberry Perl that I can't guarantee has non-test bugs.) The "make test" of the newest version of Encode ends up going into an infinite loop or "sticking" somehow such that I have to hit Ctrl-C to end it. I came back after about 8 hours (6:10pm) to see the output in the files given: Perl-Dist-WiX Test Output.txt: The output of the t/92_perl_5100.t test of Perl::Dist::WiX, copied and pasted into a file. Perl dash-V.txt: The output of "perl -V" on the perl executable that the test is generating. Perl::Dist::WiX captures the STDOUT and STDERR into files so that they don't go all over the screen and so I have information to send to you. Those files are: debug.out: The STDOUT of "make test" in C:\Perl-Dist-WiX\t\tmp92\build\Encode-2.34 that had been captured. debug.err: The STDERR of the same thing. If there is any more information that I can get to help you debug this, or anything you'd like to have me do, let me know. Note that Encode 2.33 does NOT do this, but will successfully test and build. I do not know if it fails on 5.8.x the same way. --Curtis Jewell

Subject:

debug.out

Download debug.out
application/octet-stream 851b

Message body not shown because it is not plain text.

Subject:

perl dash-V.txt

C:\Perl-Dist-WiX\t\tmp92\image\perl\bin>perl -V Summary of my perl5 (revision 5 version 10 subversion 0) configuration: Platform: osname=MSWin32, osvers=5.1, archname=MSWin32-x86-multi-thread uname='Win32 testperl 5.10.0.5 #1 Thu Jul 9 10:31:23 2009 i386' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc', ccflags =' -s -O2 -DWIN32 -DHAVE_DES_FCRYPT -DPERL_IMPLICIT_CONTE XT -DPERL_IMPLICIT_SYS -fno-strict-aliasing -DPERL_MSVCRT_READFIX', optimize='-s -O2', cppflags='-DWIN32' ccversion='', gccversion='3.4.5', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=12 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='long long', lseek size=8 alignbytes=8, prototype=define Linker and Libraries: ld='g++', ldflags ='-s -L"C:\PERL-D~2\t\tmp92\image\perl\lib\CORE" -L"C:\PER L-D~2\t\tmp92\image\c\lib"' libpth=C:\PERL-D~2\t\tmp92\image\c\lib libs= -lmsvcrt -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdlg32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lwinmm -lversion -lodbc32 -lodbccp32 perllibs= -lmsvcrt -lmoldname -lkernel32 -luser32 -lgdi32 -lwinspool -lcomdl g32 -ladvapi32 -lshell32 -lole32 -loleaut32 -lnetapi32 -luuid -lws2_32 -lmpr -lw inmm -lversion -lodbc32 -lodbccp32 libc=-lmsvcrt, so=dll, useshrplib=true, libperl=libperl510.a gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-mdll -s -L"C:\PERL-D~2\t\tmp92\image\perl\lib\CO RE" -L"C:\PERL-D~2\t\tmp92\image\c\lib"' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC USE_ITHREADS USE_LARGE_FILES USE_PERLIO Built under MSWin32 Compiled at Jul 9 2009 10:39:13 %ENV: PERL_EXTUTILS_AUTOINSTALL="--defaultdeps" @INC: C:/Perl-Dist-WiX/t/tmp92/image/perl/lib C:/Perl-Dist-WiX/t/tmp92/image/perl/site/lib .

Subject:

debug.err

Download debug.err
application/octet-stream 856b

Message body not shown because it is not plain text.

Subject:

Perl-Dist-WiX Test Output.txt

C:\Perl-Dist-WiX>prove -v t\92_perl_5100.t t\93_portable.t t\92_perl_5100.t .. 1..13 # C:\PERL-D~2\t\tmp92 ok 1 - Created C:\PERL-D~2\t\tmp92\output ok 2 - Created C:\PERL-D~2\t\tmp92\image ok 3 - Created C:\PERL-D~2\t\download ok 4 - Created C:\PERL-D~2\t\tmp92\build # [0] Starting build at Thu Jul 9 10:31:23 2009. # # Removing previous C:\PERL-D~2\t\tmp92\image # Building test dist @ Thu Jul 9 10:31:25 2009, may take several hours... (sorr y) ok 5 - The object isa t::lib::Test5100 Preparing dmake Preparing gcc-core Preparing gcc-g++ Preparing mingw-make Preparing binutils Preparing pexports Preparing mingw-runtime Preparing w32api Completed install_c_toolchain in 111 seconds Completed install_c_libraries in 0 seconds Pregenerating toolchain... Preparing perl Building perl 5.10.0... Testing perl... Installing perl... Building MSCHWERN/ExtUtils-MakeMaker-6.54.tar.gz... Building DLAND/File-Path-2.07.tar.gz... Building RKOBES/ExtUtils-Command-1.16.tar.gz... Building CHORNY/Win32API-File-0.1101.zip... Building YVES/ExtUtils-Install-1.52.tar.gz... Building RKOBES/ExtUtils-Manifest-1.56.tar.gz... Building ANDYA/Test-Harness-3.17.tar.gz... Building MSCHWERN/Test-Simple-0.92.tar.gz... Building DAGOLDEN/ExtUtils-CBuilder-0.2602.tar.gz... Building DAGOLDEN/ExtUtils-ParseXS-2.20.tar.gz... Building JPEACOCK/version-0.76.tar.gz... Building GBARR/Scalar-List-Utils-1.21.tar.gz... Building PMQS/Compress-Raw-Zlib-2.020.tar.gz... Building PMQS/Compress-Raw-Bzip2-2.020.tar.gz... Building PMQS/IO-Compress-2.020.tar.gz... Building ARJAY/Compress-Bzip2-2.09.tar.gz... Building TOMHUGHES/IO-Zlib-1.09.tar.gz... Building SMUELLER/PathTools-3.30.tar.gz... Building TJENNESS/File-Temp-0.22.tar.gz... Building JDB/Win32-WinError-0.03.tar.gz... Building BLM/Win32API-Registry-0.30.tar.gz... Building ADAMK/Win32-TieRegistry-0.26.tar.gz... Building ADAMK/File-HomeDir-0.86.tar.gz... Building PEREINAR/File-Which-0.05.tar.gz... Building ADAMK/Archive-Zip-1.30.tar.gz... Building KANE/Package-Constants-0.02.tar.gz... Building GAAS/IO-String-1.08.tar.gz... Building KANE/Archive-Tar-1.52.tar.gz... Building FERREIRA/Compress-unLZMA-0.03.tar.gz... Building ADAMK/Parse-CPAN-Meta-1.39.tar.gz... Building INGY/YAML-0.68.tar.gz... Building GAAS/Digest-MD5-2.39.tar.gz... Building GAAS/Digest-SHA1-2.12.tar.gz... Building MSHELOR/Digest-SHA-5.47.tar.gz... Building DAGOLDEN/Module-Build-0.34.tar.gz... Building ANDK/CPAN-1.9402.tar.gz... Building STSI/TermReadKey-2.30.01.tar.gz... Building ILYAZ/modules/Term-ReadLine-Perl-1.0302.tar.gz... Building RCLAMP/Text-Glob-0.08.tar.gz... Building SMUELLER/Data-Dumper-2.124.tar.gz... Building GAAS/URI-1.38.tar.gz... Building PETDANCE/HTML-Tagset-3.20.tar.gz... Building GAAS/HTML-Parser-3.61.tar.gz... Building GAAS/libwww-perl-5.829.tar.gz... Completed install_perl in 4592 seconds Running upgrade of all modules Building KANE/Archive-Extract-0.34.tar.gz... Building SMUELLER/Attribute-Handlers-0.85.tar.gz... Building SMUELLER/AutoLoader-5.68.tar.gz... Building RURBAN/B-Debug-1.11.tar.gz... Building JJORE/B-Lint-1.11.tar.gz... Building KANE/CPANPLUS-0.88.tar.gz... Building MHX/Devel-PPPort-3.19.tar.gz... Building GAAS/Digest-1.16.tar.gz... Building DANKOGAI/Encode-2.34.tar.gz... Terminating on signal SIGINT(2) Terminating on signal SIGINT(2) Terminate batch job (Y/N)? y

Fri Jul 10 00:35:42 2009 csjewell [...] cpan.org - Correspondence added

Subject:

[TEST FIX ATTACHED] Testing Encode v. 2.34 appears to infinite-loop for hours - takes ctrl-C to stop

More investigation (and fix attached) First things first, Never assume that the directory divider is forward slashes - you did so when creating $script. $blib, you got right! Therefore, starting at line 21 should be like this: my $blib = File::Spec->rel2abs( File::Spec->catdir( $FindBin::RealBin, File::Spec->updir, 'blib' ) ); my $script = File::Spec->catdir($blib, 'script', 'piconv'); (I used catdir instead of catfile, because sometimes [I'm thinking VMS here] catdir and catfile will not return the same output.) But that doesn't solve the main problem, which is that the capturing code is not Win32-compatible. Because of that, t\piconv.t does not seem to close its output, so that it hangs when redirected, and even if not, it fails. What I did in the attached test file is to make it use IPC::Run's routines (which do work) to do the capturing on Win32 systems. If it's not there on Win32 systems, the tests get skipped. (This version of the test passes on my system, btw. So I'll call this problem a test bug and force the installation of 2.34 when I go and create Strawberry. No need to hurry.)

# # $Id: piconv.t,v 0.1 2009/07/08 12:34:21 dankogai Exp $ # BEGIN { if ( $ENV{'PERL_CORE'} ) { print "1..0 # Skip: Don't know how to test this within perl's core\n"; exit 0; } } use strict; use FindBin; use File::Spec; use IPC::Open3 qw(open3); use IO::Select; use Test::More; my $WIN = $^O eq 'MSWin32'; if ($WIN) { eval { require IPC::Run; IPC::Run->VERSION(0.83); 1; } or plan skip_all => 'Win32 environments require IPC::Run 0.83 to complete this test'; } sub run_cmd (;$$); my $blib = File::Spec->rel2abs( File::Spec->catdir( $FindBin::RealBin, File::Spec->updir, 'blib' ) ); my $script = File::Spec->catdir($blib, 'script', 'piconv'); my @base_cmd = ( $^X, "-Mblib=$blib", $script ); plan tests => 5; { my ( $st, $out, $err ) = run_cmd; is( $st, 0, 'status for usage call' ); is( $out, undef ); like( $err, qr{^piconv}, 'usage' ); } { my($st, $out, $err) = run_cmd [qw(-S foobar -f utf-8 -t ascii), $script]; like($err, qr{unknown scheme.*fallback}i, 'warning for unknown scheme'); } { my ( $st, $out, $err ) = run_cmd [qw(-f utf-8 -t ascii ./non-existing/file)]; like( $err, qr{can't open}i ); } sub run_cmd (;$$) { my ( $args, $in ) = @_; my $out = "x" x 10_000; $out = ""; my $err = "x" x 10_000; $err = ""; if ($WIN) { IPC::Run->import(qw(run timeout)); my @cmd; if (defined $args) { @cmd = (@base_cmd, @$args); } else { @cmd = @base_cmd; } run(\@cmd, \$in, \$out, \$err, timeout(10)); my $st = $?; $out = undef if ($out eq ''); ( $st, $out, $err ); } else { $in ||= ''; my ( $in_fh, $out_fh, $err_fh ); use Symbol 'gensym'; $err_fh = gensym; # sigh... otherwise stderr gets just to $out_fh, not to $err_fh my $pid = open3( $in_fh, $out_fh, $err_fh, @base_cmd, @$args ) or die "Can't run @base_cmd @$args: $!"; print $in_fh $in; my $sel = IO::Select->new( $out_fh, $err_fh ); while ( my @ready = $sel->can_read ) { for my $fh (@ready) { if ( eof($fh) ) { $sel->remove($fh); last if !$sel->handles; } elsif ( $out_fh == $fh ) { my $line = <$fh>; $out .= $line; } elsif ( $err_fh == $fh ) { my $line = <$fh>; $err .= $line; } } } my $st = $?; ( $st, $out, $err ); } }

Fri Jul 10 00:35:45 2009 csjewell [...] cpan.org - Status changed from 'new' to 'open'

Sun Jul 12 20:48:35 2009 DANKOGAI [...] cpan.org - Correspondence added

Thanks, your piconv.t is now in my repository except for one line. --- /Users/dankogai/Downloads/piconv.t 2009-07-13 09:42:46.000000000 +0900 +++ t/piconv.t 2009-07-13 09:45:28.000000000 +0900 @@ -36,7 +36,7 @@ { my ( $st, $out, $err ) = run_cmd; is( $st, 0, 'status for usage call' ); - is( $out, undef ); + is( $out, $WIN ? undef : '' ); like( $err, qr{^piconv}, 'usage' ); } Now t/piconv.t should run both on *nix and Win32. Dan the Encode Maintainer On Fri Jul 10 00:35:42 2009, CSJEWELL wrote: Show quoted text

> More investigation (and fix attached) > > First things first, Never assume that the directory divider is forward > slashes - you did so when creating $script. $blib, you got right! > > Therefore, starting at line 21 should be like this: > > my $blib = > File::Spec->rel2abs( > File::Spec->catdir( $FindBin::RealBin, File::Spec->updir, 'blib' ) ); > my $script = File::Spec->catdir($blib, 'script', 'piconv'); > > (I used catdir instead of catfile, because sometimes [I'm thinking VMS > here] catdir and catfile will not return the same output.) > > But that doesn't solve the main problem, which is that the capturing > code is not Win32-compatible. > > Because of that, t\piconv.t does not seem to close its output, so that > it hangs when redirected, and even if not, it fails. > > What I did in the attached test file is to make it use IPC::Run's > routines (which do work) to do the capturing on Win32 systems. If it's > not there on Win32 systems, the tests get skipped. > > (This version of the test passes on my system, btw. So I'll call this > problem a test bug and force the installation of 2.34 when I go and > create Strawberry. No need to hurry.)

Sun Jul 12 20:48:37 2009 DANKOGAI [...] cpan.org - Status changed from 'open' to 'resolved'

Bug #47760 for Encode: Testing Encode v. 2.34 appears to infinite-loop for hours - takes ctrl-C to stop