Skip Menu |

This queue is for tickets about the ExtUtils-MakeMaker CPAN distribution.

Report information
The Basics
Id: 69528
Status: resolved
Worked: 5 min
Priority: 0/
Queue: ExtUtils-MakeMaker

People
Owner: BINGOS [...] cpan.org
Requestors: pete.a64 [...] gmail.com
perl [...] greerga.m-l.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 6.58
Fixed in: (no value)



Subject: Windows XP SP3, MSVC 9, Perl 5.12.3 Failed Tests Report
This is a build report with no work done to patch the problems given in the attached report >> Compile, Build, and Verbose Test log using Perl 5.12.3, Win32 Visual C 2008 Professional, nmake, on Windows XP SP3. I have also built and tested verbosely the ExtUtils-MakeMaker_6.58 module with no failures using a Perl 5.12.3 build: Win32 GCC 4.5.2 using dmake Version 4.8-20070327-SHAY. Thus, the failures are seemingly specific to the MS Visual C build using nmake. I have not tested this using a Visual C 6 Perl 5.12.3 build but assume that ActiveState's testers reports would feed the required information. -- Regards, Pete Armstrong pete.a64@gmail.com
Subject: ExtUtils-MakeMaker-6.58.log
Download ExtUtils-MakeMaker-6.58.log
application/octet-stream 3.1k

Message body not shown because it is not plain text.

Subject: 6.58 in blead on MSWin32 fails tests
6.58 in blead perl on MSWin32 (MSVC nmake) fails tests: ../cpan/ExtUtils-MakeMaker/t/basic.t........................FAILED 19, 62, 67, 72, 82 ../cpan/ExtUtils-MakeMaker/t/basic.t........................FAILED Non-zero exit status: 2 ../cpan/ExtUtils-MakeMaker/t/basic.t........................FAILED Bad plan. You planned 171 tests but ran 84. ../cpan/ExtUtils-MakeMaker/t/oneliner.t.....................FAILED 4, 7-11 ../cpan/ExtUtils-MakeMaker/t/oneliner.t.....................FAILED Non-zero exit status: 6 Full Test::Smoke log available here (or attached): http://m-l.org/~perl/smoke/perl/win32/blead/log06e9ce89328d9d04b2dd5693134f0ff0a14dec11.log.gz
Subject: log06e9ce89328d9d04b2dd5693134f0ff0a14dec11.log.gz

Message body not shown because it is not plain text.

Sorry, but the attached log file does not contain an error. It appears to be truncated. It stops at... cp lib/ExtUtils/MM_Any.pm blib\lib\ExtUtils\MM_Any.pm E:\usr\bin\perl.exe "-Iblib\arch" "-Iblib\lib" -MExtUtils::Command -e "cp" -- bin/instmodsh blib\script\instmodsh pl2bat.bat blib\script\instmodsh
Thank you for your report, but I'm sorry there is no way of knowing from the information given if that's a bug in blead or a bug in MakeMaker. Does 6.58 work with a stable version of Perl and your setup? If so, it's a bug in blead and please report it to p5p. If not, it's a bug in MakeMaker and please report it here. I'm going to set it as rejected, but please reopen it if it turns out to be a real bug.
From: pete.a64 [...] gmail.com
On Sun Jul 17 15:06:33 2011, MSCHWERN wrote: Show quoted text
> Sorry, but the attached log file does not contain an error. It appears > to be truncated. It stops at... > > cp lib/ExtUtils/MM_Any.pm blib\lib\ExtUtils\MM_Any.pm > E:\usr\bin\perl.exe "-Iblib\arch" "-Iblib\lib" -MExtUtils::Command -e > "cp" -- bin/instmodsh blib\script\instmodsh > pl2bat.bat blib\script\instmodsh
OK - I have attached a new test report. -- Regards, Pete Armstrong pete.a64@gmail.com
Subject: ExtUtils-MakeMaker-6.58-BXWsOD-13.11.41-07312011.rpt

Message body not shown because it is not plain text.

On Sun Jul 17 22:29:10 2011, MSCHWERN wrote: Show quoted text
> Thank you for your report, but I'm sorry there is no way of knowing from > the information given if that's a bug in blead or a bug in MakeMaker. >
It's a bug in EU::MM. Specifically in the changes made to MM_Win32.pm's quote_literal() in this commit: https://github.com/Perl-Toolchain-Gang/ExtUtils-MakeMaker/commit/1f0f4a921c578e16dc457b58297cf7dbc92e8362#lib/ExtUtils/MM_Win32.pm See the thread here for more details: http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2011-08/msg00681.html In particular, see: http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2011-08/msg00725.html which shows that the processing of command-line arguments has changed between VC6/VC7 and VC8/VC9/VC10 in such a way that the quoting being done in quote_literal() does not work correctly with VC8+, causing tests in oneliner.t to fail with VC8+. (They pass with VC6/VC7, but I suspect the quoting is still wrong anyway - just not in a way that causes the tests that we have to fail.) I note that the current version of quote_literal() is better than it was: the previous version was even more simplistic, but oneliner.t didn't fail because it hardly tested anything! The new tests added in commit 6f6c1fa2bb do not pass with the previous version of quote_literal() with any version of the compiler. However, the quoting currently being done does not at all match the Microsoft documentation of (1) cmd.exe command-line parsing: http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/cmd.mspx?mfr=true and (2) C/C++ startup command-line parsing: http://msdn.microsoft.com/en-us/library/17w5ykft%28v=VS.100%29.aspx I have found a very useful page here which puts it all together with some helpful examples (in particular, see section 8.): http://www.autohotkey.net/~deleyd/parameters/parameters.htm On the basis of that, I'm currently thinking that something along the following lines should work. I will test this (with VC6 and VC10) and post a proper patch when I have it right, unless somebody wants to beat me to it: sub quote_literal { my($self, $text) = @_; # See: http://www.autohotkey.net/~deleyd/parameters/parameters.htm#CPP # Apply the Microsoft C/C++ parsing rules $text =~ s{\\\\"}{\\\\\\\\\\"}g; # \\" -> \\\\\" $text =~ s{(?<!\\)\\"}{\\\\\\"}g; # \" -> \\\" $text =~ s{(?<!\\)"}{\\"}g; # " -> \" $text = qq{"$text"} if $text =~ /[ \t]/; # Apply the Command Prompt parsing rules (cmd.exe) my @text = split /("[^"]*")/, $text; s{([<>|&^()@!])}{^$1}g foreach grep { !/^"[^"]*"$/ } @text; $text = join('', @text); # dmake eats '{' inside double quotes and leaves alone { outside double # quotes; however it transforms {{ into { either inside and outside double # quotes. It also translates }} into }. The escaping below is not # 100% correct. if( $self->is_make_type('dmake') ) { $text =~ s/{/{{/g; $text =~ s/}}/}}}/g; } return $text; }
On Mon Aug 22 03:48:49 2011, SHAY wrote: Show quoted text
> I have found a very useful page here which puts it all together with > some helpful examples (in particular, see section 8.): > > http://www.autohotkey.net/~deleyd/parameters/parameters.htm > > On the basis of that, I'm currently thinking that something along the > following lines should work. I will test this (with VC6 and VC10) and > post a proper patch when I have it right, unless somebody wants to > beat me to it: >
I've now tested with five different perls - VC6/nmake, VC6/dmake, VC10/nmake, VC10/dmake and GCC/dmake - and the attached patch (against 6.59) gets all tests passing (and adds several more based on examples on the web page cited above). The patch also changes the handling of { and } with dmake, which wasn't working correctly otherwise (at least, not after the other changes that I have made). The dmake manual clearly states that {{ is expanded to { and }} is expanded to }, with no mention of different behaviour inside or outside of double quotes. I ran the following makefile using my current 4.12-20090907-SHAY build and Sarathy's old "4.10, PL 1" build: .USESHELL : NONE = { {{ {{{ {{{{ {{{{{ {{{{{{ }}}}}} }}}}} }}}} }}} }} } SINGLE = '{ {{ {{{ {{{{ {{{{{ {{{{{{ }}}}}} }}}}} }}}} }}} }} }' DOUBLE = "{ {{ {{{ {{{{ {{{{{ {{{{{{ }}}}}} }}}}} }}}} }}} }} }" all : @echo $(NONE) @echo $(SINGLE) @echo $(DOUBLE) and I got the same results both times: { { {{ {{ {{{ {{{ }}} }}} }} }} } } '{ { {{ {{ {{{ {{{ }}} }}} }} }} } }' "{ { {{ {{ {{{ {{{ }}} }}} }} }} } }" Finally, the patch also provides a MAKE parameter in various places where MM objects are created by the test suite, otherwise MM_Any.pm line 374 noisily warns about an uninitialized value in its lc().
Subject: quote.patch
diff -ruN ExtUtils-MakeMaker-6.59.orig/lib/ExtUtils/MM_Win32.pm ExtUtils-MakeMaker-6.59/lib/ExtUtils/MM_Win32.pm --- ExtUtils-MakeMaker-6.59.orig/lib/ExtUtils/MM_Win32.pm 2011-08-05 12:13:10.000000000 +0100 +++ ExtUtils-MakeMaker-6.59/lib/ExtUtils/MM_Win32.pm 2011-08-22 22:06:41.720976400 +0100 @@ -487,42 +487,28 @@ sub quote_literal { my($self, $text) = @_; - # DOS batch processing is hilarious: - # Quotes need to be converted into triple quotes. - # Certain special characters need to be escaped with a caret if an odd - # number of quotes came before them. - my @text = split '', $text; - my $quote_count = 0; - my %caret_chars = map { $_ => 1 } qw( < > | ); - for my $char ( @text ) { - if ( $char eq '"' ) { - $quote_count++; - $char = '"""'; - } - elsif ( $caret_chars{$char} and $quote_count % 2 ) { - $char = "^$char"; - } - elsif ( $char eq "\\" ) { - $char = "\\\\"; - } - } - $text = join '', @text; + # See: http://www.autohotkey.net/~deleyd/parameters/parameters.htm#CPP + + # Apply the Microsoft C/C++ parsing rules + $text =~ s{\\\\"}{\\\\\\\\\\"}g; # \\" -> \\\\\" + $text =~ s{(?<!\\)\\"}{\\\\\\"}g; # \" -> \\\" + $text =~ s{(?<!\\)"}{\\"}g; # " -> \" + $text = qq{"$text"} if $text =~ /[ \t]/; + + # Apply the Command Prompt parsing rules (cmd.exe) + my @text = split /("[^"]*")/, $text; + # We should also escape parentheses, but it breaks one-liners containing + # $(MACRO)s in makefiles. + s{([<>|&^@!])}{^$1}g foreach grep { !/^"[^"]*"$/ } @text; + $text = join('', @text); - # There is a terribly confusing edge case here, where this will do entirely the wrong thing: - # perl -e "use Data::Dumper; @ARGV = '%PATH%'; print Dumper( \@ARGV );print qq{@ARGV};" -- - # I have no idea how to fix this manually, much less programmatically. - # However as it is such a rare edge case i'll just leave this documentation here and hope it never happens. - - # dmake eats '{' inside double quotes and leaves alone { outside double - # quotes; however it transforms {{ into { either inside and outside double - # quotes. It also translates }} into }. The escaping below is not - # 100% correct. + # dmake expands {{ to { and }} to }. if( $self->is_make_type('dmake') ) { $text =~ s/{/{{/g; - $text =~ s/}}/}}}/g; + $text =~ s/}/}}/g; } - return qq{"$text"}; + return $text; } diff -ruN ExtUtils-MakeMaker-6.59.orig/t/MM_Win32.t ExtUtils-MakeMaker-6.59/t/MM_Win32.t --- ExtUtils-MakeMaker-6.59.orig/t/MM_Win32.t 2011-02-07 15:37:00.000000000 +0000 +++ ExtUtils-MakeMaker-6.59/t/MM_Win32.t 2011-08-22 21:25:09.394203000 +0100 @@ -107,7 +107,7 @@ # init_others(): check if all keys are created and set? # qw( TOUCH CHMOD CP RM_F RM_RF MV NOOP TEST_F LD AR LDLOADLIBS DEV_NUL ) { - my $mm_w32 = bless( { BASEEXT => 'Foo' }, 'MM' ); + my $mm_w32 = bless( { BASEEXT => 'Foo', MAKE => $Config{make} }, 'MM' ); $mm_w32->init_others(); my @keys = qw( TOUCH CHMOD CP RM_F RM_RF MV NOOP TEST_F LD AR LDLOADLIBS DEV_NULL ); @@ -124,6 +124,7 @@ NAME => 'TestMM_Win32', VERSION => '1.00', PM => { 'MM_Win32.pm' => 1 }, + MAKE => $Config{make}, }, 'MM'; # XXX Hack until we have a proper init method. diff -ruN ExtUtils-MakeMaker-6.59.orig/t/oneliner.t ExtUtils-MakeMaker-6.59/t/oneliner.t --- ExtUtils-MakeMaker-6.59.orig/t/oneliner.t 2011-08-02 16:48:10.000000000 +0100 +++ ExtUtils-MakeMaker-6.59/t/oneliner.t 2011-08-22 17:51:39.836565200 +0100 @@ -6,15 +6,16 @@ chdir 't'; +use Config; use MakeMaker::Test::Utils; -use Test::More tests => 11; +use Test::More tests => 16; use File::Spec; my $TB = Test::More->builder; BEGIN { use_ok('ExtUtils::MM') } -my $mm = bless { NAME => "Foo" }, 'MM'; +my $mm = bless { NAME => "Foo", MAKE => $Config{make} }, 'MM'; isa_ok($mm, 'ExtUtils::MakeMaker'); isa_ok($mm, 'ExtUtils::MM_Any'); @@ -47,6 +48,13 @@ try_oneliner(q{print " < \"\" < \" < \" < "}, [], q{ < "" < " < " < }, 'quotes and brackets mixed' ); try_oneliner(q{print " < \" | \" < | \" < \" < "}, [], q{ < " | " < | " < " < }, 'brackets, pipes and quotes' ); +# some examples from http://www.autohotkey.net/~deleyd/parameters/parameters.htm#CPP +try_oneliner(q{print q[ &<>^|()@ ! ]}, [], q{ &<>^|()@ ! }, 'example 8.1' ); +try_oneliner(q{print q[ &<>^|@()!"&<>^|@()! ]}, [], q{ &<>^|@()!"&<>^|@()! }, 'example 8.2' ); +try_oneliner(q{print q[ "&<>^|@() !"&<>^|@() !" ]}, [], q{ "&<>^|@() !"&<>^|@() !" }, 'example 8.3' ); +try_oneliner(q{print q[ "C:\TEST A\" ]}, [], q{ "C:\TEST A\" }, 'example 8.4' ); +try_oneliner(q{print q[ "C:\TEST %&^ A\" ]}, [], q{ "C:\TEST %&^ A\" }, 'example 8.5' ); + # XXX gotta rethink the newline test. The Makefile does newline # escaping, then the shell. diff -ruN ExtUtils-MakeMaker-6.59.orig/t/split_command.t ExtUtils-MakeMaker-6.59/t/split_command.t --- ExtUtils-MakeMaker-6.59.orig/t/split_command.t 2011-02-07 15:37:00.000000000 +0000 +++ ExtUtils-MakeMaker-6.59/t/split_command.t 2011-08-22 17:55:41.041233900 +0100 @@ -6,6 +6,7 @@ chdir 't'; +use Config; use ExtUtils::MM; use MakeMaker::Test::Utils; @@ -15,7 +16,7 @@ use Test::More tests => 7; my $perl = which_perl; -my $mm = bless { NAME => "Foo" }, "MM"; +my $mm = bless { NAME => "Foo", MAKE => $Config{make} }, "MM"; # I don't expect anything to have a length shorter than 256 chars. cmp_ok( $mm->max_exec_len, '>=', 256, 'max_exec_len' );
On Mon Aug 01 04:17:37 2011, parmstro64 wrote: Show quoted text
> On Sun Jul 17 15:06:33 2011, MSCHWERN wrote:
> > Sorry, but the attached log file does not contain an error. It appears > > to be truncated. It stops at... > > > > cp lib/ExtUtils/MM_Any.pm blib\lib\ExtUtils\MM_Any.pm > > E:\usr\bin\perl.exe "-Iblib\arch" "-Iblib\lib" -MExtUtils::Command -e > > "cp" -- bin/instmodsh blib\script\instmodsh > > pl2bat.bat blib\script\instmodsh
> > > OK - I have attached a new test report.
These are the same failures also reported on #69528, which contains a patch to fix the problem.
(See also #69481, which reported the same problem.)
I've patched this in. I skimmed the subject... but I'm going to assume you know what you're doing. :)
Working on the assumption that this is now resolved.