Bug #111455 for PAR-Packer: unable to use -x and -u with pp

Wed Jan 27 01:39:00 2016 SLAFFAN [...] cpan.org - Ticket created

Subject:

unable to use -x and -u with pp

Running pp with both the -u and -x options fails. It appears that some of the text files under the unicore directory are being executed by Module::ScanDeps. See code and results below. The "used once" warnings appear to be from Module::ScanDeps when it processes utf8_heavy.pl, and are harmless, but I have not looked into where the text files are executed. Tested on Strawberry perl 5.16.3 and 5.22.1. Regards, Shawn. C:\some_folder\x>pp -u -e "print 'blort'" C:\some_folder\x>a.exe blort C:\some_folder\x>pp -x -u -e "print 'blort'" Name "utf8::loose_property_to_file_of" used only once: possible typo at C:/strawberry_51613_x64/perl/lib/utf8_heavy.pl line 438. Name "utf8::file_to_swash_name" used only once: possible typo at C:/strawberry_51613_x64/perl/lib/utf8_heavy.pl line 440. Name "utf8::loose_property_name_of" used only once: possible typo at C:/strawberry_51613_x64/perl/lib/utf8_heavy.pl line 210. Name "utf8::stricter_to_file_of" used only once: possible typo at C:/strawberry_51613_x64/perl/lib/utf8_heavy.pl line 393. Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 30, near "007F" (Missing operator before F?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 31, near "00FF" (Missing operator before FF?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 31, near "1 Supplement" (Missing operator before Supplement?) Number found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 32, near "Supplement 0100" (Missing semicolon on previous line?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 32, near "017F" (Missing operator before F?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 33, near "024F" (Missing operator before F?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 34, near "02AF" (Missing operator before AF?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 35, near "02B0" (Missing operator before B0?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 35, near "02FF" (Missing operator before FF?) Number found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 36, near "Letters 0300" (Missing semicolon on previous line?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 36, near "036F" (Missing operator before F?) Number found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 37, near "Marks 0370" (Missing semicolon on previous line?) syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 30, near "007F" Illegal octal digit '8' at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 31, at end of line syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 31, near "00FF" syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 32, near "1 Supplement " Illegal octal digit '8' at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 33, at end of line syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 33, near "024F" syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 34, near "02AF" syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 35, near "02B0" syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 36, near "Letters 0300" syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 37, near "Marks 0370" C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt has too many errors. SYSTEM ERROR in executing C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt : 65280 at C:/strawberry_51613_x64/perl/site/lib/Module/ScanDeps.pm line 1422.

Wed Jan 27 01:50:05 2016 SLAFFAN [...] cpan.org - Ticket #111456: Ticket created

Subject:

unable to use -x and -u with pp

Running pp with both the -u and -x options fails. It appears that some of the text files under the unicore directory are being executed by Module::ScanDeps. See code and results below. The "used once" warnings appear to be from Module::ScanDeps when it processes utf8_heavy.pl, and are harmless, but I have not looked into where the text files are executed. Tested on Strawberry perl 5.16.3 and 5.22.1. Regards, Shawn. C:\some_folder\x>pp -u -e "print 'blort'" C:\some_folder\x>a.exe blort C:\some_folder\x>pp -x -u -e "print 'blort'" Name "utf8::loose_property_to_file_of" used only once: possible typo at C:/strawberry_51613_x64/perl/lib/utf8_heavy.pl line 438. Name "utf8::file_to_swash_name" used only once: possible typo at C:/strawberry_51613_x64/perl/lib/utf8_heavy.pl line 440. Name "utf8::loose_property_name_of" used only once: possible typo at C:/strawberry_51613_x64/perl/lib/utf8_heavy.pl line 210. Name "utf8::stricter_to_file_of" used only once: possible typo at C:/strawberry_51613_x64/perl/lib/utf8_heavy.pl line 393. Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 30, near "007F" (Missing operator before F?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 31, near "00FF" (Missing operator before FF?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 31, near "1 Supplement" (Missing operator before Supplement?) Number found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 32, near "Supplement 0100" (Missing semicolon on previous line?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 32, near "017F" (Missing operator before F?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 33, near "024F" (Missing operator before F?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 34, near "02AF" (Missing operator before AF?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 35, near "02B0" (Missing operator before B0?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 35, near "02FF" (Missing operator before FF?) Number found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 36, near "Letters 0300" (Missing semicolon on previous line?) Bareword found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 36, near "036F" (Missing operator before F?) Number found where operator expected at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 37, near "Marks 0370" (Missing semicolon on previous line?) syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 30, near "007F" Illegal octal digit '8' at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 31, at end of line syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 31, near "00FF" syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 32, near "1 Supplement " Illegal octal digit '8' at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 33, at end of line syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 33, near "024F" syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 34, near "02AF" syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 35, near "02B0" syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 36, near "Letters 0300" syntax error at C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt line 37, near "Marks 0370" C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt has too many errors. SYSTEM ERROR in executing C:/strawberry_51613_x64/perl/lib/unicore/Blocks.txt : 65280 at C:/strawberry_51613_x64/perl/site/lib/Module/ScanDeps.pm line 1422.

Wed Jan 27 02:42:13 2016 SLAFFAN [...] cpan.org - Ticket #111456: Correspondence added

RT seems to have filed the ticket twice on submission, so closing this one. The first ticket is https://rt.cpan.org/Ticket/Display.html?id=111455

Wed Jan 27 02:42:21 2016 SLAFFAN [...] cpan.org - Ticket #111456: Status changed from 'new' to 'resolved'

Wed Jan 27 08:38:40 2016 RSCHUPP [...] cpan.org - Correspondence added

On 2016-01-27 01:39:00, SLAFFAN wrote: Show quoted text

> Running pp with both the -u and -x options fails. It appears that some > of the text files under the unicore directory are being executed by > Module::ScanDeps. See code and results below.

Yeah, "pp -u ..." is just short for "pp -M utf8_heavy.pl ..." so combining this with "-x" gets the same result :( In fact, "pp -x -M Foo::Bar ..." with a Foo::Bar that has a %Preload rule in Module/ScanDeps.pm that adds anything NOT perlish will provoke the same problem. Other examples are charnames or LWP::MediaTypes. Turns out that "pp -x" - counter to what I would expect - does not only execute the actual input file(s), but also every module added with -M and everything it "depends" on according to the %Preload rules. The problem has been lurking for ages, but - since running "perl /path/to/module.pm" usually doesn't generate an error - nobody noticed. BTW, adding dependencies from "-M" arguments to the list of input files affects not only "-x", but the default use case as well. This behaviour also explains what's going on in #110480: try "pp -M utf8_heavy.pl ..." (no "-x") and look at the contents of the generated exe (as a zip file). It contains every non-pm file, e.g. 1000.pl, below the "unicore" directory TWICE, once as "lib/unicore/lib/Nv/1000.pl" (correct) and also as "lib/1000.pl". Can you try the patch below? Cheers, Roderich Index: lib/PAR/Packer.pm =================================================================== --- lib/PAR/Packer.pm (revision 1642) +++ lib/PAR/Packer.pm (working copy) @@ -737,7 +737,6 @@ skip => \%skip, # warn_missing => $args->{warn_missing}, ); - push @files, map {&$inc_find($_)} @preload; } push @files, @$input;

Wed Jan 27 08:38:41 2016 The RT System itself - Status changed from 'new' to 'open'

Wed Jan 27 12:59:30 2016 ether [...] cpan.org - Ticket #111456: Merged into ticket #111455

Wed Jan 27 12:59:30 2016 ether [...] cpan.org - Merged into ticket #111455

Wed Jan 27 14:59:36 2016 SLAFFAN [...] cpan.org - Correspondence added

Thanks Roderich, The patch makes the failure go away. The used once warnings still remain, but they appear harmless. Someone at a later date might use -M and want all the associated dependencies, but that could lead to all sorts of issues in itself. Maybe something can be added to the documentation for -M to clarify its interaction with -x? Regards, Shawn. On Wed Jan 27 08:38:40 2016, RSCHUPP wrote: Show quoted text

> On 2016-01-27 01:39:00, SLAFFAN wrote:

> > Running pp with both the -u and -x options fails. It appears that some > > of the text files under the unicore directory are being executed by > > Module::ScanDeps. See code and results below.

> > Yeah, "pp -u ..." is just short for "pp -M utf8_heavy.pl ..." so combining > this with "-x" gets the same result :( In fact, "pp -x -M Foo::Bar ..." with > a Foo::Bar that has a %Preload rule in Module/ScanDeps.pm that adds anything > NOT perlish will provoke the same problem. Other examples are charnames or > LWP::MediaTypes. > > Turns out that "pp -x" - counter to what I would expect - does not only > execute the actual input file(s), but also every module added with -M and > everything it "depends" on according to the %Preload rules. The problem > has been lurking for ages, but - since running "perl /path/to/module.pm" > usually doesn't generate an error - nobody noticed. > > BTW, adding dependencies from "-M" arguments to the list of input files > affects not only "-x", but the default use case as well. > This behaviour also explains what's going on in #110480: > try "pp -M utf8_heavy.pl ..." (no "-x") and look at the contents of the > generated exe (as a zip file). It contains every non-pm file, e.g. 1000.pl, > below the "unicore" directory TWICE, once as "lib/unicore/lib/Nv/1000.pl" > (correct) and also as "lib/1000.pl". > > Can you try the patch below? > > Cheers, Roderich > > > Index: lib/PAR/Packer.pm > =================================================================== > --- lib/PAR/Packer.pm (revision 1642) > +++ lib/PAR/Packer.pm (working copy) > @@ -737,7 +737,6 @@ > skip => \%skip, > # warn_missing => $args->{warn_missing}, > ); > - push @files, map {&$inc_find($_)} @preload; > } > push @files, @$input; > > > > > > > > > >

Thu Jan 28 04:48:39 2016 RSCHUPP [...] cpan.org - Correspondence added

On 2016-01-27 14:59:36, SLAFFAN wrote: Show quoted text

> The patch makes the failure go away.

Thanks for testing, will be in the next release. Show quoted text

> The used once warnings still remain, but they appear harmless.

Yes, essentially we're doing $ perl .../utf8_heavy.pl which produces the same warnings. Show quoted text

> Someone at a later date might use -M and want all the associated > dependencies, but that could lead to all sorts of issues in itself.

I don't think we're loosing anything here: for "pp -M Foo::Bar ..." we still do - scan Foo/Bar.pm - add any %Preload dependencies of Foo/Bar.pm to the list of dependencies If "-x" was also specified, we still run Foo/Bar.pm (pointless, but harmless except for possible warnings). But we don't: - scan all the %Preload dependencies of Foo/Bar.pm explicitly, but AFAICT Module::ScanDeps will scan them anyway - with "-x" run all the %Preload dependencies of Foo/Bar.pm (even more pointless, and harmful as they might include files that aren't Perl source) Cheers, Roderich

Thu Jan 28 04:48:46 2016 RSCHUPP [...] cpan.org - Status changed from 'open' to 'resolved'