Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the ExtUtils-Manifest CPAN distribution.

Report information
The Basics
Id: 45016
Status: resolved
Priority: 0/
Queue: ExtUtils-Manifest

People
Owner: Nobody in particular
Requestors: RIVY [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 1.54
Fixed in: (no value)



Subject: [PATCH] ExtUtils::Manifest::maniskip() incorrectly parses some lines with comments and internal whitespace filenames
When parsing MANIFEST files with lines containing filenames with whitespace and comments, maniskip() fails to correctly parse the line. This was made obvious on my Win32 machine when distributing/testing a module with filenames containing spaces. Here is a patch with what I believe to be the correct regular expression and code for correct filename parsing/interpretation. Sorry the regexp is complicated, but I believe it correctly parses all combinations of quoted/non-quoted filenames and comments. The portion of the regexp specifically parsing the quoted filename is based on regexps from "Mastering Regular Expressions, 2e; p. 281" and from Text::Balanced::gen_delimited_pat($;$) [v1.95]. - Roy Ivy
Subject: Manifest.pm-PATCHINFO
Download Manifest.pm-PATCHINFO
application/octet-stream 935b

Message body not shown because it is not plain text.

Subject: Manifest.pm-1.54.PATCH
--- c:\Perl\site\lib\ExtUtils\Manifest.pm Mon Nov 17 20:56:06 2008 +++ Manifest.pm.NEW Mon Apr 13 18:11:05 2009 @@ -378,11 +378,12 @@ while (<M>){ chomp; s/\r//; - next if /^#/; - next if /^\s*$/; - s/^'//; - s/'$//; - push @skip, _macify($_); + $_ =~ qr{^\s*(?:(?:'([^\\']*(?:\\.[^\\']*)*)')|([^#\s]\S*))?(?:(?:\s*)|(?:\s+(.*?)\s*))$}; + #my $comment = $3; + my $filename = $2; + if ( defined($1) ) { $filename = $1; $filename =~ s/\\(['\\])/$1/g; } + next if (not defined($filename) or not $filename); + push @skip, _macify($filename); } close M; return sub {0} unless (scalar @skip > 0);
On Mon Apr 13 19:36:00 2009, RIVY wrote: Attempting to RE-attached attachments as easily visible, plain TXT files.
--- c:\Perl\site\lib\ExtUtils\Manifest.pm Mon Nov 17 20:56:06 2008 +++ Manifest.pm.NEW Mon Apr 13 18:11:05 2009 @@ -378,11 +378,12 @@ while (<M>){ chomp; s/\r//; - next if /^#/; - next if /^\s*$/; - s/^'//; - s/'$//; - push @skip, _macify($_); + $_ =~ qr{^\s*(?:(?:'([^\\']*(?:\\.[^\\']*)*)')|([^#\s]\S*))?(?:(?:\s*)|(?:\s+(.*?)\s*))$}; + #my $comment = $3; + my $filename = $2; + if ( defined($1) ) { $filename = $1; $filename =~ s/\\(['\\])/$1/g; } + next if (not defined($filename) or not $filename); + push @skip, _macify($filename); } close M; return sub {0} unless (scalar @skip > 0);
C:\...\perl\Win32-CommandLine >perl -MExtUtils::Manifest -e "print ExtUtils::Manifest->VERSION()" 1.54 C:\...\perl\Win32-CommandLine >diff -Naur c:\Perl\site\lib\ExtUtils\Manifest.pm Manifest.pm.NEW --- c:\Perl\site\lib\ExtUtils\Manifest.pm Mon Nov 17 20:56:06 2008 +++ Manifest.pm.NEW Mon Apr 13 18:11:05 2009 @@ -378,11 +378,12 @@ while (<M>){ chomp; s/\r//; - next if /^#/; - next if /^\s*$/; - s/^'//; - s/'$//; - push @skip, _macify($_); + $_ =~ qr{^\s*(?:(?:'([^\\']*(?:\\.[^\\']*)*)')|([^#\s]\S*))?(?:(?:\s*)|(?:\s+(.*?)\s*))$}; + #my $comment = $3; + my $filename = $2; + if ( defined($1) ) { $filename = $1; $filename =~ s/\\(['\\])/$1/g; } + next if (not defined($filename) or not $filename); + push @skip, _macify($filename); } close M; return sub {0} unless (scalar @skip > 0);
On Mon Apr 13 19:38:09 2009, RIVY wrote: Show quoted text
> On Mon Apr 13 19:36:00 2009, RIVY wrote: > > Attempting to RE-attached attachments as easily visible, plain TXT files.
Thanks for the report. I'll look into this - could you supply an example or two of what failed for you, so that I can incorporate some tests for this? Thanks again.
On Sun Apr 26 15:19:58 2009, RKOBES wrote: Show quoted text
> Thanks for the report. I'll look into this - could you supply an example > or two of what failed for you, so that I can incorporate some tests for > this? Thanks again.
Sure. I hooked the maniskip() routine to show @skip after the MANIFEST.SKIP file is parsed. Here are some outputs: 1) 'MANIFEST.SKIP.default.txt' and 'MANIFEST.SKIP.default.@skip.txt' This uses the default MANIFEST.SKIP. @skip[13] is incorrect. 2) 'MANIFEST.SKIP.whitespace.txt' and 'MANIFEST.SKIP.whitespace.@skip.txt' This uses a manufactured MANIFEST.SKIP with files with internal whitespace. @skip[1], @skip[3], and @skip[4] are incorrect. Hope that helps. - Roy
test1 test2 #comment 'A note' 'B note' #comment for 'B note' 'C note' with more text # another comment 'a note with lots of whitespace'
@skip{0}=`test1` @skip{1}=`test2 #comment` @skip{2}=`A note` @skip{3}=`B note' #comment for 'B note` @skip{4}=`C note' with more text` @skip{5}=`a note with lots of whitespace`
@skip{0}=`\bRCS\b` @skip{1}=`\bCVS\b` @skip{2}=`\bSCCS\b` @skip{3}=`,v$` @skip{4}=`\B\.svn\b` @skip{5}=`\B\.git\b` @skip{6}=`\b_darcs\b` @skip{7}=`\bMANIFEST\.bak` @skip{8}=`\bMakefile$` @skip{9}=`\bblib/` @skip{10}=`\bMakeMaker-\d` @skip{11}=`\bpm_to_blib\.ts$` @skip{12}=`\bpm_to_blib$` @skip{13}=`\bblibdirs\.ts$ # 6.18 through 6.25 generated this` @skip{14}=`\bBuild$` @skip{15}=`\b_build/` @skip{16}=`~$` @skip{17}=`\.old$` @skip{18}=`\#$` @skip{19}=`\b\.#` @skip{20}=`\.bak$` @skip{21}=`\bcover_db\b`
# Avoid version control files. \bRCS\b \bCVS\b \bSCCS\b ,v$ \B\.svn\b \B\.git\b \B\.gitignore\b \b_darcs\b # Avoid Makemaker generated and utility files. \bMANIFEST\.bak \bMakefile$ \bblib/ \bMakeMaker-\d \bpm_to_blib\.ts$ \bpm_to_blib$ \bblibdirs\.ts$ # 6.18 through 6.25 generated this # Avoid Module::Build generated and utility files. \bBuild$ \b_build/ # Avoid temp and backup files. ~$ \.old$ \#$ \b\.# \.bak$ # Avoid Devel::Cover files. \bcover_db\b
On Sun Apr 26 15:19:58 2009, RKOBES wrote: Show quoted text
> Thanks for the report. I'll look into this - could you supply an example > or two of what failed for you, so that I can incorporate some tests for > this? Thanks again.
I've just posted a couple of examples with MANIFEST.SKIPs and outputs. Additionally, while hooking the code, I noticed one other possible issue. I haven't had any issues associated with this (and haven't tested it), but should the line <just below the patch supplied>: my $opts = $Is_VMS ? '(?i)' : ''; instead be: my $opts = File::Spec->case_tolerant() ? '(?i)' : ''; I don't see any other case issues after seeing this line, although I'm not exceedingly familiar with the code and only briefly perused it near $Is_VMS tokens. - Roy
Thanks very much for the test cases; this patch has been incorporated into http://github.com/rkobes/extutils-manifest. Regarding the comment about changing my $opts = $Is_VMS_mode ? '(?i)' : ''; I don't have access to a VMS machine, and I see the from the start of ExtUtils::Manifest that setting $Is_VMS_mode is different than just setting $Is_VMS. I'm worried that using the File::Spec case_tolerant function may behave differently than the current way of setting $opts, and also, it may not do the right thing for other operating systems where case tolerance is an issue. Thus, I'd like to leave the current behaviour until it's shown to be an issue.