Skip Menu |

This queue is for tickets about the Text-ParseWords CPAN distribution.

Report information
The Basics
Id: 50929
Status: rejected
Priority: 0/
Queue: Text-ParseWords

People
Owner: Nobody in particular
Requestors: shay [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 3.27
Fixed in: (no value)



Subject: Convert Text-ParseWords test file from DOS EOLs to UNIX EOLs
A recent discussion on p5p: http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2009-10/msg00967.html lead to a change being made to the release process so that certain files (mainly under win32/) that used to be converted from UNIX EOLs (in the repository) to DOS EOLs (in the release tarball) are no longer converted. The reason was simply that it is no longer necessary for these files to have DOS EOLs, since modern Windows programs cope perfectly well with UNIX EOLs, and it is simpler not to do the conversion. It was noticed at the same time that a small number of other files currently have DOS EOLs in the repository, and it would perhaps be simpler if all files in the repository had UNIX EOLs (unless any have to have DOS EOLs for some reason). The file t/ParseWords.t in Text-ParseWords has DOS EOLs. It would be helpful if a new release of Text-ParseWords could be made which converts this file to UNIX EOLs, and it would be best for people in the future looking back through the history of changes to files if nothing else was changed in this file at the same time (otherwise those other changes would get masked somewhat by the noise of the EOLs changes).
From: bitcard [...] chimpychompy.org
This is the same patch I recently submitted to p5p.
Subject: 0001-dos2unix-a-Text-ParseWords-test-file-with-DOS-EOLs.patch
From d11155eecaaa8b428e5eee000b39b9f64a8ffefd Mon Sep 17 00:00:00 2001 From: Gavin Shelley <columbusmonkey@me.com> Date: Mon, 11 Feb 2013 22:15:56 +0000 Subject: [PATCH] dos2unix a Text-ParseWords test file with DOS EOLs --- cpan/Text-ParseWords/t/ParseWords.t | 244 +++++++++++++++++------------------ 1 file changed, 122 insertions(+), 122 deletions(-) diff --git a/cpan/Text-ParseWords/t/ParseWords.t b/cpan/Text-ParseWords/t/ParseWords.t index 905ea00..3fb7ac6 100644 --- a/cpan/Text-ParseWords/t/ParseWords.t +++ b/cpan/Text-ParseWords/t/ParseWords.t @@ -1,122 +1,122 @@ -#!./perl - -use warnings; -use Text::ParseWords; -use Test::More tests => 27; - -@words = shellwords(qq(foo "bar quiz" zoo)); -is($words[0], 'foo'); -is($words[1], 'bar quiz'); -is($words[2], 'zoo'); - -{ - # Gonna get some undefined things back - no warnings 'uninitialized' ; - - # Test quotewords() with other parameters and null last field - @words = quotewords(':+', 1, 'foo:::"bar:foo":zoo zoo:'); - is(join(";", @words), qq(foo;"bar:foo";zoo zoo;)); -} - -# Test $keep eq 'delimiters' and last field zero -@words = quotewords('\s+', 'delimiters', '4 3 2 1 0'); -is(join(";", @words), qq(4; ;3; ;2; ;1; ;0)); - -# Big ol' nasty test (thanks, Joerk!) -$string = 'aaaa"bbbbb" cc\\ cc \\\\\\"dddd" eee\\\\\\"ffff" "gg"'; - -# First with $keep == 1 -$result = join('|', parse_line('\s+', 1, $string)); -is($result, 'aaaa"bbbbb"|cc\\ cc|\\\\\\"dddd" eee\\\\\\"ffff"|"gg"'); - -# Now, $keep == 0 -$result = join('|', parse_line('\s+', 0, $string)); -is($result, 'aaaabbbbb|cc cc|\\"dddd eee\\"ffff|gg'); - -# Now test single quote behavior -$string = 'aaaa"bbbbb" cc\\ cc \\\\\\"dddd\' eee\\\\\\"ffff\' gg'; -$result = join('|', parse_line('\s+', 0, $string)); -is($result, 'aaaabbbbb|cc cc|\\"dddd eee\\\\\\"ffff|gg'); - -# Make sure @nested_quotewords does the right thing -@lists = nested_quotewords('\s+', 0, 'a b c', '1 2 3', 'x y z'); -is (@lists, 3); -is (@{$lists[0]}, 3); -is (@{$lists[1]}, 3); -is (@{$lists[2]}, 3); - -# Now test error return -$string = 'foo bar baz"bach blech boop'; - -@words = shellwords($string); -is(@words, 0); - -@words = parse_line('s+', 0, $string); -is(@words, 0); - -@words = quotewords('s+', 0, $string); -is(@words, 0); - -{ - # Gonna get some more undefined things back - no warnings 'uninitialized' ; - - @words = nested_quotewords('s+', 0, $string); - is(@words, 0); - - # Now test empty fields - $result = join('|', parse_line(':', 0, 'foo::0:"":::')); - is($result, 'foo||0||||'); - - # Test for 0 in quotes without $keep - $result = join('|', parse_line(':', 0, ':"0":')); - is($result, '|0|'); - - # Test for \001 in quoted string - $result = join('|', parse_line(':', 0, ':"' . "\001" . '":')); - is($result, "|\1|"); - -} - -# Now test perlish single quote behavior -$Text::ParseWords::PERL_SINGLE_QUOTE = 1; -$string = 'aaaa"bbbbb" cc\ cc \\\\\"dddd\' eee\\\\\"\\\'ffff\' gg'; -$result = join('|', parse_line('\s+', 0, $string)); -is($result, 'aaaabbbbb|cc cc|\"dddd eee\\\\"\'ffff|gg'); - -# test whitespace in the delimiters -@words = quotewords(' ', 1, '4 3 2 1 0'); -is(join(";", @words), qq(4;3;2;1;0)); - -# [perl #30442] Text::ParseWords does not handle backslashed newline inside quoted text -$string = qq{"field1" "field2\\\nstill field2" "field3"}; - -$result = join('|', parse_line("\t", 1, $string)); -is($result, qq{"field1"|"field2\\\nstill field2"|"field3"}); - -$result = join('|', parse_line("\t", 0, $string)); -is($result, "field1|field2\nstill field2|field3"); - -SKIP: { # unicode - skip "No unicode",1 if $]<5.008; - $string = qq{"field1"\x{1234}"field2\\\x{1234}still field2"\x{1234}"field3"}; - $result = join('|', parse_line("\x{1234}", 0, $string)); - is($result, "field1|field2\x{1234}still field2|field3",'Unicode'); -} - -# missing quote after matching regex used to hang after change #22997 -"1234" =~ /(1)(2)(3)(4)/; -$string = qq{"missing quote}; -$result = join('|', shellwords($string)); -is($result, ""); - -# make sure shellwords strips out leading whitespace and trailng undefs -# from parse_line, so it's behavior is more like /bin/sh -$result = join('|', shellwords(" aa \\ \\ bb ", " \\ ", "cc dd ee\\ ")); -is($result, "aa| | bb| |cc|dd|ee "); - -$SIG{ALRM} = sub {die "Timeout!"}; -alarm(3); -@words = Text::ParseWords::old_shellwords("foo\\"); -is(@words, 1); -alarm(0); +#!./perl + +use warnings; +use Text::ParseWords; +use Test::More tests => 27; + +@words = shellwords(qq(foo "bar quiz" zoo)); +is($words[0], 'foo'); +is($words[1], 'bar quiz'); +is($words[2], 'zoo'); + +{ + # Gonna get some undefined things back + no warnings 'uninitialized' ; + + # Test quotewords() with other parameters and null last field + @words = quotewords(':+', 1, 'foo:::"bar:foo":zoo zoo:'); + is(join(";", @words), qq(foo;"bar:foo";zoo zoo;)); +} + +# Test $keep eq 'delimiters' and last field zero +@words = quotewords('\s+', 'delimiters', '4 3 2 1 0'); +is(join(";", @words), qq(4; ;3; ;2; ;1; ;0)); + +# Big ol' nasty test (thanks, Joerk!) +$string = 'aaaa"bbbbb" cc\\ cc \\\\\\"dddd" eee\\\\\\"ffff" "gg"'; + +# First with $keep == 1 +$result = join('|', parse_line('\s+', 1, $string)); +is($result, 'aaaa"bbbbb"|cc\\ cc|\\\\\\"dddd" eee\\\\\\"ffff"|"gg"'); + +# Now, $keep == 0 +$result = join('|', parse_line('\s+', 0, $string)); +is($result, 'aaaabbbbb|cc cc|\\"dddd eee\\"ffff|gg'); + +# Now test single quote behavior +$string = 'aaaa"bbbbb" cc\\ cc \\\\\\"dddd\' eee\\\\\\"ffff\' gg'; +$result = join('|', parse_line('\s+', 0, $string)); +is($result, 'aaaabbbbb|cc cc|\\"dddd eee\\\\\\"ffff|gg'); + +# Make sure @nested_quotewords does the right thing +@lists = nested_quotewords('\s+', 0, 'a b c', '1 2 3', 'x y z'); +is (@lists, 3); +is (@{$lists[0]}, 3); +is (@{$lists[1]}, 3); +is (@{$lists[2]}, 3); + +# Now test error return +$string = 'foo bar baz"bach blech boop'; + +@words = shellwords($string); +is(@words, 0); + +@words = parse_line('s+', 0, $string); +is(@words, 0); + +@words = quotewords('s+', 0, $string); +is(@words, 0); + +{ + # Gonna get some more undefined things back + no warnings 'uninitialized' ; + + @words = nested_quotewords('s+', 0, $string); + is(@words, 0); + + # Now test empty fields + $result = join('|', parse_line(':', 0, 'foo::0:"":::')); + is($result, 'foo||0||||'); + + # Test for 0 in quotes without $keep + $result = join('|', parse_line(':', 0, ':"0":')); + is($result, '|0|'); + + # Test for \001 in quoted string + $result = join('|', parse_line(':', 0, ':"' . "\001" . '":')); + is($result, "|\1|"); + +} + +# Now test perlish single quote behavior +$Text::ParseWords::PERL_SINGLE_QUOTE = 1; +$string = 'aaaa"bbbbb" cc\ cc \\\\\"dddd\' eee\\\\\"\\\'ffff\' gg'; +$result = join('|', parse_line('\s+', 0, $string)); +is($result, 'aaaabbbbb|cc cc|\"dddd eee\\\\"\'ffff|gg'); + +# test whitespace in the delimiters +@words = quotewords(' ', 1, '4 3 2 1 0'); +is(join(";", @words), qq(4;3;2;1;0)); + +# [perl #30442] Text::ParseWords does not handle backslashed newline inside quoted text +$string = qq{"field1" "field2\\\nstill field2" "field3"}; + +$result = join('|', parse_line("\t", 1, $string)); +is($result, qq{"field1"|"field2\\\nstill field2"|"field3"}); + +$result = join('|', parse_line("\t", 0, $string)); +is($result, "field1|field2\nstill field2|field3"); + +SKIP: { # unicode + skip "No unicode",1 if $]<5.008; + $string = qq{"field1"\x{1234}"field2\\\x{1234}still field2"\x{1234}"field3"}; + $result = join('|', parse_line("\x{1234}", 0, $string)); + is($result, "field1|field2\x{1234}still field2|field3",'Unicode'); +} + +# missing quote after matching regex used to hang after change #22997 +"1234" =~ /(1)(2)(3)(4)/; +$string = qq{"missing quote}; +$result = join('|', shellwords($string)); +is($result, ""); + +# make sure shellwords strips out leading whitespace and trailng undefs +# from parse_line, so it's behavior is more like /bin/sh +$result = join('|', shellwords(" aa \\ \\ bb ", " \\ ", "cc dd ee\\ ")); +is($result, "aa| | bb| |cc|dd|ee "); + +$SIG{ALRM} = sub {die "Timeout!"}; +alarm(3); +@words = Text::ParseWords::old_shellwords("foo\\"); +is(@words, 1); +alarm(0); -- 1.7.10.2 (Apple Git-33)
The attached patch against 3.29 syncs the two test files with what is currently in bleadperl, which I believe to be correct. The PERL_CORE boilerplate code has been removed from both test files, and both files have been converted from DOS EOLs to UNIX EOLs. It's not urgent, but I would be very grateful if you could apply this patch and roll a new CPAN release, ideally before 20th August when the next Perl release is due, if that's possible.
Subject: sync-with-blead.patch
diff -ruN Text-ParseWords-3.29.orig/t/ParseWords.t Text-ParseWords-3.29/t/ParseWords.t --- Text-ParseWords-3.29.orig/t/ParseWords.t 2009-09-16 22:09:47.000000000 +0100 +++ Text-ParseWords-3.29/t/ParseWords.t 2013-06-26 08:16:10.877189600 +0100 @@ -1,129 +1,122 @@ -#!./perl - -BEGIN { - if( $ENV{PERL_CORE} ) { - chdir 't' if -d 't'; - @INC = '../lib'; - } -} - -use warnings; -use Text::ParseWords; -use Test::More tests => 27; - -@words = shellwords(qq(foo "bar quiz" zoo)); -is($words[0], 'foo'); -is($words[1], 'bar quiz'); -is($words[2], 'zoo'); - -{ - # Gonna get some undefined things back - no warnings 'uninitialized' ; - - # Test quotewords() with other parameters and null last field - @words = quotewords(':+', 1, 'foo:::"bar:foo":zoo zoo:'); - is(join(";", @words), qq(foo;"bar:foo";zoo zoo;)); -} - -# Test $keep eq 'delimiters' and last field zero -@words = quotewords('\s+', 'delimiters', '4 3 2 1 0'); -is(join(";", @words), qq(4; ;3; ;2; ;1; ;0)); - -# Big ol' nasty test (thanks, Joerk!) -$string = 'aaaa"bbbbb" cc\\ cc \\\\\\"dddd" eee\\\\\\"ffff" "gg"'; - -# First with $keep == 1 -$result = join('|', parse_line('\s+', 1, $string)); -is($result, 'aaaa"bbbbb"|cc\\ cc|\\\\\\"dddd" eee\\\\\\"ffff"|"gg"'); - -# Now, $keep == 0 -$result = join('|', parse_line('\s+', 0, $string)); -is($result, 'aaaabbbbb|cc cc|\\"dddd eee\\"ffff|gg'); - -# Now test single quote behavior -$string = 'aaaa"bbbbb" cc\\ cc \\\\\\"dddd\' eee\\\\\\"ffff\' gg'; -$result = join('|', parse_line('\s+', 0, $string)); -is($result, 'aaaabbbbb|cc cc|\\"dddd eee\\\\\\"ffff|gg'); - -# Make sure @nested_quotewords does the right thing -@lists = nested_quotewords('\s+', 0, 'a b c', '1 2 3', 'x y z'); -is (@lists, 3); -is (@{$lists[0]}, 3); -is (@{$lists[1]}, 3); -is (@{$lists[2]}, 3); - -# Now test error return -$string = 'foo bar baz"bach blech boop'; - -@words = shellwords($string); -is(@words, 0); - -@words = parse_line('s+', 0, $string); -is(@words, 0); - -@words = quotewords('s+', 0, $string); -is(@words, 0); - -{ - # Gonna get some more undefined things back - no warnings 'uninitialized' ; - - @words = nested_quotewords('s+', 0, $string); - is(@words, 0); - - # Now test empty fields - $result = join('|', parse_line(':', 0, 'foo::0:"":::')); - is($result, 'foo||0||||'); - - # Test for 0 in quotes without $keep - $result = join('|', parse_line(':', 0, ':"0":')); - is($result, '|0|'); - - # Test for \001 in quoted string - $result = join('|', parse_line(':', 0, ':"' . "\001" . '":')); - is($result, "|\1|"); - -} - -# Now test perlish single quote behavior -$Text::ParseWords::PERL_SINGLE_QUOTE = 1; -$string = 'aaaa"bbbbb" cc\ cc \\\\\"dddd\' eee\\\\\"\\\'ffff\' gg'; -$result = join('|', parse_line('\s+', 0, $string)); -is($result, 'aaaabbbbb|cc cc|\"dddd eee\\\\"\'ffff|gg'); - -# test whitespace in the delimiters -@words = quotewords(' ', 1, '4 3 2 1 0'); -is(join(";", @words), qq(4;3;2;1;0)); - -# [perl #30442] Text::ParseWords does not handle backslashed newline inside quoted text -$string = qq{"field1" "field2\\\nstill field2" "field3"}; - -$result = join('|', parse_line("\t", 1, $string)); -is($result, qq{"field1"|"field2\\\nstill field2"|"field3"}); - -$result = join('|', parse_line("\t", 0, $string)); -is($result, "field1|field2\nstill field2|field3"); - -SKIP: { # unicode - skip "No unicode",1 if $]<5.008; - $string = qq{"field1"\x{1234}"field2\\\x{1234}still field2"\x{1234}"field3"}; - $result = join('|', parse_line("\x{1234}", 0, $string)); - is($result, "field1|field2\x{1234}still field2|field3",'Unicode'); -} - -# missing quote after matching regex used to hang after change #22997 -"1234" =~ /(1)(2)(3)(4)/; -$string = qq{"missing quote}; -$result = join('|', shellwords($string)); -is($result, ""); - -# make sure shellwords strips out leading whitespace and trailng undefs -# from parse_line, so it's behavior is more like /bin/sh -$result = join('|', shellwords(" aa \\ \\ bb ", " \\ ", "cc dd ee\\ ")); -is($result, "aa| | bb| |cc|dd|ee "); - -$SIG{ALRM} = sub {die "Timeout!"}; -alarm(3); -@words = Text::ParseWords::old_shellwords("foo\\"); -is(@words, 1); -alarm(0); +#!./perl + +use warnings; +use Text::ParseWords; +use Test::More tests => 27; + +@words = shellwords(qq(foo "bar quiz" zoo)); +is($words[0], 'foo'); +is($words[1], 'bar quiz'); +is($words[2], 'zoo'); + +{ + # Gonna get some undefined things back + no warnings 'uninitialized' ; + + # Test quotewords() with other parameters and null last field + @words = quotewords(':+', 1, 'foo:::"bar:foo":zoo zoo:'); + is(join(";", @words), qq(foo;"bar:foo";zoo zoo;)); +} + +# Test $keep eq 'delimiters' and last field zero +@words = quotewords('\s+', 'delimiters', '4 3 2 1 0'); +is(join(";", @words), qq(4; ;3; ;2; ;1; ;0)); + +# Big ol' nasty test (thanks, Joerk!) +$string = 'aaaa"bbbbb" cc\\ cc \\\\\\"dddd" eee\\\\\\"ffff" "gg"'; + +# First with $keep == 1 +$result = join('|', parse_line('\s+', 1, $string)); +is($result, 'aaaa"bbbbb"|cc\\ cc|\\\\\\"dddd" eee\\\\\\"ffff"|"gg"'); + +# Now, $keep == 0 +$result = join('|', parse_line('\s+', 0, $string)); +is($result, 'aaaabbbbb|cc cc|\\"dddd eee\\"ffff|gg'); + +# Now test single quote behavior +$string = 'aaaa"bbbbb" cc\\ cc \\\\\\"dddd\' eee\\\\\\"ffff\' gg'; +$result = join('|', parse_line('\s+', 0, $string)); +is($result, 'aaaabbbbb|cc cc|\\"dddd eee\\\\\\"ffff|gg'); + +# Make sure @nested_quotewords does the right thing +@lists = nested_quotewords('\s+', 0, 'a b c', '1 2 3', 'x y z'); +is (@lists, 3); +is (@{$lists[0]}, 3); +is (@{$lists[1]}, 3); +is (@{$lists[2]}, 3); + +# Now test error return +$string = 'foo bar baz"bach blech boop'; + +@words = shellwords($string); +is(@words, 0); + +@words = parse_line('s+', 0, $string); +is(@words, 0); + +@words = quotewords('s+', 0, $string); +is(@words, 0); + +{ + # Gonna get some more undefined things back + no warnings 'uninitialized' ; + + @words = nested_quotewords('s+', 0, $string); + is(@words, 0); + + # Now test empty fields + $result = join('|', parse_line(':', 0, 'foo::0:"":::')); + is($result, 'foo||0||||'); + + # Test for 0 in quotes without $keep + $result = join('|', parse_line(':', 0, ':"0":')); + is($result, '|0|'); + + # Test for \001 in quoted string + $result = join('|', parse_line(':', 0, ':"' . "\001" . '":')); + is($result, "|\1|"); + +} + +# Now test perlish single quote behavior +$Text::ParseWords::PERL_SINGLE_QUOTE = 1; +$string = 'aaaa"bbbbb" cc\ cc \\\\\"dddd\' eee\\\\\"\\\'ffff\' gg'; +$result = join('|', parse_line('\s+', 0, $string)); +is($result, 'aaaabbbbb|cc cc|\"dddd eee\\\\"\'ffff|gg'); + +# test whitespace in the delimiters +@words = quotewords(' ', 1, '4 3 2 1 0'); +is(join(";", @words), qq(4;3;2;1;0)); + +# [perl #30442] Text::ParseWords does not handle backslashed newline inside quoted text +$string = qq{"field1" "field2\\\nstill field2" "field3"}; + +$result = join('|', parse_line("\t", 1, $string)); +is($result, qq{"field1"|"field2\\\nstill field2"|"field3"}); + +$result = join('|', parse_line("\t", 0, $string)); +is($result, "field1|field2\nstill field2|field3"); + +SKIP: { # unicode + skip "No unicode",1 if $]<5.008; + $string = qq{"field1"\x{1234}"field2\\\x{1234}still field2"\x{1234}"field3"}; + $result = join('|', parse_line("\x{1234}", 0, $string)); + is($result, "field1|field2\x{1234}still field2|field3",'Unicode'); +} + +# missing quote after matching regex used to hang after change #22997 +"1234" =~ /(1)(2)(3)(4)/; +$string = qq{"missing quote}; +$result = join('|', shellwords($string)); +is($result, ""); + +# make sure shellwords strips out leading whitespace and trailng undefs +# from parse_line, so it's behavior is more like /bin/sh +$result = join('|', shellwords(" aa \\ \\ bb ", " \\ ", "cc dd ee\\ ")); +is($result, "aa| | bb| |cc|dd|ee "); + +$SIG{ALRM} = sub {die "Timeout!"}; +alarm(3); +@words = Text::ParseWords::old_shellwords("foo\\"); +is(@words, 1); +alarm(0); diff -ruN Text-ParseWords-3.29.orig/t/taint.t Text-ParseWords-3.29/t/taint.t --- Text-ParseWords-3.29.orig/t/taint.t 2009-09-16 22:09:47.000000000 +0100 +++ Text-ParseWords-3.29/t/taint.t 2013-06-26 08:16:10.877189600 +0100 @@ -1,26 +1,24 @@ -#!./perl -Tw -# [perl #33173] shellwords.pl and tainting - -BEGIN { - if ( $ENV{PERL_CORE} ) { - chdir 't' if -d 't'; - @INC = '../lib'; - require Config; - no warnings 'once'; - if ($Config::Config{extensions} !~ /\bList\/Util\b/) { - print "1..0 # Skip: Scalar::Util was not built\n"; - exit 0; - } - } -} - -use Text::ParseWords qw(shellwords old_shellwords); -use Scalar::Util qw(tainted); - -print "1..2\n"; - -print "not " if grep { not tainted($_) } shellwords("$0$^X"); -print "ok 1\n"; - -print "not " if grep { not tainted($_) } old_shellwords("$0$^X"); -print "ok 2\n"; +#!./perl -Tw +# [perl #33173] shellwords.pl and tainting + +BEGIN { + if ( $ENV{PERL_CORE} ) { + require Config; + no warnings 'once'; + if ($Config::Config{extensions} !~ /\bList\/Util\b/) { + print "1..0 # Skip: Scalar::Util was not built\n"; + exit 0; + } + } +} + +use Text::ParseWords qw(shellwords old_shellwords); +use Scalar::Util qw(tainted); + +print "1..2\n"; + +print "not " if grep { not tainted($_) } shellwords("$0$^X"); +print "ok 1\n"; + +print "not " if grep { not tainted($_) } old_shellwords("$0$^X"); +print "ok 2\n";
PR was closed, so closing this ticket.