Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the PPI CPAN distribution.

Report information
The Basics
Id: 68176
Status: resolved
Priority: 0/
Queue: PPI
People
Owner: Nobody in particular
Requestors: MOREGAN [...] cpan.org
Cc:
AdminCc:
Bug Information
Severity: Normal
Broken in: 1.215
Fixed in: (no value)

History Show all quoted text
  Fri May 13 12:15:39 2011 MOREGAN [...] cpan.org - Ticket created
Subject: Some augmented assignment operators parse as two operators
Some augmented assignment operators like .= parse as a single PPI::Token::Operator: perl -MPPI::Document -MPPI::Dumper -e 'my $d = PPI::Document->new(\"$x .= 1;"); PPI::Dumper->new($d)->print();' PPI::Document PPI::Token::Whitespace ' ' PPI::Statement PPI::Token::Operator '.=' PPI::Token::Whitespace ' ' PPI::Token::Number '1' PPI::Token::Structure ';' and some parse as two: perl -MPPI::Document -MPPI::Dumper -e 'my $d = PPI::Document->new(\"$x <<= 1;"); PPI::Dumper->new($d)->print();' PPI::Document PPI::Token::Whitespace ' ' PPI::Statement PPI::Token::Operator '<<' PPI::Token::Operator '=' PPI::Token::Whitespace ' ' PPI::Token::Number '1' PPI::Token::Structure ';' I assume that they all should parse as a single operator. For all 16 augmented operators in Perl 5.10: perl -MPPI::Document -M5.10.0 -e 'print " += ops: "; my $d = PPI::Document->new(\"my $x; $x += 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " -= ops: "; my $d = PPI::Document->new(\"my $x; $x -= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " .= ops: "; my $d = PPI::Document->new(\"my $x; $x .= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " *= ops: "; my $d = PPI::Document->new(\"my $x; $x *= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " /= ops: "; my $d = PPI::Document->new(\"my $x; $x /= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " %= ops: "; my $d = PPI::Document->new(\"my $x; $x %= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " x= ops: "; my $d = PPI::Document->new(\"my $x; $x x= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " &= ops: "; my $d = PPI::Document->new(\"my $x; $x &= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " |= ops: "; my $d = PPI::Document->new(\"my $x; $x |= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " ^= ops: "; my $d = PPI::Document->new(\"my $x; $x ^= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print "**= ops: "; my $d = PPI::Document->new(\"my $x; $x **= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print "<<= ops: "; my $d = PPI::Document->new(\"my $x; $x <<= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print ">>= ops: "; my $d = PPI::Document->new(\"my $x; $x >>= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print "&&= ops: "; my $d = PPI::Document->new(\"my $x; $x &&= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print "||= ops: "; my $d = PPI::Document->new(\"my $x; $x ||= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print "//= ops: "; my $d = PPI::Document->new(\"my $x; $x //= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' += ops: 1 -= ops: 1 .= ops: 1 *= ops: 1 /= ops: 0 %= ops: 2 x= ops: 2 &= ops: 2 |= ops: 2 ^= ops: 2 **= ops: 2 <<= ops: 2 Show quoted text
>>= ops: 2
&&= ops: 2 ||= ops: 2 //= ops: 1
  Sun Jun 05 17:37:45 2011 MOREGAN [...] cpan.org - Correspondence added
Attached patch adds basic testing of PPI::Token::Operator, looping through all the operators in %PPI::Token::Operator::OPERATOR. It also updates the list of operators in the documentation to match the list of operators actually supported. With the new tests in place, adding the augmented assignment operators not already in OPERATOR passes all tests except for 'x='
Subject: operator-tests.patch
Index: t/ppi_token_operator.t =================================================================== --- t/ppi_token_operator.t (revision 0) +++ t/ppi_token_operator.t (revision 0) @@ -0,0 +1,60 @@ +#!/usr/bin/perl + +# Unit testing for PPI, generated by Test::Inline + +use strict; +use File::Spec::Functions ':ALL'; +BEGIN { + $| = 1; + $^W = 1; + no warnings 'once'; + $PPI::XS_DISABLE = 1; + $PPI::Lexer::X_TOKENIZER ||= $ENV{X_TOKENIZER}; +} +use PPI; + +# Execute the tests +use Test::More 'no_plan'; + +# =begin testing +{ +foreach my $op ( sort keys %PPI::Token::Operator::OPERATOR ) { + my $source = $op eq '<>' ? '<>;' : "1 $op 2;"; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "operator $op parsed '$source'" ); + my $ops = $doc->find( $op eq '<>' ? 'Token::QuoteLike::Readline' : 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator $op found operators" ); + is( @$ops, 1, "operator $op found exactly once" ); + is( $ops->[0]->content(), $op, "operator $op operator text matches" ); +} + +{ + my $source = '$a = .987;'; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "parsed '$source'" ); + my $ops = $doc->find( 'Token::Number::Float' ); + is( ref $ops, 'ARRAY', "found number" ); + is( @$ops, 1, "number found exactly once" ); + is( $ops->[0]->content(), '.987', "text matches" ); + + $ops = $doc->find( 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator = found operators in number test" ); + is( @$ops, 1, "operator = found exactly once in number test" ); +} + +{ + my $source = '$a = <<PERL_END;' . "\n" . 'PERL_END'; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "parsed '$source'" ); + my $ops = $doc->find( 'Token::HereDoc' ); + is( ref $ops, 'ARRAY', "found heredoc" ); + is( @$ops, 1, "heredoc found exactly once" ); + + $ops = $doc->find( 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator = found operators in heredoc test" ); + is( @$ops, 1, "operator = found exactly once in heredoc test" ); +} +} + + +1; Index: lib/PPI/Token/Operator.pm =================================================================== --- lib/PPI/Token/Operator.pm (revision 14482) +++ lib/PPI/Token/Operator.pm (working copy) @@ -21,8 +21,8 @@ == != <=> . .. ... , & | ^ && || // ? : = += -= *= .= //= - < > <= >= <> => -> - and or dor not eq ne + /= < > <= >= <> => -> + and or xor not eq ne =head1 DESCRIPTION @@ -72,6 +72,51 @@ ##################################################################### # Tokenizer Methods +=pod + +=begin testing + +foreach my $op ( sort keys %PPI::Token::Operator::OPERATOR ) { + my $source = $op eq '<>' ? '<>;' : "1 $op 2;"; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "operator $op parsed '$source'" ); + my $ops = $doc->find( $op eq '<>' ? 'Token::QuoteLike::Readline' : 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator $op found operators" ); + is( @$ops, 1, "operator $op found exactly once" ); + is( $ops->[0]->content(), $op, "operator $op operator text matches" ); +} + +{ + my $source = '$a = .987;'; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "parsed '$source'" ); + my $ops = $doc->find( 'Token::Number::Float' ); + is( ref $ops, 'ARRAY', "found number" ); + is( @$ops, 1, "number found exactly once" ); + is( $ops->[0]->content(), '.987', "text matches" ); + + $ops = $doc->find( 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator = found operators in number test" ); + is( @$ops, 1, "operator = found exactly once in number test" ); +} + +{ + my $source = '$a = <<PERL_END;' . "\n" . 'PERL_END'; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "parsed '$source'" ); + my $ops = $doc->find( 'Token::HereDoc' ); + is( ref $ops, 'ARRAY', "found heredoc" ); + is( @$ops, 1, "heredoc found exactly once" ); + + $ops = $doc->find( 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator = found operators in heredoc test" ); + is( @$ops, 1, "operator = found exactly once in heredoc test" ); +} + +=end testing + +=cut + sub __TOKENIZER__on_char { my $t = $_[1]; my $char = substr( $t->{line}, $t->{line_cursor}, 1 );
  Sun Jun 05 17:37:46 2011 MOREGAN [...] cpan.org - Status changed from 'new' to 'open'
  Sun Jun 05 17:42:24 2011 MOREGAN [...] cpan.org - Correspondence added
Forgot to say that adding '||=' to OPERATOR causes a failure in the larger test suite: [16:40:51] t/ppi_lexer.t .................... 1/33 # Failed test 'The curly in $foo ||= { One => 1 }; isa PPI::Structure::Constructor' # at t/ppi_lexer.t line 92. # The curly in $foo ||= { One => 1 }; isn't a 'PPI::Structure::Constructor' it's a 'PPI::Token::Structure' # Looks like you failed 1 test of 33.
  Wed Nov 12 07:46:04 2014 MITHALDU [...] cpan.org - Correspondence added
The full version of the fix for this is now released to CPAN as PPI 1.218.
  Wed Nov 12 07:46:06 2014 MITHALDU [...] cpan.org - Status changed from 'open' to 'resolved'