Skip Menu |

This queue is for tickets about the Parse-RecDescent CPAN distribution.

Report information
The Basics
Id: 42348
Status: open
Priority: 0/
Queue: Parse-RecDescent

People
Owner: Nobody in particular
Requestors: polettix [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 1.96.0
Fixed in: (no value)



Subject: Negative lookahead not working
Hello, unless I completely missed the point about negative look-aheads (and this may well be!), I think there might be a bug in its handling inside Parse::RecDescent. Please find attached a minimal test file, complete with its output with "prove -v" (perl 5.8.8 and P::RD 1.96.0). Cheers, Flavio.
Subject: prd.t
# vim: filetype=perl : use strict; use warnings; use Test::More tests => 1; use Parse::RecDescent; $::RD_TRACE = 1; my $grammar = <<'END_OF_GRAMMAR'; foo : 'a' bar : 'z' not_before_bar : foo ...!bar whatever : not_before_bar foo END_OF_GRAMMAR my $parser = Parse::RecDescent->new($grammar); ok($parser->whatever($_), $_) for qw( aa ); __END__ $ prove prd.t prd...... Parse::RecDescent: Treating "foo :" as a rule declaration Parse::RecDescent: Treating "a" as a literal terminal Parse::RecDescent: Treating "bar :" as a rule declaration Parse::RecDescent: Treating "z" as a literal terminal Parse::RecDescent: Treating "not_before_bar :" as a rule declaration Parse::RecDescent: Treating "foo" as a subrule match Parse::RecDescent: Treating "...!" as a negative lookahead Parse::RecDescent: Treating "bar" as a subrule match Parse::RecDescent: Treating "whatever :" as a rule declaration Parse::RecDescent: Treating "not_before_bar" as a subrule match Parse::RecDescent: Treating "foo" as a subrule match printing code (20115) to RD_TRACE 1| whatever |Trying rule: [whatever] | 1| whatever | |"aa" 1| whatever |Trying production: [not_before_bar | | |foo] | 1| whatever |Trying subrule: [not_before_bar] | 2|not_before|Trying rule: [not_before_bar] | 2|not_before|Trying production: [foo bar] | 2|not_before|Trying subrule: [foo] | 3| foo |Trying rule: [foo] | 3| foo |Trying production: ['a'] | 3| foo |Trying terminal: ['a'] | 3| foo |>>Matched terminal<< (return value: | | |[a]) | 3| foo | |"a" 3| foo |>>Matched production: ['a']<< | 3| foo |>>Matched rule<< (return value: [a]) | 3| foo |(consumed: [a]) | 2|not_before|>>Matched subrule: [foo]<< (return | | |value: [a] | 2|not_before|Trying subrule: [bar] | 3| bar |Trying rule: [bar] | 3| bar |Trying production: ['z'] | 3| bar |Trying terminal: ['z'] | 3| bar |<<Didn't match terminal>> | 3| bar |<<Didn't match rule>> | 2|not_before|>>Matched subrule: [bar]<< (return | | |value: [] | 2|not_before|>>Matched production: [foo bar]<< | 2|not_before|>>Matched rule<< (return value: []) | 2|not_before|(consumed: [a]) | 1| whatever |<<Didn't match subrule: | | |[not_before_bar]>> | 1| whatever |<<Didn't match rule>> | prd......1/1 # Failed test 'aa' # at prd.t line 16. # Looks like you failed 1 test of 1. prd...... Dubious, test returned 1 (wstat 256, 0x100) Failed 1/1 subtests Test Summary Report ------------------- prd.t (Wstat: 256 Tests: 1 Failed: 1) Failed test: 1 Non-zero exit status: 1 Files=1, Tests=1, 0 wallclock secs ( 0.01 usr 0.01 sys + 0.00 cusr 0.07 csys = 0.09 CPU) Result: FAIL
From: mhhollomon [...] gmail.com
On Mon Jan 12 19:25:24 2009, POLETTIX wrote: Show quoted text
> Hello, > > unless I completely missed the point about negative look-aheads (and > this may well be!), I think there might be a bug in its handling inside > Parse::RecDescent. Please find attached a minimal test file, complete > with its output with "prove -v" (perl 5.8.8 and P::RD 1.96.0). > > Cheers, > > Flavio.
There is definitely a problem with the way returns from the negative look ahead is handled. If you change the rule to say ... not_before_bar : foo ...!bar { 1 } it works as expected. $_tok is assigned the return from bar, but if bar fails (as you would want) it is undefined. So, the default action of $item[-1] takes the undef and fails the production.
Subject: Patch for Negative lookahead not working
From: mhhollomon [...] gmail.com
Attached is a proof of concept patch that fixes at the specific problem reported. I suspect, however that there other paths in the code that need correcting. I ran this against some of my heavy grammars and didn't see any regressions.
Subject: neg_lookahead_bug.patch
*** RecDescent.pm.orig 2010-11-10 07:02:39.607127500 -0500 --- RecDescent.pm 2010-11-10 08:30:21.156426200 -0500 *************** sub ' . $namespace . '::' . $self->{"nam *** 441,447 **** ' : '') . ' ! $_[1] = $text; # NOT SURE THIS IS NEEDED Parse::RecDescent::_trace(q{<<Didn\'t match rule>>}, Parse::RecDescent::_tracefirst($_[1]), q{' . $self->{"name"} .'}, --- 441,447 ---- ' : '') . ' ! #$_[1] = $text; # NOT SURE THIS IS NEEDED Parse::RecDescent::_trace(q{<<Didn\'t match rule>>}, Parse::RecDescent::_tracefirst($_[1]), q{' . $self->{"name"} .'}, *************** sub code($$$$) *** 884,889 **** --- 884,890 ---- if defined $::RD_TRACE; last; } + ' . ($self->{"lookahead"}<0?'$_tok = 0;':'') . ' Parse::RecDescent::_trace(q{>>Matched action<< (return value: [} . $_tok . q{])}, Parse::RecDescent::_tracefirst($text)) *************** sub code($$$$) *** 947,952 **** --- 948,954 ---- ' . ($self->{"lookahead"} ? '$text = $_savetext and ' : '' ) .' last ' . ($self->{"lookahead"}<0?'if':'unless') . ' defined $_tok; + ' . ($self->{"lookahead"}<0?'$_tok = 0;':'') . ' push @item, $item{'.$self->{hashname}.'}=$_tok; ' . ($self->{"lookahead"} ? '$text = $_savetext;' : '' ) .' ' *************** sub code($$$$) *** 1376,1381 **** --- 1378,1384 ---- $expectation->failed(); last; } + ' . ($self->{"lookahead"}<0?'$_tok = 0;':'') . ' Parse::RecDescent::_trace(q{>>Matched subrule: [' . $self->{subrule} . ']<< (return value: [} . $_tok . q{]},
Subject: Lookahead on implied subrule not working
From: mhhollomon [...] gmail.com
On Wed Nov 10 12:32:36 2010, markhh wrote: Show quoted text
> Attached is a proof of concept patch that fixes at the specific problem > reported. I suspect, however that there other paths in the code that > need correcting.
I was right ... The form: not_before_bar : foo ...!('z' | 'x') did not work at all. The problem was that _generate was not respecting the $lookahead setting when modifying the grammar to replace the inline subrule with the autogenerated rule. Version 2 patch attached. Season to taste.
Subject: neg_lookahead_bug_v2.patch
*** RecDescent.pm.orig 2010-11-10 07:02:39.607127500 -0500 --- lib/Parse/RecDescent.pm 2010-11-10 14:12:43.222205500 -0500 *************** sub ' . $namespace . '::' . $self->{"nam *** 441,447 **** ' : '') . ' ! $_[1] = $text; # NOT SURE THIS IS NEEDED Parse::RecDescent::_trace(q{<<Didn\'t match rule>>}, Parse::RecDescent::_tracefirst($_[1]), q{' . $self->{"name"} .'}, --- 441,447 ---- ' : '') . ' ! #$_[1] = $text; # NOT SURE THIS IS NEEDED Parse::RecDescent::_trace(q{<<Didn\'t match rule>>}, Parse::RecDescent::_tracefirst($_[1]), q{' . $self->{"name"} .'}, *************** sub code($$$$) *** 884,889 **** --- 884,890 ---- if defined $::RD_TRACE; last; } + ' . ($self->{"lookahead"}<0?'$_tok = 0;':'') . ' Parse::RecDescent::_trace(q{>>Matched action<< (return value: [} . $_tok . q{])}, Parse::RecDescent::_tracefirst($text)) *************** sub code($$$$) *** 947,952 **** --- 948,954 ---- ' . ($self->{"lookahead"} ? '$text = $_savetext and ' : '' ) .' last ' . ($self->{"lookahead"}<0?'if':'unless') . ' defined $_tok; + ' . ($self->{"lookahead"}<0?'$_tok = 0;':'') . ' push @item, $item{'.$self->{hashname}.'}=$_tok; ' . ($self->{"lookahead"} ? '$text = $_savetext;' : '' ) .' ' *************** sub code($$$$) *** 1376,1381 **** --- 1378,1384 ---- $expectation->failed(); last; } + ' . ($self->{"lookahead"}<0?'$_tok = 0;':'#--no look ahead') . ' Parse::RecDescent::_trace(q{>>Matched subrule: [' . $self->{subrule} . ']<< (return value: [} . $_tok . q{]}, *************** sub _generate($$$;$$) *** 1940,1946 **** my $implicit = $rule->nextimplicit; $self->_generate("$implicit : $code",$replace,1); my $pos = pos $grammar; ! substr($grammar,$pos,0,$implicit); pos $grammar = $pos;; } elsif ($grammar =~ m/$ENDDIRECTIVEMK/gco) --- 1943,1950 ---- my $implicit = $rule->nextimplicit; $self->_generate("$implicit : $code",$replace,1); my $pos = pos $grammar; ! my $lookahead_str = ($lookahead > 0) ? '...' : ($lookahead < 0) ? '...!' : ''; ! substr($grammar,$pos,0,$lookahead_str . $implicit); pos $grammar = $pos;; } elsif ($grammar =~ m/$ENDDIRECTIVEMK/gco)