Bug #68672 for Filter-Simple: sequences of /= lines get masked as comments

Mon Jun 06 11:22:00 2011 chm [...] cpan.org - Ticket created

Subject:

sequences of /= lines get masked as comments

If you use the FILTER_ONLY with 'all' then the source
filter passes all lines as expected and the badfilter
catches and fixes the $a:$b syntax error.

If you have FILTER_ONLY with the 'code_no_comments'
argument then the code is mis-folded and the source
filter does not see the text between the two /= on
the lines and misses the syntax fix causing an error.

Subject:

badfilter.pm

package badfilter; use Filter::Simple; FILTER_ONLY code_no_comments => # all => sub { s/(\w+):\$/$1,\$/; }; 1;

Mon Jun 06 12:43:44 2011 chm [...] cpan.org - Correspondence added

The bisect.pm file with the failing lines example
did not get attached so here it is. This is what
I get if I run 'perl -c bisect.pm' as attached:

$ cat badfilter.pm
package badfilter;

use Filter::Simple;

FILTER_ONLY
   code_no_comments =>
   # all =>
sub {
    s/(\w+):\$/$1,\$/;
};

1;

$ cat bisect.pm
use badfilter;

$b /= 1;

$lu->($row:$n1,$r1);

$b /= 1;

$ perl -c bisect.pm
syntax error at bisect.pm line 5, near "$row:"
bisect.pm had compilation errors.

If I edit badfliter.pm to change from using
'code_no_comments' to 'all' in the FILTER_ONLY
specifcation, I get this:

$ cat badfilter.pm
package badfilter;

use Filter::Simple;

FILTER_ONLY
Show quoted text

# code_no_comments =>

all =>
sub {
s/(\w+):\$/$1,\$/;
};

1;

$ perl -c bisect.pm
bisect.pm syntax OK

Subject:

bisect.pm

use badfilter; $b /= 1; $lu->($row:$n1,$r1); $b /= 1;

Tue Jun 07 16:16:41 2011 chm [...] cpan.org - Correspondence added

Tracing through the code in Filter::Simple, I determined
that the problem here is that the extract_quotelike() call
from the Text::Balanced module is treating the two lines
with /= in them as the beginning and end of a multiline
match operator without the m.

Hardwiring the value of $allow_raw_match to 0 makes
the extract_quotelike() work correctly. I don't know what
would be entailed to work around the problem better.

Tue Jun 07 16:16:42 2011 chm [...] cpan.org - Status changed from 'new' to 'open'

Tue Jun 07 22:27:45 2011 damian [...] conway.org - Correspondence added

Subject:	Re: [rt.cpan.org #68672] sequences of /= lines get masked as comments
Date:	Wed, 8 Jun 2011 12:26:57 +1000
To:	bug-Filter-Simple [...] rt.cpan.org
From:	Damian Conway <damian [...] conway.org>

Show quoted text

> Hardwiring the value of $allow_raw_match to 0 makes > the extract_quotelike() work correctly. I don't know what > would be entailed to work around the problem better.

[Comment only, as I'm no longer maintaining the module] Reimplementing the filtering framework around PPI would be the only way to significantly improve its accuracy. But that would then require PPI added to the Perl core, which would be an excellent addition, but seems very unlikely to happen. :-( Damian

Wed Jun 08 07:43:54 2011 devel.chm.01 [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #68672] sequences of /= lines get masked as comments
Date:	Wed, 08 Jun 2011 07:43:46 -0400
To:	bug-Filter-Simple [...] rt.cpan.org
From:	chm <devel.chm.01 [...] gmail.com>

On 6/7/2011 10:27 PM, damian@conway.org via RT wrote: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=68672> >

>> Hardwiring the value of $allow_raw_match to 0 makes >> the extract_quotelike() work correctly. I don't know what >> would be entailed to work around the problem better.

> > [Comment only, as I'm no longer maintaining the module] > > Reimplementing the filtering framework around PPI would be the > only way to significantly improve its accuracy. But that would > then require PPI added to the Perl core, which would be an > excellent addition, but seems very unlikely to happen. :-( > > Damian

Thanks for the reply. I managed to submit reports for Filter::Simple and Text::Balanced issues for this problem: If Text::Balanced were to expose the internal functionality that would address this problem for our uses. --Chris