Bug #133619 for Regexp-Grammars: built regexp is undefined with perl 5.16.3

Wed Oct 28 05:01:00 2020 DAVEWOOD [...] cpan.org - Ticket created

Subject:

built regexp is undefined with perl 5.16.3

one of my modules is using Regexp::Grammars and cpantesters reported this test failure http://matrix.cpantesters.org/?dist=Parqus%200.04;os=linux;perl=5.16.3;reports=1 I can reproduce the issue with this code, returns undef using perl 5.16.3 ``` #!/usr/bin/env perl use 5.016; use strict; use warnings; use Regexp::Grammars; my $value_regex = qr/\w+/; my $re = eval q{qr/ ^ <[query]>* $ <rule: query> <MATCH= (\w+)> /xms}; use Data::Dumper; warn Dumper($re); ```

Wed Oct 28 15:40:12 2020 damian [...] conway.org - Correspondence added

Subject:	Re: [rt.cpan.org #133619] built regexp is undefined with perl 5.16.3
Date:	Wed, 28 Oct 2020 19:39:15 +0000
To:	bug-Regexp-Grammars [...] rt.cpan.org
From:	Damian Conway <damian [...] conway.org>

Thanks for the report, David. However, I cannot reproduce this problem (on MacOS). On my system, your example code compiles and runs as expected (i.e. it dumps a huge augmented regex) when using the latest Regex::Grammars version (1.057) on every version of Perl from 5.10 to 5.32. Given that I can't reproduce the issue, I'm unsure how to remediate it. Could you try changing your dump to: warn Dumper($@); so we can see what exception is being raised within the eval? Damian

Wed Oct 28 15:40:13 2020 The RT System itself - Status changed from 'new' to 'open'

Thu Oct 29 14:08:10 2020 DAVEWOOD [...] cpan.org - Correspondence added

my bad, the example code i submitted in the initial but report does not reflect the bug. ``` #!/usr/bin/env perl use 5.016; use strict; use warnings; use Regexp::Grammars; my $subre = qr/[\w-]+/; my $re = eval q{qr/ <logfile: parser_log > <[query]>* <rule: query> <MATCH= ($subre)> /xms}; use Data::Dumper; warn Dumper($@); ``` and this is the output $VAR1 = 'Eval-group not allowed at runtime, use re \'eval\' in regex m/(?x)(?{; *Regexp::Grammars::LOGFILE = Regexp::Grammars::_open_log(\'>>\',\'-\', \'for regex gramma.../ at (eval 1) line 1. ';

Thu Oct 29 16:49:28 2020 DCONWAY [...] cpan.org - Status changed from 'open' to 'resolved'

Thu Oct 29 16:49:38 2020 damian [...] conway.org - Correspondence added

Subject:	Re: [rt.cpan.org #133619] built regexp is undefined with perl 5.16.3
Date:	Thu, 29 Oct 2020 20:48:48 +0000
To:	bug-Regexp-Grammars [...] rt.cpan.org
From:	Damian Conway <damian [...] conway.org>

Aha. Thanks for the updated sample code. This behaviour was caused by changes to the regex parsing mechanism, which were introduced in Perl v5.18. As the two parsing mechanisms were fundamentally incompatible (in terms of which point in the parsing process variables are interpolated into regexes), Regexp::Grammars had to be retooled to support this change. Specifically, it was necessary to allow edge-cases like this fail under v5.16 and earlier, so that they could continue to work under v5.18 and later. The workaround for v5.16 and earlier is to place a "use re 'eval'" inside the string "eval", like so: use 5.016; use strict; use warnings; use Regexp::Grammars; my $subre = qr/[\w-]+/; my $re = eval q{ use re 'eval'; # WORKAROUND HERE qr/<logfile: parser_log > <[query]>* <rule: query> <MATCH= ($subre)> /xms}; use Data::Dumper; warn Dumper($@); warn Dumper($re); With this workaround, the sample code behaves correctly under Perls v5.10 to v5.32 Hope this helps, Damian

Fri Oct 30 05:51:00 2020 DAVEWOOD [...] cpan.org - Correspondence added

yeah that fixed it, thank you!