Skip Menu |

This queue is for tickets about the Marpa-XS CPAN distribution.

Report information
The Basics
Id: 72869
Status: resolved
Priority: 0/
Queue: Marpa-XS
People
Owner: jkegl [...] cpan.org
Requestors: tomas.jirotka [...] gooddata.com
Cc:
AdminCc:
Bug Information
Severity: (no value)
Broken in:
  • 0.024000
  • 0.020000
  • 0.021_000
  • 0.022000
  • 0.023_000
Fixed in:
  • 0.025_001
  • 0.026000

History Show all quoted text
  Thu Dec 01 10:25:49 2011 tomas.jirotka [...] gooddata.com - Ticket created
Subject: Error when two empty rules are in a row
Date: Thu, 1 Dec 2011 16:25:22 +0100
To: bug-Marpa-XS [...] rt.cpan.org
From: Tomáš Jirotka <tomas.jirotka [...] gooddata.com>
Hi, I have written a small grammar for a language not very different from SQL. I enclose the source code in attachment. There are several optional clauses as can be seen from the figure: CREATE METRIC id_metric AS SELECT number (BY|empty) (FOR|empty) (WHERE (TRUE|FALSE)|empty) (WITH PF|empty) Here, 'empty' states for the empty string, other lower case words are variables, upper case are keywords. On line 41, there is a rule 'TypeDef' -> 4 tokens, but the token 'AS' is evaluated as a part of MetricSelect rule. This is the real output $VAR1 = [ \[ 'Create', 'Metric', 'm', [ 'As', 'Select', [ 1 ], undef, [ 'Where', [ 'True' ] ], undef ] ] ]; while I would expect $VAR1 = [ \[ [ 'Create', [ 'Metric', 'm', 'As', [ 'Select', [ 1 ], undef, undef, [ 'Where', [ 'True' ] ], undef ] ] ] ] ]; You may try it. It is important to note that in Marpa::PP this works properly, only XS's output is wrong. Probably, the problem is in two optional keywords before WHERE. Versions: This is perl, v5.10.1 (*) built for i386-linux-thread-multi XS 0.024000 PP 0.012000 Thank you for producing this nice and useful package! Tomas Tomáš Jirotka Software Engineer GoodData Sokolovská 192/79, 186 00, Prague 8 Office: +420 226 219 421 E-mail: tomas.jirotka@gooddata.com Web: www.gooddata.com
Hi,
I have written a small grammar for a language not very different from SQL. I enclose the source code in attachment. There are several optional clauses as can be seen from the figure:

CREATE METRIC id_metric AS SELECT number (BY|empty) (FOR|empty) (WHERE (TRUE|FALSE)|empty) (WITH PF|empty)

Here, 'empty' states for the empty string, other lower case words are variables, upper case are keywords. On line 41, there is a rule 'TypeDef' -> 4 tokens, but the token 'AS' is evaluated as a part of MetricSelect rule. This is the real output

$VAR1 = [
          \[
              'Create',
              'Metric',
              'm',
              [
                'As',
                'Select',
                [
                  1
                ],
                undef,
                [
                  'Where',
                  [
                    'True'
                  ]
                ],
                undef
              ]
            ]
        ];

while I would expect

$VAR1 = [
          \[
              [
                'Create',
                [
                  'Metric',
                  'm',
                  'As',
                  [
                    'Select',
                    [
                      1
                    ],
                    undef,
                    undef,
                    [
                      'Where',
                      [
                        'True'
                      ]
                    ],
                    undef
                  ]
                ]
              ]
            ]
        ];

You may try it. It is important to note that in Marpa::PP this works properly, only XS's output is wrong. Probably, the problem is in two optional keywords before WHERE.

Versions:
This is perl, v5.10.1 (*) built for i386-linux-thread-multi
XS 0.024000
PP 0.012000


Thank you for producing this nice and useful package!
Tomas


Message body is not shown because sender requested not to inline it.




Tomáš Jirotka
Software Engineer

GoodData
Sokolovská 192/79, 186 00, Prague 8
Office:   +420 226 219 421
Web:     www.gooddata.com

  Thu Dec 01 12:39:47 2011 jkegl [...] cpan.org - Taken
  Thu Dec 01 12:44:52 2011 jkegl [...] cpan.org - Correspondence added
thanks for the very well-documented bug, which I've started work on. My current belief is that the problem is in the XS evaluation logic. By the way, although it is not directly relevent to the bug report, the tokens() method is severely deprecated in favor or the read() method, and is not documented in either Marpa::XS or Marpa::PP. However, I do not believe use of the tokens() method affects the issue here. Again, thanks for the bug report, and the kind words about Marpa -- jeffrey kegler On Thu Dec 01 10:25:49 2011, tomas.jirotka@gooddata.com wrote: Show quoted text
> Hi, > I have written a small grammar for a language not very different from > SQL. I enclose the source code in attachment. There are several > optional clauses as can be seen from the figure: > > CREATE METRIC id_metric AS SELECT number (BY|empty) (FOR|empty) (WHERE > (TRUE|FALSE)|empty) (WITH PF|empty) > > Here, 'empty' states for the empty string, other lower case words are > variables, upper case are keywords. On line 41, there is a rule > 'TypeDef' -> 4 tokens, but the token 'AS' is evaluated as a part of > MetricSelect rule. This is the real output > > $VAR1 = [ > \[ > 'Create', > 'Metric', > 'm', > [ > 'As', > 'Select', > [ > 1 > ], > undef, > [ > 'Where', > [ > 'True' > ] > ], > undef > ] > ] > ]; > > while I would expect > > $VAR1 = [ > \[ > [ > 'Create', > [ > 'Metric', > 'm', > 'As', > [ > 'Select', > [ > 1 > ], > undef, > undef, > [ > 'Where', > [ > 'True' > ] > ], > undef > ] > ] > ] > ] > ]; > > You may try it. It is important to note that in Marpa::PP this works > properly, only XS's output is wrong. Probably, the problem is in > two optional keywords before WHERE. > > Versions: > This is perl, v5.10.1 (*) built for i386-linux-thread-multi > XS 0.024000 > PP 0.012000 > > > Thank you for producing this nice and useful package! > Tomas > > > > > > > Tomáš Jirotka > Software Engineer > > GoodData > Sokolovská 192/79, 186 00, Prague 8 > Office: +420 226 219 421 > E-mail: tomas.jirotka@gooddata.com > Web: www.gooddata.com >
  Thu Dec 01 12:44:53 2011 The RT System itself - Status changed from 'new' to 'open'
  Sat Dec 03 21:31:36 2011 jkegl [...] cpan.org - Broken in 0.024000 added
  Sat Dec 03 21:42:22 2011 jkegl [...] cpan.org - Correspondence added
I have a fix, which I have just uploaded to CPAN as Marpa-XS-0.025_000. I will await cpantesters results on this, then cut an offical release. Thanks, Tomas for a very important and well done bug report. I've also added this problem to Marpa::XS's test suite as t/jirotka.t. The problem was in the evaluation logic. It only occurs under certain very unusual circumstances, circumstances which it takes a lot of internal-speak to describe. Briefly the problems is with nullablss at the start of a rule (which may be a result of an Marpa-internal rewrite) when that rule appears in a Leo chain and is not the same as the top rule. (Leo chains are an internal mechanism used to keep right-recursion from going quadratic.) Many thanks, Tomáš, for nailing down this obscure but nasty little issue so I could fix it. -- jeffrey kegler On Thu Dec 01 12:44:52 2011, JKEGL wrote: Show quoted text
> thanks for the very well-documented bug, which I've started work on. My > current belief is that the problem is in the XS evaluation logic. > > By the way, although it is not directly relevent to the bug report, the > tokens() method is severely deprecated in favor or the read() method, > and is not documented in either Marpa::XS or Marpa::PP. However, I do > not believe use of the tokens() method affects the issue here. > > Again, thanks for the bug report, and the kind words about Marpa > > -- jeffrey kegler > > On Thu Dec 01 10:25:49 2011, tomas.jirotka@gooddata.com wrote:
> > Hi, > > I have written a small grammar for a language not very different from > > SQL. I enclose the source code in attachment. There are several > > optional clauses as can be seen from the figure: > > > > CREATE METRIC id_metric AS SELECT number (BY|empty) (FOR|empty) (WHERE > > (TRUE|FALSE)|empty) (WITH PF|empty) > > > > Here, 'empty' states for the empty string, other lower case words are > > variables, upper case are keywords. On line 41, there is a rule > > 'TypeDef' -> 4 tokens, but the token 'AS' is evaluated as a part of > > MetricSelect rule. This is the real output > > > > $VAR1 = [ > > \[ > > 'Create', > > 'Metric', > > 'm', > > [ > > 'As', > > 'Select', > > [ > > 1 > > ], > > undef, > > [ > > 'Where', > > [ > > 'True' > > ] > > ], > > undef > > ] > > ] > > ]; > > > > while I would expect > > > > $VAR1 = [ > > \[ > > [ > > 'Create', > > [ > > 'Metric', > > 'm', > > 'As', > > [ > > 'Select', > > [ > > 1 > > ], > > undef, > > undef, > > [ > > 'Where', > > [ > > 'True' > > ] > > ], > > undef > > ] > > ] > > ] > > ] > > ]; > > > > You may try it. It is important to note that in Marpa::PP this works > > properly, only XS's output is wrong. Probably, the problem is in > > two optional keywords before WHERE. > > > > Versions: > > This is perl, v5.10.1 (*) built for i386-linux-thread-multi > > XS 0.024000 > > PP 0.012000 > > > > > > Thank you for producing this nice and useful package! > > Tomas > > > > > > > > > > > > > > Tomáš Jirotka > > Software Engineer > > > > GoodData > > Sokolovská 192/79, 186 00, Prague 8 > > Office: +420 226 219 421 > > E-mail: tomas.jirotka@gooddata.com > > Web: www.gooddata.com > >
> >
  Sun Dec 04 19:13:39 2011 jkegl [...] cpan.org - Correspondence added
I have uploaded a 2nd release candidate: Marpa-XS-0.025_001. I accidently left a Modern::Perl dependency in one of the tests, which was making Marpa-XS-0.025_000 fail on platforms without Modern::Perl. (I have nothing against Modern::Perl, btw, it is just that Marpa::XS uses non-core dependencies only when it has no real alternative.) The Modern::Perl issue aside, tests for Marpa-XS-0.025_000 looked good, as do those for Marpa-XS-0.025_001 so far.
  Wed Dec 07 21:30:36 2011 jkegl [...] cpan.org - Correspondence added
I have just uploaded an official release with this fix: Marpa-XS-0.026000. As is my usual practice, I plan to wait 3 days or so for cpantesters or other feedback before marking this issue resolved.
  Wed Dec 07 21:32:14 2011 jkegl [...] cpan.org - Broken in 0.020000 added
  Wed Dec 07 21:32:14 2011 jkegl [...] cpan.org - Broken in 0.021_000 added
  Wed Dec 07 21:32:14 2011 jkegl [...] cpan.org - Broken in 0.022000 added
  Wed Dec 07 21:32:14 2011 jkegl [...] cpan.org - Broken in 0.023_000 added
  Wed Dec 07 21:32:14 2011 jkegl [...] cpan.org - Fixed in 0.025_001 added
  Wed Dec 07 21:32:15 2011 jkegl [...] cpan.org - Fixed in 0.026000 added
  Sun Dec 11 01:44:38 2011 jkegl [...] cpan.org - Correspondence added
We've had a few days of cpantesters results on the offical version, as well as a total lunar eclipse. The auguries all looking well, I am closing this bug as resolved. Many thanks to Tomáš Jirotka for an excellent bug report.
  Sun Dec 11 01:44:39 2011 jkegl [...] cpan.org - Status changed from 'open' to 'resolved'
  Sun Dec 11 05:13:03 2011 tomas.jirotka [...] gooddata.com - Correspondence added
Subject: Re: [rt.cpan.org #72869] Error when two empty rules are in a row
Date: Sun, 11 Dec 2011 11:12:51 +0100
To: bug-Marpa-XS [...] rt.cpan.org
From: Tomáš Jirotka <tomas.jirotka [...] gooddata.com>
Jeffrey, thank you very much for your speedy response and fix of the bug! Tomáš Jirotka On Dec 11, 2011, at 7:44 AM, Jeffrey Kegler via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=72869 > > > We've had a few days of cpantesters results on the offical version, as > well as a total lunar eclipse. The auguries all looking well, I am > closing this bug as resolved. > > Many thanks to Tomáš Jirotka for an excellent bug report.
  Sun Dec 11 05:13:03 2011 The RT System itself - Status changed from 'resolved' to 'open'
  Tue Dec 20 23:20:39 2011 jkegl [...] cpan.org - Status changed from 'open' to 'resolved'