Bug #120195 for MarpaX-Languages-C-AST: Parsing bug

Fri Feb 10 19:27:50 2017 kai.extern [...] gmail.com - Ticket created

Subject:	Parsing bug
Date:	Sat, 11 Feb 2017 01:27:14 +0100
To:	bug-MarpaX-Languages-C-AST [...] rt.cpan.org
From:	Kai Henningsen <kai.extern [...] gmail.com>

I'm trying to parse the libuv headers: my $scan = MarpaX::Languages::C::Scan->new(content => "\n#include <uv.h>\n", filename_filter => qr[/uv]); Noticed: 1. Parsing from a string is broken: MarpaX/Languages/C/AST/Grammar/ISO_ANSI_C_2011/Scan.pm: - print($self->{_content2fh}, $opts{content}); + print {$self->{_content2fh}} $opts{content}; 2. The parser has serious problems with function pointer types. First example: "struct uv__work {\n void (*work)(struct uv__work *w);\n void (*done)(struct uv__work *w, int status);\n struct uv_loop_s* loop;\n void* wq[2];\n}" No, that is not one function with three parameters named w, w, and status, returning a struct uv__work. -- Show quoted text

-----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCM/CS/IT d s+: a++ C++ UL++++ P+++ L+++ E--- W++@ N@ !o !K w(++) O-@ M-@ V-- PS++@ PE- Y+ PGP+@ t- 5 X- !tv b++>+++ D--- G e+ h-- !r y? ------END GEEK CODE BLOCK------

Sat Feb 11 05:35:52 2017 JDDPAUSE [...] cpan.org - Correspondence added

I pushed the dev version on github into CPAN that seems to fix these two issues - this will be version 0.46.

Sat Feb 11 05:35:52 2017 The RT System itself - Status changed from 'new' to 'open'

Sat Feb 11 06:31:41 2017 kai.extern [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #120195] Parsing bug
Date:	Sat, 11 Feb 2017 11:31:14 +0000
To:	bug-MarpaX-Languages-C-AST [...] rt.cpan.org
From:	Kai Henningsen <kai.extern [...] gmail.com>

Thanks! That does indeed fir the first issue. Unfortunately, it doesn't seem to do anything about the second, I still get $fd = [ "struct uv__work", "ANON0", [ [ "struct uv__work", "w", undef, "struct uv__work *w", "" ], [ "struct uv__work", "w", undef, "struct uv__work *w", "" ], [ "int", "status", undef, "int status", "" ] ], "struct uv__work {\n void (*work)(struct uv__work *w);\n void (*done)(struct uv__work *w, int status);\n struct uv_loop_s* loop;\n void* wq[2];\n}", undef ]; Jean-Damien Durand via RT <bug-MarpaX-Languages-C-AST@rt.cpan.org> schrieb am Sa., 11. Feb. 2017 um 11:36 Uhr: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=120195 > > > I pushed the dev version on github into CPAN that seems to fix these two > issues - this will be version 0.46. > >

Sat Feb 11 08:36:56 2017 JDDPAUSE [...] cpan.org - Correspondence added

Hello, Can you please post the full test program that is producing this output. The raw AST seems ok, this is the transformation to a C::Scan like thingy that seems buggy. To be confirmed. Thanks, Regards. On Sat Feb 11 06:31:41 2017, kai.extern@gmail.com wrote: Show quoted text

> Thanks! > > That does indeed fir the first issue. Unfortunately, it doesn't seem to do > anything about the second, I still get > > $fd = [ > "struct uv__work", > "ANON0", > [ > [ > "struct uv__work", > "w", > undef, > "struct uv__work *w", > "" > ], > [ > "struct uv__work", > "w", > undef, > "struct uv__work *w", > "" > ], > [ > "int", > "status", > undef, > "int status", > "" > ] > ], > "struct uv__work {\n void (*work)(struct uv__work *w);\n void > (*done)(struct uv__work *w, int status);\n struct uv_loop_s* loop;\n > void* wq[2];\n}", > undef > ]; > > > > > Jean-Damien Durand via RT <bug-MarpaX-Languages-C-AST@rt.cpan.org> schrieb > am Sa., 11. Feb. 2017 um 11:36 Uhr: >

> > <URL: https://rt.cpan.org/Ticket/Display.html?id=120195 > > > > > I pushed the dev version on github into CPAN that seems to fix these two > > issues - this will be version 0.46. > > > >

Sat Feb 11 08:55:24 2017 kai.extern [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #120195] Parsing bug
Date:	Sat, 11 Feb 2017 13:54:41 +0000
To:	bug-MarpaX-Languages-C-AST [...] rt.cpan.org
From:	Kai Henningsen <kai.extern [...] gmail.com>

It's actually in the bug report: #include <uv.h> Those headers are from libuv 1.9.1 from http://dist.libuv.org/dist/v1.9.1/ (actually the Ubuntu package thereof, but I don't think that makes a difference). What might or might not make a difference is that the various system headers included are from Ubuntu 16.10. Jean-Damien Durand via RT <bug-MarpaX-Languages-C-AST@rt.cpan.org> schrieb am Sa., 11. Feb. 2017 um 14:37 Uhr: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=120195 > > > Hello, > > Can you please post the full test program that is producing this output. > The raw AST seems ok, this is the transformation to a C::Scan like thingy > that seems buggy. To be confirmed. > > Thanks, Regards. > > On Sat Feb 11 06:31:41 2017, kai.extern@gmail.com wrote:

> > Thanks! > > > > That does indeed fir the first issue. Unfortunately, it doesn't seem to

> do

> > anything about the second, I still get > > > > $fd = [ > > "struct uv__work", > > "ANON0", > > [ > > [ > > "struct uv__work", > > "w", > > undef, > > "struct uv__work *w", > > "" > > ], > > [ > > "struct uv__work", > > "w", > > undef, > > "struct uv__work *w", > > "" > > ], > > [ > > "int", > > "status", > > undef, > > "int status", > > "" > > ] > > ], > > "struct uv__work {\n void (*work)(struct uv__work *w);\n void > > (*done)(struct uv__work *w, int status);\n struct uv_loop_s* loop;\n > > void* wq[2];\n}", > > undef > > ]; > > > > > > > > > > Jean-Damien Durand via RT <bug-MarpaX-Languages-C-AST@rt.cpan.org>

> schrieb

> > am Sa., 11. Feb. 2017 um 11:36 Uhr: > >

> > > <URL: https://rt.cpan.org/Ticket/Display.html?id=120195 > > > > > > > I pushed the dev version on github into CPAN that seems to fix these

> two

> > > issues - this will be version 0.46. > > > > > >

> > > >

Sat Feb 11 09:03:05 2017 JDDPAUSE [...] cpan.org - Correspondence added

I mean the full perl source code - I want to know which call you do after the C::Scan->new(). Thanks.

Sat Feb 11 09:22:50 2017 kai.extern [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #120195] Parsing bug
Date:	Sat, 11 Feb 2017 14:22:27 +0000
To:	bug-MarpaX-Languages-C-AST [...] rt.cpan.org
From:	Kai Henningsen <kai.extern [...] gmail.com>

Ah, ok. It's just the beginnings of something supposed to become useful: #! /usr/bin/perl -w use strict; use Data::Dumper::Simple; $Data::Dumper::Useqq = 1; $Data::Dumper::Sortkeys = 1; no warnings qw( qw experimental::smartmatch ); $| = 1; use MarpaX::Languages::C::Scan; my $scan = MarpaX::Languages::C::Scan->new(content => "\n#include <uv.h>\n", filename_filter => qr[/uv]); #print Dumper($scan->parsed_fdecls); my $fdecls = $scan->parsed_fdecls; for my $fd (@$fdecls) { my $arg = $$fd[2]; my @args; for my $a (@$arg) { my %a = ( ty => $$a[0], nm => $$a[1], args => $$a[2], ft => $$a[3], mod => $$a[4], ); push @args, { %a }; }; my %fd = ( rt => $$fd[0], nm => $$fd[1], args => [ @args ], ft => $$fd[3], mod => $$fd[4], ); print Dumper($fd); print Dumper(%fd); }; Jean-Damien Durand via RT <bug-MarpaX-Languages-C-AST@rt.cpan.org> schrieb am Sa., 11. Feb. 2017 um 15:03 Uhr: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=120195 > > > I mean the full perl source code - I want to know which call you do after > the C::Scan->new(). > Thanks. > >

Sat Feb 11 09:43:48 2017 JDDPAUSE [...] cpan.org - Correspondence added

Ok, this is reproduced and has to do with the AST transfomation, the former being ok. Will keep you informed. Thanks for the report, Regards.

Sun Feb 12 07:05:22 2017 JDDPAUSE [...] cpan.org - Correspondence added

FYI I believe that the function declarations inside the structure should not appear in fdecls, that is concerned only by top-level declarations in the C::Scan formalism. The following commit is fixing this: https://github.com/jddurand/MarpaX-Languages-C-AST/commit/9465b8dfcc7bc42fc93ff242495624534da12136 You might want to duplicate this easy fix locally. I saw another deficiency with structure when calling $scan->cdecl(), so I want to fix the later before doing another release. Thanks, Regards. On Sat Feb 11 09:43:48 2017, JDDPAUSE wrote: Show quoted text

> Ok, this is reproduced and has to do with the AST transfomation, the > former being ok. > Will keep you informed. > Thanks for the report, Regards.

Sun Feb 12 11:59:53 2017 kai.extern [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #120195] Parsing bug
Date:	Sun, 12 Feb 2017 16:59:23 +0000
To:	bug-MarpaX-Languages-C-AST [...] rt.cpan.org
From:	Kai Henningsen <kai.extern [...] gmail.com>

Done. Seems to work as advertised. Jean-Damien Durand via RT <bug-MarpaX-Languages-C-AST@rt.cpan.org> schrieb am So., 12. Feb. 2017 um 13:06 Uhr: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=120195 > > > FYI I believe that the function declarations inside the structure should > not appear in fdecls, that is concerned only by top-level declarations in > the C::Scan formalism. > The following commit is fixing this: > > > https://github.com/jddurand/MarpaX-Languages-C-AST/commit/9465b8dfcc7bc42fc93ff242495624534da12136 > > You might want to duplicate this easy fix locally. > > I saw another deficiency with structure when calling $scan->cdecl(), so I > want to fix the later before doing another release. > > Thanks, Regards. > > On Sat Feb 11 09:43:48 2017, JDDPAUSE wrote:

> > Ok, this is reproduced and has to do with the AST transfomation, the > > former being ok. > > Will keep you informed. > > Thanks for the report, Regards.

> > > >

Thu Apr 13 01:50:59 2017 JDDPAUSE [...] cpan.org - Status changed from 'open' to 'resolved'

Thu Apr 13 01:50:59 2017 JDDPAUSE [...] cpan.org - Fixed in 0.47 added