Bug #120214 for MarpaX-Languages-C-AST: $token undef @ MarpaX/Languages/C/AST/Scope.pm:205

Sun Feb 12 12:17:02 2017 kai.extern [...] gmail.com - Ticket created

Subject:	$token undef @ MarpaX/Languages/C/AST/Scope.pm:205
Date:	Sun, 12 Feb 2017 18:16:44 +0100
To:	bug-MarpaX-Languages-C-AST [...] rt.cpan.org
From:	Kai Henningsen <kai.extern [...] gmail.com>

So I wondered if I should really be using MarpaX::Languages::C::AST directly, and not the C::Scan compatibility stuff. However, that also had a problem. Perl program used: #! /usr/bin/perl -w use strict; use Data::Dumper::Simple; $Data::Dumper::Useqq = 1; $Data::Dumper::Sortkeys = 1; no warnings qw( qw experimental::smartmatch ); $| = 1; use MarpaX::Languages::C::AST; use Scalar::Util; my $cc = 'c'; my $src = "uv0.$cc"; my $ppsrc = "uv1.$cc"; my $pppsrc = "uv2.$cc"; open UV0, '>', $src or die "$src: $!"; print UV0 "#include <uv.h>\n"; close UV0 or die "$src: $!"; system('gcc', '-E', $src, '-o', $ppsrc) and die "gcc -E $src -o $ppsrc: $! $?"; open UV1, '<', $ppsrc or die "$ppsrc: $!"; $/ = undef; my $srcpp = <UV1>; close UV1 or die "$ppsrc: $!"; my $parser = MarpaX::Languages::C::AST->new(lazy => 1); my $ast = $parser->parse(\$srcpp)->value(1); print Dumper($ast); And this produced: Use of uninitialized value $token in hash element at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Scope.pm line 205 (#1) (W uninitialized) An undefined value was used as if it were already defined. It was interpreted as a "" or a 0, but maybe it was a mistake. To suppress this warning assign a defined value to your variables. To help you figure out what was undefined, perl will try to tell you the name of the variable (if any) that was undefined. In some cases it cannot do this, so it also tells you what operation you used the undefined value in. Note, however, that perl optimizes your program and the operation displayed in the warning may not necessarily appear literally in your program. For example, "that $foo" is usually optimized into "that " . $foo, and the warning will refer to the concatenation (.) operator, even though there is no . in your program. Uncaught exception from user code: Use of uninitialized value $token in hash element at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Scope.pm line 205. MarpaX::Languages::C::AST::Scope::parseObscureTypedef(MarpaX::Languages::C::AST::Scope=HASH(0x55b910e3b8f8), undef) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback/Events.pm line 380 MarpaX::Languages::C::AST::Callback::Events::_enterOrObscureTypedef(MarpaX::Languages::C::AST::Callback=ARRAY(0x55b91209b9f0), -1, ARRAY(0x55b91588e590)) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback/Events.pm line 366 MarpaX::Languages::C::AST::Callback::Events::_declarationCheck(MarpaX::Languages::C::AST::Callback::Method=ARRAY(0x55b915412508), MarpaX::Languages::C::AST::Callback=ARRAY(0x55b91209b9f0), ARRAY(0x55b91588d950)) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback.pm line 389 MarpaX::Languages::C::AST::Callback::_fire(MarpaX::Languages::C::AST::Callback=ARRAY(0x55b91209b9f0)) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback.pm line 204 MarpaX::Languages::C::AST::Callback::exec(undef, "<process>") called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback/Events.pm line 548 MarpaX::Languages::C::AST::Callback::Events::_subFire_optimized(MarpaX::Languages::C::AST::Callback::Method=ARRAY(0x55b91544ded0), MarpaX::Languages::C::AST::Callback::Events=ARRAY(0x55b912393150), ARRAY(0x55b91588f4e8), "declarationCheck", MarpaX::Languages::C::AST::Callback=ARRAY(0x55b91209b9f0), HASH(0x55b912483ac0), HASH(0x55b9154307f0)) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback.pm line 389 MarpaX::Languages::C::AST::Callback::_fire(MarpaX::Languages::C::AST::Callback::Events=ARRAY(0x55b912393150)) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback.pm line 204 MarpaX::Languages::C::AST::Callback::exec(undef, "translationUnit\$", "^externalDeclaration", "declarationCheck\$") called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST.pm line 262 MarpaX::Languages::C::AST::_doEvents(MarpaX::Languages::C::AST=HASH(0x55b91538b8c8)) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST.pm line 171 MarpaX::Languages::C::AST::parse(MarpaX::Languages::C::AST=HASH(0x55b91538b8c8), SCALAR(0x55b9118dd398)) called at ./make_uv_wrapper.pl line 32 Am I doing something wrong? Looking at the source (and a bit of debug output), I can't see how I'd have any influence there. It seems to happen the very first time it uses this code path. -- Show quoted text

-----BEGIN GEEK CODE BLOCK----- Version: 3.12 GCM/CS/IT d s+: a++ C++ UL++++ P+++ L+++ E--- W++@ N@ !o !K w(++) O-@ M-@ V-- PS++@ PE- Y+ PGP+@ t- 5 X- !tv b++>+++ D--- G e+ h-- !r y? ------END GEEK CODE BLOCK------

Sun Feb 12 13:44:17 2017 JDDPAUSE [...] cpan.org - Correspondence added

lazy option is dangerous. You should not send an already preprocessed output, i.e: perl -Ilib -MMarpaX::Languages::C::AST -e 'MarpaX::Languages::C::AST->new->parse(\shift)' "#include <uv>" C::Scan offers a more user-friendly output, at the cost of heavy XML manipulations that make it much slower though: perl -Ilib -MData::Dumper -MMarpaX::Languages::C::Scan -e 'my $this = shift; print Dumper(MarpaX::Languages::C::Scan->new(filename => $this, filename_filter => $this)->fdecls)' /usr/include/uv.h Regards. On Sun Feb 12 12:17:02 2017, kai.extern@gmail.com wrote: Show quoted text

> So I wondered if I should really be using MarpaX::Languages::C::AST > directly, and not the C::Scan compatibility stuff. However, that also > had a > problem. > > Perl program used: > > #! /usr/bin/perl -w > > use strict; > use Data::Dumper::Simple; > $Data::Dumper::Useqq = 1; > $Data::Dumper::Sortkeys = 1; > no warnings qw( qw experimental::smartmatch ); > > $| = 1; > > use MarpaX::Languages::C::AST; > use Scalar::Util; > > my $cc = 'c'; > my $src = "uv0.$cc"; > my $ppsrc = "uv1.$cc"; > my $pppsrc = "uv2.$cc"; > > open UV0, '>', $src or die "$src: $!"; > print UV0 "#include <uv.h>\n"; > close UV0 or die "$src: $!"; > > system('gcc', '-E', $src, '-o', $ppsrc) and die "gcc -E $src -o > $ppsrc: $! > $?"; > > open UV1, '<', $ppsrc or die "$ppsrc: $!"; > $/ = undef; > my $srcpp = <UV1>; > close UV1 or die "$ppsrc: $!"; > > my $parser = MarpaX::Languages::C::AST->new(lazy => 1); > > my $ast = $parser->parse(\$srcpp)->value(1); > > print Dumper($ast); > > > And this produced: > > > Use of uninitialized value $token in hash element at > /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Scope.pm > line > 205 (#1) > (W uninitialized) An undefined value was used as if it were > already > defined. It was interpreted as a "" or a 0, but maybe it was a > mistake. > To suppress this warning assign a defined value to your variables. > > To help you figure out what was undefined, perl will try to tell you > the name of the variable (if any) that was undefined. In some cases > it cannot do this, so it also tells you what operation you used the > undefined value in. Note, however, that perl optimizes your program > and the operation displayed in the warning may not necessarily appear > literally in your program. For example, "that $foo" is usually > optimized into "that " . $foo, and the warning will refer to the > concatenation (.) operator, even though there is no . in > your program. > > Uncaught exception from user code: > Use of uninitialized value $token in hash element at > /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Scope.pm line 205. > > MarpaX::Languages::C::AST::Scope::parseObscureTypedef(MarpaX::Languages::C::AST::Scope=HASH(0x55b910e3b8f8), > undef) called at > /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback/Events.pm > line > 380 > > MarpaX::Languages::C::AST::Callback::Events::_enterOrObscureTypedef(MarpaX::Languages::C::AST::Callback=ARRAY(0x55b91209b9f0), > -1, ARRAY(0x55b91588e590)) called at > /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback/Events.pm > line > 366 > > MarpaX::Languages::C::AST::Callback::Events::_declarationCheck(MarpaX::Languages::C::AST::Callback::Method=ARRAY(0x55b915412508), > MarpaX::Languages::C::AST::Callback=ARRAY(0x55b91209b9f0), > ARRAY(0x55b91588d950)) called at > /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback.pm line > 389 > > MarpaX::Languages::C::AST::Callback::_fire(MarpaX::Languages::C::AST::Callback=ARRAY(0x55b91209b9f0)) > called at > /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback.pm > line 204 > MarpaX::Languages::C::AST::Callback::exec(undef, "<process>") > called at > /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback/Events.pm > line > 548 > > MarpaX::Languages::C::AST::Callback::Events::_subFire_optimized(MarpaX::Languages::C::AST::Callback::Method=ARRAY(0x55b91544ded0), > MarpaX::Languages::C::AST::Callback::Events=ARRAY(0x55b912393150), > ARRAY(0x55b91588f4e8), "declarationCheck", > MarpaX::Languages::C::AST::Callback=ARRAY(0x55b91209b9f0), > HASH(0x55b912483ac0), HASH(0x55b9154307f0)) called at > /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback.pm line > 389 > > MarpaX::Languages::C::AST::Callback::_fire(MarpaX::Languages::C::AST::Callback::Events=ARRAY(0x55b912393150)) > called at > /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback.pm > line 204 > MarpaX::Languages::C::AST::Callback::exec(undef, > "translationUnit\$", "^externalDeclaration", "declarationCheck\$") > called > at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST.pm line 262 > > MarpaX::Languages::C::AST::_doEvents(MarpaX::Languages::C::AST=HASH(0x55b91538b8c8)) > called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST.pm line > 171 > > MarpaX::Languages::C::AST::parse(MarpaX::Languages::C::AST=HASH(0x55b91538b8c8), > SCALAR(0x55b9118dd398)) called at ./make_uv_wrapper.pl line 32 > > > Am I doing something wrong? Looking at the source (and a bit of debug > output), I can't see how I'd have any influence there. It seems to > happen > the very first time it uses this code path.

Sun Feb 12 13:44:17 2017 The RT System itself - Status changed from 'new' to 'open'

Sat Feb 18 06:14:02 2017 kai.extern [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #120214] $token undef @ MarpaX/Languages/C/AST/Scope.pm:205
Date:	Sat, 18 Feb 2017 11:13:31 +0000
To:	bug-MarpaX-Languages-C-AST [...] rt.cpan.org
From:	Kai Henningsen <kai.extern [...] gmail.com>

Jean-Damien Durand via RT <bug-MarpaX-Languages-C-AST@rt.cpan.org> schrieb am So., 12. Feb. 2017 um 19:44 Uhr: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=120214 > > > lazy option is dangerous. You should not send an already preprocessed > output, i.e: > > perl -Ilib -MMarpaX::Languages::C::AST -e > 'MarpaX::Languages::C::AST->new->parse(\shift)' "#include <uv>" >

Ok, that doesn't actually do anything useful. C::AST does not seem to know how to preprocess anything. (Strace shows it doesn't try to touch uv.h, and the result doesn't contain any relevant data.) Also, from the docs, lazy actually seems like the SAFE option: in case there is more than one possible parse, show me all of them. C::Scan offers a more user-friendly output, at the cost of heavy XML Show quoted text

> manipulations that make it much slower though: >

Yeah, tried that, I'm not so sure about the way it returns the data. I think I'd probably prefer it less postprocessed and turned into strings I'd then have to parse myself. In any case, leaving out lazy does not help, it crashes at the exact same spot: Use of uninitialized value $token in hash element at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Scope.pm line 205 (#1) (W uninitialized) An undefined value was used as if it were already defined. It was interpreted as a "" or a 0, but maybe it was a mistake. To suppress this warning assign a defined value to your variables. To help you figure out what was undefined, perl will try to tell you the name of the variable (if any) that was undefined. In some cases it cannot do this, so it also tells you what operation you used the undefined value in. Note, however, that perl optimizes your program and the operation displayed in the warning may not necessarily appear literally in your program. For example, "that $foo" is usually optimized into "that " . $foo, and the warning will refer to the concatenation (.) operator, even though there is no . in your program. Uncaught exception from user code: Use of uninitialized value $token in hash element at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Scope.pm line 205. MarpaX::Languages::C::AST::Scope::parseObscureTypedef(MarpaX::Languages::C::AST::Scope=HASH(0x5589bb9af820), undef) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback/Events.pm line 380 MarpaX::Languages::C::AST::Callback::Events::_enterOrObscureTypedef(MarpaX::Languages::C::AST::Callback=ARRAY(0x5589bcc0cea0), -1, ARRAY(0x5589c03ff3d8)) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback/Even MarpaX::Languages::C::AST::Callback::Events::_declarationCheck(MarpaX::Languages::C::AST::Callback::Method=ARRAY(0x5589bff88268), MarpaX::Languages::C::AST::Callback=ARRAY(0x5589bcc0cea0), ARRAY(0x5589c03fe750)) called at /usr/lo MarpaX::Languages::C::AST::Callback::_fire(MarpaX::Languages::C::AST::Callback=ARRAY(0x5589bcc0cea0)) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback.pm line 204 MarpaX::Languages::C::AST::Callback::exec(undef, "<process>") called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback/Events.pm line 548 MarpaX::Languages::C::AST::Callback::Events::_subFire_optimized(MarpaX::Languages::C::AST::Callback::Method=ARRAY(0x5589bffc3c18), MarpaX::Languages::C::AST::Callback::Events=ARRAY(0x5589bcf04230), ARRAY(0x5589c0400378), "declara MarpaX::Languages::C::AST::Callback::_fire(MarpaX::Languages::C::AST::Callback::Events=ARRAY(0x5589bcf04230)) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST/Callback.pm line 204 MarpaX::Languages::C::AST::Callback::exec(undef, "translationUnit\$", "^externalDeclaration", "declarationCheck\$") called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST.pm line 262 MarpaX::Languages::C::AST::_doEvents(MarpaX::Languages::C::AST=HASH(0x5589bff01610)) called at /usr/local/share/perl/5.22.2/MarpaX/Languages/C/AST.pm line 171 MarpaX::Languages::C::AST::parse(MarpaX::Languages::C::AST=HASH(0x5589bff01610), SCALAR(0x5589bcc46168)) called at ./make_uv_wrapper.pl line 32 So, given that C::Scan and c2ast manage to parse this (although with lots of callbacks and so on), what is it missing to actually work?