Skip Menu |

This queue is for tickets about the Convert-Binary-C CPAN distribution.

Report information
The Basics
Id: 45105
Status: rejected
Priority: 0/
Queue: Convert-Binary-C

People
Owner: Nobody in particular
Requestors: bitcard [...] post2.25u.com
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in:
  • 0.71
  • 0.73
Fixed in: (no value)



Subject: definitions of constants are not considered by sourcify()
Hello, first of all, many thanks for this really amazing (and really, really immensely helpful) package! I ran into a problem that when a structures definition involves a constant, the constant's value is not considered by the sourcify() method (instead, 0 is used). This means that re-parsing the sourcified code does not give the same decoding results as obtained from decoding with the original source code. Example: Header file: static const u32 MAX_NUM_OF_TIMESTAMPS = 20; struct MyMessage { TimeStamp timeStampArr[MAX_NUM_OF_TIMESTAMPS]; }; output of sourcify(): struct MyMessage { TimeStamp timeStampArr[0]; }; Best regards, /alex
Subject: Re: [rt.cpan.org #45105] definitions of constants are not considered by sourcify()
Date: Fri, 17 Apr 2009 20:04:33 +0200
To: bug-Convert-Binary-C [...] rt.cpan.org
From: Marcus Holland-Moritz <mhx-perl [...] gmx.net>
Hi Alexander, On 2009-04-17, at 07:06:43 -0400, Alexander Ost via RT wrote: Show quoted text
> Fri Apr 17 07:06:41 2009: Request 45105 was acted upon. > Transaction: Ticket created by arost > Queue: Convert-Binary-C > Subject: definitions of constants are not considered by sourcify() > Broken in: 0.71, 0.73 > Severity: Important > Owner: Nobody > Requestors: bitcard@post2.25u.com > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=45105 > > > > first of all, many thanks for this really amazing (and really, really > immensely helpful) package!
Thanks. :-) Show quoted text
> I ran into a problem that when a structures definition involves a > constant, the constant's value is not considered by the sourcify() > method (instead, 0 is used). > > This means that re-parsing the sourcified code does not give the same > decoding results as obtained from decoding with the original source code.
It does. But see below. Show quoted text
> Example: > > Header file: > static const u32 MAX_NUM_OF_TIMESTAMPS = 20; > > struct MyMessage > { > TimeStamp timeStampArr[MAX_NUM_OF_TIMESTAMPS]; > }; > > output of sourcify(): > > struct MyMessage > { > TimeStamp timeStampArr[0]; > };
Ok, so there are actually two problems, and the latter explains why this isn't a bug. ;-) The first problem is that this does not only affect the sourcify method. It affects everything. The dimension of the array is zero internally: mhx@r2d2 ~ $ cat const.c static const int NUM = 32; struct foo { int bar[NUM]; }; mhx@r2d2 ~ $ perl -MConvert::Binary::C -MData::Dumper -e'print Dumper(Convert::Binary::C->new->parse_file("const.c")->struct)' $VAR1 = { 'identifier' => 'foo', 'align' => 1, 'context' => 'const.c(3)', 'pack' => 0, 'type' => 'struct', 'declarations' => [ { 'declarators' => [ { 'declarator' => 'bar[]', 'size' => 0, 'offset' => 0 } ], 'type' => 'int' } ], 'size' => 0 }; The second problem is that the above code isn't valid C, and gcc (and probably other compilers) will tell you: mhx@r2d2 ~ $ cc -c const.c const.c:4: error: variably modified 'bar' at file scope The semantics of a const variable are different in C and C++, which is why you can use a const variable for the declaration of a global array in C++, but not in C. As stated in the documentation, Convert::Binary::C only supports C, no C++. So, this is not (really) a bug. It's a bit unfortunate, though, that C::B::C parses the code without giving you an error message. But that's because it's syntactically correct, and there are only very few semantic checks in the C::B::C parser. Cheers, Marcus
Download signature.asc
application/pgp-signature 197b

Message body not shown because it is not plain text.

Hi Marcus, thank you for the quick reply. I must admit I wasn't aware of the different const handling in C/C++... But still, I think there's a bug. I tried a little further, and it seems I was just lucky that it seems to work with "direct" parsing. It does not when re-parsing the output of sourcify. The behaviour should be the same in both cases I suppose. Please try to decode something that involves such a const; as example, take a slighly modified version of your const.c: x [~/mac/ctest]> cat const.c static const int NUM = 2; struct foo { int i1; int bar[NUM]; }; and then A) decode a sample binary string from "direct" parsing: x [~/mac/ctest]> perl -MConvert::Binary::C -MData::Dumper -e'print Dumper(Convert::Binary::C->new->parse_file("const.c")->unpack("struct foo", pack("I*", 1, 2, 3 )))' $VAR1 = { 'i1' => 1, 'bar' => [ 2, 3 ] }; $VAR2 = { 'i1' => 2, 'bar' => [ 3 ] }; $VAR3 = { 'i1' => 3, 'bar' => [] }; The result as a whole is wrong, but _the first list element is exactly what you'd expect_ if C++ const handling was supported. B) decode the same string after re-parsing sourcify's output: x [~/mac/ctest]> perl -MConvert::Binary::C -MData::Dumper -e'print Dumper(Convert::Binary::C->new->parse(Convert::Binary::C->new->parse_file("const.c")->sourcify())->unpack("struct foo", pack("I*", 1, 2, 3 )))' $VAR1 = { 'i1' => 1, 'bar' => [] }; $VAR2 = { 'i1' => 2, 'bar' => [] }; $VAR3 = { 'i1' => 3, 'bar' => [] }; ...which is in line with the specified behaviour. Personally, I'd enjoy some fix that "extends" the current const handling in "A" to fully correct mode :) ...but I can fully understand that blurring up the current parser with C++ features doesn't look very attractive :-) I'll now work around my bad C++ consts by pre-processing the code and replacing all consts by preprocessor symbols. Best regards, /alex
Subject: Re: [rt.cpan.org #45105] definitions of constants are not considered by sourcify()
Date: Sat, 18 Apr 2009 14:31:38 +0200
To: bug-Convert-Binary-C [...] rt.cpan.org
From: Marcus Holland-Moritz <mhx-perl [...] gmx.net>
Hi Alexander, On 2009-04-18, at 06:33:21 -0400, Alexander Ost via RT wrote: Show quoted text
> Queue: Convert-Binary-C > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=45105 > > > Hi Marcus, > > thank you for the quick reply. I must admit I wasn't aware of the > different const handling in C/C++...
Not a problem. A lot of people aren't aware of this. Show quoted text
> But still, I think there's a bug. I tried a little further, and it seems > I was just lucky that it seems to work with "direct" parsing. It does > not when re-parsing the output of sourcify. The behaviour should be the > same in both cases I suppose.
And you're absolutely correct. Yes, there is a bug, albeit it's actually affecting a subtly different feature: flexible array members. Flexible array members have been introduced with the 1999 revision of the C standard, and allow you to declare array members with "arbitrary size" like this: struct flex { int a; int array[]; }; The semantics of this is that the flexible array member does not contribute to the actual size of a compound, and that it can grow arbitrarily large. There's a section on flexible array members in the C::B::C documentation. Due to the bug, what happens is that struct flex { int a; int array[]; }; is going to be sourcified as struct flex { int a; int array[0]; }; which isn't even valid ANSI/ISO-C. What bothers me is that there was actually a test case for this in the C::B::C test suite. However, the logic that evaluated the test result was complete crap (instead of failing with at least one failed subtest it was succeeding with at least one successful subtest). Once I fixed up the logic, tests started failing right away. The fix was pretty easy, though. Show quoted text
> Please try to decode something that involves such a const; as example, > take a slighly modified version of your const.c: > > x [~/mac/ctest]> cat const.c > static const int NUM = 2; > > struct foo { > int i1; > int bar[NUM]; > };
Actually, this is parsed as struct foo { int i1; int bar[]; }; so you're accidentially making use of the "flexible array member" feature. Show quoted text
> and then > > A) decode a sample binary string from "direct" parsing: > > x [~/mac/ctest]> perl -MConvert::Binary::C -MData::Dumper -e'print > Dumper(Convert::Binary::C->new->parse_file("const.c")->unpack("struct > foo", pack("I*", 1, 2, 3 )))' > $VAR1 = { > 'i1' => 1, > 'bar' => [ > 2, > 3 > ] > }; > $VAR2 = { > 'i1' => 2, > 'bar' => [ > 3 > ] > }; > $VAR3 = { > 'i1' => 3, > 'bar' => [] > }; > > The result as a whole is wrong, but _the first list element is exactly > what you'd expect_ if C++ const handling was supported.
No, the result is perfectly correct. At least under the assumption that the code is parsed as explained above. In list context, 'unpack' will unpack an array of types, which is why it returns three elements: mhx@r2d2 ~ $ cat const.c #include <stdio.h> struct foo { int i1; int bar[]; }; int main(void) { union { int a1[3]; struct foo a2[3]; } a; a.a1[0] = 1; a.a1[1] = 2; a.a1[2] = 3; printf("%d %d %d\n", a.a2[0].i1, a.a2[0].bar[0], a.a2[0].bar[1]); printf("%d %d\n", a.a2[1].i1, a.a2[1].bar[0]); printf("%d\n", a.a2[2].i1); return 0; } mhx@r2d2 ~ $ cc -o const const.c mhx@r2d2 ~ $ ./const 1 2 3 2 3 3 Another hint that flexible array members are being used is if you stuff in more input data: mhx@r2d2 ~ $ perl -MConvert::Binary::C -MData::Dumper -e'print Dumper(scalar Convert::Binary::C->new->parse_file("const.c")->unpack("foo", pack "I*", 1 .. 5))' $VAR1 = { 'i1' => 1, 'bar' => [ 2, 3, 4, 5 ] }; This also shows the effect of scalar context on the "unpack" method. Show quoted text
> B) decode the same string after re-parsing sourcify's output: > > x [~/mac/ctest]> perl -MConvert::Binary::C -MData::Dumper -e'print > Dumper(Convert::Binary::C->new->parse(Convert::Binary::C->new->parse_file("const.c")->sourcify())->unpack("struct > foo", pack("I*", 1, 2, 3 )))' > $VAR1 = { > 'i1' => 1, > 'bar' => [] > }; > $VAR2 = { > 'i1' => 2, > 'bar' => [] > }; > $VAR3 = { > 'i1' => 3, > 'bar' => [] > }; > > ...which is in line with the specified behaviour.
More or less. As I said, this isn't really valid C, but it's easier not to special-case arrays with no elements. Show quoted text
> Personally, I'd enjoy some fix that "extends" the current const handling > in "A" to fully correct mode :) ...but I can fully understand that > blurring up the current parser with C++ features doesn't look very > attractive :-)
The parser doesn't even see your constant, and "A" is fully correct (and probably what you expect if you just change it to scalar context). I've uploaded Convert::Binary::C 0.74 this morning which fixes the bug in "sourcify". You should now get "A" behaviour also after parsing code that has been sourcified. Show quoted text
> I'll now work around my bad C++ consts by pre-processing the code and > replacing all consts by preprocessor symbols.
Don't get me wrong, C++ constants aren't bad. It's just that they're not compatible with C. So, thanks for finding a bug that survived for more than five years! Cheers, Marcus -- Isn't it strange that the same people that laugh at gypsy fortune tellers take economists seriously?
Download signature.asc
application/pgp-signature 197b

Message body not shown because it is not plain text.

Hello, On Sat Apr 18 08:32:09 2009, mhx-perl@gmx.net wrote: Show quoted text
> [...] > I've uploaded Convert::Binary::C 0.74 this morning which fixes the > bug in "sourcify". You should now get "A" behaviour also after parsing > code that has been sourcified. > [...] > So, thanks for finding a bug that survived for more than five years!
Many thanks for your quick help and for explaining all the details :-) I'll try 0.74 as soon as possible. Cheers, /alex