Skip Menu |

This queue is for tickets about the Image-MetaData-JPEG CPAN distribution.

Report information
The Basics
Id: 17835
Status: resolved
Priority: 0/
Queue: Image-MetaData-JPEG

People
Owner: Nobody in particular
Requestors: brian [...] photoresearchers.com
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 0.15
Fixed in: 0.153



Subject: subroutine 'save' should check for undefined filename before calling 'open'
make test reports: t/JPEG_5_exif_Thumbnail....# Testing APP1 Exif data routines (thumbnail) t/JPEG_5_exif_Thumbnail....ok 16/26Use of uninitialized value in open at /path/to/ Image-MetaData-JPEG-0.15/blib/lib/Image/MetaData/JPEG.pm line 134. patch is attached.
Subject: Image-MetaData-JPEG-0.15-fix_save_open.patch
--- lib/Image/MetaData/JPEG.pm 2006-01-05 18:58:00.000000000 -0500 +++ lib/Image/MetaData/JPEG.pm 2006-02-23 13:02:32.000000000 -0500 @@ -124,7 +124,7 @@ # fail immediately if "read_only" is set return undef if $this->{read_only}; # if $filename is undefined, it defaults to the original name - $filename = $this->{filename} unless defined $filename; + return undef unless defined ($filename ||= $this->{filename}); # Open an IO handler for output on a file named $filename # or on an in-memory variable pointed to by $filename. # Use an indirect handler, which is closed authomatically
From: brian [...] photoresearchers.com
Wow. Disregard that last patch, as it turns out this was a bigger problem than I realized. The 'filename' argument being passed to the 'save' function is a REFERENCE, but it's being treated like a scalar value. I don't know if this was the intent or not, but going with the pass-by-reference design I further patched the routine to expect and verify the 'filename' variable as a reference. Running with the new patch, with which I'd like to assume the 'save' method is working the way it was intended, causes 'make test' to fail 10 of 17 test scripts, reporting as follows: PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/JPEG_0_records...........# Testing [Image::MetaData::JPEG::Record] t/JPEG_0_records...........ok t/JPEG_1_segments..........# Testing [Image::MetaData::JPEG::Segment] t/JPEG_1_segments..........ok t/JPEG_2_JPEG_class........# Testing [Image::MetaData::JPEG] t/JPEG_2_JPEG_class........ok 23/60# Failed test (t/JPEG_2_JPEG_class.t at line 119) t/JPEG_2_JPEG_class........ok 60/60# Looks like you failed 1 tests of 60. t/JPEG_2_JPEG_class........dubious Test returned status 1 (wstat 256, 0x100) DIED. FAILED test 24 Failed 1/60 tests, 98.33% okay t/JPEG_2_rare..............# Testing JPEG segments seldom used methods t/JPEG_2_rare..............ok t/JPEG_3_comments..........# Testing comment routines t/JPEG_3_comments..........ok 21/24Can't call method "get_comments" on an undefined value at t/JPEG_3_comments.t line 125. # Looks like you planned 24 tests but only ran 21. # Looks like your test died just after 21. t/JPEG_3_comments..........dubious Test returned status 255 (wstat 65280, 0xff00) DIED. FAILED tests 22-24 Failed 3/24 tests, 87.50% okay t/JPEG_4_app13.............# Testing APP13 IPTC basic routines t/JPEG_4_app13.............ok t/JPEG_4_app13_IPTC........# Testing APP13 IPTC format checker t/JPEG_4_app13_IPTC........ok t/JPEG_4_app13_set.........# Testing APP13 IPTC set routines t/JPEG_4_app13_set.........ok 17/53# Failed test (t/JPEG_4_app13_set.t at line 141) t/JPEG_4_app13_set.........NOK 18# 'ne' # # undef Can't call method "get_app13_data" on an undefined value at t/JPEG_4_app13_set.t line 144. # Looks like you planned 53 tests but only ran 18. # Looks like your test died just after 18. t/JPEG_4_app13_set.........dubious Test returned status 255 (wstat 65280, 0xff00) DIED. FAILED tests 18-53 Failed 36/53 tests, 32.08% okay t/JPEG_5_exif..............# Testing APP1 Exif data routines t/JPEG_5_exif..............ok t/JPEG_5_exif_GPS..........# Testing APP1 Exif data routines (GPS_DATA) t/JPEG_5_exif_GPS..........ok 11/47# Failed test (t/JPEG_5_exif_GPS.t at line 101) # got: undef # expected: 'ARRAY(0x83b5f74)' t/JPEG_5_exif_GPS..........NOK 12Can't call method "get_description" on unblessed reference at t/JPEG_5_exif_GPS.t line 105. # Looks like you planned 47 tests but only ran 12. # Looks like your test died just after 12. t/JPEG_5_exif_GPS..........dubious Test returned status 255 (wstat 65280, 0xff00) DIED. FAILED tests 12-47 Failed 36/47 tests, 23.40% okay t/JPEG_5_exif_IFD..........# Testing APP1 Exif data routines (IFD01_DATA) t/JPEG_5_exif_IFD..........ok 30/59# Failed test (t/JPEG_5_exif_IFD.t at line 186) # got: undef # expected: 'ARRAY(0x836cde4)' t/JPEG_5_exif_IFD..........NOK 31Can't call method "get_description" on unblessed reference at t/JPEG_5_exif_IFD.t line 190. # Looks like you planned 59 tests but only ran 31. # Looks like your test died just after 31. t/JPEG_5_exif_IFD..........dubious Test returned status 255 (wstat 65280, 0xff00) DIED. FAILED tests 31-59 Failed 29/59 tests, 50.85% okay t/JPEG_5_exif_IMAGE........# Testing APP1 Exif data routines (IMAGE_DATA & ROOT_DATA) t/JPEG_5_exif_IMAGE........ok 29/52# Failed test (t/JPEG_5_exif_IMAGE.t at line 189) # got: undef # expected: 'ARRAY(0x8064588)' t/JPEG_5_exif_IMAGE........NOK 30Can't call method "get_description" on unblessed reference at t/JPEG_5_exif_IMAGE.t line 193. # Looks like you planned 52 tests but only ran 30. # Looks like your test died just after 30. t/JPEG_5_exif_IMAGE........dubious Test returned status 255 (wstat 65280, 0xff00) DIED. FAILED tests 30-52 Failed 23/52 tests, 55.77% okay t/JPEG_5_exif_Interop......# Testing APP1 Exif data routines (INTEROP_DATA) t/JPEG_5_exif_Interop......ok 13/29# Failed test (t/JPEG_5_exif_Interop.t at line 90) # got: undef # expected: 'ARRAY(0x83aa6c8)' t/JPEG_5_exif_Interop......NOK 14Can't call method "get_description" on unblessed reference at t/JPEG_5_exif_Interop.t line 94. # Looks like you planned 29 tests but only ran 14. # Looks like your test died just after 14. t/JPEG_5_exif_Interop......dubious Test returned status 255 (wstat 65280, 0xff00) DIED. FAILED tests 14-29 Failed 16/29 tests, 44.83% okay t/JPEG_5_exif_Makernote....# Testing APP1 MakerNote parse / dump t/JPEG_5_exif_Makernote....ok 7/44Can't call method "get_description" on an undefined value at t/JPEG_5_exif_Makernote.t line 85. # Looks like you planned 44 tests but only ran 7. # Looks like your test died just after 7. Unbalanced string table refcount: (1) for "Pentax_Optio430.jpg" during global destruction. t/JPEG_5_exif_Makernote....dubious Test returned status 255 (wstat 65280, 0xff00) DIED. FAILED tests 8-44 Failed 37/44 tests, 15.91% okay t/JPEG_5_exif_SubIFD.......# Testing APP1 Exif data routines (SUBIFD_DATA) t/JPEG_5_exif_SubIFD.......ok 15/50# Failed test (t/JPEG_5_exif_SubIFD.t at line 125) # got: undef # expected: 'ARRAY(0x804ca28)' t/JPEG_5_exif_SubIFD.......NOK 16Can't call method "get_description" on unblessed reference at t/JPEG_5_exif_SubIFD.t line 129. # Looks like you planned 50 tests but only ran 16. # Looks like your test died just after 16. t/JPEG_5_exif_SubIFD.......dubious Test returned status 255 (wstat 65280, 0xff00) DIED. FAILED tests 16-50 Failed 35/50 tests, 30.00% okay t/JPEG_5_exif_Thumbnail....# Testing APP1 Exif data routines (thumbnail) t/JPEG_5_exif_Thumbnail....ok 7/26Can't call method "get_Exif_data" on an undefined value at t/JPEG_5_exif_Thumbnail.t line 54. # Looks like you planned 26 tests but only ran 7. # Looks like your test died just after 7. t/JPEG_5_exif_Thumbnail....dubious Test returned status 255 (wstat 65280, 0xff00) DIED. FAILED tests 8-26 Failed 19/26 tests, 26.92% okay t/JPEG_p_podchecker........# Checking documentation syntax with Pod::Checker lib/Image/MetaData/JPEG.pod pod syntax OK. t/JPEG_p_podchecker........ok 1/4lib/Image/MetaData/JPEG/Structures.pod pod syntax OK. t/JPEG_p_podchecker........ok 2/4lib/Image/MetaData/JPEG/TagLists.pod pod syntax OK. t/JPEG_p_podchecker........ok 3/4lib/Image/MetaData/JPEG/MakerNotes.pod pod syntax OK. t/JPEG_p_podchecker........ok Failed Test Stat Wstat Total Fail Failed List of Failed -------------------------------------------------------------------------- ----- t/JPEG_2_JPEG_class.t 1 256 60 1 1.67% 24 t/JPEG_3_comments.t 255 65280 24 3 12.50% 22-24 t/JPEG_4_app13_set.t 255 65280 53 36 67.92% 18-53 t/JPEG_5_exif_GPS.t 255 65280 47 36 76.60% 12-47 t/JPEG_5_exif_IFD.t 255 65280 59 29 49.15% 31-59 t/JPEG_5_exif_IMAGE.t 255 65280 52 23 44.23% 30-52 t/JPEG_5_exif_Interop.t 255 65280 29 16 55.17% 14-29 t/JPEG_5_exif_Makernote.t 255 65280 44 37 84.09% 8-44 t/JPEG_5_exif_SubIFD.t 255 65280 50 35 70.00% 16-50 t/JPEG_5_exif_Thumbnail.t 255 65280 26 19 73.08% 8-26 Failed 10/17 test scripts, 41.18% okay. 235/817 subtests failed, 71.24% okay. make: *** [test_dynamic] Error 29
--- JPEG.pm.ORIG 2006-01-05 18:58:00.000000000 -0500 +++ JPEG.pm 2006-02-23 13:51:09.000000000 -0500 @@ -124,14 +124,16 @@ # fail immediately if "read_only" is set return undef if $this->{read_only}; # if $filename is undefined, it defaults to the original name - $filename = $this->{filename} unless defined $filename; + $filename = \$this->{filename} unless defined $filename; + return undef unless ref($filename) eq 'SCALAR' && defined ($$filename); # Open an IO handler for output on a file named $filename # or on an in-memory variable pointed to by $filename. # Use an indirect handler, which is closed authomatically # when it goes out of scope (so, no need to call close()). # If open fails, it returns false and sets the special # variable $! to reflect the system error. - open(my $out, '>', $filename) || return undef; + my $out; + open($out, '>', $$filename) || return undef; # Legacy systems might need an explicit binary open. binmode($out); # For each segment in the segment list, write the content of
From: brian [...] photoresearchers.com
To follow-up: it's not clear what the tests are doing calling the 'save' method with a reference (particularly when that reference is to a unmodifiable constant), and furthermore calling the 'new' method on the *class* with a reference to a constant. For example, in t/JPEG_3_comments.t: $image->remove_all_comments(); $image->add_comment($_) for @savelist; $num = @savelist; $bufferref = \ "dummy"; $image->save($bufferref); $newim = $cname->new($bufferref); @list = $newim->get_comments(); is_deeply( \@list, \@savelist, "Save and re-read" ); What is this code trying to accomplish? I'm having trouble figuring it out. The variable '$bufferref" here is a constant value. In order for the method call to 'new' to return an in-memory object with the comments embedded (in order to compare them to the original list?), then the call to the 'save' method would either need to accept a 'Image::MetaData::JPEG' object instead of a scalar filename or else it would need to create one with its own call to 'new', unless the author's intent is for a new file to be created with the name "dummy" (which I assume is not the case, else the variable would likely have been named '$ref' instead of '$bufferref' which seems to suggest an in- memory buffer). The 'save' method expects a single optional argument representing a new filename (otherwise it defaults to the original). Even if it were modified to accept an object reference instead, this test would still fail to utilize it because it's passing '$bufferref' by *value* instead of by reference, which makes it unmodifiable. Since the method call to 'save' is being passed a reference to a string constant (but not a reference to anything ti can modify) and its return value is being ignored, I don't see how this method call accomplishes anything useful.
From: Stefano Bettelli
I had a look at your reported error during self-tests, and this is my analysis. An error occurs during test number #17 in t/JPEG_5_exif_Thumbnail: Show quoted text
>> t/JPEG_5_exif_Thumbnail....# Testing APP1 Exif data routines (thumbnail) >> t/JPEG_5_exif_Thumbnail....ok 16/26Use of uninitialized value in open
at /path/to/ Show quoted text
>> Image-MetaData-JPEG-0.15/blib/lib/Image/MetaData/JPEG.pm line 134.
This test consists in a single line of code: $image->set_Exif_data($thumb, 'THUMBNAIL'); where $thumb is a valid Image::MetaData::JPEG object reference, previously created with: $thumb = $classname->new($dataref); What happens is that $thumb is passed to an internal routine, set_Exif_thumbnail(), which can accept various types of input, and whose goal is to produce and appro- priately store a JPEG stream (a sequence of bytes). In this case, therefore, the validity of the stream is already guaranteed, and the routine only needs to extract the image in a Perl scalar; this is done by if ('Image::MetaData::JPEG' eq ref $dataref) { my $r; $dataref->save(\ $r); $dataref = \ $r; } This is not very important, it just shows who is calling save(), which contains line 134 of JPEG.pm. Anyway, save() is invoked with a reference to a scalar as argument: the effect should be that the JPEG stream is written into the scalar ($r). For some reason, this works for me but not for you ... Now, you argue: Show quoted text
>> Disregard that last patch, as it turns out this was >> a bigger problem than I realized. The 'filename' argument >> being passed to the 'save' function is a REFERENCE, >> but it's being treated like a scalar value. [...]
This is a misunderstanding; save() can accept either a scalar or a reference to a scalar. If it is a plain scalar it must contain a file name (and I agree that I have to check that it is not the empty string, which I will do in my next release); if it is a reference to a scalar, the corresponding scalar is used as "in-memory file". This is documented by 'perldoc -f open': Since v5.8.0, perl has built using PerlIO by default. Unless you've changed this (i.e. Configure -Uuseperlio), you can open file handles to "in memory" files held in Perl scalars via: open($fh, '>', \$variable) || .. This leads me to suppose that you are using an older version of Perl (and that I should include an appropriate "use VERSION" in my code), or that your version is not standard configured (and I should check for this capability to be present, which I don't know how to do presently). Show quoted text
>> I don't know if this was the intent or not, but going >> with the pass-by-reference design I further patched the >> routine to expect and verify the 'filename' variable as >> a reference. [...] >> >> - $filename = $this->{filename} unless defined $filename; >> + $filename = \$this->{filename} unless defined $filename; >> + return undef unless ref($filename) eq 'SCALAR' && defined ($$filename);
As I said, this is not true. This variable has not a fixed type: it can be a scalar or a reference to a scalar. If it is undefined it must be replaced with a scalar, not with a reference. This is the reason of all subsequent errors. Show quoted text
>> To follow-up: it's not clear what the tests are doing calling >> the 'save' method with a reference (particularly when that >> reference is to a unmodifiable constant), and furthermore calling >> the 'new' method on the *class* with a reference to a constant. >> For example, in t/JPEG_3_comments.t: >> [...] >> $bufferref = \ "dummy"; >> $image->save($bufferref); >> $newim = $cname->new($bufferref); >> [...] >> What is this code trying to accomplish? I'm having trouble >> figuring it out. The variable '$bufferref" here is a constant >> value. [...]
Well, I agree that this code (a leftover) is a bit obscure, and that the first line should be replaced by: my $buffer; $bufferref = \ $buffer; However, I disagree with your idea that $bufferref is a constant. It is a Perl variable holding the value of a reference; as such, it is surely variable. Even $bufferref is not a constant, it is just a scalar whose initial value (which is disregarded in the call to open()) is "dummy". In conclusion, if I take into account also the other errors which you showed me in a private mail, I believe that your Perl installation is either non-standard, or buggy or outdated (maybe it is pre 5.8 ?). Could you give me more details on your setup? Do you have the possibility to run the self-tests on a different machine? By the way, I develop on a Debian system with Perl 5.8.8. You can see other setups, where all self-tests where passed, in the following web page: http://cpantesters.perl.org/show/Image-MetaData-JPEG.html#Image-MetaData-JPEG-0.15 Best regards, Stefano Bettelli
From: brian [...] photoresearchers.com
On Sat Feb 25 19:25:23 2006, BETTELLI wrote: Show quoted text
> What happens is that $thumb is passed to an internal > routine, set_Exif_thumbnail(), which can accept various > types of input, and whose goal is to produce and appro- > priately store a JPEG stream (a sequence of bytes).
Aha! Yes! I think I found the problem then. See attached patch. Show quoted text
> In this case, therefore, the validity of the stream is > already guaranteed, and the routine only needs to extract > the image in a Perl scalar; this is done by > > if ('Image::MetaData::JPEG' eq ref $dataref) { > my $r; $dataref->save(\ $r); $dataref = \ $r; } > > This is not very important, it just shows who is > calling save(), which contains line 134 of JPEG.pm. > Anyway, save() is invoked with a reference to a scalar > as argument: the effect should be that the JPEG stream > is written into the scalar ($r). For some reason, this > works for me but not for you.
I don't know why it works for you. The reason is that save() contains the following line: $filename = $this->{filename} unless defined $filename; In this case, you've passed a reference to an undefined variable ($r, which you declare but do not define), so the save() method ignores the destination you've provided and defaults to the original filename, which I imagine (although I haven't investigated this completely) is also undefined in this instance. Regardless this seems incorrect, given what you've described it's trying to do, so the provided patch fixes the above code in set_Exif_thumbnail() to define $r to the empty string so that it can be written. Show quoted text
> This leads me to suppose that you are using an older > version of Perl (and that I should include an appropriate > "use VERSION" in my code), or that your version is not > standard configured (and I should check for this capability > to be present, which I don't know how to do presently).
FYI my perl is fine (v5.8.0 built for i386-linux-thread-multi), and that use of open() (about which I'd forgotten and you were right to correct me) works fine when tested. I was a bit too hasty in diagnosing the problem, which is why all previous patches should be disregarded as they only mess things up further.
--- lib/Image/MetaData/JPEG/JPEG_app1_exif.pl 2006-01-06 17:20:43.000000000 -0500 +++ lib/Image/MetaData/JPEG/JPEG_app1_exif.pl 2006-02-27 13:45:01.000000000 -0500 @@ -468,14 +468,15 @@ # if $dataref points to an Image::MetaData::JPEG object, replace it # with a reference to its bare content and set $type to 'JPEG'. if ('Image::MetaData::JPEG' eq ref $dataref) { - my $r; $dataref->save(\ $r); $dataref = \ $r; $type = 'JPEG'; } + my $r = ""; $dataref->save(\ $r); $dataref = \ $r; $type = 'JPEG'; } # $dataref must now be a scalar reference; everything else is an error return { 'ERROR' => 'not a good reference' } if ref $dataref ne 'SCALAR'; # try to recognise the content of $$dataref. If it is defined but empty, # we just need to erase the thumbnail. If it is accepted by the JPEG # ctor or $type is already 'JPEG', we consider it a regular JPEG stream. $type = 'NONE' if length $$dataref == 0; - $type = 'JPEG' if ! $type && Image::MetaData::JPEG->new($dataref, ''); + $type = 'JPEG' if ! $type && + Image::MetaData::JPEG->new($dataref, \(my $ns = '')); # If $type is not yet set, generate an error (TIFF not yet supported ...) return { 'Error' => 'unsupported thumbnail format' } unless $type; # the following lists contain all records to be erased before inserting
Subject: subroutine 'set_Exif_thumbnail' needs to define variables before passing references to them
From: brian [...] photoresearchers.com
I should also point out that after patching set_Exif_thumbnail() as described above, those warnings no longer appear and all tests pass. (I am still seeing "Unbalanced string table refcount" warnings, but if I have time I'll investigate those separately.)
Subject: Re: subroutine 'set_Exif_thumbnail' needs to define variables before passing references to them
From: brian [...] photoresearchers.com
On Mon Feb 27 13:59:09 2006, guest wrote: Show quoted text
> I don't know why it works for you. The reason is that save() contains > the following line: > > $filename = $this->{filename} unless defined $filename; > > In this case, you've passed a reference to an undefined variable > ($r, which you declare but do not define), so the save() method > ignores the destination you've provided and defaults to the original > filename <snip>
On second thought, this isn't entirely correct either. The variable $filename isn't undefined -- it's a reference with a defined value -- but rather the scalar that it points to is. The warnings I was reporting were "Use of uninitialized value in open"; note it's saying "uninitialized" and not "undefined". So clearly open()'s problem is not the reference but the memory that it points to never being initialized. However, altering the code by explicitly setting $r equal to undef when it's declared doesn't seem to work in avoiding the errors; only a defined value like "" appears to do the trick. Also, one other clarification: Show quoted text
> However, I disagree with your idea that $bufferref is a constant. > It is a Perl variable holding the value of a reference; as such, > it is surely variable. Even $bufferref is not a constant, it is > just a scalar whose initial value (which is disregarded in the > call to open()) is "dummy".
I think I mistyped the second time around, but I think I was accurate in my initial description when I wrote: Show quoted text
>>To follow-up: it's not clear what the tests are doing calling >> the 'save' method with a reference (particularly when that >> reference is to a unmodifiable constant)
$bufferref, as you said, should be defined to point to a variable, not a constant (e.g. \$buffer instead of \"dummy").
Subject: conclusion
On Lun. 27 Feb. 2006 17:57:32, guest wrote: Show quoted text
> The variable $filename isn't undefined -- it's a reference with a > defined value -- but rather the scalar that it points to is. The > warnings I was reporting were "Use of uninitialized value in open"; > note it's saying "uninitialized" and not "undefined". So clearly > open()'s problem is not the reference but the memory that it points > to never being initialized.
So, it seems the problem was found! What happens is that some versions of the Perl interpreter do not work correctly when the referenced variable is not initialised. I believe this happens because open() accesses some internal representation, without passing through the "user" layer which would autovivify it. Have a look, for instance, in the newsgroup "perl.perl5.porters" at the thread titled "In-memory read from undefined variable can return junk". This could explain why it works here but not at your site. So, I will explicitely initialise $r in set_Exif_thumbnail(), and scan through all tests to convert them along the same lines. This is the only way to guarantee a common behaviour on all interpreters (from 5.8 on), at least, I hope. However, I need some time to check that all the bits and pieces are working correctly, and I am overworked right now. I hope I will be able to send out a bug-fix release in March. Thank you very much for your collaboration! Stefano
On Lun. 27 Feb. 2006 14:04:47, guest wrote: Show quoted text
> I should also point out that after patching set_Exif_thumbnail() as > described above, those warnings no longer appear and all tests pass.
Good! Show quoted text
> (I am still seeing "Unbalanced string table refcount" warnings, but > if I have time I'll investigate those separately.)
Thanks, I can't do much on it here, since I do not see the warning with my interpreter. Be sure to open a new bug report if you find anything meaningful.