Bug #92785 for Imager: evidently a bug in either Imager (0.98) or PerlIO::via(::CBC)

Thu Feb 06 11:10:21 2014 tchatzi [...] arx.net - Ticket created

Subject:	evidently a bug in either Imager (0.98) or PerlIO::via(::CBC)
Date:	Thu, 06 Feb 2014 18:09:55 +0200
To:	bug-Imager [...] rt.cpan.org
From:	Thanos Chatziathanassiou <tchatzi [...] arx.net>

Hey there, I've encountered an issue that (unfortunately) do not have the time to properly trace: Imager dies rather violently when calling $imager->read() after using a PerlIO::via::CBC, regardless of method of reading. Some sample code to illustrate: ---8<--- #!/usr/bin/perl -w use strict; use Imager; use PerlIO::via::CBC; my $src = "/path/to/cbc-encrypted/file.jpg"; PerlIO::via::CBC->config( 'key' => "Some passphrase", 'cipher' => "Crypt::Rijndael", 'salt' => 1, 'header' => 'salt' ); my $img = Imager->new; open(my $fh, '<:via(PerlIO::via::CBC)', $src); binmode($fh); #CASE 1: pass the filehandle directly to Imager $img->read(fh => $fh, type => 'jpeg') or die "Cannot read: ", $img->errstr; #this dies with "Cannot read: Not a JPEG file: starts with random_bytes" #(eg 0x53 0x61) #note that these bytes are nowhere to be found in either the original #or the decrypted file. #this also happens with $img->read(fd => fileno($fh), type => 'jpeg') or die "Cannot read: ", $img->errstr; #CASE 2: read into scalar and pass a) by reference or b) by value my $size= (-s $src); my $offset = 0; my $imgdata; while ($size > 32) { #we can leave out a whole padding block in the end my $read = read($fh,$imgdata,262144,$offset); #warn "read $read data, $size to go\n"; $size -= $read; $offset += $read; } close($fh); $img->read(data => \$imgdata, type => 'jpeg') or die "Cannot read: ", $img->errstr; #this dies horribly with "panic: sv_setpvn called with negative strlen # at /usr/local/lib64/perl5/Imager/File/JPEG.pm line 44." $img->read(data => $imgdata, type => 'jpeg') or die "Cannot read: ", $img->errstr; #this works. ---8<--- All this on an up-to-date Centos 6.5 x86_64 system (with default perl 5.10.1) and Imager-0.98 (built from source). If I there's any more info I failed to provide, please let me know. Best Regards, Thanos Chatziathanassiou

Download smime.p7s
application/pkcs7-signature 4.4k

Message body not shown because it is not plain text.

Thu Feb 06 17:09:21 2014 tony [...] develop-help.com - Correspondence added

Subject:	Re: [rt.cpan.org #92785] evidently a bug in either Imager (0.98) or PerlIO::via(::CBC)
Date:	Fri, 7 Feb 2014 09:09:09 +1100
To:	Thanos Chatziathanassiou via RT <bug-Imager [...] rt.cpan.org>
From:	Tony Cook <tony [...] develop-help.com>

On Thu, Feb 06, 2014 at 11:10:21AM -0500, Thanos Chatziathanassiou via RT wrote: Show quoted text

> Hey there, I've encountered an issue that (unfortunately) do not have > the time to properly trace: Imager dies rather violently when calling > $imager->read() after using a PerlIO::via::CBC, regardless of method of > reading. > Some sample code to illustrate: > > ---8<--- > #!/usr/bin/perl -w > use strict; > use Imager; > use PerlIO::via::CBC; > > my $src = "/path/to/cbc-encrypted/file.jpg"; > > PerlIO::via::CBC->config( > 'key' => "Some passphrase", > 'cipher' => "Crypt::Rijndael", > 'salt' => 1, > 'header' => 'salt' > ); > > my $img = Imager->new; > > open(my $fh, '<:via(PerlIO::via::CBC)', $src); > binmode($fh);

The binmode strips the via layer. If you're on Linux, you don't need it at all, if you're on Win32 you need to strip the default :crlf layer (or :utf8 layer if you have some a UTF-8 locale and PERL_UNICODE set). Depending on the version of perl (I'm not sure when it was fixed): # recent versions open my $fh, '<:raw:via(PerlIO::via::CBC)', $src); # older versions I think including 5.10, assuming I remember the issue # correctly open my $fh, '<', $src); binmode $fh; binmode $fh, ':via(PerlIO::via::CBC)'; Show quoted text

> #CASE 1: pass the filehandle directly to Imager > $img->read(fh => $fh, type => 'jpeg') or die "Cannot read: ", $img->errstr; > #this dies with "Cannot read: Not a JPEG file: starts with random_bytes" > #(eg 0x53 0x61) > #note that these bytes are nowhere to be found in either the original > #or the decrypted file.

This one should work, with the removal of binmode, or the alternative open + binmode. Show quoted text

> #this also happens with > $img->read(fd => fileno($fh), type => 'jpeg') or die "Cannot read: ", > $img->errstr;

This one can't work, since Imager would be reading directly from the file handle and bypassing the I/O layers. Show quoted text

> > #CASE 2: read into scalar and pass a) by reference or b) by value > my $size= (-s $src); > my $offset = 0; > my $imgdata; > while ($size > 32) { #we can leave out a whole padding block in the end > my $read = read($fh,$imgdata,262144,$offset); > #warn "read $read data, $size to go\n"; > $size -= $read; > $offset += $read; > } > close($fh); > > $img->read(data => \$imgdata, type => 'jpeg') or die "Cannot read: ", > $img->errstr; > #this dies horribly with "panic: sv_setpvn called with negative strlen # > at /usr/local/lib64/perl5/Imager/File/JPEG.pm line 44."

I'll look at this, I know there have been a few bugs in perl that have caused this panic, but it's more likely to be Imager. Show quoted text

> > $img->read(data => $imgdata, type => 'jpeg') or die "Cannot read: ", > $img->errstr; > #this works.

This second one is the documented mechanism for reading from a scalar - but I'm surprised it worked with your binmode above. Show quoted text

> All this on an up-to-date Centos 6.5 x86_64 system (with default perl > 5.10.1) and Imager-0.98 (built from source). > If I there's any more info I failed to provide, please let me know.

Can you provide a sample file with key? Thanks, Tony

Thu Feb 06 17:09:22 2014 The RT System itself - Status changed from 'new' to 'open'

Fri Feb 07 08:06:31 2014 tchatzi [...] arx.net - Correspondence added

Subject:	Re: [rt.cpan.org #92785] evidently a bug in either Imager (0.98) or PerlIO::via(::CBC)
Date:	Fri, 07 Feb 2014 15:06:11 +0200
To:	bug-Imager [...] rt.cpan.org
From:	Thanos Chatziathanassiou <tchatzi [...] arx.net>

Show quoted text

>> open(my $fh, '<:via(PerlIO::via::CBC)', $src); >> binmode($fh);

> > The binmode strips the via layer. If you're on Linux, you don't need > it at all, if you're on Win32 you need to strip the default :crlf > layer (or :utf8 layer if you have some a UTF-8 locale and PERL_UNICODE > set). > > Depending on the version of perl (I'm not sure when it was fixed): > > # recent versions > open my $fh, '<:raw:via(PerlIO::via::CBC)', $src);

This seems to do the trick - specifically the ``:raw:'' part. Show quoted text

> open my $fh, '<', $src); > binmode $fh; > binmode $fh, ':via(PerlIO::via::CBC)';

any variation of binmode doesn't make any difference whatsoever and i'd be honestly surprised if it did for any OS that doesn't care about ``text'' vs ``binary'' files. Show quoted text

>> #CASE 1: pass the filehandle directly to Imager >> $img->read(fh => $fh, type => 'jpeg') or die "Cannot read: ", $img->errstr; >> #this dies with "Cannot read: Not a JPEG file: starts with random_bytes" >> #(eg 0x53 0x61) >> #note that these bytes are nowhere to be found in either the original >> #or the decrypted file.

didn't even need to specify type for this to work. Show quoted text

>> #this also happens with >> $img->read(fd => fileno($fh), type => 'jpeg') or die "Cannot read: ", >> $img->errstr;

> > This one can't work, since Imager would be reading directly from the > file handle and bypassing the I/O layers.

You're right, this doesn't work. Show quoted text

>> >> $img->read(data => \$imgdata, type => 'jpeg') or die "Cannot read: ", >> $img->errstr;

this now fails with the (still inexplicable) ``Not a JPEG file: starts with 0xf6 0x80'' or some other apparently random byte combination. Show quoted text

>> $img->read(data => $imgdata, type => 'jpeg') or die "Cannot read: ", >> $img->errstr; >> #this works.

this continues working as previously. Show quoted text

> Can you provide a sample file with key? >

Attached both encrypted and original jpgs. The pass phrase is ``8badf00d8badf00d8badf00d8badf00d'' In all, I'd say the ``:raw'' open option did the trick - obviously the default is some cooked i/o mode that messes things up for Imager and/or PerlIO::via::CBC.

Download encrypted.jpg
image/jpeg 256.3k

Message body is not shown because sender requested not to inline it.

Download unknown.jpg
image/jpeg 256.2k

Message body is not shown because sender requested not to inline it.

Download smime.p7s
application/pkcs7-signature 4.4k

Message body not shown because it is not plain text.

Sun Feb 09 20:04:05 2014 tony [...] develop-help.com - Correspondence added

Subject:	Re: [rt.cpan.org #92785] evidently a bug in either Imager (0.98) or PerlIO::via(::CBC)
Date:	Mon, 10 Feb 2014 12:03:53 +1100
To:	Thanos Chatziathanassiou via RT <bug-Imager [...] rt.cpan.org>
From:	Tony Cook <tony [...] develop-help.com>

On Fri, Feb 07, 2014 at 08:06:31AM -0500, Thanos Chatziathanassiou via RT wrote: Show quoted text

> Queue: Imager > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=92785 > >

> >> open(my $fh, '<:via(PerlIO::via::CBC)', $src); > >> binmode($fh);

> > > > The binmode strips the via layer. If you're on Linux, you don't need > > it at all, if you're on Win32 you need to strip the default :crlf > > layer (or :utf8 layer if you have some a UTF-8 locale and PERL_UNICODE > > set). > > > > Depending on the version of perl (I'm not sure when it was fixed): > > > > # recent versions > > open my $fh, '<:raw:via(PerlIO::via::CBC)', $src);

> > This seems to do the trick - specifically the ``:raw:'' part. >

> > open my $fh, '<', $src); > > binmode $fh; > > binmode $fh, ':via(PerlIO::via::CBC)';

> > any variation of binmode doesn't make any difference whatsoever and i'd > be honestly surprised if it did for any OS that doesn't care about > ``text'' vs ``binary'' files.

Some Redhat distributions have configured perl to read/write UTF-8 by default, it's simple to do: export PERL_UNICODE= LC_ALL=en_US.UTF-8 Show quoted text

> >> #CASE 1: pass the filehandle directly to Imager > >> $img->read(fh => $fh, type => 'jpeg') or die "Cannot read: ", $img->errstr; > >> #this dies with "Cannot read: Not a JPEG file: starts with random_bytes" > >> #(eg 0x53 0x61) > >> #note that these bytes are nowhere to be found in either the original > >> #or the decrypted file.

> > didn't even need to specify type for this to work. >

> >> $img->read(data => \$imgdata, type => 'jpeg') or die "Cannot read: ", > >> $img->errstr;

> > this now fails with the (still inexplicable) ``Not a JPEG file: starts > with 0xf6 0x80'' or some other apparently random byte combination.

I'll look into this. I'd expect it to fail, but the bytes shouldn't be random, but something like "SCALAR(0x26052f8)". I might make it do the sensible thing with scalar refs. Show quoted text

> > Can you provide a sample file with key? > >

> > Attached both encrypted and original jpgs. The pass phrase is > ``8badf00d8badf00d8badf00d8badf00d'' > > In all, I'd say the ``:raw'' open option did the trick - obviously the > default is some cooked i/o mode that messes things up for Imager and/or > PerlIO::via::CBC.

You might want to check your environment for PERL_UNICODE, with Imager 0.98, passing a file handle uses the handle as is, whatever layers happen to be on it (otherwise your :via(...) would get stripped.) Tony

Mon Mar 31 06:11:50 2014 TONYC [...] cpan.org - Status changed from 'open' to 'patched'

Mon Aug 25 01:25:45 2014 TONYC [...] cpan.org - Correspondence added

On Thu Feb 06 11:10:21 2014, tchatzi@arx.net wrote: Show quoted text

> Hey there, I've encountered an issue that (unfortunately) do not have > the time to properly trace: Imager dies rather violently when calling > $imager->read() after using a PerlIO::via::CBC, regardless of method of > reading.

I've made some changes which I think will fix this: - Imager::IO->new_buffer() now makes a copy of the string, rather than keeping a copy of a possible temp - Imager::IO->new_buffer() now checks if the supplied buffer is a scalar reference and if so, derefs it. Tony

Mon Aug 25 01:25:46 2014 TONYC [...] cpan.org - Status changed from 'patched' to 'resolved'