Skip Menu |

This queue is for tickets about the IO-Compress-Base CPAN distribution.

Report information
The Basics
Id: 30776
Status: resolved
Priority: 0/
Queue: IO-Compress-Base

People
Owner: Nobody in particular
Requestors: ingo [...] ingoschmiegel.de
Cc:
AdminCc:

Bug Information
Severity: Critical
Broken in: 2.008
Fixed in: (no value)



Subject: IO::Uncompress::AnyUncompress behaves non-deterministically
In the attached example, IO::Uncompress::AnyUncompress does not detect that the input file is compressed. Is this the right forum to post this bug? I use code similar to this in a bigger script which also deploys Parse::Yapp. The files read are pretty large and I would therefore very much like to use this module for automatic and transparent uncompression of the input stream. Funniliy enough my gzipp'ed input is about 90% of the time read in without being uncompressed, while 10% of the time it gets read in compressed. For any given version of my script it stays the same: Always either uncompressed or compressed. But if I change the code a little I may break or enable automatic uncompression. I'm completely puzzled by this behaviour and have no clue whatsoever what could cause this phenomenon. The example script attached does not detect that the input file is compressed and prints the garbled data. If you could have a look at it - maybe I'm doing some obvious error here. Also I did not manage to modify this test script so that it would always detect the compression and print the clear data. Bummer. Any help is highly appreciated! Cheers, Ingo
Subject: test.gz
Download test.gz
application/x-gzip 268b

Message body not shown because it is not plain text.

Subject: test.pl
use lib "./perl5/site_perl/5.8.0"; use IO::Uncompress::AnyUncompress; # handle gzip/bzip2/zip/lzf compressed input files my $fn = 'test.gz'; my $in; my $line; ### Does not detect packed format: $in = new IO::Uncompress::AnyUncompress $fn; if (!defined($in)) { die "Fatal: Failed to load file: $IO::Uncompress::AnyUncompress::AnyUncompressError\n"; } while($line = <$in>) { print $line; } exit 0;
From: PMQS [...] cpan.org
On Fri Nov 16 05:52:30 2007, cfi wrote: Show quoted text
> In the attached example, IO::Uncompress::AnyUncompress does not detect > that the input file is compressed. > > Is this the right forum to post this bug?
This forum is fine. As is mailing me directly. Show quoted text
> I use code similar to this in a bigger script which also deploys > Parse::Yapp. The files read are pretty large and I would therefore
very Show quoted text
> much like to use this module for automatic and transparent
uncompression Show quoted text
> of the input stream. > > Funniliy enough my gzipp'ed input is about 90% of the time read in > without being uncompressed, while 10% of the time it gets read in > compressed. For any given version of my script it stays the same:
Always Show quoted text
> either uncompressed or compressed. But if I change the code a little I > may break or enable automatic uncompression. I'm completely puzzled by > this behaviour and have no clue whatsoever what could cause this
phenomenon. Show quoted text
> > The example script attached does not detect that the input file is > compressed and prints the garbled data.
Hmm, not sure what is going on there - I ran you script and it worked fine. Can you check if you have the IO::Uncompress::Gunzip module installed? Something like this should do the trick. perl -MIO::Uncompress::Gunzip -e 'print qq[ver $IO::Uncompress::Gunzip::VERSION\n]' Also, do the same for IO::Uncompress::Base perl -MIO::Uncompress::Base-e 'print qq[ver $IO::Uncompress::Base::VERSION\n]' Get back to me with what that gives you please. Paul
From: ingo [...] ingoschmiegel.de
IO::Uncompress::Base got installed when I installed AnyUncompress Gunzip is missing. Is this listed as a prerequisite of AnyUncompress? I guess it's intentional if not, because everybody has different needs and may not need a gzip interface. Would it make sense to add an API to the OO version of this module so that one could check for supported compression/inflation methods. Sth like if (!defined( $obj->algorithms()->{gzip} )) { die "Cannot handle gzipped data"; } Hm, but that would make sense only if Anyuncompress could test the type of input file - even without those modules being installed. Then, additionally to the api above, it would be possible to do sth like: if ($obj->inflatable()) { die "Cannot handle ".$obj->packed_with()."'ed data"; } The inflatable() method would check with that algorithm the file is acked (one of: qw(plain gzip bzip lzf)) and would compare that with the available algorithms: sub inflatable($) { my ($self) = @_; return $self->algorithms()->{ $self->packed_with() }; } Back to my problem: I'll install that Gunzip module and recheck. Bear with me...
On Fri Nov 16 07:04:31 2007, cfi wrote: Show quoted text
> IO::Uncompress::Base got installed when I installed AnyUncompress > > Gunzip is missing. Is this listed as a prerequisite of AnyUncompress?
I Show quoted text
> guess it's intentional if not, because everybody has different needs
and Show quoted text
> may not need a gzip interface.
It isn't a prerequisite for the very reason you mention. Show quoted text
> > Would it make sense to add an API to the OO version of this module so > that one could check for supported compression/inflation methods.
Yes, it would. It's on my list of possible enhancements. Show quoted text
> Sth like > if (!defined( $obj->algorithms()->{gzip} )) { > die "Cannot handle gzipped data"; > } > Hm, but that would make sense only if Anyuncompress could test the
type Show quoted text
> of input file - even without those modules being installed. Then, > additionally to the api above, it would be possible to do sth like: > if ($obj->inflatable()) { > die "Cannot handle ".$obj->packed_with()."'ed data"; > } > The inflatable() method would check with that algorithm the file is > acked (one of: qw(plain gzip bzip lzf)) and would compare that with
the Show quoted text
> available algorithms: > sub inflatable($) { > my ($self) = @_; > return $self->algorithms()->{ $self->packed_with() }; > }
At the moment AnyUncompress has a hard-wired list of uncompression modules that it makes use of to actually do the hard work. I'll probably add a static method that will work like this if (! AnyUncompress::canUncompress("gzip") )) { die "Cannot handle gzipped data"; } I still need to work out the exact interface for this. Show quoted text
> > Back to my problem: > I'll install that Gunzip module and recheck. > Bear with me...
OK Regarding your application, if you are only ever using gzip to compress your data then you can use IO::Uncompress::Gunzip instead of AnyUncompress. If you are working with mixed compression formats, or want to future- proof your code in case you change the compression format (to, say, bzip2), then AnyUncompress is a good option. Paul
From: ingo [...] ingoschmiegel.de
Great stuff! Ok, with Gunzip.pm installed, this works. I'm now running into all sorts of trouble because I need to distribute with my script several additional modules including some that rely on binary files. But I'm also 75% on getting PAR to run. *phew* Yeah, I used the "hardcoded" gzip modules before, but "future-proofness" is exactly what I'm lookingi for. And I always wanted to check out PAR anyway ;-) Can you please close this bug? Thanks for your help!
Resolved.