Skip Menu |

This queue is for tickets about the IO-Compress-Zlib CPAN distribution.

Report information
The Basics
Id: 33231
Status: resolved
Priority: 0/
Queue: IO-Compress-Zlib

People
Owner: Nobody in particular
Requestors: ianburrell [...] gmail.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: 2.008
Fixed in: 2.009



Subject: IO::Uncompress::Gunzip errors when reading from Net::FTP data socket
We are getting errors when using an IO::Uncompress::Gunzip object that is reading from an FTP data connection socket. my $in_fh = $ftp->retr($filename); my $gz = IO::Uncompress::Gunzip->new($in_fh); print $out_fh $buffer while $gz->read($buffer, 1000) > 0; Timeout at /dp/usr/imb/perl/lib/IO/Uncompress/Base.pm line 67 When using Compress::Zlib, we get "data error" errors when doing gzread. We didn't have any trouble reading directly from the socket with Compress::Zlib 1.x. my $gz = gzopen($in_fh, 'rb') or die "Can't gzopen: $gzerrno"; my $buffer; while (my $size = $gz->gzread($buffer)) { die "Read error: $gzerrno\n" if $size < 0; print $out_fh $buffer; } This is on RHEL 4, with locally-built Perl 5.8.8, and Net::FTP 2.75.
On Wed Feb 13 15:35:57 2008, ianburrell@gmail.com wrote: Show quoted text
> We are getting errors when using an IO::Uncompress::Gunzip object that > is reading from an FTP data connection socket. > > my $in_fh = $ftp->retr($filename); > my $gz = IO::Uncompress::Gunzip->new($in_fh); > print $out_fh $buffer while $gz->read($buffer, 1000) > 0; > > Timeout at /dp/usr/imb/perl/lib/IO/Uncompress/Base.pm line 67 > > When using Compress::Zlib, we get "data error" errors when doing gzread. > > We didn't have any trouble reading directly from the socket with > Compress::Zlib 1.x. > > my $gz = gzopen($in_fh, 'rb') or die "Can't gzopen: $gzerrno"; > my $buffer; > while (my $size = $gz->gzread($buffer)) { > die "Read error: $gzerrno\n" if $size < 0; > print $out_fh $buffer; > } > > This is on RHEL 4, with locally-built Perl 5.8.8, and Net::FTP 2.75. > >
Hi Ian, do you have problems with all the gzip files you access from the ftp server, or just some? any chance you could send me one of the gzip files that is causing you problems? Paul
Subject: Re: [rt.cpan.org #33231] IO::Uncompress::Gunzip errors when reading from Net::FTP data socket
Date: Thu, 14 Feb 2008 11:56:19 -0800
To: bug-IO-Compress-Zlib [...] rt.cpan.org
From: "Ian Burrell" <ianburrell [...] gmail.com>
On Thu, Feb 14, 2008 at 1:27 AM, Paul Marquess via RT <bug-IO-Compress-Zlib@rt.cpan.org> wrote: Show quoted text
> > do you have problems with all the gzip files you access from the ftp > server, or just some? > > any chance you could send me one of the gzip files that is causing you > problems? >
It seems to happen with every .gz file. The problem only happens with Net::FTP. We worked around the problem by downloading to a temp file and reading from that. Normal pipes and sockets work fine. I think I found the problem and I think the bug is in Net::FTP. The Net::FTP:I::read method has the parameters read($buf,$size,[$timeout]). The standard handle read method is read($buf, $size, $offset). IO::Uncompress::Base::smartRead passes 0 for the $offset. But Net::FTP::I interprets that as 0 timeout which causes a "Timeout" error whenever there is no data immediately available for reading. The reason that old version of Compress::Zlib worked is that it read directly from the socket and bypassed the read method. I'll enter a bug for Net::FTP. - Ian
Subject: Re: [rt.cpan.org #33231] IO::Uncompress::Gunzip errors when reading from Net::FTP data socket
Date: Thu, 14 Feb 2008 12:22:21 -0800
To: bug-IO-Compress-Zlib [...] rt.cpan.org
From: "Ian Burrell" <ianburrell [...] gmail.com>
I entered #33268 for Net::FTP about the discrepancy in the read arguments.
On Thu Feb 14 14:56:54 2008, ianburrell@gmail.com wrote: Show quoted text
> On Thu, Feb 14, 2008 at 1:27 AM, Paul Marquess via RT > <bug-IO-Compress-Zlib@rt.cpan.org> wrote:
> > > > do you have problems with all the gzip files you access from the ftp > > server, or just some? > > > > any chance you could send me one of the gzip files that is causing you > > problems? > >
> > It seems to happen with every .gz file. The problem only happens with > Net::FTP. We worked around the problem by downloading to a temp file > and reading from that. Normal pipes and sockets work fine. > > I think I found the problem and I think the bug is in Net::FTP. The > Net::FTP:I::read method has the parameters > read($buf,$size,[$timeout]). The standard handle read method is > read($buf, $size, $offset). IO::Uncompress::Base::smartRead passes 0 > for the $offset. But Net::FTP::I interprets that as 0 timeout which > causes a "Timeout" error whenever there is no data immediately > available for reading. > > The reason that old version of Compress::Zlib worked is that it read > directly from the socket and bypassed the read method.
Yep, just got round to trying it out myself. That exactly what the problem is. I've attached a tentative patch for lib/IO/Uncompress/Base.pm that should sort it. I need to do some more testing before I release it into the wild. Give me a shout if it doesn't work for you. Paul
Subject: Re: [rt.cpan.org #33231] IO::Uncompress::Gunzip errors when reading from Net::FTP data socket
Date: Thu, 14 Feb 2008 14:18:03 -0800
To: bug-IO-Compress-Zlib [...] rt.cpan.org
From: "Ian Burrell" <ianburrell [...] gmail.com>
On Thu, Feb 14, 2008 at 1:03 PM, Paul Marquess via RT <bug-IO-Compress-Zlib@rt.cpan.org> wrote: Show quoted text
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=33231 > > > > Yep, just got round to trying it out myself. That exactly what the > problem is. > > I've attached a tentative patch for lib/IO/Uncompress/Base.pm that > should sort it. I need to do some more testing before I release it into > the wild. >
I don't see an attachment, either on the email or in RT. It probably got eaten by RT. - Ian
On Thu Feb 14 17:18:15 2008, ianburrell@gmail.com wrote: Show quoted text
> On Thu, Feb 14, 2008 at 1:03 PM, Paul Marquess via RT > <bug-IO-Compress-Zlib@rt.cpan.org> wrote:
> > > > <URL: http://rt.cpan.org/Ticket/Display.html?id=33231 > > > > > > > Yep, just got round to trying it out myself. That exactly what the > > problem is. > > > > I've attached a tentative patch for lib/IO/Uncompress/Base.pm that > > should sort it. I need to do some more testing before I release it into > > the wild. > >
> > I don't see an attachment, either on the email or in RT. It probably > got eaten by RT.
Hmmm, so it isn't. Let see if it works this time. Paul
*** IO_Compress_Base/lib/IO/Uncompress/Base.pm 2007-11-12 10:44:47.000000000 +0000 --- IO-Base/lib/IO/Uncompress/Base.pm 2008-02-14 17:59:46.000000000 +0000 *************** *** 63,70 **** # $get_size = min($get_size, *$self->{InputLengthRemaining}); #} ! if (defined *$self->{FH}) ! { *$self->{FH}->read($$out, $get_size, $offset) } elsif (defined *$self->{InputEvent}) { my $got = 1 ; while (length $$out < $size) { --- 63,78 ---- # $get_size = min($get_size, *$self->{InputLengthRemaining}); #} ! if (defined *$self->{FH}) { ! if ($offset) { ! #*$self->{FH}->read($$out, $get_size, $offset); ! my $tmp; ! *$self->{FH}->read($tmp, $get_size); ! substr($$out, $offset) = $tmp; ! } ! else ! { *$self->{FH}->read($$out, $get_size) } ! } elsif (defined *$self->{InputEvent}) { my $got = 1 ; while (length $$out < $size) {
Subject: Re: [rt.cpan.org #33231] IO::Uncompress::Gunzip errors when reading from Net::FTP data socket
Date: Thu, 14 Feb 2008 16:04:18 -0800
To: bug-IO-Compress-Zlib [...] rt.cpan.org
From: "Ian Burrell" <ianburrell [...] gmail.com>
On Thu, Feb 14, 2008 at 2:47 PM, Paul Marquess via RT <bug-IO-Compress-Zlib@rt.cpan.org> wrote: Show quoted text
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=33231 > > > > Hmmm, so it isn't. Let see if it works this time. >
The patch is there and fixes the problem. BTW, substr throws an error if the offset is beyond the end of the string instead of extending it. This block of code will pad the buffer with NULLs, like is described in the read perlfunc description: if (length($buf) < $offset) { $buf .= ('\0' x ($offset + 1 - length($buf))); } I added a patch to the Net::FTP bug which fixes it on their end but we'll see how willing they are to change the interface and break backwards compatibility to improve compatibility with other modules. - Ian
On Thu Feb 14 19:04:40 2008, ianburrell@gmail.com wrote: Show quoted text
> On Thu, Feb 14, 2008 at 2:47 PM, Paul Marquess via RT > <bug-IO-Compress-Zlib@rt.cpan.org> wrote:
> > > > <URL: http://rt.cpan.org/Ticket/Display.html?id=33231 > > > > > > > Hmmm, so it isn't. Let see if it works this time. > >
> > The patch is there and fixes the problem.
Good. Show quoted text
> BTW, substr throws an error if the offset is beyond the end of the > string instead of extending it. This block of code will pad the > buffer with NULLs, like is described in the read perlfunc description: > > if (length($buf) < $offset) { > $buf .= ('\0' x ($offset + 1 - length($buf))); > }
Thanks. I'll check that out when I find the time to carry out some rigorous testing. Show quoted text
> I added a patch to the Net::FTP bug which fixes it on their end but > we'll see how willing they are to change the interface and break > backwards compatibility to improve compatibility with other modules.
As I've noted in the bug you raised against Net::FTP, I'm agree with Graham that it's too late to mess with Net::FTP. cheers Paul