Bug #101092 for Archive-Zip: Creation of non-standard streamed zip file

Mon Dec 22 18:56:03 2014 pmqs [...] cpan.org - Ticket created

Subject:

Creation of non-standard streamed zip file

When digging around #92205 I spotted that Archive::Zip created a non-standard local header when it process a zip file that already uses streaming. Here is the relevant part from appnote.txt that discusses streaming (https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT) 4.4.4 general purpose bit flag: (2 bytes) . . . Bit 3: If this bit is set, the fields crc-32, compressed size and uncompressed size are set to zero in the local header. The correct values are put in the data descriptor immediately following the compressed data. (Note: PKZIP version 2.04g for DOS only recognizes this bit for method 8 compression, newer versions of PKZIP recognize this bit for any compression method. The enclosed file, abc.zip, contains a single member "fred" that has been streamed. This is how I created it: $ perl -MIO::Compress::Zip=zip -e 'zip \"abc" => "abc.zip", Name => "fred", Stream => 1, Method => 8' Here is a dump of the local headers section of the zip file $ zipdetails abc.zip 0000 LOCAL HEADER #1 04034B50 0004 Extract Zip Spec 14 '2.0' 0005 Extract OS 00 'MS-DOS' 0006 General Purpose Flag 0008 [Bits 1-2] 0 'Normal Compression' [Bit 3] 1 'Streamed' 0008 Compression Method 0008 'Deflated' 000A Last Mod Time 4596BBBD 'Mon Dec 22 23:29:58 2014' 000E CRC 00000000 0012 Compressed Length 00000000 0016 Uncompressed Length 00000000 001A Filename Length 0004 001C Extra Length 0000 001E Filename 'fred' 0022 PAYLOAD KLJ.. 0027 STREAMING DATA HEADER 08074B50 002B CRC 352441C2 002F Compressed Length 00000005 0033 Uncompressed Length 00000003 Key points are the presence of the Streamed bit, and that CRC, Compressed length & Uncompressed length are all zero in the local header, but populated in the Data Header. I then got Archive::Zip to process the streamed zip file as follows $ perl -MArchive::Zip -e 'my $z = Archive::Zip->new; $z->read(shift); $z->writeToFileNamed("out.zip");' abc.zip Here are the local headers from the new zip file $ zipdetails out.zip 0000 LOCAL HEADER #1 04034B50 0004 Extract Zip Spec 14 '2.0' 0005 Extract OS 00 'MS-DOS' 0006 General Purpose Flag 0008 [Bits 1-2] 0 'Normal Compression' [Bit 3] 1 'Streamed' 0008 Compression Method 0008 'Deflated' 000A Last Mod Time 4596BBBD 'Mon Dec 22 23:29:58 2014' 000E CRC 352441C2 0012 Compressed Length 00000005 0016 Uncompressed Length 00000003 001A Filename Length 0004 001C Extra Length 0000 001E Filename 'fred' 0022 PAYLOAD KLJ.. 0027 STREAMING DATA HEADER 08074B50 002B CRC 352441C2 002F Compressed Length 00000005 0033 Uncompressed Length 00000003 Note the Streaming bit is set but the values for CRC, Compressed length & Uncompressed length are all populated in the Local Header with values that match the Streaming Data Header.

Subject:

abc.zip

Download abc.zip
application/zip 127b

Message body not shown because it is not plain text.

Sun Dec 28 06:20:18 2014 pmqs [...] cpan.org - Correspondence added

Fix for this was quite straightforward, so created a patch.

Subject:

101092.patch

diff --git a/MANIFEST b/MANIFEST index 91c6e0c..8fa704a 100644 --- a/MANIFEST +++ b/MANIFEST @@ -49,6 +49,7 @@ t/13_bug_46303.t t/14_leading_separator.t t/15_decrypt.t t/16_decrypt.t +t/17_101092.t t/badjpeg/expected.jpg t/badjpeg/source.zip t/common.pm @@ -57,6 +58,7 @@ t/data/crypcomp.zip t/data/crypt.zip t/data/linux.zip t/data/perl.zip +t/data/streamed.zip t/data/winzip.zip META.yml Module YAML meta-data (added by MakeMaker) META.json Module JSON meta-data (added by MakeMaker) diff --git a/lib/Archive/Zip/Member.pm b/lib/Archive/Zip/Member.pm index 32912e9..105a95e 100644 --- a/lib/Archive/Zip/Member.pm +++ b/lib/Archive/Zip/Member.pm @@ -678,10 +678,16 @@ sub head { $self->versionNeededToExtract(), $self->{'bitFlag'}, $self->desiredCompressionMethod(), - $self->lastModFileDateTime(), $self->crc32(), $mode - ? $self->_writeOffset() # compressed size - : $self->compressedSize(), # may need to be re-written later - $self->uncompressedSize(), + $self->lastModFileDateTime(), + $self->hasDataDescriptor() + ? (0,0,0) # crc, compr & uncompr all zero if data descriptor present + : ( + $self->crc32(), + $mode + ? $self->_writeOffset() # compressed size + : $self->compressedSize(), # may need to be re-written later + $self->uncompressedSize(), + ), length($self->fileName()), length($self->localExtraField()); } diff --git a/t/common.pm b/t/common.pm index cb40473..2679736 100644 --- a/t/common.pm +++ b/t/common.pm @@ -223,4 +223,33 @@ BEGIN { } } +sub passthrough +{ + my $fromFile = shift ; + my $toFile = shift ; + my $keepTime = shift ; + + my $z = Archive::Zip->new; + $z->read($fromFile); + if ($keepTime) + { + for my $member($z->members()) + { + $member->setLastModFileDateTimeFromUnix($member->lastModTime()); + } + } + $z->writeToFileNamed($toFile); +} + +sub readFile +{ + my $name = shift ; + local $/; + open F, "<$name" + or die "Cannot open $name: $!\n"; + my $data = <F>; + close F ; + return $data; +} + 1;

Subject:

101092.tar

Download 101092.tar
application/x-tar 10k

Message body not shown because it is not plain text.

Sun Jan 04 23:46:24 2015 PHRED [...] cpan.org - Correspondence added

Change applied for 1.40 Commit reference is https://github.com/redhotpenguin/perl-Archive-Zip/commit/8c50bcc01b51d96bd29b1789ace04d34634f957c

Sun Jan 04 23:46:24 2015 The RT System itself - Status changed from 'new' to 'open'

Sun Jan 04 23:46:25 2015 PHRED [...] cpan.org - Status changed from 'open' to 'resolved'

Mon Jan 05 00:39:43 2015 PHRED [...] cpan.org - Correspondence added

Missed a test and file - adding another commit.