Skip Menu |

This queue is for tickets about the Archive-Zip CPAN distribution.

Report information
The Basics
Id: 92205
Status: resolved
Priority: 0/
Queue: Archive-Zip

People
Owner: Nobody in particular
Requestors: m.bunkus [...] linet-services.de
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: CRC error when re-writing Zip created by LibreOffice
Date: Thu, 16 Jan 2014 15:17:50 +0100
To: bug-Archive-Zip [...] rt.cpan.org
From: Moritz Bunkus <m.bunkus [...] linet-services.de>
Hey, the attached file was created with LibreOffice 4.1.4. It cannot be re-written with Archive::Zip due to the following stack trace: format error: CRC or size mismatch while skipping data descriptor at /usr/share/perl5/vendor_perl/Archive/Zip/ZipFileMember.pm line 195. Archive::Zip::ZipFileMember::_skipLocalFileHeader('Archive::Zip::ZipFileMember=HASH(0x15a6208)') called at /usr/share/perl5/vendor_perl/Archive/Zip/ZipFileMember.pm line 400 Archive::Zip::ZipFileMember::rewindData('Archive::Zip::ZipFileMember=HASH(0x15a6208)') called at /usr/share/perl5/vendor_perl/Archive/Zip/Member.pm line 1084 Archive::Zip::Member::_writeToFileHandle('Archive::Zip::ZipFileMember=HASH(0x15a6208)', 'IO::File=GLOB(0xf8b920)', 1, 13303) called at /usr/share/perl5/vendor_perl/Archive/Zip/Archive.pm line 428 Archive::Zip::Archive::writeToFileHandle('Archive::Zip::Archive=HASH(0xf8ba58)', 'IO::File=GLOB(0xf8b920)', 1) called at /usr/share/perl5/vendor_perl/Archive/Zip/Archive.pm line 396 Archive::Zip::Archive::writeToFileNamed('Archive::Zip::Archive=HASH(0xf8ba58)', 'out.zip') called at -e line 1 The resulting file is corrupt as writing stops with that error. Code to reproduce: perl -MArchive::Zip -e 'my $z = Archive::Zip->new; $z->read(shift); $z->writeToFileNamed("out.zip");' test.odt This error starts with Archive::Zip v1.31_04 (1.31_03 works nicely) and persists even today in 1.37. If I manually comment out the lines in lib/Archive/Member.pm that were introduced in 1.31_04 then it works again, and the resulting file can be unzipped with the unzip command line program. Kind regards, Moritz -- Dipl.-Inform. Moritz Bunkus Geschäftsführer/CTO LINET Services GmbH | Cyriaksring 10a | 38118 Braunschweig Tel. 0531-180508-0 | Fax 0531-180508-29 | http://www.linet-services.de Geschäftsführung: Moritz Bunkus, Philip Reetz und Timo Springmann HR B 9170 Amtsgericht Braunschweig USt-IdNr. DE 259 526 516
Download test.odt
application/vnd.oasis.opendocument.text 17.5k

Message body not shown because it is not plain text.

Download (untitled)
application/pgp-signature 198b

Message body not shown because it is not plain text.

Le Jeu 16 Jan 2014 09:18:45, m.bunkus@linet-services.de a écrit : Show quoted text
> Hey, > > the attached file was created with LibreOffice 4.1.4. It cannot be > re-written with Archive::Zip due to the following stack trace: > > format error: CRC or size mismatch while skipping data descriptor > at /usr/share/perl5/vendor_perl/Archive/Zip/ZipFileMember.pm line > 195. > Archive::Zip::ZipFileMember::_skipLocalFileHeader('Archive::Zip::ZipFileMember=HASH(0x15a6208)') > called at /usr/share/perl5/vendor_perl/Archive/Zip/ZipFileMember.pm > line 400 > Archive::Zip::ZipFileMember::rewindData('Archive::Zip::ZipFileMember=HASH(0x15a6208)') > called at /usr/share/perl5/vendor_perl/Archive/Zip/Member.pm line 1084 > Archive::Zip::Member::_writeToFileHandle('Archive::Zip::ZipFileMember=HASH(0x15a6208)', > 'IO::File=GLOB(0xf8b920)', 1, 13303) called at > /usr/share/perl5/vendor_perl/Archive/Zip/Archive.pm line 428 > Archive::Zip::Archive::writeToFileHandle('Archive::Zip::Archive=HASH(0xf8ba58)', > 'IO::File=GLOB(0xf8b920)', 1) called at > /usr/share/perl5/vendor_perl/Archive/Zip/Archive.pm line 396 > Archive::Zip::Archive::writeToFileNamed('Archive::Zip::Archive=HASH(0xf8ba58)', > 'out.zip') called at -e line 1 > > The resulting file is corrupt as writing stops with that error. > > Code to reproduce: > > perl -MArchive::Zip -e 'my $z = Archive::Zip->new; $z->read(shift); > $z->writeToFileNamed("out.zip");' test.odt > > This error starts with Archive::Zip v1.31_04 (1.31_03 works nicely) > and > persists even today in 1.37. If I manually comment out the lines > in lib/Archive/Member.pm that were introduced in 1.31_04 then it works > again, and the resulting file can be unzipped with the unzip command > line program. > > Kind regards, > Moritz
Dear, I have had the same problem on a Linux machine which use perl 5.14 and ODF::lpOD. This module creates ODT file by using Archive::Zip. On the machine, the Archive::Zip version is 1.37 and It is impossible to create ODT file : format error: CRC or size mismatch while skipping data descriptor at /usr/local/share/perl/5.14.2/ODF/lpOD/Document.pm line 413 Zip I/O error to resolv this problem, I have downloaded and installed manually Archive::Zip 1.30. Best Regards, djibel
Subject: Re: [rt.cpan.org #92205] CRC error when re-writing Zip created by LibreOffice
Date: Thu, 23 Jan 2014 17:43:45 -0800
To: bug-Archive-Zip [...] rt.cpan.org
From: Fred Moyer <fred [...] redhotpenguin.com>
This is up next in my open source queue. Any patches you might have would be most welcome. On Thu, Jan 23, 2014 at 2:51 PM, Djibril Ousmanou via RT <bug-Archive-Zip@rt.cpan.org> wrote: Show quoted text
> Queue: Archive-Zip > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=92205 > > > Le Jeu 16 Jan 2014 09:18:45, m.bunkus@linet-services.de a écrit :
>> Hey, >> >> the attached file was created with LibreOffice 4.1.4. It cannot be >> re-written with Archive::Zip due to the following stack trace: >> >> format error: CRC or size mismatch while skipping data descriptor >> at /usr/share/perl5/vendor_perl/Archive/Zip/ZipFileMember.pm line >> 195. >> Archive::Zip::ZipFileMember::_skipLocalFileHeader('Archive::Zip::ZipFileMember=HASH(0x15a6208)') >> called at /usr/share/perl5/vendor_perl/Archive/Zip/ZipFileMember.pm >> line 400 >> Archive::Zip::ZipFileMember::rewindData('Archive::Zip::ZipFileMember=HASH(0x15a6208)') >> called at /usr/share/perl5/vendor_perl/Archive/Zip/Member.pm line 1084 >> Archive::Zip::Member::_writeToFileHandle('Archive::Zip::ZipFileMember=HASH(0x15a6208)', >> 'IO::File=GLOB(0xf8b920)', 1, 13303) called at >> /usr/share/perl5/vendor_perl/Archive/Zip/Archive.pm line 428 >> Archive::Zip::Archive::writeToFileHandle('Archive::Zip::Archive=HASH(0xf8ba58)', >> 'IO::File=GLOB(0xf8b920)', 1) called at >> /usr/share/perl5/vendor_perl/Archive/Zip/Archive.pm line 396 >> Archive::Zip::Archive::writeToFileNamed('Archive::Zip::Archive=HASH(0xf8ba58)', >> 'out.zip') called at -e line 1 >> >> The resulting file is corrupt as writing stops with that error. >> >> Code to reproduce: >> >> perl -MArchive::Zip -e 'my $z = Archive::Zip->new; $z->read(shift); >> $z->writeToFileNamed("out.zip");' test.odt >> >> This error starts with Archive::Zip v1.31_04 (1.31_03 works nicely) >> and >> persists even today in 1.37. If I manually comment out the lines >> in lib/Archive/Member.pm that were introduced in 1.31_04 then it works >> again, and the resulting file can be unzipped with the unzip command >> line program. >> >> Kind regards, >> Moritz
> > Dear, > > I have had the same problem on a Linux machine which use perl 5.14 and ODF::lpOD. This module creates ODT file by using Archive::Zip. > > On the machine, the Archive::Zip version is 1.37 and It is impossible to create ODT file : > > format error: CRC or size mismatch while skipping data descriptor at /usr/local/share/perl/5.14.2/ODF/lpOD/Document.pm line 413 > Zip I/O error > > to resolv this problem, I have downloaded and installed manually Archive::Zip 1.30. > > Best Regards, > > djibel
From: jim.avera [...] gmail.com
On Thu Jan 23 20:43:54 2014, fred@redhotpenguin.com wrote: Show quoted text
> This is up next in my open source queue. Any patches you might have > would be most welcome.
Any progress on this? This is breaking ODF::lpOD's abilty to process Libre Office documents. The original reporter mentioned that the bug was introduced in v1.31_04 and that commenting out the lines introduced in that rev makes the problem go away. Does that provide enough info for a patch?
From: jim.avera [...] gmail.com
Here is the changelog for the change which (reportedly) introduced the problem: 1.31_04 Fri 14 Oct 2011 - Alan Haggai Alavi - Updated Perl dependency to 5.006 to reflect implicit dependencies in the code exposed by Perl::MinimumVersion xt test (ADAMK) - Fixes: #68446: Set compressed size and uncompressed size of an entry to 0 if either of them is 0 (HAGGAI) - Added $VERSION to crc32 (ADAMK)
I added some instrumentation to Archive::Zip and found that it is comlaining about member 'Configurations2/accelerator/current.xml'. Here is what unzip thinks about current.xml $ unzip -lv test.odt '*current.xml' Archive: test.odt Length Method Size Cmpr Date Time CRC-32 Name -------- ------ ------- ---- ---------- ----- -------- ---- 0 Defl:N 2 0% 2014-01-16 14:13 00000000 Configurations2/accelerator/current.xml -------- ------- --- ------- 0 2 0% 1 file Note that the file Legth (uncompressed size) is 0 and Size is 2 (i.e. compressed size). Deflating a zereo length file/buffer gives a 2-byte compressed buffer. Nothing illegal about that - it just wasted a couple of bytes. The other thing of note about this file is that it uses streaming for a lot of the members in the zip archive, including "current.xml". Importantly Archive::Zip is blowing up in a section of code where it deals with a streaming specific part of the zip file. I see that backing out fix #68446 has been suggested as a solution for this issue. I checked that it does seem to fix this particular issue, but breaks the issue that #68446 was suppost to be fixing. So a simle backout isn't an option here. Sorry, but no fix (don't know A::Z well enough) but I don't think it is a coincidence that this issue is triggered by zero length content and that #68446 was a fix for another zero length content issue.
Enclosed patch is a tentative fix for this issue. It assumes the fix for #101092 has already been applied. I'm not completely comfortable with how the Archive::Zip code works, so although I have supplied a test harness for the fix, the patch does need checked over by someone who knows the code.
Subject: 92205.patch
diff --git a/MANIFEST b/MANIFEST index 8fa704a..2f9d199 100644 --- a/MANIFEST +++ b/MANIFEST @@ -50,13 +50,23 @@ t/14_leading_separator.t t/15_decrypt.t t/16_decrypt.t t/17_101092.t +t/18_bug_92205.t t/badjpeg/expected.jpg t/badjpeg/source.zip t/common.pm t/data/chmod.zip t/data/crypcomp.zip t/data/crypt.zip +t/data/def.zip +t/data/defstr.zip +t/data/store.zip +t/data/storestr.zip +t/data/emptydef.zip +t/data/emptydefstr.zip +t/data/emptystore.zip +t/data/emptystorestr.zip t/data/linux.zip +t/data/mkzip.pl t/data/perl.zip t/data/streamed.zip t/data/winzip.zip diff --git a/lib/Archive/Zip/Member.pm b/lib/Archive/Zip/Member.pm index 105a95e..1092057 100644 --- a/lib/Archive/Zip/Member.pm +++ b/lib/Archive/Zip/Member.pm @@ -196,8 +196,8 @@ sub desiredCompressionMethod { $self->{'desiredCompressionMethod'} = $newDesiredCompressionMethod; if ($newDesiredCompressionMethod == COMPRESSION_STORED) { $self->{'desiredCompressionLevel'} = 0; - $self->{'bitFlag'} &= ~GPBF_HAS_DATA_DESCRIPTOR_MASK; - + $self->{'bitFlag'} &= ~GPBF_HAS_DATA_DESCRIPTOR_MASK + if $self->uncompressedSize() == 0; } elsif ($oldDesiredCompressionMethod == COMPRESSION_STORED) { $self->{'desiredCompressionLevel'} = COMPRESSION_LEVEL_DEFAULT; } @@ -704,7 +704,7 @@ sub _writeLocalFileHeader { $self->_print($fh, $signatureData) or return _ioError("writing local header signature"); - my $header = $self->head(0); + my $header = $self->head(1); $self->_print($fh, $header) or return _ioError("writing local header"); @@ -1052,12 +1052,6 @@ sub _writeToFileHandle { and ($self->compressionMethod() == COMPRESSION_STORED or $self->desiredCompressionMethod() == COMPRESSION_DEFLATED)); - # Set both compressedSize and uncompressedSize to 0 if either of them is 0 - if ($self->uncompressedSize == 0 || $self->uncompressedSize == 0) { - $self->{'compressedSize'} = 0; - $self->{'uncompressedSize'} = 0; - } - my $shouldWriteDataDescriptor = ($headerFieldsUnknown and not $fhIsSeekable); @@ -1096,7 +1090,6 @@ sub _writeData { my ($outRef, $status) = $self->readChunk($chunkSize); symlink $$outRef, $self->{'newName'}; } else { - return AZ_OK if ($self->uncompressedSize() == 0); my $status; my $chunkSize = $Archive::Zip::ChunkSize; while ($self->_readDataRemaining() > 0) { diff --git a/lib/Archive/Zip/ZipFileMember.pm b/lib/Archive/Zip/ZipFileMember.pm index 34e91c5..ba792d9 100644 --- a/lib/Archive/Zip/ZipFileMember.pm +++ b/lib/Archive/Zip/ZipFileMember.pm @@ -193,6 +193,9 @@ sub _skipLocalFileHeader { "CRC or size mismatch while skipping data descriptor") if ( $oldCrc32 != $self->{'crc32'} || $oldUncompressedSize != $self->{'uncompressedSize'}); + + $self->{'crc32'} = 0 + if $self->compressionMethod() == COMPRESSION_STORED ; } return AZ_OK; diff --git a/t/17_101092.t b/t/17_101092.t index 2b2983b..068f356 100644 --- a/t/17_101092.t +++ b/t/17_101092.t @@ -1,4 +1,4 @@ -#!/use/bin/perl +#!/usr/bin/perl use strict; diff --git a/t/common.pm b/t/common.pm index 2679736..402f12d 100644 --- a/t/common.pm +++ b/t/common.pm @@ -227,15 +227,15 @@ sub passthrough { my $fromFile = shift ; my $toFile = shift ; - my $keepTime = shift ; + my $action = shift ; my $z = Archive::Zip->new; $z->read($fromFile); - if ($keepTime) + if ($action) { for my $member($z->members()) { - $member->setLastModFileDateTimeFromUnix($member->lastModTime()); + &$action($member) ; } } $z->writeToFileNamed($toFile);
Subject: 92205.tar
Download 92205.tar
application/x-tar 20k

Message body not shown because it is not plain text.

Subject: Re: [rt.cpan.org #92205] CRC error when re-writing Zip created by LibreOffice
Date: Thu, 1 Jan 2015 16:08:14 -0800
To: bug-Archive-Zip [...] rt.cpan.org
From: Fred Moyer <fred [...] redhotpenguin.com>
Thanks, will apply these tonight On Jan 1, 2015 4:04 PM, "Paul Marquess via RT" <bug-Archive-Zip@rt.cpan.org> wrote: Show quoted text
> Queue: Archive-Zip > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=92205 > > > Enclosed patch is a tentative fix for this issue. It assumes the fix for > #101092 has already been applied. > > I'm not completely comfortable with how the Archive::Zip code works, so > although I have supplied a test harness for the fix, the patch does need > checked over by someone who knows the code. >
Thanks - ping me if you need any help.
Resolved with 1.40