Skip Menu |

This queue is for tickets about the IO-Compress CPAN distribution.

Report information
The Basics
Id: 72974
Status: resolved
Priority: 0/
Queue: IO-Compress

People
Owner: Nobody in particular
Requestors: GRAF [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in:
  • 2.039
  • 2.040
  • 2.042
  • 2.043
  • 2.044
  • 2.045
Fixed in: 2.046



Subject: Adding a directory member is broken
Until 2.037 the following code would add a directory member to the archive: $zip->newStream( Name => $path_to_member . '/', ExtAttr => ($mode << 16) | 16 ); The trailing '/' is absolutely neccessary in this case. $mode comes from lstat. Starting with 2.039 the '/' is chop'ed internally, and as a consequence, the directory is not marked as such in the archive. With UNIX Info-Zip this member is unpacked as a normal file with wrong permissions. Setting the attribute 'CanonicalName' to false has fixed the problem. But until that fix was found, existing applicatioon code broke more or less silently after upgrading to a post 2.037 version.
Hi Bernhard, I thought long & hard about making that change. In the end I decided to go for making IO-C-Zip conform the the zip standard. Will see if I can get ExtAttr to spot if a directory is being added. The issue will be getting it to work cross-platform. Either that or add a convenience method to add a directory. Glad there is a workaround and sorry for the hassle. Paul
Paul, thank you the quick response. Show quoted text
> Will see if I can get ExtAttr to spot if a directory is being added. The > issue will be getting it to work cross-platform. Either that or add a > convenience method to add a directory.
The way I used to add a directory into a ZIP wasn't documented anyway. I just found out by looking at the Archive::Zip sources and by trying. So I was walking on thin ice, but it worked. Seems, the trailing slash is indeed internally used by (un)zip to mark a directory. So I hope there will be a general solution for that problem in a coming release - if possible in a backward compatible way, so that old code does not need to be changed. Bernhard
Hi Bernhard If you don't mind my asking, in your use-case does the directory name you are adding to the Zip file actually correspond to a real directory in the file system? So if there was an addDirectory method, like this $zip->addDirectory("/some/directory"); it would create the directory in the zip file with the trailing "/", plus it would get the file attributes from the real directory. It would also fail if the directory doesn't exist. Paul
Show quoted text
> If you don't mind my asking, in your use-case does the directory name > you > are adding to the Zip file actually correspond to a real directory in > the file system? So if there was an addDirectory method, like this > > $zip->addDirectory("/some/directory"); > > it would create the directory in the zip file with the trailing "/", > plus > it would get the file attributes from the real directory. It would > also > fail if the directory doesn't exist.
Actually the directory does exist in my case, but in general making this a requirement would be a bad concept, because the data you store in an archive with IO::Compress does not necessarily have to exist somewhere on the file system. Moreover creating a directory entry in the archive is not required by ZIP at all: unzip will create all required directories when unpacking files. The only use-case for explicit directories in ZIP archives, that I'm aware of, is to define permission bits for it. My suggestion is to fix sub canonicalName in a way to not chop a trailing '/' in path names, because - as I said - this has a special meaning for ZIP. I'm attaching a patch, that implements that behaviour. Bernhard
Subject: io-compress-zip.diff
--- IO/Compress/Zip.pm.orig 2011-12-07 14:45:51.254366232 +0100 +++ IO/Compress/Zip.pm 2011-12-07 15:30:45.908398176 +0100 @@ -173,31 +173,26 @@ # . '.' # ./a a # ./a/b a/b - # ./a/b/ a/b + # ./a/b/ a/b/ # a/b/ a/b # /a/b/ a/b # c:\a\b\c.doc a/b/c.doc # on Windows # "i/o maps:whatever" i_o maps/whatever # on Macs - my $name = shift; - my $forceDir = shift ; + my $name = shift; + my ($volume, $directories, $file) = File::Spec->splitpath($name, @_); + my @dirs = map { s{/}{_}g; $_ } + File::Spec->splitdir(File::Spec->canonpath($directories)); - my ( $volume, $directories, $file ) = - File::Spec->splitpath( File::Spec->canonpath($name), $forceDir ); - - my @dirs = map { $_ =~ s{/}{_}g; $_ } - File::Spec->splitdir($directories); - - if ( @dirs > 0 ) { pop (@dirs) if $dirs[-1] eq '' } # remove empty component - push @dirs, defined($file) ? $file : '' ; + push @dirs, defined($file) ? $file : ''; my $normalised_path = join '/', @dirs; # Leading directory separators should not be stored in zip archives. # Example: - # C:\a\b\c\ a/b/c + # C:\a\b\c\ a/b/c/ # C:\a\b\c.txt a/b/c.txt - # /a/b/c/ a/b/c + # /a/b/c/ a/b/c/ # /a/b/c.txt a/b/c.txt $normalised_path =~ s{^/}{}; # remove leading separator
On Wed Dec 07 09:36:36 2011, GRAF wrote: Show quoted text
> > If you don't mind my asking, in your use-case does the directory name > > you > > are adding to the Zip file actually correspond to a real directory in > > the file system? So if there was an addDirectory method, like this > > > > $zip->addDirectory("/some/directory"); > > > > it would create the directory in the zip file with the trailing "/", > > plus > > it would get the file attributes from the real directory. It would > > also > > fail if the directory doesn't exist.
> > Actually the directory does exist in my case, but in general making this > a requirement would be a bad concept, because the data you store in an > archive with IO::Compress does not necessarily have to exist somewhere > on the file system.
I agree that there are two use-cases. One is where you want to add a directory that mirrors something that exists on a filesyetem. The second use case is where it gets created in code. I suspect the former would be the predominant use-case, but don't want to restrict it to that. Show quoted text
> Moreover creating a directory entry in the archive is not required by > ZIP at all: unzip will create all required directories when unpacking > files. The only use-case for explicit directories in ZIP archives, that > I'm aware of, is to define permission bits for it.
Yep - that's what I've found as well. Show quoted text
> My suggestion is to fix sub canonicalName in a way to not chop a > trailing '/' in path names, because - as I said - this has a special > meaning for ZIP.
Show quoted text
> I'm attaching a patch, that implements that behaviour.
Part of the reason for doing CanonocalName was to make it more difficult for folk to get unexpected results. $zip->newStream( Name => 'myfile/') ; print $zip "some data"; In the above the user thinks they have created an entry in the zip file that stores the payload "some data". A commanline unzip will show the length of the stored data correctly, but if they unzip it with a command line unzipper they end up with a directory called "myFile", rather than a file that contains "some data". Paul
Show quoted text
> Part of the reason for doing CanonocalName was to make it more difficult > for folk to get unexpected results. > > $zip->newStream( Name => 'myfile/') ; > > print $zip "some data"; > > In the above the user thinks they have created an entry in the zip file > that stores the payload "some data". A commanline unzip will show the > length of the stored data correctly, but if they unzip it with a command > line unzipper they end up with a directory called "myFile", rather than > a file that contains "some data".
I understand your intention, but your fix makes it impossible to store a directory. Also, a library should not try to be smarter than the programmer, who is using it. In your example a $zip->print() after $zip->newStream(Name => 'myfile/') should simply die, IMHO. So instead of making changes, that brake backward compatibility, you should extend the interface to make the module more developer-friendly. Look a Archive::Zip for example. It has an addMember() method, which is similar to newStream(), and it has addFile(), addDirectory(), addString(), that use addMember() inside. Bernhard
... Show quoted text
> > I understand your intention, but your fix makes it impossible to store a > directory.
Not impossible - just disable CanonicalName :-) Show quoted text
> Also, a library should not try to be smarter than the > programmer, who is using it. In your example a > > $zip->print() > > after > > $zip->newStream(Name => 'myfile/') > > should simply die, IMHO. > > So instead of making changes, that brake backward compatibility, you > should extend the interface to make the module more developer-friendly. > > Look a Archive::Zip for example. It has an addMember() method, which is > similar to newStream(), and it has addFile(), addDirectory(), > addString(), that use addMember() inside.
Convenience methods like that are on my radar. Not sure yet if they will go into IO::Compress:Zip or in a new module. Paul
Show quoted text
> > I understand your intention, but your fix makes it impossible to store a > > directory.
> > Not impossible - just disable CanonicalName :-)
Granted, but it is not possible to add a directory with a canonical name. And it still breaks backward comp. which is bad, esp. for a core module. Show quoted text
> Convenience methods like that are on my radar. Not sure yet if they will > go into IO::Compress:Zip or in a new module.
Fine. So would you please restore the old behaviour then? Having CanonicalName => 0 as default would also be a way. The convenience methods could enable this option. And then you still had to find a solution, how to implement an addDirectory method when CanonicalName is true. ;-P Bernhard
I've decided to change the default for CanonicalName to false. Update on its way to CPAN. Paul