Bug #50669 for Archive-Tar: Cannot untar CEESHEK/CGI-Application-Plugin-Authentication-0.14.tar.gz

Tue Oct 20 00:04:06 2009 ANDK [...] cpan.org - Ticket created

Subject:

Cannot untar CEESHEK/CGI-Application-Plugin-Authentication-0.14.tar.gz

The tarball CEESHEK/CGI-Application-Plugin-Authentication-0.14.tar.gz from CPAN can be untarred without problems with the tar program but trying the same with ptar gives quite a lot of garbage: (1) many directories, including the current one get a directory "PaxHeaders.11358" added which the normal tar program does not produce. (2) all paths longer than 100 bytes get truncated Let me know if you need further information.

Tue Oct 20 05:14:53 2009 jos [...] dwim.org - Correspondence added

Subject:	Re: [rt.cpan.org #50669] Cannot untar CEESHEK/CGI-Application-Plugin-Authentication-0.14.tar.gz
Date:	Tue, 20 Oct 2009 11:14:22 +0200
To:	bug-Archive-Tar [...] rt.cpan.org
From:	"Jos I. Boumans" <jos [...] dwim.org>

Hi Andreas, On Oct 20, 2009, at 6:04 AM, Andreas Koenig via RT wrote: Show quoted text

> The tarball CEESHEK/CGI-Application-Plugin-Authentication-0.14.tar.gz > from CPAN can be untarred without problems with the tar program but > trying the same with ptar gives quite a lot of garbage: > > (1) many directories, including the current one get a directory > "PaxHeaders.11358" added which the normal tar program does not > produce. > > (2) all paths longer than 100 bytes get truncated

Thanks for the report; it seems that this is a git'ism that uses the pax extended header features. It's basically the same as the gnu @LongLink implementation for long file names. Except different. From what I can tell, nothing in the wild really uses this, except git. Here's a bit more detail on where this comes from: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/ 2004-08/1973.html I'll check with Cees to see how he generated this tar file to make sure we're on the right track. If you find any more of these please let us know. Thanks, -- Jos Boumans "Whenever you find you are on the side of the majority, it is time to pause and reflect." - Mark Twain

Tue Oct 20 05:14:54 2009 The RT System itself - Status changed from 'new' to 'open'

Tue Oct 20 05:16:27 2009 jos [...] dwim.org - Correspondence added

CC:	bug-Archive-Tar [...] rt.cpan.org
Subject:	Re: [rt.cpan.org #50669] Cannot untar CEESHEK/CGI-Application-Plugin-Authentication-0.14.tar.gz
Date:	Tue, 20 Oct 2009 11:16:06 +0200
To:	ceeshek [...] gmail.com
From:	"Jos I. Boumans" <jos [...] dwim.org>

Hi Cees, On Oct 20, 2009, at 6:04 AM, Andreas Koenig via RT wrote: Show quoted text

> Tue Oct 20 00:04:06 2009: Request 50669 was acted upon. > Transaction: Ticket created by ANDK > Queue: Archive-Tar > Subject: Cannot untar CEESHEK/CGI-Application-Plugin- > Authentication-0.14.tar.gz > Broken in: 1.54 > Severity: Normal > Owner: Nobody > Requestors: ANDK@cpan.org > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=50669 > > > > The tarball CEESHEK/CGI-Application-Plugin-Authentication-0.14.tar.gz > from CPAN can be untarred without problems with the tar program but > trying the same with ptar gives quite a lot of garbage: > > (1) many directories, including the current one get a directory > "PaxHeaders.11358" added which the normal tar program does not > produce. > > (2) all paths longer than 100 bytes get truncated > > Let me know if you need further information.

We got this report that your module is tarred up in a way that Archive::Tar doesn't understand. Could you let us know, with as much detail as possible, how you generated this release tarball? OS versions, tar versions, commands invoked, etc. That way we can reproduce and hopefully fix the incompatibility. Thanks, -- Jos Boumans 'Real programmers use "cat > a.out"'

Tue Oct 20 05:26:37 2009 BINGOS [...] cpan.org - Correspondence added 5 min

http://www.opengroup.org/onlinepubs/009695399/utilities/pax.html#tag_04_100_13_03 Seems to explain some of the detail regarding pax extended headers.

Tue Oct 20 05:26:42 2009 BINGOS [...] cpan.org - Given to BINGOS

Tue Oct 20 20:23:58 2009 cees-perl [...] crtconsulting.ca - Correspondence added

Hi Jos, I kept getting reports from users that they could not open my tarballs on sun boxes that didn't have a gnu version of tar running on it (https://rt.cpan.org/Ticket/Display.html?id=23705). So I tried to tar up the file using an older method, in this case with the --posix flag to tar. I am not at the same computer right now so I can't look at my command history but I believe I ran it like this: tar czvf CGI-Application-Plugin-Authentication-0.14.tar.gz --posix CGI-Application-Plugin-Authentication-0.14 This was on a Ubuntu 9.04 box tar --version tar (GNU tar) 1.20 gzip --version gzip 1.3.12 If you know of a better way to tar up the disto so that it is compatible with older non-Gnu unixes I would really appreciate some help with that. Cheers, Cees Hek

Wed Oct 21 07:10:06 2009 jos [...] dwim.org - Correspondence added

Subject:	Re: [rt.cpan.org #50669] Cannot untar CEESHEK/CGI-Application-Plugin-Authentication-0.14.tar.gz
Date:	Wed, 21 Oct 2009 13:08:58 +0200
To:	bug-Archive-Tar [...] rt.cpan.org
From:	"Jos I. Boumans" <jos [...] dwim.org>

Hi Cees, On Oct 21, 2009, at 2:23 AM, Cees Hek via RT wrote: Show quoted text

> I kept getting reports from users that they could not open my tarballs > on sun boxes that didn't have a gnu version of tar running on it > (https://rt.cpan.org/Ticket/Display.html?id=23705). So I tried to tar > up the file using an older method, in this case with the --posix > flag to > tar. > > I am not at the same computer right now so I can't look at my command > history but I believe I ran it like this: > > tar czvf CGI-Application-Plugin-Authentication-0.14.tar.gz --posix > CGI-Application-Plugin-Authentication-0.14 > > This was on a Ubuntu 9.04 box > > tar --version > tar (GNU tar) 1.20 > > gzip --version > gzip 1.3.12 > > > If you know of a better way to tar up the disto so that it is > compatible > with older non-Gnu unixes I would really appreciate some help with > that.

Take a look at the archive::tar faq here, particularly the one about winzip or other non-posix clients and the use of the PREFIX header: http://search.cpan.org/~bingos/Archive-Tar-1.54/lib/Archive/Tar.pm# $Archive::Tar::DO_NOT_USE_PREFIX http://search.cpan.org/~bingos/Archive-Tar-1.54/lib/Archive/ Tar.pm#FAQ Basically, there's 3 different implementations of supporting file names greater than 100 characters: 1) Posix' PREFIX header (supports paths up to 255) 2) GNU's Extended Header feature (a @LongLink entry in the archive) 3) Pax' use of the pax header (a PaxHeader entry in the archive) From what we've gathered in the wild and on documentations, some support eachothers formats and all in varying degrees of backwards compatibility: Posix seems to support 1 & 3 Gnu seems to support 1 & 2 The safe way is the use of 1), which is archive tar's default, but it does limit support for older gnu tars (1.13 and before wont work). In practice this means you're best of writing archives in Posix compatible format (which with the --posix flag for gnutar seems to be format 3, not 1). The perl tar program that comes with Archive::Tar however will do exactly that (it's built for portability), so you could use that instead: * Install the latest Archive::Tar from CPAN * And then create the tar file like you normally would, but with ptar instead: $ ptar -czvf CGI-Application-Plugin-Authentication-0.14.tar.gz CGI-Application-Plugin-Authentication-0.14 * You can then verify that both gnu tar & posix tar extract this archive correctly I know it's a hassle, but it's one of the only ways we've found to support archives which contains files with paths > 100 characters. It's a mess out there :( Let me know how it goes, -- Jos Boumans 'Real programmers use "cat > a.out"'