Bug #41329 for Archive-Tar: VMS relative filenames being mangled.

Sun Nov 30 19:03:14 2008 malmberg [...] Encompasserve.org - Ticket created

Subject:

VMS relative filenames being mangled.

Test 04_resolved_issues was failing on VMS because of unconditional translations of "." to "_" in directorires. This mis-handled the case of ".." as a relative directory. Attached patch fixes.

Subject:

archive_tar_pm.gdiff

Download archive_tar_pm.gdiff
application/octet-stream 607b

Message body not shown because it is not plain text.

Thu Dec 04 13:04:34 2008 kane [...] cpan.org - Correspondence added

RT-Send-CC:

perl5-porters [...] perl.org, vmsperl [...] perl.org

Thanks, applied

Thu Dec 04 13:04:38 2008 The RT System itself - Status changed from 'new' to 'open'

Thu Dec 04 13:04:41 2008 kane [...] cpan.org - Status changed from 'open' to 'resolved'

Fri Dec 05 09:36:02 2008 wb8tyw [...] qsl.net - Correspondence added

CC:	perl5-porters [...] perl.org, vmsperl [...] perl.org
Subject:	Re: [rt.cpan.org #41329] VMS relative filenames being mangled.
Date:	Fri, 05 Dec 2008 08:35:23 -0600
To:	bug-Archive-Tar [...] rt.cpan.org
From:	"John E. Malmberg" <wb8tyw [...] qsl.net>

Jos Boumans via RT wrote: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=41329 > >

Unfortuantely while that patch fixes the Archive::Tar tests, it seems to have broken the CPANPLUS tests, which I just found out last night. Craig's alternative patch will work for VMS perl in ODS-2 mode only, which is the mode that Perl on VMS has traditionally been. I do not yet have a solution that will work for ODS-2 and ODS-5 properly. regards, -John wb8tyw@qsl.net Personal Opinion Only

Fri Dec 05 09:36:05 2008 The RT System itself - Status changed from 'resolved' to 'open'

Fri Dec 05 11:34:08 2008 craigberry [...] mac.com - Correspondence added

CC:	bug-Archive-Tar [...] rt.cpan.org, perl5-porters [...] perl.org, vmsperl [...] perl.org
Subject:	Re: [rt.cpan.org #41329] VMS relative filenames being mangled.
Date:	Fri, 05 Dec 2008 10:33:52 -0600
To:	"John E. Malmberg" <wb8tyw [...] qsl.net>
From:	"Craig A. Berry" <craigberry [...] mac.com>

On Dec 5, 2008, at 8:35 AM, John E. Malmberg wrote: Show quoted text

> Jos Boumans via RT wrote:

>> <URL: http://rt.cpan.org/Ticket/Display.html?id=41329 >

> > Unfortuantely while that patch fixes the Archive::Tar tests, it > seems to have broken the CPANPLUS tests, which I just found out last > night.

The code in question is dealing with directory components that have been split out of a Unix-format path during the process of restoring a tar archive. They will shortly be recombined into a native path so that directory can be created on disk. Calling VMS::Filespec::vmspath on each of the components is a very big hammer and it doesn't surprise me it causes problems elsewhere. Show quoted text

> Craig's alternative patch will work for VMS perl in ODS-2 mode only, > which is the mode that Perl on VMS has traditionally been.

My patch prevented the special '..' directory from being converted to '__' at the point where we otherwise convert dots to underscores in directory names on VMS. That's all it did, and I don't see anything that would prevent that from working on any volume format. The salient chunk of the patch at http://rt.cpan.org/Public/Bug/Display.html?id=39933 is just: - map tr/\./_/, @dirs if ON_VMS; + map $_ ne '..' && tr/\./_/, @dirs if ON_VMS; Show quoted text

> I do not yet have a solution that will work for ODS-2 and ODS-5 > properly.

I find this nomenclature confusing. As you well know, the level number of the On-Disk Structure is an indication of volume format. There is no such thing as an ODS-2 or ODS-5 "mode." It is true that some Unix emulation features of the C run-time and the filesystem are only supported on ODS-5 volumes. If what you are talking about here is preserving dots in directory names rather than treating them as directory delimiters, I think it would be clearer to say so in so many words. Show quoted text

________________________________________ Craig A. Berry mailto:craigberry@mac.com "... getting out of a sonnet is much more difficult than getting in." Brad Leithauser

Fri Dec 05 19:33:40 2008 malmberg [...] Encompasserve.org - Correspondence added

Subject:	Re: [rt.cpan.org #41329] VMS relative filenames being mangled.
Date:	Fri, 05 Dec 2008 18:33:21 -0600
To:	bug-Archive-Tar [...] rt.cpan.org
From:	"John E. Malmberg" <malmberg [...] Encompasserve.org>

craigberry@mac.com via RT wrote: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=41329 > > > > On Dec 5, 2008, at 8:35 AM, John E. Malmberg wrote:

Show quoted text

> I find this nomenclature confusing. As you well know, the level > number of the On-Disk Structure is an indication of volume format. > There is no such thing as an ODS-2 or ODS-5 "mode." It is true that > some Unix emulation features of the C run-time and the filesystem are > only supported on ODS-5 volumes. If what you are talking about here > is preserving dots in directory names rather than treating them as > directory delimiters, I think it would be clearer to say so in so many > words.

I was trying to use a general short hand. The supporting of multiple dots in directories is a feature of the ODS-5 volume and while it is used by the Unix emulation, it is also used for native VMS filenames. There are two cases that need to be handled: 1. Traditional VMS Perl mode of replacing extra '.' characters in directories and filenames with '_'. 2. Supporting the full file and path names that are available on an ODS-5 in both Unix and VMS format. In this case we do not want to be changing the '.'s to underscores, but when we convert them to a VMS directory format we have to prefix them with '^' in addition to putting the directory brackets around this. vmspath() should do this, but the VMS pathify() code used by perl is using a different algorithm for processing directories than vmsify() does, and vmspath() calls pathify() before calling vmsify(). This also can cause problems with symbolic link handling. So it looks like the patch that is needed is to append a '/' to the fragment and call vmsify() on it to convert it to the proper format for feeding back into catpath(). To have Archive::Tar determine when to do the substitution for the '.' would require that it also know if the extended character set feature is enabled. This is an independent feature from the Unix emulation mode. I am working on a fix for VMS pathify() code, but Archive::Tar would not be able to rely on it. I will not have that fix ready before tomorrow evening. After that I will run a test with using vmsify() instead of vmspath. -John

Sun Dec 07 08:51:20 2008 malmberg [...] Encompasserve.org - Correspondence added

From:

malmberg [...] Encompasserve.org

Since the existing vmspath() does not handle all the cases needed, attached is a replacement patch that uses vmsify() to do the proper conversion or escaping of characters as needed. Please revert the older patch before applying. A fixed vmspath() has also been submitted to blead-perl.

Download archive_tar_pm.gdiff
application/octet-stream 1.2k

Message body not shown because it is not plain text.

Tue Dec 09 08:05:46 2008 jos [...] dwim.org - Correspondence added

CC:	"Craig A. Berry" <craigberry [...] mac.com>
Subject:	Re: [rt.cpan.org #41329] VMS relative filenames being mangled.
Date:	Tue, 9 Dec 2008 14:07:03 +0100
To:	bug-Archive-Tar [...] rt.cpan.org
From:	"Jos I. Boumans" <jos [...] dwim.org>

On Dec 7, 2008, at 2:51 PM, John E. Malmberg via RT wrote: Show quoted text

> Queue: Archive-Tar > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=41329 > > > Since the existing vmspath() does not handle all the cases needed, > attached is a replacement patch that uses vmsify() to do the proper > conversion or escaping of characters as needed. > > Please revert the older patch before applying.

Hi John, this is the patch I ended up applying on top of the previous one -- just some stylistic differences: --- lib/Archive/Tar.pm (revision 5944) +++ lib/Archive/Tar.pm (local) @@ -689,9 +689,11 @@ } } - ### '.' is the directory delimiter on VMS, of which the first one - ### has to be escaped/changed by vmspath - map { $_ = VMS::Filespec::vmspath($_) } @dirs if ON_VMS; + ### '.' is the directory delimiter on VMS, which has to be escaped + ### or changed to '_' on vms. vmsify is used, because older versions + ### of vmspath do not handle this properly. + ### Must not add a '/' to an empty directory though. + map { length() ? VMS::Filespec::vmsify($_.'/') : $_ } @dirs if ON_VMS; my ($cwd_vol,$cwd_dir,$cwd_file) = File::Spec->splitpath( $cwd ); @@ -713,7 +715,8 @@ $cwd_vol, File::Spec->catdir( @cwd, @dirs ), '' ); - ### catdir() returns undef if the path is longer than 255 chars on VMS + ### catdir() returns undef if the path is longer than 255 chars on + ### older VMS systems. unless ( defined $dir ) { $^W && $self->_error( qq[Could not compose a path for '$dirs'\n] ); return;ror( qq[Could not compose a path for '$dirs'\n] ); return; -- Jos Boumans 'Real programmers use "cat > a.out"'

Sat Dec 13 11:04:13 2008 kane [...] cpan.org - Status changed from 'open' to 'resolved'