Bug #32061 for Test-Manifest: PATCH for VMS

Thu Jan 03 09:41:53 2008 cpan [...] pjedwards.co.uk - Ticket created

Subject:

PATCH for VMS

Hello and thanks for Test::Manifest 1.22 The default/standard file system on VMS is ODS-2 http://en.wikipedia.org/wiki/Files-11 ODS-2 only supports one period in a filename. http://h71000.www7.hp.com/doc/731FINAL/4506/4506pro_015.html (also see perlport#VMS) So to get the tests to run and pass on VMS I renamed the test files: 0.load.t => 0_load.t 1.get_test_files.t => 1_get_test_files.t 1.make_test_manifest.t => 1_make_test_manifest.t 99.pod.t => 99_pod.t I made the required changes to the manifests and other test files that contained the old filenames. ODS-2 is also a versioned file system, so when unlinking files, all versions must be unlinked. I've attached a diff generated on VMS using gnu diff, but because by default ODS-2 is not case sensitive you probably won't be able to apply the patch without some editing. I've not tested the patch on any other OS, but none of it is VMS specific - it should not break any other OS and may help some? Cheers, Peter (Stig) Edwards

Subject:

vms.patch

==== test-manifest-1_22/t/1_get_test_files.t#1 - test-manifest-1_22/t/1_get_test_files.t ==== --- t8271087190.tmp Thu Jan 3 09:16:21 2008 +++ test-manifest-1_22/t/1_get_test_files.t Thu Jan 3 09:13:12 2008 @@ -12,7 +12,7 @@ # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # my $expected = join " ", map { File::Spec->catfile( "t", $_ ) } qw( - 0.load.t 1.get_test_files.t 1.make_test_manifest.t + 0_load.t 1_get_test_files.t 1_make_test_manifest.t leading_space.t trailing_space.t ); @@ -34,13 +34,15 @@ { local $SIG{__WARN__} = sub { 1 }; -( unlink manifest_name() ) ? +1 while (unlink manifest_name()); + +( ! -e manifest_name() ) ? pass( "test_manifest unlinked") : fail( "test_manifest still around after unlink!"); my $string = get_t_files(); -ok( ! $string, "Nothing returned when test_manifest does not exist (scalar)" ); +ok( ! $string, "Nothing returned when test_manifest does not exist (scalar) '$string'" ); my @array = get_t_files(); @@ -53,13 +55,13 @@ copy( 'test_manifest_levels', manifest_name() ); my @expected = ( [] ); -$expected[1] = [ qw( 0.load.t 1.get_test_files.t pod_coverage.t) ]; -$expected[2] = [ qw( 0.load.t 1.get_test_files.t 1.make_test_manifest.t +$expected[1] = [ qw( 0_load.t 1_get_test_files.t pod_coverage.t) ]; +$expected[2] = [ qw( 0_load.t 1_get_test_files.t 1_make_test_manifest.t pod_coverage.t ) ]; -$expected[3] = [ qw( 0.load.t 1.get_test_files.t 1.make_test_manifest.t +$expected[3] = [ qw( 0_load.t 1_get_test_files.t 1_make_test_manifest.t leading_space.t pod_coverage.t trailing_space.t ) ]; -$expected[0] = [ qw( 0.load.t 1.get_test_files.t 1.make_test_manifest.t - leading_space.t pod_coverage.t trailing_space.t 99.pod.t ) ]; +$expected[0] = [ qw( 0_load.t 1_get_test_files.t 1_make_test_manifest.t + leading_space.t pod_coverage.t trailing_space.t 99_pod.t ) ]; foreach my $level ( 0 .. 3 ) { @@ -74,10 +76,10 @@ __END__ # this is a comment, then a blank line -0.load.t -1.get_test_files.t 1 -1.make_test_manifest.t 2 +0_load.t +1_get_test_files.t 1 +1_make_test_manifest.t 2 leading_space.t 2.9 pod_coverage.t 1 # with a comment trailing_space.t 3 # with a comment -99.pod.t 3.1 +99_pod.t 3.1 ==== test-manifest-1_22/t/1_make_test_manifest.t#1 - test-manifest-1_22/t/1_make_test_manifest.t ==== --- t8271087191.tmp Thu Jan 3 09:16:21 2008 +++ test-manifest-1_22/t/1_make_test_manifest.t Thu Jan 3 09:08:55 2008 @@ -4,9 +4,9 @@ use Test::Manifest qw(make_test_manifest); -my $test_manifest = 't/test_manifest'; +my $test_manifest = File::Spec->catfile('t','test_manifest'); -unlink $test_manifest; +1 while (unlink $test_manifest); ok( ! -e $test_manifest, 'test_manifest does not exit' ); make_test_manifest(); ==== test-manifest-1_22/t/include.t#1 - test-manifest-1_22/t/include.t ==== --- t8271087192.tmp Thu Jan 3 09:16:21 2008 +++ test-manifest-1_22/t/include.t Thu Jan 3 08:53:56 2008 @@ -17,9 +17,9 @@ # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # my @expected_tests = map { File::Spec->catfile( "t", $_ ) } qw( - 0.load.t - 99.pod.t - 1.get_test_files.t + 0_load.t + 99_pod.t + 1_get_test_files.t ); # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ==== test-manifest-1_22/t/include_in_manifest.txt#1 - test-manifest-1_22/t/include_in_manifest.txt ==== --- t8271087193.tmp Thu Jan 3 09:16:21 2008 +++ test-manifest-1_22/t/include_in_manifest.txt Thu Jan 3 08:55:59 2008 @@ -1,2 +1,2 @@ -99.pod.t -1.get_test_files.t \ No newline at end of file +99_pod.t +1_get_test_files.t \ No newline at end of file ==== test-manifest-1_22/test_manifest#1 - test-manifest-1_22/test_manifest. ==== --- t8271087194.tmp Thu Jan 3 09:16:22 2008 +++ test-manifest-1_22/test_manifest. Thu Jan 3 08:54:50 2008 @@ -1,7 +1,7 @@ # this is a comment, then a blank line -0.load.t -1.get_test_files.t -1.make_test_manifest.t +0_load.t +1_get_test_files.t +1_make_test_manifest.t leading_space.t trailing_space.t # comment at end of line \ No newline at end of file ==== test-manifest-1_22/test_manifest_levels#1 - test-manifest-1_22/test_manifest_levels. ==== --- t8271087195.tmp Thu Jan 3 09:16:22 2008 +++ test-manifest-1_22/test_manifest_levels. Thu Jan 3 08:58:18 2008 @@ -1,9 +1,9 @@ # this is a comment, then a blank line -0.load.t -1.get_test_files.t 1 -1.make_test_manifest.t 2 +0_load.t +1_get_test_files.t 1 +1_make_test_manifest.t 2 leading_space.t 2.9 pod_coverage.t 1 # with a comment trailing_space.t 3 # with a comment -99.pod.t 3.1 \ No newline at end of file +99_pod.t 3.1 \ No newline at end of file ==== test-manifest-1_22/test_manifest_with_include#1 - test-manifest-1_22/test_manifest_with_include. ==== --- t8271087196.tmp Thu Jan 3 09:16:22 2008 +++ test-manifest-1_22/test_manifest_with_include. Thu Jan 3 09:11:17 2008 @@ -1,2 +1,2 @@ -0.load.t +0_load.t ;include t/include_in_manifest.txt \ No newline at end of file

Thu Jan 03 13:32:26 2008 bdfoy [...] cpan.org - Correspondence added

On Thu Jan 03 09:41:53 2008, cpan@pjedwards.co.uk wrote: I don't have any recent experience with VMS, so I have some questions. Show quoted text

> The default/standard file system on VMS is ODS-2 > http://en.wikipedia.org/wiki/Files-11 > ODS-2 only supports one period in a filename. > http://h71000.www7.hp.com/doc/731FINAL/4506/4506pro_015.html > (also see perlport#VMS)

Both of those references show more than one period in the filename, as does perlport and perlvms. The distribution name is Test-Manifest-1.22.tar.gz, which has two periods in it. How does VMS handle that? Show quoted text

> So to get the tests to run and pass on VMS I renamed the test files:

Can you show me what happens with the stock distribution? Show quoted text

> ODS-2 is also a versioned file system, so when unlinking files, all > versions must be unlinked.

Is this a casual statement for future use, or part of the problem in this ticket?

Thu Jan 03 13:32:28 2008 The RT System itself - Status changed from 'new' to 'open'

Thu Jan 03 13:32:30 2008 bdfoy [...] cpan.org - Given to BDFOY

Thu Jan 03 17:50:27 2008 cpan [...] pjedwards.co.uk - Correspondence added

Thanks for the reply. Show quoted text

> > The default/standard file system on VMS is ODS-2 > > http://en.wikipedia.org/wiki/Files-11 > > ODS-2 only supports one period in a filename. > > http://h71000.www7.hp.com/doc/731FINAL/4506/4506pro_015.html > > (also see perlport#VMS)

> > Both of those references show more than one period in the filename, as > does perlport and perlvms.

ODS-5 does support more than one period in a filename, but ODS-2 does not. What I mean by filename is just the "temp.txt" bit of: u:[username.subdir1.subdir2]temp.txt (on VMS) /u/username/subdir1/subdir2/temp.txt (on nix) In the quote below taken from http://en.wikipedia.org/wiki/Files-11 "temp" would be the filename and "txt" the extension. ODS-2 is limited to eight levels of subdirectories, and only uppercase, alphanumeric names (plus the underscore, dash, and dollar sign) up to 39.39 characters (39 for the filename and another 39 for the extension). From http://h71000.www7.hp.com/doc/731FINAL/4506/4506pro_015.html Note that, although RMS on Alpha systems allows periods in a file name, files with such names can be created only on ODS-5 disks. John E. Malberg has recently add more support for some of the newer VMS filesystem features in 5.10.0 and it looks like the perlvms has been updated, I think any doc that mentions more than one period in a filename must be talking about ODS-5 disk - I'm wondering where you see more than one period in a filename? Also I remember a post on vmsperl (http://www.nntp.perl.org/group/perl.vmsperl/2007/12/msg14669.html) where Craig A. Berry and John talk about changing some of the new in 5.10.0 VMS::Filespec behaviour. From perlport: For ODS-2, filenames are in the format "name.extension;version". The maximum length for filenames is 39 characters, and the maximum length for extensions is also 39 characters. Version is a number from 1 to 32767. Valid characters are /[A-Z0-9$_-]/ . Show quoted text

> The distribution name is Test-Manifest-1.22.tar.gz, which has two > periods in it. How does VMS handle that?

It depends on what is creating the file, so in my case I'm using wget on VMS to grab the file: $ wget "http://search.cpan.org/CPAN/authors/id/B/BD/BDFOY/Test-Manifest-1.22.tar.gz" --17:01:23-- http://search.cpan.org/CPAN/authors/id/B/BD/BDFOY/Test-Manifest-1.22.tar.gz => `Test-Manifest-1_22_tar.gz' ... 17:01:24 (132.18 MB/s) - `Test-Manifest-1_22_tar.gz' saved [6930/6930] When wget writes a file with more than one period to ODS-2 disk on VMS it replaces all but the last period with underscore (I think SAMBA on VMS does the same, but I've seen different behaviour from other code such as FTP servers and (vms)tar, see below). I then use "gzip -d" on VMS and I end up with the file: Test-Manifest-1_22_tar. I then rename this to: Test-Manifest-1_22.tar I then use vmstar and it writes the test files with 2 periods to disk as: [.test-manifest-1_22.t]0.load_t [.test-manifest-1_22.t]1.get_test_files_t [.test-manifest-1_22.t]1.make_test_manifest_t [.test-manifest-1_22.t]99.pod_t Show quoted text

> Can you show me what happens with the stock distribution?

So if I don't rename the files above then they don't get run because they don't end in ".t", the output looks like: $ perl makefile.pl WARNING: LICENSE is not a known parameter. Checking if your kit is complete... Looks good 'LICENSE' is not a known MakeMaker parameter name. Writing Descrip.MMS for Test::Manifest $ mmk (this is make for VMS) cp [.lib]manifest.pm [.blib.lib.test]manifest.pm $ mmk test MCR <removed>.perl.perl-5_8_7_i64_root]perl.exe "-MExtUtils::Command::MM" "-e" "test_harness(0, '[.blib.lib]', '[.blib.arch]')" t/*.t t/include........... # Failed test 'Single string version of tests is right' # at t/include.t line 30. # got: '0.load.t 99.pod.t 1.get_test_files.t' # expected: '[.t]0.load.t [.t]99.pod.t [.t]1.get_test_files.t' # Failed test 'Array version of tests is right' # at t/include.t line 36. # Structures begin differing at: # $got->[0] = '0.load.t' # $expected->[0] = '[.t]0.load.t' # Looks like you failed 2 tests of 4. %SYSTEM-F-ABORT, abort dubious Test returned status 44 (wstat 1024, 0x400) (VMS status is 44) DIED. FAILED tests 3-4 Failed 2/4 tests, 50.00% okay t/leading_space..... ok t/pod_coverage...... ok t/trailing_space.... ok Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- t/include.t 44 1024 4 2 50.00% 3-4 Failed 1/4 test scripts, 75.00% okay. 2/7 subtests failed, 71.43% okay. Show quoted text

> > ODS-2 is also a versioned file system, so when unlinking files, all > > versions must be unlinked.

> > Is this a casual statement for future use, or part of the problem in > this ticket?

Sorry, the diff contains some changes to support cleaning up all versions of a file - I think if all tests are working then this is not needed, but I needed to add it to remove all versions of of the files created when trying to get all tests to pass. See http://perldoc.perl.org/perlvms.html#unlink-LIST for more info. The module Test::Portability::Files http://search.cpan.org/~saper/Test-Portability-Files-0.05/ can be used to check the portability across operating systems of the names of the files present in the distribution of a module. The tests use the advices given in "Files and Filesystems" in perlport: http://perldoc.perl.org/perlport.html#Files-and-Filesystems Also, try not to have non-word characters (except for .) in the names, and keep them to the 8.3 convention, for maximum portability, onerous a burden though this may appear. Many systems (DOS, VMS ODS-2) cannot have more than one . in their filenames. Many Thanks.

Thu Jan 03 21:32:34 2008 bdfoy [...] cpan.org - Correspondence added

From:

bdfoy [...] cpan.org

Okay, I think I have it now. This problem isn't isolated in Test::Manifest either, so I need to look at my other modules too. I'll fix them all in one shot within a couple of weeks. Thanks,

Fri Jan 04 05:19:54 2008 cpan [...] pjedwards.co.uk - Correspondence added

Thanks for caring about VMS, let me know if I can help with testing.

Sun Jan 06 15:21:22 2008 bdfoy [...] cpan.org - Correspondence added

From:

bdfoy [...] cpan.org

On Fri Jan 04 05:19:54 2008, cpan@pjedwards.co.uk wrote: Show quoted text

> Thanks for caring about VMS, let me know if I can help with testing.

I've uploaded Test::Manifest-1.22_01. If that works for you, I'll bump it up to a full release. Thanks,

Mon Jan 07 04:31:47 2008 cpan [...] pjedwards.co.uk - Correspondence added

On Sun Jan 06 15:21:22 2008, BDFOY wrote: Show quoted text

> On Fri Jan 04 05:19:54 2008, cpan@pjedwards.co.uk wrote:

> > Thanks for caring about VMS, let me know if I can help with testing.

> > I've uploaded Test::Manifest-1.22_01. If that works for you, I'll bump > it up to a full release. > > Thanks,

Thank you, the good news is that it works the first time: TEST-MANIFEST-1_22_01]$ mmk test MCR <removed>.perl.perl-5_8_7_i64_root]perl.exe "-MExtUtils::Command::MM" "-e" "test_harness(0, '[.blib.lib]', '[.bl ib.arch]')" t/*.t t/00load.................. ok t/01get_test_files........ # Test level is 0 # Test level is 1 # Test level is 2 # Test level is 3 ok t/01make_test_manifest.... ok t/99pod................... ok t/include................. ok t/leading_space........... ok t/pod_coverage............ ok t/trailing_space.......... ok All tests successful. Files=8, Tests=24, 7 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU) On subsequent runs it fails with: mmk test MCR <removed>.perl.perl-5_8_7_i64_root]perl.exe "-MExtUtils::Command::MM" "-e" "test_harness(0, '[.blib.lib]', '[.bl ib.arch]')" t/*.t t/00load.................. ok t/01get_test_files........ # Failed test 'Nothing returned when test_manifest does not exist (scalar)' # at t/01get_test_files.t line 43. # Failed test 'Nothing returned when test_manifest does not exist (list)' # at t/01get_test_files.t line 47. # Test level is 0 # Test level is 1 # Test level is 2 # Test level is 3 # Looks like you failed 2 tests of 13. %SYSTEM-F-ABORT, abort dubious Test returned status 44 (wstat 1024, 0x400) (VMS status is 44) DIED. FAILED tests 8-9 Failed 2/13 tests, 84.62% okay t/01make_test_manifest.... # Failed test 'test_manifest does not exit' # at t/01make_test_manifest.t line 10. # Looks like you failed 1 test of 2. %SYSTEM-F-ABORT, abort dubious Test returned status 44 (wstat 1024, 0x400) (VMS status is 44) DIED. FAILED test 1 Failed 1/2 tests, 50.00% okay t/99pod................... ok t/include................. ok t/leading_space........... ok t/pod_coverage............ ok t/trailing_space.......... ok Failed Test Stat Wstat Total Fail Failed List of Failed ------------------------------------------------------------------------------- t/01get_test_files.t 44 1024 13 2 15.38% 8-9 t/01make_test_manifest.t 44 1024 2 1 50.00% 1 Failed 2/8 test scripts, 75.00% okay. 3/24 subtests failed, 87.50% okay. Running 01get_test_file.t: 1..13 ok 1 - Single string version of tests is right ok 2 - Test file 0 has expected name ok 3 - Test file 1 has expected name ok 4 - Test file 2 has expected name ok 5 - Test file 3 has expected name ok 6 - Test file 4 has expected name ok 7 - test_manifest unlinked not ok 8 - Nothing returned when test_manifest does not exist (scalar) # Failed test 'Nothing returned when test_manifest does not exist (scalar)' # at [.t]01get_test_files.t line 43. not ok 9 - Nothing returned when test_manifest does not exist (list) # Failed test 'Nothing returned when test_manifest does not exist (list)' # at [.t]01get_test_files.t line 47. # Test level is 0 ok 10 - Level 0 version of tests is right # Test level is 1 ok 11 - Level 1 version of tests is right # Test level is 2 ok 12 - Level 2 version of tests is right # Test level is 3 ok 13 - Level 3 version of tests is right # Looks like you failed 2 tests of 13. Checking to see how many versions of t/test_manifest are created during a clean test run: $dir [.t]test_manifest. Directory <removed>.TEST-MANIFEST-1_22_01.T] TEST_MANIFEST.;2 TEST_MANIFEST.;1 Total of 2 files. $ type [.t]test_manifest.;2 00load.t ;include t/include_in_manifest.txt $ type [.t]test_manifest.;1 00load.t 01get_test_files.t 01make_test_manifest.t 99pod.t include.t leading_space.t pod_coverage.t trailing_space.t Because more than one version of t/test_manifest exists, when unlink is called on VMS only the most recent version is deleted/removed. If in t/01get_test_files.t the lines: ( unlink manifest_name() ) ? pass( "test_manifest unlinked") : fail( "test_manifest still around after unlink!"); are replaced with: 1 while ( unlink manifest_name() ); # http://perldoc.perl.org/perlvms.html#unlink-LIST ( ! -e manifest_name() ) ? pass( "test_manifest unlinked") : fail( "test_manifest still around after unlink!"); then the initial run of mmk test (make test) and subsequent runs all pass. Cheers.

Mon Jan 07 16:54:30 2008 bdfoy [...] cpan.org - Correspondence added

On Mon Jan 07 04:31:47 2008, cpan@pjedwards.co.uk wrote: Show quoted text

> 1 while ( unlink manifest_name() ); # > http://perldoc.perl.org/perlvms.html#unlink-LIST

I'll have to think about that. That seems like it could have some bad consequences in unusual situations. Does a glob on VMS get all of the versions? Do I have to write that in any particular way to get all of the versions? I saw the "Wildcard expansion" section in perlvms, but I don't have a VMS to play with.

Mon Jan 07 18:11:20 2008 cpan [...] pjedwards.co.uk - Correspondence added

On Mon Jan 07 16:54:30 2008, BDFOY wrote: Show quoted text

> On Mon Jan 07 04:31:47 2008, cpan@pjedwards.co.uk wrote: >

> > 1 while ( unlink manifest_name() ); # > > http://perldoc.perl.org/perlvms.html#unlink-LIST

> > I'll have to think about that. That seems like it could have some bad > consequences in unusual > situations.

You could only use it on VMS: if($^O eq 'VMS'){ # http://perldoc.perl.org/perlvms.html#unlink-LIST 1 while ( unlink manifest_name() ); } else { unlink manifest_name(); } Show quoted text

> Does a glob on VMS get all of the versions?

Yes, using VMS file spec when specifying * for the version number: perl -e "$,=""\n"";print glob('[.t]test_manifest.;*')" <removed>test-manifest-1_22_01.t]test_manifest.;2 <removed>test-manifest-1_22_01.t]test_manifest.;1 No, using *nix style glob without version number: perl -e "$,=""\n"";print glob('t/test_manifest*')" t/test_manifest. Yes, but don't use this mix of *nix and VMS version: (ugh) $ perl -e "$,=""\n"";print glob('t/test_manifest;*')" t/test_manifest.;2 t/test_manifest.;1 (Running 5.8.7) I don't think globs are the way to go here, they probably introduce more complications across platforms. I searched for "1 while (unlink" and found: http://www.perlmonks.org/?node_id=324636 http://www.perl.com/pub/a/2001/05/p5pdigest/THISWEEK-20010513.html Thanks.

Thu Jan 24 07:16:47 2008 bdfoy [...] cpan.org - Correspondence added

From:

bdfoy [...] cpan.org

I've uploaded Test::Manifest 1.022_02. It incorporates the file unlinking patch and also fixes up a path construction problem. I think this should take care of all of the problems for this issue. Thanks :)

Thu Jan 24 10:58:24 2008 cpan [...] pjedwards.co.uk - Correspondence added

From:

cpan [...] pjedwards.co.uk

I downloaded Test::Manifest 1.22_02 and tested on VMS with Perl 5.8.7. It passes all it's tests the first time and on subsequent runs. Thank you. $ perl makefile.pl WARNING: LICENSE is not a known parameter. Checking if your kit is complete... Looks good 'LICENSE' is not a known MakeMaker parameter name. Writing Descrip.MMS for Test::Manifest $ mmk cp [.lib]manifest.pm [.blib.lib.test]manifest.pm $ mmk test MCR <removed>perl.exe "-MExtUtils::Command::MM" "-e" "test_harness(0, '[.blib.lib]', '[.blib.arch]')" t/*.t t/00load.................. ok t/01get_test_files........ # Test level is 0 # Test level is 1 # Test level is 2 # Test level is 3 ok t/01make_test_manifest.... ok t/99pod................... ok t/include................. ok t/leading_space........... ok t/pod_coverage............ ok t/trailing_space.......... ok All tests successful. Files=8, Tests=24, 20 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU) $ mmk test MCR <removed>perl.exe "-MExtUtils::Command::MM" "-e" "test_harness(0, '[.blib.lib]', '[.blib.arch]')" t/*.t t/00load.................. ok t/01get_test_files........ # Test level is 0 # Test level is 1 # Test level is 2 # Test level is 3 ok t/01make_test_manifest.... ok t/99pod................... ok t/include................. ok t/leading_space........... ok t/pod_coverage............ ok t/trailing_space.......... ok All tests successful. Files=8, Tests=24, 15 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU) END

Sun Aug 31 15:08:10 2008 bdfoy [...] cpan.org - Status changed from 'open' to 'resolved'