Bug #43680 for Tie-FlatFile-Array: Shouldn't crash if index beyond end

Thu Feb 26 16:01:10 2009 dmacks [...] netspace.org - Ticket created

Subject:	Shouldn't crash if index beyond end
Date:	Thu, 26 Feb 2009 16:00:19 -0500
To:	bug-Tie-FlatFile-Array [...] rt.cpan.org
From:	Daniel Macks <dmacks [...] netspace.org>

Tie::FlatFile::Array (version 0.03) on OS X 10.4 using fink's perl5.8.8 or apple's perl5.8.6 and fink-supplied Class::Accessor (version 0.31), I get unexpected and undesireable crash when reading beyond the last defined element: #!/sw/bin/perl5.8.8 -w use strict; use Data::Dumper; use Tie::FlatFile::Array; use Fcntl; tie my @tied, 'Tie::FlatFile::Array', 'data.file', O_RDWR | O_CREAT, 0644, { packformat => 'AL'}; $tied[1] = ["A", 10]; print Dumper($tied[0]); print Dumper($tied[1]); print Dumper($tied[2]); gives: $VAR1 = [ '', 0 ]; $VAR1 = [ 'A', 10 ]; Index 2: Invalid access of FileHandle=GLOB(0x182defc) at /sw/lib/perl5-core/5.8.8/darwin-thread-multi-2level/Data/Dumper.pm line 511 The autovivification of values before vs after one defined in the middle of the array should not be different, and especially should not crash my program! If the "uninitialized" state is empty (unpacked "zero", as $tied[0]), then reading any un-set element should give that same result (cf $tied[2]). A change like ...Array::FETCH: -croak("Index $index: Invalid access of $fh") unless $data; +$data = "\0"x$self->reclen unless $data; Actually, is "boolean meaning of $data" even correct there? Given that $data is an arbitrary packed struct, it could well be valid data that happens to be false. Seems like the intent is to check for (undef) i.e., reading beyond the end of the file? dan -- Daniel Macks dmacks@netspace.org http://www.netspace.org/~dmacks

Fri Feb 27 01:31:32 2009 paduille.4061.mumia.w+nospam [...] earthlink.net - Correspondence added

Subject:	Re: [rt.cpan.org #43680] Shouldn't crash if index beyond end
Date:	Fri, 27 Feb 2009 00:29:44 -0600
To:	bug-Tie-FlatFile-Array [...] rt.cpan.org
From:	"Mumia W.." <paduille.4061.mumia.w+nospam [...] earthlink.net>

On 02/26/2009 03:01 PM, dmacks@netspace.org via RT wrote: Show quoted text

> Thu Feb 26 16:01:10 2009: Request 43680 was acted upon. > Transaction: Ticket created by dmacks@netspace.org > Queue: Tie-FlatFile-Array > Subject: Shouldn't crash if index beyond end > Broken in: (no value) > Severity: (no value) > Owner: Nobody > Requestors: dmacks@netspace.org > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=43680 > > > > Tie::FlatFile::Array (version 0.03) on OS X 10.4 using fink's > perl5.8.8 or apple's perl5.8.6 and fink-supplied Class::Accessor > (version 0.31), I get unexpected and undesireable crash when reading > beyond the last defined element: > > #!/sw/bin/perl5.8.8 -w > > use strict; > use Data::Dumper; > use Tie::FlatFile::Array; > use Fcntl; > > tie my @tied, 'Tie::FlatFile::Array', > 'data.file', O_RDWR | O_CREAT, 0644, > { packformat => 'AL'}; > $tied[1] = ["A", 10]; > print Dumper($tied[0]); > print Dumper($tied[1]); > print Dumper($tied[2]); > > gives: > > $VAR1 = [ > '', > 0 > ]; > $VAR1 = [ > 'A', > 10 > ]; > Index 2: Invalid access of FileHandle=GLOB(0x182defc) at /sw/lib/perl5-core/5.8.8/darwin-thread-multi-2level/Data/Dumper.pm line 511 > > > The autovivification of values before vs after one defined in the > middle of the array should not be different, and especially should not > crash my program! If the "uninitialized" state is empty (unpacked > "zero", as $tied[0]), then reading any un-set element should give that > same result (cf $tied[2]). A change like ...Array::FETCH: > > -croak("Index $index: Invalid access of $fh") unless $data; > +$data = "\0"x$self->reclen unless $data; > > Actually, is "boolean meaning of $data" even correct there? Given that > $data is an arbitrary packed struct, it could well be valid data that > happens to be false. Seems like the intent is to check for (undef) > i.e., reading beyond the end of the file? > > dan >

Thank you very much for your bug report. Your program didn't crash; it aborted due to an exception (an error). Accessing beyond the end of the array is an error in Tie::FlatFile::Array. I'll update the documentation to make that explicit. Although normal Perl arrays support autovivication, tied arrays do not. It would be unwise for me to let Tie::FlatFile::Array automatically create array elements that would normally be invalid, because that can too easily allow the programmer to accidentally create gargantuan files. Almost always, when a program fetches an out-of-bounds array element, it is an error. What would happen if a record was 2,000 bytes long, and the programmer accidentally fetched index 40,000,000? An 80 gigabyte file would be created--accidentally. Going out of bounds is an error. It is very easy for the programmer to set the array (file) size by assigning to the highest element he or she wishes to use. I hope this helps and thanks again.

Fri Feb 27 01:31:32 2009 The RT System itself - Status changed from 'new' to 'open'

Fri Feb 27 02:03:05 2009 dmacks [...] netspace.org - Correspondence added

Subject:	Re: [rt.cpan.org #43680] Shouldn't crash if index beyond end
Date:	Fri, 27 Feb 2009 02:02:48 -0500
To:	"Mumia W.. via RT" <bug-Tie-FlatFile-Array [...] rt.cpan.org>
From:	Daniel Macks <dmacks [...] netspace.org>

On Fri, Feb 27, 2009 at 01:31:33AM -0500, Mumia W.. via RT wrote: Show quoted text

> > > > Tie::FlatFile::Array (version 0.03) on OS X 10.4 using fink's > > perl5.8.8 or apple's perl5.8.6 and fink-supplied Class::Accessor > > (version 0.31), I get unexpected and undesireable crash when reading > > beyond the last defined element: > > > > #!/sw/bin/perl5.8.8 -w > > > > use strict; > > use Data::Dumper; > > use Tie::FlatFile::Array; > > use Fcntl; > > > > tie my @tied, 'Tie::FlatFile::Array', > > 'data.file', O_RDWR | O_CREAT, 0644, > > { packformat => 'AL'}; > > $tied[1] = ["A", 10]; > > print Dumper($tied[0]); > > print Dumper($tied[1]); > > print Dumper($tied[2]); > > > > gives: > > > > $VAR1 = [ > > '', > > 0 > > ]; > > $VAR1 = [ > > 'A', > > 10 > > ]; > > Index 2: Invalid access of FileHandle=GLOB(0x182defc) at /sw/lib/perl5-core/5.8.8/darwin-thread-multi-2level/Data/Dumper.pm line 511

> > Thank you very much for your bug report. Your program didn't crash; it > aborted due to an exception (an error). Accessing beyond the end of the > array is an error in Tie::FlatFile::Array. I'll update the documentation > to make that explicit.

That would be a big help. Whatever the technical term is for what's happening, a runtime behavior of "abort the program with an error message to the user" was highly unexpected. Maybe it could return undef--that would make it more easily trappable and handlable by the calling program. Show quoted text

> Although normal Perl arrays support autovivication, tied arrays do not. > It would be unwise for me to let Tie::FlatFile::Array automatically > create array elements that would normally be invalid, because that can > too easily allow the programmer to accidentally create gargantuan files. > Almost always, when a program fetches an out-of-bounds array element, it > is an error. What would happen if a record was 2,000 bytes long, and the > programmer accidentally fetched index 40,000,000? An 80 gigabyte file > would be created--accidentally.

Simply reading an index beyond the end of a normal perl array actually doesn't appear to *create* that element, it merely returns an undef. my @a; $a[2] = "A"; my $b = $a[10]; print scalar(@a); gives 3 not 11. So returning the zero-filled struct but not writing it to the datafile would be the analogous behavior (and avoid the huge file problem). -- Daniel Macks dmacks@netspace.org http://www.netspace.org/~dmacks

Fri Feb 27 16:55:41 2009 paduille.4061.mumia.w+nospam [...] earthlink.net - Correspondence added

Subject:	Re: [rt.cpan.org #43680] Shouldn't crash if index beyond end
Date:	Fri, 27 Feb 2009 15:35:59 -0600
To:	bug-Tie-FlatFile-Array [...] rt.cpan.org
From:	"Mumia W.." <paduille.4061.mumia.w+nospam [...] earthlink.net>

On 02/27/2009 01:03 AM, dmacks@netspace.org via RT wrote: Show quoted text

> Queue: Tie-FlatFile-Array > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=43680 > > > On Fri, Feb 27, 2009 at 01:31:33AM -0500, Mumia W.. via RT wrote:

>>> Tie::FlatFile::Array (version 0.03) on OS X 10.4 using fink's >>> perl5.8.8 or apple's perl5.8.6 and fink-supplied Class::Accessor >>> (version 0.31), I get unexpected and undesireable crash when reading >>> beyond the last defined element: >>> >>> #!/sw/bin/perl5.8.8 -w >>> >>> use strict; >>> use Data::Dumper; >>> use Tie::FlatFile::Array; >>> use Fcntl; >>> >>> tie my @tied, 'Tie::FlatFile::Array', >>> 'data.file', O_RDWR | O_CREAT, 0644, >>> { packformat => 'AL'}; >>> $tied[1] = ["A", 10]; >>> print Dumper($tied[0]); >>> print Dumper($tied[1]); >>> print Dumper($tied[2]); >>> >>> gives: >>> >>> $VAR1 = [ >>> '', >>> 0 >>> ]; >>> $VAR1 = [ >>> 'A', >>> 10 >>> ]; >>> Index 2: Invalid access of FileHandle=GLOB(0x182defc) at /sw/lib/perl5-core/5.8.8/darwin-thread-multi-2level/Data/Dumper.pm line 511

>> Thank you very much for your bug report. Your program didn't crash; it >> aborted due to an exception (an error). Accessing beyond the end of the >> array is an error in Tie::FlatFile::Array. I'll update the documentation >> to make that explicit.

> > That would be a big help. Whatever the technical term is for what's > happening, a runtime behavior of "abort the program with an error > message to the user" was highly unexpected. Maybe it could return > undef--that would make it more easily trappable and handlable by the > calling program. >

I agree, and I'm about the upload the a version (0.04_01) of the module with this change--thanks. Show quoted text

>> Although normal Perl arrays support autovivication, tied arrays do not. >> It would be unwise for me to let Tie::FlatFile::Array automatically >> create array elements that would normally be invalid, because that can >> too easily allow the programmer to accidentally create gargantuan files. >> Almost always, when a program fetches an out-of-bounds array element, it >> is an error. What would happen if a record was 2,000 bytes long, and the >> programmer accidentally fetched index 40,000,000? An 80 gigabyte file >> would be created--accidentally.

> > Simply reading an index beyond the end of a normal perl array actually > doesn't appear to *create* that element, it merely returns an undef. > > my @a; > $a[2] = "A"; > my $b = $a[10]; > print scalar(@a); > > gives 3 not 11. So returning the zero-filled struct but not writing it > to the datafile would be the analogous behavior (and avoid the huge > file problem). >

Thank you. You are exactly right. Aborting because of an out-of-bounds state makes no sense in Perl.

Fri Feb 27 20:20:54 2009 paduille.4061.mumia.w+nospam.cpan [...] earthlink.net - Correspondence added

This has been fixed in Tie-FlatFile-Array-0.0401. Tie::FlatFile::Array::FETCH now returns undef() when it gets an out-of-bounds index.

Fri Feb 27 20:20:57 2009 paduille.4061.mumia.w+nospam.cpan [...] earthlink.net - Status changed from 'open' to 'resolved'