Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the Storable CPAN distribution.

Report information
The Basics
Id: 31794
Status: open
Priority: 0/
Queue: Storable
People
Owner: Nobody in particular
Requestors: mark_young [...] hotmail.com
Cc:
AdminCc:
Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)

History Show all quoted text
  Fri Dec 21 06:10:13 2007 mark_young [...] hotmail.com - Ticket created
Subject: Reading 32-bit Storable files on 64-bit system
Date: Fri, 21 Dec 2007 11:09:42 +0000
To: <bug-storable [...] rt.cpan.org>
From: Mark Young <mark_young [...] hotmail.com>
Hi, I have a perl application that has been running for many years on a 32-bit Debian system using perl v5.8.4 and Storable v2.14. That application has stored thousands of files in Storable format using the store() command. I'm attempting to migrate to a new 64-bit system using perl v5.8.8 and Storable v2.18. I've found that retrieve() on the 64-bit system can not read the 32-bit systems' Storables. The error it produces is Byte order is not compatible at blib/lib/Storable.pm (autosplit into blib/lib/auto/Storable/_retrieve.al) line 380, at ./try line 37 The problem appears to be that retrieve() on the 64-bit system is not happy with the Storables longsize, ptrsize and byteorder. If I write out a Storable on each system and use the Storable::file_magic to read the details of the header I get these values that differ between the two systems. 32 bit system: 'hdrsize' => 15, 'version' => '2.7', 'longsize' => 4, 'ptrsize' => 4, 'byteorder' => '1234', 64 bit system: 'hdrsize' => 19, 'version' => '2.7', 'longsize' => 8, 'ptrsize' => 8, 'byteorder' => '12345678', I have subsequently found out, thanks to the perlmonks, that if I write a storable using nstore() then the resultant file can be written by either then 32-bit or 64-bit system and can be read by either the 32-bit or 64-bit system. That's useful, but unfortunately I have many thousands of files that would need to be converted. So the reason I am writing to explain this is to ask whether you would consider that the 64-bit system should be able to read the store() file written out by the 32-bit system? Shouldn't the 64-bit system be able to read the header of the Storable file and thus know how to read the contents? My ideal situation would be to be able to enable the 64-bit system to read the 32-bit systems' storables, preferably in a manner that means it could read Storables written out by either a 32-bit system or a 64-bit system, in the same way you get that capability if you'd used nstore all along. Please let me know what you think? Best Regards,Mark Young BTW - The incompatibility is the same between a 32-bit perl v5.8.8 Storable v2.18 system and a 64-bit perl v5.8.8 Storable v2.18 system.
  Wed Jul 21 12:19:33 2010 walkeraj [...] gmail.com - Correspondence added
From: walkeraj [...] gmail.com
This bug also affects me. I receive the exact same error when attempting to retrieve() a file that was created on a 32-bit system. ['Byte order is not compatible at blib/lib/Storable.pm (autosplit into blib/lib/auto/Storable/_retrieve.al) line 380'] Setting $Storable::interwork_56_64bit=1 does not solve the issue.Please let me know how I can provide further information to resolve this. On Fri Dec 21 06:10:13 2007, mark_young@hotmail.com wrote: Show quoted text
> > Hi, > > I have a perl application that has been running for many years on a > 32-bit Debian system using perl v5.8.4 and Storable v2.14. That > application has stored thousands of files in Storable format using > the store() command. I'm attempting to migrate to a new 64-bit > system using perl v5.8.8 and Storable v2.18. I've found that > retrieve() on the 64-bit system can not read the 32-bit systems' > Storables. The error it produces is > > Byte order is not compatible at blib/lib/Storable.pm (autosplit into > blib/lib/auto/Storable/_retrieve.al) line 380, at ./try line 37 > > The problem appears to be that retrieve() on the 64-bit system is not > happy with the Storables longsize, ptrsize and byteorder. If I > write out a Storable on each system and use the > Storable::file_magic to read the details of the header I get these > values that differ between the two systems. > > 32 bit system: 'hdrsize' => 15, 'version' => '2.7', 'longsize' > => 4, 'ptrsize' => 4, 'byteorder' => '1234', > > 64 bit system: 'hdrsize' => 19, 'version' => '2.7', 'longsize' > => 8, 'ptrsize' => 8, 'byteorder' => '12345678', > > I have subsequently found out, thanks to the perlmonks, that if I > write a storable using nstore() then the resultant file can be > written by either then 32-bit or 64-bit system and can be read by > either the 32-bit or 64-bit system. That's useful, but > unfortunately I have many thousands of files that would need to be > converted. So the reason I am writing to explain this is to ask > whether you would consider that the 64-bit system should be able to > read the store() file written out by the 32-bit system? Shouldn't > the 64-bit system be able to read the header of the Storable file > and thus know how to read the contents? > > My ideal situation would be to be able to enable the 64-bit system to > read the 32-bit systems' storables, preferably in a manner that > means it could read Storables written out by either a 32-bit system > or a 64-bit system, in the same way you get that capability if > you'd used nstore all along. > > Please let me know what you think? > > Best Regards,Mark Young > BTW - The incompatibility is the same between a 32-bit perl v5.8.8 > Storable v2.18 system and a 64-bit perl v5.8.8 Storable v2.18 > system.
  Wed Jul 21 12:19:33 2010 The RT System itself - Status changed from 'new' to 'open'
  Thu Jul 22 04:42:57 2010 mark_young [...] hotmail.com - Correspondence added
Subject: RE: [rt.cpan.org #31794] Reading 32-bit Storable files on 64-bit system
Date: Thu, 22 Jul 2010 08:42:46 +0000
To: <bug-storable [...] rt.cpan.org>
From: Mark Young <mark_young [...] hotmail.com>
Hi Andrew, I think I may have inadvertently deleted a mail from you. Did you mail me directly about this problem? I ended up writing a script that found and converted all of my perl programs & storables to nstore format. I never heard any response to filing that bug. Cheers, Mark Show quoted text
> Subject: [rt.cpan.org #31794] Reading 32-bit Storable files on 64-bit system > From: bug-Storable@rt.cpan.org > To: mark_young@hotmail.com > Date: Wed, 21 Jul 2010 12:19:33 -0400 > > <URL: https://rt.cpan.org/Ticket/Display.html?id=31794 > > > This bug also affects me. I receive the exact same error when > attempting to retrieve() a file that was created on a 32-bit system. > ['Byte order is not compatible at blib/lib/Storable.pm (autosplit into > blib/lib/auto/Storable/_retrieve.al) line 380'] > > Setting $Storable::interwork_56_64bit=1 does not solve the issue.Please > let me know how I can provide further information to resolve this. > > On Fri Dec 21 06:10:13 2007, mark_young@hotmail.com wrote:
> > > > Hi, > > > > I have a perl application that has been running for many years on a > > 32-bit Debian system using perl v5.8.4 and Storable v2.14. That > > application has stored thousands of files in Storable format using > > the store() command. I'm attempting to migrate to a new 64-bit > > system using perl v5.8.8 and Storable v2.18. I've found that > > retrieve() on the 64-bit system can not read the 32-bit systems' > > Storables. The error it produces is > > > > Byte order is not compatible at blib/lib/Storable.pm (autosplit into > > blib/lib/auto/Storable/_retrieve.al) line 380, at ./try line 37 > > > > The problem appears to be that retrieve() on the 64-bit system is not > > happy with the Storables longsize, ptrsize and byteorder. If I > > write out a Storable on each system and use the > > Storable::file_magic to read the details of the header I get these > > values that differ between the two systems. > > > > 32 bit system: 'hdrsize' => 15, 'version' => '2.7', 'longsize' > > => 4, 'ptrsize' => 4, 'byteorder' => '1234', > > > > 64 bit system: 'hdrsize' => 19, 'version' => '2.7', 'longsize' > > => 8, 'ptrsize' => 8, 'byteorder' => '12345678', > > > > I have subsequently found out, thanks to the perlmonks, that if I > > write a storable using nstore() then the resultant file can be > > written by either then 32-bit or 64-bit system and can be read by > > either the 32-bit or 64-bit system. That's useful, but > > unfortunately I have many thousands of files that would need to be > > converted. So the reason I am writing to explain this is to ask > > whether you would consider that the 64-bit system should be able to > > read the store() file written out by the 32-bit system? Shouldn't > > the 64-bit system be able to read the header of the Storable file > > and thus know how to read the contents? > > > > My ideal situation would be to be able to enable the 64-bit system to > > read the 32-bit systems' storables, preferably in a manner that > > means it could read Storables written out by either a 32-bit system > > or a 64-bit system, in the same way you get that capability if > > you'd used nstore all along. > > > > Please let me know what you think? > > > > Best Regards,Mark Young > > BTW - The incompatibility is the same between a 32-bit perl v5.8.8 > > Storable v2.18 system and a 64-bit perl v5.8.8 Storable v2.18 > > system.
> > >
  Mon Aug 30 19:29:13 2010 bitcard_kiddm [...] ghctechnologies.com - Correspondence added
From: bitcard_kiddm [...] ghctechnologies.com
I have the same problem as Mark on Perl 5.10 as I move into the 64-bit Linux world. At the very least the error message should be changed. Byte order means little-endian vs. big-endian. Both my Windows and Linux boxes are running on Intel chips and therefore have the same byte order. The issue is really 32-bit vs. 64-bit pointers and longs. Although network order is the proscribed workaround, switching to it degrades performance when operating on the same platform. And according to the Storeable documentation, doubles get stored as text instead of the exact machine representation with the consequence of losing +-Inf and NaN (often used to represent missing data). Since nearly all platforms now use the IEEE Standard 754 standard to store floating point numbers, it is annoying to unnecessarily lose precision when switching between platforms. It seems like reading 32-bit Storable files on 64-bit platforms would be relatively easy to implement and a great convenience. In fact I also think 32-bit platforms ought to make an effort to read 64-bit Storable files too. Yes, an exception will have to be thrown if a 64-bit integer that can not fit in 32-bits is encountered. And some very large data structures will use pointers that are too big. But those large structures aren't going to fit into the 32-bit Perl's memory space anyhow.
  Thu Feb 02 11:24:49 2017 RURBAN [...] cpan.org - Correspondence added
I'm working on a version which can read 32 from 64bit and vice versa on most platforms, when the double size and kind is the same. Only on BE (big-endian) you cannot read LE files, you need to store netorder via nstore there. It's still in work, but passes all the interload tests. https://github.com/rurban/Storable/commits/64read32 -- Reini Urban