Skip Menu |

This queue is for tickets about the CDB_File CPAN distribution.

Report information
The Basics
Id: 70025
Status: resolved
Priority: 0/
Queue: CDB_File

People
Owner: Nobody in particular
Requestors: mjy [...] pobox.com
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 0.97
Fixed in: (no value)



Subject: bus errors when .cdb used over NFS is replaced
When CDB_File has a .cdb file open in an nfs-mounted directory, it will crash with a bus error when the file is replaced or deleted. This is due to mmap() losing the mapping when the file handle becomes invalid (output is using 0.97): Program received signal SIGBUS, Bus error. memcpy () at ../sysdeps/x86_64/memcpy.S:102 102 ../sysdeps/x86_64/memcpy.S: No such file or directory. in ../sysdeps/x86_64/memcpy.S Current language: auto The current source language is "auto; currently asm". (gdb) bt #0 memcpy () at ../sysdeps/x86_64/memcpy.S:102 #1 0x00007ffff046b8b5 in cdb_read (c=0x61c8f0, buf=0x7fffffffe880 "RQ{\222j%\230\r", len=8, pos=228074858) at CDB_File.xs:221 #2 0x00007ffff046bc5e in cdb_findnext (c=0x61c8f0, key=0x638f30 "name:549389", len=11) at CDB_File.xs:289 #3 0x00007ffff046da83 in XS_CDB_File_FETCH (my_perl=0x7fffffffe880, cv=0x63a5a8) at CDB_File.xs:473 #4 0x00007ffff7b1d5f5 in Perl_pp_entersub (my_perl=0x602010) at pp_hot.c:2891 #5 0x00007ffff7b1bb36 in Perl_runops_standard (my_perl=0x602010) at run.c:40 #6 0x00007ffff7ac071c in S_run_body (my_perl=<value optimized out>) at perl.c:2431 #7 perl_run (my_perl=<value optimized out>) at perl.c:2349 #8 0x0000000000400d3c in main () When -DHASMMAP is removed from the Makefile, i.e. CDB_File is built without mmap() support, CDB_File crashes (more gently) with: Read of CDB_File failed: Protocol error at (eval 5) line 9. While the mmap() error probably cannot be avoided, it would allow better recovery on the Perl side if CDB_File didn't crash with a bus error (e.g. die'ing instead would allow catching with eval and reopening the file). How to reproduce (untested): - mount a directory over NFS - put a .cdb file in that directory - write a Perl script that opens the file with CDB_File (tie or TIEHASH() interface) and reads one key every second in an infinite loop) - replace the file on the server, e.g. by rsync'ing a new .cdb file over the original one - the Perl script will crash with a bus error
Possible solutions: - make HASMMAP code path a run-time option (it could be turned off by default if statfs() reveals that the file is in an NFS-mounted directory) - catch SIGBUS and try to recover in CDB_File.xs if the cause was an invalid file handle (recovery being a CDB protocol error that can be caught in Perl as in the non-mmap() case) - ignore and document in perldoc that use over NFS requires the non-mmap() code (~6x slower) for robust operation, it's not a frequently occurring issue after all
RT-Send-CC: michael.stevens [...] dianomi.com
On Thu Aug 04 16:49:17 2011, http://mjy.myopenid.com/ wrote: Show quoted text
> Possible solutions: > > - make HASMMAP code path a run-time option (it could be turned off by > default if statfs() reveals that the file is in an NFS-mounted
directory) Show quoted text
> > - catch SIGBUS and try to recover in CDB_File.xs if the cause was an > invalid file handle (recovery being a CDB protocol error that can be > caught in Perl as in the non-mmap() case) > > - ignore and document in perldoc that use over NFS requires the > non-mmap() code (~6x slower) for robust operation, it's not a
frequently Show quoted text
> occurring issue after all
Interested in this bug, hoping RT will let me add myself to CC.
RT-Send-CC: michael.stevens [...] dianomi.com
There's 2 people on this ticket. I've worked with files on NFS. In general, it's not really a good idea to be running database files over NFS. As I understand the problem, the issue (NFS or not) is when the file is open for read/mmap and the file handle disappears. My guess would be that you can't remove the inode that's open on a non-NFS partition until the file is closed, correct? If mmap supports it, we could probably trap the error and try re-opening the file if we see it disappear. My guess is that this would be the preference. Thoughts?
Ticket migrated to github as https://github.com/toddr/CDB_File/issues/7