Skip Menu |

This queue is for tickets about the File-Temp CPAN distribution.

Report information
The Basics
Id: 22702
Status: resolved
Priority: 0/
Queue: File-Temp

People
Owner: Nobody in particular
Requestors: tom@eborcom.com (no email address)
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: 0.17
Fixed in: 0.19



Subject: Strange interaction between File::Temp and DBD::SQLite
I get the following output from the attached tests: 1..4 DBD::SQLite::db do failed: database is locked(5) at dbdimp.c line 271 at sqlite_temp.pl line 38. not ok 1 - CREATE TABLE works for tempfile() with 2 LHS and UNLINK # Failed test (sqlite_temp.pl at line 42) DBD::SQLite::db do failed: database is locked(5) at dbdimp.c line 271 at sqlite_temp.pl line 38. not ok 2 - CREATE TABLE works for tempfile() with 2 LHS # Failed test (sqlite_temp.pl at line 42) ok 3 - CREATE TABLE works for tempfile() with 1 list context LHS ok 4 - CREATE TABLE works for tmpnam() # Looks like you failed 2 tests of 4. Basically, calling tempfile() in subtly different ways does something strange with file locking that upsets SQLite. It's not just a case of having an open filehandle, as the fourth test shows. I don't know enough to figure out if the fundamental problem exists in DBD::SQLite, the sqlite C code, File::Temp or elsewhere. But it strikes me as odd that small changes in calls to File::Temp have very different results. I see this happen with Perl 5.8.7 (on FreeBSD 5.x) and 5.8.8 (on FreeBSD 6.x). Please let me know if I can help resolve this problem. Tom
Subject: sqlite_and_file_temp.pl
#!perl use strict; use warnings; use DBI (); use File::Temp qw(tempfile tmpnam); use Test::More tests => 4; { my($fh, $filename) = tempfile(UNLINK => 1); # FAILS db_do($filename, 'tempfile() with 2 LHS and UNLINK'); } { my($fh, $filename) = tempfile; # FAILS db_do($filename, 'tempfile() with 2 LHS'); } { my(undef, $filename) = tempfile; # WORKS db_do($filename, 'tempfile() with 1 list context LHS'); } { my $filename = tmpnam(); # WORKS open (my $fh, '>', $filename); db_do($filename, 'tmpnam()'); } sub db_do { my $filename = shift; my $test_type = shift; my $dbh = DBI->connect( "DBI:SQLite:dbname=$filename", '', '', { RaiseError => 1 } ); eval { $dbh->do( "CREATE TABLE foo ( id INTEGER NOT NULL, PRIMARY KEY (id) )" ); }; ok ! $@, "CREATE TABLE works for $test_type"; $dbh->disconnect(); }
From: TJENNESS [...] cpan.org
On Mon Oct 30 13:10:12 2006, tom@eborcom.com wrote: Show quoted text
> Basically, calling tempfile() in subtly different ways does something > strange with file locking that upsets SQLite. It's not just a case of > having an open filehandle, as the fourth test shows. > > I see this happen with Perl 5.8.7 (on FreeBSD 5.x) and 5.8.8 (on FreeBSD > 6.x). > > Please let me know if I can help resolve this problem. > > Tom
The problem is that in the first two examples the file is open and in the last two examples you just have a filename. If you close($fh) before passing the filename to SQLite the tests pass fine. The third one works because you implicitly close the file by not assigning the filehandle to a variable. This would seem to be an SQLite issue rather than a File::Temp issue.
From: tom [...] eborcom.com
Thanks for your prompt reply. On Mon Oct 30 17:41:17 2006, TJENNESS wrote: Show quoted text
> The third one works because you implicitly close the file > by not assigning the filehandle to a variable.
True, but the fourth test has an open filehandle yet passes. It's this inconsistent behaviour that I don't understand. Apologies for not pointing this out more clearly earlier. I would expect the fourth to fail for the same reason as the first and second. I notice File::Temp uses sysopen() so I've written a fifth test that uses this instead of open(). This also passes, unlike the first and second.
#!perl use strict; use warnings; use DBI (); use Fcntl; use File::Temp qw(tempfile tmpnam); use Test::More tests => 5; { my($fh, $filename) = tempfile(UNLINK => 1); # FAILS db_do($filename, 'tempfile() with 2 LHS and UNLINK'); } { my($fh, $filename) = tempfile; # FAILS db_do($filename, 'tempfile() with 2 LHS'); } { my(undef, $filename) = tempfile; # WORKS db_do($filename, 'tempfile() with 1 list context LHS'); } { my $filename = tmpnam(); # WORKS open (my $fh, '>', $filename); db_do($filename, 'tmpnam() with open()'); } { my $filename = tmpnam(); # WORKS my $flags = O_CREAT | O_EXCL | O_RDWR; sysopen(my $fh, $filename, $flags, 0600); db_do($filename, 'tmpnam() with sysopen()'); } sub db_do { my $filename = shift; my $test_type = shift; my $dbh = DBI->connect( "DBI:SQLite:dbname=$filename", '', '', { RaiseError => 1 } ); eval { $dbh->do( "CREATE TABLE foo ( id INTEGER NOT NULL, PRIMARY KEY (id) )" ); }; ok ! $@, "CREATE TABLE works for $test_type"; $dbh->disconnect(); }
Subject: Re: [rt.cpan.org #22702] Strange interaction between File::Temp and DBD::SQLite
Date: Wed, 14 Nov 2007 15:29:47 -1000
To: bug-File-Temp [...] rt.cpan.org
From: Tim Jenness <t.jenness [...] jach.hawaii.edu>
More on this. The problem is that File::Temp uses O_EXLOCK when opening the files with sysopen. This means that that file is locked and no-one else can open it (and SQLite does try to get a lock). The cases that work, involve the filehandle being closed, releasing the lock and then passing the filename to SQLite which can open it without a problem. Your last sysopen works because tempfile() does not use solely those flags. I suppose my main question is whether you are using the filehandle elsewhere? It seems a bit dangerous to have a filehandle open on a SQLite database since you presumably don't want to write to the database file independently. Tim On Oct 31, 2006, at 2:07 AM, tom@eborcom.com via RT wrote: Show quoted text
> > Queue: File-Temp > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=22702 > > > Thanks for your prompt reply. > > On Mon Oct 30 17:41:17 2006, TJENNESS wrote:
>> The third one works because you implicitly close the file >> by not assigning the filehandle to a variable.
> > True, but the fourth test has an open filehandle yet passes. It's > this > inconsistent behaviour that I don't understand. Apologies for not > pointing this out more clearly earlier. >
-- Tim Jenness Joint Astronomy Centre
Subject: Re: [rt.cpan.org #22702] Strange interaction between File::Temp and DBD::SQLite
Date: Thu, 15 Nov 2007 13:59:46 +0000
To: Tim Jenness via RT <bug-File-Temp [...] rt.cpan.org>
From: Tom Hukins <tom [...] eborcom.com>
On Wed, Nov 14, 2007 at 08:30:58PM -0500, Tim Jenness via RT wrote: Show quoted text
> I suppose my main question is whether you are using the filehandle > elsewhere?
I don't use the filehandle elsewhere, but I want to keep it in scope to retain the exclusive lock to avoid other processes accidentally useing the same filename. Arguably I'm taking an overly paranoid approach, but I want the filehandle even though I never use it directly as it preserves the exclusive lock. But, as you mention, I don't really want the exclusive lock: I want to share the lock with SQLite, but I shouldn't expect File::Temp to do this. (From memory: it's over half a year since I've worked with the tests that caused me to open this ticket.) Tom
This is now fixed in that file locking can be disabled (which will allow DBD::SQLite to take a lock). See also RT #28397