Bug #18633 for DBD-SQLite: t/07busy.t hangs

Mon Apr 10 08:39:02 2006 jdhedden [...] cpan.org - Ticket created

Subject:

t/07busy.t hangs

Using Perl 5.8.8 under Cygwin, DBD-SQLite 1.12 fails testing because test file 07busy.t hangs and must be terminated with ^C. Here is the test output: Show quoted text

> perl -Mblib t/07busy.t

1..8 # Running under perl version 5.008008 for cygwin # Current time local: Mon Apr 10 08:32:25 2006 # Current time GMT: Mon Apr 10 12:32:25 2006 # Using Test.pm version 1.25 ok 1 ok 2 ok 3 ok 4 ok 5 # insert failed : DBD::SQLite::db do failed: database is locked(5) at dbdimp.c line 398 at t/07busy.t line 29. ok 6 DBD::SQLite::db do failed: database is locked(5) at dbdimp.c line 398 at t/07busy.t line 71. Issuing rollback() for database handle being DESTROY'd without explicit disconnect(). ### At this point I have to terminate the test with ^C.

Tue Apr 11 03:17:06 2006 Guest - Correspondence added

I get 1..8 # Running under perl version 5.008004 for MSWin32 # Win32::BuildNumber 810 # Current time local: Tue Apr 11 00:17:21 2006 # Current time GMT: Tue Apr 11 07:17:21 2006 # Using Test.pm version 1.25 ok 1 ok 2 ok 3 ok 4 ok 5 # insert failed : DBD::SQLite::db do failed: database is locked(5) at dbdimp.c line 398 at t\07busy.t line 29. ok 6 DBD::SQLite::db do failed: handle 2 is owned by thread 22471c not current thread 1a6a9a4 (handles can't be shared between threads and your driver may need a CLONE method added) at t\07busy.t line 71. Attempt to free unreferenced scalar: SV 0x1b48f54.

Tue Apr 11 03:17:07 2006 The RT System itself - Status changed from 'new' to 'open'

Tue Apr 11 09:04:06 2006 jdhedden [...] cpan.org - Correspondence added

From:

jdhedden [...] cpan.org

Show quoted text

> DBD::SQLite::db do failed: handle 2 is owned by thread 22471c not > current thread 1a6a9a4 (handles can't be shared between threads and your > driver may > need a CLONE method added) at t\07busy.t line 71. > Attempt to free unreferenced scalar: SV 0x1b48f54.

To the implementer: The test uses 'fork'. Under Win32, 'fork' is implemented using threads and DBI is not thread safe. Therefore, you cannot use 'fork' in your tests.

Tue Apr 11 09:34:37 2006 jdhedden [...] cpan.org - Correspondence added

Show quoted text

> To the implementer: > The test uses 'fork'. Under Win32, 'fork' is implemented using threads > and DBI is not thread safe. Therefore, you cannot use 'fork' in your

tests. Correction. You just can't have open database handles when you do a fork. Attached is a version of t/07busy.t that should work under Win32. I does work on my system and solves the problem of the hang that I observed.

#!perl use Test; BEGIN { plan tests => 8 } use DBI; unlink('foo', 'foo-journal'); my $db = DBI->connect('dbi:SQLite:foo', '', '', { RaiseError => 1, PrintError => 0, AutoCommit => 0, }); my $db2 = DBI->connect('dbi:SQLite:foo', '', '', { RaiseError => 1, PrintError => 0, AutoCommit => 0, }); ok($db2->func(3000, 'busy_timeout')); ok($db->do("CREATE TABLE Blah ( id INTEGER, val VARCHAR )")); ok($db->commit); ok($db->do("INSERT INTO Blah VALUES ( 1, 'Test1' )")); my $start = time; eval { $db2->do("INSERT INTO Blah VALUES ( 2, 'Test2' )"); }; ok($@); if ($@) { print "# insert failed : $@"; $db2->rollback; } $db->commit; ok($db2->do("INSERT INTO Blah VALUES ( 2, 'Test2' )")); $db2->commit; $db2->disconnect; undef($db2); $db->disconnect; undef($db); # Now test that two processes can write at once, assuming we commit timely. pipe(READER, WRITER); my $pid = fork; if (!defined($pid)) { # fork failed skip("No fork here", 1); skip("No fork here", 1); exit; } if ($pid) { # Parent my $db2 = DBI->connect('dbi:SQLite:foo', '', '', { RaiseError => 1, PrintError => 0, AutoCommit => 0, }); my $line = <READER>; chomp($line); ok($line, "Ready"); $db2->func(10000, 'busy_timeout'); ok($db2->do("INSERT INTO Blah VALUES (4, 'Test4' )")); $db2->commit; wait; } else { # Child my $db = DBI->connect('dbi:SQLite:foo', '', '', { RaiseError => 1, PrintError => 0, AutoCommit => 0, }); $db->do("INSERT INTO Blah VALUES ( 3, 'Test3' )"); print WRITER "Ready\n"; sleep(5); $db->commit; } exit(0); # EOF

Wed Apr 12 12:47:40 2006 nilsonsfj [...] cpan.org - Correspondence added

On Mon Apr 10 08:39:02 2006, JDHEDDEN wrote: Show quoted text

> Using Perl 5.8.8 under Cygwin, DBD-SQLite 1.12 fails testing because > test file 07busy.t hangs and must be terminated with ^C. Here is the > test output: >

> > perl -Mblib t/07busy.t

> 1..8 > # Running under perl version 5.008008 for cygwin > # Current time local: Mon Apr 10 08:32:25 2006 > # Current time GMT: Mon Apr 10 12:32:25 2006 > # Using Test.pm version 1.25 > ok 1 > ok 2 > ok 3 > ok 4 > ok 5 > # insert failed : DBD::SQLite::db do failed: database is locked(5) at > dbdimp.c line 398 at t/07busy.t line 29. > ok 6 > DBD::SQLite::db do failed: database is locked(5) at dbdimp.c line 398 at > t/07busy.t line 71. > Issuing rollback() for database handle being DESTROY'd without explicit > disconnect(). > > ### At this point I have to terminate the test with ^C.

Same issue happens with AS Perl 5.8.8. t/07busy................ok 5/8DBD::SQLite::db do failed: handle 2 is owned by th read 235004 not current thread 2a96f54 (handles can't be shared between threads and your driver may need a CLONE method added) at t/07busy.t line 71. If I kill the respective perl process, the tests keep running and all of them manage to succeed (besides, obviously, the one that hanged). -Nilson Santos F. Jr.

Wed Apr 12 12:47:47 2006 nilsonsfj [...] cpan.org - Correspondence added

On Mon Apr 10 08:39:02 2006, JDHEDDEN wrote: Show quoted text

> Using Perl 5.8.8 under Cygwin, DBD-SQLite 1.12 fails testing because > test file 07busy.t hangs and must be terminated with ^C. Here is the > test output: >

> > perl -Mblib t/07busy.t

> 1..8 > # Running under perl version 5.008008 for cygwin > # Current time local: Mon Apr 10 08:32:25 2006 > # Current time GMT: Mon Apr 10 12:32:25 2006 > # Using Test.pm version 1.25 > ok 1 > ok 2 > ok 3 > ok 4 > ok 5 > # insert failed : DBD::SQLite::db do failed: database is locked(5) at > dbdimp.c line 398 at t/07busy.t line 29. > ok 6 > DBD::SQLite::db do failed: database is locked(5) at dbdimp.c line 398 at > t/07busy.t line 71. > Issuing rollback() for database handle being DESTROY'd without explicit > disconnect(). > > ### At this point I have to terminate the test with ^C.

Same issue happens with AS Perl 5.8.8. t/07busy................ok 5/8DBD::SQLite::db do failed: handle 2 is owned by th read 235004 not current thread 2a96f54 (handles can't be shared between threads and your driver may need a CLONE method added) at t/07busy.t line 71. If I kill the respective perl process, the tests keep running and all of them manage to succeed (besides, obviously, the one that hanged). -Nilson Santos F. Jr.

Sun May 07 16:20:29 2006 Guest - Correspondence added

From:

bart.lateur [...] pandora.be

On Tue Apr 11 09:34:37 2006, JDHEDDEN wrote: Show quoted text

> You just can't have open database handles when you do a > fork. Attached is a version of t/07busy.t that should work under Win32. > I does work on my system and solves the problem of the hang that I > observed.

I've been working on the same bug as you, and at first sight, your patch would seem to work. (I wonder why this is not applied yet?) What is really the problem is that parent and child got swapped. It is the parent, the original process, which should keep the original handle open, and the child which needs to open a secondary db handle. In addition, there's a minor problem with buffering in the pipe, and the other process gets the "go" signal too late. It should get it in the sleep(5) period. My patch (attached) stays closer to the original test file than your, so it's more obvious what I changed. And it is not much. So, in summary: what happens is that the child thread attempts to use the original db handle, which belongs to the parent, and it croaks. (On Unix, with real processes, this most likely is permissible, though I recommend against it: if you may only use a db handle in one process, it's best to let the original process, the parent, have it.) The parent, waiting for the signal from the child through the pipe to continue, never gets the signal, and waits forever. Ergo, it appears to hang.

--- t/07busy.t Sun Aug 8 16:02:23 2004 +++ t/07busy.pl Sun May 7 19:54:30 2006 @@ -51,8 +51,8 @@ skip("No fork here", 1); exit; } -if ($pid) { - # parent +unless ($pid) { + # child my $db2 = DBI->connect('dbi:SQLite:foo', '', '', { RaiseError => 1, @@ -68,7 +68,9 @@ exit; } +# parent $db->do("INSERT INTO Blah VALUES ( 3, 'Test3' )"); +select WRITER; $| = 1; select STDOUT; print WRITER "Ready\n"; sleep(5); $db->commit;

Sun May 07 19:25:52 2006 Guest - Correspondence added

From:

bart.lateur [...] pandora.be

On Sun May 07 16:20:29 2006, guest wrote: Show quoted text

> What is > really the problem is that parent and child got swapped. It is the > parent, the original process, which should keep the original handle > open, and the child which needs to open a secondary db handle.

Oh wait, I see now why parent and child were set up the way they were. The child still should be the one creating the new database handle, but in the original script the parent was chosen instead because that is then the process that outputs the test results. So I've moved things around a little. A new patch is attached. It's much closer now to one of JDHEDDEN.

--- t/07busy.t Sun Aug 8 16:02:23 2004 +++ t/07busy.pl Sun May 7 23:16:00 2006 @@ -49,28 +49,26 @@ # fork failed skip("No fork here", 1); skip("No fork here", 1); - exit; -} -if ($pid) { - # parent +} elsif (!$pid) { + # child my $db2 = DBI->connect('dbi:SQLite:foo', '', '', { RaiseError => 1, PrintError => 0, AutoCommit => 0, }); + $db2->do("INSERT INTO Blah VALUES ( 3, 'Test3' )"); + select WRITER; $| = 1; select STDOUT; + print WRITER "Ready\n"; + sleep(5); + $db2->commit; +} else { + # parent my $line = <READER>; chomp($line); ok($line, "Ready"); - $db2->func(10000, 'busy_timeout'); - ok($db2->do("INSERT INTO Blah VALUES (4, 'Test4' )")); - $db2->commit; - exit; + $db->func(10000, 'busy_timeout'); + ok($db->do("INSERT INTO Blah VALUES (4, 'Test4' )")); + $db->commit; + wait; } - -$db->do("INSERT INTO Blah VALUES ( 3, 'Test3' )"); -print WRITER "Ready\n"; -sleep(5); -$db->commit; - -

Mon May 08 05:37:56 2006 Guest - Correspondence added

From:

bart.lateur [...] pandora.be

On Sun May 07 19:25:52 2006, guest wrote: Show quoted text

> So I've moved things around a little. A new patch is attached.

I forgot one thing: if the other process dies, the parent will still wait forever. That can be remedied by closing the WRITER pipe on the READER side. A patch on the patch is attached.

--- t/07busy.pl Sun May 7 23:16:00 2006 +++ t/07busy2.pl Mon May 8 09:33:26 2006 @@ -64,6 +64,7 @@ $db2->commit; } else { # parent + close WRITER; my $line = <READER>; chomp($line); ok($line, "Ready");

Mon Jun 19 07:34:16 2006 Guest - Correspondence added

From:

Alexandr Ciornii <alexchorny [...] gmail.com>

Show quoted text

>> So I've moved things around a little. A new patch is attached.

Test with patches (including close_writer.patch) works for me. perl 5.8.7 (from port), CYGWIN 1.5.19.

Thu Sep 07 19:03:41 2006 msergeant [...] cpan.org - Correspondence added

Applying, thanks.

Thu Sep 07 19:03:42 2006 msergeant [...] cpan.org - Status changed from 'open' to 'resolved'