Skip Menu |

This queue is for tickets about the DBD-SQLite CPAN distribution.

Report information
The Basics
Id: 50503
Status: resolved
Priority: 0/
Queue: DBD-SQLite

People
Owner: Nobody in particular
Requestors: logrado [...] gmail.com
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in:
  • 1.25
  • 1.26_04
Fixed in: 1.14



Subject: Segfault with fts3
The attached script fails on 1.25 and 1.26_04. It does one of three things: - segfaults - gives a double free error - hangs without any cpu activity Which of the above three it does depends on the exact script (adding or removing a print line can change its behavior), but it always fails. Here are a couple sample error messages: *** glibc detected *** double free or corruption (!prev): 0x000000000083c480 *** Abort (core dumped) Segmentation fault (core dumped) The database doesn't appear to be corrupted after the failure -- running an integrity check with the sqlite3 command line binary says the db is okay. The equivalent code without the fts3 module works fine. I did a quick test with the DBD::SQLite packaged with the latest Ubuntu (1.14), and that version seems to work.
Subject: fts3.pl
#!/usr/bin/perl # #use lib '/tmp/DBD-SQLite-1.26_04/blib/lib'; #use lib '/tmp/DBD-SQLite-1.26_04/blib/arch'; use lib '/home/ewing/tmp/DBD-SQLite-1.25/blib/lib'; use lib '/home/ewing/tmp/DBD-SQLite-1.25/blib/arch'; use DBI; use Data::Dumper; my $DB = "/tmp/test1.db"; sub insert_data { my($dbh, $inc_num, $date, $text) = @_; # "OR REPLACE" isn't standard SQL, but it sure is useful my $sth = $dbh->prepare('INSERT OR REPLACE INTO incident_fts (incident_id, all_text) VALUES (?, ?)'); $sth->execute($inc_num, $text) || die "execute failed\n"; $dbh->commit; } unlink $DB; my $dbh = DBI->connect("dbi:SQLite:$DB"); $dbh->{AutoCommit} = 0; $dbh->{RaiseError} = 1; $dbh->do(<<EOF); CREATE VIRTUAL TABLE incident_fts USING fts3 (incident_id VARCHAR, all_text VARCHAR, TOKENIZE simple) EOF $dbh->commit; insert_data($dbh, '595', time(), "sample text foo bar baz"); insert_data($dbh, '595', time(), "sample text foo bar baz"); insert_data($dbh, '595', time(), "sample text foo bar baz"); insert_data($dbh, '595', time(), "sample text foo bar baz"); $dbh->commit; my $sth = $dbh->prepare("SELECT * FROM incident_fts WHERE all_text MATCH 'bar'"); $sth->execute(); while (my $row = $sth->fetchrow_hashref("NAME_lc")) { print Dumper $row; } $dbh->commit;
This is the first of a few copies of email messages that responded to the FTS3 issue on sqlite-users@sqlite.org or privately by a list member. Mostly verbatim but trimmed for RT. Show quoted text
-------- Original Message -------- Subject: Re: [sqlite] test DBD::SQLite 1.26_05 - foreign keys! Date: Thu, 15 Oct 2009 22:35:35 -0500 From: P Kishor <punk.kish@gmail.com> Reply-To: punkish@eidesis.org, General Discussion of SQLite Database <sqlite- users@sqlite.org> To: General Discussion of SQLite Database <sqlite-users@sqlite.org> References: <4AD6C3A7.5080702@darrenduncan.net> On Thu, Oct 15, 2009 at 1:39 AM, Darren Duncan <darren@darrenduncan.net> wrote:
> I am pleased to announce that DBD::SQLite (Self Contained RDBMS in a Perl DBI > Driver) version 1.26_05 has been released on CPAN (by Adam Kennedy). > > P.S. DBD::SQLite has at least 1 known bug, also in version 1.25, with regard to > full-text search (FTS3); there is an included new failing test, which currently > is set to skip so the CPAN testers don't issue fails,
I tried to create a db with FTS3 and got the following perl(12546) malloc: *** error for object 0x1eb160: Non-aligned pointer being freed (2) *** set a breakpoint in malloc_error_break to debug perl(12546) malloc: *** error for object 0x870200: double free *** set a breakpoint in malloc_error_break to debug perl(12546) malloc: *** error for object 0x1ea668: Non-aligned pointer being freed *** set a breakpoint in malloc_error_break to debug Segmentation fault Is the above the FTS3 related error? -- Puneet Kishor http://www.punkish.org
Subject: DBD::SQLite with FTS3 finally works. here is how...
This is the second of a few copies of email messages that responded to the FTS3 issue on sqlite-users@sqlite.org or privately by a list member. Mostly verbatim but trimmed for RT. Show quoted text
-------- Original Message -------- Subject: [sqlite] DBD::SQLite with FTS3 finally works. here is how... Date: Fri, 16 Oct 2009 00:23:56 -0500 From: P Kishor <punk.kish@gmail.com> Reply-To: punkish@eidesis.org, General Discussion of SQLite Database <sqlite- users@sqlite.org> To: General Discussion of SQLite Database <sqlite-users@sqlite.org> I started with DBD::SQLite 1.26_05 and went back as much as DBD::SQLite 1.21. Kept on getting segfaults. Any further back, and I got DBD::SQLite 1.14 which bundled FTS2, so that wasn't good as well. Finally, the amazing Audrey Tang to rescue. I downloaded the classic DBD::SQLite::Amalgamation 3.6.12 (http://search.cpan.org/~audreyt/DBD-SQLite-Amalgamation-3.6.1.2/), replaced the .c and .h files in it with those from the latest tarball of original source 3.6.19, built it, and it works just fine, thank you very much. Of course, now my problem is that I have to somehow convince my shared web host provider to also do the above rigmaroo or wait until the default version from CPAN starts working again. FTS3, in my personal view, is far more useful and important than getting Foreign Keys support. That is because the latter can be emulated in the app code, but the former can't. Will be grateful when FTS3 is working again flawlessly in the publicly available CPAN version. -- Puneet Kishor http://www.punkish.org
Subject: Dan Kennedy's patch for Segfault with fts3
This is the third of a few copies of email messages that responded to the FTS3 issue on sqlite-users@sqlite.org or privately by a list member. Mostly verbatim but trimmed for RT. Show quoted text
-------- Original Message -------- Subject: Re: [sqlite] test DBD::SQLite 1.26_05 - foreign keys! Date: Sat, 17 Oct 2009 00:55:23 +0700 From: Dan Kennedy <dan@sqlite.org> To: Darren Duncan <darren@darrenduncan.net> References: <4AD6C3A7.5080702@darrenduncan.net> <cdf6db500910152035k2dd074c5j3bcd6d88275649c8@mail.gmail.com> <4AD7F883.2070002@darrenduncan.net> On Oct 16, 2009, at 11:37 AM, Darren Duncan wrote:
> Looks similar. This is the official bug report that we already > have, which was > reported on Oct 14, and on which the failing/skipping test is based: > > http://rt.cpan.org/Public/Bug/Display.html?id=50503
Hi Darren, Can you attach this patch to the ticket for me please. With some disclaimer that no testing was done apart from running the fts3.pl script. I can't figure out how to create a CPAN login... Much obliged, Dan.
--- 2/DBD-SQLite-1.26_04/dbdimp.c 2009-10-06 13:16:49.000000000 +0700 +++ DBD-SQLite-1.26_04/dbdimp.c 2009-10-17 00:43:18.000000000 +0700 @@ -154,12 +154,12 @@ sqlite_db_rollback(dbh, imp_dbh); } - while ( (pStmt = sqlite3_next_stmt(imp_dbh->db, 0))!=0 ) { - sqlite3_finalize(pStmt); - } - if (sqlite3_close(imp_dbh->db) == SQLITE_BUSY) { /* active statements! */ + while ( (pStmt = sqlite3_next_stmt(imp_dbh->db, 0))!=0 ) { + sqlite3_finalize(pStmt); + } + sqlite3_close(imp_dbh->db); warn("closing dbh with active statement handles"); } imp_dbh->db = NULL;
I just tested Dan Kennedy's patch against blead (same as 1.26_05 except for 50503 not skipping and with Dan's patch applied). While it was successful in making t/rt_50503_fts3.t pass, it also caused several other DBD::SQLite tests to fail. The results of the test suite with the patch applied are below. So the patch was not committed into blead at this time. ---------- darren-duncans-power-mac-g4-3:DBD_SQLite darrenduncan$ make test PERL_DL_NONLAZY=1 /Volumes/Spare/Perl/perl_5_10_1/bin/perl "- MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/01_compile.t ........................... 1/4 # $DBI::VERSION=1.609 t/01_compile.t ........................... ok t/02_logon.t ............................. 1/19 # sqlite_version=3.6.19 t/02_logon.t ............................. ok t/03_create_table.t ...................... ok t/04_insert.t ............................ ok t/05_select.t ............................ ok t/06_tran.t .............................. ok t/07_error.t ............................. ok t/08_busy.t .............................. ok t/09_create_function.t ................... 1/55 # Failed test 'no warnings' # at inc/Test/NoWarnings.pm line 45. # There were 2 warning(s) # Previous test 27 'SELECT noop(1.0625)' # closing dbh with active statement handles at t/09_create_function.t line 122. # at t/09_create_function.t line 122 # # ---------- # Previous test 54 'SELECT noop(1.0625)' # closing dbh with active statement handles at t/09_create_function.t line 122. # at t/09_create_function.t line 122 # # Looks like you failed 1 test of 55. t/09_create_function.t ................... Dubious, test returned 1 (wstat 256, 0x100) Failed 1/55 subtests t/10_create_aggregate.t .................. ok t/12_unicode.t ........................... ok t/13_create_collation.t .................. ok t/14_progress_handler.t .................. ok t/15_ak_dbd.t ............................ ok t/16_column_info.t ....................... ok t/17_createdrop.t ........................ ok t/18_insertfetch.t ....................... ok t/19_bindparam.t ......................... ok t/20_blobs.t ............................. ok t/21_blobtext.t .......................... ok t/22_listfields.t ........................ ok t/23_nulls.t ............................. ok t/24_numrows.t ........................... ok t/25_chopblanks.t ........................ ok t/26_commit.t ............................ ok t/27_metadata.t .......................... ok t/28_schemachange.t ...................... ok t/29_cppcomments.t ....................... ok t/30_auto_rollback.t ..................... ok t/31_bind_weird_number_param.t ........... ok t/32_inactive_error.t .................... 1/4 closing dbh with active statement handles at t/32_inactive_error.t line 16. t/32_inactive_error.t .................... ok t/33_non_latin_path.t .................... ok t/34_online_backup.t ..................... ok t/35_table_info.t ........................ ok t/36_hooks.t ............................. ok t/37_regexp.t ............................ ok t/rt_15186_prepcached.t .................. ok t/rt_21406_auto_finish.t ................. ok t/rt_25371_asymmetric_unicode.t .......... ok t/rt_25460_numeric_aggregate.t ........... ok t/rt_25924_user_defined_func_unicode.t ... ok t/rt_27553_prepared_cache_and_analyze.t .. ok t/rt_29058_group_by.t .................... ok t/rt_29629_sqlite_where_length.t ......... ok t/rt_31324_full_names.t .................. ok t/rt_32889_prepare_cached_reexecute.t .... 1/32 # Failed test 'Got a warning' # at t/rt_32889_prepare_cached_reexecute.t line 110. # got: '2' # expected: '1' # Failed test 'No warnings' # at t/rt_32889_prepare_cached_reexecute.t line 126. # got: '3' # expected: '1' # Failed test 'No warnings' # at t/rt_32889_prepare_cached_reexecute.t line 138. # got: '4' # expected: '1' # Failed test 'No warnings' # at t/rt_32889_prepare_cached_reexecute.t line 156. # got: '5' # expected: '1' # Failed test 'No warnings' # at t/rt_32889_prepare_cached_reexecute.t line 177. # got: '6' # expected: '1' # Looks like you failed 5 tests of 32. t/rt_32889_prepare_cached_reexecute.t .... Dubious, test returned 5 (wstat 1280, 0x500) Failed 5/32 subtests t/rt_36836_duplicate_key.t ............... ok t/rt_36838_unique_and_bus_error.t ........ ok t/rt_40594_nullable.t .................... ok t/rt_48393_debug_panic_with_commit.t ..... skipped: set $ENV{TEST_DBD_SQLITE_WITH_DEBUGGER} to enable this test t/rt_50503_fts3.t ........................ ok Test Summary Report ------------------- t/09_create_function.t (Wstat: 256 Tests: 55 Failed: 1) Failed test: 55 Non-zero exit status: 1 t/rt_32889_prepare_cached_reexecute.t (Wstat: 1280 Tests: 32 Failed: 5) Failed tests: 18, 20, 23, 27, 31 Non-zero exit status: 5 Files=51, Tests=825, 23 wallclock secs ( 1.01 usr 0.61 sys + 10.79 cusr 2.21 csys = 14.62 CPU) Result: FAIL Failed 2/51 test programs. 6/825 subtests failed. make: *** [test_dynamic] Error 255
Update: Dan Kennedy said by private email that: Roger put a ticket here that identifies the issue: http://www.sqlite.org/src/info/1751725cd6
As of a few days ago, updates to DBD::SQLite in version control have made the test based on this ticket pass, without breaking any other tests. The issue appears to be solved, but it is important to have further testing by any stakeholders.
I grabbed the latest version from subversion, and now I am unable to reproduce the bug. So from my point of view, this is fixed.
Marked as resolved with 1.26_06 On Wed Oct 28 19:11:21 2009, sendai wrote: Show quoted text
> I grabbed the latest version from subversion, and now I am unable to > reproduce the bug. So from my point of view, this is fixed.