Skip Menu |

This queue is for tickets about the DBD-SQLite CPAN distribution.

Report information
The Basics
Id: 84974
Status: resolved
Priority: 0/
Queue: DBD-SQLite
People
Owner: Nobody in particular
Requestors: aaron.schweiger [...] gmail.com
Cc:
AdminCc:
Bug Information
Severity: (no value)
Broken in:
  • 1.35
  • 1.37
Fixed in: (no value)

History Show all quoted text
  Wed May 01 13:24:05 2013 aaron.schweiger [...] gmail.com - Ticket created
Subject: Custom Aggregates and Group By Resulting in BusError or Seg Fault
It seems like a bug in SQLite's load extension interface (via sqlite_create_aggregate). A bit of poking suggests that the problem manifests in the destructor (under Perl_sv_clean_objs). So far, I have reproduced the bug on linux (5.10.0) and Strawberry perl (5.16.3) running on Win XP (but not ActivePerl). I actually get two different error messages depending on if I save to disk or not. See discussion at: http://perlmonks.org/?node_id=1031582 Self-contained example attached.
Subject: BrokenSQLite.pl

Message body is not shown because it is too large.

  Fri May 03 02:20:29 2013 aaron.schweiger [...] gmail.com - Correspondence added
From: aaron.schweiger [...] gmail.com
Looked at the issue some more. I re-ran the broken code through "valgrind -v" The result: ==3386== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 2 from 2) ==3386== ==3386== 1 errors in context 1 of 1: ==3386== Invalid write of size 8 ==3386== at 0x68341A8: XS_DBD__SQLite__db_selectall_arrayref (SQLite.xsi:160) ==3386== by 0x6008AC3: XS_DBI_dispatch (DBI.xs:3715) ==3386== by 0x4916F3: Perl_pp_entersub (pp_hot.c:3046) ==3386== by 0x489222: Perl_runops_standard (run.c:41) ==3386== by 0x435BD7: perl_run (perl.c:2350) ==3386== by 0x41D11C: main (perlmain.c:120) ==3386== Address 0x5ba93b8 is 8 bytes inside a block of size 4,080 free'd ==3386== at 0x4C2A82E: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==3386== by 0x48765E: Perl_av_extend (av.c:153) ==3386== by 0x4B6048: Perl_stack_grow (scope.c:38) ==3386== by 0x50C43D: Perl_pp_sort (pp_sort.c:1694) ==3386== by 0x489222: Perl_runops_standard (run.c:41) ==3386== by 0x430BA8: Perl_call_sv (perl.c:2647) ==3386== by 0x6837DFC: sqlite_db_aggr_finalize_dispatcher (dbdimp.c:1751) ==3386== by 0x6847596: sqlite3VdbeMemFinalize (sqlite3.c:57831) ==3386== by 0x688EDDB: sqlite3VdbeExec (sqlite3.c:69665) ==3386== by 0x6896E50: sqlite3_step (sqlite3.c:62394) ==3386== by 0x683A8B1: sqlite_st_fetch (dbdimp.c:995) ==3386== by 0x6833D64: dbdxst_fetchall_arrayref.isra.0 (Driver_xst.h:112) (The call to "sort" in the finalize method seems to cause the problem.) I also ran it through Perl 5.17.1. The issue was triggered in this version as well -- but stack trace was different: ==11503== by 0x4C41E8: Perl_stack_grow (scope.c:38) ==11503== by 0x497E37: S_pushav (pp_hot.c:330) ==11503== by 0x49A38C: Perl_pp_rv2av (pp_hot.c:918) ==11503== by 0x4978A2: Perl_runops_standard (run.c:42) ==11503== by 0x433DE8: Perl_call_sv (perl.c:2719) ==11503== by 0x6836680: sqlite_db_aggr_finalize_dispatcher If I had to guess -- the shared calls to "Perl_stack_grow" (found in both the stack traces above) is what is causing the problem to manifest. Looks to me like corruption of the stack. Hopefully this will help in tracing down this issue. Thanks in advance!
  Thu May 23 01:09:14 2013 aaron.schweiger [...] gmail.com - Correspondence added
From: aaron.schweiger [...] gmail.com
It turns out that bug is resolved with a change to DBI's Driver.xst. (See: https://rt.cpan.org/Public/Bug/Display.html?id=85562) 160c160,162 < ST(0) = dbdxst_fetchall_arrayref(sth, &PL_sv_undef, (maxrows_svp) ? *maxrows_svp : &PL_sv_undef); --- Show quoted text
> SV *tmp = dbdxst_fetchall_arrayref(sth, &PL_sv_undef, (maxrows_svp) ? *maxrows_svp : &PL_sv_undef); > SPAGAIN; > ST(0) = tmp;
663c665,667 < ST(0) = dbdxst_fetchall_arrayref(sth, slice, batch_row_count); --- Show quoted text
> SV *tmp = dbdxst_fetchall_arrayref(sth, slice, batch_row_count); > SPAGAIN; > ST(0) = tmp;
It appears that it was not imagined that calling fetchall_arrayref could cause a reallocation of the stack pointer. Also -- I noticed that in the function sqlite_db_aggr_step_dispatcher in dbdimp.c, DBD::SQLite has, call_method("step", G_SCALAR|G_EVAL|G_DISCARD); -- but this isn't followed by a call to SPAGAIN; -- which I guess could result in corruption of SP if the ->step() method does anything complicated. It seems to me that this could have an impact on the call to POPs in the subsequent error branch and possibly elsewhere.
  Thu May 23 01:46:08 2013 aaron.schweiger [...] gmail.com - Correspondence added
From: aaron.schweiger [...] gmail.com
Show quoted text
> Also -- I noticed that in the function sqlite_db_aggr_step_dispatcher > in dbdimp.c, DBD::SQLite has, > > call_method("step", G_SCALAR|G_EVAL|G_DISCARD); > > -- but this isn't followed by a call to SPAGAIN; -- which
Correction: neither SPAGAIN; nor PUTBACK; appear after call_method("step",....) If neither of these are needed here -- I would like to understand why not.
  Tue Aug 27 01:58:33 2013 ishigaki [...] cpan.org - Correspondence added
On Thu May 23 14:46:08 2013, Zipf wrote: Show quoted text
>
> > Also -- I noticed that in the function sqlite_db_aggr_step_dispatcher > > in dbdimp.c, DBD::SQLite has, > > > > call_method("step", G_SCALAR|G_EVAL|G_DISCARD); > > > > -- but this isn't followed by a call to SPAGAIN; -- which
> > Correction: neither SPAGAIN; nor PUTBACK; appear after > call_method("step",....) > > If neither of these are needed here -- I would like to understand why > not.
Hi. Sorry for the late reply. I believe that's because G_DISCARD is there. From perlcall pod: G_DISCARD By default, the *call_** functions place the items returned from by the Perl subroutine on the stack. If you are not interested in these items, then setting this flag will make Perl get rid of them automatically for you. Note that it is still possible to indicate a context to the Perl subroutine by using either G_SCALAR or G_ARRAY.
  Tue Aug 27 01:58:33 2013 The RT System itself - Status changed from 'new' to 'open'
  Tue Aug 27 02:03:05 2013 ishigaki [...] cpan.org - Correspondence added
Closed this ticket as this was fixed in DBI 1.628. Thanks. On Thu May 23 14:09:14 2013, Zipf wrote: Show quoted text
> It turns out that bug is resolved with a change to DBI's Driver.xst. > (See: https://rt.cpan.org/Public/Bug/Display.html?id=85562) > > 160c160,162 > < ST(0) = dbdxst_fetchall_arrayref(sth, &PL_sv_undef, > (maxrows_svp) ? *maxrows_svp : &PL_sv_undef); > ---
> > SV *tmp = dbdxst_fetchall_arrayref(sth, &PL_sv_undef, (maxrows_svp) ? > > *maxrows_svp : &PL_sv_undef); > > SPAGAIN; > > ST(0) = tmp;
> 663c665,667 > < ST(0) = dbdxst_fetchall_arrayref(sth, slice, > batch_row_count); > ---
> > SV *tmp = dbdxst_fetchall_arrayref(sth, slice, batch_row_count); > > SPAGAIN; > > ST(0) = tmp;
> > > It appears that it was not imagined that calling fetchall_arrayref > could cause a reallocation of the stack pointer. > > Also -- I noticed that in the function sqlite_db_aggr_step_dispatcher > in dbdimp.c, DBD::SQLite has, > > call_method("step", G_SCALAR|G_EVAL|G_DISCARD); > > -- but this isn't followed by a call to SPAGAIN; -- which I guess > could result in corruption of SP if the ->step() method does anything > complicated. It seems to me that this could have an impact on the > call to POPs in the subsequent error branch and possibly elsewhere.
  Tue Aug 27 02:03:06 2013 ishigaki [...] cpan.org - Status changed from 'open' to 'resolved'