Skip Menu |

This queue is for tickets about the DBD-DB2 CPAN distribution.

Report information
The Basics
Id: 28655
Status: resolved
Priority: 0/
Queue: DBD-DB2

People
Owner: opendev [...] us.ibm.com
Requestors: jrockway [...] cpan.org
Cc: aff [...] cpan.org
AdminCc:

Bug Information
Severity: Critical
Broken in:
  • 1.61
  • 1.0
  • 1.1
Fixed in: (no value)



Subject: memory corruption in dbd_preparse
Hi IBM, dbd_preparse doesn't allocate enough memory to account for expansion of ? into :p1234. The attached patch fixes the issue (although you can just change the multiplier from 4 to 7 or so if you want to punt on the issue. I doubt anyone has more than 1,000,000 bindvars in their query :) The patch is against 1.0 since 1.1 won't compile on my machine. A patch for that will follow shortly. -- Jonathan Rockway <jrockway@cpan.org>
Subject: file
Download file
application/octet-stream 2.2k

Message body not shown because it is not plain text.

Turns out that my patch had an off-by-one error :) This version fixes that. -- Jonathan Rockway <jrockway@cpan.org>
Common subdirectories: DBD-DB2-1.0/Constants and DBD-DB2-1.0-fix28655/Constants diff -u DBD-DB2-1.0/dbdimp.c DBD-DB2-1.0-fix28655/dbdimp.c --- DBD-DB2-1.0/dbdimp.c 2006-11-09 12:13:42.000000000 -0600 +++ DBD-DB2-1.0-fix28655/dbdimp.c 2007-08-08 00:44:52.000000000 -0500 @@ -1396,6 +1396,56 @@ return TRUE; } +int _bytes_required(int statement_length, int placeholders) +{ + int bytes = statement_length; + + /* each placeholder requires at least two additional bytes: + ? -> :p1 + */ + bytes += 2 * placeholders; + + /* calculate number of bytes needed to represent all numbers + from 1 to 10**(digits-1) + the pattern here is: + 1 digit : 9 + 2 digits: 9 * 21 + 3 digits: 9 * 321 + 4 digits: 9 * 4321 + etc. + + we need this because ? -> :p1, ? -> :p2, ..., ? -> :p100000 + */ + + /* get number of digits */ + int digits = 0; + double d = (double) placeholders; + while(d >= 1) + { + d /= 10.0; + digits++; + } + + /* calculate the corresponding 54321 (etc.) for these digits */ + int digit_bytes = 0; + int i; + int pos = 0; + for(i = digits-1; i > 0; i--) + { + if (pos == 0) pos = 1; + pos *= 10; /* also calculate 10**(digits-1) */ + digit_bytes *= 10; + digit_bytes += i; + } + + /* add space for 1 -> 10**(d-1) */ + bytes += 9 * digit_bytes; + + /* add the rest from 10**(d-1) to #placeholders */ + bytes += digits * (placeholders - pos); + + return bytes; +} static void dbd_preparse( imp_sth_t *imp_sth, char *statement ) @@ -1406,11 +1456,11 @@ SV *phs_sv; int idx=0, style=0, laststyle=0; - /* allocate room for copy of statement with spare capacity */ - /* for editing ':1' into ':p1' so we can use obndrv. */ - imp_sth->statement = (SQLCHAR *)safemalloc(strlen(statement) + - (DBIc_NUM_PARAMS(imp_sth)*4)); - + /* allocate "bytes" room for copy of statement with spare capacity */ + /* for expanding ? to :p1, ..., p1000, ... and :1 to :p1 (etc.) */ + int bytes = _bytes_required(strlen(statement), DBIc_NUM_PARAMS(imp_sth)); + imp_sth->statement = (SQLCHAR *) safemalloc(bytes); + /* initialise phs ready to be cloned per placeholder */ memset(&phs_tpl, '\0',sizeof(phs_tpl)); phs_tpl.sv = NULL; Common subdirectories: DBD-DB2-1.0/lib and DBD-DB2-1.0-fix28655/lib Common subdirectories: DBD-DB2-1.0/t and DBD-DB2-1.0-fix28655/t
Hi We have recently taken over the DBD:DB2 driver. However, due to legal issues we cannot look at the patch since we do not have a CLA in place to accept the patch. However, I would look into the problem and try to solve the issue. If you have a repro of the problem, that would be helpful. -- Thanks Tarun Pasrija IBM OpenSource Application Development Team India Software Labs, Bangalore (India)
CC: jrockway [...] cpan.org
Subject: Re: [rt.cpan.org #28655] memory corruption in dbd_preparse
Date: Tue, 10 Mar 2009 00:28:34 -0500
To: bug-DBD-DB2 [...] rt.cpan.org
From: Jonathan Rockway <jon [...] jrock.us>
* On Sat, Mar 07 2009, IBM OpenDev via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=28655 > > > Hi > > We have recently taken over the DBD:DB2 driver. However, due to legal > issues we cannot look at the patch since we do not have a CLA in place > to accept the patch. However, I would look into the problem and try to > solve the issue. If you have a repro of the problem, that would be helpful.
I don't have a simple script to reproduce the problem, but I can explain it for you. (It's been a year or so since I've used DB2, so my memory might be incorrect. But this is the basic idea.) Basically, when expanding placeholders, DBD::DB2 converts "?" to ":1", ":2", and so on. It allocates one extra byte of memory for each question mark before doing the replacing. This is fine when you only have nine placeholders, but as you venture into the range of ":10", ":100", and ":1000", there isn't enough memory allocated to store all the data. I noticed this causing segmentation faults in queries that had around 2000 placeholders. The fix is to allocate the correct amount of memory, 1 extra byte for 1-9, 2 extra bytes for 10-99, 3 extra bytes for 100-999, and so on. Regards, Jonathan Rockway -- print just => another => perl => hacker => if $,=$"
On Sat Mar 07 11:43:44 2009, IBMTORDB2 wrote: Show quoted text
> If you have a repro of the problem, that would be helpful.
ENV: DBD::DB2 version 1.61, perl v5.10.0 built for x86_64-linux, DB2 v9.5.0.0: The problem is reproducible using the attached perl script, which segfaults on my system when the number of bind variables is above 1000: $ perl -swl bug.28655.pl -N=1 -P=1 select foo from bar where id>? $ perl -swl bug.28655.pl -N=2 -P=1 select foo from bar where id>? and id>? $ perl -swl bug.28655.pl -N=3 -P=1 select foo from bar where id>? and id>? and id>? $ perl -swl bug.28655.pl -N=10 $ perl -swl bug.28655.pl -N=100 $ perl -swl bug.28655.pl -N=1000 $ perl -swl bug.28655.pl -N=10000 Segmentation fault $ See also attached cli trace file. The trace is run for the query that segfaults only. For values on N between ~1100 and ~1500 I get the following glibc error: $ perl -swl bug.28655.pl -N=1200 *** glibc detected *** perl: double free or corruption (!prev): 0x00000000103120c0 *** ======= Backtrace: ========= /lib64/libc.so.6[0x33e7e71834] /lib64/libc.so.6(cfree+0x8c)[0x33e7e74e7c] /opt/perl/lib/site_perl/5.10.0/x86_64-linux/auto/DBD/DB2/DB2.so(db2_st_destroy+0x7a)[0x2b94979ed3da] /opt/perl/lib/site_perl/5.10.0/x86_64-linux/auto/DBD/DB2/DB2.so(XS_DBD__DB2__st_DESTROY+0x12e)[0x2b94979e8cae] /opt/perl/lib/site_perl/5.10.0/x86_64-linux/auto/DBI/DBI.so(XS_DBI_dispatch+0x1b06)[0x2b94973bfd26] Best regards, Andreas --
Download p23723t-638693872.cli
application/octet-stream 67.9k

Message body not shown because it is not plain text.

use strict; use warnings; use DBD::DB2; our $N ||= 1; our $P ||= 0; # print flag my $DATABASE = "foo"; my $USERID = "bar"; my $PASSWORD = "*********"; my $dbh = DBI->connect("dbi:DB2:$DATABASE", "$USERID", "$PASSWORD", {PrintError => 0}); my $query = q{select foo from bar where } . join(q{ and }, ('id>?') x $N); print $query if $P; my $sth = $dbh->prepare($query); __END__
I can confirm that Jonathan's patch above seems to work for DBD-DB2-1.61 also. I can no longer get DBD-DB2 to crash even with _very_ large number N, after applying the patch. $ time perl -swl bug.28655.pl -N=100000 real 0m0.580s user 0m0.265s sys 0m0.018s $ time perl -swl bug.28655.pl -N=1000000 real 0m2.254s user 0m2.201s sys 0m0.051s $ time perl -swl bug.28655.pl -N=10000000 real 0m22.169s user 0m21.759s sys 0m0.381s $ time perl -swl bug.28655.pl -N=100000000 real 5m21.801s user 3m40.420s sys 0m7.539s The last sample makes my machine swap like crazy, but it does finish without crashing. Best regards, Andreas --
RT-Send-CC: opendev [...] us.ibm.com
Attaching the patch I created for the issue. Had been busy with some work so it took time to look into this one. Would request you to please test with your scenarios and send in your comments on this. I would thus include this in the next release. Thanks and Regards Tarun Pasrija On Tue Mar 10 09:37:47 2009, AFF wrote: Show quoted text
> I can confirm that Jonathan's patch above seems to work for DBD-DB2-1.61 > also. I can no longer get DBD-DB2 to crash even with _very_ large > number N, after applying the patch. > > $ time perl -swl bug.28655.pl -N=100000 > > real 0m0.580s > user 0m0.265s > sys 0m0.018s > $ time perl -swl bug.28655.pl -N=1000000 > > real 0m2.254s > user 0m2.201s > sys 0m0.051s > $ time perl -swl bug.28655.pl -N=10000000 > > real 0m22.169s > user 0m21.759s > sys 0m0.381s > $ time perl -swl bug.28655.pl -N=100000000 > > real 5m21.801s > user 3m40.420s > sys 0m7.539s > > The last sample makes my machine swap like crazy, but it does finish > without crashing. > > Best regards, > Andreas
-- Thanks IBM OpenSource Application Development Team India Software Labs, Bangalore (India)

Message body is not shown because it is too large.

Resolved in version 1.74. Released and update would be available by end of day tomorrow. -- Thanks Tarun Pasrija IBM OpenSource Application Development Team India Software Labs, Bangalore (India)