Skip Menu |

This queue is for tickets about the DBD-SQLite CPAN distribution.

Report information
The Basics
Id: 76395
Status: resolved
Priority: 0/
Queue: DBD-SQLite

People
Owner: Nobody in particular
Requestors: moltar [...] moltar.net
Cc: ribasushi [...] leporine.io
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: SQLite: int values over 32 bit in length produce an error "datatype mismatch"
Date: Sat, 07 Apr 2012 19:06:46 -0400
To: bug-DBIx-Class [...] rt.cpan.org
From: Roman <moltar [...] moltar.net>
When a column is defined as data_type => 'int', and an integer value greater than 32 bit max (> 2147483647) is inserted, an error is produced: "DBI Exception: DBD::SQLite::st execute failed: datatype mismatch: bind 1 type 1 as 2147483648" cpanm DBIx::Class $> DBIx::Class is up to date. (0.08196) cpanm DBD::SQLite $> DBD::SQLite is up to date. (1.35) $> sqlite3 --version 3.7.0.1 The following repo contains a test case: https://github.com/moltar/DBIx--Class-SQLite-int-overflow-bug
On Sat Apr 07 19:02:45 2012, moltar@moltar.net wrote: Show quoted text
> When a column is defined as data_type => 'int', and an integer value > greater than 32 bit max (> 2147483647) is inserted, an error is produced: > > "DBI Exception: DBD::SQLite::st execute failed: datatype mismatch: bind > 1 type 1 as 2147483648"
This is correct. In SQL-land an int is a 32 bit int. Normally SQLite doesn't care, but DBIC binds ints to their corresponding types (because of this [1]) and therefore you get an error. If you want to use values larger than int, specify your columns to use the 'bigint' datatype instead.
Subject: Re: [rt.cpan.org #76395] SQLite: int values over 32 bit in length produce an error "datatype mismatch"
Date: Fri, 13 Apr 2012 11:15:09 -0400
To: bug-DBIx-Class [...] rt.cpan.org
From: Roman <moltar [...] moltar.net>
The docs (http://www.sqlite.org/datatype3.html) say: "INTEGER. The value is a signed integer, stored in 1, 2, 3, 4, 6, or 8 bytes depending on the magnitude of the value." Unsigned 8 byte INT should go up to 18446744073709551615 On 4/13/2012 3:23 AM, Peter Rabbitson via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=76395> > > On Sat Apr 07 19:02:45 2012, moltar@moltar.net wrote:
>> When a column is defined as data_type => 'int', and an integer value >> greater than 32 bit max (> 2147483647) is inserted, an error is produced: >> >> "DBI Exception: DBD::SQLite::st execute failed: datatype mismatch: bind >> 1 type 1 as 2147483648"
> > This is correct. In SQL-land an int is a 32 bit int. Normally SQLite > doesn't care, but DBIC binds ints to their corresponding types (because > of this [1]) and therefore you get an error. If you want to use values > larger than int, specify your columns to use the 'bigint' datatype instead. >
Subject: Re: [rt.cpan.org #76395] SQLite: int values over 32 bit in length produce an error "datatype mismatch"
Date: Fri, 13 Apr 2012 11:17:29 -0400
To: bug-DBIx-Class [...] rt.cpan.org
From: Roman <moltar [...] moltar.net>
Never mind. I got it ;) On 4/13/2012 3:23 AM, Peter Rabbitson via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=76395> > > On Sat Apr 07 19:02:45 2012, moltar@moltar.net wrote:
>> When a column is defined as data_type => 'int', and an integer value >> greater than 32 bit max (> 2147483647) is inserted, an error is produced: >> >> "DBI Exception: DBD::SQLite::st execute failed: datatype mismatch: bind >> 1 type 1 as 2147483648"
> > This is correct. In SQL-land an int is a 32 bit int. Normally SQLite > doesn't care, but DBIC binds ints to their corresponding types (because > of this [1]) and therefore you get an error. If you want to use values > larger than int, specify your columns to use the 'bigint' datatype instead. >
On further examination it appears that DBD::SQLite *should* allow 64bit values bound as INT. Switching to the DBD::SQLite queue as per maintainer request. <charsbar_> hmm. I think it depends on how you compile DBD::SQLite (under 32bit/64bit) but anyway that is or was a known bug of 64bit handling <charsbar_> which I hope was fixed at least in the trunk by Yuriy <ribasushi> charsbar_: also mind if I reassign this ticket to the SQLite RT queue? <charsbar_> ribasushi: and yes, please.
On Fri Apr 13 11:31:01 2012, RIBASUSHI wrote: Show quoted text
> On further examination it appears that DBD::SQLite *should* allow 64bit > values bound as INT. Switching to the DBD::SQLite queue as per > maintainer request. > > <charsbar_> hmm. I think it depends on how you compile DBD::SQLite > (under 32bit/64bit) but anyway that is or was a known bug of 64bit handling > <charsbar_> which I hope was fixed at least in the trunk by Yuriy > <ribasushi> charsbar_: also mind if I reassign this ticket to the SQLite > RT queue? > <charsbar_> ribasushi: and yes, please.
sqlite3 internally uses i64 for integer values. sqlite3.c:63819 applyNumericAffinity() That's the range -9223372036854775808 to +9223372036854775807 I've added a testcase (see attachment) and could not repro the supposed INTMAX overflow with 32bit, 64bit and 32bit+use64bitint perls. Could the op please adjust the testcase to show the wrong behaviour? -- Reini Urban
Subject: rt_76395_int_overflow.t
#!/usr/bin/perl use strict; my $INTMAX; BEGIN { $| = 1; $^W = 1; use Config; $INTMAX = (1 << ($Config{ivsize}*4-1)) - 1; } use t::lib::Test; use Test::More tests => 7 + (2147483647 == $INTMAX ? 2 : 4); use Test::NoWarnings; use DBI qw(:sql_types); my $dbh = connect_ok(); # testing results sub intmax { my $intmax = shift; my ($statement, $sth, $result); $statement = "SELECT $intmax + 1"; $sth = $dbh->prepare($statement); ok( $sth->execute, "execute: $statement" ); $result = $sth->fetchrow_arrayref->[0]; is( $result, $intmax + 1, "result: $result" ); } intmax($INTMAX); intmax(2147483647) if 2147483647 != $INTMAX; # testing int column type, which should default to int(8) or int(4) $dbh->do('drop table if exists artist'); $dbh->do(<<'END_SQL'); create table artist ( id int not null, name text not null ) END_SQL $INTMAX = 2147483647; my ($sth, $result); ok( $dbh->do(qq/insert into artist (id,name) values($INTMAX+1, 'Leonardo')/), 'insert int INTMAX+1'); $sth = $dbh->prepare('select id from artist where name=?'); ok( $sth->execute('Leonardo'), 'bind to name' ); $result = $sth->fetchrow_arrayref->[0]; is( $result, $INTMAX+1, "result: $result" ); $sth = $dbh->prepare('select name from artist where id=?'); ok( $sth->execute($INTMAX+1), 'bind to INTMAX+1' ); $result = $sth->fetchrow_arrayref->[0]; is( $result, 'Leonardo', "result: $result" );
Subject: Re: [rt.cpan.org #76395] SQLite: int values over 32 bit in length produce an error "datatype mismatch"
Date: Wed, 03 Apr 2013 10:07:51 -0400
To: bug-DBD-SQLite [...] rt.cpan.org
From: moltar <moltar [...] moltar.net>
Hi, This test case is missing "t::lib::Test". Thanks! On 2013-04-02 11:16 AM, Reini Urban via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=76395 > > > On Fri Apr 13 11:31:01 2012, RIBASUSHI wrote:
>> On further examination it appears that DBD::SQLite *should* allow 64bit >> values bound as INT. Switching to the DBD::SQLite queue as per >> maintainer request. >> >> <charsbar_> hmm. I think it depends on how you compile DBD::SQLite >> (under 32bit/64bit) but anyway that is or was a known bug of 64bit handling >> <charsbar_> which I hope was fixed at least in the trunk by Yuriy >> <ribasushi> charsbar_: also mind if I reassign this ticket to the SQLite >> RT queue? >> <charsbar_> ribasushi: and yes, please.
> > sqlite3 internally uses i64 for integer values. > sqlite3.c:63819 applyNumericAffinity() > That's the range -9223372036854775808 to +9223372036854775807 > > I've added a testcase (see attachment) and could not repro the supposed > INTMAX overflow with 32bit, 64bit and 32bit+use64bitint perls. > > Could the op please adjust the testcase to show the wrong behaviour? >
On Tue Apr 02 11:16:23 2013, RURBAN wrote: Show quoted text
> Could the op please adjust the testcase to show the wrong behaviour?
Attaching test case, fails on latest SQLite when $Config{ivsize} == 4. The warnings are spurious - everything underneath (the roundtrip) works flawlessly.
Subject: 32bit_iv.t
use warnings; use strict; use Test::More; use DBI; my $dbh = DBI->connect('dbi:SQLite::memory:', undef, undef, { RaiseError => 1, PrintError => 0 }); $dbh->do(' CREATE TABLE t ( pk INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, int INTEGER, bigint BIGINT ) '); for my $val (qw( -9223372036854775808 -9223372036854775807 -8694837494948124658 -6848440844435891639 -5664812265578554454 -5380388020020483213 -2564279463598428141 2442753333597784273 4790993557925631491 6773854980030157393 7627910776496326154 8297530189347439311 9223372036854775806 9223372036854775807 4294967295 4294967296 -4294967296 -4294967295 -4294967294 -2147483649 -2147483648 -2147483647 -2147483646 2147483646 2147483647 2147483648 2147483649 )) { for my $col (qw(int bigint)) { for my $bindtype (undef, 'DBI::SQL_INTEGER', 'DBI::SQL_BIGINT') { my $tdesc = sprintf "value '%s' with %s bindtype on '%s' column", $val, $bindtype || 'no', $col ; my $sth = $dbh->prepare_cached( "INSERT INTO t ( $col ) VALUES ( ? )", {}, 3 ); my @w; local $SIG{__WARN__} = sub { push @w, @_ }; ok ( $sth->bind_param(1, $val, ( $bindtype and do { no strict 'refs'; &{$bindtype} } )), "Succesfully bound $tdesc", ); is_deeply( \@w, [], "No warnings during bind of $tdesc", ); ok ( eval { $sth->execute ; 1 }, "Succesfully inserted $tdesc" . ($@ ? ": $@" : ''), ); is_deeply( \@w, [], "No warnings during insertion of $tdesc", ); my $id; ok ( $id = $dbh->last_insert_id(undef, undef, 't', 'pk'), "Got id $id of inserted $tdesc", ); is_deeply( $dbh->selectall_arrayref("SELECT $col FROM t WHERE pk = $id"), [[ $val ]], "Proper roundtrip (insert/select) of $tdesc", ); } } } done_testing;
On Wed Oct 09 21:32:14 2013, RIBASUSHI wrote: Show quoted text
> On Tue Apr 02 11:16:23 2013, RURBAN wrote:
> > Could the op please adjust the testcase to show the wrong behaviour?
> > Attaching test case, fails on latest SQLite when $Config{ivsize} == 4. > The warnings are spurious - everything underneath (the roundtrip) > works flawlessly.
Updated r/rt_76396_int_overflow.t with your test, and (hopefully) fixed behavior under ivsize==4 environment with https://github.com/DBD-SQLite/DBD-SQLite/commit/517d9bf595f5a59517cab94467167425e2b357a7 Shipped 1.45_01 and let's see what happens.
On Wed Oct 22 17:36:37 2014, ISHIGAKI wrote: Show quoted text
> Updated r/rt_76396_int_overflow.t with your test, and (hopefully) > fixed behavior under ivsize==4 environment with > https://github.com/DBD-SQLite/DBD- > SQLite/commit/517d9bf595f5a59517cab94467167425e2b357a7 > > Shipped 1.45_01 and let's see what happens.
The case of 0 (false value I guess) has regressed. Add '0' to the array of tested ints to see the failure (fails on both 64 and 32 for me).
On Fri Oct 24 19:23:13 2014, RIBASUSHI wrote: Show quoted text
> On Wed Oct 22 17:36:37 2014, ISHIGAKI wrote: >
> > Updated r/rt_76396_int_overflow.t with your test, and (hopefully) > > fixed behavior under ivsize==4 environment with > > https://github.com/DBD-SQLite/DBD- > > SQLite/commit/517d9bf595f5a59517cab94467167425e2b357a7 > > > > Shipped 1.45_01 and let's see what happens.
> > The case of 0 (false value I guess) has regressed. Add '0' to the > array of tested ints to see the failure (fails on both 64 and 32 for > me).
Thanks for the heads-up. Fixed in the master: https://github.com/DBD-SQLite/DBD-SQLite/commit/0a1f3416fc19f8dc1843b3165a5f77707ca36c21
Closed as DBD::SQLite 1.46 was released. Thanks!