Skip Menu |

This queue is for tickets about the DBD-SQLite CPAN distribution.

Report information
The Basics
Id: 55558
Status: rejected
Priority: 0/
Queue: DBD-SQLite

People
Owner: Nobody in particular
Requestors: nigelm [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 1.29
Fixed in: (no value)



Subject: utf8 issue with update - data not round-tripping intact
See attached test file and DBIx::Class schema/result classes utf string goes into (ie INSERT) the db OK, roundtrips intact. However on UPDATE the string gets broken on the roundtrip. I *think* this is a problem at the DBD or sqlite layer rather than further up the DBIx::Class stack. Everything is using the current released version (new install). perl 5.10.1 - self built with default options DBIx::Class 0.08120 DBD::SQLite 1.29 DBI 1.609 Running on Mac OS X 10.6 Failing tests are the "String Length" and the regex for the utf char on $row3->str
Subject: utf8test.t
use strict; use warnings; use utf8; use Test::More; BEGIN { use_ok('TestDb'); } sub checkit { my $str = shift; my $what = shift; ok( utf8::is_utf8($str), "$what is UTF8" ); ok( utf8::valid($str), "$what valid UTF8" ); like( $str, qr/\x{2013}/, "$what contains correct char" ); } my $sqlfile = 'testdb.db'; # delete any prexisting unlink($sqlfile); my $schema = TestDb->connect( "dbi:SQLite:$sqlfile", '', '', { RaiseError => 1, sqlite_unicode => 1, } ); ok( defined($schema), "Open Db" ); $schema->deploy; my $rs = $schema->resultset('Str'); ok( $rs, 'Resultset' ); my $str = "\x{2013}"; checkit( $str, "Sample string" ); my $row = $rs->create( { str => $str } ); checkit( $row->str, "Row string" ); my $row2 = $rs->find( $row->id ); checkit( $row2->str, "Db Row string" ); $row2->str( $row2->str . $row2->str ); # duplicate str checkit( $row2->str, "Dup Row string" ); $row2->update; checkit( $row2->str, "Dup Row string post update" ); my $row3 = $rs->find( $row->id ); ok( ( length( $row2->str ) == length( $row3->str ) ), "String length" ); checkit( $row3->str, "Retrieved updated string" ); done_testing();
Subject: TestDb.pm
package TestDb; use strict; use warnings; use base qw/DBIx::Class::Schema/; __PACKAGE__->load_namespaces(); # -------------------------------------------------------------------- 1;
Subject: Str.pm
package TestDb::Result::Str; use strict; use warnings; use utf8; use base qw/DBIx::Class::Core/; __PACKAGE__->load_components(qw/UTF8Columns/); __PACKAGE__->table('str'); __PACKAGE__->add_columns( 'id' => { 'data_type' => 'integer', 'is_auto_increment' => 1, 'default_value' => undef, 'is_foreign_key' => 0, 'name' => 'id', 'is_nullable' => 0, 'size' => '11' }, 'str' => { 'data_type' => 'varchar', 'is_auto_increment' => 0, 'default_value' => undef, 'is_foreign_key' => 0, 'is_nullable' => 1, 'size' => '80', }, ); __PACKAGE__->set_primary_key('id'); __PACKAGE__->utf8_columns(qw/str/); # -------------------------------------------------------------------- 1;
From: nigelm [...] cpan.org
Should have put the test run output in... % prove --lib utf8test.t utf8test.t .. 1/? # Failed test 'String length' # at utf8test.t line 52. # Failed test 'Retrieved updated string contains correct char' # at utf8test.t line 15. # '––' # doesn't match '(?-xism:\x{2013})' # Looks like you failed 2 tests of 22. utf8test.t .. Dubious, test returned 2 (wstat 512, 0x200) Failed 2/22 subtests Test Summary Report ------------------- utf8test.t (Wstat: 512 Tests: 22 Failed: 2) Failed tests: 19, 22 Non-zero exit status: 2 Files=1, Tests=22, 0 wallclock secs ( 0.03 usr 0.01 sys + 0.22 cusr 0.03 csys = 0.29 CPU) Result: FAIL
Hi. If you set sqlite_unicode to true, you don't need to (actually, you MUSTn't) use DBIx::Class::UTF8Columns as DBD::SQLite takes care of flag handling by itself. So, just remove __PACKAGE__->utf8_columns(qw/str/); and your test will pass. Closed this ticket as "not_a_bug". On 2010-3-14 Sun 13:00:27, nigelm@cpan.org wrote: Show quoted text
> See attached test file and DBIx::Class schema/result classes > > utf string goes into (ie INSERT) the db OK, roundtrips intact. > However on UPDATE the string gets broken on the roundtrip. > > I *think* this is a problem at the DBD or sqlite layer rather than > further up > the DBIx::Class stack. > > Everything is using the current released version (new install). > perl 5.10.1 - self built with default options > DBIx::Class 0.08120 > DBD::SQLite 1.29 > DBI 1.609 > > Running on Mac OS X 10.6 > > Failing tests are the "String Length" and the regex for the utf char > on $row3->str >
From: nigelm [...] cpan.org
On Sun Mar 14 23:55:19 2010, ISHIGAKI wrote: Show quoted text
> Hi. If you set sqlite_unicode to true, you don't need to (actually, you > MUSTn't) use DBIx::Class::UTF8Columns as DBD::SQLite takes care of flag > handling by itself. So, just remove __PACKAGE__->utf8_columns(qw/str/); > and your test will pass. Closed this ticket as "not_a_bug".
Thanks for the response. I am talking this over with the DBIC guys and preparing a doc patch for UTF8Columns making it clearer what it is doing and that it is an *alternative* to native db unicode support. Cheers Nigel.
Thanks for the follow-up, and your doc patch for DBIC :) Closed this ticket again. On 2010-3-15 Mon 09:01:44, nigelm@cpan.org wrote: Show quoted text
> On Sun Mar 14 23:55:19 2010, ISHIGAKI wrote:
> > Hi. If you set sqlite_unicode to true, you don't need to (actually,
> you
> > MUSTn't) use DBIx::Class::UTF8Columns as DBD::SQLite takes care of
> flag
> > handling by itself. So, just remove __PACKAGE__- > >utf8_columns(qw/str/); > > and your test will pass. Closed this ticket as "not_a_bug".
> > Thanks for the response. > I am talking this over with the DBIC guys and preparing a doc patch > for UTF8Columns > making it clearer what it is doing and that it is an *alternative* to > native db unicode > support. > > Cheers > Nigel.