Bug #16598 for DBD-SQLite: Fine tune unicode support

Mon Dec 19 04:34:49 2005 Guest - Ticket created

Subject:

Fine tune unicode support

Currently DBD::SQLite sets the utf8 flag to true for every string when $dbh->{'unicode'} = 1. This slows down Perl on ASCII-only strings. In the attached file I've added a function "sqlite3_high_bit_set" (which is taken from DBD::Pg) which checks whether the string is ascii or utf8. And if $dbh->{unicode} = 1 and the string contains high-bit chars, only then the utf8 flag is set to true. I hope this will make it in the next version of DBD::SQLite. P.S. the attached file is taken from DBD-SQLite-1.11 package

Message body is not shown because it is too large.

Thu Mar 30 07:58:30 2006 Guest - Correspondence added

From:

dom [...] happygiraffe.net

On Mon Dec 19 04:34:49 2005, guest wrote: Show quoted text

> Currently DBD::SQLite sets the utf8 flag to true for every string when > $dbh->{'unicode'} = 1. This slows down Perl on ASCII-only strings. > In the attached file I've added a function "sqlite3_high_bit_set" > (which is taken from DBD::Pg) which checks whether the string is ascii > or utf8. > And if $dbh->{unicode} = 1 and the string contains high-bit chars, > only then the utf8 flag is set to true. > I hope this will make it in the next version of DBD::SQLite. > P.S. the attached file is taken from DBD-SQLite-1.11 package

Just to point out that there's a better way these days which I didn't know about when I did the support for DBD::Pg. Instead of calling SvUTF8_on(), call sv_utf8_decode() instead. That does the same thing, but it's part of Perl, so doesn't have to be maintained. :-) Naturally, it needs to be protected by "#ifdef sv_utf8_decode". -Dom

Thu Mar 30 07:58:32 2006 The RT System itself - Status changed from 'new' to 'open'

Thu Sep 07 23:41:01 2006 msergeant [...] cpan.org - Correspondence added

Marking resolved. At least until Unicode in DBI is properly resolved. Currently we have to turn utf-8 off on every result column otherwise weird things happen.

Thu Sep 07 23:41:02 2006 msergeant [...] cpan.org - Status changed from 'open' to 'resolved'