Bug #24471 for DBD-mysql: mysql_enable_utf8 requires UTF8 tables to work ?

Sat Jan 20 14:22:22 2007 hg [...] apteryx.fr - Ticket created

Subject:	mysql_enable_utf8 requires UTF8 tables to work ?
Date:	Sat, 20 Jan 2007 20:22:05 +0100 (CET)
To:	bug-DBD-mysql [...] rt.cpan.org
From:	Herve Guillemet <hg [...] apteryx.fr>

Hi, I'm trying the mysql_enable_utf8 option with 4.001 with the following code just after connecting : $dbh->do("SET CHARACTER SET 'utf8'"); $dbh->{mysql_enable_utf8} = 1; Data received are UTF8 but with the UTF8 flag off just as if the mysql_enable_utf8 option where not set. Could this is because my data tables are not UTF8 ? I'm a in a situation where the data are stored in latin1 in the database but I need to receive UTF8 only. Thanks,

Sat Jan 20 18:35:50 2007 patg [...] patg.net - Correspondence added

CC:	undisclosed-recipients:;
Subject:	Re: [rt.cpan.org #24471] mysql_enable_utf8 requires UTF8 tables to work ?
Date:	Sat, 20 Jan 2007 18:34:48 -0500
To:	bug-DBD-mysql [...] rt.cpan.org
From:	Patrick Galbraith <patg [...] patg.net>

Herve Guillemet via RT wrote: Show quoted text

>Sat Jan 20 14:22:22 2007: Request 24471 was acted upon. >Transaction: Ticket created by hg@apteryx.fr > Queue: DBD-mysql > Subject: mysql_enable_utf8 requires UTF8 tables to work ? > Broken in: (no value) > Severity: (no value) > Owner: Nobody > Requestors: hg@apteryx.fr > Status: new > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=24471 > > > > >Hi, > >I'm trying the mysql_enable_utf8 option with 4.001 with the following code >just after connecting : > > $dbh->do("SET CHARACTER SET 'utf8'"); > $dbh->{mysql_enable_utf8} = 1; > >Data received are UTF8 but with the UTF8 flag off just as if the >mysql_enable_utf8 option where not set. Could this is because >my data tables are not UTF8 ? >

Yes. Show quoted text

> I'm a in a situation where the data are >stored in latin1 in the database but I need to receive UTF8 only. > >

Herve, I would set UTF8 for the tables you need utf data from to ensure you get UTF8 both in retrieving and inserting data. regards, Patrick Show quoted text

>Thanks, > >

-- Patrick Galbraith, Senior Programmer Grazr - Easy feed grazing and sharing http://www.grazr.com Satyam Eva Jayate - Truth Alone Triumphs Mundaka Upanishad

Sat Jan 20 18:35:52 2007 The RT System itself - Status changed from 'new' to 'open'

Sat Jan 20 19:41:56 2007 hg [...] apteryx.fr - Correspondence added

Subject:	Re: [rt.cpan.org #24471] mysql_enable_utf8 requires UTF8 tables to work ?
Date:	Sun, 21 Jan 2007 01:41:43 +0100 (CET)
To:	Patrick Galbraith via RT <bug-DBD-mysql [...] rt.cpan.org>
From:	Herve Guillemet <hg [...] apteryx.fr>

Show quoted text

>> I'm trying the mysql_enable_utf8 option with 4.001 with the following code >> just after connecting : >> >> $dbh->do("SET CHARACTER SET 'utf8'"); >> $dbh->{mysql_enable_utf8} = 1; >> >> Data received are UTF8 but with the UTF8 flag off just as if the >> mysql_enable_utf8 option where not set. Could this is because >> my data tables are not UTF8 ? >>

> Yes. >

>> I'm a in a situation where the data are >> stored in latin1 in the database but I need to receive UTF8 only. >> >>

> Herve, > > I would set UTF8 for the tables you need utf data from to ensure you get > UTF8 both in retrieving and inserting data. >

Hi Patrick, Thanks for this reply. IMO, from the mysql client point of view, as long as it asks for UTF8 data with SET CHARACTER SET or SET NAMES, it's not its concern how the server stores the character, and it should receive the same UTF8 data whether the table is utf or latin. When under SET CHARACTER SET 'utf8', are there technical differences in the way DBD::mysql receives the data from UTF8 and latin tables ? In my case, using UTF for table storage is sub-optimal, and the perl client is based on a XML framework that needs UTF8 data. I have a currently working solution that subclasses DBI and add a layer of filtering, calling Encode::_utf8_on on all data received. But it's not so efficient and causes trouble when fetching BLOBS fo which we do not want utf8 on. Thanks again for your support, == Hervé Guillemet

Sat Jan 20 22:33:42 2007 patg [...] patg.net - Correspondence added

Subject:	Re: [rt.cpan.org #24471] mysql_enable_utf8 requires UTF8 tables to work ?
Date:	Sat, 20 Jan 2007 22:32:51 -0500
To:	bug-DBD-mysql [...] rt.cpan.org
From:	Patrick Galbraith <patg [...] patg.net>

Herve Guillemet via RT wrote: Show quoted text

> Queue: DBD-mysql > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=24471 > > > >

>>>I'm trying the mysql_enable_utf8 option with 4.001 with the following code >>>just after connecting : >>> >>> $dbh->do("SET CHARACTER SET 'utf8'"); >>> $dbh->{mysql_enable_utf8} = 1; >>> >>>Data received are UTF8 but with the UTF8 flag off just as if the >>>mysql_enable_utf8 option where not set. Could this is because >>>my data tables are not UTF8 ? >>> >>> >>>

>>Yes. >> >> >>

>>>I'm a in a situation where the data are >>>stored in latin1 in the database but I need to receive UTF8 only. >>> >>> >>> >>>

>>Herve, >> >>I would set UTF8 for the tables you need utf data from to ensure you get >>UTF8 both in retrieving and inserting data. >> >> >>

> >Hi Patrick, > >Thanks for this reply. > >IMO, from the mysql client point of view, as long as it asks for UTF8 data >with SET CHARACTER SET or SET NAMES, it's not its concern how the server >stores the character, and it should receive the same UTF8 data whether the >table is utf or latin. > >When under SET CHARACTER SET 'utf8', are there technical differences in >the way DBD::mysql receives the data from UTF8 and latin tables ? > >In my case, using UTF for table storage is sub-optimal, and the perl >client is based on a XML framework that needs UTF8 data. I have a >currently working solution that subclasses DBI and add a layer of >filtering, calling Encode::_utf8_on on all data received. But it's not so >efficient and causes trouble when fetching BLOBS fo which we do not want >utf8 on. > >Thanks again for your support, > >== Hervé Guillemet > >

Herve, I'm not the biggest expert on UTF8, so if you could, please send me some sort of script that reproduces this. That way, I have something to look at in the driver code to see why it's not returning from the db what it should. Thanks! Patrick -- Patrick Galbraith, Senior Programmer Grazr - Easy feed grazing and sharing http://www.grazr.com Satyam Eva Jayate - Truth Alone Triumphs Mundaka Upanishad

Sun Jan 21 09:40:32 2007 hg [...] apteryx.fr - Correspondence added

Subject:	Re: [rt.cpan.org #24471] mysql_enable_utf8 requires UTF8 tables to work ?
Date:	Sun, 21 Jan 2007 15:40:11 +0100 (CET)
To:	Patrick Galbraith via RT <bug-DBD-mysql [...] rt.cpan.org>
From:	Herve Guillemet <hg [...] apteryx.fr>

Hi Patrick, Please forget this report. I thought I tested the new 4.0001 release I installed, but the 3.0007 version installed by my linux distribution took precedence and was use instead. After removing the 3.0007 version, everything works fine : the UTF flag is set, both when fetching from UTF and Latin tables. Sorry about this.

Sun Jan 21 13:05:11 2007 patg [...] patg.net - Correspondence added

Subject:	Re: [rt.cpan.org #24471] mysql_enable_utf8 requires UTF8 tables to work ?
Date:	Sun, 21 Jan 2007 13:04:49 -0500
To:	bug-DBD-mysql [...] rt.cpan.org
From:	Patrick Galbraith <patg [...] patg.net>

Herve Guillemet via RT wrote: Show quoted text

> Queue: DBD-mysql > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=24471 > > > >Hi Patrick, > >Please forget this report. I thought I tested the new 4.0001 release I >installed, but the 3.0007 version installed by my linux distribution >took precedence and was use instead. >After removing the 3.0007 version, everything works fine : the UTF flag is >set, both when fetching from UTF and Latin tables. > >Sorry about this. > >

Herve, Excellent! Glad to hear this works for you! Thank you for finding this out. merci, Patrick -- Patrick Galbraith, Senior Programmer Grazr - Easy feed grazing and sharing http://www.grazr.com Satyam Eva Jayate - Truth Alone Triumphs Mundaka Upanishad

Mon Mar 05 18:49:02 2007 jimw-removetoemail [...] trainedmonkey.com - Correspondence added

Closing this, since it turned out there was no issue.

Mon Mar 05 18:49:06 2007 jimw-removetoemail [...] trainedmonkey.com - Status changed from 'open' to 'resolved'