Skip Menu |

This queue is for tickets about the DBIx-Class CPAN distribution.

Report information
The Basics
Id: 42466
Status: rejected
Priority: 0/
Queue: DBIx-Class

People
Owner: Nobody in particular
Requestors: NWELLNHOF [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Unimportant
Broken in: 0.08099_05
Fixed in: (no value)



Subject: UTF8Columns doesn't work with joined tables
When I declare columns as UTF8 with DBIx::Class::UTF8Columns on a table B and then search another table A with a join on table B, the UTF8 columns aren't decoded. I had a look at the code and it seems that this scenario simply isn't supported.
On Fri Jan 16 11:10:49 2009, NWELLNHOF wrote: Show quoted text
> When I declare columns as UTF8 with DBIx::Class::UTF8Columns on a table > B and then search another table A with a join on table B, the UTF8 > columns aren't decoded. I had a look at the code and it seems that this > scenario simply isn't supported.
This is unlikely to get fixed at all, as UTF8Columns is less and less relevant every day, and will almost certainly be deprecated in 0.09. Consider switching to native RDBMS unicode support instead. See [1] for a discussion of such a migration for mysql. If you can not do this for whatever reason (e.g. dbic is not the only thing talking to the dabatase), and are willing to send us a patch - feel free to reopen this bug. [1] http://www.gyford.com/phil/writing/2008/04/25/utf8_mysql_perl.php
On Wed May 20 17:59:52 2009, RIBASUSHI wrote: Show quoted text
> This is unlikely to get fixed at all, as UTF8Columns is less and less > relevant every day, and will almost certainly be deprecated in 0.09. > Consider switching to native RDBMS unicode support instead. See [1] for > a discussion of such a migration for mysql. > > If you can not do this for whatever reason (e.g. dbic is not the only > thing talking to the dabatase), and are willing to send us a patch - > feel free to reopen this bug.
I have to support older versions of DBD::mysql that don't handle mysql_enable_utf8 correctly. But I found that DBIx::Class::ForceUTF8 works nice for me.
On Wed May 27 12:08:41 2009, NWELLNHOF wrote: Show quoted text
> On Wed May 20 17:59:52 2009, RIBASUSHI wrote:
> > This is unlikely to get fixed at all, as UTF8Columns is less and less > > relevant every day, and will almost certainly be deprecated in 0.09. > > Consider switching to native RDBMS unicode support instead. See [1] for > > a discussion of such a migration for mysql. > > > > If you can not do this for whatever reason (e.g. dbic is not the only > > thing talking to the dabatase), and are willing to send us a patch - > > feel free to reopen this bug.
> > I have to support older versions of DBD::mysql that don't handle > mysql_enable_utf8 correctly. But I found that DBIx::Class::ForceUTF8 > works nice for me.
Hmmm... this module is a carbon copy of UTF8Columns, except for the column flags being ripped out. I much rather fix DBIC than have people use this - can you provide a bit more info? What was the fail case?
See the attached test case. I search a resultset using the "join" and "select" attributes and call "get_column" on the resulting rows. The result is not UTF8 decoded if the column name in the "select" array isn't listed in "utf8_columns". This is typically the case for columns of joined tables, or if you use the "as" attribute.
#!/usr/bin/perl use strict; package My::Schema::Table1; use base qw(DBIx::Class); __PACKAGE__->load_components(qw(PK UTF8Columns Core)); __PACKAGE__->table('table1'); __PACKAGE__->add_columns(qw(id text1)); __PACKAGE__->utf8_columns(qw(text1)); __PACKAGE__->set_primary_key('id'); __PACKAGE__->has_one(table2 => 'My::Schema::Table2', 'table1_id'); package My::Schema::Table2; use base qw(DBIx::Class); __PACKAGE__->load_components(qw(PK UTF8Columns Core)); __PACKAGE__->table('table2'); __PACKAGE__->add_columns(qw(id table1_id text2)); __PACKAGE__->utf8_columns(qw(text2)); __PACKAGE__->set_primary_key('id'); __PACKAGE__->belongs_to(table1 => 'My::Schema::Table1', 'table1_id'); package My::Schema; use base qw(DBIx::Class::Schema); __PACKAGE__->load_classes(qw(Table1 Table2)); package main; binmode(STDOUT, ':utf8'); my $schema = My::Schema->connect('dbi:SQLite:dbname=:memory:'); $schema->storage->dbh->do(<<'EOF'); create table table1 ( id integer primary key, text1 text ) EOF $schema->storage->dbh->do(<<'EOF'); create table table2 ( id integer primary key, table1_id integer references table1, text2 text ) EOF my $unicode = "\x{201c}hello\x{201d}"; my $rs1 = $schema->resultset('Table1'); my $rs2 = $schema->resultset('Table2'); $rs1->create({ id => 1, text1 => $unicode, table2 => { id => 2, text2 => $unicode, }, }); # works print($rs1->first()->table2()->text2(), "\n"); my $row = $rs1->search(undef, { join => 'table2', select => [ qw(text1 text2) ], })->first(); # works print($row->get_column('text1'), "\n"); # does not work print($row->get_column('text2'), "\n"); $row = $rs1->search(undef, { select => [ qw(text1) ], as => [ qw(text) ], })->first(); # does not work print($row->get_column('text'), "\n");
On Wed Jun 24 21:07:02 2009, NWELLNHOF wrote: Show quoted text
> See the attached test case. > > I search a resultset using the "join" and "select" attributes and call > "get_column" on the resulting rows. The result is not UTF8 decoded if > the column name in the "select" array isn't listed in "utf8_columns". > This is typically the case for columns of joined tables, or if you use > the "as" attribute.
The way you placed joined columns in the main object is discouraged, and imho should behave exactly the way shown in your test. The 'as' use-case however should work, but unfortunately there is no way to clearly solve this with the current codebase. In the mantime ForceUTF8 indeed seems like the least of all evils. I am stalling this ticket as a sign of a short-term "won't fix". The test in question is at: http://dev.catalyst.perl.org/svnweb/bast/revision?rev=6802
As UTF8Columns is becoming more and more irrelevant, it is unlikely this use case will ver get fixed. Closing ticket.