Bug #50359 for CHI: Problem with unicode

Fri Oct 09 08:15:09 2009 alexandrfedorov [...] gmail.com - Ticket created

Subject:

Problem with unicode

In example below, there is a error: error during cache set for namespace='Default', key='aa', size=1, expires='1d', cache='File': Wide character in syswrite at /usr/lib/perl5/site_perl/5.10.0/CHI/Driver/File.pm line 105. perl -v: perl, v5.10.0 built for x86_64-linux-thread-multi Cache::Cache with such data works good. use 5.010; use CHI; my $cont = chr(400); my $cache = CHI->new( driver => 'File',root_dir=>'/tmp',on_set_error => 'die'); $cache->clear; my $key = 'aa'; say "cache of $key = ",$cache->get($key) // 'undefined'; $cache->set($key,$cont,'1 day'); say "cache of $key = ",$cache->get($key);

Fri Oct 09 09:30:59 2009 alexandrfedorov [...] gmail.com - Correspondence added

I found a workaround, but i don't sure, how good it is. diff: 71c71,72 < --- Show quoted text

> use Encode; > Encode::_utf8_on($buf);

101a103,104 Show quoted text

> use Encode; > Encode::_utf8_off($data);

Fri Oct 09 09:31:01 2009 alexandrfedorov [...] gmail.com - Status changed from 'new' to 'open'

Mon Oct 12 09:51:39 2009 swartz [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #50359] Problem with unicode
Date:	Mon, 12 Oct 2009 06:51:16 -0700
To:	bug-CHI [...] rt.cpan.org
From:	Jonathan Swartz <swartz [...] pobox.com>

Thanks for reporting this bug, Alexandr. You'll see in the test suite (lib/CHI/t/Driver.pm line 99) that I punted on unicode keys and values because I wasn't sure how to handle them: # Several problems with trying to handle unicode keys and values. # * They generate 'Wide character in print' warnings when logging about the gets/sets # * Causes metadata packing to break # * Storable won't touch them # not sure how to handle these. Maybe there's no way. Even if we automatically # encoded all keys and values, what to do about complex values where wide strings # might be lurking? # 'utf8_partial' => "abc\x{263A}def", # 'utf8_all' => "\x{263A}\x{263B}\x{263C}", Unfortunately I'm fuzzier about unicode issues than I ought to be. If you can come up with a patch that allows the above tests to work, please submit it! Thanks Jon On Oct 9, 2009, at 6:31 AM, Alexandr Fedorov via RT wrote: Show quoted text

> Queue: CHI > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=50359 > > > I found a workaround, but i don't sure, how good it is. > diff: > > 71c71,72 > < > ---

>> use Encode; >> Encode::_utf8_on($buf);

> 101a103,104

>> use Encode; >> Encode::_utf8_off($data);

>

Tue Oct 20 14:28:03 2009 alexandrfedorov [...] gmail.com - Correspondence added

I has strenuously studied how Perl unicode works, and has come to a conclusion than perlunifaq explains all =) <i>When should I decode or encode? Whenever you're communicating text with anything that is external to your perl process, like a database, a text file, a socket, or another program. Even if the thing you're communicating with is also written in Perl.</i> I.e. you need add layer :utf8 to opening files within file-driver All tests that i tried works good with that.

Fri Jun 25 09:00:57 2010 JSWARTZ [...] cpan.org - Correspondence added

I've resolved issues with utf8 in keys and values in CHI 0.36. Let me know if this solution still does not solve your problem. Thx Jon

Fri Jun 25 09:01:30 2010 JSWARTZ [...] cpan.org - Status changed from 'open' to 'resolved'