| Subject: | UTF8 Encoding issue - wide character errors in IO::Handle when Template Toolkit Renders. |
I'm using webservice::solr as a catalyst model (extending catalyst::model rather than using the dedicated adapter). catalyst is 5.90042. OS is mint 15 (ubuntu 13.04) x64. solr instance is 4.4 and the data comes from postgres, both the collection and source data are configured as utf8.
I have a long text field in my solr database, which when rendered by the view results in the following error and no data returned to the client browser.
[error] Caught exception in engine "Wide character in syswrite at /usr/lib/perl/5.14/IO/Handle.pm line 216, <DATA> line 1003."
I'm attaching a text file giving an example of the raw text.
As a workaround I wrote a macro in template toolkit:
[% MACRO utfclean(field) PERL %]
use utf8;
my $field = $stash->get('field');
utf8::encode($field);
print $field ;
[% END %]
[% utfclean( doc.value_for('description') ) %]
| Subject: | example.wsr.txt |
Original entry (explodes)
This program showcases Piffaro's legendary instrumentarium and versatile musicians, in music from French, Flemish and English composers from the late 15th through the mid 16th centuries. The winner of Piffaroâs 2013 recorder competition, Martin Bernstein, joins the ensemble, blending his recorder sound with the group, and thrilling all with his virtuosity and fine musicianship.
Entity version is OK
This program showcases Piffaro's legendary instrumentarium and versatile musicians, in music from French, Flemish and English composers from the late 15th through the mid 16th centuries. The winner of Piffaro’s 2013 recorder competition, Martin Bernstein, joins the ensemble, blending his recorder sound with the group, and thrilling all with his virtuosity and fine musicianship.