Bug #20305 for Text-Restructured: Any utf-8 handling ?

Wed Jul 05 14:21:46 2006 Guest - Ticket created

Subject:

Any utf-8 handling ?

I have utf-8 content, which has characters ää and öö (hopefully seen as utf-8 encoded here too). Works fine in plain text, but breaks tables etc (workaround is to count 2-char letters and add extra space, but...). But also header content with utf-8-encoded chars gives interesting results. The command line I've tried is: perl -CD -I blib/lib ./blib/script/prest input.rst output.html (i.e. nothing weird there. However I did the following steps to make prest tool available: $ rm -f Text-Restructured-0.003_013 $ tar zxvf Text-Restructured-0.003_013.tar.gz $ cd Text-Restructured-0.003_013 $ perl Makefile.PL -k $ make $ rm -rf blib/arch blib/bin blib/man1 blib/man3 blib/lib/auto $ rm blib/lib/Text/.exists blib/script/.exists $ rm -rf blib/lib/Text/Restructured/Directive $ for f in latex toc dom xml index xref; do rm -f blib/lib/Text/Restructured/Writer/$f.wrt; done

Subject:

utf8test.rst

Download utf8test.rst
application/octet-stream 151b

Message body not shown because it is not plain text.

Thu Jul 06 15:45:15 2006 nodine [...] cpan.org - TimeEstimated changed from (no value) to '240'

Thu Jul 06 15:45:16 2006 nodine [...] cpan.org - TimeWorked changed from (no value) to '120'

Thu Jul 06 15:45:16 2006 nodine [...] cpan.org - Status changed from 'new' to 'open'

Thu Jul 06 15:45:17 2006 nodine [...] cpan.org - Given to NODINE

Thu Jul 06 15:45:18 2006 nodine [...] cpan.org - Broken in 0.003014 added

Thu Jul 06 15:45:19 2006 nodine [...] cpan.org - Broken in 0.003_011 added

Thu Jul 06 15:45:19 2006 nodine [...] cpan.org - Broken in 0.003_012 added

Thu Jul 06 15:55:55 2006 nodine [...] cpan.org - Correspondence added

On Wed Jul 05 14:21:46 2006, guest wrote: Show quoted text

> I have utf-8 content, which has characters ää and öö (hopefully seen > as utf-8 encoded here too). Works fine in plain text, but breaks > tables etc (workaround is to count 2-char letters and add extra space, > but...). But also header content with utf-8-encoded chars gives > interesting results. > > The command line I've tried is: > > perl -CD -I blib/lib ./blib/script/prest input.rst output.html

I assume you mean with this command line that you're directing stdout to output.html (i.e., "> output.html"). I'm not sure about the encoding of the special characters you have in your section title; they're showing up funny for me in a number of environments. However, the table worked for me using either perl 5.8.0. With perl 5.8.8, if I used "-CSD" it worked, but not with just "-CD". That makes sense, since the output is written to STDOUT. With the section title, 5.8.8 with -CSD gives utf8 "\xE4" does not map to Unicode at blib/lib/Text/Restructured.pm line 2100, <> line 16.

Thu Jul 06 17:01:39 2006 Guest - Correspondence added

From:

Tomi Ollila

On Thu Jul 06 15:55:55 2006, NODINE wrote: Show quoted text

> On Wed Jul 05 14:21:46 2006, guest wrote:

> > I have utf-8 content, which has characters ää and öö (hopefully seen > > as utf-8 encoded here too). Works fine in plain text, but breaks > > tables etc (workaround is to count 2-char letters and add extra space, > > but...). But also header content with utf-8-encoded chars gives > > interesting results. > > > > The command line I've tried is: > > > > perl -CD -I blib/lib ./blib/script/prest input.rst output.html

> > I assume you mean with this command line that you're directing > stdout to output.html (i.e., "> output.html"). > > I'm not sure about the encoding of the special characters you have in > your section title; they're showing up funny for me in a number of > environments.

Sorry for that. I screwed that later using xemacs in different character endocing mode and figured out my mistake just after sending the report. Show quoted text

> However, the table worked for me using either perl 5.8.0. > With perl 5.8.8, if I used "-CSD" it worked, but not with just "-CD". > That makes sense, since the output is written to STDOUT.

Thanks! That fixed my problem. Show quoted text

> With the > section title, 5.8.8 with -CSD gives > > utf8 "\xE4" does not map to Unicode at blib/lib/Text/Restructured.pm > line 2100, <> line 16.

Yes. 0xE4 us ä in latin1 charset:/. Thank you once more. I'm so new with unicode in perl; getting -CD to a command line was interesting enough (and I think I failed to add those in some tests even i thought so) Anyway I consider this bug report invalid (but maybe it would be good to add to readme/other documentation).

Thu Jul 06 17:05:30 2006 nodine [...] cpan.org - Status changed from 'open' to 'resolved'

Thu Jul 06 17:05:30 2006 nodine [...] cpan.org - Fixed in 0.003014 added

Thu Jul 06 17:05:30 2006 nodine [...] cpan.org - Broken in 0.003_013 deleted

Thu Jul 06 17:05:31 2006 nodine [...] cpan.org - Broken in 0.003014 deleted

Thu Jul 06 17:05:31 2006 nodine [...] cpan.org - Broken in 0.003_011 deleted

Thu Jul 06 17:05:31 2006 nodine [...] cpan.org - Broken in 0.003_012 deleted