Bug #123341 for MIME-tools: Re: Bug#879205: MIME::Words::encode_mimewords: double-encodes (produces Mojibake), produces too long lines

On Fri, 20 Oct 2017 16:25:44 -0400 "Thorsten Glaser via RT" <bug-MIME-tools@rt.cpan.org> wrote: Show quoted text
I realized that after I sent the reply. However, even with: my $sample = "Re: Bildungsurlaub f\x{FC}r CCC-Fahrt? [THD#1424195]"; The output is: Out: Re: Bildungsurlaub =?UTF-8?Q?f=C3=BCr=20?=CCC-Fahrt? [THD#1424195] which is correct. Show quoted text
No, probably the original input was in UTF-8 already and was re-encoded by the call to Encode::encode. That's why I always run tests using \x{..} Unicode escapes rather than typing Unicode characters > \x{07f} directly into source code. Regards, Dianne.

Hi, This is not a bug in MIME::tools. The OP misunderstands how Perl works. He typed UTF-8 source code in and is double encoding it. Here's a test program: #=================================================================== use MIME::Words; use Encode; my $sample = "Re: Bildungsurlaub für CCC-Fahrt? [THD#1424195]"; my $utf8 = Encode::encode('UTF-8', $sample); my $out = MIME::Words::encode_mimewords($utf8, Charset => 'UTF-8'); print "Out: $out\n"; #=================================================================== If I run: perl test-utf8.pl Output is: Out: Re: Bildungsurlaub =?UTF-8?Q?f=C3=83=C2=BCr=20?=CCC-Fahrt? [THD#1424195] But that's because the word "für" is *already* UTF-8. If I tell Perl to convert UTF-8 in the source code to native Perl Unicode, the result is very different: perl -Mutf8 test-utf8.pl Output is: Out: Re: Bildungsurlaub =?UTF-8?Q?f=C3=BCr=20?=CCC-Fahrt? [THD#1424195] The OP should read "perldoc utf8" and should also not use UTF-8 directly as Perl source code; use \x{FC} rather than ü, etc. Regards, Dianne.

Dianne Skoll via RT dixit: Show quoted text
Hm, probably. I’d say you just found a bug in OTRS then ;-) (Just now it’s going to be another tricky thing to figure out where exactly and how to fix that. Might report this to the OTRS developers.) One thing I don’t understand is how this was *not* double- encoded in the old version of MIME tools? Thanks, //mirabilos -- 18:47⎜<mirabilos:#!/bin/mksh> well channels… you see, I see everything in the same window anyway 18:48⎜<xpt:#!/bin/mksh> i know, you have some kind of telnet with automatic pong 18:48⎜<mirabilos:#!/bin/mksh> haha, yes :D 18:49⎜<mirabilos:#!/bin/mksh> though that's more tinyirc – sirc is more comfy

Dianne Skoll via RT dixit: Show quoted text
Yes, that was on Debian wheezy. I reported this as bug in Debian against wheezy (which is still supported-ish) first, then as a separate bug against sid because I tried to see if it was still reproducible, and got a different result. Let me dig out version numbers… Original system: otrs2 3.3.18-1~deb7u1 perl 5.14.2-21+deb7u5 libmime-tools-perl 5.503-1 New system: perl 5.26.0-8 libmime-tools-perl 5.508-1 In addition to that, OTRS would have the original subject in a Perl string already, whereas I tried¹ to draft a testcase until I succeeded reproducing the original bug. ① tried, because I don’t really know Perl — I just can program bye, //mirabilos -- (gnutls can also be used, but if you are compiling lynx for your own use, there is no reason to consider using that package) -- Thomas E. Dickey on the Lynx mailing list, about OpenSSL