Bug #103101 for Data-Visitor: encoding error at configure time

Wed Mar 25 07:39:27 2015 zefram [...] fysh.org - Ticket created

Subject:	encoding error at configure time
Date:	Wed, 25 Mar 2015 11:39:15 +0000
To:	bug-Data-Visitor [...] rt.cpan.org
From:	Zefram <zefram [...] fysh.org>

$ /opt/perl-5.20.2/bin/perl Makefile.PL Checking if your kit is complete... Looks good Generating a Unix-style Makefile Writing Makefile for Data::Visitor "\x{00c3}" does not map to ascii at /opt/perl-5.20.2/lib/site_perl/5.20.2/ExtUtils/MakeMaker.pm line 1183. "\x{00bc}" does not map to ascii at /opt/perl-5.20.2/lib/site_perl/5.20.2/ExtUtils/MakeMaker.pm line 1183. "\x{00c3}" does not map to ascii at /opt/perl-5.20.2/lib/site_perl/5.20.2/ExtUtils/MakeMaker.pm line 1183. "\x{00bc}" does not map to ascii at /opt/perl-5.20.2/lib/site_perl/5.20.2/ExtUtils/MakeMaker.pm line 1183. "\x{00c3}" does not map to ascii at /opt/perl-5.20.2/lib/site_perl/5.20.2/ExtUtils/MakeMaker.pm line 1183. "\x{00bc}" does not map to ascii at /opt/perl-5.20.2/lib/site_perl/5.20.2/ExtUtils/MakeMaker.pm line 1183. "\x{00c3}" does not map to ascii at /opt/perl-5.20.2/lib/site_perl/5.20.2/ExtUtils/MakeMaker.pm line 1183. "\x{00bc}" does not map to ascii at /opt/perl-5.20.2/lib/site_perl/5.20.2/ExtUtils/MakeMaker.pm line 1183. Writing MYMETA.yml and MYMETA.json The problematic characters here arise from the author name (Marcel's surname) specified in Makefile.PL. There are two problems here. Firstly, EU:MM is trying to write this non-ASCII text to files for which it declares :encoding(locale), so there is no guarantee that non-ASCII text is encodable. I don't know who to blame between EU:MM (using an encoding that's inadequate for the job) and Data-Visitor's Makefile.PL (including non-portable characters in the data that's to be stored by a low-powered mechanism). Secondly, the string that Makefile.PL passes to EU:MM doesn't correctly represent Marcel's name, it's *the UTF-8 encoding of* Marcel's name. The Makefile.PL source specifies the UTF-8 octets in the form of octal escapes. The result is that EU:MM attempts to encode text that's already encoded. On my system, apparently after giving the "does not map to ascii" warning it falls back to using UTF-8 encoding, and so the MYMETA files end up with the name double-UTF8-encoded (octets 0xc3 0x83 0xc2 0xbc in the place of the single non-ASCII character of the real name). I'm guessing that whoever packages this distro has a Latin-1 locale, and so ends up with no warnings and a single layer of UTF-8 encoding in the MYMETA files. -zefram

Wed Mar 25 12:53:08 2015 ether [...] cpan.org - Correspondence added

On 2015-03-25 04:39:27, zefram@fysh.org wrote: Show quoted text

> The problematic characters here arise from the author name (Marcel's > surname) specified in Makefile.PL. There are two problems here.

Vanilla (and modern) EUMM does support wide characters properly, so given that Module::Install will likely never be fully fixed, it would be sensible to move off of it. I shall do so.

Wed Mar 25 12:53:09 2015 The RT System itself - Status changed from 'new' to 'open'

Wed Mar 25 12:53:15 2015 ether [...] cpan.org - Taken

Wed Mar 25 13:20:02 2015 zefram [...] fysh.org - Correspondence added

Subject:	Re: [rt.cpan.org #103101] encoding error at configure time
Date:	Wed, 25 Mar 2015 17:19:50 +0000
To:	Karen Etheridge via RT <bug-Data-Visitor [...] rt.cpan.org>
From:	Zefram <zefram [...] fysh.org>

Karen Etheridge via RT wrote: Show quoted text

>Vanilla (and modern) EUMM does support wide characters properly, so >given that Module::Install will likely never be fully fixed, it would >be sensible to move off of it. I shall do so.

I don't see where Module::Install gets involved in Data-Visitor. That looks like it's using EU:MM directly. Did you get confused with [rt.cpan.org #103112], which is about the same problem showing up via Module::Install in Module-Install's test suite? -zefram

Wed Mar 25 16:42:25 2015 ether [...] cpan.org - Correspondence added

On 2015-03-25 10:20:02, zefram@fysh.org wrote: Show quoted text

> Karen Etheridge via RT wrote:

> >Vanilla (and modern) EUMM does support wide characters properly, so > >given that Module::Install will likely never be fully fixed, it would > >be sensible to move off of it. I shall do so.

> > I don't see where Module::Install gets involved in Data-Visitor. > That looks like it's using EU:MM directly. Did you get confused with > [rt.cpan.org #103112], which is about the same problem showing up via > Module::Install in Module-Install's test suite?

Looks like it! Sorry, too many tabs open at one time :D

Wed Mar 25 16:43:44 2015 ether [...] cpan.org - Given to DOY

Mon Aug 03 15:29:24 2020 ether [...] cpan.org - Correspondence added

This is now solved, with a package retooling.

Mon Aug 03 15:29:26 2020 ether [...] cpan.org - Status changed from 'open' to 'resolved'

Mon Aug 03 15:29:26 2020 ether [...] cpan.org - Fixed in 0.31 added