Skip Menu |

This queue is for tickets about the ExtUtils-MakeMaker CPAN distribution.

Report information
The Basics
Id: 106461
Status: open
Priority: 0/
Queue: ExtUtils-MakeMaker

People
Owner: Nobody in particular
Requestors: vitaliy.tokarev [...] gmail.com
SREZIC [...] cpan.org
Cc: ribasushi [...] leporine.io
AdminCc:

Bug Information
Severity: Unimportant
Broken in: 7.04
Fixed in: (no value)



Subject: Miscorrect usage of ":encoding" pragma?
Hello, When installing some modules via CPAN I've got next messages: "\x{00f6}" does not map to ascii at /home/tvv/.stablestaticperl-5.22.0-1.001/perl/lib/ExtUtils/MakeMaker.pm line 1182. Environment =========== * GNU/Linux Debian 7.8 * Perl: staticperl + stableperl 5.22.0 * ExtUtils::MakeMaker 7.04 (this bug affected versions up to 7.05_27 as I see) * locale LANG=en_US.UTF-8 LANGUAGE=en_US.UTF-8 LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL= How to test? ============ perl -MCPAN -e shell test Module::Compile Documentation "perldoc open" and "perldoc encoding" contains nothing about ":encoding(locale)". Instead of it there are notes about usage of ":locale" pragma. So, if I apply the patch below the warning message disappears: --- /home/tvv/.stablestaticperl-5.22.0-1.001/perl/lib/ExtUtils/MakeMaker.pm.orig 2015-08-14 22:37:35.000000000 +0300 +++ /home/tvv/.stablestaticperl-5.22.0-1.001/perl/lib/ExtUtils/MakeMaker.pm 2015-08-14 22:37:48.000000000 +0300 @@ -1172,7 +1172,7 @@ unlink($finalname, "MakeMaker.tmp", $Is_VMS ? 'Descrip.MMS' : ()); open(my $fh,">", "MakeMaker.tmp") or die "Unable to open MakeMaker.tmp: $!"; - binmode $fh, ':encoding(locale)' if $CAN_DECODE; + binmode $fh, ':locale' if $CAN_DECODE; for my $chunk (@{$self->{RESULT}}) { my $to_write = $chunk; Is it a typo or am I wrong? Can you fix that? Thanks!
On 2015-08-14 12:44:23, gh0stwizard wrote: Show quoted text
> Hello, > > When installing some modules via CPAN I've got next messages: > > "\x{00f6}" does not map to ascii at /home/tvv/.stablestaticperl- > 5.22.0-1.001/perl/lib/ExtUtils/MakeMaker.pm line 1182.
What distribution were you installing when this happened?
On Fri Aug 14 15:44:23 2015, gh0stwizard wrote: Show quoted text
> Hello, > > When installing some modules via CPAN I've got next messages: > > "\x{00f6}" does not map to ascii at /home/tvv/.stablestaticperl- > 5.22.0-1.001/perl/lib/ExtUtils/MakeMaker.pm line 1182. > > > Environment > =========== > > * GNU/Linux Debian 7.8 > * Perl: staticperl + stableperl 5.22.0 > * ExtUtils::MakeMaker 7.04 (this bug affected versions up to 7.05_27 > as I see) > * locale > LANG=en_US.UTF-8 > LANGUAGE=en_US.UTF-8 > LC_CTYPE="en_US.UTF-8" > LC_NUMERIC="en_US.UTF-8" > LC_TIME="en_US.UTF-8" > LC_COLLATE="en_US.UTF-8" > LC_MONETARY="en_US.UTF-8" > LC_MESSAGES="en_US.UTF-8" > LC_PAPER="en_US.UTF-8" > LC_NAME="en_US.UTF-8" > LC_ADDRESS="en_US.UTF-8" > LC_TELEPHONE="en_US.UTF-8" > LC_MEASUREMENT="en_US.UTF-8" > LC_IDENTIFICATION="en_US.UTF-8" > LC_ALL= > > > How to test? > ============ > > perl -MCPAN -e shell > test Module::Compile > > > Documentation "perldoc open" and "perldoc encoding" contains nothing > about ":encoding(locale)". Instead of it there are notes about usage > of ":locale" pragma. > > So, if I apply the patch below the warning message disappears: > > --- /home/tvv/.stablestaticperl-5.22.0- > 1.001/perl/lib/ExtUtils/MakeMaker.pm.orig 2015-08-14 > 22:37:35.000000000 +0300 > +++ /home/tvv/.stablestaticperl-5.22.0- > 1.001/perl/lib/ExtUtils/MakeMaker.pm 2015-08-14 22:37:48.000000000 > +0300 > @@ -1172,7 +1172,7 @@ > unlink($finalname, "MakeMaker.tmp", $Is_VMS ? 'Descrip.MMS' : > ()); > open(my $fh,">", "MakeMaker.tmp") > or die "Unable to open MakeMaker.tmp: $!"; > - binmode $fh, ':encoding(locale)' if $CAN_DECODE; > + binmode $fh, ':locale' if $CAN_DECODE; > > for my $chunk (@{$self->{RESULT}}) { > my $to_write = $chunk; > > > Is it a typo or am I wrong? Can you fix that? > > Thanks!
The :locale from open.pm/encoding.pm is not a true PerlIO layer. And given the actual layer is unlikely to be installed, that succeeds because it actually fails to apply silently. This is certainly not the right fix. It would be helpful to know if ExtUtils::MakeMaker::Locale loads without issues, and if so what it thinks the locale's encodings are. The error message suggests Encode::Locale is incorrectly picking it up as ASCII, EUMM::Locale should give the same answer (this potential mismatch worries me a bit, and the fallback for this scenario only seems to take the latter into account). Leon
From: vitaliy.tokarev [...] gmail.com
On Fri Aug 14 17:40:45 2015, LEONT wrote: Show quoted text
> On Fri Aug 14 15:44:23 2015, gh0stwizard wrote:
> > Hello, > > > > When installing some modules via CPAN I've got next messages: > > > > "\x{00f6}" does not map to ascii at /home/tvv/.stablestaticperl- > > 5.22.0-1.001/perl/lib/ExtUtils/MakeMaker.pm line 1182. > > > > > > Environment > > =========== > > > > * GNU/Linux Debian 7.8 > > * Perl: staticperl + stableperl 5.22.0 > > * ExtUtils::MakeMaker 7.04 (this bug affected versions up to 7.05_27 > > as I see) > > * locale > > LANG=en_US.UTF-8 > > LANGUAGE=en_US.UTF-8 > > LC_CTYPE="en_US.UTF-8" > > LC_NUMERIC="en_US.UTF-8" > > LC_TIME="en_US.UTF-8" > > LC_COLLATE="en_US.UTF-8" > > LC_MONETARY="en_US.UTF-8" > > LC_MESSAGES="en_US.UTF-8" > > LC_PAPER="en_US.UTF-8" > > LC_NAME="en_US.UTF-8" > > LC_ADDRESS="en_US.UTF-8" > > LC_TELEPHONE="en_US.UTF-8" > > LC_MEASUREMENT="en_US.UTF-8" > > LC_IDENTIFICATION="en_US.UTF-8" > > LC_ALL= > > > > > > How to test? > > ============ > > > > perl -MCPAN -e shell > > test Module::Compile > > > > > > Documentation "perldoc open" and "perldoc encoding" contains nothing > > about ":encoding(locale)". Instead of it there are notes about usage > > of ":locale" pragma. > > > > So, if I apply the patch below the warning message disappears: > > > > --- /home/tvv/.stablestaticperl-5.22.0- > > 1.001/perl/lib/ExtUtils/MakeMaker.pm.orig 2015-08-14 > > 22:37:35.000000000 +0300 > > +++ /home/tvv/.stablestaticperl-5.22.0- > > 1.001/perl/lib/ExtUtils/MakeMaker.pm 2015-08-14 > > 22:37:48.000000000 > > +0300 > > @@ -1172,7 +1172,7 @@ > > unlink($finalname, "MakeMaker.tmp", $Is_VMS ? 'Descrip.MMS' : > > ()); > > open(my $fh,">", "MakeMaker.tmp") > > or die "Unable to open MakeMaker.tmp: $!"; > > - binmode $fh, ':encoding(locale)' if $CAN_DECODE; > > + binmode $fh, ':locale' if $CAN_DECODE; > > > > for my $chunk (@{$self->{RESULT}}) { > > my $to_write = $chunk; > > > > > > Is it a typo or am I wrong? Can you fix that? > > > > Thanks!
> > The :locale from open.pm/encoding.pm is not a true PerlIO layer. And > given the actual layer is unlikely to be installed, that succeeds > because it actually fails to apply silently. This is certainly not the > right fix. > > It would be helpful to know if ExtUtils::MakeMaker::Locale loads > without issues, and if so what it thinks the locale's encodings are. > The error message suggests Encode::Locale is incorrectly picking it up > as ASCII, EUMM::Locale should give the same answer (this potential > mismatch worries me a bit, and the fallback for this scenario only > seems to take the latter into account). > > Leon
I've to modify MakeMaker.pm (version 7.04), function "sub flush": 1168 my $finalname = $self->{MAKEFILE}; 1169 print "Generating a $type $finalname\n"; 1170 print "Writing $finalname for $self->{NAME}\n"; 1171 1172 unlink($finalname, "MakeMaker.tmp", $Is_VMS ? 'Descrip.MMS' : ()); 1173 1174 warn ">>> CAN_DECODE = $CAN_DECODE\n"; 1175 1176 open(my $fh,">", "MakeMaker.tmp") 1177 or die "Unable to open MakeMaker.tmp: $!"; 1178 1179 if ( $CAN_DECODE ) { 1180 my $mode = binmode $fh, ':locale'; 1181 warn sprintf ">>> BINMODE = %s\n", $mode ? $mode : $!; 1182 } 1183 1184 for my $chunk (@{$self->{RESULT}}) { Testing on an installation the module "Module::Compile". Futher investigation shows next things: cpan[1]> test Module::Compile Reading '/home/tvv/.stablestaticperl-5.22.0-1.001/cpan/Metadata' Database was generated on Fri, 14 Aug 2015 11:41:02 GMT Running test for module 'Module::Compile' Checksum for /home/tvv/.stablestaticperl-5.22.0-1.001/cpan/sources/authors/id/I/IN/INGY/Module-Compile-0.35.tar.gz ok Scanning cache /home/tvv/.stablestaticperl-5.22.0-1.001/cpan/build for sizes ............................................................................DONE tmp-2798 for tmp-2798: No such file or directory at /home/tvv/.stablestaticperl-5.22.0-1.001/perl/ lib/CPAN/Distribution.pm line 468. Configuring I/IN/INGY/Module-Compile-0.35.tar.gz with Makefile.PL Checking if your kit is complete... Looks good Generating a Unix-style Makefile Writing Makefile for Module::Compile Show quoted text
>>> CAN_DECODE = 1 >>> BINMODE = No such file or directory
Writing MYMETA.yml and MYMETA.json INGY/Module-Compile-0.35.tar.gz /home/tvv/.stablestaticperl-5.22.0-1.001/perl/bin/perl Makefile.PL MAP_TARGET=perl -- OK Running make for I/IN/INGY/Module-Compile-0.35.tar.gz cp lib/Module/Optimize.pm blib/lib/Module/Optimize.pm cp lib/Module/Compile.pod blib/lib/Module/Compile.pod ... If I push back my changes to an original: 1179 if ( $CAN_DECODE ) { 1180 my $mode = binmode $fh, ':encoding(locale)'; 1181 warn sprintf ">>> BINMODE = %s\n", $mode ? $mode : $!; 1182 } Then I got next output: Configuring I/IN/INGY/Module-Compile-0.35.tar.gz with Makefile.PL Checking if your kit is complete... Looks good Generating a Unix-style Makefile Writing Makefile for Module::Compile Show quoted text
>>> CAN_DECODE = 1 >>> BINMODE = 1
"\x{00f6}" does not map to ascii at /home/tvv/.stablestaticperl-5.22.0-1.001/perl/lib/ExtUtils/Mak eMaker.pm line 1189. "\x{00f6}" does not map to ascii at /home/tvv/.stablestaticperl-5.22.0-1.001/perl/lib/ExtUtils/Mak eMaker.pm line 1189. "\x{00f6}" does not map to ascii at /home/tvv/.stablestaticperl-5.22.0-1.001/perl/lib/ExtUtils/Mak eMaker.pm line 1189. "\x{00f6}" does not map to ascii at /home/tvv/.stablestaticperl-5.22.0-1.001/perl/lib/ExtUtils/Mak eMaker.pm line 1189. Writing MYMETA.yml and MYMETA.json INGY/Module-Compile-0.35.tar.gz /home/tvv/.stablestaticperl-5.22.0-1.001/perl/bin/perl Makefile.PL MAP_TARGET=perl -- OK Running make for I/IN/INGY/Module-Compile-0.35.tar.gz ... So I was wrong and usage ":encoding(locale)" is correct. Futher investigation. I have look into encoding.pm and run cpan shell with PERL_ENCODE_DEBUG=1 environemnt. And now I see where is problem: ... Generating a Unix-style Makefile Writing Makefile for Module::Compile Show quoted text
>>> CAN_DECODE = 1
find_alias(Encode, locale)->name = ascii at /home/tvv/.stablestaticperl-5.22.0-1.001/perl/lib/Enco de/Alias.pm line 75. Show quoted text
>>> BINMODE = 1
"\x{00f6}" does not map to ascii at /home/tvv/.stablestaticperl-5.22.0-1.001/perl/lib/ExtUtils/Mak eMaker.pm line 1189. ... Well, quick looking at Encode::* module(s) does not give any glue about what is "locale" encoding is. From encoding.pm, I've try this command: perl -MI18N::Langinfo -le 'I18N::Langinfo->import(qw(langinfo CODESET)); print langinfo( CODESET() );' and its returns "UTF-8". I'll try to investigate the problem futher. Thanks!
From: vitaliy.tokarev [...] gmail.com
I've look in EUMM/Locale.pm and I comment out the line 128: use Data::Dump; ddx $ENCODING_LOCALE, $ENCODING_LOCALE_FS, $ENCODING_CONSOLE_IN, $ENCODING_CONSOLE_IN, $ENCODING_CONSOLE_OUT; and I've got next output: # Locale.pm:128: ( # "ANSI_X3.4-1968", # "ANSI_X3.4-1968", # "ANSI_X3.4-1968", # "ANSI_X3.4-1968", # )
From: vitaliy.tokarev [...] gmail.com
On Sat Aug 15 05:36:17 2015, gh0stwizard wrote: Show quoted text
> I've look in EUMM/Locale.pm and I comment out the line 128: > > use Data::Dump; ddx $ENCODING_LOCALE, $ENCODING_LOCALE_FS, > $ENCODING_CONSOLE_IN, $ENCODING_CONSOLE_IN, $ENCODING_CONSOLE_OUT; > > and I've got next output: > > # Locale.pm:128: ( > # "ANSI_X3.4-1968", > # "ANSI_X3.4-1968", > # "ANSI_X3.4-1968", > # "ANSI_X3.4-1968", > # )
I've found why I got this strange result. It was inside staticperl script which sets LC_ALL=C. In that case langinfo( CODESET ) returns "ANSI_X3.4-1968". I've run "~/staticperl cpan" to open CPAN shell instead of "perl -MCPAN -e shell". As I see ExtUtils/MakeMaker.pm contains next lines: 14 eval { ExtUtils::MakeMaker::Locale::reinit('UTF-8') } 15 if $CAN_DECODE and $ExtUtils::MakeMaker::Locale::ENCODING_LOCALE eq 'US-ASCII'; Perhaps it should be expanded/changed? Thanks!
Subject: Re: [rt.cpan.org #106461] Miscorrect usage of ":encoding" pragma?
Date: Sat, 15 Aug 2015 13:30:54 +0200
To: bug-ExtUtils-MakeMaker [...] rt.cpan.org
From: Leon Timmermans <fawaka [...] gmail.com>
On Sat, Aug 15, 2015 at 12:18 PM, Vitaliy Tokarev via RT < bug-ExtUtils-MakeMaker@rt.cpan.org> wrote: Show quoted text
> Queue: ExtUtils-MakeMaker > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=106461 > > > On Sat Aug 15 05:36:17 2015, gh0stwizard wrote:
> > I've look in EUMM/Locale.pm and I comment out the line 128: > > > > use Data::Dump; ddx $ENCODING_LOCALE, $ENCODING_LOCALE_FS, > > $ENCODING_CONSOLE_IN, $ENCODING_CONSOLE_IN, $ENCODING_CONSOLE_OUT; > > > > and I've got next output: > > > > # Locale.pm:128: ( > > # "ANSI_X3.4-1968", > > # "ANSI_X3.4-1968", > > # "ANSI_X3.4-1968", > > # "ANSI_X3.4-1968", > > # )
> > I've found why I got this strange result. It was inside staticperl script > which sets LC_ALL=C. In that case langinfo( CODESET ) returns > "ANSI_X3.4-1968". I've run "~/staticperl cpan" to open CPAN shell instead > of "perl -MCPAN -e shell". >
That is unfortunate. Show quoted text
> As I see ExtUtils/MakeMaker.pm contains next lines: > > 14 eval { ExtUtils::MakeMaker::Locale::reinit('UTF-8') } > 15 if $CAN_DECODE and $ExtUtils::MakeMaker::Locale::ENCODING_LOCALE > eq 'US-ASCII'; > > Perhaps it should be expanded/changed? >
It's overriding ExtUtils::MakeMaker::Locale, but ':encoding(locale)' is using Encode::Locale. I suspect the easiest solution is to use something like ":encoding($ExtUtils::MakeMaker::Locale::ENCODING_LOCALE)" instead. Leon
From: ntyni [...] iki.fi
On Sat Aug 15 07:31:37 2015, fawaka@gmail.com wrote: Show quoted text
> On Sat, Aug 15, 2015 at 12:18 PM, Vitaliy Tokarev via RT <
Show quoted text
> It's overriding ExtUtils::MakeMaker::Locale, but ':encoding(locale)' is > using Encode::Locale. I suspect the easiest solution is to use something > like ":encoding($ExtUtils::MakeMaker::Locale::ENCODING_LOCALE)" instead.
As a data point, we're seeing this in Debian / 5.22.2 with the Vitacilina distribution, where it actually causes sporadic crashes when running Makefile.PL because of https://rt.cpan.org/Public/Bug/Display.html?id=65541 in Encode. There are some debugger traces in https://bugs.debian.org/835989 for the Encode side. -- Niko Tyni ntyni@debian.org
I have same errors trying to install modules into docker. How to reproduce: 1. install docker 2. try to build image from attached Dockerfile
Subject: Dockerfile
Download Dockerfile
application/octet-stream 1.9k

Message body not shown because it is not plain text.