Skip Menu |

This queue is for tickets about the ExtUtils-MakeMaker CPAN distribution.

Report information
The Basics
Id: 100268
Status: resolved
Worked: 30 min
Priority: 0/
Queue: ExtUtils-MakeMaker

People
Owner: ETJ [...] cpan.org
Requestors: ppisar [...] redhat.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: 7.02
Fixed in: 7.05_03

Attachments
0001-Write-UTF-8-encoded-chunk-if-Locale-Encode-is-not-av.patch



Subject: t/unicode.t fails if Locale::Encode is not available: Wide character in print
Since EU::MM 7.00, Locale::Encode module is bundled (is that necessary, will it become a core module?). If I unbundle the module and remove it from the system as it's optional everywhere in the EU::MM, t/unicode.t from 7.02 fails: $ prove -l -v t/unicode.t t/unicode.t .. 1..9 ok 1 - setup ok 2 - chdir'd to Problem-Module # Locale env vars: LANG=en_US.UTF-8 not ok 3 - no warning ok 4 - utf8 abstract # Failed test 'no warning' # at t/unicode.t line 49. # got: '3' # expected: '0' # Wide character in print at /home/test/fedora/perl-ExtUtils-MakeMaker/ExtUtils-MakeMaker-7.02/lib/ExtUtils/MakeMaker.pm line 1202. # Wide character in print at /home/test/fedora/perl-ExtUtils-MakeMaker/ExtUtils-MakeMaker-7.02/lib/ExtUtils/MakeMaker.pm line 1202. # Wide character in print at /home/test/fedora/perl-ExtUtils-MakeMaker/ExtUtils-MakeMaker-7.02/lib/ExtUtils/MakeMaker.pm line 1202. ok 5 - Exit code of make == 0 [...] The line 1202 is the print in this code: unlink($finalname, "MakeMaker.tmp", $Is_VMS ? 'Descrip.MMS' : ()); open(my $fh,">", "MakeMaker.tmp") or die "Unable to open MakeMaker.tmp: $!"; binmode $fh, ':encoding(locale)' if $CAN_DECODE; for my $chunk (@{$self->{RESULT}}) { my $to_write = "$chunk\n"; if (!$CAN_DECODE && $] > 5.008) { utf8::encode $to_write; } print $fh "$chunk\n" or die "Can't write to MakeMaker.tmp: $!"; } The fix is to print the $to_write instead of the $chunk. It looks like an overlooking because this branch happens only Locale::Encode is not available.
From: ppisar [...] redhat.com
Dne Út 11.lis.2014 08:36:24, ppisar napsal(a): Show quoted text
> The fix is to print the $to_write instead of the $chunk. It looks like > an overlooking because this branch happens only Locale::Encode is not > available.
Attached patch implements it. -- Petr
Subject: 0001-Write-UTF-8-encoded-chunk-if-Locale-Encode-is-not-av.patch
From 079898d941daec04fe277f850fe571a3b8b18fde Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petr=20P=C3=ADsa=C5=99?= <ppisar@redhat.com> Date: Tue, 11 Nov 2014 14:30:08 +0100 Subject: [PATCH] Write UTF-8-encoded chunk if Locale::Encode is not available MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CPAN RT#100268 Signed-off-by: Petr Písař <ppisar@redhat.com> --- lib/ExtUtils/MakeMaker.pm | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/ExtUtils/MakeMaker.pm b/lib/ExtUtils/MakeMaker.pm index 028925b..8dfdc47 100644 --- a/lib/ExtUtils/MakeMaker.pm +++ b/lib/ExtUtils/MakeMaker.pm @@ -1175,11 +1175,11 @@ sub flush { binmode $fh, ':encoding(locale)' if $CAN_DECODE; for my $chunk (@{$self->{RESULT}}) { - my $to_write = "$chunk\n"; + my $to_write = "$chunk"; if (!$CAN_DECODE && $] > 5.008) { utf8::encode $to_write; } - print $fh "$chunk\n" + print $fh "$to_write\n" or die "Can't write to MakeMaker.tmp: $!"; } -- 1.9.3
On Tue Nov 11 08:37:45 2014, ppisar wrote: Show quoted text
> Dne Út 11.lis.2014 08:36:24, ppisar napsal(a):
> > The fix is to print the $to_write instead of the $chunk. It looks like > > an overlooking because this branch happens only Locale::Encode is not > > available.
> > Attached patch implements it. > > -- Petr >
By "bundled" do you mean that it is included as ExtUtils::MakeMaker::Locale? As distinct from included in the bundled/ directory of the distribution? Are you saying that you have removed ExtUtils::MakeMaker::Locale? or that you removed Encode::Locale? The latter will have no difference (my entire testing environment doesn't have Encode::Locale on perls from v5.20.1 all the way back to v5.6.2).
Subject: Re: [rt.cpan.org #100268] t/unicode.t fails if Locale::Encode is not available: Wide character in print
Date: Tue, 11 Nov 2014 15:22:35 +0100
To: BINGOS via RT <bug-ExtUtils-MakeMaker [...] rt.cpan.org>
From: Petr Pisar <ppisar [...] redhat.com>
On Tue, Nov 11, 2014 at 09:01:21AM -0500, BINGOS via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=100268 > > > By "bundled" do you mean that it is included as ExtUtils::MakeMaker::Locale? >
Yes. I removed the ExtUtils::MakeMaker::Locale as it is a word-by-word copy of Encode::Locale from Encode-Locale-1.03. Of course I changed the other code to attempt to load the Encode::Locale instead of ExtUtils::MakeMaker::Locale. Show quoted text
> As distinct from included in the bundled/ directory of the distribution? >
There is no Locale.pm in the bundled directory. However if you ask, yes, I prune this directory too. Show quoted text
> Are you saying that you have removed ExtUtils::MakeMaker::Locale? or that > you removed Encode::Locale? >
Both of them. Based on the $CAN_DECODE boolean, I got impression that the feature is optional. -- Petr
Download (untitled)
application/pgp-signature 213b

Message body not shown because it is not plain text.

If the Perl installation can't encode UTF-8, it doesn't try. This means wide characters can end up attempting to get printed. It seems you've carefully disabled EUMM's UTF-8 encoding capability, then reported a "bug" that it's then not doing UTF-8 encoding. Did I miss something?
Subject: Re: [rt.cpan.org #100268] t/unicode.t fails if Locale::Encode is not available: Wide character in print
Date: Thu, 20 Nov 2014 07:55:28 +0100
To: Ed J via RT <bug-ExtUtils-MakeMaker [...] rt.cpan.org>
From: Petr Pisar <ppisar [...] redhat.com>
On Wed, Nov 19, 2014 at 07:39:09PM -0500, Ed J via RT wrote: Show quoted text
> If the Perl installation can't encode UTF-8, it doesn't try.
The Encode::Locale is not about "can't encode UTF-8". It's about ability to set encoding of a handle according to current locale. The code I quoted in my original post does the recoding if Encode::Locale is not available (utf8::encode $to_write;). Show quoted text
> This means wide characters can end up attempting to get printed.
Yes, because there is a bug in the code I quoted in my original post and a fix is available in the patch I sent. Without the patch, the "utf8::encode $to_write;" line is completly pointless because the $to_write is never used. Show quoted text
> It seems you've carefully disabled EUMM's UTF-8 encoding capability, then > reported a "bug" that it's then not doing UTF-8 encoding. >
I disabled the Encode::Locale presence because all the usages are properly handled as optional, yet the test suite cannot cope with the missing feature. That was point of my subquestion why do you handle the Encode::Locale in the library as optional while the test suite does not. And whether this inconsistent status is just temporary, or you want to ship EU::MM with bundled Encode::Locale library (travested as EU::MM::Locale) within the perl in the future. (Because from maintanance point of view, bundling is really bad practise.) In my opinion you should either apply the patch and keep handling the Encode::Locale as optional, or you should handle Encode::Locale as hard dependency and then you can remove all the !$CAN_DECODE branches and coditions. -- Petr
Download (untitled)
application/pgp-signature 213b

Message body not shown because it is not plain text.

A fix has been applied with commit adf284c. Thanks for reporting the bug. If you deliberately delete part of the distribution, you can't really blame the maintainers if it doesn't work properly anymore. It's not being treated as "optional", EUMM is simply robustly maintaining what functionality it can even in a degraded environment.
Closing since released in 7.05_03.