Skip Menu |

This queue is for tickets about the Term-ReadLine-Gnu CPAN distribution.

Maintainer(s)' notes

When you report a bug, please provide the following information; 

- output of
	perl -V
	perl Makefile.PL verbose
	make test TEST_VERBOSE=1
	perl -Mblib t/00checkver.t
	echo $TERM
- terminal emulator which you are using
- compiler which is used to compile the GNU Readline Library (libreadline.a) if you can know.
Read INSTALL in the distribution for more details.

Report information
The Basics
Id: 104239
Status: resolved
Priority: 0/
Queue: Term-ReadLine-Gnu
People
Owner: HAYASHI [...] cpan.org
Requestors: ambs [...] cpan.org
Cc:
AdminCc:
Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 1.32

History Show all quoted text
  Tue May 05 14:41:54 2015 ambs [...] cpan.org - Ticket created
Subject: Enable UTF8 in different ways
Hello I noticed there is code to handle unicode if the -C flag is used ($^{UNICODE} variable). I feel most people do not use the -C flag at all, and it would be nice if it was possible to just enable unicode read/output from standard Perl code. The patch in attach is doing so. I am using it on a tool that uses unicode in most directions (printing in the OUT handler, reading unicode from readline, unputting unicode strings in readline method, etc) and it is working perfectly. So, if you are happy with it, I would be very grateful if the patch could be applied. Best regards, Alberto
Subject: trg.diff
Index: Gnu.pm =================================================================== --- Gnu.pm (revision 485) +++ Gnu.pm (working copy) @@ -141,7 +141,7 @@ preput => 1, attribs => 1, newTTY => 1, tkRunning => Term::ReadLine::Stub->Features->{'tkRunning'}, ornaments => Term::ReadLine::Stub->Features->{'ornaments'}, - stiflehistory => 1, + stiflehistory => 1, unicode => 0, ); # keep rl_readline_version value for efficiency @@ -350,7 +350,7 @@ # from ReadLine.pm: convert to the internal representation from UTF-8 # see 'perldoc perlvar' - if ((${^UNICODE} & 1 || defined ${^ENCODING}) && + if (($Features{unicode} || ${^UNICODE} & 1 || defined ${^ENCODING}) && utf8::valid($line)) { #utf8::upgrade($line); utf8::decode($line); @@ -505,6 +505,19 @@ select($sel); } +=item C<enableUTF8> + +activates unicode support, binmoding output file handle, +and preencoding input. + +=cut + +sub enableUTF8 { + my ($self) = @_; + $Features{unicode} = 1; + binmode $self->OUT, ":utf8"; +} + =back =cut
  Tue May 05 16:43:31 2015 ether [...] cpan.org - Correspondence added
binmode $self->OUT, ":utf8"; should be: binmode $self->OUT, ":encoding(UTF-8)";
  Tue May 05 16:43:31 2015 The RT System itself - Status changed from 'new' to 'open'
  Sat May 16 22:16:46 2015 HAYASHI [...] cpan.org - Correspondence added
Hi, Thank you for your patch. But I'm negative on it. It should be applied on ReadLine.pm first. I have been trying to make Term-ReadLine-Gnu compatible with it. By the way I think "use utf8:all;" can be your alternative solution.
  Sat May 16 22:16:47 2015 HAYASHI [...] cpan.org - Status changed from 'open' to 'rejected'
  Sun Aug 09 06:30:36 2015 ambs [...] cpan.org - Correspondence added
On Sat May 16 22:16:46 2015, HAYASHI wrote: Show quoted text
> Hi, > > Thank you for your patch. > > But I'm negative on it. It should be applied on ReadLine.pm first. I > have been trying to make Term-ReadLine-Gnu compatible with it.
Having further options doesn't make it incompatible. Anyway, will try to find out who maintains Term::ReadLine and see if I can convince him to fix this problem. Show quoted text
> By the way I think "use utf8:all;" can be your alternative solution.
I am against the usage of hammers. Will, for now, ressurect the patch in my machine, so I can use unicode with Term::ReadLine::Gnu :(
  Tue Mar 01 11:09:33 2016 HAYASHI [...] cpan.org - Correspondence added
I released Term-ReadLine-Gnu-1.29 which fixed this bug. Now TRG calls utf8::decode() for a UTF8-enabled input filehandle which has the utf8 PerlIO layer.
  Tue Mar 01 11:09:35 2016 HAYASHI [...] cpan.org - Status changed from 'rejected' to 'resolved'
  Tue Mar 01 11:09:37 2016 HAYASHI [...] cpan.org - Taken
  Tue Mar 01 11:09:38 2016 HAYASHI [...] cpan.org - Fixed in 1.29 added
  Sun Jun 05 06:12:42 2016 HAYASHI [...] cpan.org - Correspondence added
Hi, Show quoted text
> binmode $self->OUT, ":utf8"; > > should be: > > binmode $self->OUT, ":encoding(UTF-8)";
Why ":encoding(UTF-8)" is better than ":utf8"? http://perldoc.perl.org/open.html uses ":utf8". If you have a good reference (URL) for the reason, let me konow. I want put it in my comment. Thanks.
  Sun Jun 05 06:19:28 2016 ambs [...] perl-hackers.net - Correspondence added
Subject: Re: [rt.cpan.org #104239] Enable UTF8 in different ways
Date: Sun, 5 Jun 2016 11:19:02 +0100
To: bug-Term-ReadLine-Gnu [...] rt.cpan.org, ambs [...] cpan.org
From: Alberto Simoes <ambs [...] perl-hackers.net>
On 05/06/16 11:12, Hiroo_HAYASHI via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=104239 > > > Hi, >
>> binmode $self->OUT, ":utf8"; >> >> should be: >> >> binmode $self->OUT, ":encoding(UTF-8)";
> > Why ":encoding(UTF-8)" is better than ":utf8"? > > http://perldoc.perl.org/open.html uses ":utf8". > If you have a good reference (URL) for the reason, let me konow. I want put it in my comment.
Hi From perlio: :utf8 (CAUTION: This layer does not validate byte sequences. For reading input, you should instead use ":encoding(utf8)" instead of bare ":utf8".) I notice you are opening for output, so is not that important. Although some folks hanging on #p5p usually suggest the :encoding usage both for input and output. Best Alberto
  Sun Jun 05 09:55:52 2016 HAYASHI [...] cpan.org - Correspondence added
Hi, Thank you for your quick response. http://perldoc.perl.org/PerlIO.html I did not check pages under "Core modules". Thanks! Show quoted text
> :utf8 > > (CAUTION: This layer does not validate byte sequences. For reading > input, you should instead use ":encoding(utf8)" instead of bare > ":utf8".) > > I notice you are opening for output, so is not that important. > Although > some folks hanging on #p5p usually suggest the :encoding usage both > for > input and output.
  Tue Jun 07 23:36:58 2016 HAYASHI [...] cpan.org - Correspondence added
Hi, Last night I released Term::ReadLine::Gnu-1.32. It includes improvement of UTF-8 support. I decided to have enableUTF8 method which you proposed. ---- enableUTF8 Enables UTF-8 support. If STDIN is in UTF-8 by the -C command-line switch or PERL_UNICODE environment variable, or IN file handle has utf8 IO layer, then UTF-8 support is also enabled. In other cases you need this enableUTF8 method. ---- On Sun, 9 Aug 2015 10:30:36 GMT, AMBS wrote: Show quoted text
> On Sat May 16 22:16:46 2015, HAYASHI wrote:
> > Hi, > > > > Thank you for your patch. > > > > But I'm negative on it. It should be applied on ReadLine.pm first. I > > have been trying to make Term-ReadLine-Gnu compatible with it.
> > Having further options doesn't make it incompatible. Anyway, will try > to find out who maintains Term::ReadLine and see if I can convince him > to fix this problem. >
> > By the way I think "use utf8:all;" can be your alternative solution.
> > I am against the usage of hammers. > > Will, for now, ressurect the patch in my machine, so I can use unicode > with Term::ReadLine::Gnu :(
  Tue Jun 07 23:36:58 2016 HAYASHI [...] cpan.org - Fixed in 1.32 added
  Tue Jun 07 23:36:59 2016 HAYASHI [...] cpan.org - Fixed in 1.29 deleted
  Wed Jun 08 05:43:46 2016 ambs [...] perl-hackers.net - Correspondence added
Subject: Re: [rt.cpan.org #104239] Enable UTF8 in different ways
Date: Wed, 8 Jun 2016 10:43:30 +0100
To: bug-Term-ReadLine-Gnu [...] rt.cpan.org, ambs [...] cpan.org
From: Alberto Simoes <ambs [...] perl-hackers.net>
On 08/06/16 04:36, Hiroo_HAYASHI via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=104239 > > > Hi, > > Last night I released Term::ReadLine::Gnu-1.32. It includes improvement of UTF-8 support. > I decided to have enableUTF8 method which you proposed. > > ---- > enableUTF8 > > Enables UTF-8 support. > > If STDIN is in UTF-8 by the -C command-line switch or PERL_UNICODE environment variable, or IN file handle has utf8 IO layer, then UTF-8 support is also enabled. In other cases you need this enableUTF8 method. > ---- >
Just great. Thanks!