Skip Menu |

This queue is for tickets about the Tk CPAN distribution.

Report information
The Basics
Id: 123543
Status: open
Priority: 0/
Queue: Tk

People
Owner: Nobody in particular
Requestors: tonkin [...] cpan.org
Cc: CAC [...] cpan.org
AdminCc:

Bug Information
Severity: Critical
Broken in: 804.034
Fixed in: (no value)



Subject: Tk crashes with high unicode characters
Hi, Tk raises a fatal exception when handed a (very) high unicode character. Here is a test script demonstrating the issue: ############################# use strict; use warnings; use feature qw/ say /; binmode STDOUT, ':utf8'; use Try::Tiny; use Tk; my @tests = ("\x{1f596}", "\x{04e7}"); my $mw = tkinit; for my $char ( @tests ) { say "Testing with $char"; my $mw = tkinit; my $t = $mw->Text->pack; $t->insert("end", $char); $t->markSet('insert','end'); $t->focusForce; try { $t->update; say "Updated OK"; } catch { say "Caught fatal exception: $_"; }; $t->destroy; } __END__ ##################### Output: $ perl ~/dev/tk.pl Testing with 🖖 UCS-2BE:code point "\x{1f596}" too high at /home/nick/dev/tk.pl line 13. UCS-2BE:code point "\x{1f596}" too high at /home/nick/dev/tk.pl line 13. Caught fatal exception: UCS-2BE:code point "\x{1f596}" too high at /home/nick/dev/tk.pl line 13. Testing with ӧ Updated OK
On 2017-11-07 10:18:00, TONKIN wrote: Show quoted text
> Hi, > > Tk raises a fatal exception when handed a (very) high unicode > character. Here is a test script demonstrating the issue: > > ############################# > use strict; use warnings; use feature qw/ say /; > binmode STDOUT, ':utf8'; > use Try::Tiny; > use Tk; > > my @tests = ("\x{1f596}", "\x{04e7}"); > my $mw = tkinit; > > for my $char ( @tests ) { > say "Testing with $char"; > my $mw = tkinit; > my $t = $mw->Text->pack; > $t->insert("end", $char); > $t->markSet('insert','end'); > $t->focusForce; > try { > $t->update; > say "Updated OK"; > } catch { > say "Caught fatal exception: $_"; > }; > $t->destroy; > } > > __END__ > ##################### > > Output: > $ perl ~/dev/tk.pl > > Testing with 🖖 > UCS-2BE:code point "\x{1f596}" too high at /home/nick/dev/tk.pl line > 13. > UCS-2BE:code point "\x{1f596}" too high at /home/nick/dev/tk.pl line > 13. > Caught fatal exception: UCS-2BE:code point "\x{1f596}" too high at > /home/nick/dev/tk.pl line 13. > > Testing with ӧ > Updated OK
Can you give me more information on the environment (OS, perl version, Tk version, locale environment variables, maybe else...)? It does not fail for me on a debian/jessie and freebsd9 machine. It does fail on a Windows machine, though the error message is different: UCS-2LE:code point "\x{1f596}" too high at tk.pl line 31. And given your error message is using forward slashes I think your system is not Windows...
On Tue Nov 07 13:48:56 2017, SREZIC wrote: Show quoted text
> On 2017-11-07 10:18:00, TONKIN wrote:
> > Hi, > > > > Tk raises a fatal exception when handed a (very) high unicode > > character. Here is a test script demonstrating the issue: > > > > ############################# > > use strict; use warnings; use feature qw/ say /; > > binmode STDOUT, ':utf8'; > > use Try::Tiny; > > use Tk; > > > > my @tests = ("\x{1f596}", "\x{04e7}"); > > my $mw = tkinit; > > > > for my $char ( @tests ) { > > say "Testing with $char"; > > my $mw = tkinit; > > my $t = $mw->Text->pack; > > $t->insert("end", $char); > > $t->markSet('insert','end'); > > $t->focusForce; > > try { > > $t->update; > > say "Updated OK"; > > } catch { > > say "Caught fatal exception: $_"; > > }; > > $t->destroy; > > } > > > > __END__ > > ##################### > > > > Output: > > $ perl ~/dev/tk.pl > > > > Testing with 🖖 > > UCS-2BE:code point "\x{1f596}" too high at /home/nick/dev/tk.pl line > > 13. > > UCS-2BE:code point "\x{1f596}" too high at /home/nick/dev/tk.pl line > > 13. > > Caught fatal exception: UCS-2BE:code point "\x{1f596}" too high at > > /home/nick/dev/tk.pl line 13. > > > > Testing with ӧ > > Updated OK
> > Can you give me more information on the environment (OS, perl version, > Tk version, locale environment variables, maybe else...)? It does not > fail for me on a debian/jessie and freebsd9 machine. It does fail on a > Windows machine, though the error message is different: > > UCS-2LE:code point "\x{1f596}" too high at tk.pl line 31. > > And given your error message is using forward slashes I think your > system is not Windows...
That's right :-) I'm on Ubuntu Xenial: $ uname -a Linux yoda 4.10.0-37-generic #41~16.04.1-Ubuntu SMP Fri Oct 6 22:42:59 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux $ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.3 LTS Release: 16.04 Codename: xenial $perl -V Summary of my perl5 (revision 5 version 26 subversion 1) configuration: Platform: osname=linux osvers=4.10.0-35-generic archname=x86_64-linux uname='linux yoda 4.10.0-35-generic #39~16.04.1-ubuntu smp wed sep 13 09:02:42 utc 2017 x86_64 x86_64 x86_64 gnulinux ' config_args='-de -Dprefix=/home/nick/perl5/perlbrew/perls/perl-5.26.1 -Aeval:scriptdir=/home/nick/perl5/perlbrew/perls/perl-5.26.1/bin' hint=recommended useposix=true d_sigaction=define useithreads=undef usemultiplicity=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cc' ccflags ='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' optimize='-O2' cppflags='-fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion='' gccversion='5.4.0 20160609' gccosandvers='' intsize=4 longsize=8 ptrsize=8 doublesize=8 byteorder=12345678 doublekind=3 d_longlong=define longlongsize=8 d_longdbl=define longdblsize=16 longdblkind=3 ivtype='long' ivsize=8 nvtype='double' nvsize=8 Off_t='off_t' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='cc' ldflags =' -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/x86_64-linux-gnu/5/include-fixed /usr/include/x86_64-linux-gnu /usr/lib /lib/x86_64-linux-gnu /lib/../lib /usr/lib/x86_64-linux-gnu /usr/lib/../lib /lib libs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.23.so so=so useshrplib=false libperl=libperl.a gnulibc_version='2.23' Dynamic Linking: dlsrc=dl_dlopen.xs dlext=so d_dlsymun=undef ccdlflags='-Wl,-E' cccdlflags='-fPIC' lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong' Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_MALLOC_WRAP PERL_OP_PARENT PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF Locally applied patches: Devel::PatchPerl 1.48 Built under linux Compiled at Oct 4 2017 08:33:06 %ENV: PERLBREW_BASHRC_VERSION="0.80" PERLBREW_HOME="/home/nick/.perlbrew" PERLBREW_MANPATH="/home/nick/perl5/perlbrew/perls/perl-5.26.1/man" PERLBREW_PATH="/home/nick/perl5/perlbrew/bin:/home/nick/perl5/perlbrew/perls/perl-5.26.1/bin" PERLBREW_PERL="perl-5.26.1" PERLBREW_ROOT="/home/nick/perl5/perlbrew" PERLBREW_VERSION="0.80" PERL_CPANM_OPT=" --mirror cpan.mirror.constant.com --mirror cpan.hoovism-http.com --mirror http://ftp.wayne.edu/cpan/ --mirror ftp://mirror.cogentco.com/pub/CPAN/ --mirror ftp://mirror.cc.columbia.edu/pub/software/cpan/ --mirror http://develooper.com --mirror http://cpan.cse.msu.edu/ --mirror http://mirror.cc.columbia.edu/pub/software/cpan/ --mirror http://www.cpan.org/ --mirror http://cpan.perlnow.com/ --mirror http://cpan.mirror.facebook.net/ --mirror http://www.perl.com/CPAN/ --mirror http://cpan.pair.com/ --mirror http://mirror.uta.edu/CPAN/ --mirror http://cpan.cs.utah.edu/ --mirror-only --cascade-search" @INC: /home/nick/perl5/perlbrew/perls/perl-5.26.1/lib/site_perl/5.26.1/x86_64-linux /home/nick/perl5/perlbrew/perls/perl-5.26.1/lib/site_perl/5.26.1 /home/nick/perl5/perlbrew/perls/perl-5.26.1/lib/5.26.1/x86_64-linux /home/nick/perl5/perlbrew/perls/perl-5.26.1/lib/5.26.1

Message body is not shown because it is too large.

The long message above explains that the test succeeds on the system perl, which is 5.22 with lots of DEBPKG:fixes, and using the vendor-packaged Tk.
On 2017-11-07 14:12:15, TONKIN wrote: Show quoted text
> The long message above explains that the test succeeds on the system > perl, which is 5.22 with lots of DEBPKG:fixes, and using the vendor- > packaged Tk.
Another idea: is Tk compiled with XFT support (the system perl's Tk typically is)? Can you send me the output of perl -MTk::Config -e 'warn $Tk::Config::xlib' ? If "-lXft" does not appear in the output, can you install the necessary packages (see README.linux) and rebuild Tk again?
On 2017-11-07 16:08:58, SREZIC wrote: Show quoted text
> On 2017-11-07 14:12:15, TONKIN wrote:
> > The long message above explains that the test succeeds on the system > > perl, which is 5.22 with lots of DEBPKG:fixes, and using the vendor- > > packaged Tk.
> > Another idea: is Tk compiled with XFT support (the system perl's Tk > typically is)? Can you send me the output of > > perl -MTk::Config -e 'warn $Tk::Config::xlib' > > ? If "-lXft" does not appear in the output, can you install the > necessary packages (see README.linux) and rebuild Tk again?
I tried myself the other way: "perl Makefile.PL XFT=0" to create a Tk without XFT support. With that Tk I could reproduce the problem. Tk is using Encode for encoding stuff. Either Encode::FB_QUIET() or Encode::FB_PERLQQ() are set to avoid errors in case some characters could not be encoded/decoded. However, this does not seem to work with ucs2-be: $ perl5.26.1 -MEncode -e '$enc = Encode->getEncoding("ucs2-be"); $x = "\x{1f596}"; warn $enc->encode($x, Encode::FB_PERLQQ())' UCS-2BE:code point "\x{1f596}" too high at -e line 1. $ perl5.26.1 -MEncode -e '$enc = Encode->getEncoding("ucs2-be"); $x = "\x{1f596}"; warn $enc->encode($x, Encode::FB_QUIET())' UCS-2BE:code point "\x{1f596}" too high at -e line 1. For ascii, which supports a much smaller character set than ucs2, it works: $ perl5.26.1 -MEncode -e '$enc = Encode->getEncoding("US-ASCII"); $x = "\x{1f596}"; warn $enc->encode($x, Encode::FB_PERLQQ());' \x{1f596} at -e line 1. So this smells like a problem in Encode.pm...
Subject: Re: [rt.cpan.org #123543] Tk crashes with high unicode characters
Date: Tue, 7 Nov 2017 16:59:06 -0500
To: bug-Tk [...] rt.cpan.org
From: Nick Tonkin <1nickt [...] gmail.com>
Hi Slaven. As you surmised, the failing perl did not have XFT support, whereas the system perl does. I read the README.linux (apologies for not finding it earlier) and installed the system dependencies. Just reinstalled Tk and I see that as documented, it is building with XFT support. I see a lot of tests with high unicode characters that I didn't see on the previous builds. (BTW what a lot of fun to watch the Tk test suite on a fast machine!) And ... the test script now succeeds. I will leave it to you to close the issue or not, given the unhandled exception, and report the relevant issue to the Encode maintainers ? Thanks for helping me get squared away. On Tue, Nov 7, 2017 at 4:45 PM, Slaven_Rezic via RT <bug-Tk@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=123543 > > > On 2017-11-07 16:08:58, SREZIC wrote:
> > On 2017-11-07 14:12:15, TONKIN wrote:
> > > The long message above explains that the test succeeds on the system > > > perl, which is 5.22 with lots of DEBPKG:fixes, and using the vendor- > > > packaged Tk.
> > > > Another idea: is Tk compiled with XFT support (the system perl's Tk > > typically is)? Can you send me the output of > > > > perl -MTk::Config -e 'warn $Tk::Config::xlib' > > > > ? If "-lXft" does not appear in the output, can you install the > > necessary packages (see README.linux) and rebuild Tk again?
> > I tried myself the other way: "perl Makefile.PL XFT=0" to create a Tk > without XFT support. With that Tk I could reproduce the problem. > > Tk is using Encode for encoding stuff. Either Encode::FB_QUIET() or > Encode::FB_PERLQQ() are set to avoid errors in case some characters could > not be encoded/decoded. However, this does not seem to work with ucs2-be: > > $ perl5.26.1 -MEncode -e '$enc = Encode->getEncoding("ucs2-be"); $x > = "\x{1f596}"; warn $enc->encode($x, Encode::FB_PERLQQ())' > UCS-2BE:code point "\x{1f596}" too high at -e line 1. > $ perl5.26.1 -MEncode -e '$enc = Encode->getEncoding("ucs2-be"); $x > = "\x{1f596}"; warn $enc->encode($x, Encode::FB_QUIET())' > UCS-2BE:code point "\x{1f596}" too high at -e line 1. > > For ascii, which supports a much smaller character set than ucs2, it works: > > $ perl5.26.1 -MEncode -e '$enc = Encode->getEncoding("US-ASCII"); $x > = "\x{1f596}"; warn $enc->encode($x, Encode::FB_PERLQQ());' > \x{1f596} at -e line 1. > > So this smells like a problem in Encode.pm... >
This ticket might technically be resolved, though I wonder if it is related to #127786.