Bug #127786 for Tk: UCS-2BE:code point "\x{1d771}" too high

Thu Nov 22 16:07:35 2018 philiprbrenan [...] gmail.com - Ticket created

Subject:

I get: UCS-2BE:code point "\x{1d771}" too high at /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/Tk/Widget.pm line 205. from: use strict; use utf8; use Tk; my $mw = MainWindow->new; $mw->Label (-text=>'𝝰 𝝱 𝝲')->pack; $mw->Button(-text=>'𝝰 𝝱 𝝲', -command=>sub {exit})->pack; MainLoop; -- Thanks, Phil <https://www.eff.org/privacybadger> Philip R Brenan <https://www.eff.org/privacybadger>

Thu Nov 22 16:55:07 2018 SREZIC [...] cpan.org - Correspondence added

On 2018-11-22 16:07:35, philiprbrenan@gmail.com wrote: Show quoted text

> I get: > > UCS-2BE:code point "\x{1d771}" too high at > /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/Tk/Widget.pm line 205. > > from: > > use strict; > use utf8; > use Tk; > my $mw = MainWindow->new; > $mw->Label (-text=>'𝝰 𝝱 𝝲')->pack; > $mw->Button(-text=>'𝝰 𝝱 𝝲', -command=>sub {exit})->pack; > MainLoop; >

Which Linux and which Tk version? Your script runs fine on my Debian/jessie system, either with the system perl (Tk 804.032) or with a self-compiled 5.26.1 (Tk 804.034). Regards, Slaven

Thu Nov 22 16:55:08 2018 The RT System itself - Status changed from 'new' to 'open'

Thu Nov 22 17:02:11 2018 SREZIC [...] cpan.org - Correspondence added

On 2018-11-22 16:55:07, SREZIC wrote: Show quoted text

> On 2018-11-22 16:07:35, philiprbrenan@gmail.com wrote:

> > I get: > > > > UCS-2BE:code point "\x{1d771}" too high at > > /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/Tk/Widget.pm line 205. > > > > from: > > > > use strict; > > use utf8; > > use Tk; > > my $mw = MainWindow->new; > > $mw->Label (-text=>'𝝰 𝝱 𝝲')->pack; > > $mw->Button(-text=>'𝝰 𝝱 𝝲', -command=>sub {exit})->pack; > > MainLoop; > >

> > Which Linux and which Tk version? Your script runs fine on my > Debian/jessie system, either with the system perl (Tk 804.032) or with > a self-compiled 5.26.1 (Tk 804.034).

Another idea: grepping through the code shows lines where ucs-2be is mapped to iso10646-1. I remember the latter being a popular encoding for some (old) X11 fonts. So probably it's a font issue --- maybe your Tk was compiled without XFT=1 (e.g. because libxft etc. was not available), or maybe there are no true type fonts available. If XFT support is available can be shown with the following oneliner: $ perl -MTk::Config -E 'say $Tk::Config::xlib' -L/usr/lib/x86_64-linux-gnu -lXft Regards, Slaven

Thu Nov 22 17:22:24 2018 philiprbrenan [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #127786]
Date:	Thu, 22 Nov 2018 22:21:54 +0000
To:	bug-Tk [...] rt.cpan.org
From:	Philip R Brenan <philiprbrenan [...] gmail.com>

Hi *Slaven*: perl -MTk::Config -E 'say $Tk::Config::xlib' -I/usr/lib/x86_64-linux-gnu -lXft -L/usr/lib/x86_64-linux-gnu cat /proc/version Linux version 4.15.0-33-generic (buildd@lcy01-amd64-024) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018 This is on Kubuntu 18.04. On Thu, Nov 22, 2018 at 10:02 PM Slaven_Rezic via RT <bug-Tk@rt.cpan.org> wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=127786 > > > On 2018-11-22 16:55:07, SREZIC wrote:

> > On 2018-11-22 16:07:35, philiprbrenan@gmail.com wrote:

> > > I get: > > > > > > UCS-2BE:code point "\x{1d771}" too high at > > > /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/Tk/Widget.pm line 205. > > > > > > from: > > > > > > use strict; > > > use utf8; > > > use Tk; > > > my $mw = MainWindow->new; > > > $mw->Label (-text=>'𝝰 𝝱 𝝲')->pack; > > > $mw->Button(-text=>'𝝰 𝝱 𝝲', -command=>sub {exit})->pack; > > > MainLoop; > > >

> > > > Which Linux and which Tk version? Your script runs fine on my > > Debian/jessie system, either with the system perl (Tk 804.032) or with > > a self-compiled 5.26.1 (Tk 804.034).

> > Another idea: grepping through the code shows lines where ucs-2be is > mapped to iso10646-1. I remember the latter being a popular encoding for > some (old) X11 fonts. So probably it's a font issue --- maybe your Tk was > compiled without XFT=1 (e.g. because libxft etc. was not available), or > maybe there are no true type fonts available. If XFT support is available > can be shown with the following oneliner: > > $ perl -MTk::Config -E 'say $Tk::Config::xlib' > -L/usr/lib/x86_64-linux-gnu -lXft > > Regards, > Slaven > >

-- Thanks, Phil <https://www.eff.org/privacybadger> Philip R Brenan <https://www.eff.org/privacybadger>

Sat Feb 02 09:19:42 2019 SREZIC [...] cpan.org - Subject changed from (no value) to 'UCS-2BE:code point "\x{1d771}" too high'

Sat Feb 02 09:32:03 2019 SREZIC [...] cpan.org - Correspondence added

On 2018-11-22 17:22:24, philiprbrenan@gmail.com wrote: Show quoted text

> Hi *Slaven*: > > perl -MTk::Config -E 'say $Tk::Config::xlib' -I/usr/lib/x86_64-linux-gnu > -lXft > -L/usr/lib/x86_64-linux-gnu > > cat /proc/version > Linux version 4.15.0-33-generic (buildd@lcy01-amd64-024) (gcc version 7.3.0 > (Ubuntu 7.3.0-16ubuntu3)) #36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018 > > This is on Kubuntu 18.04. >

Then maybe missing fonts? If I execute the script using strace like this: strace -eopen,openat perl test.pl |& grep fonts then at the end two files looking like font files are listed: openat(AT_FDCWD, "/usr/share/texmf/fonts/opentype/public/tex-gyre/texgyreheros-bold.otf", O_RDONLY) = 4 openat(AT_FDCWD, "/usr/share/fonts/truetype/dejavu/DejaVuSans-Bold.ttf", O_RDONLY) = 4 Can you try this on your system? What files are listed there? Regards, Slaven Show quoted text

> > On Thu, Nov 22, 2018 at 10:02 PM Slaven_Rezic via RT <bug-Tk@rt.cpan.org> > wrote: >

> > <URL: https://rt.cpan.org/Ticket/Display.html?id=127786 > > > > > On 2018-11-22 16:55:07, SREZIC wrote:

> > > On 2018-11-22 16:07:35, philiprbrenan@gmail.com wrote:

> > > > I get: > > > > > > > > UCS-2BE:code point "\x{1d771}" too high at > > > > /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/Tk/Widget.pm line 205. > > > > > > > > from: > > > > > > > > use strict; > > > > use utf8; > > > > use Tk; > > > > my $mw = MainWindow->new; > > > > $mw->Label (-text=>'𝝰 𝝱 𝝲')->pack; > > > > $mw->Button(-text=>'𝝰 𝝱 𝝲', -command=>sub {exit})->pack; > > > > MainLoop; > > > >

> > > > > > Which Linux and which Tk version? Your script runs fine on my > > > Debian/jessie system, either with the system perl (Tk 804.032) or with > > > a self-compiled 5.26.1 (Tk 804.034).

> > > > Another idea: grepping through the code shows lines where ucs-2be is > > mapped to iso10646-1. I remember the latter being a popular encoding for > > some (old) X11 fonts. So probably it's a font issue --- maybe your Tk was > > compiled without XFT=1 (e.g. because libxft etc. was not available), or > > maybe there are no true type fonts available. If XFT support is available > > can be shown with the following oneliner: > > > > $ perl -MTk::Config -E 'say $Tk::Config::xlib' > > -L/usr/lib/x86_64-linux-gnu -lXft > > > > Regards, > > Slaven > > > >

>

Sat Feb 02 09:32:05 2019 SREZIC [...] cpan.org - Status changed from 'open' to 'stalled'

Thu Jun 06 12:16:55 2019 CAC [...] cpan.org - Correspondence added

I wonder if this issue is expected behavior due to the historical limitation in Tcl for not being able to handle non-BMP Unicode characters (i.e. codepoints above U+FFFF; support for non-BMP Unicode characters is planned for Tcl/Tk 8.7 and 9.0).

Thu Jun 06 12:16:56 2019 The RT System itself - Status changed from 'stalled' to 'open'

Thu Jun 06 12:17:22 2019 CAC [...] cpan.org - Cc CAC added

Fri Jun 07 02:29:58 2019 SREZIC [...] cpan.org - Correspondence added

On 2019-06-06 12:16:55, CAC wrote: Show quoted text

> I wonder if this issue is expected behavior due to the historical > limitation in > Tcl for not being able to handle non-BMP Unicode characters (i.e. > codepoints > above U+FFFF; support for non-BMP Unicode characters is planned for > Tcl/Tk 8.7 > and 9.0).

This wouldn't explain why the script works on some systems (e.g. mine).

Fri Jun 07 02:50:44 2019 SREZIC [...] cpan.org - Correspondence added

On 2019-06-07 02:29:58, SREZIC wrote: Show quoted text

> On 2019-06-06 12:16:55, CAC wrote:

> > I wonder if this issue is expected behavior due to the historical > > limitation in > > Tcl for not being able to handle non-BMP Unicode characters (i.e. > > codepoints > > above U+FFFF; support for non-BMP Unicode characters is planned for > > Tcl/Tk 8.7 > > and 9.0).

> > This wouldn't explain why the script works on some systems (e.g. mine).

Also, the error message looks like it was generated by perl code, not by tcl code: $ perl -MEncode -e 'encode("UCS-2BE", "\x{1d771}", Encode::FB_WARN)' UCS-2BE:code point "\x{1d771}" too high at /usr/lib/x86_64-linux-gnu/perl/5.20/Encode.pm line 161.

Fri Jun 07 10:54:07 2019 prbrenan [...] cpan.org - Correspondence added

Now working as expected. cat /proc/version Linux version 4.15.0-51-generic (buildd@lgw01-amd64-059) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #55-Ubuntu SMP Wed May 15 14:27:21 UTC 2019 Tk is up to date (804.034) On Sat Feb 02 09:32:03 2019, SREZIC wrote: Show quoted text

> On 2018-11-22 17:22:24, philiprbrenan@gmail.com wrote:

> > Hi *Slaven*: > > > > perl -MTk::Config -E 'say $Tk::Config::xlib' -I/usr/lib/x86_64-linux- > > gnu > > -lXft > > -L/usr/lib/x86_64-linux-gnu > > > > cat /proc/version > > Linux version 4.15.0-33-generic (buildd@lcy01-amd64-024) (gcc version > > 7.3.0 > > (Ubuntu 7.3.0-16ubuntu3)) #36-Ubuntu SMP Wed Aug 15 16:00:05 UTC 2018 > > > > This is on Kubuntu 18.04. > >

> > Then maybe missing fonts? If I execute the script using strace like > this: > > strace -eopen,openat perl test.pl |& grep fonts > > then at the end two files looking like font files are listed: > > openat(AT_FDCWD, "/usr/share/texmf/fonts/opentype/public/tex- > gyre/texgyreheros-bold.otf", O_RDONLY) = 4 > openat(AT_FDCWD, "/usr/share/fonts/truetype/dejavu/DejaVuSans- > Bold.ttf", O_RDONLY) = 4 > > Can you try this on your system? What files are listed there? > > Regards, > Slaven >

> > > > On Thu, Nov 22, 2018 at 10:02 PM Slaven_Rezic via RT <bug- > > Tk@rt.cpan.org> > > wrote: > >

> > > <URL: https://rt.cpan.org/Ticket/Display.html?id=127786 > > > > > > > On 2018-11-22 16:55:07, SREZIC wrote:

> > > > On 2018-11-22 16:07:35, philiprbrenan@gmail.com wrote:

> > > > > I get: > > > > > > > > > > UCS-2BE:code point "\x{1d771}" too high at > > > > > /usr/local/lib/x86_64-linux-gnu/perl/5.26.1/Tk/Widget.pm line > > > > > 205. > > > > > > > > > > from: > > > > > > > > > > use strict; > > > > > use utf8; > > > > > use Tk; > > > > > my $mw = MainWindow->new; > > > > > $mw->Label (-text=>'𝝰 𝝱 𝝲')->pack; > > > > > $mw->Button(-text=>'𝝰 𝝱 𝝲', -command=>sub {exit})->pack; > > > > > MainLoop; > > > > >

> > > > > > > > Which Linux and which Tk version? Your script runs fine on my > > > > Debian/jessie system, either with the system perl (Tk 804.032) or > > > > with > > > > a self-compiled 5.26.1 (Tk 804.034).

> > > > > > Another idea: grepping through the code shows lines where ucs-2be > > > is > > > mapped to iso10646-1. I remember the latter being a popular > > > encoding for > > > some (old) X11 fonts. So probably it's a font issue --- maybe your > > > Tk was > > > compiled without XFT=1 (e.g. because libxft etc. was not > > > available), or > > > maybe there are no true type fonts available. If XFT support is > > > available > > > can be shown with the following oneliner: > > > > > > $ perl -MTk::Config -E 'say $Tk::Config::xlib' > > > -L/usr/lib/x86_64-linux-gnu -lXft > > > > > > Regards, > > > Slaven > > > > > >

> >

Sat Jul 20 00:23:51 2019 CAC [...] cpan.org - Correspondence added

On Fri Jun 07 01:50:44 2019, SREZIC wrote:
Show quoted text

> On 2019-06-07 02:29:58, SREZIC wrote:

> > On 2019-06-06 12:16:55, CAC wrote:

> > > I wonder if this issue is expected behavior due to the historical
> > > limitation in
> > > Tcl for not being able to handle non-BMP Unicode characters (i.e.
> > > codepoints
> > > above U+FFFF; support for non-BMP Unicode characters is planned for
> > > Tcl/Tk 8.7
> > > and 9.0).

> >
> > This wouldn't explain why the script works on some systems (e.g.
> > mine).

>
> Also, the error message looks like it was generated by perl code, not
> by tcl code:
>
> $ perl -MEncode -e 'encode("UCS-2BE", "\x{1d771}", Encode::FB_WARN)'
> UCS-2BE:code point "\x{1d771}" too high at /usr/lib/x86_64-linux-
> gnu/perl/5.20/Encode.pm line 161.

You are right, Encode.pm is where the error message comes from, and the example seems to work anyways. I was thinking of TCL_UTF_MAX being an issue, but I now see where Perl/Tk appears to set it to 13 instead of 3. Sorry for the confusion.

I wonder if this ticket is related to #123543.