Skip Menu |

This queue is for tickets about the Tk-JFileDialog CPAN distribution.

Report information
The Basics
Id: 128958
Status: resolved
Worked: 8 hours (480 min)
Priority: 0/
Queue: Tk-JFileDialog

People
Owner: turnerjw784 [...] yahoo.com
Requestors: welleozean [...] googlemail.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 2.20



Subject: [Win] unicode in path
Date: Fri, 29 Mar 2019 10:40:46 +0000
To: bug-Tk-JFileDialog [...] rt.cpan.org
From: welle Ozean <welleozean [...] googlemail.com>
Hello, all paths/files containing nonLatin characters are simply skipped by JFileDialog.I am on Windows 10, encoding 1252. A path containing for example "ü" doesn't show up. Best, Welle <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virenfrei. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
Confirmed, will fix... Jim On Fri Mar 29 06:40:55 2019, welleozean@googlemail.com wrote: Show quoted text
> Hello, > all paths/files containing nonLatin characters are simply skipped by > JFileDialog.I am on Windows 10, encoding 1252. A path containing for > example "ü" doesn't show up. > Best, > Welle > > <http://www.avg.com/email- > signature?utm_medium=email&utm_source=link&utm_campaign=sig- > email&utm_content=webmail> > Virenfrei. > www.avg.com > <http://www.avg.com/email- > signature?utm_medium=email&utm_source=link&utm_campaign=sig- > email&utm_content=webmail> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
Changed from bug to feature request. I think I have this fixed, but I'm not feelin' the warm fuzzies a/b it. It seems that Perl converts strings to an internal "utf-8" format but can't seem to talk to the file-system, which is storing things in strings of simple 8-bit characters. The program fails whenever attempting to take the file-name with the high ascii characters, do anything with it, and then pass it to a file-test function, ie. "-f" or open() or even `stat $fn`, etc, IF Perl has set it's internal "UTF-8 flag" for the string. It fails on both Windows and Linux for me! I did a lot of Google research on this this weekend, and found tons of docs on handling unicode DATA from files, but precious little on dealing with file NAMES! I'm a newbie at this unicode-stuff, since I've never run into this b/c I only use standard low-ASCII characters in filenames but was led to believe that Perl now handled this pretty well under the hood, now however, I'm not so sure! Anyway, since I'm not sure if this is the best, or even the proper way to fix this (but the only way I've found so far), I'm adding a new option flag to JFileDialog: "-nonLatinFilenames". Set this to a TRUE value, ie. 1 in your program to have it handle non-lower Ascii and unicode filenames (ie. characters 128-255). Or, to be more Kosher: $jfm->configure(-nonLatinFilenames => 1) if (Tk::JFileDialog::VERSION >= 2.11); Please backup your current JFileDialog.pm file, then copy this new one (attached) over it and see if this addresses your issues, then let me know, including if you have any additional ideas / patches on how to better address this! NOTE: You will also likely need to modify your Perl application that accepts these file names from JFileDialog, or from elsewhere as well wherever it does a system-call, such as open() or a file-test, ie. "-f $fid" in a similar fashon, particularly if it does any string-minipulation with it beforehand (such that Perl's internal UTF-8 flag gets set for it). If this addresses the issue to your satisfaction, I'll do a new full-release. Regards, Jim On Sat Mar 30 12:15:35 2019, TURNERJW wrote: Show quoted text
> Confirmed, will fix... > > Jim > > > On Fri Mar 29 06:40:55 2019, welleozean@googlemail.com wrote:
> > Hello, > > all paths/files containing nonLatin characters are simply skipped by > > JFileDialog.I am on Windows 10, encoding 1252. A path containing for > > example "ü" doesn't show up. > > Best, > > Welle > > > > <http://www.avg.com/email- > > signature?utm_medium=email&utm_source=link&utm_campaign=sig- > > email&utm_content=webmail> > > Virenfrei. > > www.avg.com > > <http://www.avg.com/email- > > signature?utm_medium=email&utm_source=link&utm_campaign=sig- > > email&utm_content=webmail> > > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> >
Subject: JFileDialog.pm

Message body is not shown because it is too large.

Subject: Re: [rt.cpan.org #128958] [Win] unicode in path
Date: Mon, 1 Apr 2019 15:46:35 +0100
To: bug-Tk-JFileDialog [...] rt.cpan.org
From: welle Ozean <welleozean [...] googlemail.com>
Hi Jim, I am well aware of the 'madness' of Perl+file-system, especially on Windows. Having a Tk application that needs to run on potentially any Window system in terms of locales, languages, etc, I had to fight a lot to find some workable solution. However, I never found the definitive solution. And the Web doesn't help. Many general suggestions here and there, but no fool-proof concept (different to handling unicode content which is fantastic in Perl). I tried your patch, and at least on my machine it seems to be a huge step forward. I now can read file names with a larger set of characters (even Chinese ?!? Is this possible?). Generally speaking, I do not know what is going on under the hood (you seem to make changes in the encodings of the Perl internal representation of characters, which is out of my reach), but it seems to work quite well now. As I will be confronted with bugs on several machines with all possible languages and locales, I can report on it (as an extensive real-world test). PS: when on Windows, I normally use Win32::LongPath to improve (enormously) the handling of file names. Thank you for your great job Welle <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Virenfrei. www.avg.com <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> Am Mo., 1. Apr. 2019 um 06:01 Uhr schrieb Jim Turner via RT < bug-Tk-JFileDialog@rt.cpan.org>: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=128958 > > > Changed from bug to feature request. > > I think I have this fixed, but I'm not feelin' the warm fuzzies a/b it. > It seems that Perl converts strings to an internal "utf-8" format but can't > seem to talk to the file-system, which is storing things in strings of > simple 8-bit characters. The program fails whenever attempting to take the > file-name with the high ascii characters, do anything with it, and then > pass it to a file-test function, ie. "-f" or open() or even `stat $fn`, > etc, IF Perl has set it's internal "UTF-8 flag" for the string. It fails > on both Windows and Linux for me! I did a lot of Google research on this > this weekend, and found tons of docs on handling unicode DATA from files, > but precious little on dealing with file NAMES! I'm a newbie at this > unicode-stuff, since I've never run into this b/c I only use standard > low-ASCII characters in filenames but was led to believe that Perl now > handled this pretty well under the hood, now however, I'm not so sure! > > Anyway, since I'm not sure if this is the best, or even the proper way to > fix this (but the only way I've found so far), I'm adding a new option flag > to JFileDialog: "-nonLatinFilenames". Set this to a TRUE value, ie. 1 in > your program to have it handle non-lower Ascii and unicode filenames (ie. > characters 128-255). Or, to be more Kosher: > $jfm->configure(-nonLatinFilenames => 1) if (Tk::JFileDialog::VERSION >= > 2.11); > > Please backup your current JFileDialog.pm file, then copy this new one > (attached) over it and see if this addresses your issues, then let me know, > including if you have any additional ideas / patches on how to better > address this! > > NOTE: You will also likely need to modify your Perl application that > accepts these file names from JFileDialog, or from elsewhere as well > wherever it does a system-call, such as open() or a file-test, ie. "-f > $fid" in a similar fashon, particularly if it does any string-minipulation > with it beforehand (such that Perl's internal UTF-8 flag gets set for it). > If this addresses the issue to your satisfaction, I'll do a new > full-release. > > Regards, > > Jim > > > On Sat Mar 30 12:15:35 2019, TURNERJW wrote:
> > Confirmed, will fix... > > > > Jim > > > > > > On Fri Mar 29 06:40:55 2019, welleozean@googlemail.com wrote:
> > > Hello, > > > all paths/files containing nonLatin characters are simply skipped by > > > JFileDialog.I am on Windows 10, encoding 1252. A path containing for > > > example "ü" doesn't show up. > > > Best, > > > Welle > > > > > > <http://www.avg.com/email- > > > signature?utm_medium=email&utm_source=link&utm_campaign=sig- > > > email&utm_content=webmail> > > > Virenfrei. > > > www.avg.com > > > <http://www.avg.com/email- > > > signature?utm_medium=email&utm_source=link&utm_campaign=sig- > > > email&utm_content=webmail> > > > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> > > >
> > > >
Fixed in v2.20 and closing. Thanks, Jim On Mon Apr 01 09:46:43 2019, welleozean@googlemail.com wrote: Show quoted text
> Hi Jim, > > I am well aware of the 'madness' of Perl+file-system, especially on > Windows. Having a Tk application that needs to run on potentially any > Window system in terms of locales, languages, etc, I had to fight a > lot to > find some workable solution. However, I never found the definitive > solution. And the Web doesn't help. Many general suggestions here and > there, but no fool-proof concept (different to handling unicode > content > which is fantastic in Perl). > > I tried your patch, and at least on my machine it seems to be a huge > step > forward. I now can read file names with a larger set of characters > (even > Chinese ?!? Is this possible?). Generally speaking, I do not know what > is > going on under the hood (you seem to make changes in the encodings of > the > Perl internal representation of characters, which is out of my reach), > but > it seems to work quite well now. As I will be confronted with bugs on > several machines with all possible languages and locales, I can report > on > it (as an extensive real-world test). > > PS: when on Windows, I normally use Win32::LongPath to improve > (enormously) > the handling of file names. > > Thank you for your great job > Welle > > <http://www.avg.com/email- > signature?utm_medium=email&utm_source=link&utm_campaign=sig- > email&utm_content=webmail> > Virenfrei. > www.avg.com > <http://www.avg.com/email- > signature?utm_medium=email&utm_source=link&utm_campaign=sig- > email&utm_content=webmail> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> > > Am Mo., 1. Apr. 2019 um 06:01 Uhr schrieb Jim Turner via RT < > bug-Tk-JFileDialog@rt.cpan.org>: >
> > <URL: https://rt.cpan.org/Ticket/Display.html?id=128958 > > > > > Changed from bug to feature request. > > > > I think I have this fixed, but I'm not feelin' the warm fuzzies a/b > > it. > > It seems that Perl converts strings to an internal "utf-8" format but > > can't > > seem to talk to the file-system, which is storing things in strings > > of > > simple 8-bit characters. The program fails whenever attempting to > > take the > > file-name with the high ascii characters, do anything with it, and > > then > > pass it to a file-test function, ie. "-f" or open() or even `stat > > $fn`, > > etc, IF Perl has set it's internal "UTF-8 flag" for the string. It > > fails > > on both Windows and Linux for me! I did a lot of Google research on > > this > > this weekend, and found tons of docs on handling unicode DATA from > > files, > > but precious little on dealing with file NAMES! I'm a newbie at this > > unicode-stuff, since I've never run into this b/c I only use standard > > low-ASCII characters in filenames but was led to believe that Perl > > now > > handled this pretty well under the hood, now however, I'm not so > > sure! > > > > Anyway, since I'm not sure if this is the best, or even the proper > > way to > > fix this (but the only way I've found so far), I'm adding a new > > option flag > > to JFileDialog: "-nonLatinFilenames". Set this to a TRUE value, ie. > > 1 in > > your program to have it handle non-lower Ascii and unicode filenames > > (ie. > > characters 128-255). Or, to be more Kosher: > > $jfm->configure(-nonLatinFilenames => 1) if > > (Tk::JFileDialog::VERSION >= > > 2.11); > > > > Please backup your current JFileDialog.pm file, then copy this new > > one > > (attached) over it and see if this addresses your issues, then let me > > know, > > including if you have any additional ideas / patches on how to better > > address this! > > > > NOTE: You will also likely need to modify your Perl application that > > accepts these file names from JFileDialog, or from elsewhere as well > > wherever it does a system-call, such as open() or a file-test, ie. "- > > f > > $fid" in a similar fashon, particularly if it does any string- > > minipulation > > with it beforehand (such that Perl's internal UTF-8 flag gets set for > > it). > > If this addresses the issue to your satisfaction, I'll do a new > > full-release. > > > > Regards, > > > > Jim > > > > > > On Sat Mar 30 12:15:35 2019, TURNERJW wrote:
> > > Confirmed, will fix... > > > > > > Jim > > > > > > > > > On Fri Mar 29 06:40:55 2019, welleozean@googlemail.com wrote:
> > > > Hello, > > > > all paths/files containing nonLatin characters are simply skipped > > > > by > > > > JFileDialog.I am on Windows 10, encoding 1252. A path containing > > > > for > > > > example "ü" doesn't show up. > > > > Best, > > > > Welle > > > > > > > > <http://www.avg.com/email- > > > > signature?utm_medium=email&utm_source=link&utm_campaign=sig- > > > > email&utm_content=webmail> > > > > Virenfrei. > > > > www.avg.com > > > > <http://www.avg.com/email- > > > > signature?utm_medium=email&utm_source=link&utm_campaign=sig- > > > > email&utm_content=webmail> > > > > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> > > > > >
> > > > > > > >