Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the PPI CPAN distribution.

Report information
The Basics
Id: 30773
Status: resolved
Priority: 0/
Queue: PPI

People
Owner: Nobody in particular
Requestors: dam [...] modsoftsys.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: 1.201
Fixed in: (no value)



Subject: erroneous skip of t/14_charsets.t: missing "use utf8;"
Hi, During "make test" run, t/14_charsets.t is always skipped here, regardless of the active locale. t/14_charsets....................ok 11/11 skipped: Unicode-incompatible locale in use I think this is because perl doesn't know that the source is encoded in utf-8 (despite the BOM). Adding explicit "use utf8;" somewhere in the beginning makes the test to run (and pass). perl 5.8.8 (from Debian/unstable) Trivial patch attached. If you prefer me fiddling with CVS, my SF username is "dbi". Thanks, dam Debian Perl Group
Subject: t-14_charsets.t-is-utf8.patch
Index: t/14_charsets.t # Explicitly state source is utf8, as a-umlaut is then matched with \w # as a test for utf-8 compatibility =================================================================== --- a/t/14_charsets.t (revision 9390) +++ b/t/14_charsets.t (working copy) @@ -18,6 +18,7 @@ use PPI; use Params::Util '_INSTANCE'; use Test::More tests => 11; +use utf8; sub good_ok { my $source = shift;
Subject: Re: [rt.cpan.org #30773] erroneous skip of t/14_charsets.t: missing "use utf8;"
Date: Sat, 17 Nov 2007 12:23:15 +1100
To: bug-PPI [...] rt.cpan.org
From: "Adam Kennedy" <adamkennedybackup [...] gmail.com>
I don't mind you fiddling directly, since I'm not particularly multilingual person. Other people have done most of the unicode work. I'm no longer using the sourceforge CVS, but I can add you directly to the new SVN repository? If you have a CPAN login I'll add you based on that, otherwise I'll do it based on an arbitrary email address of your choice. Adam K On 16/11/2007, Damyan Ivanov via RT <bug-PPI@rt.cpan.org> wrote: Show quoted text
> > > Fri Nov 16 04:16:01 2007: Request 30773 was acted upon. > Transaction: Ticket created by dam@modsoftsys.com > Queue: PPI > Subject: erroneous skip of t/14_charsets.t: missing "use utf8;" > Broken in: 1.201 > Severity: (no value) > Owner: Nobody > Requestors: dam@modsoftsys.com > Status: new > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=30773 > > > > Hi, > > During "make test" run, t/14_charsets.t is always skipped here, > regardless of the active locale. > > t/14_charsets....................ok > 11/11 skipped: Unicode-incompatible locale in use > > I think this is because perl doesn't know that the source is encoded in > utf-8 (despite the BOM). Adding explicit "use utf8;" somewhere in the > beginning makes the test to run (and pass). > > perl 5.8.8 (from Debian/unstable) > > Trivial patch attached. If you prefer me fiddling with CVS, my SF > username is "dbi". > > > Thanks, > dam > Debian Perl Group > >
I took a stab at BOM support in PPI back in 2005, but it never got accepted to the PPI trunk. I think Adam just doesn't like me. :-) http://parseperl.cvs.sourceforge.net/parseperl/PPI/lib/PPI/Tokenizer.pm?r1=1.31&r2=1.32 http://parseperl.cvs.sourceforge.net/parseperl/PPI/lib/PPI/Token/BOM.pm?hideattic=0&revision=1.1&view=markup I would love to see this revived! Chris
Subject: Re: [rt.cpan.org #30773] erroneous skip of t/14_charsets.t: missing "use utf8;"
Date: Sat, 17 Nov 2007 14:12:28 +1100
To: bug-PPI [...] rt.cpan.org
From: "Adam Kennedy" <adamkennedybackup [...] gmail.com>
Now would probably be a good time to look at doing this again, since PPI is stable, the outstanding functionality is all in, and we're in a support period now. On 17/11/2007, Chris Dolan via RT <bug-PPI@rt.cpan.org> wrote: Show quoted text
> > > Queue: PPI > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=30773 > > > I took a stab at BOM support in PPI back in 2005, but it never got > accepted to the PPI trunk. I think Adam just doesn't like me. :-) > > > http://parseperl.cvs.sourceforge.net/parseperl/PPI/lib/PPI/Tokenizer.pm?r1=1.31&r2=1.32 > > http://parseperl.cvs.sourceforge.net/parseperl/PPI/lib/PPI/Token/BOM.pm?hideattic=0&revision=1.1&view=markup > > I would love to see this revived! > > Chris >
No time here... The patch is almost trivial but the tests are a little bit hard to write -- need to build a binary string with different BOM values. If someone applies the patches, I can test against a real-world Locale::Maketext file that has a UTF-8 BOM. Chris On Fri Nov 16 22:12:45 2007, adamkennedybackup@gmail.com wrote: Show quoted text
> Now would probably be a good time to look at doing this again, since > PPI is > stable, the outstanding functionality is all in, and we're in a > support > period now. > > On 17/11/2007, Chris Dolan via RT <bug-PPI@rt.cpan.org> wrote:
> > > > > > Queue: PPI > > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=30773 > > > > > I took a stab at BOM support in PPI back in 2005, but it never got > > accepted to the PPI trunk. I think Adam just doesn't like me. :-) > > > > > >
>
http://parseperl.cvs.sourceforge.net/parseperl/PPI/lib/PPI/Tokenizer.pm?r1=1.31&r2=1.32 Show quoted text
> > > >
>
http://parseperl.cvs.sourceforge.net/parseperl/PPI/lib/PPI/Token/BOM.pm?hideattic=0&revision=1.1&view=markup Show quoted text
> > > > I would love to see this revived! > > > > Chris > >
From: dam [...] modsoftsys.com
On Fri Nov 16 20:24:04 2007, adamkennedybackup@gmail.com wrote: Show quoted text
> I don't mind you fiddling directly, since I'm not particularly > multilingual person.
Thanks for letting me in. Patch committed (r2412). t/14_charsets.t now passes, instead of skipping. -- dam
Since this fixed, i'm closing the ticket. :)