Bug #83825 for YAML-Syck: Wide characters are quoted erroneously on PowerPC

Fri Mar 08 04:10:38 2013 ppisar [...] redhat.com - Ticket created

Subject:

Wide characters are quoted erroneously on PowerPC

Since 1.22, tests fail on PPC and PPC64 platforms in t/yaml_utf.t test because the wide charters are quotes unnecessarily there: $ uname -m x86_64 $ perl -MYAML::Syck -e 'use utf8; local $YAML::Syck::ImplicitUnicode = 1; print Dump(q{♥}), qq{\n}' Wide character in print at -e line 1. --- ♥ $ uname -m ppc64 $ perl -Iblib/lib -Iblib/arch -MYAML::Syck -e 'use utf8; local $YAML::Syck::ImplicitUnicode = 1; print Dump(q{♥}), qq{\n}' Wide character in print at -e line 1. --- "♥" Problem is causes by these two commits: commit f63d57472f62e60b2cdf981f08be01850612311d Author: Lubomir Rintel <lubo.rintel@gooddata.com> Date: Fri Nov 9 18:02:22 2012 +0100 Fix non-printable character detection commit 07a319d848fc17ddb6463d732ceed55c18b1b415 Author: Lubomir Rintel <lubo.rintel@gooddata.com> Date: Fri Nov 9 18:09:05 2012 +0100 Quote if non-printable characters are present

Fri Mar 08 06:12:49 2013 ppisar [...] redhat.com - Correspondence added

Subject:	Re: [rt.cpan.org #83825] AutoReply: Wide characters are quoted erroneously on PowerPC
Date:	Fri, 8 Mar 2013 12:12:30 +0100
To:	Bugs in YAML-Syck via RT <bug-YAML-Syck [...] rt.cpan.org>
From:	Petr Pisar <ppisar [...] redhat.com>

Show quoted text

> Since 1.22, tests fail on PPC and PPC64 platforms in t/yaml_utf.t test > because the wide charters are quotes unnecessarily there: >

And it's getting more coplicated. It quotes on PPC because if ( ! ( (unsigned)cursor[i] == 0x9 || (unsigned)cursor[i] == 0xA || (unsigned)cursor[i] == 0xD || ( (unsigned)cursor[i] >= 0x20 && (unsigned)cursor[i] <= 0x7E ) || (unsigned)cursor[i] == 0x85 || (unsigned)cursor[i] >= 0xa0 ) ) { flags |= SCAN_NONPRINT; } matches UTF-8 encoded bit-stream (♥ is e2 99 a5), but on x86_64, the typecast results in: (gdb) p (unsigned)cursor[0] $5 = 4294967266 (gdb) p (unsigned)cursor[1] $6 = 4294967193 (gdb) p (unsigned)cursor[2] $7 = 4294967205 because (unsigned) means (unsigned int) which is 64-bit wide on x86_64, thus it never matches. (Well the test checking "\0" passes only because of series of zeros). -- Petr

Download (untitled)
application/pgp-signature 230b

Message body not shown because it is not plain text.

Fri Mar 08 07:13:25 2013 ppisar [...] redhat.com - Correspondence added

Subject:	Re: [rt.cpan.org #83825] AutoReply: Wide characters are quoted erroneously on PowerPC
Date:	Fri, 8 Mar 2013 13:12:59 +0100
To:	Bugs in YAML-Syck via RT <bug-YAML-Syck [...] rt.cpan.org>
From:	Petr Pisar <ppisar [...] redhat.com>

On Fri, Mar 08, 2013 at 04:10:39AM -0500, Bugs in YAML-Syck via RT wrote: Show quoted text

> Since 1.22, tests fail on PPC and PPC64 platforms in t/yaml_utf.t test > because the wide charters are quotes unnecessarily there: >

Attached patch remedies the situation so that all tests pass. However proper fix needs to be able to recognize which encoding is used and decode it to Unicode code points to obey <http://yaml.org/spec/1.0/index.html#id2559514> properly. -- Petr

Message body is not shown because sender requested not to inline it.

Download (untitled)
application/pgp-signature 230b

Message body not shown because it is not plain text.

Fri May 31 18:30:43 2013 NHORNE [...] cpan.org - Correspondence added

I believe YAML::Syck also fails on ARM processors. See this test report: http://www.cpantesters.org/cpan/report/258c24f6-c66f-11e2-9147-1b06c9e8bbde

Fri May 31 18:30:44 2013 The RT System itself - Status changed from 'new' to 'open'

Sun Jul 27 07:31:13 2014 fraserbn [...] gmail.com - Correspondence added

From:

fraserbn [...] gmail.com

On Fri May 31 18:30:43 2013, NHORNE wrote: Show quoted text

> I believe YAML::Syck also fails on ARM processors. See this test > report: http://www.cpantesters.org/cpan/report/258c24f6-c66f-11e2- > 9147-1b06c9e8bbde

Can confirm -- this fails on Android ARM.

Sat Aug 02 07:51:10 2014 fraserbn [...] gmail.com - Correspondence added

From:

fraserbn [...] gmail.com

On Sun Jul 27 07:31:13 2014, Hugmeir wrote: Show quoted text

> On Fri May 31 18:30:43 2013, NHORNE wrote:

> > I believe YAML::Syck also fails on ARM processors. See this test > > report: http://www.cpantesters.org/cpan/report/258c24f6-c66f-11e2- > > 9147-1b06c9e8bbde

> > Can confirm -- this fails on Android ARM. >

...and for what it's worth, the patch attached to this ticket makes the module install cleanly on Android and OS X.

Sat Sep 06 11:47:21 2014 NHORNE [...] cpan.org - Broken in 1.28_01 added

Sat Sep 06 11:47:22 2014 NHORNE [...] cpan.org - Broken in 1.22 deleted

Sat Sep 06 11:47:22 2014 NHORNE [...] cpan.org - Severity Important added

Sat Sep 06 13:28:46 2014 NHORNE [...] cpan.org - Requestor NHORNE added

Sun Dec 14 03:33:46 2014 TODDR [...] cpan.org - Correspondence added

Thanks for the patch. I've taken the patch via https://github.com/toddr/YAML-Syck/commit/b9578dc70fc8ca2bfee990bcfd6e5b275ec0cb7d I've released to CPAN as dev release 1.29_01 so I can assure CPAN testers pass it before giving a production release. Ping me if you don't see 1.29 in a week.

Sun Dec 14 03:33:48 2014 TODDR [...] cpan.org - Status changed from 'open' to 'patched'

Mon Dec 15 10:59:42 2014 TODDR [...] cpan.org - Correspondence added

On Sun Dec 14 03:33:46 2014, TODDR wrote: Show quoted text

> Thanks for the patch. > > I've taken the patch via https://github.com/toddr/YAML- > Syck/commit/b9578dc70fc8ca2bfee990bcfd6e5b275ec0cb7d > > I've released to CPAN as dev release 1.29_01 so I can assure CPAN > testers pass it before giving a production release. Ping me if you > don't see 1.29 in a week.

Released to CPAN as 1.29

Mon Dec 15 10:59:45 2014 TODDR [...] cpan.org - Status changed from 'patched' to 'resolved'