Bug #68319 for Inline-Python: dict test fails for 0.39 on some Debian systems

Thu May 19 11:44:09 2011 mateusz.pavlic [...] gmail.com - Ticket created

Subject:	dict test fails for 0.39 on some Debian systems
Date:	Thu, 19 May 2011 17:43:59 +0200
To:	bug-Inline-Python [...] rt.cpan.org
From:	Mateusz Pavlic <mateusz.pavlic [...] gmail.com>

Hello, I always have this error on the one of my debian systems: PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t t/00init.t ............. ok t/01testpl.t ........... ok t/02testpl.t ........... ok t/03parse.t ............ ok t/04func.t ............. ok t/05JAxH.t ............. ok t/06dict.t ............. 1/6 exceptions.KeyError: u'f\xc3\xb6\xc3\xb6' at line 14 # Looks like you planned 6 tests but ran 4. # Looks like your test exited with 255 just after 4. t/06dict.t ............. Dubious, test returned 255 (wstat 65280, 0xff00) (...) I tried with Inline-0.46 and Inline-0.48. 1708 $>cat /etc/debian_version 6.0.1 uname -a Linux xxxxxxxxxx 2.6.35.5-c3 #1 SMP Fri Sep 24 18:43:54 CEST 2010 x86_64 GNU/Linux perl -v This is perl, v5.10.1 (*) built for x86_64-linux-gnu-thread-multi (with 51 registered patches, see perl -V for more detail) python Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48) [GCC 4.4.5] on linux2 Type "help", "copyright", "credits" or "license" for more information. On the other, very similar debian system tests work just fine every time. (both systems just upgraded with apt-get, while bad machine is real hardware, good one is xen virtual). cat /etc/debian_version 6.0.1 uname -a Linux yyyyyyyyyyyy 2.6.32-5-amd64 #1 SMP Wed Jan 12 03:40:32 UTC 2011 x86_64 GNU/Linux perl -v This is perl, v5.10.1 (*) built for x86_64-linux-gnu-thread-multi (with 51 registered patches, see perl -V for more detail) python Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48) [GCC 4.4.5] on linux2 I found one interesting thing on "x" machine (where failures occur). It is somehow related to blib_test/lib/*files. Please look: PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/00init.t t/06dict.t t/00init.t .. ok t/06dict.t .. 1/6 exceptions.KeyError: u'f\xc3\xb6\xc3\xb6' at line 14 # Looks like you planned 6 tests but ran 4. # Looks like your test exited with 255 just after 4. t/06dict.t .. Dubious, test returned 255 (wstat 65280, 0xff00) Failed 2/6 subtests But if I skip 00init which removes files from blib_test and files from previous run can be found, test is OK: PERL_DL_NONLAZY=1 /usr/bin/perl "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/06dict.t t/06dict.t .. ok All tests successful. Yes, it only occurs for 06dict test. Other tests are OK. also when I copy compiled files from "good" machine to "bad", and run make test, test fails. If I copy compiled files from "bad" machine to "good", all tests are OK! I'm thinkig of some filesystem/cache tuning or something but I cannot find anything reasonable. And still I can compile and test 0.38. Also compilation of 0.39 goes OK but tests fail. Please help if you have any idea ;-) Regards, Mateusz Pavlic

Thu May 19 12:04:46 2011 nine [...] detonation.org - Correspondence added

Subject:	Re: [rt.cpan.org #68319] dict test fails for 0.39 on some Debian systems
Date:	Thu, 19 May 2011 18:04:31 +0200
To:	bug-Inline-Python [...] rt.cpan.org
From:	Stefan Seifert <nine [...] detonation.org>

On Thursday 19 May 2011 17:44:09 you wrote: Show quoted text

> Hello, > I always have this error on the one of my debian systems: > > t/06dict.t ............. 1/6 exceptions.KeyError: u'f\xc3\xb6\xc3\xb6'

I have seen this error in many CPAN testers reports but have never been able to find a machine where I could reproduce this. That is until now. Seems like it finally happens for me as well, so I can start debugging. Show quoted text

> I found one interesting thing on "x" machine (where failures occur). > It is somehow related to blib_test/lib/*files.

I see this as well. That's a great hint, I'll see if I can find out more. Stefan

Thu May 19 12:04:47 2011 The RT System itself - Status changed from 'new' to 'open'

Thu May 19 12:19:12 2011 nine [...] detonation.org - Correspondence added

I think I got it :) https://github.com/niner/inline-python- pm/commit/d5cc8b72455c5d64be9520b02a9008083a0c9d53 Your hint gave it to me. Seems like the Python code is cached in blib_test. When read from there, it's properly recognized as UTF-8. If not, Python thinks it's latin-1 and the test fails. Fixed by adding an explicit encoding for the Python code. Could you test the fix? I'm quite tempted to release a version just with this test fix just to see many test reports go green :)

Thu May 19 12:24:47 2011 mateusz.pavlic [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #68319] dict test fails for 0.39 on some Debian systems
Date:	Thu, 19 May 2011 18:24:38 +0200
To:	bug-Inline-Python [...] rt.cpan.org
From:	Mateusz Pavlic <mateusz.pavlic [...] gmail.com>

I'll test it in 10 minutes. MP On 19 May 2011 18:19, Stefan Seifert via RT <bug-Inline-Python@rt.cpan.org> wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=68319 > > > I think I got it :) > > https://github.com/niner/inline-python- > pm/commit/d5cc8b72455c5d64be9520b02a9008083a0c9d53 > > Your hint gave it to me. Seems like the Python code is cached in blib_test. When read > from there, it's properly recognized as UTF-8. If not, Python thinks it's latin-1 and the test > fails. Fixed by adding an explicit encoding for the Python code. > > Could you test the fix? I'm quite tempted to release a version just with this test fix just to > see many test reports go green :) >

Thu May 19 12:29:14 2011 mateusz.pavlic [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #68319] dict test fails for 0.39 on some Debian systems
Date:	Thu, 19 May 2011 18:29:06 +0200
To:	bug-Inline-Python [...] rt.cpan.org
From:	Mateusz Pavlic <mateusz.pavlic [...] gmail.com>

Everyting fine with niner-inline-python-pm-d5cc8b7.tar.gz :-) I haven't tested it working in "production" yet - I'll do it tomorrow or next week, but tests after compilation are OK now. Thanks!!! MP On 19 May 2011 18:19, Stefan Seifert via RT <bug-Inline-Python@rt.cpan.org> wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=68319 > > > I think I got it :) > > https://github.com/niner/inline-python- > pm/commit/d5cc8b72455c5d64be9520b02a9008083a0c9d53 > > Your hint gave it to me. Seems like the Python code is cached in blib_test. When read > from there, it's properly recognized as UTF-8. If not, Python thinks it's latin-1 and the test > fails. Fixed by adding an explicit encoding for the Python code. > > Could you test the fix? I'm quite tempted to release a version just with this test fix just to > see many test reports go green :) >

Fri Jul 29 07:46:51 2011 perl [...] intermine.org - Correspondence added

RT-Send-CC:

nine [...] detonation.org

I can confirm this issue and the fact that the coding declaration fix works. Is there any ETA on getting a 0.40 release? Alex

Fri Jul 29 10:49:03 2011 nine [...] detonation.org - Correspondence added

Finally after a year of wondering why this fails on some machines but not on others, I found the final piece of the puzzle. The fix I commited was correct indeed, but it broke the test on my own system. But today I found out why: I have an old Digest::MD5 version installed. Inline generates an MD5 hash of the executed code. Older Digest::MD5 versions have a bug that downgrades UTF-8 enabled arguments to latin-1. This is why for me the Python interpreter never had problems with UTF-8 encoded code: it always got latin-1. And that's the reason why it complained when I told it that the code is in UTF-8. Inline::Python 0.40 requires Digest::MD5 >= 2.50, even though it doesn't use it directly. I opened a bug report for Inline itself so that hopefully gets the requirement as well. 0.40 is on it's way to the CPAN servers :)

Fri Jul 29 10:49:04 2011 nine [...] detonation.org - Status changed from 'open' to 'resolved'