Skip Menu |

This queue is for tickets about the autodie CPAN distribution.

Report information
The Basics
Id: 80412
Status: resolved
Priority: 0/
Queue: autodie

People
Owner: Nobody in particular
Requestors: yves [...] cpan.org
Cc: perl5-porters [...] perl.org
AdminCc:

Bug Information
Severity: Normal
Broken in: 2.12
Fixed in: 2.13



CC: perl5-porters [...] perl.org
Subject: autodie hash order dependency
I have been working on making it possible to add new hash algorithms to perl core and on making the hash seed random per process for security reasons. In my work on this I have encountered a hash order dependency bug in the t/hints_pod_examples.t. I could not actually explain the bug but making hash order processing deterministic solved the problem. The attached patch contains some fixes and a version bump. It would be really nice if this or an equivalent fix could get applied sometime soon as I would like to merge my hash changes to core soon and this is a roadblock. Thanks, Yves
Subject: autodie.patch
commit 420f9bf288956d21861eed96530892987372758f Author: Yves Orton <demerphq@gmail.com> Date: Mon Aug 27 08:54:50 2012 +0200 fix a hash key order dependency in cpan/autodie/t/hints_pod_examples.t At the same time make part of the internals deterministic Just In Case. Version bump on autodie to 2.13 as well. diff --git a/cpan/autodie/lib/Fatal.pm b/cpan/autodie/lib/Fatal.pm index 87d9da4..ce17af9 100644 --- a/cpan/autodie/lib/Fatal.pm +++ b/cpan/autodie/lib/Fatal.pm @@ -40,7 +40,7 @@ use constant ERROR_58_HINTS => q{Non-subroutine %s hints for %s are not supporte use constant MIN_IPC_SYS_SIMPLE_VER => 0.12; # All the Fatal/autodie modules share the same version number. -our $VERSION = '2.12'; +our $VERSION = '2.13'; our $Debug ||= 0; @@ -118,6 +118,7 @@ my %TAGS = ( ':2.10' => [qw(:default)], ':2.11' => [qw(:default)], ':2.12' => [qw(:default)], + ':2.13' => [qw(:default)], ); # chmod was only introduced in 2.07 @@ -409,7 +410,9 @@ sub _install_subs { my $pkg_sym = "${pkg}::"; - while(my ($sub_name, $sub_ref) = each %$subs_to_reinstate) { + # It does not hurt to do this in a predictable order, and might help debugging. + foreach my $sub_name (sort keys %$subs_to_reinstate) { + my $sub_ref= $subs_to_reinstate->{$sub_name}; my $full_path = $pkg_sym.$sub_name; diff --git a/cpan/autodie/lib/autodie.pm b/cpan/autodie/lib/autodie.pm index a2360e3..71a6a5e 100644 --- a/cpan/autodie/lib/autodie.pm +++ b/cpan/autodie/lib/autodie.pm @@ -8,7 +8,7 @@ our @ISA = qw(Fatal); our $VERSION; BEGIN { - $VERSION = '2.12'; + $VERSION = '2.13'; } use constant ERROR_WRONG_FATAL => q{ diff --git a/cpan/autodie/lib/autodie/exception.pm b/cpan/autodie/lib/autodie/exception.pm index cd06639..45c723d 100644 --- a/cpan/autodie/lib/autodie/exception.pm +++ b/cpan/autodie/lib/autodie/exception.pm @@ -14,7 +14,7 @@ use overload use if ($] >= 5.010), overload => '~~' => "matches"; -our $VERSION = '2.12'; +our $VERSION = '2.13'; my $PACKAGE = __PACKAGE__; # Useful to have a scalar for hash keys. diff --git a/cpan/autodie/lib/autodie/exception/system.pm b/cpan/autodie/lib/autodie/exception/system.pm index d3047a8..0489b61 100644 --- a/cpan/autodie/lib/autodie/exception/system.pm +++ b/cpan/autodie/lib/autodie/exception/system.pm @@ -5,7 +5,7 @@ use warnings; use base 'autodie::exception'; use Carp qw(croak); -our $VERSION = '2.12'; +our $VERSION = '2.13'; my $PACKAGE = __PACKAGE__; diff --git a/cpan/autodie/lib/autodie/hints.pm b/cpan/autodie/lib/autodie/hints.pm index 71c8be3..36715e9 100644 --- a/cpan/autodie/lib/autodie/hints.pm +++ b/cpan/autodie/lib/autodie/hints.pm @@ -5,7 +5,7 @@ use warnings; use constant PERL58 => ( $] < 5.009 ); -our $VERSION = '2.12'; +our $VERSION = '2.13'; =head1 NAME diff --git a/cpan/autodie/t/hints_pod_examples.t b/cpan/autodie/t/hints_pod_examples.t index a3c6f0f..21a85fd 100644 --- a/cpan/autodie/t/hints_pod_examples.t +++ b/cpan/autodie/t/hints_pod_examples.t @@ -152,22 +152,43 @@ my $perl58_fix = ( ); # Some of the tests provide different hints for scalar or list context - -while (my ($test, $exception_expected) = each %scalar_tests) { - eval " +# NOTE: these tests are sensitive to order (not sure why) therefore +# this loop must use a sorted list of keys . Otherwise there is an occasional +# failure like this: +# +# Failed test 'scalar test - zero_scalar("")' +# at cpan/autodie/t/hints_pod_examples.t line 168. +# got: 'Can't zero_scalar(''): at cpan/autodie/t/hints_pod_examples.t line 157 +# ' +# expected: '' +# +# +# my $scalar = zero_scalar(""); +# 1; + + +foreach my $test (sort keys %scalar_tests) { + my $exception_expected= $scalar_tests{$test}; + my $ok= eval(my $code= " $perl58_fix my \$scalar = $test; - "; + 1; + "); if ($exception_expected) { - isnt("$@", "", "scalar test - $test"); + isnt($ok ? "" : "$@", "", "scalar test - $test") + or diag($code); } else { - is($@, "", "scalar test - $test"); + is($ok ? "" : "$@", "", "scalar test - $test") + or diag($code); } } -while (my ($test, $exception_expected) = each %list_tests) { + +# this set of test is not *known* to be order dependent however we sort it anyway out caution +foreach my $test (sort keys %list_tests) { + my $exception_expected= $list_tests{$test}; eval " $perl58_fix my \@array = $test;
Subject: Re: [rt.cpan.org #80412] autodie hash order dependency
Date: Sat, 27 Oct 2012 12:04:27 +1100
To: bug-autodie [...] rt.cpan.org
From: Paul Fenwick <pjf [...] perltraining.com.au>
On 10/26/2012 07:17 PM, Yves via RT wrote: Show quoted text
> I have been working on making it possible to add new hash algorithms to > perl core and on making the hash seed random per process for security > reasons.
You rock. Show quoted text
> In my work on this I have encountered a hash order dependency bug in the > t/hints_pod_examples.t. I could not actually explain the bug but making > hash order processing deterministic solved the problem. The attached > patch contains some fixes and a version bump.
Weird. I've noticed the patch changes _install_subs to be deterministic in order, too. Just to check, is that just a precaution, or do we actually see things break with the new hash implementation? The patch looks awesome, but I'm worried if there's anything in autodie which is order dependent. I know it's asking even more from you, but do you have a copy of a failing output from the unpatched hints_pod_examples.t ? Even without your patches to Perl's hash code, I should be able to manually change the ordering of the tests used and see if I can reproduce it on an unpatched Perl, and hopefully figure out what's causing the root of the failures. Your patch rocks, by the way. If I can't find the underlying cause in a couple of days, I'll apply your patch and push to the CPAN so this isn't blocking you any more. Do feel free to nudge me on this if I haven't done so. Paul -- Paul Fenwick <pjf@perltraining.com.au> | http://perltraining.com.au/ Director of Training | Ph: +61 3 9354 6001 Perl Training Australia | Fax: +61 3 9354 2681
CC: perl5-porters [...] perl.org
Subject: Re: [rt.cpan.org #80412] autodie hash order dependency
Date: Sat, 27 Oct 2012 10:01:54 +0200
To: bug-autodie [...] rt.cpan.org
From: demerphq <demerphq [...] gmail.com>
On 27 October 2012 03:04, Paul Fenwick via RT <bug-autodie@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=80412 > > > On 10/26/2012 07:17 PM, Yves via RT wrote:
[....] Show quoted text
> I've noticed the patch changes _install_subs to be deterministic in order, > too. Just to check, is that just a precaution, or do we actually see things > break with the new hash implementation?
From my point of view it is just a precaution. When I was debugging things this was the first possible hash ordering issue i saw. I just re-checked, and making that change alone does nothing. So you could hypothetically stick with the current code. Although I personally would want a deterministic ordering. Show quoted text
> The patch looks awesome, but I'm worried if there's anything in autodie > which is order dependent. I know it's asking even more from you, but do you > have a copy of a failing output from the unpatched hints_pod_examples.t ?
Attached is tar.gz of the test output if I revert my patch and run it a few times. There are multiple fails and at least one pass in the tar file. With hash randomization the test output is different every run. Show quoted text
> Even without your patches to Perl's hash code, I should be able to manually > change the ordering of the tests used and see if I can reproduce it on an > unpatched Perl, and hopefully figure out what's causing the root of the > failures.
I have to admit I found those tests non obvious so I didn't dig very deep in to how changing the test execution order would cause this. Show quoted text
> If I can't find the underlying cause in a couple of days, I'll apply your > patch and push to the CPAN so this isn't blocking you any more. Do feel > free to nudge me on this if I haven't done so.
Thanks, Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
Download autodie_rpt.tar.gz
application/x-gzip 3.1k

Message body not shown because it is not plain text.

Subject: Re: [rt.cpan.org #80412] autodie hash order dependency
Date: Sat, 3 Nov 2012 13:54:08 +0100
To: bug-autodie [...] rt.cpan.org
From: demerphq <demerphq [...] gmail.com>
On 27 October 2012 03:04, Paul Fenwick via RT <bug-autodie@rt.cpan.org> wrote: Show quoted text
> If I can't find the underlying cause in a couple of days, I'll apply your > patch and push to the CPAN so this isn't blocking you any more. Do feel > free to nudge me on this if I haven't done so.
Nudge. :-) Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
On Sat Nov 03 08:54:17 2012, demerphq@gmail.com wrote: Show quoted text
> Nudge. :-)
Oops! New autodie pushed to the CPAN with your pathces applied. I still hope to investigate what's going on under the hood, but hopefully this means autodie won't be a roadblock any more. Sorry about the late apply, and keep being awesome! Paul
CC: yves [...] cpan.org, perl5-porters [...] perl.org
Subject: Re: [rt.cpan.org #80412] autodie hash order dependency
Date: Thu, 8 Nov 2012 09:02:39 +0100
To: bug-autodie [...] rt.cpan.org
From: demerphq <demerphq [...] gmail.com>
On 8 November 2012 04:29, PJF via RT <bug-autodie@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=80412 > > > On Sat Nov 03 08:54:17 2012, demerphq@gmail.com wrote: >
>> Nudge. :-)
> > Oops! New autodie pushed to the CPAN with your pathces applied. I > still hope to investigate what's going on under the hood, but hopefully > this means autodie won't be a roadblock any more. > > Sorry about the late apply, and keep being awesome!
Nice. You applied that in a way I didnt have to anything more than cherry-pick my commit. :-) So, I have pushed this to blead with 91d419acae8f9fecd22ae9054297725fbddda0a2 Thanks for the support! Yves -- perl -Mre=debug -e "/just|another|perl|hacker/"
Hi, According to the Changes file, this bug was fixed in v2.13. Marking it as resolved and adding "fixed in" version for it. ~Niels