Skip Menu |

This queue is for tickets about the Tie-Hash-Indexed CPAN distribution.

Report information
The Basics
Id: 81449
Status: patched
Priority: 0/
Queue: Tie-Hash-Indexed

People
Owner: Nobody in particular
Requestors: ANDK [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in:
  • 0.05
  • 0.05_01
  • 0.05_02
  • 0.05_03
Fixed in: (no value)



Subject: Test failures due to hash randomisation in perl 5.17.6
Since bleadperl v5.17.5-518-g7dc8663 your tests are failing frequently. That commit introduced hash key randomization and it seems at least the test t/101_basic.t is hit by that. Find a sample fail report at: http://www.cpantesters.org/cpan/report/76500b20-318a-11e2-8d7f-bdada290f8f5 You can read more about the commit at http://perl5.git.perl.org/perl.git/commit/7dc8663964c66a698d31bbdc8e8abed69bddeec3 You may have to run the test several times until the randomization causes a fail. HTH&&Thanks&&Regards,
I couldn't think of a sensible fix for this since the randomization means the number of buckets will change, so I marked the test as invalid on 5.17.6+
Subject: hash_randomization.patch
diff -rupN orig/t/101_basic.t new/t/101_basic.t --- orig/t/101_basic.t 2007-08-24 13:18:54.000000000 +0000 +++ new/t/101_basic.t 2013-06-11 19:08:33.129745242 +0000 @@ -24,6 +24,9 @@ ok(1); $scalar = $] < 5.008003 || $] == 5.009 ? 'skip: no scalar context for tied hashes' : ''; +$scalar = $] > 5.017005 + ? 'skip: hash randomization changes make this test invalid' : ''; + $broken_untie = $] == 5.009003 ? 'skip: broken untie' : ''; tie %h, 'Tie::Hash::Indexed';
Would you mind making a release? T:H:I has 14 reverse dependencies, so it would probably pay off for some users. https://metacpan.org/requires/distribution/Tie-Hash-Indexed?sort=[[2,1]] Thanks && Regards from Berlin,
Ditto +1 from me. The upcoming MCE 1.7 release will come with MCE::Shared allowing ordered hashes with full support for deep data sharing. Your module is one of three allowed. use Tie::Hash::Indexed; use MCE::Flow max_workers => 4; use MCE::Shared ordered => 1; my $db = mce_share_m {}; # Tie::Hash::Indexed behind the scene with MCE::Mutex locking $db->{key1} = 'foo'; $db->{key2} = 'bar'; $db->{key3} = 'baz'; $db->{counter} = 0; mce_flow sub { $db->lock(); $db->{counter} += 1; $db->unlock(); }; print $db->{counter}, "\n"; # 4 Unfortunately, am placed in an uncomfortable place and having to make a decision on wether to allow ordered keys from your incredible module. I am not totally sure if your module is working with Perl v5.18 and above. Look at T::H::I go... :-) 288,000 FETCH's/STORE's Tie::Hash::Indexed 0.307 secs Hash::Ordered 0.443 secs Tie::IxHash 0.519 secs +1 for Marcus and T::H::I Regards, - Mario
The results just posted were from the Manager side, not workers. Anyway, T::H::I is very fast. The underlying methods are called when possible behind the scene for maximum performance; e.g. ->STORE, ->FETCH, etc. Thank you for T::H::I.
Hi Marcus, The 101_basic.t script is the reason for failing. The following diff allows Tie::Hash::Indexed to build on newer Perl versions. Thus, zero changes to the underlying XS code. diff -uar Tie-Hash-Indexed-0.05/t/101_basic.t Tie-Hash-Indexed-0.05-fix/t/101_basic.t --- Tie-Hash-Indexed-0.05/t/101_basic.t 2007-08-24 09:18:54.000000000 -0400 +++ Tie-Hash-Indexed-0.05-fix/t/101_basic.t 2016-05-11 09:18:30.000000000 -0400 @@ -39,20 +39,17 @@ ok(exists $h{foo}); ok(exists $h{bar}); ok(!exists $h{xxx}); -$s = &scalar_h; -skip($scalar, $s =~ /^(\d+)\/\d+$/ && $1 == scalar keys %h); +ok(scalar keys %h, 4); $h{xxx} = 5; ok(join(',', keys %h), 'foo,bar,zoo,baz,xxx'); ok(exists $h{xxx}); -$s = &scalar_h; -skip($scalar, $s =~ /^(\d+)\/\d+$/ && $1 == scalar keys %h); +ok(scalar keys %h, 5); $h{foo} = 6; ok(join(',', keys %h), 'foo,bar,zoo,baz,xxx'); ok(exists $h{foo}); -$s = &scalar_h; -skip($scalar, $s =~ /^(\d+)\/\d+$/ && $1 == scalar keys %h); +ok(scalar keys %h, 5); while (my($k,$v) = each %h) { $key .= $k; Regards, Mario
Hey! Sorry for the looooong wait. Given how simple the fix is (and how stupid my original tests were), there's really no excuse for such a late update. :-/ I've (more or less) applied your patch and added some additional tests. Also, the github repo is finally up-to-date. You can give T:H:I 0.05_01 a spin which is just entering CPAN. I'll release 0.06 once 0.05_01 has gone through some testing. Cheers, Marcus
Hi Marcus, Currently, Tie::Hash::Indexed and Tie::Hash::LRU are the fastest solutions for Perl. Since the long delay, Pure-Perl solutions have reached new levels. MCE::Shared::Ordhash, based on Hash::Ordered, includes an optimized tombstone deletion without impacting the rest of the library. No matter how hard I tried (many many hours), I was unable to reach XS levels :) That's okay now :) The following is a small benchmark script which performs random deletion for 1 million keys. Basically, random deletion is no longer a problem for folks without a C compiler. use strict; use warnings; use List::Util 'shuffle'; use Time::HiRes 'time'; use Tie::Hash::Indexed; use MCE::Shared::Ordhash; use Hash::Ordered; srand 1618; my $c = shift || 1; # (1) Tie::Hash::Indexed # (2) MCE::Shared::Ordhash # (3) Hash::Ordered my $max = 1_000_000; my @rand = shuffle(1..$max); my $oh1 = tie my %h1, 'Tie::Hash::Indexed'; my $oh2 = tie my %h2, 'MCE::Shared::Ordhash'; my $oh3 = tie my %h3, 'Hash::Ordered'; my $oh; if ($c eq '1') { print ref($oh1), "\n"; $oh = $oh1 } elsif ($c eq '2') { print ref($oh2), "\n"; $oh = $oh2 } elsif ($c eq '3') { print ref($oh3), "\n"; $oh = $oh3 } else { print "usage: $0 [ 1 | 2 | 3 ]\n" } my $s = time; $oh->STORE($_,$_) for 1..$max; printf "store : %0.3f secs.\n", time - $s; my $d = time; $oh->DELETE($_) for @rand; printf "delete : %0.3f secs.\n", time - $d; printf "elapse : %0.3f secs.\n", time - $s; print "\n"; __END__ $ perl bench.pl 1 Tie::Hash::Indexed store : 0.752 secs. delete : 1.229 secs. elapse : 1.981 secs. $ perl bench.pl 2 MCE::Shared::Ordhash store : 1.034 secs. delete : 2.600 secs. elapse : 3.634 secs. $ perl bench.pl 3 Hash::Ordered store : 1.317 secs. delete : 4.090 secs. elapse : 5.407 secs. T::H::I is really nice to have in any Perl toolbox. Thank you, Mario
The following provides a parallel demonstration for sharing an ordered hash between workers. use strict; use warnings; use List::Util 'shuffle'; use Time::HiRes 'time'; use MCE::Hobo; use MCE::Shared; use Tie::Hash::Indexed; use MCE::Shared::Ordhash; use Hash::Ordered; srand 1618; my $c = shift || 1; # (1) Tie::Hash::Indexed # (2) MCE::Shared::Ordhash # (3) Hash::Ordered my $max = 1_000_000; my @rand = shuffle(1..$max); my $seq = MCE::Shared->sequence( { chunk_size => 500, bounds_only => 1 }, 1, $max ); my $oh1 = MCE::Shared->share( Tie::Hash::Indexed->TIEHASH ); my $oh2 = MCE::Shared->share( MCE::Shared::Ordhash->new ); my $oh3 = MCE::Shared->share( Hash::Ordered->new ); my $oh; if ($c eq '1') { print $oh1->blessed, "\n"; $oh = $oh1 } elsif ($c eq '2') { print $oh2->blessed, "\n"; $oh = $oh2 } elsif ($c eq '3') { print $oh3->blessed, "\n"; $oh = $oh3 } else { print "usage: $0 [ 1 | 2 | 3 ]\n" } my $s = time; sub _store { while ( my ( $beg, $end ) = $seq->next ) { $oh->STORE($_,$_) for $beg..$end; } }; MCE::Hobo->create('_store') for 1..8; MCE::Hobo->waitall; printf "store : %0.3f secs.\n", time - $s; $seq->rewind; my $d = time; sub _delete { while ( my ( $beg, $end ) = $seq->next ) { $oh->DELETE($_) for $beg..$end; } }; MCE::Hobo->create('_delete') for 1..8; MCE::Hobo->waitall; printf "delete : %0.3f secs.\n", time - $d; printf "elapse : %0.3f secs.\n", time - $s; print "\n"; __END__ $ perl bench_shared.pl 1 Tie::Hash::Indexed store : 2.803 secs. delete : 2.553 secs. elapse : 5.356 secs. $ perl bench_shared.pl 2 MCE::Shared::Ordhash store : 3.093 secs. delete : 3.810 secs. elapse : 6.903 secs. $ perl bench_shared.pl 3 Hash::Ordered store : 3.434 secs. delete : 4.325 secs. elapse : 7.760 secs. It is really nice and cool for T::H::I to run on Perl 5.18 and later :) Thank you, Mario
T:H:I 0.05_01 was missing a file, T:H:I 0.05_02 just entered CPAN.
Le 2016-05-12 10:51:34, MHX a écrit :
Show quoted text
> T:H:I 0.05_01 was missing a file, T:H:I 0.05_02 just entered CPAN.

Time for a stable release?

-- 
Olivier Mengué - http://perlresume.org/DOLMEN
The tests are still falling in 5.22.2 Can you please make a new release?
On Tue Apr 04 05:01:59 2017, NKH wrote: Show quoted text
> The tests are still falling in 5.22.2 > > Can you please make a new release?
And still failing with perl-5.27.9: ##### [Tie-Hash-Indexed-0.05] 522 $ bleadprove -vb t/101_basic.t t/101_basic.t .. 1..32 # Running under perl version 5.027009 for linux # Current time local: Wed Feb 21 21:33:52 2018 # Current time GMT: Thu Feb 22 02:33:52 2018 # Using Test.pm version 1.31 ok 1 ok 2 ok 3 ok 4 ok 5 ok 6 ok 7 not ok 8 ok 9 ok 10 not ok 11 ok 12 ok 13 not ok 14 ok 15 ok 16 ok 17 ok 18 ok 19 ok 20 ok 21 ok 22 ok 23 ok 24 ok 25 ok 26 ok 27 ok 28 ok 29 ok 30 ok 31 ok 32 # Failed test 8 in t/101_basic.t at line 43 # t/101_basic.t line 43 is: skip($scalar, $s =~ /^(\d+)\/\d+$/ && $1 == scalar keys %h); # Failed test 11 in t/101_basic.t at line 49 # t/101_basic.t line 49 is: skip($scalar, $s =~ /^(\d+)\/\d+$/ && $1 == scalar keys %h); # Failed test 14 in t/101_basic.t at line 55 # t/101_basic.t line 55 is: skip($scalar, $s =~ /^(\d+)\/\d+$/ && $1 == scalar keys %h); Failed 3/32 subtests Test Summary Report ------------------- t/101_basic.t (Wstat: 0 Tests: 32 Failed: 3) Failed tests: 8, 11, 14 Files=1, Tests=32, 0 wallclock secs ( 0.03 usr + 0.00 sys = 0.03 CPU) Result: FAIL #####
And still failing in perl 5.26.1 Manifying 1 pod document "/usr/bin/perl" -MExtUtils::Command::MM -e 'cp_nonempty' -- Indexed.bs blib/arch/auto/Tie/Hash/Indexed/Indexed.bs 644 PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t # Failed test 8 in t/101_basic.t at line 43 # t/101_basic.t line 43 is: skip($scalar, $s =~ /^(\d+)\/\d+$/ && $1 == scalar keys %h); # Failed test 11 in t/101_basic.t at line 49 # t/101_basic.t line 49 is: skip($scalar, $s =~ /^(\d+)\/\d+$/ && $1 == scalar keys %h); # Failed test 14 in t/101_basic.t at line 55 # t/101_basic.t line 55 is: skip($scalar, $s =~ /^(\d+)\/\d+$/ && $1 == scalar keys %h); t/101_basic.t ..... Failed 3/32 subtests
Subject: Test failures due to perl 5.17.6 and perl 5.25.3
Bleadperl v5.25.2-28-g8bf4c40 introduced that scalar %hash == keys %hash and as such broke some tests in MHX/Tie-Hash-Indexed-0.05_03.tar.gz : https://perl5.git.perl.org/perl.git/commit/8bf4c40 The bleadperl BBC ticket for this change was https://rt.perl.org/Public/Bug/Display.html?id=128482, closed because nobody had reported Tie-Hash-Indexed-0.05_03 (possibly because it is a dev release).