Skip Menu |

This queue is for tickets about the Shannon-Entropy CPAN distribution.

Report information
The Basics
Id: 127573
Status: resolved
Priority: 0/
Queue: Shannon-Entropy

People
Owner: Nobody in particular
Requestors: john.imrie [...] oleeo.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Speed of execution.
Date: Mon, 5 Nov 2018 09:05:31 +0000
To: bug-Shannon-Entropy [...] rt.cpan.org
From: john <john.imrie [...] oleeo.com>
Hi there, This question https://stackoverflow.com/questions/51624871/calculating-the-entropy-of-a-32mb-file-in-perl-what-is-the-quickest-method/51628560#51628560 on Stack Overflow prompted me to rewrite your entropy function According to the person who wrote the original question this version is a significant speed up. Here's the new version in full. sub entropy { my ($entropy, $len, $p, %t) = (0, length($_[0])); my @chars = split '', $_[0]; $t{$_}++ foreach @chars; foreach (values %t) { $p = $_/$len; $entropy -= $p * log $p ; } return $entropy / log 2; } John
Subject: Re: [rt.cpan.org #127573] Speed of execution.
Date: Tue, 6 Nov 2018 10:24:45 +0700
To: bug-Shannon-Entropy [...] rt.cpan.org
From: "LNATION ." <thisusedtobeanemail [...] gmail.com>
Hey, can you run the following benchmark and let me know if you get the same result as me? use Benchmark qw(:all); my $string = 'aBDCS4REGW4aFAsdSDRsHRJKK32DBSEO0WGNHSGDJRSSJHSFFSLFSHJHJAMMHBMADBADanhkAdgdDJKJKdfdggdH4DBMBHADfsgfJsaJdJSHtSEGkDNGjThWHgFSfJRdWaTGHD8JKHADBHK7HE0WGSF9uSFiKHBhhSafRK5HH1AD3aaAKH9ohWJHdSFuNtSVr9SFSL75GLfJhDAJAEyAfMVjFSk2FDG43F1HBFSFSFSfsdfwe342sfd'; cmpthese(-10, { old => sub { my ($entropy, $len, $p, %t) = ($string, length($string)); return (map { $p = $_/$len; $entropy -= $p * log $p } (map { $t{$_}++ } split '', $_[0]) ? values %t : () and $entropy / log 2); }, new => sub { my ($entropy, $len, $p, %t) = ($string, length($string)); my @chars = split '', $_[0]; $t{$_}++ foreach @chars; foreach (values %t) { $p = $_/$len; $entropy -= $p * log $p ; } return $entropy / log 2; }, }); [image: Screenshot 2018-11-06 at 10.23.03.png] On Mon, Nov 5, 2018 at 4:05 PM john via RT <bug-Shannon-Entropy@rt.cpan.org> wrote: Show quoted text
> Mon Nov 05 04:05:44 2018: Request 127573 was acted upon. > Transaction: Ticket created by john.imrie@oleeo.com > Queue: Shannon-Entropy > Subject: Speed of execution. > Broken in: (no value) > Severity: (no value) > Owner: Nobody > Requestors: john.imrie@oleeo.com > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=127573 > > > > Hi there, > > This question > https://stackoverflow.com/questions/51624871/calculating-the-entropy-of-a-32mb-file-in-perl-what-is-the-quickest-method/51628560#51628560 > on Stack Overflow prompted me to rewrite your entropy function > > According to the person who wrote the original question this version is a > significant speed up. > > Here's the new version in full. > > sub entropy { > my ($entropy, $len, $p, %t) = (0, length($_[0])); > my @chars = split '', $_[0]; > $t{$_}++ foreach @chars; > > foreach (values %t) { > $p = $_/$len; > $entropy -= $p * log $p ; > } > > return $entropy / log 2; > } > > John >
Screenshot 2018-11-06 at 10.23.03.png
On Mon Nov 05 22:25:19 2018, thisusedtobeanemail@gmail.com wrote: Show quoted text
> Hey, > > can you run the following benchmark and let me know if you get the > same > result as me? > > use Benchmark qw(:all); > my $string = > 'aBDCS4REGW4aFAsdSDRsHRJKK32DBSEO0WGNHSGDJRSSJHSFFSLFSHJHJAMMHBMADBADanhkAdgdDJKJKdfdggdH4DBMBHADfsgfJsaJdJSHtSEGkDNGjThWHgFSfJRdWaTGHD8JKHADBHK7HE0WGSF9uSFiKHBhhSafRK5HH1AD3aaAKH9ohWJHdSFuNtSVr9SFSL75GLfJhDAJAEyAfMVjFSk2FDG43F1HBFSFSFSfsdfwe342sfd'; > cmpthese(-10, { > old => sub { > my ($entropy, $len, $p, %t) = (0, length($string)); > return (map { $p = $_/$len; $entropy -= $p * log $p } (map { > $t{$_}++ } split '', $string) ? values %t : () and $entropy / log 2); > }, > new => sub { > my ($entropy, $len, $p, %t) = (0, length($string)); > my @chars = split '', $string; > $t{$_}++ foreach @chars; > > foreach (values %t) { > $p = $_/$len; > $entropy -= $p * log $p ; > } > > return $entropy / log 2; > }, > }); > > [image: Screenshot 2018-11-06 at 10.23.03.png] > > On Mon, Nov 5, 2018 at 4:05 PM john via RT <bug-Shannon- > Entropy@rt.cpan.org> > wrote: >
> > Mon Nov 05 04:05:44 2018: Request 127573 was acted upon. > > Transaction: Ticket created by john.imrie@oleeo.com > > Queue: Shannon-Entropy > > Subject: Speed of execution. > > Broken in: (no value) > > Severity: (no value) > > Owner: Nobody > > Requestors: john.imrie@oleeo.com > > Status: new > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=127573 > > > > > > > Hi there, > > > > This question > > https://stackoverflow.com/questions/51624871/calculating-the-entropy- > > of-a-32mb-file-in-perl-what-is-the-quickest-method/51628560#51628560 > > on Stack Overflow prompted me to rewrite your entropy function > > > > According to the person who wrote the original question this version > > is a > > significant speed up. > > > > Here's the new version in full. > > > > sub entropy { > > my ($entropy, $len, $p, %t) = (0, length($_[0])); > > my @chars = split '', $_[0]; > > $t{$_}++ foreach @chars; > > > > foreach (values %t) { > > $p = $_/$len; > > $entropy -= $p * log $p ; > > } > > > > return $entropy / log 2; > > } > > > > John > >
Subject: Re: [rt.cpan.org #127573] Speed of execution.
Date: Tue, 6 Nov 2018 10:45:18 +0700
To: bug-Shannon-Entropy [...] rt.cpan.org
From: "LNATION ." <thisusedtobeanemail [...] gmail.com>
updated code: use Benchmark qw(:all); my $string = 'zSQ32dfW2Ffc43lOp3fgr5gbaBDCS4REGW4aFAsdSDRsHRJKK32DBSEO0WGNHSGDJR SSJHSFFSLFSHJHJAMMHBMADBADanhkAdgdDJKJKdfds123243453dsfdge34ldgdbdbkbhggdldjgkdggdH4DBMBHADfsgfJsaJdJSHtSEGkDNGjThWHgFSfJRdWaTGHD8JKHADBHK7HE0WGSF9uSFiKHBhhSafRK5HH1AD3aaAKH9ohWJHdSFuNtSVr9SFSL75GLfJhDAJAEyAfMVjFSk2FDG43F1HBFSFSFSfsdfwe342sfd'; cmpthese(-10, { old => sub { my ($entropy, $len, $p, %t) = (0, length($string)); return (map { $p = $_/$len; $entropy -= $p * log $p } (map { $t{$_}++ } split '', $string) ? values %t : () and $entropy / log 2); }, new => sub { my ($entropy, $len, $p, %t) = (0, length($string)); my @chars = split '', $string; $t{$_}++ foreach @chars; foreach (values %t) { $p = $_/$len; $entropy -= $p * log $p ; } return $entropy / log 2; }, anew => sub { my ($entropy, $len, $p, %t) = (0, length($string)); $t{$_}++ foreach split '', $string; foreach (values %t) { $p = $_/$len; $entropy -= $p * log $p ; } return $entropy / log 2; }, }); in which case anew is fastest :) On Tue, Nov 6, 2018 at 10:24 AM LNATION . <thisusedtobeanemail@gmail.com> wrote: Show quoted text
> Hey, > > can you run the following benchmark and let me know if you get the same > result as me? > > use Benchmark qw(:all); > my $string = > 'aBDCS4REGW4aFAsdSDRsHRJKK32DBSEO0WGNHSGDJRSSJHSFFSLFSHJHJAMMHBMADBADanhkAdgdDJKJKdfdggdH4DBMBHADfsgfJsaJdJSHtSEGkDNGjThWHgFSfJRdWaTGHD8JKHADBHK7HE0WGSF9uSFiKHBhhSafRK5HH1AD3aaAKH9ohWJHdSFuNtSVr9SFSL75GLfJhDAJAEyAfMVjFSk2FDG43F1HBFSFSFSfsdfwe342sfd'; > cmpthese(-10, { > old => sub { > my ($entropy, $len, $p, %t) = ($string, length($string)); > return (map { $p = $_/$len; $entropy -= $p * log $p } (map { > $t{$_}++ } split '', $_[0]) ? values %t : () and $entropy / log 2); > }, > new => sub { > my ($entropy, $len, $p, %t) = ($string, length($string)); > my @chars = split '', $_[0]; > $t{$_}++ foreach @chars; > > foreach (values %t) { > $p = $_/$len; > $entropy -= $p * log $p ; > } > > return $entropy / log 2; > }, > }); > > [image: Screenshot 2018-11-06 at 10.23.03.png] > > On Mon, Nov 5, 2018 at 4:05 PM john via RT < > bug-Shannon-Entropy@rt.cpan.org> wrote: >
>> Mon Nov 05 04:05:44 2018: Request 127573 was acted upon. >> Transaction: Ticket created by john.imrie@oleeo.com >> Queue: Shannon-Entropy >> Subject: Speed of execution. >> Broken in: (no value) >> Severity: (no value) >> Owner: Nobody >> Requestors: john.imrie@oleeo.com >> Status: new >> Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=127573 > >> >> >> Hi there, >> >> This question >> https://stackoverflow.com/questions/51624871/calculating-the-entropy-of-a-32mb-file-in-perl-what-is-the-quickest-method/51628560#51628560 >> on Stack Overflow prompted me to rewrite your entropy function >> >> According to the person who wrote the original question this version is a >> significant speed up. >> >> Here's the new version in full. >> >> sub entropy { >> my ($entropy, $len, $p, %t) = (0, length($_[0])); >> my @chars = split '', $_[0]; >> $t{$_}++ foreach @chars; >> >> foreach (values %t) { >> $p = $_/$len; >> $entropy -= $p * log $p ; >> } >> >> return $entropy / log 2; >> } >> >> John >>
>
Screenshot 2018-11-06 at 10.23.03.png