Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the Dumbbench CPAN distribution.

Report information
The Basics
Id: 68647
Status: open
Priority: 0/
Queue: Dumbbench
People
Owner: Nobody in particular
Requestors: jnareb [...] gmail.com
Cc:
AdminCc:
Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)

History Show all quoted text
  Sat Jun 04 11:59:02 2011 jnareb [...] gmail.com - Ticket created
CC: Steffen Mueller <smueller [...] cpan.org>
Subject: Running 'dumbbench -v -v -v -- ./program' might return *negative* result
Date: Sat, 4 Jun 2011 17:58:44 +0200
To: bug-Dumbbench [...] rt.cpan.org
From: Jakub Narebski <jnareb [...] gmail.com>
Running dumbbench to benchmark external command might return *negative* results: $ dumbbench -v -v -v -i 25 -- ./test-something Running initial timing for warming up the cache... [...] Running 125 initial timings... Running timing 1... [...] Iterating until target precision reached... Reached relative precision 0.000320688275706346 (neeed 0.05). Running initial timing for warming up the cache... [...] Running 25 initial timings... Running timing 1... [...] Running timing 25... [...] Iterating until target precision reached... Reached relative precision 0.000802326211875739 (neeed 0.1). Ran 25 iterations (0 outliers). Rounded run time per iteration: -1.8e-03 +/- 1.3e-03 (-74.6%) Raw: -0.00178688812255867 +/- 0.00133240317867483 While $ time ./test-eigenv real 0m1.542s user 0m1.537s sys 0m0.003s Versions: $ perl -MDumbbench -wle 'print $Dumbbench::VERSION' 0.04 $ perl --version This is perl 5, version 12, subversion 3 (v5.12.3) built for x86_64-linux (with 13 registered patches, see perl -V for more detail) $ uname -a Linux grom 2.6.36-gentoo-r4 #5 SMP Mon Jan 17 20:53:04 CET 2011 x86_64 AMD Opteron(tm) Processor 6134 AuthenticAMD GNU/Linux P.S. I was unable to login to rt.cpan.org using my OpenID provider... -- Jakub Narebski Poland
  Thu Nov 10 02:47:09 2011 smueller [...] cpan.org - Correspondence added
Sorry for the long delay in replying. Could you retry with a newer version of Dumbbench, please? The version released in August had some fixes for dry-runs of external commands. Best regards, Steffen
  Thu Nov 10 02:47:10 2011 The RT System itself - Status changed from 'new' to 'open'
  Fri Nov 11 17:17:23 2011 jnareb [...] gmail.com - Correspondence added
Subject: Re: [rt.cpan.org #68647] Running 'dumbbench -v -v -v -- ./program' might return *negative* result
Date: Fri, 11 Nov 2011 23:17:30 +0100
To: bug-Dumbbench [...] rt.cpan.org
From: Jakub Narebski <jnareb [...] gmail.com>
Steffen Mueller via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=68647 > > > Sorry for the long delay in replying. Could you retry with a newer > version of Dumbbench, please? The version released in August had some > fixes for dry-runs of external commands.
Running Dumbbench 0.06 (upgraded from 0.04) still gives error for a very short running command. I guess that for this case it might happen that time of calibrating loop is longer than actually running a program. Try e.g. $ dumbbench -v -v -v -- echo The result is cmd: Iterating until target precision reached... cmd: Reached relative precision 0.00615156933222919 (neeed 0.1). cmd: Ran 20 iterations (0 outliers). cmd: Rounded run time per iteration: -1.0232e-03 +/- 8.1e-06 (-0.8%) cmd: Raw: -0.00102324821416614 +/- 8.12395073145449e-06 It would be better to inform user that 'dummbench' cannot reliably benchmark given program. -- Jakub Narebski Poland