Skip Menu |

This queue is for tickets about the Finance-Quote CPAN distribution.

Report information
The Basics
Id: 50424
Status: resolved
Worked: 50 min
Priority: 0/
Queue: Finance-Quote

People
Owner: eco [...] ecocode.net
Requestors: user42 [...] zip.com.au
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: yahoo "cap" expand B for billions
Date: Tue, 13 Oct 2009 11:11:22 +1100
To: bug-Finance-Quote [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
The "cap" from yahoo seems to sometimes come back with a B for billions, eg. for Ford use strict; use warnings; use Finance::Quote; my $q = Finance::Quote->new; my %quotes = $q->fetch('yahoo','F'); print $quotes{'F','cap'},"\n"; gives 24.546B It'd be good if that was expanded to a plain number, so it can be used directly in arithmetic without having to crunch down a source-specific format. (If an abbreviation is wanted for display then a plain number is the best starting point, since any abbreviation will be language dependent, and you can choose how many decimal places or sig figures are desired.) Perhaps along the following lines,

Message body is not shown because sender requested not to inline it.

Message body is not shown because sender requested not to inline it.

Hello ! Replying to this bug for keeping references... Yep, that would be great if you could update this patch and test file ! Best -- erik On Mon Oct 12 20:12:06 2009, user42@zip.com.au wrote: Show quoted text
> The "cap" from yahoo seems to sometimes come back with a B for > billions, eg. for Ford > > use strict; > use warnings; > use Finance::Quote; > my $q = Finance::Quote->new; > my %quotes = $q->fetch('yahoo','F'); > print $quotes{'F','cap'},"\n"; > > gives > > 24.546B > > It'd be good if that was expanded to a plain number, so it can be used > directly in arithmetic without having to crunch down a source-specific > format. > > (If an abbreviation is wanted for display then a plain number is the > best starting point, since any abbreviation will be language dependent, > and you can choose how many decimal places or sig figures are desired.) > > Perhaps along the following lines, >
-- Erik
Subject: Re: [rt.cpan.org #50424] yahoo "cap" expand B for billions
Date: Tue, 15 Feb 2011 09:17:54 +1100
To: bug-Finance-Quote [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
Fixed up and done as a format-patch,
From 5ade9d46eee237c9bf5d7a6828de992084bed8b2 Mon Sep 17 00:00:00 2001 From: Kevin Ryde <user42@zip.com.au> Date: Tue, 15 Feb 2011 09:16:34 +1100 Subject: [PATCH] Yahoo "cap" field expand "B" for billions like "1.5B" as "1500000000" so it can be used as a number, or shown with a non-English abbreviation, etc. --- lib/Finance/Quote/Yahoo/Base.pm | 61 +++++++++++++++++++++++++++++++++++++++ t/yahoo_base.t | 38 ++++++++++++++++++++++++ 2 files changed, 99 insertions(+), 0 deletions(-) create mode 100644 t/yahoo_base.t diff --git a/lib/Finance/Quote/Yahoo/Base.pm b/lib/Finance/Quote/Yahoo/Base.pm index b737230..c3313bd 100644 --- a/lib/Finance/Quote/Yahoo/Base.pm +++ b/lib/Finance/Quote/Yahoo/Base.pm @@ -250,6 +250,14 @@ sub yahoo_request { $info{$symbol,"time"} = $quoter->isoTime($info{$symbol,"time"}); } + # "cap" from Yahoo::USA sometimes has "B" for + # billions suffix, eg. from "F" Ford -- expand that + # to a plain number for ease of use + if (defined($info{$symbol,"cap"})) { + $info{$symbol,"cap"} + = _B_to_billions ($info{$symbol,"cap"}); + } + # Convert prices (when needed). E.G. Some London sources # return in pence. Yahoo denotes this with GBP vs GBp # We'd like them to return in pounds (divide by 100). @@ -325,6 +333,59 @@ sub yahoo_request { return \%info; } +# If $str ends with a B like "20B" or "1.6B" then expand it as billions like +# "20000000000" or "1600000000". +# +# This is done with string manipulations so floating-point rounding doesn't +# produce spurious digits for values like "1.6" which aren't exactly +# representable in binary. +# +# Is "B" for billions the only abbreviation from Yahoo? +# Could extend and rename this if there's also millions or thousands. +# +# For reference, if the value was just for use within perl then simply +# substituting to exponential "1.5e9" might work. But expanding to full +# digits seems a better idea as the value is likely to be printed directly +# as a string. +# +sub _B_to_billions { + my ($str) = @_; + ### _B_to_billions(): $str + if ($str =~ s/B$//i) { + $str = _decimal_shiftup ($str, 9); + } + return $str; +} + +# $str is a number like "123" or "123.45" +# return it with the decimal point moved $shift places to the right +# must have $shift>=1 +# eg. _decimal_shiftup("123",3) -> "123000" +# _decimal_shiftup("123.45",1) -> "1234.5" +# _decimal_shiftup("0.25",1) -> "2.5" +# +sub _decimal_shiftup { + my ($str, $shift) = @_; + + # delete decimal point and set $after to count of chars after decimal. + # Leading "0" as in "0.25" is deleted too giving "25" so as not to end up + # with something that might look like leading 0 for octal. + my $after = ($str =~ s/(^0)?\.(.*)/$2/ ? length($2) : 0); + + $shift -= $after; + # now $str is an integer and $shift is relative to the end of $str + + if ($shift >= 0) { + # moving right, eg. "1234" becomes "12334000" + return $str . ('0' x $shift); # extra zeros appended + } else { + # negative means left, eg. "12345" becomes "12.345" + # no need to prepend zeros since demanding initial $shift>=1 + substr ($str, $shift,0, '.'); # new '.' at shifted spot from end + return $str; + } +} + 1; =head1 NAME diff --git a/t/yahoo_base.t b/t/yahoo_base.t new file mode 100644 index 0000000..776fa9d --- /dev/null +++ b/t/yahoo_base.t @@ -0,0 +1,38 @@ +#!/usr/bin/perl -w +use strict; +use Finance::Quote::Yahoo::Base; +use Test::More; + +plan tests => 18; + +#------------------------------------------------------------------------------ +# _decimal_shiftup() + +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('1',1), '10'); +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('1',2), '100'); + +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('1.',1), '10'); +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('1.',2), '100'); + +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('1.5',1), '15'); +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('1.5',2), '150'); +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('1.5',3), '1500'); + +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('56',1), '560'); +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('56',2), '5600'); + +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('1.2345678901234',3), + '1234.5678901234'); +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('0.12345678',1), + '1.2345678'); +is(Finance::Quote::Yahoo::Base::_decimal_shiftup('0.00001',1), + '0.0001'); + +#------------------------------------------------------------------------------ +# _B_to_billions() + +is(Finance::Quote::Yahoo::Base::_B_to_billions('1B'), '1000000000'); +is(Finance::Quote::Yahoo::Base::_B_to_billions('1.5B'), '1500000000'); +is(Finance::Quote::Yahoo::Base::_B_to_billions('1.23456789876B'), '1234567898.76'); + +exit 0; -- 1.7.2.3
Kevin, Thanks for your patch ! I applied it in a 'billions' branch on github : https://github.com/pfenwick/finance- quote/tree/billions Created a branch since I want to move some part of code to Quote.pm. Best -- Erik
Kevin, Been thinking why you didn't use arithmetics and sprintf in _decimal_shiftup. Rewrote it using these and got following benchmark results (based on your tests): Benchmark: timing 500000 iterations of _decimal_shiftup, sprintf... _decimal_shiftup: 26 wallclock secs (25.11 usr + 0.02 sys = 25.13 CPU) @ 19896.54/s (n=500000) sprintf: 23 wallclock secs (22.97 usr + 0.01 sys = 22.98 CPU) @ 21758.05/s (n=500000) Rate _decimal_shiftup sprintf _decimal_shiftup 19897/s -- -9% sprintf 21758/s 9% -- (Benchmark script including my version is attached). I think most time in your routine is lost in the regexp, so maybe rewriting this can reduce the gap, but that won't bring it to 0... I'm willing to move the _decimal_shift and _B_to_billions subs to Quote.pm, so a little optimization won't harm ;) My version passes your tests, but don't hesitate to tell me if you can think of any circumstances which would void my version... Best -- Erik
Subject: dec.pl
use Benchmark qw(:all); my $results = timethese (500000,{_decimal_shiftup=>test1 , sprintf=>test2} ); cmpthese($results); sub test1 { _decimal_shiftup('1',1) ."\n"; _decimal_shiftup('1',2) ."\n"; _decimal_shiftup('1.',1) ."\n"; _decimal_shiftup('1.',2) ."\n"; _decimal_shiftup('1.5',1) ."\n"; _decimal_shiftup('1.5',2) ."\n"; _decimal_shiftup('1.5',3) ."\n"; _decimal_shiftup('56',1) ."\n"; _decimal_shiftup('56',2) ."\n"; _decimal_shiftup('1.2345678901234',3) ."\n"; _decimal_shiftup('0.12345678',1) ."\n"; _decimal_shiftup('0.00001',1) ."\n"; } sub test2 { shift_dec('1',1) ."\n"; shift_dec('1',2) ."\n"; shift_dec('1.',1) ."\n"; shift_dec('1.',2) ."\n"; shift_dec('1.5',1) ."\n"; shift_dec('1.5',2) ."\n"; shift_dec('1.5',3) ."\n"; shift_dec('56',1) ."\n"; shift_dec('56',2) ."\n"; shift_dec('1.2345678901234',3) ."\n"; shift_dec('0.12345678',1) ."\n"; shift_dec('0.00001',1) ."\n"; } sub shift_dec { my ($f, $shift_dec) = @_ ; my $precision = length($1) if ($f=~ /\.(\d+)/) || 0 ; my $new_precision = ($shift_dec>$precision) ? 0 : $precision-$shift_dec ; $f *= (10**$shift_dec) ; return sprintf("%.".$new_precision."f",$f) ; } sub _decimal_shiftup { my ($str, $shift) = @_; my $after = ($str =~ s/(^0)?\.(.*)/$2/ ? length($2) : 0); $shift -= $after; if ($shift >= 0) { return $str . ('0' x $shift); # extra zeros appended } else { substr ($str, $shift,0, '.'); # new '.' at shifted spot from end return $str; } }
Subject: Re: [rt.cpan.org #50424] yahoo "cap" expand B for billions
Date: Thu, 17 Feb 2011 09:37:37 +1100
To: bug-Finance-Quote [...] rt.cpan.org
From: Kevin Ryde <user42 [...] zip.com.au>
"Erik Colson via RT" <bug-Finance-Quote@rt.cpan.org> writes: Show quoted text
> > Been thinking why you didn't use arithmetics and sprintf in > _decimal_shiftup.
In general it can suffer spurious non-zero digits for an input like "1.6B" if the 1.6 is put into a float and out again. I expect a bit of cooking could make an example giving ".999999", or changing the last digit, or possibly also depending on the FPU rounding mode. The shift bit is slightly ugly. I'd be happy for someone else's module to do some lightweight decimal arithmetic like that, but do I think it's important to stay in decimal or in integers to maintain accuracy. (Speed of course is swamped by the download time.)
I guess this was solved and moved into master in feb 2011 -- Erik