Skip Menu |

This queue is for tickets about the Finance-Quote CPAN distribution.

Report information
The Basics
Id: 56469
Status: resolved
Worked: 15 min
Priority: 0/
Queue: Finance-Quote

People
Owner: eco [...] ecocode.net
Requestors: Marc.Ellenrieder [...] gmx.de
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Follow-Up on Bug #49414: General currency rate exchange rate problem in 1.17
Date: Sat, 10 Apr 2010 21:35:11 +0200
To: bug-Finance-Quote [...] rt.cpan.org
From: "Marc M. Ellenrieder" <Marc.Ellenrieder [...] gmx.de>
Hi, Bug #49414 seems to be a fairly common problem with exchange rates. See the below example: 1) perl currency-lookup.pl EUR RUB EUR->RUB = 39.269 2) perl currency-lookup.pl RUB EUR RUB->EUR = 2.5407 The example 2 is correct only for 100 RUB = 2.5407 EUR. The bug is present in Finance-Quote 1.17. After going through the source code of the Finance-Quote file Quote.pm I discovered, that this is a bug of the Yahoo page itself, since example 2) calls http://uk.finance.yahoo.com/q?s=RUBEUR=X and Yahoo itself yields the above (incorrect) value. Unless someone finds the correct URL to get Yahoo to quote the right value, we are somehow lost.
From: Ciprian.Enache [...] gmail.com
On Sat Apr 10 15:35:35 2010, Marc.Ellenrieder@gmx.de wrote: Show quoted text
> > Hi, > > Bug #49414 seems to be a fairly common problem with exchange rates. > See the > below example: > > 1) > perl currency-lookup.pl EUR RUB > EUR->RUB = 39.269 > > 2) > perl currency-lookup.pl RUB EUR > RUB->EUR = 2.5407 > > The example 2 is correct only for 100 RUB = 2.5407 EUR. > > The bug is present in Finance-Quote 1.17. After going through the > source code > of the Finance-Quote file Quote.pm I discovered, that this is a bug of > the > Yahoo page itself, since example 2) calls > > http://uk.finance.yahoo.com/q?s=RUBEUR=X > > and Yahoo itself yields the above (incorrect) value. Unless someone > finds the > correct URL to get Yahoo to quote the right value, we are somehow > lost.
This is also impacting other currencies such as the Indonesian Rupiah and the Thai Baht.
From: sbrabec [...] suse.cz
I reported the same problem with CZK/USD currency pair to Yahoo. The problem was acknowledged by Yahoo on Thu, 05 Nov 2009 as KMM87712558V70692L0KM, but not fixed until today. There is no was to fix it in the Finance:Quote module. (Unless somebody wants to parse the graph using OCR.) http://uk.finance.yahoo.com/q?s=CZKUSD=X
From: mta [...] umich.edu
I've attached a patch that works around this problem. It queries two Yahoo sites for exchange rates and compares the results. If they are off by a factor of 100 (more or less) it assumes that this problem exists for the current currency pair and fixes it.
Subject: Quote.pm.diff
--- lib/Finance/Quote.pm 2009-10-05 13:39:57.000000000 -0400 +++ lib/Finance/Quote.pm 2010-09-02 01:03:48.000000000 -0400 @@ -41,9 +41,11 @@ use vars qw/@ISA @EXPORT @EXPORT_OK @EXPORT_TAGS $VERSION $TIMEOUT %MODULES %METHODS $AUTOLOAD - $YAHOO_CURRENCY_URL $USE_EXPERIMENTAL_UA/; + $YAHOO_CURRENCY_URL $YAHOO_CURRENCY_QUOTES_URL + $USE_EXPERIMENTAL_UA/; $YAHOO_CURRENCY_URL = "http://uk.finance.yahoo.com/q?s="; +$YAHOO_CURRENCY_QUOTES_URL = "http://uk.finance.yahoo.com/d/quotes.csv?e=.csv&f=l&s="; @ISA = qw/Exporter/; @EXPORT = (); @@ -238,36 +240,88 @@ return $amount if ($from eq $to); # Trivial case. - my $ua = $this->user_agent; + sub primary_url { + my ($this,$from,$to) = @_; - my $data = $ua->request(GET "${YAHOO_CURRENCY_URL}$from$to%3DX")->content; - # The web page returns utf8 content which gives a warning when parsing $data - # in HTML::Parser - my $tb = HTML::TreeBuilder->new_from_content(decode_utf8($data)); - - # Find the <div> with the data - my $div = $tb->look_down('id','yfi_quote_summary_data'); - # Make sure there's a <div> to parse. - return undef unless $div; - - # The first <b> should contain the quote - my $rate_element=$div->look_down('_tag','b'); - # Make sure there's a <b> to parse. - return undef unless $rate_element; - - my $exchange_rate=$rate_element->as_text; - - $exchange_rate =~ s/,// ; # solve a bug when conversion rate - # involves thousands. yahoo inserts - # a comma when thousands occur - - { - local $^W = 0; # Avoid undef warnings. - - # We force this to a number to avoid situations where - # we may have extra cruft, or no amount. - return undef unless ($exchange_rate+0); + my $ua = $this->user_agent; + + my $data = $ua->request(GET "${YAHOO_CURRENCY_URL}$from$to%3DX")->content; + # The web page returns utf8 content which gives a warning when parsing $data + # in HTML::Parser + my $tb = HTML::TreeBuilder->new_from_content(decode_utf8($data)); + + # Find the <div> with the data + my $div = $tb->look_down('id','yfi_quote_summary_data'); + # Make sure there's a <div> to parse. + return undef unless $div; + + # The first <b> should contain the quote + my $rate_element=$div->look_down('_tag','b'); + # Make sure there's a <b> to parse. + return undef unless $rate_element; + + my $exchange_rate=$rate_element->as_text; + + $exchange_rate =~ s/,// ; # solve a bug when conversion rate + # involves thousands. yahoo inserts + # a comma when thousands occur + { + local $^W = 0; # Avoid undef warnings. + return undef unless ($exchange_rate+0); + } + return $exchange_rate * 1.0; } + + + sub secondary_url { + my ($this,$from,$to) = @_; + # Check against alternative URL as the original URL gives a rate + # multiplied by 100 for some currencies, for instance for NOKGBP=X + # (compare with GBPNOK=X) + # + # The alternative "CSV download" URL does not have this problem + # and don't require any screenscrapingm but the resolution is + # lower, typically just 2 digits, so we should only use it if we + # find a large difference from the original exchange rate. + # + # See http://www.gummy-stuff.org/Yahoo-data.htm for details about the + # fields for the f= parameter. + my $ua = $this->user_agent; + my $quotes_rate = $ua->request(GET "${YAHOO_CURRENCY_QUOTES_URL}$from$to%3DX")->content; + { + # Avoid undef warnings. + local $^W = 0; + return undef unless ($quotes_rate+0); + } + return $quotes_rate * 1.0; + } + + my $primary_rate = primary_url($this, $from, $to); + my $secondary_rate = secondary_url($this, $from, $to); + my $exchange_rate = $primary_rate; + + if ( $primary_rate && $secondary_rate ) { + my $ratio = $primary_rate / $secondary_rate; + if ( $ratio < 0.5 || $ratio > 2 ) { + # Large difference, ignore $primary_rate + # (This seems to happen for NOK->GBP, DKK->USD) + if ( $ratio < 106 && $ratio > 94 ) { + # off by factor of 100, this is the typical error + $exchange_rate = $exchange_rate / 100; + } + else { + $exchange_rate = $secondary_rate; + } + # TODO: Work out the difference in scale (typically 100) + # and multiply the original $exchange_rate + } + } elsif ( !$primary_rate && $secondary_rate ) { + # $primary rate failed, fall back to secondary + $exchange_rate = $secondary_rate; + } elsif ( ! $exchange_rate ) { + return undef; + } + return ($exchange_rate * $amount); }
From: mta [...] umich.edu
On Sun Feb 05 16:39:36 2012, malexander wrote: Show quoted text
> I've attached a patch that works around this problem. It queries two > Yahoo sites for exchange rates and compares the results. If they are > off by a factor of 100 (more or less) it assumes that this problem > exists for the current currency pair and fixes it.
This patch, by the way, comes from a message sent to the Finance::Quote mailing list on April 26, 2010 by Stian Soiland-Reyes. The message is archived at <http://sourceforge.net/mailarchive/forum.php?thread_name=j2l68db3ea71004261347lac34e2c6q5f6dd0ef978ee782%40mail.gmail.com&forum_name=finance-quote-devel>
Hi Thanks for the work ! We've changed the currency routine and it seems now that Yahoo returns correct results. I've added 2 tests in currency.t to check for anomalies of this kind. Tests added to the YahooApi2012 branch and will be merged into master later today. Don't hesitate to reopen this bug if you get any other anomalies... best -- Erik