| Subject: | from sourceforge : Tiaacref quote source is broken (POST versus GET) |
It appears that recently (around 23 October 2007) TIAA-CREF switched from
using the GET form submission method to the POST method. As a result, the
code in Tiaacref.pm no longer works (Finance::Quote version 1.13). In
other words, Tiaacref can no longer be used as a quote source. Here's the
diff for the change that I made to get this working with Gnucash. Note
that I'm not convinced that this is a general solution (in particular, I
don't know if $cntt is ever non-zero at line 294; I hope not because I
don't see where $urlt/$TIAA_URL are defined). In any case, here's the
diff...
--- /usr/lib/perl5/site_perl/5.8.8/Finance/Quote/Tiaacref.pm-old
2007-10-28 09:46:10.000000000 -0700
+++ /usr/lib/perl5/site_perl/5.8.8/Finance/Quote/Tiaacref.pm-new
2007-10-28 11:09:31.000000000 -0700
@@ -45,5 +45,5 @@
# URLs of where to obtain information.
# This used to be different for the CREF and TIAA annuities, but this
changed.
-$CREF_URL = ("https://www3.tiaa-cref.org/ddata/DownloadData?");
+$CREF_URL = ("https://www3.tiaa-cref.org/ddata/DownloadData");
sub methods { return (tiaacref=>\&tiaacref); }
@@ -257,7 +257,9 @@
my($cntc,$cntt); #counters for each of the two url containers
my($reply,$qdata); #the reply from TIAA-CREF's cgi and a
buffer for the data
+ my(@postc); @holds post vars
# $url = $TIAACREF_URL;
$urlc = $CREF_URL;
+ @postc = ();
$urlt = $TIAA_URL;
#Initialize counters for the two types of URL. If either counter is zero,
then
@@ -270,5 +272,5 @@
if ($tiaacref_locs{$fund} == 1) {
$cntc++;
- $urlc .= "f" . $cntc . "=" . $tiaacref_vals{$fund}
. "&";
+ push(@postc, "f" . $cntc => $tiaacref_vals{$fund});
} else {
$urlt .= $fund . "=yes&";
@@ -281,5 +283,5 @@
}
}
- $urlc .= "days=1";
+ push(@postc, days => '1', dldformat => '1');
$urlt .= "selected=1";
$qdata ="";
@@ -287,5 +289,5 @@
$ua = $quoter->user_agent;
if ($cntc) {
- $reply = $ua->request(GET $urlc);
+ $reply = $ua->request(POST $urlc, [@postc]);
if ($reply ->is_success) {
$qdata .= $reply->content;
Submitted By:
E Lewis - eclewis
Date Submitted:
2007-10-28 18:35
Followups:
Comments
Date: 2008-06-03 03:00
Sender: jcabanela
Logged In: YES
user_id=205804
Originator: NO
I can confirm these edits make the TIAA-CREF module work whereas without
them it no longer does.
Date: 2008-01-20 07:13
Sender: wigie
Logged In: YES
user_id=1986406
Originator: NO
GET still seems to work (POST not necessary), but the site continues to
undergo changes. Several funds in Tiaacref.pm don't exist anymore, and
many new ones were created.
Riffing on dbreiser's diff, here's what works for me as of 20 Jan 2008:
--- Tiaacref.pm.old 2008-01-20 02:07:56.000000000 -0500
+++ Tiaacref.pm 2008-01-20 01:59:29.000000000 -0500
@@ -280,7 +280,7 @@
$info{$fund,"errormsg"} = "Bad symbol";
}
}
- $urlc .= "days=1";
+ $urlc .= "days=1&dldformat=1&maxsize=9999";
$urlt .= "selected=1";
$qdata ="";
Date: 2008-01-03 23:26
Sender: dbreiser
Logged In: YES
user_id=1044137
Originator: NO
hmm. I guess I can't add a file if it isn't my bug.
Date: 2008-01-03 23:20
Sender: dbreiser
Logged In: YES
user_id=1044137
Originator: NO
Here's a simpler diff that worked for me 1/3/2008. I don't know if they
changed the site again between 10/28 and Jan/08, but the change to me looks
like one required when they added Excel file format downloads. (also added
to bug report as a file)
--- Tiaacref.pm 2008-01-03 18:08:21.000000000 -0500
+++ newTiaacref.pm 2008-01-03 18:04:52.000000000 -0500
@@ -280,7 +280,7 @@
$info{$fund,"errormsg"} = "Bad symbol";
}
}
- $urlc .= "days=1";
+ $urlc .= "days=1&dldformat=1";
$urlt .= "selected=1";
$qdata ="";
Date: 2007-10-28 18:44
Sender: eclewis
Logged In: YES
user_id=1923929
Originator: YES
Sorry about the terrible formatting. Yuck. Also, there's a bug in the
diff "@holds post vars" should be "#holds post vars".