Bug #37679 for Geo-Coder-Yahoo: utf8 not handled correctly

Mon Jul 14 16:08:07 2008 SREZIC [...] cpan.org - Ticket created

Subject:

utf8 not handled correctly

Geo::Coder::Yahoo seems to not handle utf8 correctly. My test script use Data::Dumper; use Geo::Coder::Yahoo; my $geocoder = Geo::Coder::Yahoo->new(appid => 'geo-coder-yahoo-pm-test'); my $location = $geocoder->geocode(location => 'Berlin, Dudenstr. 24' ); print Data::Dumper->new([$location],[qw()])->Indent(1)->Useqq(1)->Dump; generates the following result: $VAR1 = [ { "country" => "DE", "longitude" => "13.381901", "state" => "Germany", "city" => "10965 Berlin", "zip" => "", "latitude" => "52.485176", "address" => "Dudenstra\303\237e 24", "precision" => "address" } ]; The resulting address should contain *one* unicode character (the sharp s, codepoint 0xdf). This is with perl5.10.0 on a FreeBSD machine. Regards, Slaven

Mon Jul 14 17:08:47 2008 SREZIC [...] cpan.org - Correspondence added

On Mon Jul 14 16:08:07 2008, SREZIC wrote: Show quoted text

> Geo::Coder::Yahoo seems to not handle utf8 correctly. [...]

Not only output is problematic, but also the incoming parameters. The correct street is found if using input parameters which are not having the utf8 flag on: use Data::Dumper; use Geo::Coder::Yahoo; my $geocoder = Geo::Coder::Yahoo->new(appid => 'geo-coder-yahoo-pm-test'); my $str = "Berlin, Dudenstra\xdfe 24"; #utf8::upgrade($str); my $location = $geocoder->geocode(location => $str); print Data::Dumper->new([$location],[qw()])->Indent(1)->Useqq(1)->Dump; If the utf8::upgrade call above is activated, then the result set is different and looks wrong. Regards, Slaven

Mon Jul 14 17:09:20 2008 SREZIC [...] cpan.org - Status changed from 'new' to 'open'

Wed Jul 16 13:43:58 2008 ask [...] perl.org - Correspondence added

Subject:	Re: [rt.cpan.org #37679] utf8 not handled correctly
Date:	Wed, 16 Jul 2008 10:40:36 -0700
To:	"bug-Geo-Coder-Yahoo [...] rt.cpan.org" <bug-Geo-Coder-Yahoo [...] rt.cpan.org>
From:	Ask Bjørn Hansen <ask [...] perl.org>

Funny - I as just working with this a few days ago. I will look at the output problem - it worked for me though. :-/. Any chance you could write a failing test case? There is git repository at git.develooper.com. For the input I think that is a problem with the Yahoo API actually. I normalized unicode to ASCII in my app to work around it, I haven't had time to figure out if that really is needed, but it seems like it. -- http://develooper.com On Jul 14, 2008, at 14:08, "Slaven_Rezic via RT" <bug-Geo-Coder-Yahoo@rt.cpan.org Show quoted text

> wrote:

Show quoted text

> Queue: Geo-Coder-Yahoo > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=37679 > > > On Mon Jul 14 16:08:07 2008, SREZIC wrote:

>> Geo::Coder::Yahoo seems to not handle utf8 correctly. [...]

> > Not only output is problematic, but also the incoming parameters. The > correct street is found if using input parameters which are not having > the utf8 flag on: > > use Data::Dumper; > use Geo::Coder::Yahoo; > my $geocoder = Geo::Coder::Yahoo->new(appid => 'geo-coder-yahoo-pm- > test'); > my $str = "Berlin, Dudenstra\xdfe 24"; > #utf8::upgrade($str); > my $location = $geocoder->geocode(location => $str); > print Data::Dumper->new([$location],[qw()])->Indent(1)->Useqq(1)-

> >Dump;

> > If the utf8::upgrade call above is activated, then the result set is > different and looks wrong. > > Regards, > Slaven

Wed Jul 16 14:07:22 2008 ask [...] perl.org - Correspondence added

Subject:	Re: [rt.cpan.org #37679] utf8 not handled correctly
Date:	Wed, 16 Jul 2008 10:40:36 -0700
To:	"bug-Geo-Coder-Yahoo [...] rt.cpan.org" <bug-Geo-Coder-Yahoo [...] rt.cpan.org>
From:	Ask Bjørn Hansen <ask [...] perl.org>

Funny - I as just working with this a few days ago. I will look at the output problem - it worked for me though. :-/. Any chance you could write a failing test case? There is git repository at git.develooper.com. For the input I think that is a problem with the Yahoo API actually. I normalized unicode to ASCII in my app to work around it, I haven't had time to figure out if that really is needed, but it seems like it. -- http://develooper.com On Jul 14, 2008, at 14:08, "Slaven_Rezic via RT" <bug-Geo-Coder-Yahoo@rt.cpan.org Show quoted text

> wrote:

Show quoted text

> Queue: Geo-Coder-Yahoo > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=37679 > > > On Mon Jul 14 16:08:07 2008, SREZIC wrote:

>> Geo::Coder::Yahoo seems to not handle utf8 correctly. [...]

> > Not only output is problematic, but also the incoming parameters. The > correct street is found if using input parameters which are not having > the utf8 flag on: > > use Data::Dumper; > use Geo::Coder::Yahoo; > my $geocoder = Geo::Coder::Yahoo->new(appid => 'geo-coder-yahoo-pm- > test'); > my $str = "Berlin, Dudenstra\xdfe 24"; > #utf8::upgrade($str); > my $location = $geocoder->geocode(location => $str); > print Data::Dumper->new([$location],[qw()])->Indent(1)->Useqq(1)-

> >Dump;

> > If the utf8::upgrade call above is activated, then the result set is > different and looks wrong. > > Regards, > Slaven

Wed Jul 16 18:38:25 2008 slaven [...] rezic.de - Correspondence added

Subject:	Re: [rt.cpan.org #37679] utf8 not handled correctly
Date:	17 Jul 2008 00:35:54 +0200
To:	bug-Geo-Coder-Yahoo [...] rt.cpan.org
From:	Slaven Rezic <slaven [...] rezic.de>

"ask@perl.org via RT" <bug-Geo-Coder-Yahoo@rt.cpan.org> writes: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=37679 > > > Funny - I as just working with this a few days ago. > > I will look at the output problem - it worked for me though. :-/. Any > chance you could write a failing test case? There is git repository > at git.develooper.com.

I made some changes which are available at git://bbbike.dyndns.org/Geo-Coder-Yahoo (be gentle, these are my first steps with git :-) (and be prepared that the machine is not always online) Regards, Slaven Show quoted text

> > For the input I think that is a problem with the Yahoo API actually. > I normalized unicode to ASCII in my app to work around it, I haven't > had time to figure out if that really is needed, but it seems like it. > > -- > http://develooper.com > > On Jul 14, 2008, at 14:08, "Slaven_Rezic via RT" <bug-Geo-Coder-Yahoo@rt.cpan.org

> > wrote:

>

> > Queue: Geo-Coder-Yahoo > > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=37679 > > > > > On Mon Jul 14 16:08:07 2008, SREZIC wrote:

> >> Geo::Coder::Yahoo seems to not handle utf8 correctly. [...]

> > > > Not only output is problematic, but also the incoming parameters. The > > correct street is found if using input parameters which are not having > > the utf8 flag on: > > > > use Data::Dumper; > > use Geo::Coder::Yahoo; > > my $geocoder = Geo::Coder::Yahoo->new(appid => 'geo-coder-yahoo-pm- > > test'); > > my $str = "Berlin, Dudenstra\xdfe 24"; > > #utf8::upgrade($str); > > my $location = $geocoder->geocode(location => $str); > > print Data::Dumper->new([$location],[qw()])->Indent(1)->Useqq(1)-

> > >Dump;

> > > > If the utf8::upgrade call above is activated, then the result set is > > different and looks wrong. > > > > Regards, > > Slaven

>

-- Slaven Rezic - slaven <at> rezic <dot> de tktimex - time recording tool http://sourceforge.net/projects/ptktools/

Thu Jul 17 02:50:43 2008 slaven [...] rezic.de - Correspondence added

Subject:	Re: [rt.cpan.org #37679] utf8 not handled correctly
Date:	17 Jul 2008 08:00:06 +0200
To:	bug-Geo-Coder-Yahoo [...] rt.cpan.org
From:	Slaven Rezic <slaven [...] rezic.de>

Slaven Rezic <slaven@rezic.de> writes: Show quoted text

> "ask@perl.org via RT" <bug-Geo-Coder-Yahoo@rt.cpan.org> writes: >

> > <URL: http://rt.cpan.org/Ticket/Display.html?id=37679 > > > > > Funny - I as just working with this a few days ago. > > > > I will look at the output problem - it worked for me though. :-/. Any > > chance you could write a failing test case? There is git repository > > at git.develooper.com.

> > I made some changes which are available at > git://bbbike.dyndns.org/Geo-Coder-Yahoo > (be gentle, these are my first steps with git :-) (and be prepared that > the machine is not always online)

If you tried it already: the firewall was not opened for the git daemon. So please try again. Regards, Slaven Show quoted text

> > Regards, > Slaven >

> > > > For the input I think that is a problem with the Yahoo API actually. > > I normalized unicode to ASCII in my app to work around it, I haven't > > had time to figure out if that really is needed, but it seems like it. > > > > -- > > http://develooper.com > > > > On Jul 14, 2008, at 14:08, "Slaven_Rezic via RT" <bug-Geo-Coder-Yahoo@rt.cpan.org

> > > wrote:

> >

> > > Queue: Geo-Coder-Yahoo > > > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=37679 > > > > > > > On Mon Jul 14 16:08:07 2008, SREZIC wrote:

> > >> Geo::Coder::Yahoo seems to not handle utf8 correctly. [...]

> > > > > > Not only output is problematic, but also the incoming parameters. The > > > correct street is found if using input parameters which are not having > > > the utf8 flag on: > > > > > > use Data::Dumper; > > > use Geo::Coder::Yahoo; > > > my $geocoder = Geo::Coder::Yahoo->new(appid => 'geo-coder-yahoo-pm- > > > test'); > > > my $str = "Berlin, Dudenstra\xdfe 24"; > > > #utf8::upgrade($str); > > > my $location = $geocoder->geocode(location => $str); > > > print Data::Dumper->new([$location],[qw()])->Indent(1)->Useqq(1)-

> > > >Dump;

> > > > > > If the utf8::upgrade call above is activated, then the result set is > > > different and looks wrong. > > > > > > Regards, > > > Slaven

> >

> > -- > Slaven Rezic - slaven <at> rezic <dot> de > > tktimex - time recording tool > http://sourceforge.net/projects/ptktools/ >

-- Slaven Rezic - slaven <at> rezic <dot> de BBBike - route planner for cyclists in Berlin WWW version: http://www.bbbike.de Perl/Tk version for Unix and Windows: http://bbbike.sourceforge.net

Thu Jul 17 06:28:47 2008 ask [...] perl.org - Correspondence added

Subject:	Re: [rt.cpan.org #37679] utf8 not handled correctly
Date:	Thu, 17 Jul 2008 03:28:25 -0700
To:	bug-Geo-Coder-Yahoo [...] rt.cpan.org
From:	Ask Bjørn Hansen <ask [...] perl.org>

On Jul 16, 2008, at 15:38, slaven@rezic.de via RT wrote: Show quoted text

> I made some changes which are available at > git://bbbike.dyndns.org/Geo-Coder-Yahoo > (be gentle, these are my first steps with git :-)

Looks good; thanks. I added another test and tagged and released 0.41. - ask

Thu Jul 17 06:34:28 2008 ABH [...] cpan.org - Correspondence added

http://git.develooper.com/?p=Geo-Coder-Yahoo.git;a=summary http://tmp.askask.com/2008/07/Geo-Coder-Yahoo-0.41.tar.gz

Thu Jul 17 06:34:56 2008 ABH [...] cpan.org - Status changed from 'open' to 'resolved'