Bug #35173 for Geo-Coder-Google: Google returns non-utf8 unless the user-agent says Mozilla

Fri Apr 18 01:38:05 2008 DROLSKY [...] cpan.org - Ticket created

Subject:

Google returns non-utf8 unless the user-agent says Mozilla

Google, in their apparently infinite wisdom, return non-utf8 data unless the user agent looks like Firefox or IE (or maybe others). The user-agent string this module is using doesn't cut it. A good example is this URI - http://maps.google.com/maps/geo?q=B%C4%9Blohorsk%C3%A1+80%2C+6%2C+Czech+Republic&output=json&key=[your key] The canonical address _should_ come back as "Bělohorská 80, 169 00 Praha 6, Praha, Czech Republic" but without messing with the user agent, the e with a caron (ě) comes back as a space! Thanks, Google. The fix is to just lie about the user-agent, like this: Mozilla/5.0 (compatible; Geo::Coder::Google/0.03; Google, please stop smoking crack)

Mon Sep 07 07:02:54 2009 SREZIC [...] cpan.org - Correspondence added

On Fri Apr 18 01:38:05 2008, DROLSKY wrote: Show quoted text

> Google, in their apparently infinite wisdom, return non-utf8 data > unless > the user agent looks like Firefox or IE (or maybe others). The > user-agent string this module is using doesn't cut it. > > A good example is this URI - >

http://maps.google.com/maps/geo?q=B%C4%9Blohorsk%C3%A1+80%2C+6%2C+Czech+Republic&output=json&key=[your Show quoted text

> key] > > The canonical address _should_ come back as "Bělohorská 80, 169 00 > Praha > 6, Praha, Czech Republic" but without messing with the user agent, the > e > with a caron (ě) comes back as a space! > > Thanks, Google. > > The fix is to just lie about the user-agent, like this: > > Mozilla/5.0 (compatible; Geo::Coder::Google/0.03; Google, please stop > smoking crack)

And thanks to the already available "ua" method, one can fix it without changing the module: my $google = Geo::Coder::Google->new(apikey => $apikey); $google->ua->agent("Mozilla/5.0 (compatible; Geo::Coder::Google/$Geo::Coder::Google::VERSION; Google, please stop smoking crack; http://rt.cpan.org/Public/Bug/Display.html?id=35173)"); Regards, Slaven

Mon Sep 07 07:02:55 2009 The RT System itself - Status changed from 'new' to 'open'

Mon Sep 07 10:36:46 2009 DROLSKY [...] cpan.org - Correspondence added

On Mon Sep 07 07:02:54 2009, SREZIC wrote: Show quoted text

> On Fri Apr 18 01:38:05 2008, DROLSKY wrote:

> > Google, in their apparently infinite wisdom, return non-utf8 data > > unless > > the user agent looks like Firefox or IE (or maybe others). The > > user-agent string this module is using doesn't cut it. > > > > A good example is this URI - > >

http://maps.google.com/maps/geo?q=B%C4%9Blohorsk%C3%A1+80%2C+6%2C+Czech+Republic&output=json&key=[your Show quoted text

> > key] > > > > The canonical address _should_ come back as "Bělohorská 80, 169 00 > > Praha > > 6, Praha, Czech Republic" but without messing with the user agent,

> the

> > e > > with a caron (ě) comes back as a space! > > > > Thanks, Google. > > > > The fix is to just lie about the user-agent, like this: > > > > Mozilla/5.0 (compatible; Geo::Coder::Google/0.03; Google, please

> stop

> > smoking crack)

> > And thanks to the already available "ua" method, one can fix it > without > changing the module: > > my $google = Geo::Coder::Google->new(apikey => $apikey); > $google->ua->agent("Mozilla/5.0 (compatible; > Geo::Coder::Google/$Geo::Coder::Google::VERSION; Google, please stop > smoking crack; http://rt.cpan.org/Public/Bug/Display.html?id=35173)");

Yeah, but if you fix it in the module directly then people don't have to figure this out. It's completely undocumented by Google, and the module doesn't mention this at all either. At the very least, the module docs could mention this.

Mon Sep 07 10:45:29 2009 miyagawa [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #35173] Google returns non-utf8 unless the user-agent says Mozilla
Date:	Mon, 7 Sep 2009 23:45:10 +0900
To:	bug-Geo-Coder-Google [...] rt.cpan.org
From:	Tatsuhiko Miyagawa <miyagawa [...] gmail.com>

http://github.com/miyagawa/geo-coder-google/tree/master patch/fork please? Thank you. On Mon, Sep 7, 2009 at 11:36 PM, DROLSKY via RT<bug-Geo-Coder-Google@rt.cpan.org> wrote: Show quoted text

> Queue: Geo-Coder-Google > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=35173 > > > On Mon Sep 07 07:02:54 2009, SREZIC wrote:

>> On Fri Apr 18 01:38:05 2008, DROLSKY wrote:

>> > Google, in their apparently infinite wisdom, return non-utf8 data >> > unless >> > the user agent looks like Firefox or IE (or maybe others). The >> > user-agent string this module is using doesn't cut it. >> > >> > A good example is this URI - >> >

> http://maps.google.com/maps/geo?q=B%C4%9Blohorsk%C3%A1+80%2C+6%2C+Czech+Republic&output=json&key=[your

>> > key] >> > >> > The canonical address _should_ come back as "Bělohorská 80, 169 00 >> > Praha >> > 6, Praha, Czech Republic" but without messing with the user agent,

>> the

>> > e >> > with a caron (ě) comes back as a space! >> > >> > Thanks, Google. >> > >> > The fix is to just lie about the user-agent, like this: >> > >> > Mozilla/5.0 (compatible; Geo::Coder::Google/0.03; Google, please

>> stop

>> > smoking crack)

>> >> And thanks to the already available "ua" method, one can fix it >> without >> changing the module: >> >> my $google = Geo::Coder::Google->new(apikey => $apikey); >> $google->ua->agent("Mozilla/5.0 (compatible; >> Geo::Coder::Google/$Geo::Coder::Google::VERSION; Google, please stop >> smoking crack; http://rt.cpan.org/Public/Bug/Display.html?id=35173)");

> > Yeah, but if you fix it in the module directly then people don't have to > figure this out. It's completely undocumented by Google, and the module > doesn't mention this at all either. > > At the very least, the module docs could mention this. > >

-- Tatsuhiko Miyagawa