Bug #58829 for Net-Twitter: UTF8

Fri Jun 25 06:53:28 2010 jps [...] burda.com - Ticket created

Subject:	UTF8 - signature failure
Date:	Fri, 25 Jun 2010 12:53:15 +0200
To:	bug-Net-Twitter [...] rt.cpan.org
From:	Jean Paul Schmetz <jps [...] burda.com>

Hi, I have a working application using Net::Twitter v.3.13006 (thank you for that: adapting to oauth was a walk in the park) I have a problem posting a status update with real UTF-8 character e.g. posting 배우 권상우 씨 뺑소니 혐의 입건 will generate an "Incorrect signature" whereas posting iPhone 4 Teardown - iFixit will succesfully go through The string tests as true on both: $flag1 = utf8::is_utf8($mytext); $flag2 = utf8::valid($mytext); and prints correctly back to the web site (set as utf8) so the string should be OK. This is the feedback string I get to see on the site: MY TEST: 1-1 Something bad happened while tweeting 배우 권상우 씨 뺑소니 혐의 입건 . Please Try later: Incorrect signature basically indicates that my utf8 flags are both 1 and 1, that something went wrong (ie. my $result = eval {$twitterapi->update($mytext)}; generated an error). Print the string back and gives me $@ I noticed that you posted an update a few days ago re: this topic but somehow it seems still broken unless I did something wrong somewhere best, Jean Paul Schmetz

Fri Jun 25 10:42:05 2010 marc [...] questright.com - Correspondence added

Subject:	Re: [rt.cpan.org #58829] UTF8 - signature failure
Date:	Fri, 25 Jun 2010 07:41:48 -0700
To:	Jean Paul Schmetz via RT <bug-Net-Twitter [...] rt.cpan.org>
From:	Marc Mims <marc [...] questright.com>

* Jean Paul Schmetz via RT <bug-Net-Twitter@rt.cpan.org> [100625 03:53]: Show quoted text

> I have a working application using Net::Twitter v.3.13006 (thank you for > that: adapting to oauth was a walk in the park) > > I have a problem posting a status update with real UTF-8 character e.g. > posting > > 배우 권상우 씨 뺑소니 혐의 입건 > > will generate an "Incorrect signature" whereas posting > > iPhone 4 Teardown - iFixit > > will succesfully go through > > The string tests as true on both: > > $flag1 = utf8::is_utf8($mytext); > $flag2 = utf8::valid($mytext); > > and prints correctly back to the web site (set as utf8) so the string > should be OK. > > This is the feedback string I get to see on the site: > > MY TEST: 1-1 > Something bad happened while tweeting > 배우 권상우 씨 뺑소니 혐의 입건 > . Please Try later: > Incorrect signature > > > basically indicates that my utf8 flags are both 1 and 1, that something > went wrong (ie. my $result = eval {$twitterapi->update($mytext)}; > generated an error). Print the string back and gives me $@ > > I noticed that you posted an update a few days ago re: this topic but > somehow it seems still broken unless I did something wrong somewhere

I have not been able to reproduce the problem. I posted this status message with the attached script: http://twitter.com/semifor_test/status/17018282331 Will you please run the attached script and report back? Either populate ENV with your OAuth credentials or edit the Net::Twitter constructor to include them. -Marc

Message body is not shown because sender requested not to inline it.

Fri Jun 25 10:42:07 2010 The RT System itself - Status changed from 'new' to 'open'

Fri Jun 25 12:57:59 2010 jps [...] burda.com - Correspondence added

Subject:	Re: [rt.cpan.org #58829] UTF8 - signature failure
Date:	Fri, 25 Jun 2010 18:57:44 +0200
To:	bug-Net-Twitter [...] rt.cpan.org
From:	Jean Paul Schmetz <jps [...] burda.com>

I have identified what cause the failure in my code: if you add: use encoding 'utf8'; to your code (which I have in mine) unicode.pl returns with the signature error. I have to check why we have the line in our code but I think it is necessary to avoid a bunch of "wide Character' issues (e.g. your script would cause one by simple 'print "$utf8_string\n";'. hope this helps jp On 6/25/10 4:42 PM, marc@questright.com via RT wrote: Show quoted text

> my $nt = Net::Twitter->new( > traits => [qw/API::REST OAuth/], > consumer_key => $ENV{TWITTER_CONSUMER_KEY}, > consumer_secret => $ENV{TWITTER_CONSUMER_SECRET}, > access_token => $ENV{TWITTER_ACCESS_TOKEN}, > access_token_secret => $ENV{TWITTER_ACCESS_TOKEN_SECRET}, > );

Fri Jun 25 14:02:19 2010 marc [...] questright.com - Correspondence added

Subject:	Re: [rt.cpan.org #58829] UTF8 - signature failure
Date:	Fri, 25 Jun 2010 11:02:03 -0700
To:	Jean Paul Schmetz via RT <bug-Net-Twitter [...] rt.cpan.org>
From:	Marc Mims <marc [...] questright.com>

* Jean Paul Schmetz via RT <bug-Net-Twitter@rt.cpan.org> [100625 09:58]: Show quoted text

> Queue: Net-Twitter > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=58829 > > > I have identified what cause the failure in my code: > > if you add: > > use encoding 'utf8'; > > to your code (which I have in mine) unicode.pl returns with the > signature error. > > I have to check why we have the line in our code but I think it is > necessary to avoid a bunch of "wide Character' issues (e.g. your script > would cause one by simple 'print "$utf8_string\n";'. > > hope this helps

Net::OAuth calls URI::Escape::uri_escape_utf8($str,'^\w.~-') on the status. That behaves differently depending upon whether "use encoding 'utf8'" is present, or not. See the attached script. I don't know if this is a bug in your code (i.e., is the use of the encoding pragma correct?), a bug in Net::OAuth, or a bug in URI::Escape. I'll see if we can get some assistance with this. -Marc

Message body is not shown because sender requested not to inline it.

Fri Jun 25 14:10:43 2010 miyagawa [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #58829] UTF8 - signature failure
Date:	Fri, 25 Jun 2010 11:10:29 -0700
To:	bug-Net-Twitter [...] rt.cpan.org
From:	Tatsuhiko Miyagawa <miyagawa [...] gmail.com>

On Fri, Jun 25, 2010 at 9:58 AM, Jean Paul Schmetz via RT <bug-Net-Twitter@rt.cpan.org> wrote: Show quoted text

> > I have identified what cause the failure in my code: > > if you add: > > use encoding 'utf8'; > > to your code (which I have in mine) unicode.pl returns with the > signature error. > > I have to check why we have the line in our code but I think it is > necessary to avoid a bunch of "wide Character' issues (e.g. your script > would cause one by simple 'print "$utf8_string\n";'.

Don't use 'use encoding' - it has a bunch of nasty side effects. If you want to encode STDOUT when you print decoded strings, then do so with binmode. There's also encoding::stdio and encoding::source module available for perl 5.10+ which is exactly there to solve the side-effect problem of encoding pragma. HTH.

Fri Jun 25 14:27:26 2010 MMIMS [...] cpan.org - Correspondence added

RT-Send-CC:

miyagawa [...] gmail.com

miyagawa++ # I owe you another beer :) We'll blame pragma encoding and close this one.

Fri Jun 25 14:27:27 2010 MMIMS [...] cpan.org - Status changed from 'open' to 'resolved'

Bug #58829 for Net-Twitter: UTF8 - signature failure

Preferred bug tracker