Skip Menu |

This queue is for tickets about the Net-Twitter-OAuth CPAN distribution.

Report information
The Basics
Id: 46423
Status: rejected
Priority: 0/
Queue: Net-Twitter-OAuth

People
Owner: MMIMS [...] cpan.org
Requestors: chris [...] oak-wood.co.uk
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 0.05
Fixed in: (no value)



Subject: Status updates fail when 'wide' characters are used
When a status update is posted that includes 'wide' characters, authorisation fails. The following exception is thrown: post on http://twitter.com/statuses/update.json?oauth_timestamp=1243456768&oauth_consumer_key=QZ2pIVQF2G6JllVkOFTPw&status=Another+test+status+4+%C3%A9%C2%A6%C2%96%C3%A9%C2%83%C2%BD%C3%A5%C2%9C%C2%8F&oauth_signature=QYLE3Zzml2ZYY77wMPgAHcBXM1w%3D&oauth_nonce=2610890340&oauth_token=42584122-14QBiEvv7kY0C1UYBAj3hW9sKXyAFEJGlXr3u66ib&oauth_version=1.0&oauth_signature_method=HMAC-SHA1 failed: 401 Unauthorized at /usr/local/lib/perl5/site_perl/5.8.8/Net/OAuth/Simple.pm line 567 Twitter is returning "Failed to validate oauth signature or token". I'm guessing this may be because the post is signed before it is URL encoded. Perl v5.8.8 built for amd64-freebsd Net::Twitter::OAuth 0.5
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Wed, 27 May 2009 15:19:09 -0700
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
Good catch, but the module (Net::OAuth::Message) gets those parameters and then encode them using URI::uri_escape_utf8 so it actually should work against those wide characters as well. How do you pass those characters, in UTF8 encoded bytes, or decoded strings? On Wed, May 27, 2009 at 1:57 PM, Chris Hastie via RT <bug-Net-Twitter-OAuth@rt.cpan.org> wrote: Show quoted text
> Wed May 27 16:57:32 2009: Request 46423 was acted upon. > Transaction: Ticket created by tipichris >       Queue: Net-Twitter-OAuth >     Subject: Status updates fail when 'wide' characters are used >   Broken in: 0.05 >    Severity: Important >       Owner: Nobody >  Requestors: chris@oak-wood.co.uk >      Status: new >  Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=46423 > > > > When a status update is posted that includes 'wide' characters, > authorisation fails. The following exception is thrown: > > post on > http://twitter.com/statuses/update.json?oauth_timestamp=1243456768&oauth_consumer_key=QZ2pIVQF2G6JllVkOFTPw&status=Another+test+status+4+%C3%A9%C2%A6%C2%96%C3%A9%C2%83%C2%BD%C3%A5%C2%9C%C2%8F&oauth_signature=QYLE3Zzml2ZYY77wMPgAHcBXM1w%3D&oauth_nonce=2610890340&oauth_token=42584122-14QBiEvv7kY0C1UYBAj3hW9sKXyAFEJGlXr3u66ib&oauth_version=1.0&oauth_signature_method=HMAC-SHA1 > failed: 401 Unauthorized at > /usr/local/lib/perl5/site_perl/5.8.8/Net/OAuth/Simple.pm line 567 > > Twitter is returning "Failed to validate oauth signature or token". > > I'm guessing this may be because the post is signed before it is URL > encoded. > > Perl v5.8.8 built for amd64-freebsd > Net::Twitter::OAuth 0.5 >
-- Tatsuhiko Miyagawa
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Wed, 27 May 2009 15:24:05 -0700
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
Show quoted text
Actually, your status message is encoded like this: Another+test+status+4+%C3%A9%C2%A6%C2%96%C3%A9%C2%83%C2%BD%C3%A5%C2%9C%C2%8F&oauth_signature= Looks at those %C3 and %C2 in the head: they're double-encoded to UTF-8 assuming it was originally latin-1. Show quoted text
> print uri_unescape "Another+test+status+4+%C3%A9%C2%A6%C2%96%C3%A9%C2%83%C2%BD%C3%A5%C2%9C%C2%8F"
Another+test+status+4+é¦é½å Show quoted text
> print Encode::decode_utf8 uri_unescape "Another+test+status+4+%C3%A9%C2%A6%C2%96%C3%A9%C2%83%C2%BD%C3%A5%C2%9C%C2%8F"
Another+test+status+4+首都圏 Can you try passing them as a decoded strings rather than UTF-8 bytes? I'm not sure if Net::Twitter itself barfs on it with un-encoded strings though: my guess is it should just work. -- Tatsuhiko Miyagawa
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Wed, 27 May 2009 15:32:48 -0700
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
Yes, decoding strings before passing to ->update just worked: http://twitter.com/miyagawa/status/1940376104 May 27, 2009 at 3:24 PM, Tatsuhiko Miyagawa <miyagawa@gmail.com> wrote: Show quoted text
> Can you try passing them as a decoded strings rather than UTF-8 bytes? > I'm not sure if Net::Twitter itself barfs on it with un-encoded > strings though: my guess is it should just work.
-- Tatsuhiko Miyagawa
On Wed May 27 18:33:16 2009, miyagawa@gmail.com wrote: Show quoted text
> Yes, decoding strings before passing to ->update just worked: > http://twitter.com/miyagawa/status/1940376104 > > May 27, 2009 at 3:24 PM, Tatsuhiko Miyagawa <miyagawa@gmail.com> wrote:
> > Can you try passing them as a decoded strings rather than UTF-8 bytes? > > I'm not sure if Net::Twitter itself barfs on it with un-encoded > > strings though: my guess is it should just work.
>
I originally started by encoding to UTF-8 because that is the way I have successfully used Net::Twitter $status = "A test status 6 £"; $status = encode('UTF-8', $status); $twitresult = $twit->update({status => $status}); which gives post on http://twitter.com/statuses/update.json?oauth_timestamp=1243497296&oauth_consumer_key=QZ2pIVQF2G6JllVkOFTPw&status=A+test+status+6+%C3%82%C2%A3&oauth_signature=oZi6fe%2BTDK0G2L68v8neNzlJqwI%3D&oauth_nonce=425647448&oauth_token=36725782-Lgb3nAJIm8fo6gd2Evf6qdcmT9zNFOyjNnSWhbMOu&oauth_version=1.0&oauth_signature_method=HMAC-SHA1 failed: 401 Unauthorized at /usr/local/lib/perl5/site_perl/5.8.8/Net/OAuth/Simple.pm line 567. removing the 'encode' $status = "A test status 6 £"; # $status = encode('UTF-8', $status); $twitresult = $twit->update({status => $status}); gives http://twitter.com/statuses/update.json?oauth_timestamp=1243497345&oauth_consumer_key=QZ2pIVQF2G6JllVkOFTPw&status=A+test+status+6+%C2%A3&oauth_signature=yKA%2FIoIz1eI3o%2Fgo3%2BZfkyEVDWM%3D&oauth_nonce=2803819267&oauth_token=36725782-Lgb3nAJIm8fo6gd2Evf6qdcmT9zNFOyjNnSWhbMOu&oauth_version=1.0&oauth_signature_method=HMAC-SHA1 failed: 401 Unauthorized at /usr/local/lib/perl5/site_perl/5.8.8/Net/OAuth/Simple.pm line 567. whilst $status = "A test status 6 £"; $status = decode('utf8', $status); $twitresult = $twit->update({status => $status}); gives post on http://twitter.com/statuses/update.json?oauth_timestamp=1243498290&oauth_consumer_key=QZ2pIVQF2G6JllVkOFTPw&status=A+test+status+6+%A3&oauth_signature=lrXbVsk3YE4VMVHtKEP2n7qlp20%3D&oauth_nonce=1775320411&oauth_token=36725782-Lgb3nAJIm8fo6gd2Evf6qdcmT9zNFOyjNnSWhbMOu&oauth_version=1.0&oauth_signature_method=HMAC-SHA1 failed: 401 Unauthorized at /usr/local/lib/perl5/site_perl/5.8.8/Net/OAuth/Simple.pm line 567. Loosing the '£' works fine.
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Thu, 28 May 2009 01:29:00 -0700
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
Does your script have "use utf8" in the head? If not, do that, and try utf8::upgrade($status) since URI::Escape::uri_escape_utf8() would probably encodes it as latin-1. On Thu, May 28, 2009 at 1:13 AM, Chris Hastie via RT <bug-Net-Twitter-OAuth@rt.cpan.org> wrote: Show quoted text
>       Queue: Net-Twitter-OAuth >  Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=46423 > > > On Wed May 27 18:33:16 2009, miyagawa@gmail.com wrote:
>> Yes, decoding strings before passing to ->update just worked: >> http://twitter.com/miyagawa/status/1940376104 >> >> May 27, 2009 at 3:24 PM, Tatsuhiko Miyagawa <miyagawa@gmail.com> wrote:
>> > Can you try passing them as a decoded strings rather than UTF-8 bytes? >> > I'm not sure if Net::Twitter itself barfs on it with un-encoded >> > strings though: my guess is it should just work.
>>
> > I originally started by encoding to UTF-8 because that is the way I have > successfully used Net::Twitter > > $status = "A test status 6 £"; > $status = encode('UTF-8', $status); > $twitresult = $twit->update({status => $status}); > > which gives > post on > http://twitter.com/statuses/update.json?oauth_timestamp=1243497296&oauth_consumer_key=QZ2pIVQF2G6JllVkOFTPw&status=A+test+status+6+%C3%82%C2%A3&oauth_signature=oZi6fe%2BTDK0G2L68v8neNzlJqwI%3D&oauth_nonce=425647448&oauth_token=36725782-Lgb3nAJIm8fo6gd2Evf6qdcmT9zNFOyjNnSWhbMOu&oauth_version=1.0&oauth_signature_method=HMAC-SHA1 > failed: 401 Unauthorized at > /usr/local/lib/perl5/site_perl/5.8.8/Net/OAuth/Simple.pm line 567. > > removing the 'encode' > > $status = "A test status 6 £"; > # $status = encode('UTF-8', $status); > $twitresult = $twit->update({status => $status}); > > gives > > http://twitter.com/statuses/update.json?oauth_timestamp=1243497345&oauth_consumer_key=QZ2pIVQF2G6JllVkOFTPw&status=A+test+status+6+%C2%A3&oauth_signature=yKA%2FIoIz1eI3o%2Fgo3%2BZfkyEVDWM%3D&oauth_nonce=2803819267&oauth_token=36725782-Lgb3nAJIm8fo6gd2Evf6qdcmT9zNFOyjNnSWhbMOu&oauth_version=1.0&oauth_signature_method=HMAC-SHA1 > failed: 401 Unauthorized at > /usr/local/lib/perl5/site_perl/5.8.8/Net/OAuth/Simple.pm line 567. > > whilst > $status = "A test status 6 £"; > $status = decode('utf8', $status); > $twitresult = $twit->update({status => $status}); > > gives > post on > http://twitter.com/statuses/update.json?oauth_timestamp=1243498290&oauth_consumer_key=QZ2pIVQF2G6JllVkOFTPw&status=A+test+status+6+%A3&oauth_signature=lrXbVsk3YE4VMVHtKEP2n7qlp20%3D&oauth_nonce=1775320411&oauth_token=36725782-Lgb3nAJIm8fo6gd2Evf6qdcmT9zNFOyjNnSWhbMOu&oauth_version=1.0&oauth_signature_method=HMAC-SHA1 > failed: 401 Unauthorized at > /usr/local/lib/perl5/site_perl/5.8.8/Net/OAuth/Simple.pm line 567. > > Loosing the '£' works fine. > >
-- Tatsuhiko Miyagawa
On Thu May 28 04:29:20 2009, miyagawa@gmail.com wrote: Show quoted text
> Does your script have "use utf8" in the head? If not, do that, and try > utf8::upgrade($status) since URI::Escape::uri_escape_utf8() would > probably encodes it as latin-1. >
No joy. Similar to the decode option earlier: post on http://twitter.com/statuses/update.json?oauth_timestamp=1243499882&oauth_consumer_key=QZ2pIVQF2G6JllVkOFTPw&status=A+test+status+6+%A3&oauth_signature=RDUqpxk4HZWZE31E44mCrOmPWqc%3D&oauth_nonce=1983972719&oauth_token=36725782-Lgb3nAJIm8fo6gd2Evf6qdcmT9zNFOyjNnSWhbMOu&oauth_version=1.0&oauth_signature_method=HMAC-SHA1 failed: 401 Unauthorized at /usr/local/lib/perl5/site_perl/5.8.8/Net/OAuth/Simple.pm line 567. It's worth pointing out the '£' was only an example. The characters in the original post certainly wouldn't have been latin-1: 首都圏
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Thu, 28 May 2009 01:51:45 -0700
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
On Thu, May 28, 2009 at 1:41 AM, Chris Hastie via RT <bug-Net-Twitter-OAuth@rt.cpan.org> wrote: Show quoted text
> > No joy. Similar to the decode option earlier: > > post on > http://twitter.com/statuses/update.json?oauth_timestamp=1243499882&oauth_consumer_key=QZ2pIVQF2G6JllVkOFTPw&status=A+test+status+6+%A3&oauth_signature=RDUqpxk4HZWZE31E44mCrOmPWqc%3D&oauth_nonce=1983972719&oauth_token=36725782-Lgb3nAJIm8fo6gd2Evf6qdcmT9zNFOyjNnSWhbMOu&oauth_version=1.0&oauth_signature_method=HMAC-SHA1 > failed: 401 Unauthorized at > /usr/local/lib/perl5/site_perl/5.8.8/Net/OAuth/Simple.pm line 567. > > It's worth pointing out the '£' was only an example. The characters in > the original post certainly wouldn't have been latin-1: 首都圏
Do they also fail? I see £ is differnent from 首都圏 because £ is now encoded in %A3 (as latin-1) instead of %C2%A3, its UTF-8 representation. Anyway, try eg/tweet-hello.pl (the latest from github) on UTF-8 aware console: Works for me: http://twitter.com/miyagawa/status/1945384384 http://twitter.com/miyagawa/status/1945384384 -- Tatsuhiko Miyagawa
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Thu, 28 May 2009 21:09:59 +0100
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Chris Hastie <chris [...] oak-wood.co.uk>
On 28/05/09 09:52, miyagawa@gmail.com via RT wrote: Show quoted text
>> It's worth pointing out the '£' was only an example. The characters in >> the original post certainly wouldn't have been latin-1: 首都圏
> > Do they also fail? I see £ is differnent from 首都圏 because £ is now > encoded in %A3 (as latin-1) instead of %C2%A3, its UTF-8 > representation.
Yes. I've also tried getting characters in a way more allied to how they are likely to turn up in the app for real: $status = HTML::Entities::decode("A test status 6 &#44592;&#47084;&#44592;"); Show quoted text
> Anyway, try eg/tweet-hello.pl (the latest from github) on UTF-8 aware console: > Works for me: http://twitter.com/miyagawa/status/1945384384
The place I looked at github appears to be empty. Am I missing something? http://github.com/miyagawa/net-twitter-oauth/tree/master BTW, the other side of me trying to fathom this out is at http://code.google.com/p/twitter-api/issues/detail?id=624 Thanks for your help.
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Thu, 28 May 2009 13:26:01 -0700
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
For the fuck's sake Github broke repos with mixed case. I guess you can still check it with git://github.com/miyagawa/net-twitter-oauth.git On Thu, May 28, 2009 at 1:10 PM, Chris Hastie via RT <bug-Net-Twitter-OAuth@rt.cpan.org> wrote: Show quoted text
> Queue: Net-Twitter-OAuth > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=46423 > > > On 28/05/09 09:52, miyagawa@gmail.com via RT wrote: >
>>> It's worth pointing out the '£' was only an example. The characters in >>> the original post certainly wouldn't have been latin-1: 首都圏
>> >> Do they also fail? I see £ is differnent from 首都圏 because £ is now >> encoded in %A3 (as latin-1) instead of %C2%A3, its UTF-8 >> representation.
> > Yes. I've also tried getting characters in a way more allied to how they > are likely to turn up in the app for real: > > $status = HTML::Entities::decode("A test status 6 > &#44592;&#47084;&#44592;"); > >
>> Anyway, try eg/tweet-hello.pl (the latest from github) on UTF-8 aware console: >> Works for me: http://twitter.com/miyagawa/status/1945384384
> > The place I looked at github appears to be empty. Am I missing something? > http://github.com/miyagawa/net-twitter-oauth/tree/master > > BTW, the other side of me trying to fathom this out is at > http://code.google.com/p/twitter-api/issues/detail?id=624 > > Thanks for your help. > > >
-- Tatsuhiko Miyagawa
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Thu, 28 May 2009 13:32:12 -0700
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
wanted to push the new version to CPAN and PAUSE is down, so here's the full copy: #!/usr/bin/perl use strict; use FindBin; use lib "$FindBin::Bin/../lib"; use Net::Twitter::OAuth; my $client = Net::Twitter::OAuth->new( consumer_key => $ENV{TWITTER_CONSUMER_KEY}, consumer_secret => $ENV{TWITTER_CONSUMER_SECRET}, ); unless ($client->oauth->authorized) { # The client is not yet authorized: Do it now print "Authorize this app at ", $client->oauth->get_authorization_url, " and hit RET\n"; system "open", $client->oauth->get_authorization_url; <STDIN>; # wait for input $client->oauth->request_access_token; } binmode STDOUT, ":utf8"; my $tweets = $client->friends_timeline; for my $tweet (@$tweets) { printf "%s: %s (%s)\n", $tweet->{user}{name}, $tweet->{text}, $tweet->{created_at}; } print "\nSomething to say: "; binmode STDIN, ":encoding(utf-8)"; chomp(my $tweet = <STDIN>); my $res = $client->update({ status => $tweet }); print "Tweeted: http://twitter.com/$res->{user}{screen_name}/status/$res->{id}\n"; 2009/5/28 Tatsuhiko Miyagawa <miyagawa@gmail.com>: Show quoted text
> For the fuck's sake Github broke repos with mixed case. I guess you > can still check it with > git://github.com/miyagawa/net-twitter-oauth.git > > On Thu, May 28, 2009 at 1:10 PM, Chris Hastie via RT > <bug-Net-Twitter-OAuth@rt.cpan.org> wrote:
>> Queue: Net-Twitter-OAuth >> Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=46423 > >> >> On 28/05/09 09:52, miyagawa@gmail.com via RT wrote: >>
>>>> It's worth pointing out the '£' was only an example. The characters in >>>> the original post certainly wouldn't have been latin-1: 首都圏
>>> >>> Do they also fail? I see £ is differnent from 首都圏 because £ is now >>> encoded in %A3 (as latin-1) instead of %C2%A3, its UTF-8 >>> representation.
>> >> Yes. I've also tried getting characters in a way more allied to how they >> are likely to turn up in the app for real: >> >> $status = HTML::Entities::decode("A test status 6 >> &#44592;&#47084;&#44592;"); >> >>
>>> Anyway, try eg/tweet-hello.pl (the latest from github) on UTF-8 aware console: >>> Works for me: http://twitter.com/miyagawa/status/1945384384
>> >> The place I looked at github appears to be empty. Am I missing something? >> http://github.com/miyagawa/net-twitter-oauth/tree/master >> >> BTW, the other side of me trying to fathom this out is at >> http://code.google.com/p/twitter-api/issues/detail?id=624 >> >> Thanks for your help. >> >> >>
> > > > -- > Tatsuhiko Miyagawa >
-- Tatsuhiko Miyagawa
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Thu, 28 May 2009 13:35:17 -0700
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
On Thu, May 28, 2009 at 1:10 PM, Chris Hastie via RT <bug-Net-Twitter-OAuth@rt.cpan.org> wrote: Show quoted text
> > Yes. I've also tried getting characters in a way more allied to how they > are likely to turn up in the app for real: > > $status = HTML::Entities::decode("A test status 6 > &#44592;&#47084;&#44592;");
Works fine for me (assuming they're Korean Hangul characters): http://twitter.com/miyagawa/status/1951462759 Here's the full code: #!/usr/bin/perl use strict; use FindBin; use lib "$FindBin::Bin/../lib"; use HTML::Entities; use Net::Twitter::OAuth; my $client = Net::Twitter::OAuth->new( consumer_key => $ENV{TWITTER_CONSUMER_KEY}, consumer_secret => $ENV{TWITTER_CONSUMER_SECRET}, ); # The client is not yet authorized: Do it now print "Authorize this app at ", $client->oauth->get_authorization_url, " and hit RET\n"; system "open", $client->oauth->get_authorization_url; <STDIN>; # wait for input $client->oauth->request_access_token; my $tweet = HTML::Entities::decode("A test status 6&#44592;&#47084;&#44592;"); my $res = $client->update({ status => $tweet }); print "Tweeted: http://twitter.com/$res->{user}{screen_name}/status/$res->{id}\n"; -- Tatsuhiko Miyagawa
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Thu, 28 May 2009 22:36:31 +0100
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Chris Hastie <chris [...] oak-wood.co.uk>
On 28/05/09 21:35, miyagawa@gmail.com via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=46423 > > > On Thu, May 28, 2009 at 1:10 PM, Chris Hastie via RT > <bug-Net-Twitter-OAuth@rt.cpan.org> wrote:
>> Yes. I've also tried getting characters in a way more allied to how they >> are likely to turn up in the app for real: >> >> $status = HTML::Entities::decode("A test status 6 >> &#44592;&#47084;&#44592;");
> > Works fine for me (assuming they're Korean Hangul characters): > http://twitter.com/miyagawa/status/1951462759
Ho hum, seems like our mileage differs, as they say. I have managed to get things to work with this patch to Net/OAuth/Message.pm --- Message.pm.orig Thu May 28 20:27:21 2009 +++ Message.pm Thu May 28 20:27:32 2009 @@ -86,7 +86,7 @@ sub encode { my $str = shift; $str = "" unless defined $str; - return URI::Escape::uri_escape_utf8($str,'^\w.~-'); + return URI::Escape::uri_escape($str,'^\w.~-'); } sub decode { With this applied, I need to do an encode('UTF-8', $status) to get things to work. I've tried various things, encoding or not, patched or not, and watched the value of $request_url being hit in Net::OAuth::Simple::_make_request, and the value of $request->signature_base_string Here's a summary - I've only included the relevant parts Works: patched, encoded status=A+test+status+6+%EA%B8%B0%EB%9F%AC%EA%B8%B0 %26status%3DA%2520test%2520status%25206%2520%25EA%25B8%25B0%25EB%259F%25AC%25EA%25B8%25B0 Unpatched, unencoded &status=A+test+status+6+ %26status%3DA%2520test%2520status%25206%2520%25EA%25B8%25B0%25EB%259F%25AC%25EA%25B8%25B0 Unpatched, encoded &status=A+test+status+6+%EA%B8%B0%EB%9F%AC%EA%B8%B0 %26status%3DA%2520test%2520status%25206%2520%25C3%25AA%25C2%25B8%25C2%25B0%25C3%25AB%25C2%259F%25C2%25AC%25C3%25AA%25C2%25B8%25C2%25B0 You'll see that compared to what eventually worked, on an unpatched code I get the right signature_base_string if I don't encode, but the request_url has the wide characters stripped. To get the request URL right I have to encode, but that does something odd to signature_base_string.
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Thu, 28 May 2009 15:02:47 -0700
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
On Thu, May 28, 2009 at 2:36 PM, Chris Hastie via RT <bug-Net-Twitter-OAuth@rt.cpan.org> wrote: Show quoted text
>>> $status = HTML::Entities::decode("A test status 6 >>> &#44592;&#47084;&#44592;");
>> >> Works fine for me (assuming they're Korean Hangul characters): >> http://twitter.com/miyagawa/status/1951462759
> > Ho hum, seems like our mileage differs, as they say. > > I have managed to get things to work with this patch to Net/OAuth/Message.pm
I guess we should probably make sure we use the same versions (unpatched) in the tool chain. I'm using perl 5.8.8 on OSX with these modules: strict 1.03 FindBin 1.47 Carp 1.04 Exporter 5.58 Cwd 3.3 vars 1.01 warnings::register 1.01 warnings 1.05 XSLoader 0.06 Config (undefined) File::Basename 2.74 re 0.05 File::Spec 3.3 File::Spec::Unix 3.3 lib 0.5565 base 2.07 Net::Twitter 2.12 URI::Escape 3.29 JSON::Any 1.19 constant 1.05 JSON::XS 2.232 overload 1.04 attributes 0.06 LWP::UserAgent 5.826 HTTP::Request 5.818 HTTP::Message 5.826 HTTP::Headers 5.822 Storable 2.18 AutoLoader 5.68 FileHandle 2.01 IO::File 1.14 Symbol 1.06 SelectSaver 1.01 IO::Seekable 1.1 IO::Handle 1.27 IO 1.23 Fcntl 1.05 Exporter::Heavy 5.58 DynaLoader 1.05 Log::Agent (undefined) URI 1.37 HTTP::Response 5.824 HTTP::Status 5.817 HTTP::Date 5.810 Time::Local 1.11 integer 1.00 LWP 5.826 LWP::Protocol 5.826 LWP::MemberMixin (undefined) Net::OAuth::Simple 1.1 CGI 3.43 CGI::Util 1.5_01 Math::Random::MT (undefined) Data::Dumper 2.121_08 bytes 1.02 Net::OAuth::Request 0.14 Net::OAuth::Message -1, set by base.pm Class::Data::Inheritable 0.08 Class::Accessor 0.33 Sub::Name 0.04 UNIVERSAL::require 0.13 UNIVERSAL 1.01 Net::OAuth::RequestTokenRequest (undefined) Net::OAuth::AccessTokenRequest -1, set by base.pm Net::OAuth::ProtectedResourceRequest (undefined) HTTP::Config 5.815 URI::http (undefined) URI::_server (undefined) URI::_generic (undefined) URI::_query (undefined) Net::OAuth::SignatureMethod::HMAC_SHA1 (undefined) Digest::HMAC_SHA1 1.01 Digest::SHA1 2.12 Digest::base 1.00 Digest::HMAC 1.01 MIME::Base64 3.07 HTTP::Request::Common 5.824 LWP::Protocol::http (undefined) Errno 1.0901 Net::HTTP 5.819 IO::Socket::INET 1.31 IO::Socket 1.30 Socket 1.78 IO::Socket::UNIX 1.23 Net::HTTP::Methods 5.824 IO::Select 1.17 Compress::Zlib 2.019 Scalar::Util 1.21 List::Util 1.21 IO::Compress::Base::Common 2.019 File::GlobMapper 1.000 File::Glob 1.05 utf8 1.06 Compress::Raw::Zlib 2.019 IO::Compress::Gzip 2.019 IO::Compress::RawDeflate 2.019 IO::Compress::Base 2.019 IO::Compress::Adapter::Deflate 2.019 IO::Compress::Gzip::Constants 2.019 IO::Compress::Zlib::Extra 2.019 IO::Uncompress::Gunzip 2.019 IO::Uncompress::RawInflate 2.019 IO::Uncompress::Base 2.019 IO::Uncompress::Adapter::Inflate 2.019 HTML::HeadParser 3.60 HTML::Entities 3.60 HTML::Parser 3.60 I got this output with this http://gist.github.com/119608 saved as Versions.pm and run the script with perl -MVersions -- Tatsuhiko Miyagawa
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Wed, 03 Jun 2009 20:10:37 +0100
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Chris Hastie <chris [...] oak-wood.co.uk>
On 28/05/09 23:03, miyagawa@gmail.com via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=46423 > > > On Thu, May 28, 2009 at 2:36 PM, Chris Hastie via RT > <bug-Net-Twitter-OAuth@rt.cpan.org> wrote:
>>>> $status = HTML::Entities::decode("A test status 6 >>>> &#44592;&#47084;&#44592;");
>>> Works fine for me (assuming they're Korean Hangul characters): >>> http://twitter.com/miyagawa/status/1951462759
>> Ho hum, seems like our mileage differs, as they say. >> >> I have managed to get things to work with this patch to Net/OAuth/Message.pm
> > I guess we should probably make sure we use the same versions > (unpatched) in the tool chain.
OK, sorry for the delay responding. FreeBSD 6.2-RELEASE amd64 perl, v5.8.8 built for amd64-freebsd HTML::Entities 1.35 strict 1.03 vars 1.01 warnings::register 1.01 warnings 1.05 Carp 1.04 Exporter 5.58 HTML::Parser 3.56 XSLoader 0.06 Encode 2.26 base 2.07 Encode::Alias 2.10 bytes 1.02 Encode::Config 2.05 Encode::ConfigLocal (undefined) Encode::Encoding 2.05 Net::Twitter::OAuth 0.05 Net::Twitter 2.12 URI::Escape 3.28 JSON::Any 1.19 constant 1.05 JSON::XS (undefined) JSON 2.14 overload 1.04 JSON::XS (undefined) JSON::PP 2.22010 B 1.09_01 JSON::PP58 1.02 Scalar::Util 1.19 List::Util 1.19 LWP::UserAgent 2.036 HTTP::Request 1.40 HTTP::Message 1.57 HTTP::Headers 1.64 URI 1.35 HTTP::Response 1.53 HTTP::Status 1.28 HTTP::Date 1.47 Time::Local 1.17 Config (undefined) BSDPAN (undefined) BSDPAN::Override (undefined) Config (undefined) Exporter::Heavy 5.58 integer 1.00 LWP 5.808 LWP::Debug (undefined) LWP::Protocol 1.46 LWP::MemberMixin (undefined) Net::OAuth::Simple 1.1 CGI 3.37 CGI::Util 1.5_01 Math::Random::MT (undefined) Data::Dumper 2.121_08 Net::OAuth::Request 0.14 Net::OAuth::Message -1, set by base.pm Class::Data::Inheritable 0.08 Class::Accessor 0.33 Sub::Name (undefined) UNIVERSAL::require 0.13 UNIVERSAL 1.01 Net::OAuth::RequestTokenRequest (undefined) Net::OAuth::AccessTokenRequest -1, set by base.pm Net::OAuth::ProtectedResourceRequest (undefined) Net::Twitter::OAuth::UserAgent (undefined) URI::http (undefined) URI::_server (undefined) URI::_generic (undefined) URI::_query (undefined) Net::OAuth::SignatureMethod::HMAC_SHA1 (undefined) Digest::HMAC_SHA1 1.01 Digest::SHA1 2.11 DynaLoader 1.05 AutoLoader 5.60 Digest::base 1.00 Digest::HMAC 1.01 MIME::Base64 3.07 HTTP::Request::Common 1.28 LWP::Protocol::http (undefined) Net::HTTP 1.00 IO::Socket::INET 1.29 IO::Socket 1.29 IO::Handle 1.25 Symbol 1.06 SelectSaver 1.01 IO 1.22 Socket 1.78 Errno 1.0901 IO::Socket::UNIX 1.22 Net::HTTP::Methods 1.02 IO::Select 1.17 Compress::Zlib 2.012 IO::Compress::Base::Common 2.012 File::GlobMapper 1.000 File::Glob 1.05 utf8 1.06 Compress::Raw::Zlib 2.012 IO::Compress::Gzip 2.012 IO::Compress::RawDeflate 2.012 IO::Compress::Base 2.012 IO::File 1.13 IO::Seekable 1.1 Fcntl 1.05 File::Spec 3.25 File::Spec::Unix 1.5 IO::Compress::Adapter::Deflate 2.012 IO::Compress::Gzip::Constants 2.012 IO::Compress::Zlib::Extra 2.012 IO::Uncompress::Gunzip 2.012 IO::Uncompress::RawInflate 2.012 IO::Uncompress::Base 2.012 IO::Uncompress::Adapter::Inflate 2.012 -- Chris Hastie
Subject: Re: [rt.cpan.org #46423] Status updates fail when 'wide' characters are used
Date: Wed, 3 Jun 2009 12:26:11 -0700
To: bug-Net-Twitter-OAuth [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
It's "your" turn to make sure everything's up-to-date, and while you're away we shipped Net::Twitter 3.0 with OAuth roles bundled. On Wed, Jun 3, 2009 at 12:11 PM, Chris Hastie via RT <bug-Net-Twitter-OAuth@rt.cpan.org> wrote: Show quoted text
>       Queue: Net-Twitter-OAuth >  Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=46423 > > > On 28/05/09 23:03, miyagawa@gmail.com via RT wrote:
>> <URL: https://rt.cpan.org/Ticket/Display.html?id=46423 > >> >> On Thu, May 28, 2009 at 2:36 PM, Chris Hastie via RT >> <bug-Net-Twitter-OAuth@rt.cpan.org> wrote:
>>>>> $status = HTML::Entities::decode("A test status 6 >>>>> &#44592;&#4708O4;&#44592;");
>>>> Works fine for me (assuming they're Korean Hangul characters): >>>> http://twitter.com/miyagawa/status/1951462759
>>> Ho hum, seems like our mileage differs, as they say. >>> >>> I have managed to get things to work with this patch to Net/OAuth/Message.pm
>> >> I guess we should probably make sure we use the same versions >> (unpatched) in the tool chain.
> > OK, sorry for the delay responding. > > FreeBSD 6.2-RELEASE amd64 > perl, v5.8.8 built for amd64-freebsd > > HTML::Entities 1.35 > strict 1.03 > vars 1.01 > warnings::register 1.01 > warnings 1.05 > Carp 1.04 > Exporter 5.58 > HTML::Parser 3.56 > XSLoader 0.06 > Encode 2.26 > base 2.07 > Encode::Alias 2.10 > bytes 1.02 > Encode::Config 2.05 > Encode::ConfigLocal (undefined) > Encode::Encoding 2.05 > Net::Twitter::OAuth 0.05 > Net::Twitter 2.12 > URI::Escape 3.28 > JSON::Any 1.19 > constant 1.05 > JSON::XS (undefined) > JSON 2.14 > overload 1.04 > JSON::XS (undefined) > JSON::PP 2.22010 > B 1.09_01 > JSON::PP58 1.02 > Scalar::Util 1.19 > List::Util 1.19 > LWP::UserAgent 2.036 > HTTP::Request 1.40 > HTTP::Message 1.57 > HTTP::Headers 1.64 > URI 1.35 > HTTP::Response 1.53 > HTTP::Status 1.28 > HTTP::Date 1.47 > Time::Local 1.17 > Config (undefined) > BSDPAN (undefined) > BSDPAN::Override (undefined) > Config (undefined) > Exporter::Heavy 5.58 > integer 1.00 > LWP 5.808 > LWP::Debug (undefined) > LWP::Protocol 1.46 > LWP::MemberMixin (undefined) > Net::OAuth::Simple 1.1 > CGI 3.37 > CGI::Util 1.5_01 > Math::Random::MT (undefined) > Data::Dumper 2.121_08 > Net::OAuth::Request 0.14 > Net::OAuth::Message -1, set by base.pm > Class::Data::Inheritable 0.08 > Class::Accessor 0.33 > Sub::Name (undefined) > UNIVERSAL::require 0.13 > UNIVERSAL 1.01 > Net::OAuth::RequestTokenRequest (undefined) > Net::OAuth::AccessTokenRequest -1, set by base.pm > Net::OAuth::ProtectedResourceRequest (undefined) > Net::Twitter::OAuth::UserAgent (undefined) > URI::http (undefined) > URI::_server (undefined) > URI::_generic (undefined) > URI::_query (undefined) > Net::OAuth::SignatureMethod::HMAC_SHA1 (undefined) > Digest::HMAC_SHA1 1.01 > Digest::SHA1 2.11 > DynaLoader 1.05 > AutoLoader 5.60 > Digest::base 1.00 > Digest::HMAC 1.01 > MIME::Base64 3.07 > HTTP::Request::Common 1.28 > LWP::Protocol::http (undefined) > Net::HTTP 1.00 > IO::Socket::INET 1.29 > IO::Socket 1.29 > IO::Handle 1.25 > Symbol 1.06 > SelectSaver 1.01 > IO 1.22 > Socket 1.78 > Errno 1.0901 > IO::Socket::UNIX 1.22 > Net::HTTP::Methods 1.02 > IO::Select 1.17 > Compress::Zlib 2.012 > IO::Compress::Base::Common 2.012 > File::GlobMapper 1.000 > File::Glob 1.05 > utf8 1.06 > Compress::Raw::Zlib 2.012 > IO::Compress::Gzip 2.012 > IO::Compress::RawDeflate 2.012 > IO::Compress::Base 2.012 > IO::File 1.13 > IO::Seekable 1.1 > Fcntl 1.05 > File::Spec 3.25 > File::Spec::Unix 1.5 > IO::Compress::Adapter::Deflate 2.012 > IO::Compress::Gzip::Constants 2.012 > IO::Compress::Zlib::Extra 2.012 > IO::Uncompress::Gunzip 2.012 > IO::Uncompress::RawInflate 2.012 > IO::Uncompress::Base 2.012 > IO::Uncompress::Adapter::Inflate 2.012 > > > > > > > > -- > Chris Hastie > >
-- Tatsuhiko Miyagawa
The conversation on this bug ended over a year ago. Net::Twitter has received a great deal use in the field with wide characters. I think it's safe to assume this issue is resolved. -Marc