Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the Net-Twitter CPAN distribution.

Report information
The Basics
Id: 58493
Status: resolved
Priority: 0/
Queue: Net-Twitter

People
Owner: MMIMS [...] cpan.org
Requestors: TURUGINA [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 3.13003
Fixed in: (no value)



Subject: upload() multi-byte status fails with Net::OAuth 0.26
since its version 0.26, Net::OAuth requires non-ASCII parameters well decoded to make signature. Net::Twitter now passes that kind of parameters in UTF-8 octets form to Net::OAuth. ( as a result, Net::OAuth makes invalid signature for non-ASCII parameters ) I attach a patch file to solve this problem. it is for lib/Net/Twiiter/Role/OAuth.pm and t/unicode.t . please check it. thx.
Subject: Net-Twitter_3.13003_for_Net-OAuth_0.26.diff
=== modified file 'lib/Net/Twitter/Role/OAuth.pm' --- lib/Net/Twitter/Role/OAuth.pm 2010-06-17 14:49:16 +0000 +++ lib/Net/Twitter/Role/OAuth.pm 2010-06-17 16:23:28 +0000 @@ -5,6 +5,7 @@ use URI; use Digest::SHA; use List::Util qw/first/; +use Encode qw/decode_utf8/; requires qw/_add_authorization_header ua/; @@ -167,7 +168,12 @@ my $uri = $msg->uri->clone; $uri->query($msg->content) if $msg->content_type eq 'application/x-www-form-urlencoded'; - my $args = { $uri->query_form }; + ## Net::OAuth 0.26 and above requires decoded string. + ## See perldoc Net::OAuth section I18N + my $args = + $Net::OAuth::VERSION ge '0.26' ? + { map { utf8::is_utf8($_) ? $_ : decode_utf8($_) } $uri->query_form } : + { $uri->query_form }; local $Net::OAuth::SKIP_UTF8_DOUBLE_ENCODE_CHECK = 1; === modified file 't/unicode.t' --- t/unicode.t 2010-06-17 14:49:16 +0000 +++ t/unicode.t 2010-06-17 16:24:51 +0000 @@ -9,7 +9,7 @@ eval "use LWP::UserAgent 5.819"; plan skip_all => 'LWP::UserAgent >= 5.819 required' if $@; -plan tests => 9; +plan tests => 11; my $req; my $ua = LWP::UserAgent->new; @@ -36,6 +36,16 @@ $nt->access_token('token'); $nt->access_token_secret('secret'); +my $meta = $nt->meta; +$meta->make_mutable; +$meta->add_around_method_modifier('_make_oauth_request', sub { + my ($orig, $self, $type, %args) = @_; + + ok utf8::is_utf8($args{extra_params}{status}), "status must be decoded"; + $self->$orig($type, %args); + }); +$meta->make_immutable; + # "Hello world!" in traditional Chinese if Google translate is correct my $status = "\x{4E16}\x{754C}\x{60A8}\x{597D}\x{FF01}";
On Thu Jun 17 12:45:46 2010, TURUGINA wrote: Show quoted text
> since its version 0.26, Net::OAuth requires non-ASCII parameters well > decoded to make signature. > > Net::Twitter now passes that kind of parameters in UTF-8 octets form to > Net::OAuth. ( as a result, Net::OAuth makes invalid signature for > non-ASCII parameters ) > > I attach a patch file to solve this problem. > it is for lib/Net/Twiiter/Role/OAuth.pm and t/unicode.t . > > please check it.
I can't claim to be a unicode expert, but I think the problem here is the caller should be passing in a decoded string, not relying on Net::Twitter to decode it. If you pass Net::Twitter an octet string that has not been decoded, it assumes (like Perl itself), that the string is latin-1. It does a utf8::upgrade on the string to handle that case. If your octet string is encoded in utf-8 (or any other encoding), you must decode it before passing it to Net::Twitter. Otherwise, how can Net::Twitter know what character set the octet string's encoding? I'm going to ask a unicode expert to review this bug and comment. Thanks for the report!
Subject: Re: [rt.cpan.org #58493] upload() multi-byte status fails with Net::OAuth 0.26
Date: Fri, 18 Jun 2010 11:06:57 -0700
To: bug-Net-Twitter [...] rt.cpan.org
From: Tatsuhiko Miyagawa <miyagawa [...] gmail.com>
On Fri, Jun 18, 2010 at 10:59 AM, Marc Mims via RT <bug-Net-Twitter@rt.cpan.org> wrote: Show quoted text
>> >> I attach a patch file to solve this problem. >> it is for lib/Net/Twiiter/Role/OAuth.pm and t/unicode.t . >> >> please check it.
> > I can't claim to be a unicode expert, but I think the problem here is > the caller should be passing in a decoded string, not relying on > Net::Twitter to decode it.
I think I can claim to be a unicode expert :) and i'm +1 on it. Checking the utf8 flag in the module code is ALWAYS wrong. See perldoc perlunitut (that comes with perl 5.10.1 or available on perldoc.org). Show quoted text
> If you pass Net::Twitter an octet string that has not been decoded, it > assumes (like Perl itself), that the string is latin-1.  It does a > utf8::upgrade on the string to handle that case. > > If your octet string is encoded in utf-8 (or any other encoding), you > must decode it before passing it to Net::Twitter.  Otherwise, how can > Net::Twitter know what character set the octet string's encoding? > > I'm going to ask a unicode expert to review this bug and comment. > > Thanks for the report! >
-- Tatsuhiko Miyagawa
RT-Send-CC: miyagawa [...] gmail.com
Ah! There *is* a bug. Although originally decoded, arguments lost their decoded state when extracted from the request in Net::Twitter::Role::OAuth::_add_authorization_header. I broke it in a recent update to use OAuth authentication headers instead for oauth in body params. Apologies. The fix is easy and actually simplifies the code. I'll ship 3.13004 to CPAN very soon. -Marc
Fixed in 3.13005, just shipped to CPAN. Mostly fixed in 3.13004, shipped a few hours ago. However, that version would not handle update_profile_image and update_profile_background_image correctly. 3.13005 also includes an unrelated fix to the synthetic "since" arg filtering.