Bug #132661 for LWP-Authen-OAuth2: Bug in LWP::Authen::OAuth2::AccessToken refresh code

Hi, Further observation: OAuth2.pm line 104 is the return line of the request subroutine: sub request { my ($self, $request, @rest) = @_; return $self->access_token->request($self, $request, @rest); } The error message on this line: "Can't call method "request" on unblessed reference" suggests that $self->access_token is no longer "blessed" once an old existing access_token has been replaced by a new one in refresh_access_token / _set_tokens. AccessToken.pm line 166 is if ($self->expires_in < $oauth2->access_token->expires_in) { In this case, the culprit seems to be that $oauth2->access_token is no longer "blessed" once an old existing access_token has been replaced by a new one in refresh_access_token / _set_tokens. The two cases therefore come down to the same underlying issue. Observation: In OAuth2.pm, when an initial $token_string is loaded via method load_token_string(), the critical line is Oauth2.pm line 320: $self->{access_token} = $class->from_ref($tokens); where the from_ref( ) function (in AccessToken.pm) actually blesses the $tokens hash references as being part of the LWP::Authen::OAuth2::AccessToken::Bearer class Also if prerefresh does not return anything (i.e. forces a refresh), refresh_access_token branches to ServiceProvider->refreshed_tokens, which also (in construct_tokens) does a "from_ref" (ServiceProvider.pm line 290). In the branch at fault, no such "from_ref" blessing occurs. I believe it should be inserted around OAuth2.pm line 221. More generally, it seems to me that the checks in load_token_string should also be applied to the possibly new token_string being returned from the prerefresh handler -- as that is a user function. Thus, my suggestion is to replace Oauth2.pm lines 219-221 with the following: if ($data and not $@) { # Assume I got it. # ----- Ruud 2020/05/21 insert (copied from load_token_string) ------- my $class = $data->{_class} or croak("No _class in token_string '$tokens'"); eval {load($class)}; if ($@) { croak("Can't load access token class '$class': $@"); } # Necessary: $class->from_ref blesses the new tokens as part of a class (e.g. LWP::Authen::OAuth2::AccessToken::Bearer) $tokens = $class->from_ref($data); # ----- Ruud 2020/05/21 end of insert ------- # $tokens = $data; # Ruud 2020/05/21 commented out, replaced by above lines I stepped through this in the debugger and it looks good -- it now also passes my testcases. I thereby look forward to running parallel jobs, using a single set of tokens, that is refreshed only as needed, with all the running jobs taking on the refreshed tokens. Respectfully submitting the above as a patch to OAuth2.pm. Thanks, Ruud. --------------------------- Ruud A. Haring, From: "Bugs in LWP-Authen-OAuth2 via RT" <bug-LWP-Authen-OAuth2@rt.cpan.org> To: ruud@us.ibm.com Date: 05/20/2020 01:17 Subject: [EXTERNAL] [rt.cpan.org #132661] AutoReply: Bug in LWP::Authen::OAuth2::AccessToken refresh code Greetings, This message has been automatically generated in response to the creation of a trouble ticket regarding: "Bug in LWP::Authen::OAuth2::AccessToken refresh code", a summary of which appears below. There is no need to reply to this message right now. Your ticket has been assigned an ID of [rt.cpan.org #132661]. Your ticket is accessible on the web at: https://urldefense.proofpoint.com/v2/url?u=https-3A__rt.cpan.org_Ticket_Display.html-3Fid-3D132661&d=DwIDaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=boUvkdeRe-_ytBHkjH2S7g&m=wz5_GQK5Q5pdx-vwFawphOqQZaTFRYXxaON8KAWPdx0&s=dQXX0-DfHnA7Q1GH8JkNOGQ-wYM3bohPJuZYZYUHcks&e= Please include the string: [rt.cpan.org #132661] in the subject line of all future correspondence about this issue. To do so, you may reply to this message. Thank you, bug-LWP-Authen-OAuth2@rt.cpan.org ------------------------------------------------------------------------- Hi, Have been using a Perl script based on LWP::Authen::OAuth2 for a long time to run nightly maintenance on data we have at an external service provider (Box.com). Works great -- thanks! Once initial authentication has been set up via web browser (i.e. manually), I use save_tokens to save off the $token_string to a file ( oauth2_tokens ). From that point onwards I have been running jobs automatically (cron jobs): each new job picks up the previous job's $token_string from the file, refreshes the tokens and saves the new $token_string off again. With the refresh_token being valid for up to 60 days, this scheme enables automated nightly runs -- but multiple jobs per night need to run sequentially: 1 job at a time. With workload increasing, I was intrigued to read in https://urldefense.proofpoint.com/v2/url?u=https-3A__metacpan.org_pod_LWP-3A-3AAuthen-3A-3AOAuth2&d=DwIDaQ&c=jf_iaSHvJObTbx-siA1ZOg&r=boUvkdeRe-_ytBHkjH2S7g&m=wz5_GQK5Q5pdx-vwFawphOqQZaTFRYXxaON8KAWPdx0&s=xL74G-rDm6KDKVjlomeN7rVyqD3FduC-xVF4DZTmod8&e= about the prerefresh handler: "A handler to be called before attempting to refresh tokens. It is passed the $oauth2 object. If it returns a token string, that will be used to generate tokens instead of going to the service provider. The purpose of this hook is so that, even if you have multiple processes accessing an API simultaneously, only one of them will try to refresh tokens with the service provider. (Service providers may dislike having multiple refresh requests arrive at once from the same consumer for the same user.)" This appears to enable token management for parallel operations of multiple jobs, sharing tokens as long as they are valid, and refreshing tokens only once for all parallel jobs, when the tokens are about to expire. My scenario: some long-running process A is starting with a fresh set of tokensA; some process B can run in parallel to A, re-using tokensA (if enough time before expiration); some process C starts later with not enough time before tokensA is about to expire. So it refreshes the tokens to tokensC; which immediately invalidates tokensA; process A (if still running) now stalls and needs to pick up the newest token set (tokensC). Note 1: The documentation does not describe that method $oauth2->refresh_access_token() is required to cause a call to prerefresh() -- if I need that in the program's initial refresh_tokens operation. My original call: $oauth2->request_tokens( grant_type => 'refresh_token', 'refresh_token' => $refresh_token ) bypassed prerefresh(). Note 2: The documentation does not describe how (with what parameters) the prerefresh() callback function is called I found that the following works: sub prerefresh { my ( $oauth2, $token_type, $current_refresh_token ) = @_; # $oauth2 object; $token_type='refresh_token' ... Note 3: If prerefresh() returns nothing, LWP::Authen::OAuth2 executes the token refresh with the service provider, and everything works fine. However, if prerefresh() (invoked by $oauth2->refresh_access_token() ) returns the JSON $token_string (straight from the save_tokens file) then OAuth2 crashes with: Can't call method "request" on unblessed reference at /usr/local/share/perl5/site_perl/5.26/LWP/Authen/OAuth2.pm line 104. Under another scenario ($oauth2->get() on process A running into the above token-invalidation by another process ) I got: Can't call method "expires_in" on unblessed reference at /usr/local/share/perl5/site_perl/5.26/LWP/Authen/OAuth2/AccessToken.pm line 166. I'm afraid that I can't lay my finger on what causes this unblessed reference to pop up, when prerefresh returns a token string. Any suggestions appreciated. Environment: Windows 10/Cygwin -- my debug env't (production is on AIX) perl 5, version 26, subversion 3 (v5.26.3) built for x86_64-cygwin-threads-multi LWP::Authen::OAuth2 : our $VERSION = '0.16'; LWP::Authen::OAuth2::AccessToken: our $VERSION = '0.02'; Thanks so much, Ruud. --------------------------- Ruud A. Haring, Principal Research Staff Member, IBM Thomas J. Watson Research Center, P.O. Box 218, Yorktown Heights, NY 10598, USA. phone: +1-914-945-2856 e-mail: ruud@us.ibm.com

Thomas, Thanks -- v0.17 works fine for me. Ruud.