Bug #93913 for Net-Google-Drive-Simple: truncated downloads

Sun Mar 16 22:05:35 2014 fjjmarques [...] free.fr - Ticket created

Subject:	truncated downloads
Date:	Mon, 17 Mar 2014 03:05:14 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

Hello, When trying the "file-download" script included in a distribution of "net-google-drive" cloned from GitHub ("simple.pm" indicates version 0.08), all of 20 downloaded files where truncated to approx. 5 kb. No error was reported. I ran a test download of an mp3 file with curl using the url returned by a "simple.pm" based script and the auth. token from the .yml file generated by the "google-drive-init" script. The download was successful, suggesting that the problem lies in the "simple.pm" download "sub". OS: Windows XP-sp3 perl version: Srawberry perl 5.18. --- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Mon Mar 17 01:08:36 2014 MSCHILLI [...] cpan.org - Correspondence added

Net::Google::Drive::Simple's download() method uses LWP::UserAgent's request() method to obtain the data. I wonder if your version of it does the right thing for Win32, which is set the file descriptor to write the response data to to binmode(). Can you check that your version of LWP::Protocol has a collect() method that uses bindmode( $fh ) after opening the response file?

Mon Mar 17 01:08:36 2014 The RT System itself - Status changed from 'new' to 'open'

Sun Mar 23 02:03:31 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Sun, 23 Mar 2014 07:03:16 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

Being very new to perl, I'm not sure to be able to answer your question without further investigation. However, according to my perldoc and to what I see in LWP::Protocol::file.pm, there indeed seems to be such a binmode. Tell me if you need further details. On 17/03/2014 06:08, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > Net::Google::Drive::Simple's download() method uses LWP::UserAgent's request() method to obtain the data. I wonder if your version of it does the right thing for Win32, which is set the file descriptor to write the response data to to binmode(). Can you check that your version of LWP::Protocol has a collect() method that uses bindmode( $fh ) after opening the response file?

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Sun Mar 23 16:30:16 2014 MSCHILLI [...] cpan.org - Correspondence added

On Sun Mar 23 02:03:31 2014, fjjmarques@free.fr wrote: Show quoted text

> there indeed seems to be such a > binmode. Tell me if you need further details.

You can test this out yourself with a snippet of code like #!/usr/local/bin/perl -w use strict; use LWP::UserAgent; use HTTP::Request::Common; my $ua = LWP::UserAgent->new(); my $req = GET "http://news.yahoo.com"; my $resp = $ua->request( $req, "test" ); print $resp->code(), "\n"; If the test file "test" after this is less than about 2K in size, the problem is with LWP::UserAgent.

Sun Mar 23 16:36:52 2014 MSCHILLI [...] cpan.org - Correspondence added

On Sun Mar 23 16:30:16 2014, MSCHILLI wrote: Show quoted text

> my $req = GET "http://news.yahoo.com";

Actually, since your files seem to be truncated at 5.5k, try something bigger, like http://files.dlink.com.au/Products/DP-301U/Manuals/DP-301U_B2_Manual_v3.10.pdf which has 5MB.

Mon Mar 24 15:58:03 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Mon, 24 Mar 2014 20:57:46 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

Thank you for the test snippet. I ran it and got a 5313 ko "test" pdf file. On 23/03/2014 21:36, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Sun Mar 23 16:30:16 2014, MSCHILLI wrote:

>> my $req = GET "http://news.yahoo.com";

> Actually, since your files seem to be truncated at 5.5k, try something bigger, like > > http://files.dlink.com.au/Products/DP-301U/Manuals/DP-301U_B2_Manual_v3.10.pdf > > which has 5MB.

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Mon Mar 24 22:31:43 2014 MSCHILLI [...] cpan.org - Correspondence added

Closing because it seems to be caused by LWP::UserAgent, not Net::Google::Drive::Simple.

Mon Mar 24 22:31:43 2014 MSCHILLI [...] cpan.org - Status changed from 'open' to 'resolved'

Tue Mar 25 02:47:57 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Tue, 25 Mar 2014 07:47:42 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

I don't understand. It seems to me that the test you provided to me proved exactly the contrary. On 25/03/2014 03:31, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > Closing because it seems to be caused by LWP::UserAgent, not Net::Google::Drive::Simple.

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Tue Mar 25 02:58:37 2014 MSCHILLI [...] cpan.org - Correspondence added

On Mon Mar 24 15:58:03 2014, fjjmarques@free.fr wrote: Show quoted text

> Thank you for the test snippet. I ran it and got a 5313 ko "test" pdf > file.

I interpreted the above to say that you downloaded the 5MB file and got 5k. Is this not the case?

Tue Mar 25 02:58:37 2014 MSCHILLI [...] cpan.org - Status changed from 'resolved' to 'open'

Tue Mar 25 03:07:59 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Tue, 25 Mar 2014 08:07:46 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

No, as mentioned before the file I received was 5313 kilobytes in size, not 5313 bytes. On 25/03/2014 07:58, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Mon Mar 24 15:58:03 2014, fjjmarques@free.fr wrote:

>> Thank you for the test snippet. I ran it and got a 5313 ko "test" pdf >> file.

> I interpreted the above to say that you downloaded the 5MB file and got 5k. Is this not the case?

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Tue Mar 25 03:11:38 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Tue, 25 Mar 2014 08:11:13 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

I forgot to add that the downloded file was a readable pdf and was not corrupted in any way, to make things perfectly clear. On 25/03/2014 07:58, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Mon Mar 24 15:58:03 2014, fjjmarques@free.fr wrote:

>> Thank you for the test snippet. I ran it and got a 5313 ko "test" pdf >> file.

> I interpreted the above to say that you downloaded the 5MB file and got 5k. Is this not the case?

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Tue Mar 25 15:51:56 2014 MSCHILLI [...] cpan.org - Correspondence added

On Tue Mar 25 03:07:59 2014, fjjmarques@free.fr wrote: Show quoted text

> No, as mentioned before the file I received was 5313 kilobytes in > size,

Oh, I see "and got a 5313 ko 'test' pdf file" means 5MB :). In this case, it seems LWP works OK on your platform, but for some reason Google Drive is not giving you what you expect. Are you sure those files are intact on the server? Have you used and verified them using a different client?

Tue Mar 25 15:55:10 2014 MSCHILLI [...] cpan.org - Correspondence added

Show quoted text

> In this case, it seems LWP works OK on your platform

Also, can you run your Simple.pm script on a different platform (like Mac or Linux) to verify that your platform or perl core isn't the problem?

Tue Mar 25 17:37:11 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Tue, 25 Mar 2014 22:36:54 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

I don't think this would solve the problem in any way. I have no doubt that Simple.pm does run as expected in a given context, at the very least on that particular *nix distribution, with this specific set of perl core and accessory modules versions. However what I would like is to see Simple.pm run on *my* platform. Would it be be simpler to adapt your script (and maybe some other module) to make it work in my context or the reverse? The analogy may not be very well chosen, but if a take say a pair of shoes that are not fit to me, are my feet the problem or are the shoes a problem? Of course it depends on the point of view. IMHO, that's where lies the whole difficulty of defining what a "bug" is. Of course I'm not demanding anything from you. I'm just a computer enthusiast trying to learn a bit about programming and I'm just beginning to see how big the "compatibility problem" is. On 25/03/2014 20:55, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > >

>> In this case, it seems LWP works OK on your platform

> Also, can you run your Simple.pm script on a different platform (like Mac or Linux) to verify that your platform or perl core isn't the problem? > >

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Tue Mar 25 17:41:13 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Tue, 25 Mar 2014 22:41:00 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

I forgot to add that, as mentioned on my first post, I did check that my files were intact using curl, among other ways. Simple.pm did provide me the download links and a valid access token that I passed to curl. I'll try to pass them to LWP as soon as I figure out how to use a custom header with LWP. On 25/03/2014 20:51, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Tue Mar 25 03:07:59 2014, fjjmarques@free.fr wrote:

>> No, as mentioned before the file I received was 5313 kilobytes in >> size,

> Oh, I see "and got a 5313 ko 'test' pdf file" means 5MB :). > > In this case, it seems LWP works OK on your platform, but for some reason Google Drive is not giving you what you expect. Are you sure those files are intact on the server? Have you used and verified them using a different client?

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Tue Mar 25 21:53:26 2014 MSCHILLI [...] cpan.org - Correspondence added

On Tue Mar 25 17:37:11 2014, fjjmarques@free.fr wrote: Show quoted text

> I don't think this would solve the problem in any way. I have no doubt > that Simple.pm does run as expected in a given context

You're mistaken, I just wanted to verify if the problem is platform/installation specific, if it has to do with your google account, or if the adapted script (would you mind posting it, btw with the secrets removed?) is doing anything unsupported. I have no way of accessing your Google account, don't have access to your platform, and therefore can't reproduce your problem. You trying it on a different platform would rule out several possible issues. Show quoted text

> The analogy may not be very well chosen, but if a take say a pair of > shoes that are not fit to me, are my feet the problem or are the shoes

It depends, what if you wanted to wear ski boots on the beach? :)

Wed Mar 26 00:09:27 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Wed, 26 Mar 2014 05:09:12 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

I' m quite sure there is no problem regarding the files I wanted to download. What I did was the following: 1 - run the "google-dirive-init" script included in the "net-google-drive/eg" folder I cloned from Github. This created the yaml file from which I took a (fresh) access_token. I got the download urls from the attached "geturls" script, a simple copy and paste from Simple.pm. 2 - download the files with curl as follows: curl -Sv -H "Authorization: Bearer $access_token" "https://doc-0o-bs-docs.googleusercontent.com/docs/securesc/r1o99735rvf8kooaubt896qcull11dft/0lkhuka7gsmbnpqi4j7o4431ucgsbnk3/1395014400000/17167261938173040669/17167261938173040669/0B_NQoUcaLnLleVlvSlkxYkxsTm8?h=16653014193614665626&e=download&gd=true" -o "outfile.mp3" The downloaded files were not corrupted, showing that the access_token was valid and the files well kept on the server side. However, the files I first got after running the (attached) "file-download" script - also included in the "eg" folder - were all truncated. I first thought they might contain some meta-data rather than the actual audio, but this was not the case. They were just aborted downloads, which is indeed rather weird. I repeated the test twice with the same result. I have only one computer and no access to a Mac platform, however I do have a pair of Linux distributions (Ubuntu and Mint) on bootable usb keys. I'll try Simple.pm in these conditions in the next hours. One question, if I may: on what platform(s) and with wich perl version did you test Simple.pm? On 26/03/2014 02:53, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Tue Mar 25 17:37:11 2014, fjjmarques@free.fr wrote:

>> I don't think this would solve the problem in any way. I have no doubt >> that Simple.pm does run as expected in a given context

> You're mistaken, I just wanted to verify if the problem is platform/installation specific, if it has to do with your google account, or if the adapted script (would you mind posting it, btw with the secrets removed?) is doing anything unsupported. > > I have no way of accessing your Google account, don't have access to your platform, and therefore can't reproduce your problem. You trying it on a different platform would rule out several possible issues. >

>> The analogy may not be very well chosen, but if a take say a pair of >> shoes that are not fit to me, are my feet the problem or are the shoes

> It depends, what if you wanted to wear ski boots on the beach? :) > >

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Message body is not shown because sender requested not to inline it.

Wed Mar 26 15:35:11 2014 MSCHILLI [...] cpan.org - Correspondence added

On Wed Mar 26 00:09:27 2014, fjjmarques@free.fr wrote: Show quoted text

> 2 - download the files with curl as follows: > curl -Sv -H "Authorization: Bearer $access_token"

So, the question is, what is different in the request object between the curl call and the LWP::UserAgent call? The latter you can find out by adding "$DB::single = 1;" to Simple.pm like this: my $req = HTTP::Request->new( GET => $url, HTTP::Headers->new( Authorization => "Bearer " . $self->{ cfg }->{ access_token }) ); $DB::single = 1; my $ua = LWP::UserAgent->new(); my $resp = $ua->request( $req, $local_file ); and then run the script in the debugger and print out the request object: perl -d scriptname DB<1> c xyz.pdf ([y]/n) [y]> y Downloading lme-2013-12-snp.pdf Net::Google::Drive::Simple::download(lib/Net/Google/Drive/Simple.pm:455): 455: my $ua = LWP::UserAgent->new(); DB<1> x $req 0 HTTP::Request=HASH(0x7ffd6ab8d318) '_content' => '' '_headers' => HTTP::Headers=HASH(0x7ffd6ab8ea48) ... Another thing you might want to try is use the download() method without a filename in the script: Replace my $c = $gd->download( $file, $name ); by my $data = $gd->download( $file ); and then print print "data length: ", length( $data ), "\n"; to figure out how many bytes have been downloaded.

Wed Mar 26 15:36:49 2014 MSCHILLI [...] cpan.org - Correspondence added

On Wed Mar 26 00:09:27 2014, fjjmarques@free.fr wrote: Show quoted text

> One question, if I may: on what platform(s) and with wich perl version > did you test Simple.pm?

It's in use on various platforms, and personally I've tested it on Linux and Mac OSX with perl-5.16.

Wed Mar 26 19:34:53 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Wed, 26 Mar 2014 23:34:35 +0000
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

I haven't tried your latest suggestions yet, but I did a quick test of the "file-download" script with Perl 5.14.2 on Linux Mint 16 Petra, running "live" from an usb key. For the moment I get: Failed with 400: Bad Request at /usr/local/share/perl/5.14.2/Net/Google/Drive/Simple.pm line 529. No luck! On 26/03/2014 19:35, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Wed Mar 26 00:09:27 2014, fjjmarques@free.fr wrote:

>> 2 - download the files with curl as follows: >> curl -Sv -H "Authorization: Bearer $access_token"

> So, the question is, what is different in the request object between the curl call and the LWP::UserAgent call? > > The latter you can find out by adding "$DB::single = 1;" to Simple.pm like this: > > my $req = HTTP::Request->new( > GET => $url, > HTTP::Headers->new( Authorization => > "Bearer " . $self->{ cfg }->{ access_token }) > ); > > $DB::single = 1; > > my $ua = LWP::UserAgent->new(); > my $resp = $ua->request( $req, $local_file ); > > and then run the script in the debugger and print out the request object: > > perl -d scriptname > DB<1> c > xyz.pdf ([y]/n) [y]> y > Downloading lme-2013-12-snp.pdf > Net::Google::Drive::Simple::download(lib/Net/Google/Drive/Simple.pm:455): > 455: my $ua = LWP::UserAgent->new(); > DB<1> x $req > 0 HTTP::Request=HASH(0x7ffd6ab8d318) > '_content' => '' > '_headers' => HTTP::Headers=HASH(0x7ffd6ab8ea48) > ... > > Another thing you might want to try is use the download() method without a filename in the script: Replace > > my $c = $gd->download( $file, $name ); > > by > > my $data = $gd->download( $file ); > > and then print > > print "data length: ", length( $data ), "\n"; > > to figure out how many bytes have been downloaded.

Wed Mar 26 19:52:58 2014 MSCHILLI [...] cpan.org - Correspondence added

On Wed Mar 26 19:34:53 2014, fjjmarques@free.fr wrote: Show quoted text

> Failed with 400: Bad Request at > /usr/local/share/perl/5.14.2/Net/Google/Drive/Simple.pm line 529.

Completely different issue, most likely related to an invalid access token.

Wed Mar 26 21:22:48 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Thu, 27 Mar 2014 01:22:29 +0000
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

Perfectly right. I hadn't noticed that the credentials I had this time were for "offline access" only! The Google "developer's console" is not always extremely clear, and receiving a generic 400 instead of a 401 somewhat confused me. With the appropriate credentials, file downloading is now working fine for me using Perl 5.14.2 on Mint. So now I guess we could indeed say there is some kind of "sky shoes on a beach" problem attached to Simple.pm and/or Windows with perl - depending on the perspective :-) I wouldn't be suprised to find in the end that some quotes or similar were missing somewhere. Five minutes ago I had the suprise to see that a (very simple) script running on Windows-Msys bash needed some modification to run on Mint, for example. As I said, I'm (very) far from being an experienced programmer. Well, At least I can now go make myself something to eat before returning to Windows and meet the perl debugger. On 26/03/2014 23:52, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Wed Mar 26 19:34:53 2014, fjjmarques@free.fr wrote:

>> Failed with 400: Bad Request at >> /usr/local/share/perl/5.14.2/Net/Google/Drive/Simple.pm line 529.

> Completely different issue, most likely related to an invalid access token.

Thu Mar 27 00:45:26 2014 MSCHILLI [...] cpan.org - Correspondence added

On Wed Mar 26 21:22:48 2014, fjjmarques@free.fr wrote: Show quoted text

> With the appropriate credentials, file downloading is now working fine > for me using Perl 5.14.2 on Mint.

Nice, we're getting closer! Don't forget to try the other thing I've suggested in the previous post, namely using the download() method without specifying a file name, gobbling up the incoming bytes in a scalar and measuring its length.

Thu Mar 27 19:44:29 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Fri, 28 Mar 2014 00:44:16 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

Following your instructions I tested the "file-download" script in the debugger. The results of the first test don't sem to indicate anything unexpected, for instance: DB<1> c small.jpg ([y]/n) [y]> y Downloading small.jpg Net::Google::Drive::Simple::download(C:/strawberry/perl/site/lib/Net/Google/Drive/Simple.pm:455): 455: my $ua = LWP::UserAgent->new(); DB<1> x $req 0 HTTP::Request=HASH(0x2ad6f14) '_content' => '' '_headers' => HTTP::Headers=HASH(0x2ad6d04) 'authorization' => 'Bearer $access_token' '_method' => 'GET' '_uri' => URI::https=SCALAR(0x2a96604) -> '$src' Where $src is the actual link and $access_token actual token, of course. All files are approx. 6 kilobytes. Next, after replacement of my $c = $gd->download( $file, $name ); by my $data = $gd->download( $file );: It seems that the expected data are actually downloaded this time, considering the much increased time lapse between successive prompts for downloading, however nothing is written to my hard disc. Here is part of the printed output: Show quoted text

>perl eg/file-download

small.jpg ([y]/n) [y]> y Downloading small.jpg data length: 35174 ScanImage09.jpg ([y]/n) [y]> y Downloading ScanImage09.jpg data length: 1823420 09. Goodbye To Childhood (Alternate Take).mp3 ([y]/n) [y]> y Downloading 09. Goodbye To Childhood (Alternate Take).mp3 data length: 14024664 08. Riot (Second Alternate Take).mp3 ([y]/n) [y]> y Downloading 08. Riot (Second Alternate Take).mp3 data length: 11224337 Note that the printed sizes are correct. Very strange, at least to me! On 27/03/2014 05:45, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Wed Mar 26 21:22:48 2014, fjjmarques@free.fr wrote:

>> With the appropriate credentials, file downloading is now working fine >> for me using Perl 5.14.2 on Mint.

> Nice, we're getting closer! Don't forget to try the other thing I've suggested in the previous post, namely using the download() method without specifying a file name, gobbling up the incoming bytes in a scalar and measuring its length.

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Fri Mar 28 02:06:11 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Fri, 28 Mar 2014 07:05:59 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

I've just tested the attached script with both Strawberry Perl 5.18 a freshly installed ActivePerl 5.16, on Windows. Guess what: while the former gave me the "habitual" 5kb file, perl 5.16 did work! I've yet to test Simple.pm with it, but I expect it to work as well. So now the question is: what is the change introduced by v5.18 that causes this truncation behavior? Anyway, maybe you should add a warning for perl 5.18 users, at least on Windows. I wonder what would happen on Linux. One question: do you think I could use the Straberry module library with ActivePerl, to avoid reinstalling all required modules? On 27/03/2014 05:45, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Wed Mar 26 21:22:48 2014, fjjmarques@free.fr wrote:

>> With the appropriate credentials, file downloading is now working fine >> for me using Perl 5.14.2 on Mint.

> Nice, we're getting closer! Don't forget to try the other thing I've suggested in the previous post, namely using the download() method without specifying a file name, gobbling up the incoming bytes in a scalar and measuring its length.

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Message body is not shown because sender requested not to inline it.

Fri Mar 28 02:36:17 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Fri, 28 Mar 2014 07:36:07 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

Well, installation of the missing modules was fast with the perl package manager. No problem with file downloading using Net::Google::Drive::Simple with Perl 5.16 on the (old) WindowsXP-sp3. On 27/03/2014 05:45, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Wed Mar 26 21:22:48 2014, fjjmarques@free.fr wrote:

>> With the appropriate credentials, file downloading is now working fine >> for me using Perl 5.14.2 on Mint.

> Nice, we're getting closer! Don't forget to try the other thing I've suggested in the previous post, namely using the download() method without specifying a file name, gobbling up the incoming bytes in a scalar and measuring its length.

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Fri Mar 28 15:20:16 2014 MSCHILLI [...] cpan.org - Correspondence added

On Thu Mar 27 19:44:29 2014, fjjmarques@free.fr wrote: Show quoted text

> downloading, however nothing is written to my hard disc.

Yes, that's correct, without the file parameter, it won't write to the disk, the data stays in memory, and the length() call verified that the downloaded size is correct. So, it seems that somehow on Strawberry perl 5.18 on Win32 there's a problem writing binary data to disk, even while using the binary() switch as required on Win32. I'm quite surprised to see that, since you have confirmed early on that downloading a PDF file with LWP::UserAgent worked fine with the correct byte count. Can you put one of the files that you've tested this with onto a web server and then run the LWP::UserAgent test you did early on again? Something must be missing here.

Sat Mar 29 19:41:53 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Sun, 30 Mar 2014 00:41:40 +0100
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

Well, while "plain" (no special authorization) downloading with LWP::UserAgent works with both perl5.16 and perl5.18, there seems to be a problem specifically with the use of custom headers. Using the latest script I send you ("gootest.pl"), I tried to download files from Google and then from Box.com, which uses a similar authentication system (the header must also include an "Authorization: Bearer $access_token" string). No problem with perl 5.16. However with Strawberry perl 5.18, the files downloaded from Google are approx. 5 kb, while those downloaded from Box.com are simply empty, 0 kb files. I'm currently studying the Http::Headers documentation to see if I can find any hint of what's happening here. On 28/03/2014 20:20, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Thu Mar 27 19:44:29 2014, fjjmarques@free.fr wrote:

>> downloading, however nothing is written to my hard disc.

> Yes, that's correct, without the file parameter, it won't write to the disk, the data stays in memory, and the length() call verified that the downloaded size is correct. > > So, it seems that somehow on Strawberry perl 5.18 on Win32 there's a problem writing binary data to disk, even while using the binary() switch as required on Win32. I'm quite surprised to see that, since you have confirmed early on that downloading a PDF file with LWP::UserAgent worked fine with the correct byte count. Can you put one of the files that you've tested this with onto a web server and then run the LWP::UserAgent test you did early on again? Something must be missing here.

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Sun Mar 30 04:54:33 2014 fjjmarques [...] free.fr - Correspondence added

Subject:	Re: [rt.cpan.org #93913] truncated downloads
Date:	Sun, 30 Mar 2014 10:54:23 +0200
To:	bug-Net-Google-Drive-Simple [...] rt.cpan.org
From:	fm <fjjmarques [...] free.fr>

I forgot to mention that I also uploaded some .mp3 files to a server that doesn't require authentication. Again, there was no problem re-downloading these files with the "LWPtest" script and Strawberry perl 5.18. On 28/03/2014 20:20, Michael_Schilli via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=93913 > > > On Thu Mar 27 19:44:29 2014, fjjmarques@free.fr wrote:

>> downloading, however nothing is written to my hard disc.

> Yes, that's correct, without the file parameter, it won't write to the disk, the data stays in memory, and the length() call verified that the downloaded size is correct. > > So, it seems that somehow on Strawberry perl 5.18 on Win32 there's a problem writing binary data to disk, even while using the binary() switch as required on Win32. I'm quite surprised to see that, since you have confirmed early on that downloading a PDF file with LWP::UserAgent worked fine with the correct byte count. Can you put one of the files that you've tested this with onto a web server and then run the LWP::UserAgent test you did early on again? Something must be missing here.

--- Ce courrier électronique ne contient aucun virus ou logiciel malveillant parce que la protection avast! Antivirus est active. http://www.avast.com

Mon Mar 31 00:18:45 2014 MSCHILLI [...] cpan.org - Correspondence added

On Sat Mar 29 19:41:53 2014, fjjmarques@free.fr wrote: Show quoted text

> files from Google and then from Box.com, which uses a similar > authentication system (the header must also include an "Authorization: > Bearer $access_token" string). No problem with perl 5.16. However with > Strawberry perl 5.18, the files downloaded from Google are approx. 5 > kb,

That's quite an interesting find, I wouldn't have suspected that the request header has any effect on the truncation of the response.

Thu Jan 03 10:51:34 2019 me [...] eboxr.com - Correspondence added

Ticket migrated to github as https://github.com/mschilli/net-google-drive-simple/issues/26

Thu Jan 03 10:51:34 2019 me [...] eboxr.com - Status changed from 'open' to 'resolved'