Skip Menu |

This queue is for tickets about the Net-SFTP-Foreign CPAN distribution.

Report information
The Basics
Id: 32026
Status: resolved
Worked: 2 hours (120 min)
Priority: 0/
Queue: Net-SFTP-Foreign
People
Owner: Nobody in particular
Requestors: gerry.lin [...] db.com
Cc:
AdminCc:
Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)

History Show all quoted text
  Mon Dec 31 16:29:53 2007 gerry.lin [...] db.com - Ticket created
Subject: SFTP::Foreign::get Failed after 100 files(version 1.31)
Date: Mon, 31 Dec 2007 16:29:16 -0500
To: bug-Net-SFTP-Foreign [...] rt.cpan.org
From: Gerry Lin <gerry.lin [...] db.com>
Hi Salvador Thank you for this useful package which has helped us a lot. However, SFTP::Foreign::get will fail when try to download list of files over 100. Here is the code and error message, Show quoted text
>>> beginning of code
use strict; use Net::SFTP::Foreign; # version 1.31 os aix5.1 perl 5.8 64bit my @f_list=<{*}>; # get list of files over 100 my ($host,$dir)=(q{user_id@remote_host_name},q{remote_host_path_dir}); use Net::SFTP::Foreign; my $sftp = Net::SFTP::Foreign->new($host); $sftp->setcwd($dir); my ($f_cnt,$f)=(0); foreach $f(@f_list) { $sftp->get($f,$f) or die "unable to get [$f] sftp error-msg: ".$sftp->error; $f_cnt++; print STDERR "$f_cnt files downloaded\n"; } <<< end of code error message from $sftp->error, "Couldn't open remote file 'remote_file_full_path" where this 101th remote file does exist. regards Gerry Lin +1 201 593 3170(Desk) +1 201 341 2405(Mobile) --- This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
  Sun Jan 06 16:26:36 2008 sfandino [...] yahoo.com - Correspondence added
CC: gerry.lin [...] db.com
Subject: Re: [rt.cpan.org #32026] SFTP::Foreign::get Failed after 100 files(version 1.31)
Date: Sun, 6 Jan 2008 13:26:21 -0800 (PST)
To: bug-Net-SFTP-Foreign [...] rt.cpan.org
From: Salvador Fandino <sfandino [...] yahoo.com>
Hi Gerry, I am unable to reproduce the problem on my system. Have you been able to transfer the same files using the sftp program (you can use the -b option to pass a batchfile to sftp)? Which OS and SSH software is running on the remote side? Could you download version 1.33 from CPAN and add at the beginning of your script this line: $Net::SFTP::Foreign::debug = 7; Then run the script and send my the (compressed) output by email (my address is sfandino@yahoo.com). Cheers, - Salva Show quoted text
----- Original Message ----
> From: Gerry Lin via RT <bug-Net-SFTP-Foreign@rt.cpan.org> > Sent: Monday, December 31, 2007 10:29:55 PM > Subject: [rt.cpan.org #32026] SFTP::Foreign::get Failed after 100 files(version 1.31) > > > Mon Dec 31 16:29:53 2007: Request 32026 was acted upon. > Transaction: Ticket created by gerry.lin@db.com > Queue: Net-SFTP-Foreign > Subject: SFTP::Foreign::get Failed after 100 files(version 1.31) > Broken in: (no value) > Severity: (no value) > Owner: Nobody > Requestors: gerry.lin@db.com > Status: new > Ticket > > > Hi Salvador > > Thank you for this useful package which has helped us a lot. However, > SFTP::Foreign::get will fail when try to download list of files > over >
100.
> Here is the code and error message, > >
> >>> beginning of code
> use strict; > > use Net::SFTP::Foreign; # version 1.31 os aix5.1 perl 5.8 64bit > > > my @f_list=<{*}>; # get list of files over 100 > > > my ($host,$dir)=(q{user_id@remote_host_name},q{remote_host_path_dir}); > > use Net::SFTP::Foreign; > my $sftp = Net::SFTP::Foreign->new($host); > > $sftp->setcwd($dir); > > my ($f_cnt,$f)=(0); > foreach $f(@f_list) { > $sftp->get($f,$f) or die "unable to get [$f] sftp error-msg: > ".$sftp->error; > $f_cnt++; > print STDERR "$f_cnt files downloaded\n"; > } > <<< end of code > > error message from $sftp->error, "Couldn't open remote file > 'remote_file_full_path" where this 101th remote file does exist. > > > regards > > Gerry Lin > +1 201 593 3170(Desk) > +1 201 341 2405(Mobile) > > --- > > This e-mail may contain confidential and/or privileged information. > If >
you
> are not the intended recipient (or have received this e-mail in error) > please notify the sender immediately and destroy this e-mail. Any > unauthorized copying, disclosure or distribution of the material > in >
this
> e-mail is strictly forbidden. > > > -----Inline Attachment Follows----- > >
> Hi Salvador >
>
> Thank you for this useful package > which >
has helped us a lot. However, SFTP::Foreign::get will fail when try
> to >
download list of files over 100. Here is the code and error
> message, >
>
>
> >>> beginning of
> code >
> use > strict; >
>
> use Net::SFTP::Foreign; # version > 1.31 >
os aix5.1 perl 5.8
> 64bit >
>
>
> my @f_list=<{*}>; # get > list >
of files over
> 100 >
>
>
> my > ($host,$dir)=(q{user_id@remote_host_name},q{remote_host_path_dir}); >
>
> use > Net::SFTP::Foreign; >
> my $sftp = > Net::SFTP::Foreign->new($host); >
>
> $sftp->setcwd($dir); >
>
> my > ($f_cnt,$f)=(0); >
> foreach $f(@f_list) > { >
> $sftp->get($f,$f) or > die >
"unable to get [$f] sftp error-msg:
> ".$sftp->error; >
> > $f_cnt++; >
> print STDERR "$f_cnt > files >
downloaded\n";
>
> } >
> <<< end of > code >
>
> error message from > $sftp->error, >
"Couldn't open remote file 'remote_file_full_path" where
> this >
101th remote file does exist.
>
>
>
> regards >
>
> Gerry Lin >
+1 201 593 3170(Desk)
>
+1 201 341
> 2405(Mobile) >
>
---
>
>
This e-mail may contain confidential and/or privileged information. If you
>
are not the intended recipient (or have received this e-mail in error)
>
please notify the sender immediately and destroy this e-mail. Any
>
unauthorized copying, disclosure or distribution of the material in this
>
e-mail is strictly forbidden.
> >
____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
  Sun Jan 06 16:26:39 2008 The RT System itself - Status changed from 'new' to 'open'
  Mon Jan 07 18:37:51 2008 sfandino [...] yahoo.com - Correspondence added
CC: bug-Net-SFTP-Foreign [...] rt.cpan.org
Subject: Re: [rt.cpan.org #32026] SFTP::Foreign::get Failed after 100 files(version 1.31)
Date: Mon, 7 Jan 2008 15:37:33 -0800 (PST)
To: Gerry Lin <gerry.lin [...] db.com>
From: Salvador Fandino <sfandino [...] yahoo.com>
Hi Gerry, Show quoted text
> 4) Foreign.pm.close_rfh_after_get, modified
Foreign.pm with one line added to close rfh at end of "sub get" The SFTP file handlers have a DESTROY method that closes the associated remote file handle automatically, but for some reason it is not being called on your system. This is the debugging output I get on my linux box running perl 5.8.8: # queueing msg len: 51, code:3, id:52 ... [1] # waiting for message... [1] # got it!, len:13, code:102, id:52, status: - # new remote file '/tmp/sftp_max/dat/tiny_file011.dat' open, rid: 00 00 00 00 | .... # queueing msg len: 25, code:5, id:53 ... [1] # waiting for message... [1] # got it!, len:13, code:103, id:53, status: - # queueing msg len: 25, code:5, id:54 ... [1] # waiting for message... [1] # got it!, len:28, code:101, id:54, status: 1 # Net::SFTP::Foreign::FileHandle=GLOB(0x833597c)->DESTROY called (sftp: Net::SFTP::Foreign=HASH(0x81515e8)) # queueing msg len: 13, code:4, id:55 ... [1] # waiting for message... [1] # got it!, len:24, code:101, id:55, status: 0 # closing file handle, return: 1, rid: 00 00 00 00 | .... +++11 files downloaded see how the sftp close command (code 4) is called from the DESTROY callback. On your log, there is no such call: # queueing msg len: 51, code:3, id:42 ... [1] # waiting for message... [1] # got it!, len:13, code:102, id:42, status: - # new remote file '/var/tmp/sftp_max/tiny_file011.dat' open, rid: 00 00 00 0a | .... # queueing msg len: 25, code:5, id:43 ... [1] # waiting for message... [1] # got it!, len:19, code:103, id:43, status: - # queueing msg len: 25, code:5, id:44 ... [1] # waiting for message... [1] # got it!, len:28, code:101, id:44, status: 1 +++11 files downloaded Which Perl version are you using? can you include the output from "perl -V" Cheers, - Salva Show quoted text
____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping
  Mon Jan 07 21:30:37 2008 gerry.lin [...] db.com - Correspondence added
CC: bug-Net-SFTP-Foreign [...] rt.cpan.org
Subject: Re: [rt.cpan.org #32026] SFTP::Foreign::get Failed after 100 files(version 1.31)
Date: Mon, 7 Jan 2008 21:30:00 -0500
To: sfandino [...] yahoo.com
From: Gerry Lin <gerry.lin [...] db.com>
Hi Salva, Our current perl version is 5.8.0 built almost 6 years ago. And it looks like it should be upgraded to perl5.8.8. And below is -V output, Show quoted text
>>>
/db/pub/infra/CPAN/perl/64-5.8/bin/perl -V Summary of my perl5 (revision 5.0 version 8 subversion 0) configuration: Platform: osname=aix, osvers=5.1.0.0, archname=aix-64all uname='aix hbrfissd1 1 5 0026a19a4c00 ' config_args='' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=define use64bitall=define uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='xlc_r', ccflags ='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=16384 -qnoansialias -DUSE_NATIVE_DLOPEN -q64 -DUSE_64_BIT_ALL -q64', optimize='-O -q64', cppflags='-D_ALL_SOURCE -D_ANSI_C_SOURCE -D_POSIX_SOURCE -qmaxmem=16384 -qnoansialias -DUSE_NATIVE_DLOPEN' ccversion='5.0.2.6', gccversion='', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=87654321 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='ld', ldflags ='-brtl -q64 -b64' libpth=/lib /usr/lib /usr/ccs/lib libs=-lbind -lnsl -ldbm -ldl -lld -lm -lc -lcrypt -lbsd -lPW perllibs=-lbind -lnsl -ldl -lld -lm -lc -lcrypt -lbsd -lPW libc=/usr/ccs/lib/libc.a, so=a, useshrplib=false, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_aix.xs, dlext=so, d_dlsymun=undef, ccdlflags=' -bE:/db/pub/infra/CPAN/perl/64-5.8/lib/5.8.0/aix-64all/CORE/perl.exp' cccdlflags=' ', lddlflags='-b64 -bhalt:4 -bM:SRE -bI:$(PERL_INC)/perl.exp -bE:$(BASEEXT).exp -bnoentry -lc' Characteristics of this binary (from libperl): Compile-time options: USE_64_BIT_INT USE_64_BIT_ALL USE_LARGE_FILES Built under aix Compiled at Feb 13 2003 13:30:52 %ENV: PERL5LIB=":/db/pub/infra/CPAN/perl/64-5.8/perl-lib:/db/pub/infra/CPAN/perl/64-5.8/perl-lib" PERL_WORKING_DIR="/db/pub/infra/CPAN/perl/64-5.8" @INC: /db/pub/infra/CPAN/perl/64-5.8/perl-lib /db/pub/infra/CPAN/perl/64-5.8/perl-lib /db/pub/infra/CPAN/perl/64-5.8/lib/5.8.0/aix-64all /db/pub/infra/CPAN/perl/64-5.8/lib/5.8.0 /db/pub/infra/CPAN/perl/64-5.8/lib/site_perl/5.8.0/aix-64all /db/pub/infra/CPAN/perl/64-5.8/lib/site_perl/5.8.0 /db/pub/infra/CPAN/perl/64-5.8/lib/site_perl . [dteprod@nyhshutl06d3():NYFISSU] /applications/dte/sql <<< Gerry Lin +1 201 593 3170(Desk) +1 201 341 2405(Mobile) Salvador Fandino <sfandino@yahoo.com> 01/07/2008 06:37 PM To Gerry Lin/NewYork/DBNA/DeuBa@DBAmericas cc bug-Net-SFTP-Foreign@rt.cpan.org Subject Re: [rt.cpan.org #32026] SFTP::Foreign::get Failed after 100 files(version 1.31) Hi Gerry, Show quoted text
> 4) Foreign.pm.close_rfh_after_get, modified Foreign.pm with one line
added to close rfh at end of "sub get" The SFTP file handlers have a DESTROY method that closes the associated remote file handle automatically, but for some reason it is not being called on your system. This is the debugging output I get on my linux box running perl 5.8.8: # queueing msg len: 51, code:3, id:52 ... [1] # waiting for message... [1] # got it!, len:13, code:102, id:52, status: - # new remote file '/tmp/sftp_max/dat/tiny_file011.dat' open, rid: 00 00 00 00 | .... # queueing msg len: 25, code:5, id:53 ... [1] # waiting for message... [1] # got it!, len:13, code:103, id:53, status: - # queueing msg len: 25, code:5, id:54 ... [1] # waiting for message... [1] # got it!, len:28, code:101, id:54, status: 1 # Net::SFTP::Foreign::FileHandle=GLOB(0x833597c)->DESTROY called (sftp: Net::SFTP::Foreign=HASH(0x81515e8)) # queueing msg len: 13, code:4, id:55 ... [1] # waiting for message... [1] # got it!, len:24, code:101, id:55, status: 0 # closing file handle, return: 1, rid: 00 00 00 00 | .... +++11 files downloaded see how the sftp close command (code 4) is called from the DESTROY callback. On your log, there is no such call: # queueing msg len: 51, code:3, id:42 ... [1] # waiting for message... [1] # got it!, len:13, code:102, id:42, status: - # new remote file '/var/tmp/sftp_max/tiny_file011.dat' open, rid: 00 00 00 0a | .... # queueing msg len: 25, code:5, id:43 ... [1] # waiting for message... [1] # got it!, len:19, code:103, id:43, status: - # queueing msg len: 25, code:5, id:44 ... [1] # waiting for message... [1] # got it!, len:28, code:101, id:44, status: 1 +++11 files downloaded Which Perl version are you using? can you include the output from "perl -V" Cheers, - Salva Never miss a thing. Make Yahoo your homepage. --- This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
  Tue Jan 08 03:49:38 2008 sfandino [...] yahoo.com - Correspondence added
Subject: Re: [rt.cpan.org #32026] SFTP::Foreign::get Failed after 100 files(version 1.31)
Date: Tue, 8 Jan 2008 00:49:21 -0800 (PST)
To: bug-Net-SFTP-Foreign [...] rt.cpan.org
From: Salvador Fandino <sfandino [...] yahoo.com>
hi, Show quoted text
> Our current perl version is 5.8.0 built almost 6 years ago. And > it looks like it should be upgraded to perl5.8.8.
Yes, you should, 5.8.0 was very buggy. The problem with Net::SFTP::Foreign is probably caused by this perl 5.8.0 bug: http://rt.perl.org/rt3/Public/Bug/Display.html?id=18038 I am not going to work-around it in the module as it relies in the DESTROY method being called on time too heavily, it is not just the "get" method. Cheers, - Salva Show quoted text
____________________________________________________________________________________ Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
  Tue Jan 08 04:01:56 2008 salva [...] cpan.org - Correspondence added 120 min
caused by a perl 5.8.0 bug
  Tue Jan 08 04:01:59 2008 salva [...] cpan.org - Status changed from 'open' to 'resolved'