Bug #35888 for Net-SFTP-Foreign: put() on hpux fails with: Connection to remote server is broken

Wed May 14 16:09:29 2008 Tom.Warkentin [...] intecbilling.com - Ticket created

Subject:	put() on hpux fails with: Connection to remote server is broken
Date:	Wed, 14 May 2008 14:07:51 -0600
To:	<bug-Net-SFTP-Foreign [...] rt.cpan.org>
From:	"Tom Warkentin" <Tom.Warkentin [...] intecbilling.com>

Distribution: Net-SFTP-Foreign-1.35 Perl Version: 5.8.7 built for PA-RISC2.0-LP64 Operating System: HP-UX bmw B.11.11 U 9000/800 1121911527 unlimited-user license Local SSH Version: OpenSSH_4.3p2-hpn, OpenSSL 0.9.7i 14 Oct 2005 HP-UX Secure Shell-A.04.30.006, HP-UX Secure Shell version Remote SSH Version: OpenSSH_4.5p1+sftpfilecontrol-v1.1-hpn12v14, OpenSSL 0.9.7l 28 Sep 2006 HP-UX Secure Shell-A.04.50.003, HP-UX Secure Shell version Sending a file using the Net::SFTP::Foreign::put method consistently fails with "Connection to remote server is broken" with largish files. After further debugging, it appears that the remote server just closes the connection when largish packets of data are sent. A work around is to pass in a block_size of 483 to get rid of this error. However, this does not result in a successful transfer. After explicitly setting the block size, I got errors like "bad packet sequence, expected 11, got 12". The sequence numbers were not the same each time the error was produced. By passing in a queue_size of 1, I was able to finally get a successful "put" of a file larger than 483 bytes. The same problem occurs on a "put" transfer from an AIX machine. If you want the OS/SSH/Perl details for this as well, I can dig it up when I get some time. The only difference is that the magic number is not 483 bytes but some other slightly larger value (I haven't figured out the limit by trial and error on AIX but it is somewhere between 483 and 1000 bytes). "get" transfers do not have this problem on any OSes tested. Also, the sftp command that comes with OpenSSH works fine for sending or receiving files. This leads me to believe that there is some logic error in the put method that is causing this problem but I have not yet been able to track it down. If you need any more information or have ideas to try, let me know... Thanks, Tom This e-mail and any attachments are confidential and may also be legally privileged and/or copyright material of Intec Telecom Systems PLC (or its affiliated companies). If you are not an intended or authorised recipient of this e-mail or have received it in error, please delete it immediately and notify the sender by e-mail. In such a case, reading, reproducing, printing or further dissemination of this e-mail or its contents is strictly prohibited and may be unlawful. Intec Telecom Systems PLC does not represent or warrant that an attachment hereto is free from computer viruses or other defects. The opinions expressed in this e-mail and any attachments may be those of the author and are not necessarily those of Intec Telecom Systems PLC.

Thu May 15 18:03:30 2008 salva [...] cpan.org - Correspondence added

From:

salva [...] cpan.org

On Wed May 14 16:09:29 2008, Tom.Warkentin@intecbilling.com wrote: Show quoted text

> Distribution: Net-SFTP-Foreign-1.35 > > Perl Version: 5.8.7 built for PA-RISC2.0-LP64 > > Operating System: HP-UX bmw B.11.11 U 9000/800 1121911527 unlimited-user > license > > Local SSH Version: OpenSSH_4.3p2-hpn, OpenSSL 0.9.7i 14 Oct 2005 > > HP-UX Secure Shell-A.04.30.006, HP-UX Secure Shell version > > Remote SSH Version: OpenSSH_4.5p1+sftpfilecontrol-v1.1-hpn12v14, OpenSSL > 0.9.7l 28 Sep 2006 > > HP-UX Secure Shell-A.04.50.003, HP-UX Secure Shell version > > Sending a file using the Net::SFTP::Foreign::put method consistently > fails with "Connection to remote server is broken" with largish files. > After further debugging, it appears that the remote server just closes > the connection when largish packets of data are sent.

how is your script handling authentication to the remote server? are you using public-keys or a password? Show quoted text

> A work around is to pass in a block_size of 483 to get rid of this > error. However, this does not result in a successful transfer. After > explicitly setting the block size, I got errors like "bad packet > sequence, expected 11, got 12". The sequence numbers were not the same > each time the error was produced. By passing in a queue_size of 1, I > was able to finally get a successful "put" of a file larger than 483 > bytes.

Net::SFTP::Foreign aggressively pipelines SFTP commands in order to maximize the throughput and probably the server doesn't like it. Have you tried reducing both values at the same time to something reasonable, for instance: block_size => 2048, queue_size => 2. Anyway, in order to generate debugging information, set... $Net::SFTP::Foreign::debug = 25; at the beginning of the script, and send me the generated output for both failing cases (not setting the block size and setting it to 483). Cheers, - Salva

Thu May 15 18:03:32 2008 The RT System itself - Status changed from 'new' to 'open'

Thu May 15 19:17:56 2008 Tom.Warkentin [...] intecbilling.com - Correspondence added

Subject:	RE: [rt.cpan.org #35888] put() on hpux fails with: Connection to remote server is broken
Date:	Thu, 15 May 2008 17:17:31 -0600
To:	<bug-Net-SFTP-Foreign [...] rt.cpan.org>
From:	"Tom Warkentin" <Tom.Warkentin [...] intecbilling.com>

My script is configurable to use username/password authentication or public-keys. Either method of authentication works fine for retrieving file listing information and for transferring small files without specifying the magic 483 byte block size and queue size of 1. Based on your suggestions below, I ran four tests with debug to transfer a 5200 byte file. All tests used username/password authentication using Expect to wait for the appropriate prompts. I added the code to set the debug value after the SFTP Foreign object was created with a transport passed as the only parameter to the constructor. When using Expect, I seem to remember the debug output confusing Expect so that is why I did this. The first test did not pass in a block_size or queue_size and the transfer failed with the "Connection to remote server is broken" error. See the stderr.dbg_noblk_noque file for the debug output. The second test passed in a block_size of 2048 and a queue_size of 2. The transfer failed with the same error. See the stderr.dbg_2048_2 file for the debug output. The third test passed in a block_size of 484 but no queue_size and it failed with the error. See the stderr.dbg_484_noque file for the debug output. The fourth test passed in a block size of 483 and a queue_size of 1. The transfer completed successfully. See the stderr.dbg_483_1 file for the debug output. Let me know if there are any other scenarios you'd like me to try. I have also requested that OpenSSH/OpenSSL on the box be upgraded to the latest versions supported by HP. If this resolves the issue, I will keep you posted... Cheers, Tom Show quoted text

-----Original Message----- From: Salvador Fandino Garcia via RT [mailto:bug-Net-SFTP-Foreign@rt.cpan.org] Sent: Thursday, May 15, 2008 4:04 PM To: Tom Warkentin Subject: [rt.cpan.org #35888] put() on hpux fails with: Connection to remote server is broken <URL: http://rt.cpan.org/Ticket/Display.html?id=35888 > On Wed May 14 16:09:29 2008, Tom.Warkentin@intecbilling.com wrote:

> Distribution: Net-SFTP-Foreign-1.35 > > Perl Version: 5.8.7 built for PA-RISC2.0-LP64 > > Operating System: HP-UX bmw B.11.11 U 9000/800 1121911527 unlimited-user > license > > Local SSH Version: OpenSSH_4.3p2-hpn, OpenSSL 0.9.7i 14 Oct 2005 > > HP-UX Secure Shell-A.04.30.006, HP-UX Secure Shell version > > Remote SSH Version: OpenSSH_4.5p1+sftpfilecontrol-v1.1-hpn12v14, OpenSSL > 0.9.7l 28 Sep 2006 > > HP-UX Secure Shell-A.04.50.003, HP-UX Secure Shell version > > Sending a file using the Net::SFTP::Foreign::put method consistently > fails with "Connection to remote server is broken" with largish files. > After further debugging, it appears that the remote server just closes > the connection when largish packets of data are sent.

how is your script handling authentication to the remote server? are you using public-keys or a password?

> A work around is to pass in a block_size of 483 to get rid of this > error. However, this does not result in a successful transfer. After > explicitly setting the block size, I got errors like "bad packet > sequence, expected 11, got 12". The sequence numbers were not the same > each time the error was produced. By passing in a queue_size of 1, I > was able to finally get a successful "put" of a file larger than 483 > bytes.

Net::SFTP::Foreign aggressively pipelines SFTP commands in order to maximize the throughput and probably the server doesn't like it. Have you tried reducing both values at the same time to something reasonable, for instance: block_size => 2048, queue_size => 2. Anyway, in order to generate debugging information, set... $Net::SFTP::Foreign::debug = 25; at the beginning of the script, and send me the generated output for both failing cases (not setting the block size and setting it to 483). Cheers, - Salva This e-mail and any attachments are confidential and may also be legally privileged and/or copyright material of Intec Telecom Systems PLC (or its affiliated companies). If you are not an intended or authorised recipient of this e-mail or have received it in error, please delete it immediately and notify the sender by e-mail. In such a case, reading, reproducing, printing or further dissemination of this e-mail or its contents is strictly prohibited and may be unlawful. Intec Telecom Systems PLC does not represent or warrant that an attachment hereto is free from computer viruses or other defects. The opinions expressed in this e-mail and any attachments may be those of the author and are not necessarily those of Intec Telecom Systems PLC.

Download stderr.dbg_noblk_noque
application/octet-stream 24.6k

Message body not shown because it is not plain text.

Download stderr.dbg_483_1
application/octet-stream 30.9k

Message body not shown because it is not plain text.

Download stderr.dbg_2048_2
application/octet-stream 20.4k

Message body not shown because it is not plain text.

Download stderr.dbg_484_noque
application/octet-stream 25.8k

Message body not shown because it is not plain text.

Fri May 16 06:53:53 2008 sfandino [...] yahoo.com - Correspondence added

Subject:	Re: [rt.cpan.org #35888] put() on hpux fails with: Connection to remote server is broken
Date:	Fri, 16 May 2008 03:53:03 -0700 (PDT)
To:	bug-Net-SFTP-Foreign [...] rt.cpan.org
From:	Salvador Fandino <sfandino [...] yahoo.com>

Hi, The debugging output on your last mail doesn't show the precise cause of the problem, I believe that it is in the _do_io_unix function that probably has hidden dependencies on Linux semantics for some IO operations. I will send you a new version of the module with improved debugging support in a private email. Install it and set... $Net::SFTP::Foreign::debug = 1|2|32|64; ... and run the tests again. Hopefully, this time we will get more interesting information. To minimize the possible causes of the problem, don't use Expect, just let the script ask for the password on the command line or use public-key authentication. It would also help if you could capture the script interaction with the OS using strace, truss or ktrace (or whatever HP-UX has to monitor processes), but don't trace the child ssh processes. Cheers and thank you for your collaboration! - Salva

Thu May 22 17:21:42 2008 salva [...] cpan.org - Correspondence added 960 min

The bug is solved in the new module version 1.37_06 available from CPAN. Now the interprocess communication performed through a PTY is limited to the authentication phase. Afterwards, a regular pipe is used.

Thu May 22 17:21:46 2008 salva [...] cpan.org - Status changed from 'open' to 'resolved'