Bug #78745 for libnet: ASCII put result has different lengths in 1.16 vs. 1.18

Thu Aug 02 14:08:40 2012 voyle.e.james.ctr [...] mail.mil - Ticket created

Subject:	ASCII put result has different lengths in 1.16 vs. 1.18
Date:	Thu, 2 Aug 2012 18:08:37 +0000
To:	"bug-libnet [...] rt.cpan.org" <bug-libnet [...] rt.cpan.org>
From:	"James, Voyle E (Ed) CTR USARMY HQDA ITA BA (US)" <voyle.e.james.ctr [...] mail.mil>

Hello, We have been running Net::FTP v2.75 with A.pm v1.16 via Perl 5.8.8 on our old machine. We switched to a new machine running Net::FTP v2.75 with A.pm v1.18 via Perl 5.12.4. The attached logs are from the copy of a file to another node, and a check run that does a size comparison. There is a copy and check from both 1.16 and 1.18. The file wc on both old and new machines is: 91 358 5762 INVOICE_0001_JUL12.REPORT On the old machine the copy reports 5853 transferred. The check log adds (1 x #lines = 91) to the local size (5762) giving 5853, and reports that it matches the remote size (5853). On the new machine the copy reports 5846 transferred. The check log adds (1 x #lines = 91) to the local size (5762) giving 5853, and reports that it doesn't match the remote size (5846). When I look at a raw dump of the remote 5846 file, there appear to be seven lines without a carriage return. Please let me know if I can help with more information. Thank you, Ed Ed James, voyle.e.james.ctr@mail.mil 703-545-3616-voice 571-256-3314-fax US Army Information Technology Agency, Business Administration Systems and Database Administrator Hoffman II 6N53-WS45, 200 Stovall Street, Alexandria, VA 22332-0027

Download web-iv-jul12-check-prod-1.16.log
application/octet-stream 10k

Message body not shown because it is not plain text.

Download web-iv-jul12-check-prod-1.18.log
application/octet-stream 10k

Message body not shown because it is not plain text.

Download web-iv-jul12-copy-prod-1.16.log
application/octet-stream 8.1k

Message body not shown because it is not plain text.

Download web-iv-jul12-copy-prod-1.18.log
application/octet-stream 8.1k

Message body not shown because it is not plain text.

Thu Aug 02 14:19:26 2012 voyle.e.james.ctr [...] mail.mil - Correspondence added

Subject:	RE: [rt.cpan.org #78745] AutoReply: ASCII put result has different lengths in 1.16 vs. 1.18
Date:	Thu, 2 Aug 2012 18:19:27 +0000
To:	"bug-libnet [...] rt.cpan.org" <bug-libnet [...] rt.cpan.org>
From:	"James, Voyle E (Ed) CTR USARMY HQDA ITA BA (US)" <voyle.e.james.ctr [...] mail.mil>

Additional info: I copied just A.pm v1.16 to the new machine over A.pm v1.18, reran the tests, and got a match on sizes. This was to eliminate possible differences in the other modules used by Net::FTP as suspects. Thank you, Ed Ed James, voyle.e.james.ctr@mail.mil 703-545-3616-voice 571-256-3314-fax US Army Information Technology Agency, Business Administration Systems and Database Administrator Hoffman II 6N53-WS45, 200 Stovall Street, Alexandria, VA 22332-0027 Show quoted text

-----Original Message----- From: Bugs in libnet via RT [mailto:bug-libnet@rt.cpan.org] Sent: Thursday, August 02, 2012 2:09 PM To: James, Voyle E (Ed) CTR USARMY HQDA ITA BA (US) Subject: [rt.cpan.org #78745] AutoReply: ASCII put result has different lengths in 1.16 vs. 1.18 Greetings, This message has been automatically generated in response to the creation of a trouble ticket regarding: "ASCII put result has different lengths in 1.16 vs. 1.18", a summary of which appears below. There is no need to reply to this message right now. Your ticket has been assigned an ID of [rt.cpan.org #78745]. Your ticket is accessible on the web at: https://rt.cpan.org/Ticket/Display.html?id=78745 Please include the string: [rt.cpan.org #78745] in the subject line of all future correspondence about this issue. To do so, you may reply to this message. Thank you, bug-libnet@rt.cpan.org ------------------------------------------------------------------------- Hello, We have been running Net::FTP v2.75 with A.pm v1.16 via Perl 5.8.8 on our old machine. We switched to a new machine running Net::FTP v2.75 with A.pm v1.18 via Perl 5.12.4. The attached logs are from the copy of a file to another node, and a check run that does a size comparison. There is a copy and check from both 1.16 and 1.18. The file wc on both old and new machines is: 91 358 5762 INVOICE_0001_JUL12.REPORT On the old machine the copy reports 5853 transferred. The check log adds (1 x #lines = 91) to the local size (5762) giving 5853, and reports that it matches the remote size (5853). On the new machine the copy reports 5846 transferred. The check log adds (1 x #lines = 91) to the local size (5762) giving 5853, and reports that it doesn't match the remote size (5846). When I look at a raw dump of the remote 5846 file, there appear to be seven lines without a carriage return. Please let me know if I can help with more information. Thank you, Ed Ed James, voyle.e.james.ctr@mail.mil 703-545-3616-voice 571-256-3314-fax US Army Information Technology Agency, Business Administration Systems and Database Administrator Hoffman II 6N53-WS45, 200 Stovall Street, Alexandria, VA 22332-0027

Thu Jan 30 03:51:10 2014 shay [...] cpan.org - Correspondence added

On Thu Aug 02 14:19:26 2012, voyle.e.james.ctr@mail.mil wrote: Show quoted text

> Additional info: > > I copied just A.pm v1.16 to the new machine over A.pm v1.18, reran the > tests, and got a match on sizes. > > This was to eliminate possible differences in the other modules used > by Net::FTP as suspects. >

Please can you try libnet-1.24 and report back whether this is still a problem? I cannot reproduce it, but I also couldn't using 1.18 either (on my Windows machine) so I may not be doing the right thing. If the bug still exists in 1.24 then please attach a sample file that does not transfer correctly and a script showing the commands used.

Thu Jan 30 03:51:11 2014 The RT System itself - Status changed from 'new' to 'open'

Thu Feb 05 08:34:31 2015 shay [...] cpan.org - Correspondence added

I have now reproduced what was likely to be the problem with A.pm v1.18, which didn't occur with A.pm v1.16, and it is indeed now fixed with libnet-1.24 and later. We know that the bug must be in A.pm since copying the old v1.16 into the new setup fixed the problem. The only changes to A.pm between v1.16 (in libnet-1.19) and v1.18 (in libnet-1.21) were: 3b3fad04dc Handle CRLF that cross a block boundary c490800ca0 Do not remove last character of line when adding \r 1f9f03272f Run through perltidy The latter commit was certainly a no-op in this regard. The combined effect of the former two commits was to change: (my $tmp = substr($buf,0,$size)) =~ s/\r?\n/\015\012/sg; to: my $nr = (my $tmp = substr($buf, 0, $size)) =~ tr/\r\n/\015\012/; $tmp =~ s/([^\015])\012/$1\015\012/sg if $nr; $tmp =~ s/^\012/\015\012/ unless ${*$data}{'net_ftp_outcr'}; ${*$data}{'net_ftp_outcr'} = substr($tmp, -1) eq "\015"; That was fine up to a point, but had a bug in the case of two consecutive \012 bytes in the data, which was fixed by: 24eb861945 Fix incorrect handling of CRLF in Net::FTP in v1.19 (libnet-1.24). That commit changed the above code to: my $nr = (my $tmp = substr($buf, 0, $size)) =~ tr/\r\n/\015\012/; $tmp =~ s/(?<!\015)\012/\015\012/sg if $nr; $tmp =~ s/^\015// if ${*$data}{'net_ftp_outcr'}; ${*$data}{'net_ftp_outcr'} = substr($tmp, -1) eq "\015"; The breakage and the fix can be seen in these one-liners: This is what v1.16 did (correct in this case): Show quoted text

>perl -e "$str=qq[X\012\012]; $str=~s/\r?\n/$1\015\012/sg; binmode STDOUT; print $str"|od -c

0000000 X \r \n \r \n 0000005 This is what v1.18 did (wrong): Show quoted text

>perl -e "$str=qq[X\012\012]; $str=~s/([^\015])\012/$1\015\012/sg; binmode STDOUT; print $str"|od -c

0000000 X \r \n \n 0000004 This is what v1.19 did (correct): Show quoted text

>perl -e "$str=qq[X\012\012]; $str=~s/(?<!\015)\012/$1\015\012/sg; binmode STDOUT; print $str"|od -c

0000000 X \r \n \r \n 0000005 I am therefore marking this ticket as resolved since I strongly suspect that the bug in question is the one that has already been fixed as of A.pm v1.19.

Thu Feb 05 08:34:35 2015 shay [...] cpan.org - Status changed from 'open' to 'resolved'

Thu Feb 05 08:34:35 2015 shay [...] cpan.org - Fixed in 1.24 added