Bug #75743 for Term-ReadLine-Perl: rl_scroll_nextline inserts line full of spaces into the output before the prompt

I think there's a bug in the implementation of rl_scroll_nextline in Term::ReadLine::Perl. The effect of the bug is that a full line of spaces is placed into the output before the prompt is written. Since there is no newline at the end of this, copy and paste from some terminal emulators then ends up with the prompt displaced across the screen (more details about which terminals are affected later). When users copy and paste logs from programs using Term::ReadLine::Perl the results are messy. Visually, you get a blank line before the prompt. Although this does not look unattractive, this is different from Term::ReadLine::Stub, the behaviour with rl_scroll_nextline turned off and the behaviour on unaffected terminal emulators. I think the bug is in the latest version (1.0303) and has been there since the code came in (version 1.0204). I can see the problems in various versions of Perl 5.8 and in Linux versions ranging from 2.4 to 2.6. So, I don't think this is OS or Perl version related. I think the bug is caused by two spaces between \b and \r in the line that implements rl_scroll_nextline. Removing those two spaces fixes the problem. Since the two spaces start in column 80 of the line, my suspicion is that someone was indenting this section of code by two spaces in an 80 column wide editor and moved down a screen line but not a physical line. Here's my proposed diff: -------------------------------------------------------------------------------- --- readline-orig.pm 2012-03-13 15:06:15.044848000 +0000 +++ readline.pm 2012-03-13 15:06:48.996220000 +0000 @@ -1531,7 +1531,7 @@ # on XTerm one needs to increase the number by 1. - print $term_OUT ' ' x ($rl_screen_width - !$rl_last_pos_can_backspace) . "\b \r" + print $term_OUT ' ' x ($rl_screen_width - !$rl_last_pos_can_backspace) . "\b\r" if $rl_scroll_nextline; if ($dumb_term) { -------------------------------------------------------------------------------- Here's a test case and reference output. In each case, I've copied and pasted the output out of a PuTTY window or one of the other affected terminal emulators and then replaced spaces with dots in the output. I'm starting with the current Stub implementation as a reference followed by the current code and then the output with my proposed fix. In case the long lines in the following examples get mangled, here's the test Perl code formatted more normally: use Term::ReadLine; my $term = new Term::ReadLine "test"; for(;;) { print $term->readline("prompt> "), "\n"; } ================================================================================ Stub implementation ------------------- $ PERL_RL=Stub perl -e 'use Term::ReadLine; my $term = new Term::ReadLine "test"; for(;;) { print $term->readline("prompt> "), "\n"; }' Show quoted text
foo Show quoted text
bar Show quoted text
Current Perl implementation --------------------------- $ PERL_RL=Perl perl -e 'use Term::ReadLine; my $term = new Term::ReadLine "test"; for(;;) { print $term->readline("prompt> "), "\n"; }' ................................................................................prompt>.foo foo ................................................................................prompt>.bar bar ................................................................................prompt> [In case the previous lines have got mangled, the lines containing the prompts have 80 dots before the prompt. Note that on screen, with an 80 column terminal the prompts appear in the left hand column with blank lines above them.] Perl implementation with proposed patch --------------------------------------- $ PERL_RL=Perl perl -e 'use Term::ReadLine; my $term = new Term::ReadLine "test"; for(;;) { print $term->readline("prompt> "), "\n"; }' Show quoted text
foo Show quoted text
bar Show quoted text
================================================================================ Really, any test should also make sure that rl_scroll_newline works as intended even if just one character has been output before the prompt. Something like this might be suitable: use Term::ReadLine; my $term = new Term::ReadLine "test"; select($term->OUT); for(;;) { print "*"; print $term->readline("prompt> "), "\n"; } The problem shows in: PuTTY (version 0.62) gnome-terminal (version 2.16.0) Terminal (version 0.2.8 from xfce 4.4) screen (versions 3.09.15 (FAU) 13-Mar-03 and 4.00.03 (FAU) 23-Oct-06) konsole (Qt: 3.3.6; KDE: 3.5.4-22.el5_3 Red Hat; Konsole: 1.6.4) xterm with reverse wraparound enabled (not default?) The problem does not show in: xterm with reverse wraparound disabled (default?) I've marked default with a question mark as xterm's settings are at the mercy of lots of configuration files. It's what I get on my systems on the two versions of xterm I've tried (XFree86 4.3.99.5(179) and XTerm(215)). I think the difference in behaviour is to do with how the various terminals deal with reaching the right hand column. To simplify the example, I will assume an 80 character display. I will number the columns with column 1 being the left hand edge of the screen and column 80 being the right hand edge. If I say the cursor is in column N then the next character will be output in column N. All terminal emulators agree that after 79 spaces have been output, the cursor is in column 80. When the 80th character is output, many terminals now say the cursor is in column 81. If another character is output, the terminals go to column 1 on the next line, then output the character. If, on the other hand, with the cursor in column 81, a backspace is received the cursor goes back to column 80. With xterm (with reverse wraparound disabled) things are slightly different. With the cursor in column 80 when the next character is output, it appears that xterm places the character in column 80, leaves the cursor in column 80 and sets a flag to say the line is full. If another character is output then xterm goes to column 1 of the next line then outputs the character. If, on the other hand, a backspace is received then the cursor goes to column 79. The difference is easy to see by outputting the following sequences: 79 spaces, dot 79 spaces, backspace, dot 80 spaces, dot 80 spaces, backspace, dot All terminals agree that the first sequence puts a dot in the right hand column. All terminals agree that the second sequence puts a dot in the second column from the right. All terminals agree that the third sequence puts a dot in the left hand column of the next line. It's only the last sequence where there's a disagreement. Terminal emulators except xterm with reverse wraparound disabled put the dot in the right hand column (that is the positioning for 80 spaces plus backspace is equal to the positioning for 79 spaces). Xterm with reverse wraparound puts the dot in the second column from the right. Actually, the sample code in the comments above the rl_scroll_nextline shows the problem quite well: perl -we "print 1 x shift, qq(\b2\r3); sleep 2" 80 (I find I need to set $| to 1 to get that example to work). All the terminals except xterm with reverse wraparound disabled display: 31111111111111111111111111111111111111111111111111111111111111111111111111111112 xterm with reverse wraparound disabled displays: 31111111111111111111111111111111111111111111111111111111111111111111111111111121 I know enough about terminal emulation to know that handling the last column is tricky. So, I'm not going to say xterm's behaviour is wrong. All I'm going to say is that Term::ReadLine::Perl appears to be relying on behaviour that's peculiar to xterm with some options. -- Steven Singer Software Engineer Cambridge Silicon Radio Ltd. Tel: +44 (0)1223 692000 Fax: +44 (0)1223 692001 Churchill House, Cambridge Business Park, Cowley Road, Cambridge CB4 0WZ, UK Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom More information can be found at www.csr.com. Follow CSR on Twitter at http://twitter.com/CSR_PLC and read our blog at www.csr.com/blog