Bug #43889 for Win32-Clipboard: Win32::Clipboard::GetText() doesn't convert newlines

Thu Mar 05 16:45:46 2009 KWittrock [...] web.de - Ticket created

Subject:	Win32::Clipboard::GetText() doesn't convert newlines
Date:	Thu, 5 Mar 2009 17:06:44 +0100
To:	<bug-Win32-Clipboard [...] rt.cpan.org>
From:	"K. Wittrock" <KWittrock [...] web.de>

Dear maintainer of Win32::Clipboard, MS Windows uses \r\n for Newline, while Perl uses \n. So Win32::Clipboard::GetText() should perform a "$text =~ s/\r$//mg;" before returning the text. Otherwise the \r is treated as part of the text by Perl. I use Windows XP with SP3 and ActivePerl 5.8.8. I observed the error with Win32::Clipboard versions 0.52 and 0.55. Regards Klaus Wittrock

Fri Mar 06 07:00:27 2009 dbecoll [...] roadrunner.com - Correspondence added

Subject:	Re: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines
Date:	Fri, 06 Mar 2009 04:00:09 -0800
To:	bug-Win32-Clipboard [...] rt.cpan.org
From:	Bill Luebkert <dbecoll [...] roadrunner.com>

K. Wittrock via RT wrote: Show quoted text

> > MS Windows uses \r\n for Newline, while Perl uses \n. So Win32::Clipboard::GetText() > should perform a "$text =~ s/\r$//mg;" before returning the text. Otherwise the \r > is treated as part of the text by Perl.

That's kinda debatable. GetText returns the string contained in the clipboard - I see no conversion documented or implied. But it would be nice if it stated how it handled newlines just for this type of situation. The text in the clipboard could possibly be text without \r's too. If you always strip the \r, you eliminate the ability to get a DOS CRLF from the clipboard even if you wanted one. I'd rather leave it and document it or add a strip option than change it.

Fri Mar 06 07:00:27 2009 The RT System itself - Status changed from 'new' to 'open'

Sun Mar 08 16:34:03 2009 KWittrock [...] web.de - Correspondence added

Subject:	Re: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines
Date:	Sun, 8 Mar 2009 13:07:47 +0100
To:	<bug-Win32-Clipboard [...] rt.cpan.org>
From:	"K. Wittrock" <KWittrock [...] web.de>

Show quoted text

----- Original Message ----- From: "Bill Luebkert via RT" <bug-Win32-Clipboard@rt.cpan.org> To: <KWittrock@web.de> Sent: Friday, March 06, 2009 1:00 PM Subject: Re: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines

> <URL: http://rt.cpan.org/Ticket/Display.html?id=43889 > > > K. Wittrock via RT wrote:

>> >> MS Windows uses \r\n for Newline, while Perl uses \n. So Win32::Clipboard::GetText() >> should perform a "$text =~ s/\r$//mg;" before returning the text. Otherwise the \r >> is treated as part of the text by Perl.

> > That's kinda debatable. GetText returns the string contained in the > clipboard - I see no conversion documented or implied. But it would > be nice if it stated how it handled newlines just for this type of > situation. > > The text in the clipboard could possibly be text without \r's too.

A \n without a preceeding \r is not a line end marker on Windows. It is not touched by my suggestion (note the $ and the mg in the regex). To see the difference, open a text file that was created on Linux by Notepad.

> If you always strip the \r, you eliminate the ability to get a DOS > CRLF from the clipboard even if you wanted one. > > I'd rather leave it and document it or add a strip option than change > it.

Thank you for reacting so fast on my report. I thought over your arguments and I believe ttat adding a strip option would be a practical solution. I suggest that you add a method "Binary" to Clipboard to turn line end conversion on or off. Hopefully the default will be Binary off (= do convert line end markers from Windows \r\n to Perl \n). I think this will be what the user needs in most cases. Here are some examples of the trouble that is caused by not converting: chomp will not remove trailing whitespace, because it is no longer trailing. /a$/ will not match lines ending with "a", although you know there are some. Or: Open a text file by Notepad, use "ctrl-a ctrl-c" to send it to the clipboard, fetch it by GetText, and save it as a new file. Both files should be identical, but they aren't. At the end of each line you have \r\r\n (hex 0d0d0a). Of course I can do the "$text =~ s/\r$//mg;" by myself after each call to GetText. And I can automate this with a small subroutine myGetText. Then this should be told in the docu. But I would prefer a solution within GetText. Thank you very much for your kind help. Please feel free to contact me if you want additional info. Kind regards Klaus Wittrock

Mon Mar 09 06:01:04 2009 dbecoll [...] roadrunner.com - Correspondence added

Subject:	Re: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines
Date:	Mon, 09 Mar 2009 03:00:26 -0700
To:	bug-Win32-Clipboard [...] rt.cpan.org
From:	Bill Luebkert <dbecoll [...] roadrunner.com>

K. Wittrock via RT wrote: Show quoted text

> > A \n without a preceeding \r is not a line end marker on Windows. It is not touched by my suggestion (note the $ and the mg in the > regex). To see the difference, open a text file that was created on Linux by Notepad.

Not implying otherwise - just stating that \r\n may or may not be in the clipboard. Just because you're on Windoze, doesn't mean that there will always be Windoze text in the clipboard. Show quoted text

> Of course I can do the "$text =~ s/\r$//mg;" by myself after each call to GetText. And I can automate this with a small subroutine > myGetText. Then this should be told in the docu. But I would prefer a solution within GetText.

At least if they document it, they are covering themselves. I suspect changing the normal behavior could be problematic for some, so I would be against changing the default behavior.

Mon Mar 09 09:18:05 2009 KWittrock [...] web.de - Correspondence added

Subject:	Re: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines
Date:	Mon, 9 Mar 2009 14:16:39 +0100
To:	<bug-Win32-Clipboard [...] rt.cpan.org>
From:	"K. Wittrock" <KWittrock [...] web.de>

Show quoted text

----- Original Message ----- From: "Bill Luebkert via RT" <bug-Win32-Clipboard@rt.cpan.org> To: <KWittrock@web.de> Sent: Monday, March 09, 2009 11:01 AM Subject: Re: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines <URL: https://rt.cpan.org/Ticket/Display.html?id=43889 >

> K. Wittrock via RT wrote:

>> >> A \n without a preceeding \r is not a line end marker on Windows. It is not touched by my suggestion (note the $ and the mg in >> the >> regex). To see the difference, open a text file that was created on Linux by Notepad.

> > Not implying otherwise - just stating that \r\n may or may not be in the clipboard. > Just because you're on Windoze, doesn't mean that there will always be Windoze > text in the clipboard.

But the likelihood is certainly high. That's why I suggested to have Binary off as the default.

>> Of course I can do the "$text =~ s/\r$//mg;" by myself after each call to GetText. And I can automate this with a small >> subroutine >> myGetText. Then this should be told in the docu. But I would prefer a solution within GetText.

> > At least if they document it, they are covering themselves. I suspect changing > the normal behavior could be problematic for some, so I would be against changing > the default behavior.

Perl developers are often anxious about backward compatibility. Ok, go ahead.

Mon Mar 09 11:43:07 2009 dbecoll [...] roadrunner.com - Correspondence added

Subject:	Re: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines
Date:	Mon, 09 Mar 2009 08:41:46 -0700
To:	bug-Win32-Clipboard [...] rt.cpan.org
From:	Bill Luebkert <dbecoll [...] roadrunner.com>

K. Wittrock via RT wrote: Show quoted text

> > Perl developers are often anxious about backward compatibility. Ok, go ahead.

Go ahead and what ? I'm not the maintainer - just another user. All I'm saying is that the maintainer should document the newline behavior and possibly make an addition as you requested for an option to convert them.

Mon Mar 09 14:10:03 2009 KWittrock [...] web.de - Correspondence added

Subject:	Re: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines
Date:	Mon, 9 Mar 2009 19:08:32 +0100
To:	<bug-Win32-Clipboard [...] rt.cpan.org>
From:	"K. Wittrock" <KWittrock [...] web.de>

Show quoted text

----- Original Message ----- From: "Bill Luebkert via RT" <bug-Win32-Clipboard@rt.cpan.org> To: <KWittrock@web.de> Sent: Monday, March 09, 2009 4:43 PM Subject: Re: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines <URL: https://rt.cpan.org/Ticket/Display.html?id=43889 >

> K. Wittrock via RT wrote:

>> >> Perl developers are often anxious about backward compatibility. Ok, go ahead.

> > Go ahead and what ? I'm not the maintainer - just another user. > > All I'm saying is that the maintainer should document the newline behavior > and possibly make an addition as you requested for an option to convert > them.

Sorry, I wasn't aware of this. I got your email in response to my error report, so I assumed it came from the maintainer. Did you pass your suggestion to CPAN somehow, so it can be seen by the real maintainer? I don't know how to do this, I'm not very familiar with CPAN. What i wanted to say in my last reply is that backward compatibility should be weighted against other concerns such as comfort, ease of use, consistancy. I feel that backward compatibility sometimes causes confusing complexity in Perl. Maintainers should not only consider the past, but also the future. Kind regards Klaus Wittrock

Mon Mar 09 14:51:27 2009 jand [...] ActiveState.com - Correspondence added

Subject:	RE: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines
Date:	Mon, 9 Mar 2009 11:26:41 -0700
To:	<bug-Win32-Clipboard [...] rt.cpan.org>
From:	"Jan Dubois" <jand [...] activestate.com>

Show quoted text

> K. Wittrock via RT wrote:

> > Perl developers are often anxious about backward compatibility. Ok, > > go ahead.

> > Go ahead and what ? I'm not the maintainer - just another user. > > All I'm saying is that the maintainer should document the newline > behavior and possibly make an addition as you requested for an option > to convert them.

I'm maintaining Win32::Clipboard, and I'll eventually make some changes as indicated. I find it very useful to see these changes discussed a little in public first, which is why I've set up the RT queues to copy all these messages to libwin32@perl.org as well. I think the libwin32 modules are so old that they should not be changed in incompatible ways, even if their interfaces (and implementations) are sometimes quite gross. But they are a product of their time, have been "documented" in published books and sample code all over the net, so we should not break this body of existing code/knowledge. Win32::Clipboard is especially hard to change as it has taken up a lot of syntactic space that otherwise could be used to extend the functionality in a compatible way (it uses both object and function interfaces, and has already up a lot of the obvious method names). One other area that needs addressing is the CF_UNICODETEXT support, which currently also just returns the raw data, and not a Perl string with the correctly set up Unicode characters instead. There is then another issue about Perl's internal encoding schizophrenia, that sometimes it treas 8-bit data as being CP_ANSI encoded, and sometimes as Latin1, but I don't want to go into that here. Any update to the Win32::Clipboard interface should take these issues into account and allow for further extensibility in the future. My current way of thinking would be to leave the functional interface alone, but add additional options to the constructor of the Win32::Clipboard object. That should completely solve the backwards compatibility issue and still provide a more perlish interface for new code (you could mix the old and the new APIs in the same program, which is important if you use modules, which in turn use Win32::Clipboard). I won't get around to actually *doing* anything about this until later in April, but feel free to continue to file bugs / discuss changes; this will help in shaping the final implementation. Cheers, -Jan

Tue Mar 10 17:05:36 2009 KWittrock [...] web.de - Correspondence added

Subject:	Re: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines
Date:	Tue, 10 Mar 2009 19:31:48 +0100
To:	<bug-Win32-Clipboard [...] rt.cpan.org>
From:	"K. Wittrock" <KWittrock [...] web.de>

Show quoted text

----- Original Message ----- From: "jand@ActiveState.com via RT" <bug-Win32-Clipboard@rt.cpan.org> To: <KWittrock@web.de> Sent: Monday, March 09, 2009 7:51 PM Subject: RE: [rt.cpan.org #43889] Win32::Clipboard::GetText() doesn't convert newlines <URL: https://rt.cpan.org/Ticket/Display.html?id=43889 >

>> K. Wittrock via RT wrote:

>> > Perl developers are often anxious about backward compatibility. Ok, >> > go ahead.

>> >> Go ahead and what ? I'm not the maintainer - just another user. >> >> All I'm saying is that the maintainer should document the newline >> behavior and possibly make an addition as you requested for an option >> to convert them.

> > I'm maintaining Win32::Clipboard, and I'll eventually make some changes > as indicated. I find it very useful to see these changes discussed a > little in public first, which is why I've set up the RT queues to copy > all these messages to libwin32@perl.org as well. > > I think the libwin32 modules are so old that they should not be changed > in incompatible ways, even if their interfaces (and implementations) are > sometimes quite gross. But they are a product of their time, have been > "documented" in published books and sample code all over the net, so we > should not break this body of existing code/knowledge. > > Win32::Clipboard is especially hard to change as it has taken up a lot > of syntactic space that otherwise could be used to extend the > functionality in a compatible way (it uses both object and function > interfaces, and has already up a lot of the obvious method names). > > One other area that needs addressing is the CF_UNICODETEXT support, > which currently also just returns the raw data, and not a Perl string > with the correctly set up Unicode characters instead. There is then > another issue about Perl's internal encoding schizophrenia, that > sometimes it treas 8-bit data as being CP_ANSI encoded, and sometimes as > Latin1, but I don't want to go into that here. > > Any update to the Win32::Clipboard interface should take these issues > into account and allow for further extensibility in the future. > > My current way of thinking would be to leave the functional interface > alone, but add additional options to the constructor of the > Win32::Clipboard object. That should completely solve the backwards > compatibility issue and still provide a more perlish interface for new > code (you could mix the old and the new APIs in the same program, which > is important if you use modules, which in turn use Win32::Clipboard).

Thank you for your explanations. I admit that there are aspects of backward compatibility that I'm not aware with the small Perl scripts which I write.

> > I won't get around to actually *doing* anything about this until later > in April, .....

As I stated earlier, the conversion of line end markers isn't a high priority matter, particulary since I know how to do it afterwards. So April is more than I could expect. Greetings Klaus