Bug #55651 for CGI-Application-Dispatch-Server: [PATCH] Use CRLF instead of LF for response

Wed Mar 17 02:46:40 2010 nazri [...] healthiercompany.com - Ticket created

Subject:	[PATCH] Use CRLF instead of LF for response
Date:	Wed, 17 Mar 2010 14:45:22 +0800
To:	bug-CGI-Application-Dispatch-Server [...] rt.cpan.org
From:	Nazri <nazri [...] healthiercompany.com>

Hi there, Thanks for writing CGI::Application::Dispatch::Server. We are using it for development and found out that it does not work well with facebook server (the profile tab). We tracked down this problem and it turned out that facebook server do not handle HTTP response sent with LF line endings. Attached patch make CGI::Application::Dispatch::Server use CRLF line endings, as suggested in [1]. The version we are using is 0.52, and the attached patch is against that version. We're not sure if this blanket change is good for all cases. nazri + reza. [1] http://tools.ietf.org/html/rfc2616#section-6 -- hundred-and-one symptoms of being an internet addict: 101. U can read htis w/o ny porblm and cant figur eout Y its evn listd.

Message body is not shown because sender requested not to inline it.

Wed Mar 17 09:30:49 2010 mark [...] summersault.com - Correspondence added

Subject:	Re: [rt.cpan.org #55651] [PATCH] Use CRLF instead of LF for response
Date:	Wed, 17 Mar 2010 09:29:59 -0400
To:	bug-CGI-Application-Dispatch-Server [...] rt.cpan.org
From:	Mark Stosberg <mark [...] summersault.com>

Show quoted text

> Hi there, > > Thanks for writing CGI::Application::Dispatch::Server. > > We are using it for development and found out that it does not work well with > facebook server (the profile tab). > > We tracked down this problem and it turned out that facebook server > do not handle HTTP response sent with LF line endings. > > Attached patch make CGI::Application::Dispatch::Server use CRLF line > endings, as suggested in [1]. > > The version we are using is 0.52, and the attached patch is against that > version. > > We're not sure if this blanket change is good for all cases.

Thanks for the report. I agree with the principle of your patch, but this module is no longer maintained or recommended, in favor CGI::Application::Server. I recommend you switch to use that module (which is hopefully easy), or I can offer to make you a co-maintainer of this module, and you can upload a new release yourself which includes this improvement. Mark

Wed Mar 17 09:30:56 2010 The RT System itself - Status changed from 'new' to 'open'

Tue Mar 23 04:50:05 2010 nazri [...] healthiercompany.com - Correspondence added

Subject:	Re: [rt.cpan.org #55651] [PATCH] Use CRLF instead of LF for response
Date:	Tue, 23 Mar 2010 16:49:52 +0800
To:	"mark [...] summersault.com via RT" <bug-CGI-Application-Dispatch-Server [...] rt.cpan.org>
From:	Nazri <nazri [...] healthiercompany.com>

On 17/03/2010, mark@summersault.com via RT wrote: Show quoted text

> Thanks for the report. I agree with the principle of your patch, but > this module is no longer maintained or recommended, in favor > CGI::Application::Server. > > I recommend you switch to use that module (which is hopefully easy), or > I can offer to make you a co-maintainer of this module, and you can > upload a new release yourself which includes this improvement.

I've tried running my application using CGI::Application::Server. Following are the hiccups, or maybe I'm just stupid :) 1. From CGI::Application::Server perldoc: $server->entry_points({ '/' => 'MyCGIApp::DefaultApp', '/index.cgi' => 'MyCGIApp', '/admin' => 'MyCGIApp::Admin', '/account' => 'MyCGIApp::Account::Dispatch', '/users' => $object, '/static' => '/usr/local/htdocs', }); Looking at CGI::Application::Server:handle_request it seems like the first four entries will not work at all because all four of them fails the "Is-it-a-directory-or-an-instance-of-a-dispatcher" test as all four of them are string scalars. To work around this I had to OO-ize my dispatcher, created an instance of it ($object) and used an entry similar to the fifth entry above. 2. I had to add an explicit entries for each of top-level directories where static content are being served. I was under the impression that request paths that are not handled by the dispatcher will be automagically treated as static content, that is, I had to write: my $doc_root = './htdocs'; $server->document_root($doc_root); $server->entry_points({ '/' => $dispatch, '/stylesheets' => $doc_root, '/javascript' => $doc_root, '/images' => $doc_root, }); instead of the simpler: $server->document_root('./htdocs'); $server->entry_points({ '/' => $dispatch, }); While I managed to use CGI::Application::Server successfully for my application, it still suffer from the same problem with the "header-sent-using-LF-instead-of-CRLF" issue. Going through these 'exercises' I now think that the proper place to fix this issue might be in HTTP::(Response|Headers)::as_string, instead of CGI::Application::Server or CGI::Application::Dispatch::Server. nazri -- hundred-and-one symptoms of being an internet addict: 135. You cut classes or miss work so you can stay home and browse the web.

Tue Mar 23 09:40:13 2010 mark [...] summersault.com - Correspondence added

Subject:	Re: [rt.cpan.org #55651] [PATCH] Use CRLF instead of LF for response
Date:	Tue, 23 Mar 2010 09:40:02 -0400
To:	bug-CGI-Application-Dispatch-Server [...] rt.cpan.org
From:	Mark Stosberg <mark [...] summersault.com>

Nazri, I'm sorry your conversion to CGI::Application::Server wasn't as easy as you hoped. Show quoted text

> 2. I had to add an explicit entries for each of top-level directories where > static content are being served. I was under the impression that request > paths that are not handled by the dispatcher will be automagically treated > as static content, that is, I had to write: > > my $doc_root = './htdocs'; > $server->document_root($doc_root); > $server->entry_points({ > '/' => $dispatch, > '/stylesheets' => $doc_root, > '/javascript' => $doc_root, > '/images' => $doc_root, > }); > > instead of the simpler: > > $server->document_root('./htdocs'); > $server->entry_points({ > '/' => $dispatch, > });

I'm not certain about the intended behavior for CGI::Application::Server is. You could contact the developers or file a "wish" report if you think it should be changed. Show quoted text

> While I managed to use CGI::Application::Server successfully for my > application, it still suffer from the same problem with the > "header-sent-using-LF-instead-of-CRLF" issue.

CGI::Application uses CGI.pm to build the headers and send the linebreaks between the header and the body. In turn, it decides which character sequence to use based on which platform it is running on. Show quoted text

> Going through these 'exercises' I now think that the proper place to fix this > issue might be in HTTP::(Response|Headers)::as_string, instead of > CGI::Application::Server or CGI::Application::Dispatch::Server.

Having looked closely at the code for HTTP::Headers, I think you'll find that is already spec-compliant. I suggest adding some debugging code to your copy of CGI.pm in the "header()" method to see if that is the source of your unexpected linebreaks. ( But, as a maintainer of CGI.pm, I don't expect you'll find a bug there ). You could also try to see if you could write a test case against just CGI.pm alone or HTTP::Headers alone and see if they fail. Mark

Tue Mar 23 22:08:17 2010 nazri [...] healthiercompany.com - Correspondence added

Subject:	Re: [rt.cpan.org #55651] [PATCH] Use CRLF instead of LF for response
Date:	Wed, 24 Mar 2010 10:08:01 +0800
To:	"mark [...] summersault.com via RT" <bug-CGI-Application-Dispatch-Server [...] rt.cpan.org>
From:	Nazri <nazri [...] healthiercompany.com>

On 23/03/2010, mark@summersault.com via RT wrote: Show quoted text

> I'm not certain about the intended behavior for CGI::Application::Server > is. You could contact the developers or file a "wish" report if you > think it should be changed.

Thank you for the suggestion. I don't see immediate need on my part to proceed with the above at this moment. Show quoted text

> CGI::Application uses CGI.pm to build the headers and send the > linebreaks between the header and the body. In turn, it decides which > character sequence to use based on which platform it is running on.

Looking at CGI.pm, it defaults to CRLF (which is spec-compliant) when the platform is not VMS or the platform is not using EBCDIC character encoding. The author commented that it is 'really annoying'. I think the solution he implemented is pragmatic. Show quoted text

> Having looked closely at the code for HTTP::Headers, I think you'll find > that is already spec-compliant.

Are you sure? The HTTP::Headers::as_string (as of 5.827) defaults to LF ("\n") as the header terminator, which to me, is not rfc2616 [1]- compliant and not pragmatic when it is used for sending an HTTP request. It seems like the library expects that their users be aware of the LF-CRLF issues when it comes to sending an HTTP request. Maybe it would be more pragmatic if it made use the same logic as CGI.pm when choosing a default for the header terminator. Hmm maybe I should bring this issue to the library authors. They may have their own reason for the current behavior, or it is just a bug. Show quoted text

> ( But, as a maintainer of CGI.pm, I don't expect you'll find a bug there ).

You are right there. CGI.pm is all good. Thank you. nazri [1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html#sec2.2 -- hundred-and-one symptoms of being an internet addict: 192. Your boss asks you to "go fer" coffee and you come up with 235 FTP sites.

Wed Mar 24 08:13:30 2010 mark [...] summersault.com - Correspondence added

Subject:	Re: [rt.cpan.org #55651] [PATCH] Use CRLF instead of LF for response
Date:	Wed, 24 Mar 2010 08:13:16 -0400
To:	bug-CGI-Application-Dispatch-Server [...] rt.cpan.org
From:	Mark Stosberg <mark [...] summersault.com>

Show quoted text

> > Having looked closely at the code for HTTP::Headers, I think you'll find > > that is already spec-compliant.

> > Are you sure? The HTTP::Headers::as_string (as of 5.827) defaults to LF > ("\n") as the header terminator, which to me, is not rfc2616 [1]- > compliant and not pragmatic when it is used for sending an HTTP request. > > It seems like the library expects that their users be aware of the > LF-CRLF issues when it comes to sending an HTTP request. Maybe it would > be more pragmatic if it made use the same logic as CGI.pm when choosing > a default for the header terminator. > > Hmm maybe I should bring this issue to the library authors. They may > have their own reason for the current behavior, or it is just a bug.

I think you are right here. This second reference also suggests you are correct. In RFC 2616, see "3.7.1 Canonicalization and Text Defaults" http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html Show quoted text

> [1] http://www.w3.org/Protocols/rfc2616/rfc2616-sec2.html#sec2.2

HTTP::Headers is on github if you'd like to submit a patch that way: http://github.com/gisle/libwww-perl I think you'll find Gisle is receptive. Mark