Bug #40536 for Catalyst-Runtime: Catalyst::Engine::FastCGI serving binary data

Thu Oct 30 12:44:37 2008 simonw [...] zynet.net - Ticket created

Subject:	Catalyst::Engine::FastCGI serving binary data
Date:	Thu, 30 Oct 2008 16:43:43 +0000
To:	bug-Catalyst-Runtime [...] rt.cpan.org
From:	Simon Waters <simonw [...] zynet.net>

In 5.7008 sub "write" gained these lines # XXX: We can't use Engine's write() method because syswrite # appears to return bogus values instead of the number of bytes # written: http://www.fastcgi.com/om_archive/mail-archive/0128.html # Prepend the headers if they have not yet been sent if ( my $headers = delete $self->{_header_buf} ) { " } In our case $buffer is binary data (image/jpeg). If $headers is in utf8 after this $buffer is UTF8 encoded, our jpeg data is now corrupt. If I replace "$buffer = $headers . $buffer;" with "*STDOUT->syswrite($headers);" it works as expected. I'm not sure it is a/the bug, or our usage of $c->res->body this is wrong. We don't see the same problem with the pajax MyApp_server.pl, only with FastCGI. Simon

Thu Oct 30 12:58:41 2008 AGRUNDMA [...] cpan.org - Correspondence added

The main reason for this code is to reduce the number of TCP packets, so headers and body are sent together. I don't have a problem putting the old code back but we should figure out if we can detect when it will fail and only send the extra syswrite if necessary.

Thu Oct 30 12:58:43 2008 The RT System itself - Status changed from 'new' to 'open'

Fri Oct 31 05:36:19 2008 simonw [...] zynet.net - Correspondence added

Subject:	Re: [rt.cpan.org #40536] Catalyst::Engine::FastCGI serving binary data
Date:	Fri, 31 Oct 2008 09:35:38 +0000
To:	bug-Catalyst-Runtime [...] rt.cpan.org
From:	Simon Waters <simonw [...] zynet.net>

On Thursday 30 October 2008 16:58:43 AGRUNDMA via RT wrote: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=40536 > > > The main reason for this code is to reduce the number of TCP packets, so > headers and body are sent together. I don't have a problem putting the > old code back but we should figure out if we can detect when it will > fail and only send the extra syswrite if necessary.

It will fail when headers is UTF8 encoded, and buffers is suppose to be binary data. I'm unclear why headers is UTF8 encoded in this instance. But since the HTTP standard allows arbitary characters in certain headers it has to be allowed for here I believe. Also I'm unclear why it is only the FASTCGI code failing here, might be worth contrasting with the other engines.

Mon Nov 03 06:08:29 2008 simonw [...] zynet.net - Correspondence added

Subject:	Re: [rt.cpan.org #40536] Catalyst::Engine::FastCGI serving binary data
Date:	Mon, 3 Nov 2008 11:08:11 +0000
To:	bug-Catalyst-Runtime [...] rt.cpan.org
From:	Simon Waters <simonw [...] zynet.net>

The workaround suggested is broken. Suggestions on how best to address this appreciated. We are trying "$buffer = Encode::encode_utf8($header) . $buffer" but that seems to fail on the live server for reasons unclear, but works on our test server. Simon

Wed Nov 05 09:22:02 2008 simonw [...] zynet.net - Correspondence added

Subject:	Re: [rt.cpan.org #40536] Catalyst::Engine::FastCGI serving binary data
Date:	Wed, 5 Nov 2008 14:21:48 +0000
To:	bug-Catalyst-Runtime [...] rt.cpan.org
From:	Simon Waters <simonw [...] zynet.net>

I'm having trouble reproducing this fully on our test server. The problem persists of the live server Debian Sarge, Apache 2.0.54, and Perl 5.8.8. The test server is Debian Etch, Apache 2.2.3, Perl 5.8.8. The workaround we are using now is to call "$c->res->write('');" before outputting binary content, which seems to work without side effects on both servers. I'm thinking there must be some version mismatch. But the bug report can be closed until further information is forthcoming.

Sun Dec 21 09:21:58 2008 DAKKAR [...] cpan.org - Correspondence added

I'm experiencing the same problems, on a clean install of MojoMojo: I get double-utf8-encoded output. I have added the line: utf8::encode($headers) if utf8::is_utf8($headers); before the "$buffer = $headers . $buffer;", and it seems to work. It's interesting to note that Catalyst::Engine::HTTP does not exhibit this problem.

Mon Nov 30 22:55:35 2009 bobtfish [...] bobtfish.net - Cc DAKKAR added

Mon Nov 30 22:56:00 2009 bobtfish [...] bobtfish.net - AdminCc AGRUNDMA added

Mon Nov 30 22:56:13 2009 bobtfish [...] bobtfish.net - Taken

Sat Jan 09 20:42:13 2010 bobtfish [...] bobtfish.net - Correspondence added

This is a bug in FCGI which is corrected in the FCGI-0.68_01 dev release.

Sat Jan 09 20:42:14 2010 bobtfish [...] bobtfish.net - Status changed from 'open' to 'resolved'