Bug #104754 for App-Wallflower: Scramble Images

Thu May 28 13:46:25 2015 ASB [...] cpan.org - Ticket created

Subject:

Scramble Images

When running wallflower, the image is stored to static/image.jpg, but it's content is all messed up. Please find attached an example. This is the command I used: wallflower --application script.cgi --dir static/ Here is an example application: #!perl use Mojolicious::Lite; get '/' => sub { my $c = shift; $c->render(template => 'index'); }; app->start; __DATA__ @@ index.html.ep <html> <body> <h1>Image</h1> <img src="/image.jpg" /> </body> </html> @@ image.jpg (base64) R0lGODlhoACgAPfKAGet+oe16PTy72Sq94+55/fz8Gyv+cdQUKenp4CAgGex/qVsfoKWzX19fV5eXl1dXVRUVNzc3MpNSubm5s5NSsbGxouLi7GxsZKSkmxsbGhoaFZWVmOf4sZNTWuv+ri4uNHR0W6q8Wa1/6F2lMlOTJycnHJycs1KR81KRllZWWuu971aY4i153Kk2nV1dWum5W2x/dvb28nJyWRkZG2x/Gqv+G2m5oeHh3C3/2mv+ZmZmby8vJC55sxeXui3t2Sp9vj07leFuGag4s1hYc1kZJC66Pv27mah42Kf4vf08G+q63Gq6sxfX8pKSMpaWWay/stcXG6y/sdOTeezs2qt955xoKdnkmat+vfz74e152yv+G6y/GSp98lKSGyu+ERgf8NYXlmJvrFbgmS1/2ix/I+DuV2Sy2yy/Gy0/2Od22mo72qq6tQxRm6p759yitFGQGe4/9RHP2W0/VqMwm2y/sxLRm2x+12RykJceViIvaN0kG2w+22r84+Du12TzUhnimmw+dMzSWmp8Gqu916SzWys9e/v73Sj6Gyv+miw+3Cs9IqJw46EqJl4q2Op9muw+6Jsmmus9X+Y0lR/rmCX1Hif3rtPcGey/Gel6m+2/2ex/Wmw+2K4/4KVzGqu+G+1//n28pO961+U0V6Uz1N9rKZ1kmaz/maj56BvnnOj6MBXXsVEYH2X2Gex/EVigWez/FJ9q7tQcWym5W6y/YOUymyz/2ay/W2y/Wuw+mut9VeGu3Wi5m6k4WWg4mKb2aZse6lpeGK3/6FunNBLRpZ7rmuv+VNTU2yl5Gut9vDw8P///wAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACH/C05FVFNDQVBFMi4wAwEAAAAh/wtYTVAgRGF0YVhNUDw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaUh6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IkFkb2JlIFhNUCBDb3JlIDUuMC1jMDYwIDYxLjEzNDc3NywgMjAxMC8wMi8xMi0xNzozMjowMCAgICAgICAgIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9IiIgeG1sbnM6eG1wPSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvIiB4bWxuczp4bXBNTT0iaHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLyIgeG1sbnM6c3RSZWY9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9zVHlwZS9SZXNvdXJjZVJlZiMiIHhtcDpDcmVhdG9yVG9vbD0iQWRvYmUgUGhvdG9zaG9wIENTNSBXaW5kb3dzIiB4bXBNTTpJbnN0YW5jZUlEPSJ4bXAuaWlkOjEzQzJENEEzNTU1QjExRTNBNjVDQzI4MjMzM0E4MzhCIiB4bXBNTTpEb2N1bWVudElEPSJ4bXAuZGlkOjEzQzJENEE0NTU1QjExRTNBNjVDQzI4MjMzM0E4MzhCIj4gPHhtcE1NOkRlcml2ZWRGcm9tIHN0UmVmOmluc3RhbmNlSUQ9InhtcC5paWQ6MTNDMkQ0QTE1NTVCMTFFM0E2NUNDMjgyMzMzQTgzOEIiIHN0UmVmOmRvY3VtZW50SUQ9InhtcC5kaWQ6MTNDMkQ0QTI1NTVCMTFFM0E2NUNDMjgyMzMzQTgzOEIiLz4gPC9yZGY6RGVzY3JpcHRpb24+IDwvcmRmOlJERj4gPC94OnhtcG1ldGE+IDw/eHBhY2tldCBlbmQ9InIiPz4B//79/Pv6+fj39vX08/Lx8O/u7ezr6uno5+bl5OPi4eDf3t3c29rZ2NfW1dTT0tHQz87NzMvKycjHxsXEw8LBwL++vby7urm4t7a1tLOysbCvrq2sq6qpqKempaSjoqGgn56dnJuamZiXlpWUk5KRkI+OjYyLiomIh4aFhIOCgYB/fn18e3p5eHd2dXRzcnFwb25tbGtqaWhnZmVkY2JhYF9eXVxbWllYV1ZVVFNSUVBPTk1MS0pJSEdGRURDQkFAPz49PDs6OTg3NjU0MzIxMC8uLSwrKikoJyYlJCMiISAfHh0cGxoZGBcWFRQTEhEQDw4NDAsKCQgHBgUEAwIBAAAh+QQFMgDKACwAAAAAoACgAAAI/wBb2DhGsKDBgwgTKlzIsKHDhxAjSiQoi5ebXwsyatyYERijNS1e0DBAsqTJkyhTqlzJsqXLlzBjvqSBo+asR4rADKPAk8KJnyd6UohTCs6LYwaQ1eAUbEyNJ8iiSp1KtarVq1izat3KtatXqoh6ffmCR9SZECtIHFjbgUiPHkQ6rD2AYoQIggZypIplSVibPoC+Ch5MuLBhql4+ZYqSSyoMP7DSBNGFBq1atlB8+IAid23du0hNLVpVxooYSHIOq17N2rCBU39cTbIjlcYoUr6C5KlleW4HJlOmMOlM1y5eZHxeVakSCJWt1tCjS0esZk4YSlqkqhBEyMwdTMV6r/+VMsSJkyFS5n4+3oqVlUZsiF2aTr++ahV0ouyhqgIGjFtaeBKCKnVIYKAEXTTRRBcHSvCGHqAllcMuh5CRSCT2ZajhdIVI0gkDIIYoIoi0VILLcUrlgMwgKmzo4ouHaaLAjDTWSOMTmxiAFxVcDODjj0AGKeSQRBZp5JFIJqmkkj848sOSS3JBhY7HeLAEDwRkqeWWXHbp5ZdghinmmGSWWWYRoRRhppk8LOEBQQCwYEQBdNZp55145qnnnnz26eefgP6JRRKgJIFFoIAawQIAcAYgQDKQRirppJRWaumlmGaq6aacduqppwIEwOgxADj66amopqrqqqxqGuqopT7/2uqstNZqq6qvNirrrbz26qutuZJq6q/EFmvspsHGeuyyzC6b7LDNRittrc/uOu212H5abbbcdpvptt6GK24y4I5rLrblnqtus+mu626xuR6h7Lv0EvvqERzMW+++t77KQb7Q8itwq/4CbO3ACKPqLxL6JuwwqKIiIUTDD1f8rahCCHuwxRxT2m7HIEP6ccgdj0yyxSaf/HDKKifMcssDvwwzvzLPXG/NNr+Lc87r7szzuT7/PG7QQodLdNHdHo10tkovfW3TTksLddTsiqor1ftOjfWxWm8Nr9Uae60z2BSLPTTZAZt9Nqxpq2002hu7nTTccpvbdd3U0o3322zH/733tHf/zWrgguOqd+HoHk6vMYyfy7gxvBJ9QwMaPG7MAw0gEMOvjxfbgOWgh94ApJ33qzinoYOOgK+l//p56qmPnkzrefd96uM7VKA7Aq8bg0GvtPd6AQLEF/948cVfQHrjptv+afDJXPB4BbxCb6z1y0Pe/NXPMz8pAoz/fiv2xJJPvuHOe2p9DIybUL33njYgO6rmw08r0difn6r+ltqvvv+R4t+p8AfA2V2OUjvAQOU0cAPlTYpxq4sBBh6gPQNW8H+qqt8Fk7EDxj0ABKki4AYhBQLGzS8ZE+id5RowAUlBEAQU9J4AKVXATWlQUjC8HAhDeDobAvAGEIzUBP9McLkLtBAEJTChC42BgMoZQ34BrCGmpJipG0JqAjF0IA/T1ynoJfFyLYQUBnT4PcZRb3mVw0AYozhCH2awhqUbIuO0uEXu/c8YF9BdBZqIu0ixj4mVoqD4LHhCNtKvjaiDI/OAiEeC9VBTsGOcBs4IKfAZY3OU+pwG2LiDSs3wgYj0YSg790U6os+OXZTk40xQAkpGaoyRbF3+qGjJWFpudZBUJB6VOCsRbkqFkTSkJ2lpS9WJ8lKPi+EHe/nIKlIRUq9LJCI/ucQ3jvJxjDTBGk8Ztu6FUlJfxKQzp/nMYVoTmXNMBiMHyc2y5fKbkZJeI9/Zv3LSEJ5T1OUVszj/uGbmE58olOQ260lOgN7znASNVAU8uMM6dvOOnPqiGseZ0NsZdJjXvOAXtdnOttFzU3I0hgYQoEcEYOCEs7xoNfenT0kR0Xcd9ds/O5XCYArzniqNIkIxOqkcznOA/kRnTiugQA82oJVLLOghd4pTSllymUDlIuKkFtSpfk2qVmUW4bLaqa1yFVlV/WrkwipWYJG1rPc7K1odidW19sqrbvWYWuPqUHfStXaovOv2HqrXvdq1r/1sK2ADm9fBsrWwhl0VXPu6WL029q6PpWtk4zpZt1Z2rZdFa2bLulmxdvarn+VqaLM6WquWdqqnRVxqC7dawbX2b6/dW2zxNtu6utVWbrd1W27Vtluz9VZsv/VacLc2XKwVl2rHjVpynbbcpTUXac8tWnSFNt2fVZdn181Zdm223Zl1F2bfbVl4VTbek5WXZOcNWbJYAAQBuPe98I2vfOdL3/ra9774za9+98tf/gJhUQTxgBKyEIACG/jACE6wghfM4AY7+MEQjrCEJzzhLCjhTS/YggeuAIAOe/jDIA6xiEdM4hKb+MQoTrGKV7ziK3hgCy8QyERmTOMa2/jGOJaIDVoQEAAh+QQFMgDKACwWAEIAcwAXAAAI/wBnONigrKAyCA4SyDDIsKHDhxAjSpwI0QFFgxYvXkygsaPHjxU7ZvyoA4HJBCMzgFzJkqKFBDBjGowZ00JLhjYLIrjJs6fPjxyVqfxJtKjRhcpSGOWZLNnSp1BZNo1KFIJDHRkIbpiR0yFHGRms+pxK9WpBCB8mplU2UlmEthgjeP0g9ifZsgbpHlwrcUbBoG6VQrAg94OLgnCVJSDINvHKu3gjiO0a8fBBuQVVom0YdCdDghkwj3WKN3BBygYtmESw2KAOg0gBM7Q6lKHjm5Cp+lWGWuIGz38LIm1okTHD10VzQ7Xc26DxFC6AG6xdWlnT69izYy+a87bI6ga1i2bPXlTs5pbewVsnXXZ3CtEgLQ9XH559VJu7qX/M2Ry8cqiSncYSZhvAR99/UHl2XnyZGeiffXhZ9t5KESilzAYJrJZABukRhWBUFurX0Vv0NfQhVHrx1hICWZ3lQHSlGWJIiUGhFRAAOw== __END__

Subject:

image.jpg

Download image.jpg
image/jpeg 3.8k

Wed Jun 03 03:11:44 2015 book [...] cpan.org - Correspondence added

Subject:	Re: [rt.cpan.org #104754] Scramble Images
Date:	Wed, 3 Jun 2015 09:11:35 +0200
To:	Alexander Becker via RT <bug-App-Wallflower [...] rt.cpan.org>
From:	"Philippe Bruhat (BooK)" <book [...] cpan.org>

On Thu, May 28, 2015 at 01:46:26PM -0400, Alexander Becker via RT wrote: Show quoted text

> > This is the command I used: > wallflower --application script.cgi --dir static/ >

I copy/pasted your application in a script.cgi file on my Linux box, and ran: $ wallflower -a script.cgi -d static 200 / => static/index.html [74] 200 /image.jpg => static/image.jpg [3973] $ file static/image.jpg static/image.jpg: GIF image data, version 89a, 160 x 160 $ identify static/image.jpg static/image.jpg[0] GIF 160x160 160x160+0+0 8-bit sRGB 256c 3.97KB 0.000u 0:00.000 static/image.jpg[1] GIF 115x23 160x160+22+66 8-bit sRGB 256c 3.97KB 0.000u 0:00.000 So this is a GIF image with two frames. Looking at it with my browser, it looks fine (shows a Tk window with the text "Perl + Tk" with a blinking "+"). Running the application in parallel with `plackup -p 5000 script.cgi`, I did: $ lwp-request -des http://localhost:5000/image.jpg 200 OK Date: Tue, 02 Jun 2015 22:38:57 GMT Date: Tue, 02 Jun 2015 22:38:57 GMT Accept-Ranges: bytes ETag: "beb5ffb6b02170dea496b600198ee13e" Server: HTTP::Server::PSGI Content-Length: 3973 Content-Type: image/jpeg Last-Modified: Tue, 02 Jun 2015 22:38:41 GMT Client-Date: Tue, 02 Jun 2015 22:38:57 GMT Client-Peer: 127.0.0.1:5000 Client-Response-Num: 1 Apart from the fact that a GIF image is served with a content-type of image/jpeg, I didn't see anything weird. I then downloaded the attached image, and compared it with the one downloaded by wallflower: $ ls -l image.jpg static/image.jpg -rw-r----- 1 book book 3978 Jun 3 00:52 image.jpg -rw-r--r-- 1 book book 3973 Jun 3 00:40 static/image.jpg So there's 5 extra bytes, it seems. When comparing two hexdumps (basically comparing the text output of running the `hexdump` command on both files), the first difference I found was an extra 0x0d... Could if be an issue with DOS/Unix end-of-lines? (You're on Windows, aren't you?) What about opening the file for writing in binary mode? Does the following patch work for you? diff --git a/lib/Wallflower.pm b/lib/Wallflower.pm index b34aa34..9771c8a 100644 --- a/lib/Wallflower.pm +++ b/lib/Wallflower.pm @@ -125,7 +125,7 @@ sub get { # get a file to save the content in my $dir = ( $file = $target )->dir; $dir->mkpath if !-e $dir; - open my $fh, '>', $file or croak "Can't open $file for writing: $!"; + open my $fh, '> :raw :bytes', $file or croak "Can't open $file for writing: $!"; # copy content to the file if ( ref $content eq 'ARRAY' ) { -- Philippe Bruhat (BooK) When the employee is a fool, so is the employer. (Moral from Groo The Wanderer #26 (Epic))

Wed Jun 03 03:11:45 2015 The RT System itself - Status changed from 'new' to 'open'

Wed Jun 03 13:51:18 2015 ASB [...] cpan.org - Correspondence added

Am Mi 03. Jun 2015, 03:11:44, BOOK schrieb: Show quoted text

> On Thu, May 28, 2015 at 01:46:26PM -0400, Alexander Becker via RT > wrote:

> > > > This is the command I used: > > wallflower --application script.cgi --dir static/ > >

> > I copy/pasted your application in a script.cgi file on my Linux box, > and ran: > > $ wallflower -a script.cgi -d static > 200 / => static/index.html [74] > 200 /image.jpg => static/image.jpg [3973] > > $ file static/image.jpg > static/image.jpg: GIF image data, version 89a, 160 x 160 > > $ identify static/image.jpg > static/image.jpg[0] GIF 160x160 160x160+0+0 8-bit sRGB 256c 3.97KB > 0.000u 0:00.000 > static/image.jpg[1] GIF 115x23 160x160+22+66 8-bit sRGB 256c 3.97KB > 0.000u 0:00.000 > > So this is a GIF image with two frames. Looking at it with my browser, > it > looks fine (shows a Tk window with the text "Perl + Tk" with a > blinking "+"). > > Running the application in parallel with `plackup -p 5000 script.cgi`, > I did: > > $ lwp-request -des http://localhost:5000/image.jpg > 200 OK > Date: Tue, 02 Jun 2015 22:38:57 GMT > Date: Tue, 02 Jun 2015 22:38:57 GMT > Accept-Ranges: bytes > ETag: "beb5ffb6b02170dea496b600198ee13e" > Server: HTTP::Server::PSGI > Content-Length: 3973 > Content-Type: image/jpeg > Last-Modified: Tue, 02 Jun 2015 22:38:41 GMT > Client-Date: Tue, 02 Jun 2015 22:38:57 GMT > Client-Peer: 127.0.0.1:5000 > Client-Response-Num: 1 > > Apart from the fact that a GIF image is served with a content-type of > image/jpeg, I didn't see anything weird. > > I then downloaded the attached image, and compared it with the one > downloaded by wallflower: > > $ ls -l image.jpg static/image.jpg > -rw-r----- 1 book book 3978 Jun 3 00:52 image.jpg > -rw-r--r-- 1 book book 3973 Jun 3 00:40 static/image.jpg > > So there's 5 extra bytes, it seems. When comparing two hexdumps > (basically > comparing the text output of running the `hexdump` command on both > files), > the first difference I found was an extra 0x0d... Could if be an issue > with DOS/Unix end-of-lines? (You're on Windows, aren't you?) What > about > opening the file for writing in binary mode? > > Does the following patch work for you? > > diff --git a/lib/Wallflower.pm b/lib/Wallflower.pm > index b34aa34..9771c8a 100644 > --- a/lib/Wallflower.pm > +++ b/lib/Wallflower.pm > @@ -125,7 +125,7 @@ sub get { > # get a file to save the content in > my $dir = ( $file = $target )->dir; > $dir->mkpath if !-e $dir; > - open my $fh, '>', $file or croak "Can't open $file for > writing: $!"; > + open my $fh, '> :raw :bytes', $file or croak "Can't open > $file for writing: $!"; > > # copy content to the file > if ( ref $content eq 'ARRAY' ) {

Yes, it worked! Once the file was written in binary mode, the image could be displayed. You were also right about the OS, I'm on Windows here.

Wed Jun 03 16:58:58 2015 book [...] cpan.org - Correspondence added

On Wed Jun 03 13:51:18 2015, ASB wrote: Show quoted text

> Am Mi 03. Jun 2015, 03:11:44, BOOK schrieb:

> > > > Does the following patch work for you? > >

> > Yes, it worked! > Once the file was written in binary mode, the image could be displayed. >

So, after further investigation, the :bytes layer is unnecessary, since the content MUST be bytes according to the PSGI standard: https://metacpan.org/pod/distribution/PSGI/PSGI.pod#Body I'll apply the patch with only :raw and make a release shortly. Thanks! -- BooK

Wed Jun 03 17:36:29 2015 ASB [...] cpan.org - Correspondence added

Am Mi 03. Jun 2015, 16:58:58, BOOK schrieb: Show quoted text

> On Wed Jun 03 13:51:18 2015, ASB wrote:

> > Am Mi 03. Jun 2015, 03:11:44, BOOK schrieb:

> > > > > > Does the following patch work for you? > > >

> > > > Yes, it worked! > > Once the file was written in binary mode, the image could be displayed. > >

> > So, after further investigation, the :bytes layer is unnecessary, since > the content MUST be bytes according to the PSGI standard: > https://metacpan.org/pod/distribution/PSGI/PSGI.pod#Body > > I'll apply the patch with only :raw and make a release shortly. > > Thanks! > > -- BooK

I quickly confirmed it. Omitting the :bytes option yields the same results. Thanks for patching!

Wed Jun 24 04:05:10 2015 book [...] cpan.org - Status changed from 'open' to 'resolved'

Wed Jun 24 04:05:10 2015 book [...] cpan.org - Fixed in 1.006 added