Bug #104449 for App-Wallflower: url option does not work

Thu May 14 16:18:00 2015 ASB [...] cpan.org - Ticket created

Subject:

url option does not work

I have a Plack application that generates HTML content with links like "/cgi-bin/myapp/index.cgi/page.html". After running wallflower, I get HTML pages where the link is reduced to "/page.html". This is not what I want, because the pages will be located in the folder /myapp (e.g. http://127.0.0.1/myapp/page.html). This was the command used: wallflower --application "C:\Apache\cgi-bin\myapp\index.cgi" So I tried the --url option to change the links (NB: localhost is the target server): wallflower --application "C:\Apache\cgi-bin\myapp\index.cgi" --url "http://localhost/myapp" But: doing so only yields a "404 /" What am I doing wrong? Could you eventually provide an example of the URL option in the tutorial (incl. a before and after example)? Here is some additional info: * Perl 5.16.3 * Windows 8.1 x64 * App::Wallflower 1.005

Fri May 15 00:59:21 2015 book [...] cpan.org - Correspondence added

On Thu May 14 16:18:00 2015, ASB wrote: Show quoted text

> I have a Plack application that generates HTML content with links like > "/cgi-bin/myapp/index.cgi/page.html". > > After running wallflower, I get HTML pages where the link is reduced > to "/page.html". > This is not what I want, because the pages will be located in the > folder /myapp (e.g. http://127.0.0.1/myapp/page.html). > > This was the command used: > wallflower --application "C:\Apache\cgi-bin\myapp\index.cgi" > > So I tried the --url option to change the links (NB: localhost is the > target server): > wallflower --application "C:\Apache\cgi-bin\myapp\index.cgi" --url > "http://localhost/myapp" > > But: doing so only yields a "404 /" > > What am I doing wrong? > Could you eventually provide an example of the URL option in the > tutorial (incl. a before and after example)?

The url option is for mounting Plack applications, as done by Plack::App::URLMap. It does not influence URL produced by your application, just the URL the application will reply to. Could you show me an example of your application (just a small subset that exhibits the behaviour you're describing)? That way, I could better understand what's happening, and provide a correct answer. Once we've got an answer to your question, I can use it to produce an example in the tutorial. Regards, -- BooK

Fri May 15 00:59:22 2015 The RT System itself - Status changed from 'new' to 'open'

Fri May 15 06:45:22 2015 ASB [...] cpan.org - Correspondence added

Am Fr 15. Mai 2015, 00:59:21, BOOK schrieb: Show quoted text

> On Thu May 14 16:18:00 2015, ASB wrote:

> > I have a Plack application that generates HTML content with links like > > "/cgi-bin/myapp/index.cgi/page.html". > > > > After running wallflower, I get HTML pages where the link is reduced > > to "/page.html". > > This is not what I want, because the pages will be located in the > > folder /myapp (e.g. http://127.0.0.1/myapp/page.html). > > > > This was the command used: > > wallflower --application "C:\Apache\cgi-bin\myapp\index.cgi" > > > > So I tried the --url option to change the links (NB: localhost is the > > target server): > > wallflower --application "C:\Apache\cgi-bin\myapp\index.cgi" --url > > "http://localhost/myapp" > > > > But: doing so only yields a "404 /" > > > > What am I doing wrong? > > Could you eventually provide an example of the URL option in the > > tutorial (incl. a before and after example)?

> > The url option is for mounting Plack applications, as done by > Plack::App::URLMap. It does not influence URL produced by your > application, just the URL the application will reply to. > > Could you show me an example of your application (just a small subset > that exhibits the behaviour you're describing)? That way, I could > better understand what's happening, and provide a correct answer. > > Once we've got an answer to your question, I can use it to produce > an example in the tutorial. > > Regards, > > -- BooK

Hi BooK, please find below a working sample: [snip] #!perl use Mojolicious::Lite; any '/' => sub { my $c = shift; $c->render(template => "index"); }; any '/page.html' => sub{ my $c = shift; $c->render(text => 'this is a page'); }; app->start; __DATA__ @@ index.html.ep <html> <p>This is a <%= link_to '/page.html' => begin %>link to page</a><% end %>.</p> </html> [/snip]

Tue May 19 05:26:19 2015 book [...] cpan.org - Correspondence added

Thanks for the code example. First, you were using the --url option correctly. :-) Here are the things I tried: $ wallflower --application app.psgi --dir static/ 200 / => static/index.html [91] 200 /page.html => static/page.html [14] Using the --url option to "mount" the app under /myapp: $ wallflower --application app.psgi --dir static/ --url http://localhost/myapp 404 / So, same issue as you had. Until I realized that this is the correct behaviour, because (thanks to the --url option), your application now answers to the /myapp URL, and not / any more. wallflower starts spidering the application from /, but you can pass it a list of URL on the command-line: $ wallflower --application app.psgi --dir static/ --url http://localhost/myapp /myapp/ 200 /myapp/ => static/myapp/index.html [97] 200 /myapp/page.html => static/myapp/page.html [14] And it now works as expected! Note that the initial URL *has* to be "/myapp/" with a final slash, to create a directory "myapp" with an "index.html" file inside. If you forget the final slash, you'll get an error: $ wallflower --application app.psgi --dir static/ --url http://localhost/myapp /myapp 200 /myapp => static/myapp [97] Can't open static/myapp/page.html for writing: Not a directory at /home/book/perl5/lib/perl5/App/Wallflower.pm line 130. which is caused by the fact that wallflower puts the result of requesting "/myapp" into the "static/myapp" file, and later on can't create the "static/myapp" directory to put the content of the "/myapp/page.html" URL in it. Regards, -- BooK

Fri May 22 18:18:01 2015 ASB [...] cpan.org - Correspondence added

Am Di 19. Mai 2015, 05:26:19, BOOK schrieb: Show quoted text

> Thanks for the code example. > > First, you were using the --url option correctly. :-) > > Here are the things I tried: > > $ wallflower --application app.psgi --dir static/ > 200 / => static/index.html [91] > 200 /page.html => static/page.html [14] > > Using the --url option to "mount" the app under /myapp: > > $ wallflower --application app.psgi --dir static/ --url > http://localhost/myapp > 404 / > > So, same issue as you had. Until I realized that this is the correct > behaviour, > because (thanks to the --url option), your application now answers to > the /myapp URL, > and not / any more. > > wallflower starts spidering the application from /, but you can pass > it a list of URL > on the command-line: > > $ wallflower --application app.psgi --dir static/ --url > http://localhost/myapp /myapp/ > 200 /myapp/ => static/myapp/index.html [97] > 200 /myapp/page.html => static/myapp/page.html [14] > > And it now works as expected! > > Note that the initial URL *has* to be "/myapp/" with a final slash, to > create a directory "myapp" > with an "index.html" file inside. > > If you forget the final slash, you'll get an error: > > $ wallflower --application app.psgi --dir static/ --url > http://localhost/myapp /myapp > 200 /myapp => static/myapp [97] > Can't open static/myapp/page.html for writing: Not a directory at > /home/book/perl5/lib/perl5/App/Wallflower.pm line 130. > > which is caused by the fact that wallflower puts the result of > requesting "/myapp" into the > "static/myapp" file, and later on can't create the "static/myapp" > directory to put the > content of the "/myapp/page.html" URL in it. > > Regards, > > -- BooK

Thanks, it worked for me now. 5 additional things came to my attention: 1. A file extension can also be a cgi file: Although you put a lot of work into the tutorial section about "URL SEMANTICS COMPARED TO DIRECTORY SEMANTICS", I ran into this problem. It was not that obvious. As I'm running the script in CGI mode (as I will convert it into an offline page anyway, speed doesn't matter), I had a home URL named http://localhost/cgi-bin/myapp/myapp.cgi This is similar to a URL without extension. I had to change the URL to: http://localhost/cgi-bin/myapp/myapp.cgi/index.html 2. wallflower is a bit tricky to debug It was a bit tricky to debug. I modified the Wallflower.pm file and added STDERR output messages. Logging would be nice. Would you accept a patch, once the module is available on github? Or is there a simple way I didn't get? 3. undocumented --filter option There is no documentation for --filter. Not in the wallpaper tutorial, not in the main manpage, and I didn't find a test for it. Is it just a list of URLs? Or can there be place hodlers? 4. There is no link to the tutorial in wallflower's manpage. There is only this one link that referrs to the section "URL SEMANTICS COMPARED TO DIRECTORY SEMANTICS". Could you please add it to SEE ALSO section? 5. it just works Although there are some issues as pointed out before, it just works. Thank you for this great piece of software! I really enjoy it.

Fri May 22 18:18:02 2015 ASB [...] cpan.org - Status changed from 'open' to 'resolved'

Fri May 29 17:51:57 2015 book [...] cpan.org - Correspondence added

Many thanks for your comments. On Fri May 22 18:18:01 2015, ASB wrote: Show quoted text

> > 5 additional things came to my attention: > > 1. A file extension can also be a cgi file: > > Although you put a lot of work into the tutorial section about "URL > SEMANTICS COMPARED TO DIRECTORY SEMANTICS", I ran into this problem. > It was not that obvious. As I'm running the script in CGI mode (as I > will convert it into an offline page anyway, speed doesn't matter), I > had a home URL named http://localhost/cgi-bin/myapp/myapp.cgi > > This is similar to a URL without extension. I had to change the URL > to: > http://localhost/cgi-bin/myapp/myapp.cgi/index.html >

What would be a solution to that issue? I'm of the opinion that, if you want your web application to convert smoothly to a static site, you have to put some effort into it, as a static website imposes some restrictions on the type of URL that can be served (no query-string, for example). Wallflower does a naïve conversion from the URL it's given: https://github.com/book/App-Wallflower/blob/master/lib/Wallflower.pm#L50 There might be something that could be done with PATH_INFO, but I don't see it yet. Show quoted text

> 2. wallflower is a bit tricky to debug > > It was a bit tricky to debug. I modified the Wallflower.pm file and > added STDERR output messages. Logging would be nice. Would you accept > a patch, once the module is available on github? > Or is there a simple way I didn't get?

I'd be interested to know the type of logging you've added. I've been adding some options, but they only control which URL and status codes are shown. Show quoted text

> 3. undocumented --filter option > > There is no documentation for --filter. Not in the wallpaper tutorial, > not in the main manpage, and I didn't find a test for it. Is it just a > list of URLs? Or can there be place hodlers?

I think the only documentation is here: https://github.com/book/App-Wallflower/blob/master/bin/wallflower#L39 https://github.com/book/App-Wallflower/blob/master/bin/wallflower#L66 The arguments are URL from which Wallflower will start. The --filter option turns it into a "filter", so you can do: script-that-prints-url | wallflower --filter or wallflower --filter urls.txt Show quoted text

> 4. There is no link to the tutorial in wallflower's manpage. There is > only this one link that referrs to the section "URL SEMANTICS COMPARED > TO DIRECTORY SEMANTICS". > > Could you please add it to SEE ALSO section?

Will do. Show quoted text

> 5. it just works > > Although there are some issues as pointed out before, it just works. > Thank you for this great piece of software! I really enjoy it.

Thanks!