Bug #28820 for WWW-Mechanize: update_html doesn't update everything

Mon Aug 13 17:01:35 2007 kernel [...] foster.nce.ubc.ca - Ticket created

Subject:	update_html doesn't update everything
Date:	Mon, 13 Aug 2007 14:00:38 -0700
To:	bug-WWW-Mechanize [...] rt.cpan.org
From:	P Fudd <kernel [...] foster.nce.ubc.ca>

Hi.. I'm using WWW::Mechanize to take a web page, fill in a form, and submit it. First problem: the page uses javascript to populate a select field. Selecting something that should be there gives an unexpected 'illegal value' error (a select box). So, I get the page data with $mech->content, rewrite it, and reparse it with $mech->update_html. This adds 600 options to two of the select boxes on the page, just like the javascript does in a browser. Second problem: $mech->value still gives 'illegal value' error, as before. Using $mech->param instead of $mech->value solves this problem; $mech->param uses the updated html. Third problem: Errors with $mech->param refer to lines in the HTML::Form module, not lines in my program. Not much of a problem, but should be fixed. $mech->value does the right thing and refers to my program. This is from WWW::Mechanize 1.20; I'll try it again with the latest version later if I can install it on ActiveState perl. Thanks for writing such a useful module, I use WWW::Mechanize every day now!

Mon Aug 13 17:07:14 2007 andy [...] petdance.com - Correspondence added

Subject:	Re: [rt.cpan.org #28820] update_html doesn't update everything
Date:	Mon, 13 Aug 2007 16:06:21 -0500
To:	bug-WWW-Mechanize [...] rt.cpan.org
From:	Andy Lester <andy [...] petdance.com>

First, 1.20 is a year old. Show quoted text

> > First problem: the page uses javascript to populate a select field. > Selecting something that should be there gives an unexpected 'illegal > value' error (a select box). > > So, I get the page data with $mech->content, rewrite it, and > reparse it > with $mech->update_html. This adds 600 options to two of the select > boxes on the page, just like the javascript does in a browser.

Please reduce this to a single problem with code that you can show. Show quoted text

> > Second problem: $mech->value still gives 'illegal value' error, as > before. > > Using $mech->param instead of $mech->value solves this problem; > $mech->param uses the updated html.

Again, please show some code that makes this happen that I can reproduce. Show quoted text

> > Third problem: Errors with $mech->param refer to lines in the > HTML::Form > module, not lines in my program. > > Not much of a problem, but should be fixed. $mech->value does the > right > thing and refers to my program.

And here as well. When you get these, please send each one as a separate ticket, because they could well be unrelated. Thanks, Andy -- Andy Lester => andy@petdance.com => www.petdance.com => AIM:petdance

Mon Aug 13 17:07:16 2007 The RT System itself - Status changed from 'new' to 'open'

Mon Aug 13 18:32:46 2007 bitcard [...] foster.nce.ubc.ca - Correspondence added

From:

bitcard [...] foster.nce.ubc.ca

Ok, see attached. Two files are attached. a.html : a form with a select box, updated by javascript doit.pl : a WWW::Mechanize script that attempts to work around the javascript problem. This has been tested against 1.30 running on linux, and the problem is worse; the $mech->update_html function is not having any effect.

One

#!/usr/bin/perl -w use strict; use WWW::Mechanize; my $mech = WWW::Mechanize->new(); $mech->get("http://localhost/settings/mascot/a.html"); my $form=$mech->form_with_fields("myChoice"); if (!defined $form) { die("myChoice was not found"); } $form->param("myChoice","One"); eval { $form->param("myChoice","Two"); }; if ($@) {warn("$@");} eval { $form->value("myChoice","Two"); }; if ($@) {warn("$@");} my $more=<<EOF; <OPTION>Two</OPTION> <OPTION>Three</OPTION> EOF my $html=$mech->content(); $html=~s/(<SELECT[^>]*>)/$1\n$more/s; $mech->update_html($html); #print $html; eval {$form->param("myChoice","Two");}; if ($@) {warn("$@");} eval {$form->value("myChoice","Two");}; if ($@) {warn("$@");}

Mon Aug 13 18:49:14 2007 bitcard [...] foster.nce.ubc.ca - Correspondence added

From:

bitcard [...] foster.nce.ubc.ca

Ooops, the form was not re-obtained after the update_html call. I'm uploading a fixed version of the test program. The error message from $form->param() is referring to line 512 in HTML/Form, instead of line 12 of doit.pl.

#!/usr/bin/perl -w use strict; use WWW::Mechanize; my $mech = WWW::Mechanize->new(); $mech->get("http://localhost/settings/mascot/a.html"); my $form=$mech->form_with_fields("myChoice"); if (!defined $form) { die("myChoice was not found"); } $form->param("myChoice","One"); eval { $form->param("myChoice","Two"); }; if ($@) {warn("$@");} eval { $form->value("myChoice","Two"); }; if ($@) {warn("$@");} my $more=<<EOF; <OPTION>Two</OPTION> <OPTION>Three</OPTION> EOF my $html=$mech->content(); $html=~s/(<SELECT[^>]*>)/$1\n$more/s; $mech->update_html($html); #print $html; $form=$mech->form_with_fields("myChoice"); eval {$form->param("myChoice","Two");}; if ($@) {warn("$@");} eval {$form->value("myChoice","Two");}; if ($@) {warn("$@");}

Mon Aug 13 18:54:14 2007 bitcard [...] foster.nce.ubc.ca - Correspondence added

Subject:	HTML::Form->param() error messages point to line in HTML::Form.
From:	bitcard [...] foster.nce.ubc.ca

At this point, is it really a WWW::Mechanize problem, or an HTML::Form problem?

Mon Aug 13 18:57:57 2007 bitcard [...] foster.nce.ubc.ca - Correspondence added

From:

bitcard [...] foster.nce.ubc.ca

Updated a.html... action="a.html", not action="GET"

One

Fri Nov 09 13:48:45 2007 PETDANCE [...] cpan.org - Correspondence added

If you find this is actually still a problem with Mech, please report it to http://code.google.com/p/www-mechanize/issues/list

Fri Nov 09 13:48:46 2007 PETDANCE [...] cpan.org - Status changed from 'open' to 'resolved'

Bug #28820 for WWW-Mechanize: update_html doesn't update everything

Preferred bug tracker