Bug #118855 for WWW-Mechanize-FireFox: I love WWW::Mechanize::Firefox

Sun Nov 20 17:28:12 2016 jeff.stalzer [...] ieee.org - Ticket created

Subject:	I love WWW::Mechanize::Firefox
Date:	Sun, 20 Nov 2016 17:28:03 -0500
To:	bug-WWW-Mechanize-FireFox [...] rt.cpan.org
From:	Jeff I Stalzer <jeffstalzer [...] comcast.net>

Max, I have used WWW::Mechanize::Firefox for years and really love it. It is an outstanding piece of code. I use it to scrape my on-line banking every month. Every once in a while the bank decides to change their web site (sometimes just the internals) which causes me to re-code parts of my scraping solution. Recently they threw me a curve. The site displays check record "header summaries" that have to be expanded w/ a button to get detailed info downloaded from the site. I was using: @links = $mech3->selector('button'); to get a list of links and then one by one I did: $mech3->highlight_node($links[$ww]) ; $mech3->click( { dom=>$links[$ww], synchronize=>0 } ); These last two Mechanize::Firefox commands no longer work on the bank web page (although I do get a list of buttons) and I have had to manually "click the button" and then scrape the page. The bank changed the "button" to: <button class="expand-btn collapsibleControl" aria-controls="paymentDetail_20161114085508080608" data-billid="20161114085508080608" data-activitypaymenttypecode="0"><span class="expand-status visuallyhidden" id="hide-status" aria-hidden="true">Hide payment details for TruGreen Processing Center<span class="sub-detail">TruGreen</span> Any thoughts on a solution or work around? I'm using WWW-Mechanize-Firefox-0.78 on Win 7 w/ FireFox 50.0 Thanks, Jeff Stalzer

Mon Nov 21 12:18:31 2016 corion [...] cpan.org - Correspondence added

Hello Jeff, Show quoted text

> I have used WWW::Mechanize::Firefox for years and really love it. It is > an outstanding piece of code.

Thanks for using WWW::Mechanize::Firefox and reporting back! Show quoted text

> The bank changed the "button" to: > <button class="expand-btn collapsibleControl" > aria-controls="paymentDetail_20161114085508080608" > data-billid="20161114085508080608" > data-activitypaymenttypecode="0"><span class="expand-status > visuallyhidden" id="hide-status" aria-hidden="true">Hide payment details > for TruGreen Processing Center<span class="sub-detail">TruGreen</span> > > Any thoughts on a solution or work around?

Not really, and without any code to replicate this, I'm not really sure how I could reproduce your situation. Maybe you can click the items by id, or maybe Firefox broke some API or maybe the HTML is really badly behaved, it's hard to tell from here. You could help me help you better by providing a more to-the-point description of how "not work" manifests itself. Does the highlighting highlight the wrong HTML elements? Does it not highlight any HTML elements? Are you certain that the elements you see there are the relevant elements that the script finds? To me it seems that these HTML elements are hidden through CSS and also not accessible to screen readers. Please note that this site is not a support forum for WWW::Mechanize::Firefox. As stated in the documentation, please go to http://perlmonks.org/ to post support queries. -max

Mon Nov 21 12:18:32 2016 The RT System itself - Status changed from 'new' to 'open'

Mon Nov 21 12:18:32 2016 corion [...] cpan.org - Fixed in 0.78 added

Tue Nov 22 09:26:51 2016 jeff.stalzer [...] ieee.org - Correspondence added

Subject:	Re: [rt.cpan.org #118855] I love WWW::Mechanize::Firefox
Date:	Tue, 22 Nov 2016 09:26:40 -0500
To:	bug-WWW-Mechanize-FireFox [...] rt.cpan.org
From:	Jeff I Stalzer <jeffstalzer [...] comcast.net>

Thanks for your response. Your questions led me to re-examine my assumptions and test them. Turns out the change that caused the highlight link and expand link to not do either was an index problem. The web site has some links above the ones I want to work w/ and that number changed so my index was starting in the wrong place. when corrected both highlight link and expand link work exactly as they should. Thanks, Jeff stalzer Max Maischein via RT wrote: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=118855 > > > Hello Jeff,

>> I have used WWW::Mechanize::Firefox for years and really love it. It is >> an outstanding piece of code.

> Thanks for using WWW::Mechanize::Firefox and reporting back! >

>> The bank changed the "button" to: >> <button class="expand-btn collapsibleControl" >> aria-controls="paymentDetail_20161114085508080608" >> data-billid="20161114085508080608" >> data-activitypaymenttypecode="0"><span class="expand-status >> visuallyhidden" id="hide-status" aria-hidden="true">Hide payment details >> for TruGreen Processing Center<span class="sub-detail">TruGreen</span> >> >> Any thoughts on a solution or work around?

> Not really, and without any code to replicate this, I'm not really sure how I could reproduce your situation. Maybe you can click the items by id, or maybe Firefox broke some API or maybe the HTML is really badly behaved, it's hard to tell from here. > > You could help me help you better by providing a more to-the-point description of how "not work" manifests itself. Does the highlighting highlight the wrong HTML elements? Does it not highlight any HTML elements? Are you certain that the elements you see there are the relevant elements that the script finds? To me it seems that these HTML elements are hidden through CSS and also not accessible to screen readers. > > Please note that this site is not a support forum for WWW::Mechanize::Firefox. As stated in the documentation, please go to http://perlmonks.org/ to post support queries. > > -max > >