Bug #29922 for Tk-XMLViewer: Widget is unresponsive while parsing large files

Thu Oct 11 10:19:50 2007 gyles19 [...] visi.com - Ticket created

Subject:

Widget is unresponsive while parsing large files

I have large xml files (100k or better) containing GPS POI information. When I ask XMLViewer to display one, the app hangs for 10+ minutes while the parser handlers fill the viewer widget. I added a call to $w->update inside _flush. The app now responds during the parsing process, but it's still taking a very long time to build the viewer. This may not be the best place to put the update call, but it served to illustrate the issue. I've attached a zipped xml file for demonstration purposes. (I captured the data myself so there are no entanglements on it should you wish to use it as test data.) I'm using Perl 5.8.8, Tk 804.027, Red Hat Linux 7.3 running on a Thinkpad with 1gig of ram and a 1Ghz cpu.

Subject:

large_sample.zip

Download large_sample.zip
application/zip 22.6k

Message body not shown because it is not plain text.

Thu Oct 11 18:12:01 2007 slaven [...] rezic.de - Correspondence added

CC:	undisclosed-recipients:;
Subject:	Re: [rt.cpan.org #29922] Widget is unresponsive while parsing large files
Date:	11 Oct 2007 23:06:02 +0200
To:	bug-Tk-XMLViewer [...] rt.cpan.org
From:	Slaven Rezic <slaven [...] rezic.de>

"gyles19@visi.com via RT" <bug-Tk-XMLViewer@rt.cpan.org> writes: Show quoted text

> Thu Oct 11 10:19:50 2007: Request 29922 was acted upon. > Transaction: Ticket created by gyles19@visi.com > Queue: Tk-XMLViewer > Subject: Widget is unresponsive while parsing large files > Broken in: (no value) > Severity: Normal > Owner: Nobody > Requestors: gyles19@visi.com > Status: new > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=29922 > > > > I have large xml files (100k or better) containing GPS POI information. > When I ask XMLViewer to display one, the app hangs for 10+ minutes while the > parser handlers fill the viewer widget. > > I added a call to $w->update inside _flush. The app now responds during > the parsing process, but it's still taking a very long time to build the > viewer. This may not be the best place to put the update call, but it > served to illustrate the issue. > > I've attached a zipped xml file for demonstration purposes. (I captured the > data myself so there are no entanglements on it should you wish to use it as > test data.) > > I'm using Perl 5.8.8, Tk 804.027, Red Hat Linux 7.3 running on a Thinkpad > with 1gig of ram and a 1Ghz cpu. > >

I had success with defered tagAdd calls. Please try the development version which should appear shortly at CPAN: ---------------------------------------------------------------------- The uploaded file Tk-XMLViewer-0.18_91.tar.gz has entered CPAN as file: $CPAN/authors/id/S/SR/SREZIC/Tk-XMLViewer-0.18_91.tar.gz size: 12953 bytes md5: 58bc40f16a4291ffb4c5a8a3312064b6 No action is required on your part Request entered by: SREZIC (Slaven Rezić) Request entered on: Thu, 11 Oct 2007 20:57:16 GMT Request completed: Thu, 11 Oct 2007 20:57:41 GMT ---------------------------------------------------------------------- Regards, Slaven -- Slaven Rezic - slaven <at> rezic <dot> de Berlin Perl Mongers - http://berlin.pm.org

Thu Oct 11 18:12:04 2007 The RT System itself - Status changed from 'new' to 'open'

Thu Oct 11 19:03:47 2007 slaven [...] rezic.de - Correspondence added

CC:	undisclosed-recipients:;
Subject:	Re: [rt.cpan.org #29922] Widget is unresponsive while parsing large files
Date:	11 Oct 2007 22:11:46 +0200
To:	bug-Tk-XMLViewer [...] rt.cpan.org
From:	Slaven Rezic <slaven [...] rezic.de>

"gyles19@visi.com via RT" <bug-Tk-XMLViewer@rt.cpan.org> writes: Show quoted text

> Thu Oct 11 10:19:50 2007: Request 29922 was acted upon. > Transaction: Ticket created by gyles19@visi.com > Queue: Tk-XMLViewer > Subject: Widget is unresponsive while parsing large files > Broken in: (no value) > Severity: Normal > Owner: Nobody > Requestors: gyles19@visi.com > Status: new > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=29922 > > > > I have large xml files (100k or better) containing GPS POI information. > When I ask XMLViewer to display one, the app hangs for 10+ minutes while the > parser handlers fill the viewer widget. > > I added a call to $w->update inside _flush. The app now responds during > the parsing process, but it's still taking a very long time to build the > viewer. This may not be the best place to put the update call, but it > served to illustrate the issue. > > I've attached a zipped xml file for demonstration purposes. (I captured the > data myself so there are no entanglements on it should you wish to use it as > test data.) > > I'm using Perl 5.8.8, Tk 804.027, Red Hat Linux 7.3 running on a Thinkpad > with 1gig of ram and a 1Ghz cpu. >

Well, your XML is not really that large. 120kB should be handled gracefully by today's computers. There seems to be some kind of exponential growth. If I create smaller files from your sample file and try to load them into XMLViewer, then I get the following results: $ for i in truckstops-*; do echo $i; time perl -MTk::XMLViewer -MTk -e '$v=tkinit->XMLViewer->pack;$v->insertXML(-file => shift);' $i; done truckstops-1000.gpx perl -MTk::XMLViewer -MTk -e $i 0,18s user 0,05s system 41% cpu 0,562 total truckstops-10000.gpx perl -MTk::XMLViewer -MTk -e $i 0,50s user 0,01s system 91% cpu 0,559 total truckstops-30000.gpx perl -MTk::XMLViewer -MTk -e $i 5,26s user 0,04s system 42% cpu 12,431 total truckstops-60000.gpx perl -MTk::XMLViewer -MTk -e $i 52,28s user 0,10s system 35% cpu 2:26,89 total truckstops-90000.gpx (still running) I suspect it's some Tk::Text slowness. Time for some profiling and optimization! Regards, Slaven -- Slaven Rezic - slaven <at> rezic <dot> de tktimex - time recording tool http://sourceforge.net/projects/ptktools/

Thu Oct 11 20:59:16 2007 gyles19 [...] visi.com - Correspondence added

Subject:	Re: [rt.cpan.org #29922] Widget is unresponsive while parsing large files
Date:	Thu, 11 Oct 2007 19:50:48 -0500 (CDT)
To:	"slaven [...] rezic.de via RT" <bug-Tk-XMLViewer [...] rt.cpan.org>
From:	Joi Ellis <gyles19 [...] visi.com>

On Thu, 11 Oct 2007, slaven@rezic.de via RT wrote: Show quoted text

> I had success with defered tagAdd calls. Please try the development > version which should appear shortly at CPAN:

Wow, you work fast! Show quoted text

> ---------------------------------------------------------------------- > The uploaded file > > Tk-XMLViewer-0.18_91.tar.gz > > has entered CPAN as > > file: $CPAN/authors/id/S/SR/SREZIC/Tk-XMLViewer-0.18_91.tar.gz > size: 12953 bytes > md5: 58bc40f16a4291ffb4c5a8a3312064b6

I've built and installed this latest. It does appear to be much faster, but now the element nesting is incorrect. It seems to indent, but not outdent when the element ends. I captured a pair of gifs to illustrate what I'm seeing. -- Joi Ellis gyles19@visi.com No matter what we think of Linux versus FreeBSD, etc., the one thing I really like about Linux is that it has Microsoft worried. Anything that kicks a monopoly in the pants has got to be good for something. - Chris Johnson

Download outdent.gif
image/gif 124.9k

Message body is not shown because sender requested not to inline it.

Download outdent2.gif
image/gif 124.9k

Message body is not shown because sender requested not to inline it.

Fri Oct 12 02:41:41 2007 slaven [...] rezic.de - Correspondence added

Subject:	Re: [rt.cpan.org #29922] Widget is unresponsive while parsing large files
Date:	12 Oct 2007 08:40:11 +0200
To:	bug-Tk-XMLViewer [...] rt.cpan.org
From:	Slaven Rezic <slaven [...] rezic.de>

"gyles19@visi.com via RT" <bug-Tk-XMLViewer@rt.cpan.org> writes: Show quoted text

> Queue: Tk-XMLViewer > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=29922 > > > On Thu, 11 Oct 2007, slaven@rezic.de via RT wrote: >

> > I had success with defered tagAdd calls. Please try the development > > version which should appear shortly at CPAN:

> > Wow, you work fast! >

> > ---------------------------------------------------------------------- > > The uploaded file > > > > Tk-XMLViewer-0.18_91.tar.gz > > > > has entered CPAN as > > > > file: $CPAN/authors/id/S/SR/SREZIC/Tk-XMLViewer-0.18_91.tar.gz > > size: 12953 bytes > > md5: 58bc40f16a4291ffb4c5a8a3312064b6

> > I've built and installed this latest. It does appear to be much faster, > but now the element nesting is incorrect. It seems to indent, but > not outdent when the element ends. > > I captured a pair of gifs to illustrate what I'm seeing.

Fixed in version 0.18_92 Regards, Slaven -- Slaven Rezic - slaven <at> rezic <dot> de Berlin Perl Mongers - http://berlin.pm.org

Fri Oct 12 06:29:12 2007 gyles19 [...] visi.com - Correspondence added

Subject:	Re: [rt.cpan.org #29922] Widget is unresponsive while parsing large files
Date:	Fri, 12 Oct 2007 05:20:35 -0500 (CDT)
To:	"slaven [...] rezic.de via RT" <bug-Tk-XMLViewer [...] rt.cpan.org>
From:	Joi Ellis <gyles19 [...] visi.com>

On Fri, 12 Oct 2007, slaven@rezic.de via RT wrote: Show quoted text

> > Fixed in version 0.18_92 > > Regards, > Slaven

Yes, that's fixed. I'm leaving town this morning so any further emails you send will probably be delayed until I get back in about 6 weeks. Thanks for your help! -- Joi Ellis gyles19@visi.com No matter what we think of Linux versus FreeBSD, etc., the one thing I really like about Linux is that it has Microsoft worried. Anything that kicks a monopoly in the pants has got to be good for something. - Chris Johnson

Fri Oct 12 17:52:55 2007 SREZIC [...] cpan.org - Correspondence added

I made an "official" distribution Tk-XMLViewer-0.19.tar.gz Regards, Slaven

Fri Oct 12 17:52:57 2007 SREZIC [...] cpan.org - Status changed from 'open' to 'resolved'