Subject: | Memory issues when processing large MIF files |
Date: | Mon, 4 Jun 2007 12:50:19 -0400 |
To: | <bug-FrameMaker-MifTree [...] rt.cpan.org> |
From: | "Amy Louise Green" <algreen [...] softimage.com> |
Basic problem:
The larger the MIF file, the more time it seems to take
(quasi-exponentially) to parse the file. For really big files (~24MB) I
eventually get a Windows Application error on perl.exe saying the memory
cannot be written to at that address, etc. For example, it takes ~15
seconds to parse a 0.5 MB file, ~4 minutes for 6.7 MB file, ~39 minutes
for 15 MB file, and a whopping 4.25 hours to parse a 20 MB file.
That last one only worked after I opened it and stripped out all hidden
conditional text because the original version (23.8MB) was the one that
killed perl.exe with the memory error. I also tried splitting into 2
pieces and running them individually, and the two halves were 10.2MB and
11MB, taking between 1.25 and 1.5 hours each to parse.
FWIW, the largest file I was able to process without getting the Windows
Application error was 21.9MB.
Environment details:
* Distribution name and version: FrameMaker-MifTree-0.075
* Perl version: v5.8.0 built for MSWin32-x86-multi-thread,
ActivePerl Build 805
* Operating System vendor and version: Windows XP Professional SP2
* Operating environment: AMD Athlon, 1.10 GHz, 1 GB RAM