Show quoted text>>>>> On Thu, 16 Nov 2006 11:59:39 -0500, "Mark Overmeer via RT" <bug-Mail-Box@rt.cpan.org> said:
Show quoted text
Show quoted text movr> I would have written that as:
movr> my $mgr = Mail::Box::Manager->new;
movr> my $mbox = $mgr->open($mboxfile) or die;
movr> foreach my $msg ($folder->messages)
movr> { print "progress: ".$msg->seqnr/$folder->nrMessages."\n";
movr> my $text = $msg->body->decoded->string;
movr> # do something with $text;
movr> }
Much prettier, thanks. I didn't find out how to serialize a message
except via print.
Show quoted text movr> The "open" will not consume too much memory: for mbox folders it will
movr> run through the file and collect headers with a minimal number of
movr> fields. But then, when you start looking inside the message, the
movr> memory use will be considerable. Don't forget that each message
movr> builds an expensive header structure, and with multiparts even quite
movr> a few of them.
Show quoted text movr> There are tests in the test-set which check for memory-leaks (using weak-
movr> links), and I am not aware of any.
Show quoted text movr> There are two approaches to avoid the memory use explosion:
movr> 1) reopen the folder for each few-hundred processed messages
movr> 2) use $msg->destruct ... the memory of the message will
movr> be cleared. But then: do not open the folder with write
movr> permission.
Thanks, the second option seems to help a lot and is perfectly well
suited for me. The process now grows to 640 MB for a 440 MB mbox and
it is faster too. This will enable me to do the jobs of converting
these mailboxes at hand. Thanks much.
I'm still surprised that it's so much. Seems like you're holding the
whole mbox in memory despite you have a filehandle for it. But that's
OK for a perl program. Maybe worth a mention in the manpage?
Show quoted text movr> This is typically a place where the memory hungry perl variables
movr> bite. Unless you have more facts to suspect a leak, I think there
movr> isn't.
It's your call to decide on that. I wasn't even suspecting a leak in
the first place, just simply complaining about use of too much memory.
Thanks much for your help on this.
--
andreas
