Skip Menu |

This queue is for tickets about the Mail-Box CPAN distribution.


Subject: Problems with autodetection of MH folders
Date: Sun, 23 Sep 2007 21:15:22 +0200
To: bug-Mail-Box [...] rt.cpan.org
From: gregor herrmann <gregor+debian [...] comodo.priv.at>
Hi, we have received a bug against the Debian package of Mail::Box that describes that MH folders are incorrectly detected as mboxes. The full report is available at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=442912 Please consider investigating this issue. Thanks in advance, Gregor (Debian Perl Group) -- .''`. http://info.comodo.priv.at/ | gpg key ID: 0x00F3CFE4 : :' : debian: the universal operating system - http://www.debian.org/ `. `' member of https://www.vibe.at/ | how to reply: http://got.to/quote/ `- NP: Rolling Stones: It's Only Rock & Roll
Download signature.asc
application/pgp-signature 189b

Message body not shown because it is not plain text.

Perl is quite memory hungry... in MailBox case, especially the headers take a lot (remember that any scaler uses at least 24 bytes) Opening is LAZY, and therefore still fits in memory in your case. When you are using the message it is triggered to be read as a whole... and therefore the memory needs grow. See $msg->destruct to manually cleanup used messages... or re-open the folder each time you have processed a few thousands messages.
Sorry, I mixed it up. This answer is for report http://rt.cpan.org/Ticket/Display.html?id=29580
Ok. Autodection is not flawless, but this can be fixed. Could you either test this, or send me a MH set which fails: Mail/Box.pm sub openRelatedFolder(@) { my $self = shift; my @options = (%{$self->{MB_init_options}}, @_); $self->{MB_manager} - ? $self->{MB_manager}->open(@options) + ? $self->{MB_manager}->open(type => ref($self), @options) : (ref $self)->new(@options); }
Subject: Re: [rt.cpan.org #29579] Problems with autodetection of MH folders
Date: Mon, 24 Sep 2007 18:33:41 +0200
To: Mark Overmeer via RT <bug-Mail-Box [...] rt.cpan.org>
From: gregor herrmann <gregor+debian [...] comodo.priv.at>
On Mon, 24 Sep 2007 02:57:49 -0400, Mark Overmeer via RT wrote: Show quoted text
> Ok. Autodection is not flawless, but this can be fixed. Could you > either test this, or send me a MH set which fails:
Thanks for your quick help, I've passed this on to the original submitter, let's see if it helps or if he can provide an exmaple. Cheers, gregor -- .''`. http://info.comodo.priv.at/ | gpg key ID: 0x00F3CFE4 : :' : debian: the universal operating system - http://www.debian.org/ `. `' member of https://www.vibe.at/ | how to reply: http://got.to/quote/ `- NP: Janis Joplin: Ball And Chain
Download signature.asc
application/pgp-signature 189b

Message body not shown because it is not plain text.

Subject: Re: [rt.cpan.org #29579] Problems with autodetection of MH folders
Date: Mon, 24 Sep 2007 21:47:46 +0200
To: Mark Overmeer via RT <bug-Mail-Box [...] rt.cpan.org>
From: gregor herrmann <gregor+debian [...] comodo.priv.at>
On Mon, 24 Sep 2007 02:57:49 -0400, Mark Overmeer via RT wrote: Show quoted text
> Ok. Autodection is not flawless, but this can be fixed. Could you > either test this, or send me a MH set which fails:
The bug submitter has applied the patch and provides an example folder -- both at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=442912 Cheers, gregor -- .''`. http://info.comodo.priv.at/ | gpg key ID: 0x00F3CFE4 : :' : debian: the universal operating system - http://www.debian.org/ `. `' member of https://www.vibe.at/ | how to reply: http://got.to/quote/ `- NP: Neil Young: Heart of Gold
Download signature.asc
application/pgp-signature 189b

Message body not shown because it is not plain text.

Subject: [rt.cpan.org #29579] (Fwd) Fw: [sylpheed:31818] Re: MH filenames
Date: Wed, 10 Oct 2007 17:44:52 +0200
To: bug-Mail-Box [...] rt.cpan.org
From: gregor herrmann <gregor+debian [...] comodo.priv.at>
Hi Mark, first let me thank you for implementing the fix from #29579 so quickly! Celejar and me have looked into the issue regarding the detection of MH folders a little bit further (for details please look at http://bugs.debian.org/442912 ). Celejar has found out that auto-detecting works if the folder contains a file named "1", and I've found at least to lines in the code (in lib/Mail/Box/MH.pm and in lib/Mail/Box/Mbox.pm) where you explicitly test for the existence of $directory/1 or $filename/1. I have still no idea, if a file named 1 has to exist in an MH folder, and I failed to find something like an MH folder specification. Celejar has asked on the sylpheed list, and the answer quoted below indicates that 1 is not needed, which means your code should test for $directory/\d+ instead. I'm not sure if your current implementation does the correct thing or if the comment below is right; but I'd like to ask you to look into that issue again. Thanks alot, gregor ----- Forwarded message from Celejar <celejar@gmail.com> ----- From: Celejar <celejar@gmail.com> To: gregor herrmann <gregor+debian@comodo.priv.at> Subject: Fw: [sylpheed:31818] Re: MH filenames Date: Wed, 10 Oct 2007 08:54:49 -0400 [..] Begin forwarded message: Date: Tue, 9 Oct 2007 20:46:17 -0400 From: Don Koch <aardvark@krl.com> To: sylpheed@sraoss.jp Subject: [sylpheed:31818] Re: MH filenames On Mon, 8 Oct 2007 14:46:47 -0400 Celejar wrote: Show quoted text
> The developer / maintainer of the Mail::Box::MH Perl module seems to > assume that (non-empty) MH folders always contain a message with the > filename '1' [0]. Sylpheed clearly doesn't follow that convention. > Does anyone have any information about the MH specification? > > [0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=442912#93 (see the > rest of the bug report for context) > > Celejar
There is no requirement for a '1' file to be present. If a user deletes the first mail file and doesn't pack the folder, there will be no such file. An invalid assumption was made on the part of the author of said code. A non-empty folder is one in which there is at least one file that has a name with only digits in it. -d [..] Show quoted text
----- End forwarded message ----- -- .''`. http://info.comodo.priv.at/ | gpg key ID: 0x00F3CFE4 : :' : debian: the universal operating system - http://www.debian.org/ `. `' member of https://www.vibe.at/ | how to reply: http://got.to/quote/ `- NP: Rolling Stones: Highland
Download signature.asc
application/pgp-signature 189b

Message body not shown because it is not plain text.

Somehow, your remarks got lost, and therefore ignored. Let's get back in the debate. On Wed Oct 10 11:46:03 2007, gregor+debian@comodo.priv.at wrote: Show quoted text
> Celejar and me have looked into the issue regarding the detection of > MH folders a little bit further (for details please look at > http://bugs.debian.org/442912 ). > > Celejar has found out that auto-detecting works if the folder > contains a file named "1", and I've found at least to lines in the > code (in lib/Mail/Box/MH.pm and in lib/Mail/Box/Mbox.pm) where you > explicitly test for the existence of $directory/1 or $filename/1.
In the MH case, the "1" is used as first fast attempt to detect it, if fails, a slower approach is taken. This seems sound. On the other hand, the mbox auto-detect which is tried before the MH is called is too lazy. I have improved it from if(-d $filename) # fake empty folder, with sub-folders { return 1 unless -f File::Spec->catfile($filename, '1') # MH || -d File::Spec->catdir($filename, 'cur'); # Maildir } into if(-d $filename) # fake empty folder, with sub-folders { return 0 if -d File::Spec->catdir($filename, 'cur'); # Maildir local *DIR; if(opendir DIR, $filename) { my @f = grep !/^\.\.?$/, readdir DIR; return 0 if @f && ! grep /\D/, @f; # MH closedir DIR; } return 1; } [untested, but looks healthy] Please keep in mind that auto-detection can not always distinguish the folder-types, although comflicts are rare. Thanks for your effort, sorry for the delay.
Subject: Re: [rt.cpan.org #29579] Problems with autodetection of MH folders
Date: Tue, 13 Nov 2007 17:22:51 +0100
To: Mark Overmeer via RT <bug-Mail-Box [...] rt.cpan.org>
From: gregor herrmann <gregor+debian [...] comodo.priv.at>
On Tue, 13 Nov 2007 03:25:27 -0500, Mark Overmeer via RT wrote: Show quoted text
> Somehow, your remarks got lost, and therefore ignored. Let's > get back in the debate.
No problem, and thanks for digging up the lost mail! Show quoted text
> > Celejar has found out that auto-detecting works if the folder > > contains a file named "1", and I've found at least to lines in the > > code (in lib/Mail/Box/MH.pm and in lib/Mail/Box/Mbox.pm) where you > > explicitly test for the existence of $directory/1 or $filename/1.
> In the MH case, the "1" is used as first fast attempt to detect it, > if fails, a slower approach is taken. This seems sound.
Ok. Show quoted text
> On the other hand, the mbox auto-detect which is tried before the > MH is called is too lazy. I have improved it from > > if(-d $filename) # fake empty folder, with sub-folders > { return 1 unless -f File::Spec->catfile($filename, '1') # MH > || -d File::Spec->catdir($filename, 'cur'); # Maildir > } > > into > > if(-d $filename) # fake empty folder, with sub-folders > { return 0 if -d File::Spec->catdir($filename, 'cur'); # Maildir > local *DIR; > if(opendir DIR, $filename) > { my @f = grep !/^\.\.?$/, readdir DIR; > return 0 if @f && ! grep /\D/, @f; # MH > closedir DIR; > } > return 1; > } > > [untested, but looks healthy]
It looks indeed good; a short test (with the original submitters test folder from sylpheed) shows another problem: $ find inbox/ inbox/ inbox/new inbox/new/.sylpheed_mark inbox/new/.sylpheed_cache inbox/2 inbox/1 inbox/3 inbox/6 inbox/.sylpheed_mark inbox/4 inbox/.sylpheed_cache inbox/5 "inbox" is not detected as an MH folder because of new and .syl* Changing the above line to return 0 if @f && grep /^\d+$/, @f; # MH catches this case but obviously leads to false positives in other cases (mbox files with only-digits-filenames). I don't know if there is a solution that fits all cases ... Show quoted text
> Please keep in mind that auto-detection can not always distinguish > the folder-types, although comflicts are rare.
Right. Show quoted text
> Thanks for your effort, sorry for the delay.
Thank you! Cheers, gregor -- .''`. http://info.comodo.priv.at/ | gpg key ID: 0x00F3CFE4 : :' : debian: the universal operating system - http://www.debian.org/ `. `' member of https://www.vibe.at/ | how to reply: http://got.to/quote/ `-
Download signature.asc
application/pgp-signature 189b

Message body not shown because it is not plain text.

CC: undisclosed-recipients: ;
Subject: Re: [rt.cpan.org #29579] Problems with autodetection of MH folders
Date: Tue, 13 Nov 2007 18:56:29 +0100
To: "gregor+debian [...] comodo.priv.at via RT" <bug-Mail-Box [...] rt.cpan.org>
From: NLnet webmaster <webmaster [...] nlnet.nl>
* gregor+debian@comodo.priv.at via RT (bug-Mail-Box@rt.cpan.org) [071113 16:23]: Show quoted text
> It looks indeed good; a short test (with the original submitters test > folder from sylpheed) shows another problem: > > $ find inbox/ > inbox/ > inbox/new > inbox/new/.sylpheed_mark > inbox/new/.sylpheed_cache > inbox/2 > inbox/1 > inbox/3 > inbox/6 > inbox/.sylpheed_mark > inbox/4 > inbox/.sylpheed_cache > inbox/5 > > "inbox" is not detected as an MH folder because of new and .syl*
Blurk... and maildir is using new (and cur) This 'new' thing make sylpheed using MH+own extensions :( Show quoted text
> Changing the above line to > return 0 if @f && grep /^\d+$/, @f; # MH > catches this case but obviously leads to false positives in other > cases (mbox files with only-digits-filenames). > > I don't know if there is a solution that fits all cases ...
Ok, next attempt: if(-d $filename) { # Maildir and MH Sylpheed have a 'new' sub-directory return 0 if -d File::Spec->catdir($filename, 'new'); local *DIR; if(opendir DIR, $filename) { my @f = grep !/^\./, readdir DIR; # skip . .. and hidden return 0 if @f && ! grep /\D/, @f; # MH closedir DIR; } return 1; # faked empty Mbox sub-folder (with subsub-folders?) } -- Regards, MarkOv ------------------------------------------------------------------------ Mark Overmeer MSc MARKOV Solutions Mark@Overmeer.net solutions@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Subject: Re: [rt.cpan.org #29579] Problems with autodetection of MH folders
Date: Wed, 14 Nov 2007 16:37:14 +0100
To: NLnet webmaster via RT <bug-Mail-Box [...] rt.cpan.org>
From: gregor herrmann <gregor+debian [...] comodo.priv.at>
On Tue, 13 Nov 2007 12:57:26 -0500, NLnet webmaster via RT wrote: Show quoted text
> > $ find inbox/ > > inbox/ > > inbox/new > > inbox/new/.sylpheed_mark > > inbox/new/.sylpheed_cache > > inbox/2 > > inbox/1 > > inbox/3 > > inbox/6 > > inbox/.sylpheed_mark > > inbox/4 > > inbox/.sylpheed_cache > > inbox/5 > > > > "inbox" is not detected as an MH folder because of new and .syl*
> > Blurk... and maildir is using new (and cur) > This 'new' thing make sylpheed using MH+own extensions :(
Right, that's a little strange. Show quoted text
> if(-d $filename) > { # Maildir and MH Sylpheed have a 'new' sub-directory > return 0 if -d File::Spec->catdir($filename, 'new'); > > local *DIR; > if(opendir DIR, $filename) > { my @f = grep !/^\./, readdir DIR; # skip . .. and hidden > return 0 if @f && ! grep /\D/, @f; # MH > closedir DIR; > } > return 1; # faked empty Mbox sub-folder (with subsub-folders?) > }
Looks good and indeed works for this example. It would be great if you implemented this patch in a future version of Mail::Box. Thanks alot for your patience in sorting this out, gregor -- .''`. http://info.comodo.priv.at/ | gpg key ID: 0x00F3CFE4 : :' : debian: the universal operating system - http://www.debian.org/ `. `' member of https://www.vibe.at/ | how to reply: http://got.to/quote/ `- NP: Die Tontauben: jonny
Download signature.asc
application/pgp-signature 189b

Message body not shown because it is not plain text.

Yes, this patch will be part of MailBox 2.079
Subject: Re: [rt.cpan.org #29579] Problems with autodetection of MH folders
Date: Wed, 14 Nov 2007 21:38:01 +0100
To: Mark Overmeer via RT <bug-Mail-Box [...] rt.cpan.org>
From: gregor herrmann <gregor+debian [...] comodo.priv.at>
On Wed, 14 Nov 2007 11:15:46 -0500, Mark Overmeer via RT wrote: Show quoted text
> Yes, this patch will be part of MailBox 2.079
Great, thanks! gregor -- .''`. http://info.comodo.priv.at/ | gpg key ID: 0x00F3CFE4 : :' : debian: the universal operating system - http://www.debian.org/ `. `' member of https://www.vibe.at/ | how to reply: http://got.to/quote/ `- NP: Johnny Cash: I'm Leaving Now
Download signature.asc
application/pgp-signature 189b

Message body not shown because it is not plain text.