Skip Menu |

This queue is for tickets about the Mail-IMAPClient CPAN distribution.

Report information
The Basics
Id: 42932
Status: resolved
Priority: 0/
Queue: Mail-IMAPClient

People
Owner: Nobody in particular
Requestors: PLOBBES [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in:
  • 2.2.9
  • 3.13
Fixed in: (no value)



Subject: is_parent bad arrayref and Massage needs to quote additional characters
Two problems are addressed in the bug report (sorry, perhaps technically I should have split these out): * [Issue#1] If list() returns undef when called in is_parent, $list does not contain an arrayref thus @$list is invalid. In my case, this happened via a call to selectable(). Error seen would be something like: Use of uninitialized value in pattern match (m//) at .../lib/Mail/IMAPClient.pm line 2303. * [Issue#2] Massage has (for a long time) only quoted for a limited number of cases, but most (if not all?) IMAP want certain special characters to be quoted more often. For example, any folder name containing an asterisk should be quoted if I read the RFC's correctly. Here's my RFC interpretation of what should be quoted (also included in patch): # rfc3501: # atom-specials = "(" / ")" / "{" / SP / CTL / list-wildcards / # quoted-specials / resp-specials # list-wildcards = "%" / "*" # quoted-specials = DQUOTE / "\" # resp-specials = "]" # rfc2060: CTL ::= <any ASCII control character and DEL, 0x00 - 0x1f, 0x7f> Another approach that at least Thunderbird seems to take is quote anything that isn't going to be sent as a literal (not sure if/when thunderbird will even use a literal)... or we can just use the updated regexp in the patch. Either would suffice. Without quoting, I would get errors like: 33 CREATE Star* 33 BAD parse error: excess characters at end of command A patch is attached! PS. If you're looking for a co-maintainer let me know!
Subject: imapclient-3.13.bad_ref-fix_quote.patch
--- IMAPClient.pm.ORIG 2009-01-06 04:01:22.000000000 -0500 +++ IMAPClient.pm 2009-01-31 01:38:39.000000000 -0500 @@ -2255,7 +2255,7 @@ sub is_parent { my ($self, $folder) = (shift, shift); - my $list = $self->list(undef, $folder) || "NO NO BAD BAD"; + my $list = $self->list(undef, $folder) || return undef; my $line; for(my $m = 0; $m < @$list; $m++) @@ -2300,7 +2300,8 @@ sub selectable { my ($self, $f) = @_; - not( grep /NoSelect/i, $self->list("", $f) ); + my $list = $self->list("", $f) or return undef; + not( grep /NoSelect/i, @$list ); } sub append @@ -2763,12 +2764,19 @@ sub Quote($) { $_[0]->Massage($_[1], NonFolderArg) } +# rfc3501: +# atom-specials = "(" / ")" / "{" / SP / CTL / list-wildcards / +# quoted-specials / resp-specials +# list-wildcards = "%" / "*" +# quoted-specials = DQUOTE / "\" +# resp-specials = "]" +# rfc2060: CTL ::= <any ASCII control character and DEL, 0x00 - 0x1f, 0x7f> sub Massage($;$) { my ($self, $name, $notFolder) = @_; $name =~ s/^\"(.*)\"$/$1/ unless $notFolder; $name =~ /["\\]/ ? "{".length($name)."}\r\n$name" - : $name =~ /[\s{}()]/ ? qq["$name"] + : $name =~ /[(){}\s[:cntrl:]%\*\[\]]/ ? qq["$name"] : $name; }
Subject: Re: [rt.cpan.org #42932] is_parent bad arrayref and Massage needs to quote additional characters
Date: Mon, 2 Feb 2009 09:13:56 +0100
To: Phil Lobbes via RT <bug-Mail-IMAPClient [...] rt.cpan.org>
From: Mark Overmeer <mark [...] overmeer.net>
* Phil Lobbes via RT (bug-Mail-IMAPClient@rt.cpan.org) [090131 07:23]: Show quoted text
> Sat Jan 31 02:22:58 2009: Request 42932 was acted upon. > Transaction: Ticket created by PLOBBES > Queue: Mail-IMAPClient > Subject: is_parent bad arrayref and Massage needs to quote additional > > Two problems are addressed in the bug report (sorry, perhaps technically > I should have split these out):
Next time please ;-) Show quoted text
> * [Issue#1] If list() returns undef when called in is_parent, $list does > not contain an arrayref thus @$list is invalid. In my case, this > happened via a call to selectable(). > > Error seen would be something like: > Use of uninitialized value in pattern match (m//) at > .../lib/Mail/IMAPClient.pm line 2303.
Clearly two bugs on its own. is_parent() can never have worked. - not( grep /NoSelect/i, $self->list("", $f) ); + my $list = $self->list("", $f) or return undef; + not( grep /NoSelect/i, @$list ); It seems something else is wrong. The list should deliver an empty list is LIST context. In my person view at least. Maybe you can investigate why "undef" is returned. (Mail::IMAPClient is not my own code: I only maintain it because it is well-used but was not maintained any longer. I often have to guess the principles behind constructions. On the other hand, I try to resolve problems, avoid work-arounds) Show quoted text
> * [Issue#2] Massage has (for a long time) only quoted for a limited > Here's my RFC interpretation of what should be quoted > # rfc2060: CTL ::= <any ASCII control character and DEL, 0x00 - 0x1f, 0x7f>
- : $name =~ /[\s{}()]/ ? qq["$name"] + : $name =~ /[(){}\s[:cntrl:]%\*\[\]]/ ? qq["$name"] Of course, in a one-to-one translation, we do not need to escape } and [ I have added a line of comment here. Show quoted text
> PS. If you're looking for a co-maintainer let me know!
Don't underestimate the complications in this module. I would really like to get rit of the module (I am not even an IMAP user myself), but it needs active maintenance over a long period of time. And some developmen... the whole manual must get redone: far too verbose. I await your response to one remaining problem. Two patches applied. -- Regards, MarkOv ------------------------------------------------------------------------ Mark Overmeer MSc MARKOV Solutions Mark@Overmeer.net solutions@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Subject: Re: [rt.cpan.org #42932] is_parent bad arrayref and Massage needs to quote additional characters
Date: Mon, 02 Feb 2009 23:27:40 -0500
To: bug-Mail-IMAPClient [...] rt.cpan.org
From: Phil Lobbes <phil [...] perkpartners.com>
Show quoted text
> > * [Issue#1] If list() returns undef when called in is_parent, $list > > does not contain an arrayref thus @$list is invalid. In my case, > > this happened via a call to selectable(). > > > > Error seen would be something like: > > Use of uninitialized value in pattern match (m//) at > > .../lib/Mail/IMAPClient.pm line 2303.
> > Clearly two bugs on its own. is_parent() can never have worked. > > - not( grep /NoSelect/i, $self->list("", $f) ); > + my $list = $self->list("", $f) or return undef; > + not( grep /NoSelect/i, @$list ); > It seems something else is wrong. The list should deliver an > empty list is LIST context. In my person view at least. Maybe > you can investigate why "undef" is returned.
In some error conditions, list() will return undef instead of (). In my case, the server I was talking to would return something like this when the folder name was not escaped properly: 118 NO Mailbox does not exist, or must be subscribed to. You'll notice in list() this happens: $self->_imap_command( qq[LIST "$reference" $target] ) or return undef; Effective we were ending up with: $ perl -we 'grep ( /NoSelect/, undef ); ' Use of uninitialized value in pattern match (m//) at -e line 1. Given our quoting problems, the IMAP server response would cause _imap_command to return false. Looking through the code, it appears there are numerous ways to get _imap_command() to return undef and several calls that don't deal so well with that. I've got a few on my list to investigate right now... expect more patches to come, but I'll have a new bug opened for those. One at a time :-). Show quoted text
> (Mail::IMAPClient is not my own code: I only maintain it because it is > well-used but was not maintained any longer. I often have to guess > the principles behind constructions. On the other hand, I try to > resolve problems, avoid work-arounds)
I know you're not the original author. Take a look at the Change history you'll see some patches were submitted by me a *long* time ago and I appreciate the fact that you're working to make the module better! Show quoted text
> > * [Issue#2] Massage has (for a long time) only quoted for a limited > > Here's my RFC interpretation of what should be quoted > > # rfc2060: CTL ::= <any ASCII control character and DEL, 0x00 - 0x1f, 0x7f>
> > - : $name =~ /[\s{}()]/ ? qq["$name"] > + : $name =~ /[(){}\s[:cntrl:]%\*\[\]]/ ? qq["$name"] > > Of course, in a one-to-one translation, we do not need to escape } and > [ I have added a line of comment here.
Agreed, I'm less interested in RFC strictness and more interested in minimally compliant and *most* interested in a module that works in most (if not all) cases... we're at least one step closer now. Show quoted text
> > PS. If you're looking for a co-maintainer let me know!
> > Don't underestimate the complications in this module. I would really > like to get rit of the module (I am not even an IMAP user myself), but > it needs active maintenance over a long period of time. And some > developmen... the whole manual must get redone: far too verbose.
I know the complexity, and am well aware of the history. My offer still stands... I'm happy to co-maintain... and I'm actively using the module and working on IMAP servers. I work for Zimbra (a Yahoo! company) and among a host of other things, I work on email migrations. I'm also a CPAN author of a few modules and have been doing Perl and email for *years*. Show quoted text
> I await your response to one remaining problem. Two patches applied.
Did I miss anything? Phil
Subject: Re: [rt.cpan.org #42932] is_parent bad arrayref and Massage needs to quote additional characters
Date: Tue, 3 Feb 2009 10:10:40 +0100
To: "phil [...] perkpartners.com via RT" <bug-Mail-IMAPClient [...] rt.cpan.org>
From: Mark Overmeer <mark [...] overmeer.net>
* phil@perkpartners.com via RT (bug-Mail-IMAPClient@rt.cpan.org) [090203 04:28]: Show quoted text
> Queue: Mail-IMAPClient > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=42932 > >
> > > * [Issue#1] If list() returns undef when called in is_parent, $list > > > does not contain an arrayref thus @$list is invalid. In my case, > > > this happened via a call to selectable(). > > > > > > Error seen would be something like: > > > Use of uninitialized value in pattern match (m//) at > > > .../lib/Mail/IMAPClient.pm line 2303.
> > > > Clearly two bugs on its own. is_parent() can never have worked. > > > > - not( grep /NoSelect/i, $self->list("", $f) ); > > + my $list = $self->list("", $f) or return undef; > > + not( grep /NoSelect/i, @$list ); > > It seems something else is wrong. The list should deliver an > > empty list is LIST context. In my person view at least. Maybe > > you can investigate why "undef" is returned.
> > In some error conditions, list() will return undef instead of (). In my > case, the server I was talking to would return something like this when > the folder name was not escaped properly: > > 118 NO Mailbox does not exist, or must be subscribed to. > > You'll notice in list() this happens: > > $self->_imap_command( qq[LIST "$reference" $target] ) > or return undef;
Valid reason. And losing error conditions is a sin as well. When I write new code, I use Log::Report to create exceptions. Then, _imap_command would simply cast an error, and the user's application use a try { } block. But we have to live with the current concept. Show quoted text
> Effective we were ending up with: > $ perl -we 'grep ( /NoSelect/, undef ); ' > Use of uninitialized value in pattern match (m//) at -e line 1.
Yep. sub selectable { my ($self, $f) = @_; my $info = $self->list("", $f); defined $info ? not(grep /NoSelect/i, @$list) : undef; } Show quoted text
> ... expect more patches to come, but I'll > have a new bug opened for those. One at a time :-).
Or for a class of same bugs. Some items need more time to fix. It is confusing when different kinds of items are in one bug-report, where some need more discussions and time, where other are easy to fix. I like to keep a clear work- list. Show quoted text
> I know the complexity, and am well aware of the history. My offer still > stands... I'm happy to co-maintain... and I'm actively using the module > and working on IMAP servers. I work for Zimbra (a Yahoo! company) and > among a host of other things, I work on email migrations. I'm also a > CPAN author of a few modules and have been doing Perl and email for > *years*.
Ok, gladly accepted. You do not want to become the single maintainer? Or maybe over time? Do you prefer sending in patches (via RT or directly) or a CVS repository? Show quoted text
> > I await your response to one remaining problem. Two patches applied.
> Did I miss anything?
1) is_parent 2) selectable 3) quote characters selectable() was the last to be discussed. -- Regards, MarkOv ------------------------------------------------------------------------ Mark Overmeer MSc MARKOV Solutions Mark@Overmeer.net solutions@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
fixes will be available in 3.14