Skip Menu |

This queue is for tickets about the Mail-IMAPClient CPAN distribution.

Report information
The Basics
Id: 30733
Status: resolved
Priority: 0/
Queue: Mail-IMAPClient

People
Owner: Nobody in particular
Requestors: bernd [...] firmix.at
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Mail-IMAPClient-2.2.9 and broken connections
Date: Wed, 14 Nov 2007 17:23:10 +0100
To: bug-Mail-IMAPClient [...] rt.cpan.org
From: Bernd Petrovitsch <bernd [...] firmix.at>
Hi! We are using Mail-IMAPClient-2.2.9 on a Debian/Sarge on x86_64 hardware. We use it against Cyrus-Imapd-2.2.13 where user mailboxes are stored. Every user has a "spam" folder there and a janitor job removes every night old enough spam mails and sends the user a overview of the remaining. What happens as seen from the outside? Once in a while the janitor job busy loops on broken IMAP connections and stracing the process gives ---- snip ---- write(12, "5378 SELECT user/91070z2/Spam\r\n", 31) = -1 EPIPE (Broken pipe) --- SIGPIPE (Broken pipe) @ 0 (0) --- write(12, "5378 SELECT user/91070z2/Spam\r\n", 31) = -1 EPIPE (Broken pipe) --- SIGPIPE (Broken pipe) @ 0 (0) --- write(12, "5378 SELECT user/91070z2/Spam\r\n", 31) = -1 EPIPE (Broken pipe) --- SIGPIPE (Broken pipe) @ 0 (0) --- write(12, "5378 SELECT user/91070z2/Spam\r\n", 31) = -1 EPIPE (Broken pipe) --- SIGPIPE (Broken pipe) @ 0 (0) --- ---- snip ---- (and fast of course!). We have (for various reasons and routinely) SIGPIPE ignored (as the strace above also tells). And AFAICS we check for errors on every possibility in our application code. Following into the Mail-IMAPClient module, I come to the "sub _send_line" at line 1372 and find some strange (IMHO of course) code on line 1458: ---- snip ---- 1458 until ($total >= length($string)) { 1459 my $ret = 0; 1460 $!=0; 1461 $ret = syswrite( 1462 $self->Socket, 1463 $string, 1464 length($string)-$total, 1465 $total 1466 ); 1467 $ret||=0; 1468 if ($! == &EAGAIN ) { 1469 if ( $self->{Maxtemperrors} !~ /^unlimited/i 1470 and $temperrs++ > ($self->{Maxtemperrors}||10) 1471 ) { 1472 $self->LastError("Persistent '${!}' errors\n"); 1473 $self->_debug("Persistent '${!}' errors\n") 1474 return undef; 1475 } 1476 $optimize = 1; 1477 } else { 1478 # avoid infinite loops on syswrite error 1479 return undef unless(defined $ret); 1480 } ---- snip ---- What happens more in detail here? - syswrite() is called and returns "undef" (stored in $ret) and sets $!. - line 1467: '$ret = 0' - $! is &EPIPE so we take the "else" branch. - But here we are not returning because $ret is defined - it has 0 in there. What is IMHO strange? - The "unless(defined $ret)" seems wrong to me since $ret will always be defined there - either by the return value of syswrite() - but then we shouldn't get there anyways - and otherwise through line 1467. Are we the first ones which ignore SIGPIPE to stumble over that? Did I overlook something? Alas, testing and tracing is not really or easily possible since the system and application in question has around 23.000 mailboxes ATM and is the central mail server of a medium sized Austrian ISP (medium sized for Austrian circumstances). The short term work-around is to enable SIGPIPE at that end so that the process vanishes (and can be consequently restarted). Bernd -- Firmix Software GmbH http://www.firmix.at/ mobil: +43 664 4416156 fax: +43 1 7890849-55 Embedded Linux Development and Services
Subject: Re: [rt.cpan.org #30733] Mail-IMAPClient-2.2.9 and broken connections
Date: Wed, 14 Nov 2007 22:48:53 +0100
To: Bernd Petrovitsch via RT <bug-Mail-IMAPClient [...] rt.cpan.org>
From: Mark Overmeer <solutions [...] overmeer.net>
* Bernd Petrovitsch via RT (bug-Mail-IMAPClient@rt.cpan.org) [071114 17:10]: Show quoted text
> Wed Nov 14 12:10:49 2007: Request 30733 was acted upon. > Transaction: Ticket created by bernd@firmix.at > Queue: Mail-IMAPClient > Subject: Mail-IMAPClient-2.2.9 and broken connections
After four years without maintenance, I took over and hope to get a new release out. See search.cpan.org for 2.99_06. There are zillions of bugfixes (and newly created bug) There will be no bugfix release based on the 2.2.9 code. So: be warned for the new release: it's a big change! Requires more extensive testing than usual. Show quoted text
> Once in a while the janitor job busy loops on broken > IMAP connections and stracing the process gives > ---- snip ---- > write(12, "5378 SELECT user/91070z2/Spam\r\n", 31) = -1 EPIPE (Broken pipe) > --- SIGPIPE (Broken pipe) @ 0 (0) --- > > We have (for various reasons and routinely) SIGPIPE ignored (as the > strace above also tells).
Show quoted text
> What happens more in detail here? > - syswrite() is called and returns "undef" (stored in $ret) and sets $!.
The loop is left, writing stops. No nice recovery. In this case, we could attempt a reconnect, calling connect()->login() Show quoted text
> - But here we are not returning because $ret is defined - it has > 0 in there.
True, that was a bug. Show quoted text
> Are we the first ones which ignore SIGPIPE to stumble over that? > Did I overlook something?
Probably that people found many more important bugs ;-( Show quoted text
> Alas, testing and tracing is not really or easily possible since the > system and application in question has around 23.000 mailboxes ATM and > is the central mail server of a medium sized Austrian ISP (medium sized > for Austrian circumstances).
Then... create a smaller test-set. That's what other people do. Show quoted text
> The short term work-around is to enable SIGPIPE at that end so that the > process vanishes (and can be consequently restarted).
yes: die $! if $!==EPIPE; -- Regards, MarkOv ------------------------------------------------------------------------ Mark Overmeer MSc MARKOV Solutions Mark@Overmeer.net solutions@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
CC: DJKERNEN__NO_SOLICITING__ [...] cpan.org
Subject: Re: [rt.cpan.org #30733] Mail-IMAPClient-2.2.9 and broken connections
Date: Fri, 16 Nov 2007 12:50:07 +0100
To: bug-Mail-IMAPClient [...] rt.cpan.org
From: Bernd Petrovitsch <bernd [...] firmix.at>
On Mit, 2007-11-14 at 16:50 -0500, Mark Overmeer via RT wrote: Show quoted text
> <URL: http://rt.cpan.org/Ticket/Display.html?id=30733 > > > * Bernd Petrovitsch via RT (bug-Mail-IMAPClient@rt.cpan.org) [071114 17:10]:
> > Wed Nov 14 12:10:49 2007: Request 30733 was acted upon. > > Transaction: Ticket created by bernd@firmix.at > > Queue: Mail-IMAPClient > > Subject: Mail-IMAPClient-2.2.9 and broken connections
> > After four years without maintenance, I took over and hope to
Ah, I didn't know that. Show quoted text
> get a new release out. See search.cpan.org for 2.99_06. There > are zillions of bugfixes (and newly created bug) There will be > no bugfix release based on the 2.2.9 code.
There is 2.99_07 now there. Show quoted text
> So: be warned for the new release: it's a big change! Requires > more extensive testing than usual.
Show quoted text
> > Once in a while the janitor job busy loops on broken > > IMAP connections and stracing the process gives > > ---- snip ---- > > write(12, "5378 SELECT user/91070z2/Spam\r\n", 31) = -1 EPIPE (Broken pipe) > > --- SIGPIPE (Broken pipe) @ 0 (0) --- > > > > We have (for various reasons and routinely) SIGPIPE ignored (as the > > strace above also tells).
>
> > What happens more in detail here? > > - syswrite() is called and returns "undef" (stored in $ret) and sets $!.
> > The loop is left, writing stops. No nice recovery. In this case, we > could attempt a reconnect, calling connect()->login()
Yes, if it's cleanly possible to recover from that error. Show quoted text
> > - But here we are not returning because $ret is defined - it has > > 0 in there.
> > True, that was a bug. >
> > Are we the first ones which ignore SIGPIPE to stumble over that? > > Did I overlook something?
> > Probably that people found many more important bugs ;-( >
> > Alas, testing and tracing is not really or easily possible since the > > system and application in question has around 23.000 mailboxes ATM and > > is the central mail server of a medium sized Austrian ISP (medium sized > > for Austrian circumstances).
> > Then... create a smaller test-set. That's what other people do.
We have actually have there a shadow cluster (in Vmware instances) for testing functional changes. But it doesn't have the load (by far!) of the real cluster. It never happened there AFAIK. I can easily create a test situation of the above described bug. But I don't know if that is really the bug from real life. Bernd -- Firmix Software GmbH http://www.firmix.at/ mobil: +43 664 4416156 fax: +43 1 7890849-55 Embedded Linux Development and Services
Subject: Re: [rt.cpan.org #30733] Mail-IMAPClient-2.2.9 and broken connections
Date: Fri, 16 Nov 2007 13:55:42 +0100
To: Bernd Petrovitsch via RT <bug-Mail-IMAPClient [...] rt.cpan.org>
From: Mark Overmeer <solutions [...] overmeer.net>
* Bernd Petrovitsch via RT (bug-Mail-IMAPClient@rt.cpan.org) [071116 11:51]: Show quoted text
> > The loop is left, writing stops. No nice recovery. In this case, we
> > > Alas, testing and tracing is not really or easily possible since the > > > system and application in question has around 23.000 mailboxes ATM and > > > is the central mail server of a medium sized Austrian ISP (medium sized > > > for Austrian circumstances).
> > > > Then... create a smaller test-set. That's what other people do.
> > We have actually have there a shadow cluster (in Vmware instances) for > testing functional changes. But it doesn't have the load (by far!) of > the real cluster. It never happened there AFAIK. > I can easily create a test situation of the above described bug. But I > don't know if that is really the bug from real life.
You can just kill server-side clients at random interfalls. That should trigger your EPIPEs. Any chance you will do that, or should I close this ticket? -- Regards, MarkOv ------------------------------------------------------------------------ Mark Overmeer MSc MARKOV Solutions Mark@Overmeer.net solutions@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
CC: DJKERNEN__NO_SOLICITING__ [...] cpan.org
Subject: Re: [rt.cpan.org #30733] Mail-IMAPClient-2.2.9 and broken connections
Date: Fri, 16 Nov 2007 15:35:14 +0100
To: bug-Mail-IMAPClient [...] rt.cpan.org
From: Bernd Petrovitsch <bernd [...] firmix.at>
On Fre, 2007-11-16 at 07:56 -0500, Mark Overmeer via RT wrote: Show quoted text
> <URL: http://rt.cpan.org/Ticket/Display.html?id=30733 > > > * Bernd Petrovitsch via RT (bug-Mail-IMAPClient@rt.cpan.org) [071116 11:51]:
> > > The loop is left, writing stops. No nice recovery. In this case, we
> > > > Alas, testing and tracing is not really or easily possible since the > > > > system and application in question has around 23.000 mailboxes ATM and > > > > is the central mail server of a medium sized Austrian ISP (medium sized > > > > for Austrian circumstances).
> > > > > > Then... create a smaller test-set. That's what other people do.
> > > > We have actually have there a shadow cluster (in Vmware instances) for > > testing functional changes. But it doesn't have the load (by far!) of > > the real cluster. It never happened there AFAIK. > > I can easily create a test situation of the above described bug. But I > > don't know if that is really the bug from real life.
> > You can just kill server-side clients at random interfalls. That > should trigger your EPIPEs. Any chance you will do that, or should > I close this ticket?
Close it as it looks like it's fixed in 2.99_07. If I actually do it, I can write you an email anways;-) Bernd -- Firmix Software GmbH http://www.firmix.at/ mobil: +43 664 4416156 fax: +43 1 7890849-55 Embedded Linux Development and Services
closed, fixed in 2.99