| Subject: | MCE locks up if a worker quits in a non-graceful manner |
| Date: | Thu, 04 Feb 2016 16:57:10 +0000 |
| To: | "bug-MCE [...] rt.cpan.org" <bug-MCE [...] rt.cpan.org> |
| From: | "Benjamin C. McKeown" <mckeowbc [...] gmail.com> |
I've noticed that if a MCE worker dies in such a way that its END block
does not get called then the MCE manager will block on a socket call and
never terminate. We see this occurring occasionally when using XS modules
in a worker that can cause Perl to die in uncontrollable manner. We've
also seen it occasionally happen if a Perl worker runs out of memory.
The follow code demonstrates the behavior.
---
use MCE::Loop;
mce_loop {
my ($mce, $chunk_ref, $chunk_id) = @_;
foreach my $item (@{$chunk_ref}) {
if($item > 100 and $item % 9 == 0) {
MCE->say(\*STDERR,"Worker " . MCE->wid . " dying in a bad way
from item $item");
kill 9,$$;
}
}
} 1..10_000;
---
I've verified that the problem still exists in 1.699_009.
I've been able to get MCE to exit gracefully in this scenario by patching
MCE::Core::Manager to wrap all reads from $_DAT_R_SOCK with a select
statement, then call waitpid and cleanup the _total_running and
_total_workers counts if no data is received after a reasonable timeout.