Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the Parallel-ForkManager CPAN distribution.

Report information
The Basics
Id: 102305
Status: open
Priority: 0/
Queue: Parallel-ForkManager
People
Owner: Nobody in particular
Requestors: DROLSKY [...] cpan.org
Cc:
AdminCc:
Bug Information
Severity: Wishlist
Broken in: (no value)
Fixed in: (no value)

History Show all quoted text
  Mon Feb 23 15:38:34 2015 DROLSKY [...] cpan.org - Ticket created
Subject: Provide a way to do a real blocking wait for child processes
I wanted to use this code in a manager process to spin up a few children and then keep spinning up new children whenever an old one exited. However, internally, PFM always ends up doing some sort of busy loop, and the public ->wait_for_available_procs() method does a _very_ busy loop. In a system where you expect the children to be long-lived, this means that the parent process is going to be burning a lot of CPU doing nothing. I could patch ->wait_for_available_procs() to use a blocking wait, or add a separate method. It seems like a method named "->wait..." should actually block, though.
  Mon Feb 23 15:54:27 2015 yanick+cpan [...] babyl.dyndns.org - Correspondence added
On Mon Feb 23 15:38:34 2015, DROLSKY wrote: Show quoted text
> I could patch ->wait_for_available_procs() to use a blocking wait, or > add a separate method. It seems like a method named "->wait..." should > actually block, though.
Yeah. The code of `_waitpid_blocking` was modified to protect us from the case where the children procs have been reaped by another 'waitpid', but that indeed cause troubles. Hmm... Perhaps changing the hot loop for a configurable timeout would do the trick?
  Mon Feb 23 15:54:27 2015 The RT System itself - Status changed from 'new' to 'open'
  Mon Feb 23 15:58:25 2015 DROLSKY [...] cpan.org - Correspondence added
On Mon Feb 23 15:54:27 2015, YANICK wrote: Show quoted text
> On Mon Feb 23 15:38:34 2015, DROLSKY wrote:
> > I could patch ->wait_for_available_procs() to use a blocking wait, or > > add a separate method. It seems like a method named "->wait..." > > should > > actually block, though.
> > > Yeah. The code of `_waitpid_blocking` was modified to protect us from > the case where the children procs have been reaped by another > 'waitpid', but that indeed cause troubles. Hmm... Perhaps changing the > hot loop for a configurable timeout would do the trick?
Why not just document that if you reap the processes yourself then the blocking method will block forever.
  Mon Feb 23 16:07:20 2015 yanick [...] babyl.dyndns.org - Correspondence added
Subject: Re: [rt.cpan.org #102305] Provide a way to do a real blocking wait for child processes
Date: Mon, 23 Feb 2015 16:07:10 -0500
To: bug-Parallel-ForkManager [...] rt.cpan.org
From: Yanick Champoux <yanick [...] babyl.dyndns.org>
On 2015-02-23 03:58 PM, Dave Rolsky via RT wrote: Show quoted text
> Why not just document that if you reap the processes yourself then the blocking method will block forever.
Because "yourself" can also be "another module that you are using". Which is exactly what was happening and triggered the modification of the code. I'll try to find the time to craft a patch tonight that might deal with the blocking calls better. Stay tuned. `/.
  Mon Feb 23 16:18:05 2015 DROLSKY [...] cpan.org - Correspondence added
ether on IRC just pointed me to Parallel::Prefork (https://metacpan.org/release/Parallel-Prefork) which is a much better fit for what I was doing.
  Mon Feb 23 16:21:33 2015 yanick [...] babyl.dyndns.org - Correspondence added
Subject: Re: [rt.cpan.org #102305] Provide a way to do a real blocking wait for child processes
Date: Mon, 23 Feb 2015 16:21:23 -0500
To: bug-Parallel-ForkManager [...] rt.cpan.org, undisclosed-recipients:;
From: Yanick Champoux <yanick [...] babyl.dyndns.org>
On 2015-02-23 04:18 PM, Dave Rolsky via RT wrote: Show quoted text
> Queue: Parallel-ForkManager > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=102305 > > > ether on IRC just pointed me to Parallel::Prefork (https://metacpan.org/release/Parallel-Prefork) which is a much better fit for what I was doing. >
Excellent. And fwiw, there's also https://metacpan.org/release/MCE that might be of interest. `/.