Skip Menu |

This queue is for tickets about the POE-XS-Queue-Array CPAN distribution.

Report information
The Basics
Id: 43902
Status: resolved
Priority: 0/
Queue: POE-XS-Queue-Array

People
Owner: TONYC [...] cpan.org
Requestors: acferen [...] yahoo.com
Cc:
AdminCc:

Bug Information
Severity: Critical
Broken in: 0.005
Fixed in: (no value)



Subject: Causes app to exit on MSWin32 if threading is used
Everything works fine until a thread finishes. As soon as a thread stops running the entire perl app closes with memory errors. Some examples that work without POE-XS-Queue-Array, but that fail when POE-XS-Queue-Array is installed. 1) http://poe.perl.org/?POE_Cookbook/Child_Processes_3 2) make test for POE-Component-IKC-0.2002 3) I didn't test all of them, but I suspect any of the other child process scripts from the cookbook will fail. Tested on C:\>perl -v This is perl, v5.8.8 built for MSWin32-x86-multi-thread (with 18 registered patches, see perl -V for more detail) Copyright 1987-2007, Larry Wall Binary build 822 [280952] provided by ActiveState http://www.ActiveState.com Built Jul 31 2007 19:34:48
On Fri Mar 06 10:08:58 2009, acferen@yahoo.com wrote: Show quoted text
> Everything works fine until a thread finishes. As soon as a thread > stops running the entire perl app closes with memory errors. > > Some examples that work without POE-XS-Queue-Array, but that fail when > POE-XS-Queue-Array is installed. > 1) http://poe.perl.org/?POE_Cookbook/Child_Processes_3 > 2) make test for POE-Component-IKC-0.2002 > 3) I didn't test all of them, but I suspect any of the other child > process scripts from the cookbook will fail.
Hi, POE::XS::Queue::Array doesn't support threads. This may change in the future, but I hadn't seen much demand for it, since typically threads and POE aren't mixed. I expect you're only seeing it because of the perl on Win32 fork emulation. So for now the only solution is to remove POE::XS::Queue::Array. Tony
On Sun Mar 08 18:48:39 2009, TONYC wrote: Show quoted text
> On Fri Mar 06 10:08:58 2009, acferen@yahoo.com wrote:
> > Everything works fine until a thread finishes. As soon as a thread > > stops running the entire perl app closes with memory errors. > > > > Some examples that work without POE-XS-Queue-Array, but that fail when > > POE-XS-Queue-Array is installed. > > 1) http://poe.perl.org/?POE_Cookbook/Child_Processes_3 > > 2) make test for POE-Component-IKC-0.2002 > > 3) I didn't test all of them, but I suspect any of the other child > > process scripts from the cookbook will fail.
> > Hi, > > POE::XS::Queue::Array doesn't support threads. > > This may change in the future, but I hadn't seen much demand for it, > since typically threads and POE aren't mixed. I expect you're only > seeing it because of the perl on Win32 fork emulation. > > So for now the only solution is to remove POE::XS::Queue::Array. > > Tony
Fair enough, but I'm not sure about POE and threads not being mixed. Wheel::Run is bundled with POE and used in a number of modules and several examples from http://poe.perl.org/?POE_Cookbook. Using Wheel::Run on Win32 will absolutely tickle this bug. -Andrew
Subject: Re: [rt.cpan.org #43902] Causes app to exit on MSWin32 if threading is used
Date: Tue, 10 Mar 2009 09:19:32 +1100
To: Andrew Feren via RT <bug-POE-XS-Queue-Array [...] rt.cpan.org>
From: tonyc [...] cpan.org
On Mon, Mar 09, 2009 at 11:47:04AM -0400, Andrew Feren via RT wrote: Show quoted text
> Queue: POE-XS-Queue-Array > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=43902 > > > On Sun Mar 08 18:48:39 2009, TONYC wrote:
> > On Fri Mar 06 10:08:58 2009, acferen@yahoo.com wrote:
> > > Everything works fine until a thread finishes. As soon as a thread > > > stops running the entire perl app closes with memory errors. > > > > > > Some examples that work without POE-XS-Queue-Array, but that fail when > > > POE-XS-Queue-Array is installed. > > > 1) http://poe.perl.org/?POE_Cookbook/Child_Processes_3 > > > 2) make test for POE-Component-IKC-0.2002 > > > 3) I didn't test all of them, but I suspect any of the other child > > > process scripts from the cookbook will fail.
> > > > Hi, > > > > POE::XS::Queue::Array doesn't support threads. > > > > This may change in the future, but I hadn't seen much demand for it, > > since typically threads and POE aren't mixed. I expect you're only > > seeing it because of the perl on Win32 fork emulation. > > > > So for now the only solution is to remove POE::XS::Queue::Array. > > > > Tony
> > Fair enough, but I'm not sure about POE and threads not being mixed. > Wheel::Run is bundled with POE and used in a number of modules and > several examples from http://poe.perl.org/?POE_Cookbook. > > Using Wheel::Run on Win32 will absolutely tickle this bug.
But only on Win32 - everything else uses a real fork() typically. I spent some time on it last night, unfortunately dealing with the cruft from threads at the XS level is under-documented. Tony
On Fri Mar 06 10:08:58 2009, acferen@yahoo.com wrote: Show quoted text
> Everything works fine until a thread finishes. As soon as a thread > stops running the entire perl app closes with memory errors.
I've uploaded POE-XS-Queue-Array 0.005_01 to CPAN, if you could give it a go I'd appreciate it. Tony
Subject: Re: [rt.cpan.org #43902] Causes app to exit on MSWin32 if threading is used
Date: Fri, 13 Mar 2009 09:39:34 -0400
To: bug-POE-XS-Queue-Array [...] rt.cpan.org
From: Andrew Feren <acferen [...] yahoo.com>
0.005_01 still fails. (only tested on Win32) My test consisted of 1. copied http://poe.perl.org/?POE_Cookbook/Child_Processes_3 to use a test case 2. Ran test script successfully 3. build and install POE-XS-Queue-Array 4. Rerun the test script. 1. failed. The script died right after the first task ran. 5. Removed POE-XS-Queue-Array 6. Rerun the test script. 1. success If you have other patches I'll be happy to give them a try. In a few weeks I'll have a little time and can try and get some more info about exactly where the crash is occurring. Thanks, -Andrew TONYC via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=43902 > > > On Fri Mar 06 10:08:58 2009, acferen@yahoo.com wrote: >
>> Everything works fine until a thread finishes. As soon as a thread >> stops running the entire perl app closes with memory errors. >>
> > I've uploaded POE-XS-Queue-Array 0.005_01 to CPAN, if you could give it > a go I'd appreciate it. > > Tony > > >
Subject: Re: [rt.cpan.org #43902] Causes app to exit on MSWin32 if threading is used
Date: Mon, 16 Mar 2009 00:21:43 +1100
To: Andrew Feren via RT <bug-POE-XS-Queue-Array [...] rt.cpan.org>
From: Tony Cook <tony [...] develop-help.com>
Thanks, I've uploaded 0.005_02 which uses an entirely different approach. In 0.005_01 I tried to use the CLONE() mechanism to do what perl does for perl data structures and duplicate the queue into the new thread. This worked fine for ithreads, but not for fork() emulation. Unfortunately by the time CLONE() runs there isn't enough information to detect whether a pseudo-fork() or a thread->create() is being done, and to update the stash list used for CLONE() calls. So what I've done is to implemente the CLONE_SKIP() method added in 5.8.7, so that the fork() child process will receive a non-blessed reference instead of a queue object. This works fine for the sample code you linked to. Tony On Fri, Mar 13, 2009 at 09:40:14AM -0400, Andrew Feren via RT wrote: Show quoted text
> Queue: POE-XS-Queue-Array > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=43902 > > > 0.005_01 still fails. (only tested on Win32) > > My test consisted of > > 1. copied http://poe.perl.org/?POE_Cookbook/Child_Processes_3 to use > a test case > 2. Ran test script successfully > 3. build and install POE-XS-Queue-Array > 4. Rerun the test script. > 1. failed. The script died right after the first task ran. > 5. Removed POE-XS-Queue-Array > 6. Rerun the test script. > 1. success > > If you have other patches I'll be happy to give them a try. In a few > weeks I'll have a little time and can try and get some more info about > exactly where the crash is occurring. > > Thanks, > -Andrew > > TONYC via RT wrote:
> > <URL: https://rt.cpan.org/Ticket/Display.html?id=43902 > > > > > On Fri Mar 06 10:08:58 2009, acferen@yahoo.com wrote: > >
> >> Everything works fine until a thread finishes. As soon as a thread > >> stops running the entire perl app closes with memory errors. > >>
> > > > I've uploaded POE-XS-Queue-Array 0.005_01 to CPAN, if you could give it > > a go I'd appreciate it. > > > > Tony > > > > > >
Subject: Re: [rt.cpan.org #43902] Causes app to exit on MSWin32 if threading is used
Date: Mon, 16 Mar 2009 09:03:16 -0400
To: bug-POE-XS-Queue-Array [...] rt.cpan.org
From: Andrew Feren <acferen [...] yahoo.com>
tony@develop-help.com via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=43902 > > > Thanks, > > I've uploaded 0.005_02 which uses an entirely different approach. > > In 0.005_01 I tried to use the CLONE() mechanism to do what perl does > for perl data structures and duplicate the queue into the new thread. > > This worked fine for ithreads, but not for fork() emulation. > > Unfortunately by the time CLONE() runs there isn't enough information > to detect whether a pseudo-fork() or a thread->create() is being done, > and to update the stash list used for CLONE() calls. > > So what I've done is to implemente the CLONE_SKIP() method added in > 5.8.7, so that the fork() child process will receive a non-blessed > reference instead of a queue object. > > This works fine for the sample code you linked to. > > Tony >
Yep. Works here too. I'll leave it installed on my dev box and let you know if I see any other issues. Thanks! -Andrew