Skip Menu |

This queue is for tickets about the Forks-Super CPAN distribution.

Report information
The Basics
Id: 127016
Status: resolved
Priority: 0/
Queue: Forks-Super

People
Owner: Nobody in particular
Requestors: heptite [...] gmail.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 0.96



Subject: Can't get a forked process to "block" stdin/stdout filehandles
Date: Fri, 31 Aug 2018 21:25:21 -0600 (Mountain Daylight Time)
To: bug-Forks-Super [...] rt.cpan.org
From: "Christian J. Robinson" <heptite [...] gmail.com>
Under Windows 10, using Strawberry Perl, I am trying to use Forks::Super to create a sub-process that handles the job of gathering a small amount of data from a website (via an API key). The sub-process (child) is supposed to gather that data then print it to STDOUT, and wait for the parent process to read that data before it does it all over again. The parent process only tries to read the data from the child every 20 minutes, and that data is hopefully immediately available. Unfortunately the whole thing seems to fail to work as expected: $pid = fork {child_fh => ['in', 'out', 'pipe', 'block'], on_busy => 'fail'}; die "fork failed: $!" unless defined $pid; if ($pid) { $child_fh = $pid->{child_stdout}; $line = <$child_fh>; print ">>$line<<\n"; } else { while (1) { @aqi = &getAQI; print STDOUT join(' ', @aqi)."\n"; } } The &getAQI subroutine can take up to a minute to complete, and meanwhile the parent process IMMEDIATELY outputs ">><<" instead of waiting for the child process to gather the data and send it. Moreover, the child process seems to rapidly loop instead of waiting for the parent to gather the data before proceeding to run the data gathering subroutine again. I've tried using IO::Select::can_read/can_write on the file handles to wait for data, but that doesn't work either. Am I making a fundamental mistake, or have I discovered a bug? perl -v output: This is perl 5, version 24, subversion 0 (v5.24.0) built for MSWin32-x64-multi-thread -- Christian J. Robinson <heptite@gmail.com> -- https://christianrobinson.name/ And on the seventh day, He exited from append mode.
Reproduced
Forks::Super sets attributes on IO handles to indicate how the user wants them to behave. In lib/Forks/Super/Job/Ipc.pm, for example, there are dozens of assignments like $$fh->{is_socket} = 1 The distribution also uses an experimental set of tied classes, Forks::Super::Tie::IPCxxxHandle, for most interprocess communication. My vision was that these objects would be able to monitor how the I/O handles are used in an application and someday would be able to recommend how to tune the Forks::Super module to run your application more effectively. This issue arises because an attribute is set on the Windows socket handle (pipes on Windows are always emulated with sockets) to indicate that it should be a blocking socket handle _before_ the socket handle is tied. The block attribute is not visible when you query the tied filehandle. To fix this, I am checking for the attribute on the original handle and updating the handle used by the tied handle class each time a readline operation is done on the socket handle. This seems to resolve the problem. On older version of Forks::Super, a workaround is to disable the use of these tied handle classes. Some ways to do this are: set the environment variable NO_TIES=1 prior to loading Forks::Super set the variables $Forks::Super::Job::Ipc::USE_TIE_SH = $Forks::Super::Job::Ipc::USE_TIE_FH = $Forks::Super::Job::Ipc::USE_TIE_PH = 0 in your program, preferrably before the first `fork` call edit Forks/Super/Job/Ipc.pm, lines 53-59, to ensure that the $USE_TIE_xxx variables are set to false values Thank you for using Forks::Super and thank you for the bug report. Please continue to let me know about any other difficulty you have using this distribution. -- Marty O'Brien
Subject: Re: [rt.cpan.org #127016] Can't get a forked process to "block" stdin/stdout filehandles
Date: Thu, 13 Sep 2018 13:37:54 -0600 (Mountain Daylight Time)
To: Marty O'Brien via RT <bug-Forks-Super [...] rt.cpan.org>
From: "Christian J. Robinson" <heptite [...] gmail.com>
This has not resolved my problem. The behavior is different, in that the first iteration of the loop it does appear to block, but then it reverts to the old behavior where there appears to be no blocking.
Re-opening. Christian, do you have a short code sample that would demonstrate the current issue?
Subject: Re: [rt.cpan.org #127016] Can't get a forked process to "block" stdin/stdout filehandles
Date: Thu, 13 Sep 2018 13:55:46 -0600 (Mountain Daylight Time)
To: Marty O'Brien via RT <bug-Forks-Super [...] rt.cpan.org>
From: "Christian J. Robinson" <heptite [...] gmail.com>
The same code snippet I already provided, but I'll add: sub getAQI { print STDERR "in getAQI\n"; sleep 1; return ('--','--'); } Then it outputs something like this: in getAQI Show quoted text
>>-- --
<< in getAQI in getAQI in getAQI in getAQI in getAQI in getAQI in getAQI Terminating on signal SIGINT(2)
Hi Christian, I'm still not sure you have demonstrated a problem. In the script you provided, the parent process code is just if ($pid) { $child_fh = $pid->{child_stdout}; $line = <$child_fh>; print ">>$line<<\n"; } ... so it is expected that it will only read and display one line of output from the child process. If you put a loop into the parent code, it does appear to read and display more than one line from the child, and with an appropriate pause between lines of output. On Thu Sep 13 15:55:51 2018, heptite@gmail.com wrote: Show quoted text
> The same code snippet I already provided, but I'll add: > > sub getAQI > { > print STDERR "in getAQI\n"; > sleep 1; > return ('--','--'); > } > > Then it outputs something like this: > > in getAQI
> >>-- --
> << > in getAQI > in getAQI > in getAQI > in getAQI > in getAQI > in getAQI > in getAQI > Terminating on signal SIGINT(2)
Subject: Re: [rt.cpan.org #127016] Can't get a forked process to "block" stdin/stdout filehandles
Date: Thu, 13 Sep 2018 14:24:39 -0600 (Mountain Daylight Time)
To: Marty O'Brien via RT <bug-Forks-Super [...] rt.cpan.org>
From: "Christian J. Robinson" <heptite [...] gmail.com>
My actual script invokes a Win32::GUI instance that has a repeating timer that will read from the child process every 20 minutes. The problem is that the child process just continually prints "in getAQI" every second, even though the parent isn't reading from the child. I have to do the fork before I instantiate Win32::GUI, but I wonder if Win32::GUI is incompatible with Forks::Super?
Subject: Re: [rt.cpan.org #127016] Can't get a forked process to "block" stdin/stdout filehandles
Date: Thu, 13 Sep 2018 16:38:24 -0400
To: bug-Forks-Super [...] rt.cpan.org
From: "Marty O'Brien" <mobrule [...] gmail.com>
So you want the child to block until the parent can read all of its output? On Thu, Sep 13, 2018 at 4:25 PM Christian J. Robinson via RT < bug-Forks-Super@rt.cpan.org> wrote: Show quoted text
> Queue: Forks-Super > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=127016 > > > My actual script invokes a Win32::GUI instance that has a repeating > timer that will read from the child process every 20 minutes. The > problem is that the child process just continually prints "in getAQI" > every second, even though the parent isn't reading from the child. > > I have to do the fork before I instantiate Win32::GUI, but I wonder if > Win32::GUI is incompatible with Forks::Super? >
Subject: Re: [rt.cpan.org #127016] Can't get a forked process to "block" stdin/stdout filehandles
Date: Thu, 13 Sep 2018 14:45:27 -0600
To: bug-Forks-Super [...] rt.cpan.org
From: "Christian J. Robinson" <heptite [...] gmail.com>
I want the child to block until the parent reads, then gather new data and block until the parent reads again, repeated indefinitely. I wouldn't even bother forking to gather the data but the Win32::GUI would freeze every time the data is gathered, which is undesirable. On Thu, Sep 13, 2018 at 2:38 PM, Marty O'Brien via RT < bug-Forks-Super@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=127016 > > > So you want the child to block until the parent can read all of its output? > > On Thu, Sep 13, 2018 at 4:25 PM Christian J. Robinson via RT < > bug-Forks-Super@rt.cpan.org> wrote: >
> > Queue: Forks-Super > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=127016 > > > > > My actual script invokes a Win32::GUI instance that has a repeating > > timer that will read from the child process every 20 minutes. The > > problem is that the child process just continually prints "in getAQI" > > every second, even though the parent isn't reading from the child. > > > > I have to do the fork before I instantiate Win32::GUI, but I wonder if > > Win32::GUI is incompatible with Forks::Super? > >
> >
-- Christian J. Robinson <heptite@gmail.com>
Subject: Re: [rt.cpan.org #127016] Can't get a forked process to "block" stdin/stdout filehandles
Date: Thu, 13 Sep 2018 17:17:59 -0400
To: bug-Forks-Super [...] rt.cpan.org
From: "Marty O'Brien" <mobrule [...] gmail.com>
Got you. That's not really what the `block` attribute is for -- it's for blocking on read operations while an input stream reader waits for more input. To block on writing, I can see a few different things to try. 1. If the output is always the same number of bytes (fixed record length), there is a way of setting the buffer size of the socket/pipe so that after one record has been written, the socket buffer is full and will block until the other process reads from the socket. I would have to refresh my memory about how to do that. 2. You could use a blocking queue with a max size of one element, like Forks::Queue, to pass messages from child to parent instead of standard filehandles. Or at a more basic level, have the parent read from a file and then erase the file, and have the child sleep while that file exists, and then create a new file with the same name. 3. The `sync` option in a Forks::Super::fork call is kind of a solution in search of a problem, but this may be an application. The concept is that the processes share one or more synchronization objects with the property that only one process can "acquire" an object at a time. So consider sharing two synchronization objects between parent and child, let's call them A and B. At the time of the fork, the parent possesses A and the child possesses B. When the parent is ready to read, it follows this process: release A acquire B acquire A read data from child release B When the child is ready to write data, it follows this process write data to parent release B acquire A acquire B release A I think this scheme would not deadlock and would make the child wait after writing until the parent could read it (it's untested, though). On Thu, Sep 13, 2018 at 4:46 PM Christian J. Robinson via RT < bug-Forks-Super@rt.cpan.org> wrote: Show quoted text
> Queue: Forks-Super > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=127016 > > > I want the child to block until the parent reads, then gather new data and > block until the parent reads again, repeated indefinitely. I wouldn't even > bother forking to gather the data but the Win32::GUI would freeze every > time the data is gathered, which is undesirable. > > On Thu, Sep 13, 2018 at 2:38 PM, Marty O'Brien via RT < > bug-Forks-Super@rt.cpan.org> wrote: >
> > <URL: https://rt.cpan.org/Ticket/Display.html?id=127016 > > > > > So you want the child to block until the parent can read all of its
> output?
> > > > On Thu, Sep 13, 2018 at 4:25 PM Christian J. Robinson via RT < > > bug-Forks-Super@rt.cpan.org> wrote: > >
> > > Queue: Forks-Super > > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=127016 > > > > > > > My actual script invokes a Win32::GUI instance that has a repeating > > > timer that will read from the child process every 20 minutes. The > > > problem is that the child process just continually prints "in getAQI" > > > every second, even though the parent isn't reading from the child. > > > > > > I have to do the fork before I instantiate Win32::GUI, but I wonder if > > > Win32::GUI is incompatible with Forks::Super? > > >
> > > >
> > > -- > Christian J. Robinson <heptite@gmail.com> >
Marking as resolved.