Subject: | In pure-perl loop, IO events can be indefinitely starved by always-ready timers |
Date: | Thu, 18 Aug 2016 17:07:27 -0400 |
To: | bug-AnyEvent [...] rt.cpan.org |
From: | Zac Bentley <zbbentley [...] gmail.com> |
In the pure-perl AnyEvent::Loop, there's a starvation condition that can
occur if timer callbacks block for long enough that another timer is always
ready when they finish.
A full description and proposed solution are below.
Sample code and a bad workaround are available on this stackoverflow
question:
http://stackoverflow.com/questions/37769891/why-wont-anyeventchild-callbacks-ever-run-if-interval-timer-events-are-always/37845061#37845061
Simple outline of the issue:
Setup:
Use the pure perl backend on a POSIX OS (reproduced on OSX and RHEL6) with
Time::HiRes, without Async::Interrupt, without AIO. Much of that is
probably unnecessary; that's just my setup.
Steps to reproduce:
1. Schedule an interval timer event immediately, and then every N seconds
where N>0.
2. Set the timer's callback to block (do something synchronous that cannot
be interrupted by a signal, e.g. busy wait or use sigblock(2)) for M
seconds, where M>N.
3. Set a signal watcher (really just a wrapped IO event) for any catchable
signal S.
4. Run the event loop.
5. Wait some amount of time and then externally deliver signal S to the
running process.
Expected behavior:
Interval callback fires once, and then continues firing roughly every M
seconds (though it wants to run every N).
When signal is delivered, signal watcher callback runs either immediately
after the currently-executing interval callback, or after the immediately
following execution of that same callback.
Actual behavior:
Interval callback fires once, and then continues firing roughly every M
seconds (though it wants to run every N).
Signal callback is never delivered.
Suspected issue:
AnyEvent::Loop::one_event finds and runs timer events. If no timer events
are ready, it polls for IO events, and if none of either are ready, it runs
idle events. The if/else behavior between timers and IO events should be
removed: both sets of events should populate an internal queue, which
should be drained before re-polling.
Basically, the current logic should go from this:
1. Check for timer events, if any are found run them.
2. Else, select() for IO events, with a timeout of the time until the next
timer, and if any are found run them
3. Else, run idle events if any.
...to this:
1. If there are events in @queue, pop one from the head and run it.
2. Else
a. Check for timer events and add them to the tail of @queue.
b. Select() for IO events and add them to the tail of @queue. This
should be run uncondtionally. The timeout for select() should be:
scalar(@queue) ? 0 : $wait, where $wait is the existing "time until next
timer" value.
c. If there are events in @queue, pop one from the head and run it.
d. Else, run idle events if any.
This can be implemented without consuming any additional memory.
However, this adds overhead to one_event(). Select()s will be run more
frequently: they will now be run whenever timers are due. I think that
tradeoff is valid, since it ensures that all events will eventually be
processed in roughly the order they "occur" (in concept), regardless of how
much blocking time is spent outside of the event loop.
I am not sure how other event loop implementations deal with issues like
this. I know that many browser event loops use a backing queue like the one
proposed above, but am not familiar with others. Unless all of the other
AnyEvent backends mimic this behavior exactly, though (in such a way that
fixing this issue would make the pure-perl loop the only deviant), fixing
this still seems useful.
Thanks!
Zac