Skip Menu |

This queue is for tickets about the future CPAN distribution.

Report information
The Basics
Id: 96685
Status: open
Priority: 0/
Queue: future

People
Owner: Nobody in particular
Requestors: TEAM [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: 0.28
Fixed in: (no value)



Subject: sub-future cancellation for aggregates
Hi, The current behaviour when cancelling sub-futures seems unintuitive. Given a dependent Future (via ->needs_all) with 2 subs, when each sub-Future is cancelled that should reduce the pending count. Once all subs are ready (cancelled/failed/complete), should the dependent not also be marked as ready? Currently, it seems that cancelling all the subs leaves the dependent Future untouched - likewise, cancelling one of the subs would leave $pending non-zero so the dependent would never be marked as ready no matter what any of the other subs do. If this is the case, I'd expect the same to apply to other dependent Futures, ->needs_any for example - I can put together a better set of test cases (and/or patches) if so. cheers, Tom
Subject: 2014-06-24-needs_all_sub_cancellation.patch
diff -uNPr Future-0.28-ej0TdH/t/12needs_all.t Future-0.28-H0UgpN/t/12needs_all.t --- Future-0.28-ej0TdH/t/12needs_all.t 2014-06-08 22:57:28.000000000 +0100 +++ Future-0.28-H0UgpN/t/12needs_all.t 2014-06-24 14:42:16.751997673 +0100 @@ -122,6 +122,20 @@ is( $c2, undef, '$future->cancel ignores ready subs' ); } +# cancel detection +{ + my $f1 = Future->new; + my $f2 = Future->new; + + my $future = Future->needs_all( $f1, $f2 ); + + $f1->cancel; + $f2->cancel; + + ok( $future->is_ready, 'marked as ready after all subs are cancelled'); + ok( $future->is_cancelled, 'marked as cancelled after all subs are cancelled'); +} + # needs_all on none { my $f = Future->needs_all( () );
From IRL discussion; in summary: ->wait_all needs to treat cancellation as a success - the combination as a whole didn't fail. The called completion code itself will fail if it calls ->get on one of the futures that was cancelled. ->wait_any and ->needs_any need to treat cancellation as a failure of one component; just stop watching it and continue waiting on the others. If the last component is cancelled, it will fail with a specific message indicating so ->needs_all needs to treat cancellation as a failure; cancel the remaining components and cause the combination to fail (reuse message from above). -- Paul Evans
Subject: Future cancellation propagation bugs
Another entire cancellation problem that isn't currently handled is divergent trees: my $ffirst = Future->new; my @fs = map { $ffirst->then(sub{ Future->done(1) }) } 0 .. 9; $fs[0]->cancel; will currently cancel $ffirst, and therefore fail $fs[1] to $fs[9]. These probably ought to refcount somehow, and not bother cancelling $ffirst until the last one goes. -- Paul Evans
Attached patch handles -some- of this; the initial concern about aggregate convergent trees. Still under consideration is how to handle divergent trees, and also how to handle simple sequencing. I've added a long blob of notes in the documentation under a "KNOWN ISSUES" section, so we can at least release this and continue considering design. -- Paul Evans
Subject: rt96685.patch
=== modified file 'lib/Future.pm' --- lib/Future.pm 2014-06-08 21:57:09 +0000 +++ lib/Future.pm 2014-06-26 18:23:14 +0000 @@ -1335,7 +1335,8 @@ Returns a new C<Future> instance that will indicate it is ready once all of the sub future objects given to it indicate that they are ready, either by -success or failure. Its result will a list of its component futures. +success, failure or cancellation. Its result will a list of its component +futures. When given an empty list this constructor returns a new immediately-done future. @@ -1369,7 +1370,6 @@ weaken( my $weakself = $self ); my $sub_on_ready = sub { - return if $_[0]->{cancelled}; return unless $weakself; $pending--; @@ -1392,7 +1392,9 @@ the sub future objects given to it indicate that they are ready, either by success or failure. Any remaining component futures that are not yet ready will be cancelled. Its result will be the result of the first component future -that was ready; either success or failure. +that was ready; either success or failure. Any component futures that are +cancelled are ignored, apart from the final component left; at which point the +result will be a failure. When given an empty list this constructor returns an immediately-failed future. @@ -1435,27 +1437,36 @@ return $self; } + my $pending = 0; + weaken( my $weakself = $self ); my $sub_on_ready = sub { - return if $_[0]->{cancelled}; return unless $weakself; - - foreach my $sub ( @subs ) { - $sub->{ready} or $sub->cancel; + return if $weakself->{result} or $weakself->{failure}; # don't recurse on child ->cancel + + return if --$pending and $_[0]->{cancelled}; + + if( $_[0]->{cancelled} ) { + $weakself->{failure} = [ "All component futures were cancelled" ]; } - - if( $_[0]->{failure} ) { + elsif( $_[0]->{failure} ) { $weakself->{failure} = [ $_[0]->failure ]; } else { $weakself->{result} = [ $_[0]->get ]; } + + foreach my $sub ( @subs ) { + $sub->{ready} or $sub->cancel; + } + $weakself->_mark_ready; }; foreach my $sub ( @subs ) { # No need to test $sub->{ready} since we know none of them are $sub->on_ready( $sub_on_ready ); + $pending++; } return $self; @@ -1467,7 +1478,8 @@ sub future objects given to it indicate that they have completed successfully, or when any of them indicates that they have failed. If any sub future fails, then this will fail immediately, and the remaining subs not yet ready will be -cancelled. +cancelled. Any component futures that are cancelled will cause an immediate +failure of the result. If successful, its result will be a concatenated list of the results of all its component futures, in corresponding order. If it fails, its failure will @@ -1522,14 +1534,21 @@ weaken( my $weakself = $self ); my $sub_on_ready = sub { - return if $_[0]->{cancelled}; return unless $weakself; + return if $weakself->{result} or $weakself->{failure}; # don't recurse on child ->cancel - if( my @failure = $_[0]->failure ) { + if( $_[0]->{cancelled} ) { + $weakself->{failure} = [ "All component futures were cancelled" ]; foreach my $sub ( @subs ) { $sub->cancel if !$sub->{ready}; } + $weakself->_mark_ready; + } + elsif( my @failure = $_[0]->failure ) { $weakself->{failure} = \@failure; + foreach my $sub ( @subs ) { + $sub->cancel if !$sub->{ready}; + } $weakself->_mark_ready; } else { @@ -1554,7 +1573,9 @@ the sub future objects given to it indicate that they have completed successfully, or when all of them indicate that they have failed. If any sub future succeeds, then this will succeed immediately, and the remaining subs -not yet ready will be cancelled. +not yet ready will be cancelled. Any component futures that are cancelled are +ignored, apart from the final component left; at which point the result will +be a failure. If successful, its result will be that of the first component future that succeeded. If it fails, its failure will be that of the last component future @@ -1618,22 +1639,26 @@ weaken( my $weakself = $self ); my $sub_on_ready = sub { - return if $_[0]->{cancelled}; return unless $weakself; - - $pending--; - - if( my @failure = $_[0]->failure ) { + return if $weakself->{result} or $weakself->{failure}; # don't recurse on child ->cancel + + return if --$pending and $_[0]->{cancelled}; + + if( $_[0]->{cancelled} ) { + $weakself->{failure} = [ "All component futures were cancelled" ]; + $weakself->_mark_ready; + } + elsif( my @failure = $_[0]->failure ) { $pending and return; $weakself->{failure} = \@failure; $weakself->_mark_ready; } else { - foreach my $sub ( @subs ) { - $sub->cancel if !$sub->{ready}; - } $weakself->{result} = [ $_[0]->get ]; + foreach my $sub ( @subs ) { + $sub->cancel if !$sub->{ready}; + } $weakself->_mark_ready; } }; === modified file 't/10wait_all.t' --- t/10wait_all.t 2014-03-26 15:09:41 +0000 +++ t/10wait_all.t 2014-06-26 18:23:14 +0000 @@ -129,6 +129,26 @@ is( $c2, undef, '$future->cancel ignores ready subs' ); } +# cancelled dependents +{ + my $f1 = Future->new; + my $f2 = Future->new; + + my $future = Future->wait_all( $f1, $f2 ); + + $f1->done( "result" ); + $f2->cancel; + + ok( $future->is_ready, '$future of cancelled sub is ready after final cancellation' ); + + is_deeply( [ $future->done_futures ], + [ $f1 ], + '->done_futures with cancellation' ); + is_deeply( [ $future->cancelled_futures ], + [ $f2 ], + '->cancelled_futures with cancellation' ); +} + # wait_all on none { my $f = Future->wait_all( () ); === modified file 't/11wait_any.t' --- t/11wait_any.t 2014-03-26 15:09:41 +0000 +++ t/11wait_any.t 2014-06-26 18:23:14 +0000 @@ -108,6 +108,38 @@ is( $c1, 1, '$future->cancel marks subs cancelled' ); } +# cancelled dependents +{ + my $f1 = Future->new; + my $f2 = Future->new; + + my $future = Future->wait_any( $f1, $f2 ); + + $f1->cancel; + + ok( !$future->is_ready, '$future not yet ready after first cancellation' ); + + $f2->done( "result" ); + + ok( $future->is_ready, '$future is ready' ); + + is_deeply( [ $future->done_futures ], + [ $f2 ], + '->done_futures with cancellation' ); + is_deeply( [ $future->cancelled_futures ], + [ $f1 ], + '->cancelled_futures with cancellation' ); + + my $f3 = Future->new; + $future = Future->wait_any( $f3 ); + + $f3->cancel; + + ok( $future->is_ready, '$future is ready after final cancellation' ); + + like( scalar $future->failure, qr/ cancelled/, 'Failure mentions cancelled' ); +} + # wait_any on none { my $f = Future->wait_any( () ); === modified file 't/12needs_all.t' --- t/12needs_all.t 2014-03-26 15:09:41 +0000 +++ t/12needs_all.t 2014-06-26 18:23:14 +0000 @@ -122,6 +122,20 @@ is( $c2, undef, '$future->cancel ignores ready subs' ); } +# cancelled dependents +{ + my $f1 = Future->new; + my $f2 = Future->new; + + my $future = Future->needs_all( $f1, $f2 ); + + $f1->cancel; + + ok( $future->is_ready, '$future of cancelled sub is ready after first cancellation' ); + + like( scalar $future->failure, qr/ cancelled/, 'Failure mentions cancelled' ); +} + # needs_all on none { my $f = Future->needs_all( () ); === modified file 't/13needs_any.t' --- t/13needs_any.t 2014-03-26 15:09:41 +0000 +++ t/13needs_any.t 2014-06-26 18:23:14 +0000 @@ -157,6 +157,36 @@ is( $c2, undef, '$future->cancel ignores ready subs' ); } +# cancelled dependents +{ + my $f1 = Future->new; + my $f2 = Future->new; + + my $future = Future->needs_any( $f1, $f2 ); + + $f1->cancel; + + ok( !$future->is_ready, '$future not yet ready after first cancellation' ); + + $f2->done( "result" ); + + is_deeply( [ $future->done_futures ], + [ $f2 ], + '->done_futures with cancellation' ); + is_deeply( [ $future->cancelled_futures ], + [ $f1 ], + '->cancelled_futures with cancellation' ); + + my $f3 = Future->new; + $future = Future->needs_any( $f3 ); + + $f3->cancel; + + ok( $future->is_ready, '$future is ready after final cancellation' ); + + like( scalar $future->failure, qr/ cancelled/, 'Failure mentions cancelled' ); +} + # needs_any on none { my $f = Future->needs_any( () );
This is my opinion on the "KNOWN ISSUES". - Cancellation of Non-Final Sequence Futures I'm not sure what is the correct behavior. On the other hand, I agree on the behavior of wait_all(), wait_any(), needs_all() and needs_any() introduced in 0.29. That's because those methods have enough semantics to decide what is most likely to be the correct behavior. - Cancellation of Divergent Flow I think reference-count is not necessary. Cancelling a single subsequent future (say, $f1) should immediately cancel the original future ($f_initial). That is because, in my opinion, calling cancel() is like saying "Stop whatever you're doing NOW!", not "I don't care your result anymore". If the consumer loses interest on $f_initial's result, they can just ignore it in then() callbacks or even throw $f1 away. They don't need to call cancel(). cancel() method is an active request to stop something in progress. - FIY(1): Future::Q I felt cancel() was tricky when I designed the spec of Future::Q module, so I decided to keep it as simple as possible. In Future::Q, cancellation of a future just propagates in BOTH directions. For example, suppose you have my $f2 = $f1->then(\&callback1); my $f3 = $f2->then(\&callback2); and those futures are all pending. Then, if you do $f2->cancel(), it cancels both $f1 and $f3, and none of the callbacks is executed. So if you have a very complex tree of pending futures, cancelling ANY future always cancels the entire tree of futures. I'm not so sure if this behavior is useful, because I've never used cancel() as a matter of fact. Nonetheless I like it because it's simple, understandable, predictable and easy to implement. - FIY(2): q.js One of the authors of q.js (JavaScript Promise module) explains why cancellation is tricky (and so q.js does not have cancellation feature). https://github.com/kriskowal/q/issues/64 Note that the situation is a bit different from us, because q.js usually gives its users "promises", that is, read-only futures. He mentions reference-counting.
s/FIY/FYI/g; Silly typo..
I am coming to the conclusion that there are two ways to handle this: a) Add a ->reuse method which returns the Future instance itself, but marks that the Future needs one extra ->cancel before it will take effect. E.g. my $f_initial = ... my $f1 = Future->needs_all( $f_initial ); my $f2 = Future->needs_all( $f_initial->reuse ); or b) Have DESTROY make an implicit ->cancel on its parents, and no longer use 'cancel' when dropping a convergent Future. This means that Perl's refcounting alone would handle it. Allegedly. Of these two I massively prefer 'a', because relying on Perl refcounting seems far too subtle and awkward. -- Paul Evans
Firstly, I'm also in agreement on the new wait_all(), wait_any(), needs_all() and needs_any() behaviours in 0.29: they're consistent and from my initial testing work well in resolving the issues that originally prompted this ticket. Thanks for that! On the wider issue: Cancellation is indeed useful when considered as something stronger than "I am no longer interested": it provides a way to stop a task, and there are situations where this can be very useful. I use cancellation in this sense in a lot of code, and find the lack of cancellation in other systems (Promises/A, etc.) very limiting. Although I do agree in some respects with the simplicity of the recursive cancellation approach used in Future::Q (cancelling all futures in both directions), in practice it's less useful than the current, "manual" Future behaviour. An example: Given a web framework which uses Futures to represent outstanding requests, at one given point you might have two Futures representing independent, active web requests. Both of these depend on a third Future - a long-running database request, maybe. If the client happens to disconnect early, cancellation of the request Future is a natural way to represent this. If one of those clients disconnects, and you cancel the corresponding Future, that cancellation will now propagate to the database request. That means your other web request will also be cancelled. This becomes more complicated in larger applications, and I find it's quite common to have interleaved trees which could easily result in a complete application shutdown due to the ripple effect as those cancellations propagate outwards. I'd argue that $f should never be cancelled in the following code: my $f = Future->new; Future->needs_all($f)->cancel; Future->needs_all($f); Propagating the cancellation here is essentially conflating two meanings of cancellation: "stop doing this task", and "you're no longer important". If a piece of code is given a Future, and later decides that the Future is no longer important, it often does not (and I'd argue, should not) know enough about the component Futures (if any) to know whether they are needed any more. Each Future can be used in multiple dependent Futures *at any point during its lifecycle*. I think this also means neither refcounting option will be viable - the current refcount is a transient thing, and may increase before the Future would have been marked ready. Might be some mileage in a callback, similar to ->on_cancel, which will be triggered if we're used by a dependent future which is subsequently cancelled. Could possibly also add a ->cancel_recursively method (and if partial propagation is required, that would need to be handled by iterating through the tree manually). In summary: my preference would be to keep the current "manual" cancellation behaviour. Attaching ->on_cancel handlers seems sufficient to work around any limitations with the manual approach. Tom
Tom, Thanks for the explanation. I understand the situation where you don't want the immediate propagation of cancel(). Paul, The option (a) looks good to me. It would be simple enough both in concept and in implementation.
One related issue is the sequence handling in ->else_done: $ cat test.pl use Future; my $f = Future->new->set_label("original future"); my $f2 = $f->else_done->set_label("->else_done"); $f->cancel; $ mirai-trace test.pl Future=HASH(0x9ac24a0) created at test.pl:2 Future=HASH(0x9ac24a0) was given a label, and it was: original future Future=HASH(0x9ac27d4) created at .../Future.pm:1040 Future=HASH(0x9ac27d4) was given a label, and it was: ->else_done Future=HASH(0x9ac24a0) was marked ready: cancelled, elapsed time 0.002s Future=HASH(0x9ac27d4) went away, it was pending and had the label ->else_done Future=HASH(0x9ac24a0) went away, it was cancelled and had the label original future This comes up in IO::Async-using code with cases such as: my $f = Future->new; $notifier->adopt_future( $f->else_done ); $f->cancel; # notifier's ->else_done future is still pending Pretty sure that the correct behaviour here should be ->else_done marked as cancelled when the original was cancelled. If I can find a spare moment I'll try to go through the other similar methods to see what else might have similar issues. cheers, Tom
Written the basic sequencing methods up as test cases, ignoring ->needs_all etc. This is how I think they should work, needs feedback though! cheers, Tom
Subject: 2015-01-28-future.t
use strict; use warnings; use Test::More; use Future; subtest '->then($code)' => sub { my $f = Future->new->set_label('leaf'); my $then = $f->then(sub { Future->fail("should not be called") })->set_label('leaf->then'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($then->is_ready, '->then is ready'); ok($then->is_cancelled, '->then is cancelled when dependent is cancelled'); }; subtest '->then($code, $code)' => sub { my $f = Future->new->set_label('leaf'); my $then = $f->then(sub { Future->fail("should not be called") }, sub { Future->fail("also should not be called") })->set_label('leaf->then(CV,CV)'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($then->is_ready, '->then is ready'); ok($then->is_cancelled, '->then is cancelled when dependent is cancelled'); }; subtest '->then_with_f($code)' => sub { my $f = Future->new->set_label('leaf'); my $then_with_f = $f->then_with_f(sub { Future->fail("should not be called") })->set_label('leaf->then_with_f'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($then_with_f->is_ready, '->then_with_f is ready'); ok($then_with_f->is_cancelled, '->then_with_f is cancelled when dependent is cancelled'); }; subtest '->then_done($code)' => sub { my $f = Future->new->set_label('leaf'); my $then_done = $f->then_done("should not be called")->set_label('leaf->then_done'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($then_done->is_ready, '->then_done is ready'); ok($then_done->is_cancelled, '->then_done is cancelled when dependent is cancelled'); }; subtest '->then_fail($code)' => sub { my $f = Future->new->set_label('leaf'); my $then_fail = $f->then_fail("should not be called")->set_label('leaf->then_fail'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($then_fail->is_ready, '->then_fail is ready'); ok($then_fail->is_cancelled, '->then_fail is cancelled when dependent is cancelled'); }; subtest '->else' => sub { my $f = Future->new->set_label('leaf'); my $else = $f->else(sub { Future->fail("should not be called") })->set_label('leaf->else'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($else->is_ready, '->else is ready'); ok($else->is_cancelled, '->else is cancelled when dependent is cancelled'); }; subtest '->else_with_f' => sub { my $f = Future->new->set_label('leaf'); my $else_with_f = $f->else_with_f(sub { Future->fail("should not be called") })->set_label('leaf->else_with_f'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($else_with_f->is_ready, '->else_with_f is ready'); ok($else_with_f->is_cancelled, '->else_with_f is cancelled when dependent is cancelled'); }; subtest '->transform' => sub { my $f = Future->new->set_label('leaf'); my $transform = $f->transform( done => sub { Future->fail("should not be called") }, fail => sub { Future->fail("also should not be called") } )->set_label('leaf->transform'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($transform->is_ready, '->transform is ready'); ok($transform->is_cancelled, '->transform is cancelled when dependent is cancelled'); }; subtest '->without_cancel($code, $code)' => sub { my $f = Future->new->set_label('leaf'); my $without_cancel = $f->without_cancel->set_label('leaf->without_cancel'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($without_cancel->is_ready, '->without_cancel is ready'); ok($without_cancel->is_cancelled, '->without_cancel is cancelled when dependent is cancelled'); }; subtest '->followed_by($code)' => sub { { my $f = Future->new->set_label('leaf'); my $followed_by = $f->followed_by(sub { Future->fail("should not be called") })->set_label('leaf->followed_by'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($followed_by->is_ready, '->followed_by is ready'); ok($followed_by->failure, '->followed_by executed code which returned Future->fail'); } { my $f = Future->new->set_label('leaf'); my $followed_by = $f->followed_by(sub { Future->done("this is fine") })->set_label('leaf->followed_by'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($followed_by->is_ready, '->followed_by is ready'); ok($followed_by->is_done, '->followed_by executed code which returned Future->done'); } }; done_testing;
Subject: 2015-01-28-future.tap
Download 2015-01-28-future.tap
application/octet-stream 4.8k

Message body not shown because it is not plain text.

Updated test case (mostly typos), and a patch in case anyone else wants to try this version of the logic. Note that this is more a proof-of-concept than an official patch. All IO::Async tests pass with this applied, there's 3 failures in the Future testsuite, all caused by refcount differences. None of them were obvious leaks on first glance, will be able to confirm after running with this for a few days. cheers, Tom On Wed Jan 28 16:04:10 2015, TEAM wrote: Show quoted text
> Written the basic sequencing methods up as test cases, ignoring > ->needs_all etc. This is how I think they should work, needs feedback > though! > > cheers, > > Tom
Subject: 2015-01-28-future.t
use strict; use warnings; use Test::More; use Future; subtest '->then($code)' => sub { my $f = Future->new->set_label('leaf'); my $then = $f->then(sub { Future->fail("should not be called") })->set_label('leaf->then'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($then->is_ready, '->then is ready'); ok($then->is_cancelled, '->then is cancelled when dependent is cancelled'); }; subtest '->then($code, $code)' => sub { my $f = Future->new->set_label('leaf'); my $then = $f->then(sub { Future->fail("should not be called") }, sub { Future->fail("also should not be called") })->set_label('leaf->then(CV,CV)'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($then->is_ready, '->then is ready'); ok($then->is_cancelled, '->then is cancelled when dependent is cancelled'); }; subtest '->then_with_f($code)' => sub { my $f = Future->new->set_label('leaf'); my $then_with_f = $f->then_with_f(sub { Future->fail("should not be called") })->set_label('leaf->then_with_f'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($then_with_f->is_ready, '->then_with_f is ready'); ok($then_with_f->is_cancelled, '->then_with_f is cancelled when dependent is cancelled'); }; subtest '->then_done($code)' => sub { my $f = Future->new->set_label('leaf'); my $then_done = $f->then_done("should not be called")->set_label('leaf->then_done'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($then_done->is_ready, '->then_done is ready'); ok($then_done->is_cancelled, '->then_done is cancelled when dependent is cancelled'); }; subtest '->then_fail($value)' => sub { my $f = Future->new->set_label('leaf'); my $then_fail = $f->then_fail("should not be called")->set_label('leaf->then_fail'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($then_fail->is_ready, '->then_fail is ready'); ok($then_fail->is_cancelled, '->then_fail is cancelled when dependent is cancelled'); }; subtest '->else' => sub { my $f = Future->new->set_label('leaf'); my $else = $f->else(sub { Future->fail("should not be called") })->set_label('leaf->else'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($else->is_ready, '->else is ready'); ok($else->is_cancelled, '->else is cancelled when dependent is cancelled'); }; subtest '->else_with_f' => sub { my $f = Future->new->set_label('leaf'); my $else_with_f = $f->else_with_f(sub { Future->fail("should not be called") })->set_label('leaf->else_with_f'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($else_with_f->is_ready, '->else_with_f is ready'); ok($else_with_f->is_cancelled, '->else_with_f is cancelled when dependent is cancelled'); }; subtest '->transform' => sub { my $f = Future->new->set_label('leaf'); my $transform = $f->transform( done => sub { Future->fail("should not be called") }, fail => sub { Future->fail("also should not be called") } )->set_label('leaf->transform'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($transform->is_ready, '->transform is ready'); ok($transform->is_cancelled, '->transform is cancelled when dependent is cancelled'); }; subtest '->without_cancel($code, $code)' => sub { my $f = Future->new->set_label('leaf'); my $without_cancel = $f->without_cancel->set_label('leaf->without_cancel'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($without_cancel->is_ready, '->without_cancel is ready'); ok($without_cancel->is_cancelled, '->without_cancel is cancelled when dependent is cancelled'); }; subtest '->followed_by($code)' => sub { { my $f = Future->new->set_label('leaf'); my $followed_by = $f->followed_by(sub { Future->fail("should be called") })->set_label('leaf->followed_by'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($followed_by->is_ready, '->followed_by is ready'); ok($followed_by->failure, '->followed_by executed code which returned Future->fail'); } { my $f = Future->new->set_label('leaf'); my $followed_by = $f->followed_by(sub { Future->done("this is fine") })->set_label('leaf->followed_by'); $f->cancel; ok($f->is_ready, 'leaf is ready'); ok($followed_by->is_ready, '->followed_by is ready'); ok($followed_by->is_done, '->followed_by executed code which returned Future->done'); } }; done_testing;
Subject: 2015-01-28-sequence-future-cancel.patch
diff --git a/lib/Future.pm b/lib/Future.pm index a5edc8f..4e21607 100644 --- a/lib/Future.pm +++ b/lib/Future.pm @@ -196,6 +196,8 @@ use constant { CB_SEQ_IMDONE => 1<<7, # $code is in fact immediate ->done result CB_SEQ_IMFAIL => 1<<8, # $code is in fact immediate ->fail result + + CB_SEQ_ONCANCEL => 1<<9, # $code wants to be called even on dependent ->cancel }; use constant CB_ALWAYS => CB_DONE|CB_FAIL|CB_CANCEL; @@ -370,7 +372,8 @@ sub _mark_ready my $f2; if( $done and $flags & CB_SEQ_ONDONE or - $fail and $flags & CB_SEQ_ONFAIL ) { + $fail and $flags & CB_SEQ_ONFAIL or + $cancelled and $flags & CB_SEQ_ONCANCEL) { if( $flags & CB_SEQ_IMDONE ) { $fseq->done( @$code ); @@ -940,7 +943,7 @@ sub cancel return $self if $self->{ready}; $self->{cancelled}++; - foreach my $code ( reverse @{ $self->{on_cancel} || [] } ) { + foreach my $code ( reverse @{ delete $self->{on_cancel} || [] } ) { my $is_future = blessed( $code ) && $code->isa( "Future" ); $is_future ? $code->cancel : $code->( $self ); @@ -1011,6 +1014,9 @@ sub _sequence return $f1 if $f1->is_done and not( $flags & CB_SEQ_ONDONE ) or $f1->failure and not( $flags & CB_SEQ_ONFAIL ); + # Immediate-cancel - don't call our code, but return something that's already cancelled + return Future->new->cancel if $f1->is_cancelled and not( $flags & CB_SEQ_ONCANCEL ); + if( $flags & CB_SEQ_IMDONE ) { return Future->done( @$code ); } @@ -1039,6 +1045,7 @@ sub _sequence my $fseq = $f1->new; $fseq->on_cancel( $f1 ); + $f1->on_cancel( $fseq ) unless $flags & CB_SEQ_ONCANCEL; push @{ $f1->{callbacks} }, [ CB_DONE|CB_FAIL|$flags, $code, $fseq ]; weaken( $f1->{callbacks}[-1][2] ); @@ -1272,7 +1279,7 @@ sub followed_by my $self = shift; my ( $code ) = @_; - return $self->_sequence( $code, CB_SEQ_ONDONE|CB_SEQ_ONFAIL|CB_SELF ); + return $self->_sequence( $code, CB_SEQ_ONDONE|CB_SEQ_ONFAIL|CB_SEQ_ONCANCEL|CB_ALWAYS|CB_SELF ); } =head2 $future = $f1->and_then( \&code ) @@ -1338,15 +1345,9 @@ sub without_cancel my $self = shift; my $new = $self->new; - $self->on_ready( sub { - my $self = shift; - if( $self->failure ) { - $new->fail( $self->failure ); - } - else { - $new->done( $self->get ); - } - }); + $self->on_done( sub { $new->done(@_) } ); + $self->on_fail( sub { $new->fail(@_) } ); + $self->on_cancel( sub { $new->cancel } ); return $new; }
Some additional inspiration may be found in Bluebird (a JS promisy-futury library): http://bluebirdjs.com/docs/api/cancellation.html -- Paul Evans
Updated patch for 0.34. Note that this still has refcount test failures. I've been using a slightly-modified version of this in production for a couple of years now, seems stable enough even in larger applications. Without this, things tend to leak memory over time, especially for situations like websockets or servers that rely on Future cancellation to clean up on user disconnect. On 2015-01-28 18:20:03, TEAM wrote: Show quoted text
> Updated test case (mostly typos), and a patch in case anyone else > wants to try this version of the logic. Note that this is more a > proof-of-concept than an official patch. > > All IO::Async tests pass with this applied, there's 3 failures in the > Future testsuite, all caused by refcount differences. None of them > were obvious leaks on first glance, will be able to confirm after > running with this for a few days. > > cheers, > > Tom > > On Wed Jan 28 16:04:10 2015, TEAM wrote:
> > Written the basic sequencing methods up as test cases, ignoring > > ->needs_all etc. This is how I think they should work, needs feedback > > though! > > > > cheers, > > > > Tom
Subject: 2017-01-02-sequence-future-cancel.patch
commit 1cf850f79f56ad29f6285e50ada2809b4d0806b7 Author: Tom Molesworth <tom@binary.com> Date: Mon Jan 2 23:04:56 2017 +0800 Reapply https://rt.cpan.org/Public/Bug/Display.html?id=96685 diff --git a/lib/Future.pm b/lib/Future.pm index 2f66b64..3f0833a 100644 --- a/lib/Future.pm +++ b/lib/Future.pm @@ -220,6 +220,7 @@ use constant { CB_SEQ_IMDONE => 1<<7, # $code is in fact immediate ->done result CB_SEQ_IMFAIL => 1<<8, # $code is in fact immediate ->fail result + CB_SEQ_ONCANCEL => 1<<9, # $code wants to be called even on dependent ->cancel }; use constant CB_ALWAYS => CB_DONE|CB_FAIL|CB_CANCEL; @@ -417,7 +418,8 @@ sub _mark_ready my $f2; if( $done and $flags & CB_SEQ_ONDONE or - $fail and $flags & CB_SEQ_ONFAIL ) { + $fail and $flags & CB_SEQ_ONFAIL or + $cancelled and $flags & CB_SEQ_ONCANCEL) { if( $flags & CB_SEQ_IMDONE ) { $fseq->done( @$code ); @@ -979,7 +981,7 @@ sub cancel return $self if $self->{ready}; $self->{cancelled}++; - foreach my $code ( reverse @{ $self->{on_cancel} || [] } ) { + foreach my $code ( reverse @{ delete $self->{on_cancel} || [] } ) { my $is_future = blessed( $code ) && $code->isa( "Future" ); $is_future ? $code->cancel : $code->( $self ); @@ -1039,6 +1041,9 @@ sub _sequence return $f1 if $f1->is_done and not( $flags & CB_SEQ_ONDONE ) or $f1->failure and not( $flags & CB_SEQ_ONFAIL ); + # Immediate-cancel - don't call our code, but return something that's already cancelled + return Future->new->cancel if $f1->is_cancelled and not( $flags & CB_SEQ_ONCANCEL ); + if( $flags & CB_SEQ_IMDONE ) { return Future->done( @$code ); } @@ -1067,6 +1072,7 @@ sub _sequence my $fseq = $f1->new; $fseq->on_cancel( $f1 ); + $f1->on_cancel( $fseq ) unless $flags & CB_SEQ_ONCANCEL; # TODO: if anyone cares about the op name, we might have to synthesize it # from $flags @@ -1455,7 +1461,7 @@ sub followed_by my $self = shift; my ( $code ) = @_; - return $self->_sequence( $code, CB_SEQ_ONDONE|CB_SEQ_ONFAIL|CB_SELF ); + return $self->_sequence( $code, CB_SEQ_ONDONE|CB_SEQ_ONFAIL|CB_SEQ_ONCANCEL|CB_ALWAYS|CB_SELF ); } =head2 without_cancel @@ -1477,15 +1483,9 @@ sub without_cancel my $self = shift; my $new = $self->new; - $self->on_ready( sub { - my $self = shift; - if( $self->failure ) { - $new->fail( $self->failure ); - } - else { - $new->done( $self->get ); - } - }); + $self->on_done( sub { $new->done(@_) } ); + $self->on_fail( sub { $new->fail(@_) } ); + $self->on_cancel( sub { $new->cancel } ); return $new; } diff --git a/t/2015-01-28-future.t b/t/2015-01-28-future.t new file mode 100644 index 0000000..848140f --- /dev/null +++ b/t/2015-01-28-future.t @@ -0,0 +1,115 @@ +use strict; +use warnings; + +use Test::More; +use Future; + +subtest '->then($code)' => sub { + my $f = Future->new->set_label('leaf'); + my $then = $f->then(sub { Future->fail("should not be called") })->set_label('leaf->then'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($then->is_ready, '->then is ready'); + ok($then->is_cancelled, '->then is cancelled when dependent is cancelled'); +}; + +subtest '->then($code, $code)' => sub { + my $f = Future->new->set_label('leaf'); + my $then = $f->then(sub { Future->fail("should not be called") }, sub { Future->fail("also should not be called") })->set_label('leaf->then(CV,CV)'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($then->is_ready, '->then is ready'); + ok($then->is_cancelled, '->then is cancelled when dependent is cancelled'); +}; + +subtest '->then_with_f($code)' => sub { + my $f = Future->new->set_label('leaf'); + my $then_with_f = $f->then_with_f(sub { Future->fail("should not be called") })->set_label('leaf->then_with_f'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($then_with_f->is_ready, '->then_with_f is ready'); + ok($then_with_f->is_cancelled, '->then_with_f is cancelled when dependent is cancelled'); +}; + +subtest '->then_done($code)' => sub { + my $f = Future->new->set_label('leaf'); + my $then_done = $f->then_done("should not be called")->set_label('leaf->then_done'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($then_done->is_ready, '->then_done is ready'); + ok($then_done->is_cancelled, '->then_done is cancelled when dependent is cancelled'); +}; + +subtest '->then_fail($value)' => sub { + my $f = Future->new->set_label('leaf'); + my $then_fail = $f->then_fail("should not be called")->set_label('leaf->then_fail'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($then_fail->is_ready, '->then_fail is ready'); + ok($then_fail->is_cancelled, '->then_fail is cancelled when dependent is cancelled'); +}; + +subtest '->else' => sub { + my $f = Future->new->set_label('leaf'); + my $else = $f->else(sub { Future->fail("should not be called") })->set_label('leaf->else'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($else->is_ready, '->else is ready'); + ok($else->is_cancelled, '->else is cancelled when dependent is cancelled'); +}; + +subtest '->else_with_f' => sub { + my $f = Future->new->set_label('leaf'); + my $else_with_f = $f->else_with_f(sub { Future->fail("should not be called") })->set_label('leaf->else_with_f'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($else_with_f->is_ready, '->else_with_f is ready'); + ok($else_with_f->is_cancelled, '->else_with_f is cancelled when dependent is cancelled'); +}; + +subtest '->transform' => sub { + my $f = Future->new->set_label('leaf'); + my $transform = $f->transform( + done => sub { Future->fail("should not be called") }, + fail => sub { Future->fail("also should not be called") } + )->set_label('leaf->transform'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($transform->is_ready, '->transform is ready'); + ok($transform->is_cancelled, '->transform is cancelled when dependent is cancelled'); +}; + +subtest '->without_cancel($code, $code)' => sub { + my $f = Future->new->set_label('leaf'); + my $without_cancel = $f->without_cancel->set_label('leaf->without_cancel'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($without_cancel->is_ready, '->without_cancel is ready'); + ok($without_cancel->is_cancelled, '->without_cancel is cancelled when dependent is cancelled'); +}; + +subtest '->followed_by($code)' => sub { + { + my $f = Future->new->set_label('leaf'); + my $followed_by = $f->followed_by(sub { + Future->fail("should be called") + })->set_label('leaf->followed_by'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($followed_by->is_ready, '->followed_by is ready'); + ok($followed_by->failure, '->followed_by executed code which returned Future->fail'); + } + { + my $f = Future->new->set_label('leaf'); + my $followed_by = $f->followed_by(sub { + Future->done("this is fine") + })->set_label('leaf->followed_by'); + $f->cancel; + ok($f->is_ready, 'leaf is ready'); + ok($followed_by->is_ready, '->followed_by is ready'); + ok($followed_by->is_done, '->followed_by executed code which returned Future->done'); + } +}; + +done_testing; +