`cancelawait` keyword to abort an async function call

[andrewrk]

I've spent many hours in the past trying to solve this, and never quite tied up all the loose ends, but I think I've done it this time.

Related Proposals:

• add "select" syntax to the language to await the first function in a given set that completes add "select" syntax to the language to await the first function in a given set that completes #5263
• a way to check async frame liveness Proposal: a way to check async frame liveness #3164
• ability to annotate functions which allocate resources, with a way to deallocate the returned resources ability to annotate functions which allocate resources, with a way to deallocate the returned resources #782

Problem 1: Error Handling & Resource Management

Typical async await usage when multiple async functions are "in-flight", written naively, looks like this:

fn asyncAwaitTypicalUsage(allocator: *Allocator) !void {
    var download_frame = async fetchUrl(allocator, "https://example.com/");
    var file_frame = async readFile(allocator, "something.txt");

    const download_text = try await download_frame; // NO GOOD!!!
    defer allocator.free(download_text);

    const file_text = try await file_frame;
    defer allocator.free(file_text);
}

Spot the problem? If the first try returns an error, the in-flight file_frame becomes invalid memory while the readFile function is still using the memory. This is nasty undefined behavior. It's too easy to do this on accident.

Problem 2: The Await Result Location

Function calls directly write their return values into the result locations. This is important for pinned memory, and will become more noticeable when these are implemented:

• result location: ability to refer to the return result location before the return statement result location: ability to refer to the return result location before the return statement #2765
• result locations: unwrap optional and error unions so that the payload can be non-copied result locations: unwrap optional and error unions so that the payload can be non-copied #2761
• Ability to mark a struct field as "pinned" in memory Ability to mark a struct field as "pinned" in memory #3803
• make aggregate types non-copyable by default; provide copyable attribute make aggregate types non-copyable by default; provide copyable attribute #3804

However this breaks when using async and await. It is possible to use the advanced builtin @asyncCall and pass a result location pointer to async, but there is not a way to do it with await. The duality is messy, and a function that relies on pinning its return value will have its guarantees broken when it becomes an async function.

Solution

I've tried a bunch of other ideas before, but nothing could quite give us good enough semantics. But now I've got something that solves both problems. The key insight was making obtaining a result location pointer for the return statement of an async function, implicitly a suspend point. This suspends the async function at the return statement, to be resumed by the await site, which will pass it a result location pointer. The crucial point here is that it also provides a suspension point that can be used for cancelawait to activate. If an async function is cancelled, then it resumes, but instead of returning a value, it runs the errdefer and defer expressions that are in scope. So - async functions will simply have to retain the property that idiomatic code already has, which is that all the cleanup that possibly needs to be done is in scope in a defer at a return statement.

I think this is the best of both worlds, between automatically running a function up to the first suspend point, and what e.g. Rust does, not running a function until await is called. A function can introduce an intentional copy of the result data, if it wishes to run the logic in the return expression before an await result pointer is available. It means async function frames can get smaller, because they no longer need the return value in the frame.

Now this leaves the problem of blocking functions which are used with async/await, and what cancelawait does to them. The proposal #782 is open for that purpose, but it has a lot of flaws. Again, here, the key insight of await working properly with result location pointers was the answer. If we move the function call of non-suspending functions used with async/await to happen at the await site instead of the async site, then cancelawait becomes a no-op. async will simply copy the parameters into the frame, and await would do the actual function call. Note that function parameters must be copied anyway for all function calls, so this comes at no penalty, and in fact should be better all around because we don't have "undoing" of allocated resources but we have simply not doing extra work in the first place.

Example code:

fn asyncAwaitTypicalUsage(allocator: *Allocator) !void {
    var download_frame = async fetchUrl(allocator, "https://example.com/");
    errdefer cancelawait download_frame;

    var file_frame = async readFile(allocator, "something.txt");
    errdefer cancelawait file_frame;

    const download_text = try await download_frame;
    defer allocator.free(download_text);

    const file_text = try await file_frame;
    defer allocator.free(file_text);
}

Now, calling an async function looks like any resource allocation that needs to be cleaned up when returning an error. It works like await in that it is a suspend point, however, it discards the return value, and it atomically sets a flag in the function's frame which is observable from within.

Cancellation tokens and propagating whether an async function has been cancelled I think can be out of scope of this proposal. It's possible to build higher level cancellation abstractions on top of this primitive. For example, #5263 (comment) could be improved with the availability of cancelawait. But more importantly, cancelawait makes it possible to casually use async/await on arbitrary functions in a maintainable and correct way.

I really don't like the idea of an implicit suspend point. It smells an awful lot like hidden control flow. Perhaps we should require async functions to retrieve their result location explicitly? Wait, no, then that's function colouring. Hmm.

(Also, is there a specific reason that the keyword can't just be cancel? cancelawait is a bit unwieldy.)

[andrewrk]

I really don't like the idea of an implicit suspend point.

I should clarify, there is already a suspend point at a return statement in an async function. Also the fact that it is at return makes it explicit I suppose. Anyway, the point is this doesn't add a suspend point, it moves it a little bit earlier so that the return expression will have the await result pointer before being evaluated, and so that the defers have not been executed yet.

[kprotty]

Sounds like a nice proposal, with moving execution into await instead of async for non-suspend functions being quite the change. Was left with a few questions after reading it over:

What and how would cancellation look like for normal calls to async functions? (e.g. _ = someAsyncFn()). Does it introduce an implicit try, do catch unreachable, etc.?

Also, is execution deferred until await instead of async for functions that dont suspend based on compile time analysis, or is this change a global property? If the latter, does this mean that async no longer runs until the first suspend point? That sounds like it would remove the ability to start async functions concurrently, which is why I feel like i'm misunderstanding it here.

[andrewrk]

What and how would cancellation look like for normal calls to async functions?

No cancellation possible for these. The result location and the awaiter resume handle are both available from the very beginning of the call. When it gets to return, no suspend occurs; it writes the return value to the result location, runs the (non-error) defers, and then tail-resumes the callee.

Also, is execution deferred until await instead of async for functions that dont suspend based on compile time analysis

At the end of the compilation process, every function is assigned a calling convention. Async functions have an async calling convention. So the compiler does have to "color" functions internally for code generation purposes. So it's based on compile time analysis. (That's status quo already)

[kprotty]

For the last part, as I understand it now, doing var frame = async someAsyncFn() runs someAsyncFn() up until its suspend point if any. If the result location is already available at the beginning of the async fn, does that mean that the execution of someAsyncFn() now begins at its frame's await point (since that were the result location is specified)?

The reason I find this significant is because, if that is true, then it changes the current assumptions on what the async keyword currently does. It would now just "setup the frame", instead of "setup the frame and run until first suspend".
If the "run" step is now only possible at await, what does this mean for trying to run other code while an async function is suspended? Originally, after the call to async someAsyncFn(), the async fn would then be running concurrently. Now that only await can start running the async fn, there no longer seems to be a way to express concurrency given await effectively serializes the async procedure.

[SpexGuy]

First, I want to note that result location semantics can already be (and may already be) supported for calls to async functions that do not use the async keyword. This gives us the rule: "the async keyword does not support result location semantics". Any call that does not use the async keyword can retain result location semantics, which means that two-coloring is not a problem. I think this rule is fine. It's simple, easy to explain and understand, and easy to see in code. I also think that passing the result location into @asyncCall is a decent solution for cases where the async keyword and result locations are both required. If you're returning a large value from an async function, something is going to be slow. Our choice is whether to make that slowness obvious (copying the value a couple times) or hidden (performing indirect jumps to do computation at the await site which involves writing large amounts of memory).

That said, I see two fatal inconsistencies between blocking and async functions with this proposal. I think they are much more subtle and hard to catch than problems with result location semantics, so IMO it would be better for the language not to support result location semantics for async calls than to take on these new problems. These two examples are related but subtly different. Fixing one will not fix the other.

cancelawait with side effects

If a function has side effects, this definition of cancelawait behaves very differently for async functions vs blocking functions. With async functions, the side effects will have happened when the cancelawait completes. But with this definition for blocking functions, it will not have triggered. This is especially problematic if the side effect is to free memory, as in this example:

// x is consumed by b.
fn b(x: *Thing) void {
    defer Thing.free(x);
    // do other stuff
}

fn a() void {
    var x: *Thing = Thing.alloc();
    
    // ownership of x is passed to b, it will clean up
    var frame = async b(x);

    // if b is async, this will clean up x.
    // if b is blocking, this will not clean up x.
    errdefer cancelawait frame;
    
    // ...
    
    await frame;
}

Return statements with side effects

This proposal can cause undesirable behavior when nested. Consider this async function:

pub fn fetchUrl(allocator: *Allocator, url: []const u8) callconv(.Async) !FetchResult {
    const urlInfo = nosuspend parseUrl(url);
    return try fetchUrlInternal(allocator, urlInfo);
}

Assume for a moment that fetchUrlInternal is blocking. According to the semantics above, it cannot run until the function is awaited, because if the function is cancelawaited its side effects will not happen. For consistency, this rule should also hold for async functions.

But that means that when fetchUrlInternal is async, the meat of this function cannot begin executing until the await happens. This means that if a user spawns 5 frames and then awaits each of them, each will not begin fetching its url until the previous one has completely finished. Essentially the async code has been "linearized", forced to run in order by this constraint.

The alternative is to allow async function calls in the return expression to begin executing asynchronously, and have the await or cancelawait in the parent be passed on to the child. But this causes a significant semantic difference between blocking and async functions, because side effects in async functions will execute but side effects in blocking functions will not.

The proposal addresses this a bit:

A function can introduce an intentional copy of the result data, if it wishes to run the logic in the return expression before an await result pointer is available.

But this is an extremely subtle difference in code for something so dramatically different in execution. I don't think this is a good idea.

---

It's not explicitly stated in the proposal, but cancelawait must be allowed on completed async functions. Otherwise the example given is buggy:

fn asyncAwaitTypicalUsage(allocator: *Allocator) !void {
    var download_frame = async fetchUrl(allocator, "https://example.com/");
    errdefer cancelawait download_frame;

    var file_frame = async readFile(allocator, "something.txt");
    errdefer cancelawait file_frame;

    // if this returns error, download_frame is awaited twice
    const download_text = try await download_frame;
    defer allocator.free(download_text);
    
    // if this returns error, download_frame and file_frame are both awaited twice
    const file_text = try await file_frame;
    defer allocator.free(file_text);
}

Fixing this is actually the only useful thing cancelawait does in this example. The calling code already needs to know how to clean up the return value, so that knowledge is not abstracted. And the returned values are slices, which are trivially fast to copy. In fact, this form incurs a significant new performance problem, because the processor now needs to make an indirect jump into the return stubs of fetchUrl and readFile which contain the code to copy the slice into the result location, instead of just copying 16 bytes out of the frame. In theory a sufficiently smart compiler could recognize that the stub is known in this case and inline it, but this is more work that has to happen at every async function in the program, and could have a negative impact on build times and debug performance.

I think this use is important, but it can be accomplished more directly. Here's my counterproposal:

Keep cancelawait, but don't have it run defers or errdefers. For a function that returns T, cancelawait returns ?T. If the function has been awaited or cancelawaited, cancelawait returns null. Otherwise it returns the return value.

This would allow the above example to be written as follows:

fn asyncAwaitTypicalUsage(allocator: *Allocator) !void {
    var download_frame = async fetchUrl(allocator, "https://example.com/");
    errdefer if (cancelawait download_frame) |text| allocator.free(text);

    var file_frame = async readFile(allocator, "something.txt");
    errdefer if (cancelawait file_frame) |text| allocator.free(text);

    const download_text = try await download_frame;
    defer allocator.free(download_text);
    
    const file_text = try await file_frame;
    defer allocator.free(file_text);
}

This is still much less efficient than avoiding defer/errdefer/try and putting the cleanup code at each return statement, because there are now atomic checks that must be made to implement cancelawait. And the optimizer will never be able to get to that level of efficiency, because it can't prove that download_frame will not trigger something that will cause file_frame to be awaited elsewhere and then return an error. But at least the code is a bit cleaner than including bools alongside each frame to prevent double-awaits.

[andrewrk]

For the last part, as I understand it now, doing var frame = async someAsyncFn() runs someAsyncFn() up until its suspend point if any. If the result location is already available at the beginning of the async fn, does that mean that the execution of someAsyncFn() now begins at its frame's await point (since that were the result location is specified)?

In the exmaple var frame = async someAsyncFn() the result location for return is not available yet, not until await happens. However, it would still setup the frame and run until first suspend, just like status quo. Here's an example that highlights the difference between status quo and this proposal:

This Proposal

fn main() void {
    seq('a');
    var frame1 = async foo();
    seq('c');
    var frame2 = async bar();
    seq('e');
    const x = await frame1;
    seq('k');
    const y = await frame2;
    seq('m');
}

fn foo() i32 {
    defer seq('j');
    seq('b');
    operationThatSuspends();
    seq('f');
    return util();
}

fn util() i32 {
    seq('g');
    operationThatSuspends();
    seq('i');
    return 1234;
}

fn bar() i32 {
    defer seq('l');
    seq('d');
    operationThatSuspends();
    seq('h');
    return 1234;
}

[kprotty]

it would still setup the frame and run until first suspend, just like status quo

Ah ok, think that was where my misunderstanding was. My last point of confusion was related to how non-suspending async fns are handled:

If we move the function call of non-suspending functions used with async/await to happen at the await site instead of the async site

Is this change in semantics something applied by compile time analysis or through some other observation? If its compile time defined, what happens to the result values of async f() started functions before they're await'ed which conditionally suspend at runtime? Running until suspend at async would discard the result value as theres not yet a provided result location. Running at await would serialize the async function as explained eariler.

[frmdstryr]

Instead of a new keyword, why couldn't a frame just have a cancel function?

errdefer download_task.cancel();

[kprotty]

@frmdstryr Nice idea. Would it make sense to extend this to other frame functionality? suspend probably wouldn't be feasible to be a frame method since it needs to support block execution.

download_task.resume();

download_task.await();

EDIT: removed async since its a calling convention and is invoked on the function rather than the frame

These aren't methods though -- they're built-in functionality. Writing them as methods is misleading, and breaks the principle of all control flow as keywords.

[kprotty]

@EleanorNB All control flow isn't currently keywords as function calls themselves are a form of control flow and can have control flow inside them as well. If I understand correctly, resume currently updates some atomic state and tail-calls into the frame's func, while await updates some atomic state and possibly suspends. Given both don't require source level control like async/suspend do, them being methods instead of keywords seems to be pretty fitting. One example of this is Rust where await is a field-property keyword of Futures/Frames and the resume equivalent is a poll() method on the Future/Frame as well.