Commands and Reactors

[sunfishcode]

There seem to be two distinct modes of program execution that applications broadly fit into: Commands and Reactors.

• A Command has a "main" function, and when this function returns the program terminates.
• A Reactor has a "setup" function, and when this function returns the program remains live, allowing functions registered as callbacks to be called. There is an event loop in the wasm runtime, rather than in the application.

Reactors could run in the main thread of a browser, but they may also have uses in a variety of settings where applications will primarily be responding to external events. Putting the event loop in the runtime gives the runtime the flexiblity to use a single event loop for multiple purposes.

(I briefly mentioned these ideas here, but I want to give them more visibility here.)

Emscripten provides a kind of hybrid approach, where instances can stick around after "main" exits to allow them to be called by callbacks. For WASI, it may be useful to make an explicit static distinction between Commands and Reactors, because:

• We could limit the APIs available to Reactors. For example, if we want to run Reactors in the main thread of a browser, we could prohibit them from using synchronous I/O APIs.
• We could limit the APIs available to Commands too, for example not allowing them to register for external event-loop APIs since there is no external event loop in a Command.
• Allowing runtimes to know that a program is a Reactor up front can enable some useful optimizations, like switching JIT tiers between turns of the event loop, in a very simple way.

Currently, WASI programs combined with clang fit the Command model, with _start being the "main" function. It might make sense to rename it to __wasi_command_main or something. Programs could declare themselves to be Reactors by exporting a function named something like __wasi_reactor_setup or so.

Some interesting questions:

• Is a static distinction between Command and Reactor too limiting?
• Are there programs which don't fit into either Command or Reactor models?
• Are the restrictions imposed by browser main thread execution too limiting for non-Web use cases?

[npmccallum]

Why isn't Reactor just a WASM library used on top of a Command?

[dschuff]

You could create such a library, and it could be useful. But I think there's still value in having a reactor-type model explicitly modeled by WASI; in particular because as mentioned the main thread on the web inherently has this model, and its event loop is baked into the platform. That means that if WASI mandates a command-style model, it will be come much more awkward and less useful on the web.

[npmccallum]

As someone who has written such a main loop abstraction, it is very hard to coerce all the platforms an implementations into the same behavior. I think there is negative value in trying to implement a main loop into WASI.

[PoignardAzur]

A relevant thought experiment would be the following use case:

How do you handle a grep program?

One the one hand, it definitely has the semantics of a command: it has a main function, reads input files sequentially and prints lines sequentially into its output file descriptor. It's very deterministic and doesn't "react" to anything happening around it.

On the other hand, a wasm grep implementation could definitely benefit from async/await and an event loop. The programs tend to be I/O bound, and async/await allows them to process text while the next block is being loaded in memory, without relying on multithreading or additional logic.

Using the guidelines @sunfishcode suggested above:

• You don't want a grep-like program to run in the main thread of a browser.

• You want grep to have access to some asynchronous features. You might still limit its access to the event-loop API, because grep is sequential and usually doesn't need to, say, race between different files and output a line as soon as one of them finds a match.

• There are probably optimizations to be gained from having semantic information about grep, but I'm not sure which ones. I guess that knowing that it doesn't spawn callbacks make it easier to dispose of?

I'm not really sure what the right semantics are, but I think that grep should still be classified as a command, provided commands have a way to keep executing past blocking syscalls. They could still be more limited than reactors (eg be only allowed one call stack), but I'm not sure what benefits these limitations would actually bring.

[PoignardAzur]

Also, another question: what effect do you expect the Commands/Reactors distinction to have on the type system and library interoperability?

In the rationale, you propose:

• In a Reactor, WASI APIs are available, but all functions have an additional argument, which specifies a function to call as a continuation once the I/O completes. This way, we can use the same conceptual APIs, but adapt them to run in an callback-based async environment.
• In a Command, WASI APIs don't have callback parameters. Whether or not they're non-blocking is an open question (see the previous question).

Which means that every syscall would have two signatures: __wasi_fd_read(fd, buffers) and __wasi_fd_read(fd, buffers, callback).

Wouldn't that split the library ecosystem in two? With every I/O library developer either picking a side, or making a mynetworklib_command and a mynetworklib_reactor version?

I think a better solution might be to have a common function signature for Reactor-mode and Command-mode read: async __wasi_fd_read(...). When compiled in Reactor mode, that function would then get a callback parameter added. In Command mode, it would block, or maybe return a promise-like object that indicates when the read is done.

This would, of course, require async to be added to the type system, with only async functions being able to call other async functions. Additionally, in Command mode, async functions would only be able to use features that don't require an event loop. So, Promise.all and Promise.race can work, but Promise.then can't.

That way, developers could write library that can be used interchangeably in Commands and Reactors. Ideally, you'd also want a __wasi_async_addevent syscall that Commands could shim to provide their own event loop for libraries that do require one.

[dumblob]

Somehow I still can't grasp what would be the reason for such (weird) differentiation.

• We could limit the APIs available to Reactors. For example, if we want to run Reactors in the main thread of a browser, we could prohibit them from using synchronous I/O APIs.

Sounds to me like a disguised attempt to get at least some finer grained capability support than the current coarse grained one - see #1 .

• We could limit the APIs available to Commands too, for example not allowing them to register for external event-loop APIs since there is no external event loop in a Command.

Again the same - it sounds to me like a disguised attempt to get at least some finer grained capability support than the current coarse grained one - see #1 . @kentonv what do you think?

Allowing runtimes to know that a program is a Reactor up front can enable some useful optimizations, like switching JIT tiers between turns of the event loop, in a very simple way.

Compared e.g. to the approach @npmccallum proposed above, there could be some minor gain, but only for one thread out of N (nowadays from hundreds up to many thousands) and only in case the Reactor runs in the very same thread as the main loop (which is not necessarily the case and is becoming less and less probable over the time as we're having in our pockets HW capable of running tens and hundreds of threads in parallel and software trying to accommodate...).

Also I'm pretty confident, that optimization of switching JIT tiers is possible (and not much more difficult) also without knowing about the distinction between Reactors and Commands in advance in compile time.

But maybe there are other reasons for such distinction and I'm missing them...

[kentonv]

@dumblob Hmm, not sure what you mean -- I'm not seeing the connection between this and fine-grained capabilities.

My opinion, FWIW (disclaimer: I don't know a whole lot about WASI and I'm probably missing important context):

In the context of both capability systems and the Web platform, event loops and Promises have "won". async/await has provided a finishing blow by making async code almost as easy to write as sync code, while keeping all the advantages of async. It seems like WASI should focus on async as the main way it expects to be used.

That said, supporting legacy C/C++/etc. code written in synchronous style seems like a desirable goal.

I'd argue for creating only one set of I/O APIs that operates in an async way. All such calls should return a "promise descriptor". A special call, then, would be used to register a callback to call when a promise completes.

In order to support legacy synchronous code, you could then offer a special await call which takes a promise descriptor and actually blocks until it completes. This would actually run the event loop in the meantime, which might mean calling callbacks registered with then. The await call would only be permitted when control entered WASM in an "await-friendly" way. Invoking the main function of a "command"-style program would be defined as await-friendly. Note that this means that "command"-style programs would still have an event loop -- they just control when it runs by using await. I suspect this will be important as command-style programs will likely commonly want to use async-style libraries, which means they need to support event loops in the same way.

You might then consider offering a create_fiber() call which allows an await-friendly context to be created even in "reactor"-style programs. This would work just like Windows fibers or node-fibers. This would permit reactor-style programs to use libraries that were written in a synchronous style. Again, this enables the use of legacy code -- any new code should probably use async/await which makes fibers obsolete.

You might also consider supporting Promise Pipelining: Consider an I/O call which eventually produces a file descriptor as its result, such as open(). It needs to return a promise initially, but then awaiting the promise produces a file descriptor. In these cases, the promise descriptor itself should also be usable as a proxy for the eventual file descriptor, allowing you to pass it to any other async syscall that would work on the eventual file descriptor. So, you can call read() and pass in the promise descriptor returned by open(), without first waiting for that promise to complete. The semantics is: if the promise completes successfully, then the read() operates on the resulting file. If open() ends up failing, then read() fails with the same error. This trick can reduce the number of round trips needed e.g. when the file resides on a remote filesystem. Promise Pipelining is a core feature of Cap'n Proto RPC (derived from E/CapTP); you can read more here: https://capnproto.org/rpc.html

[dumblob]

@dumblob Hmm, not sure what you mean -- I'm not seeing the connection between this and fine-grained capabilities.

I quote prohibit them from using synchronous I/O APIs. and for example not allowing them to register for external event-loop APIs - this is not possible with the current fd-oriented capability system - so it sounds to me like "an extension" to the current capability system.

[sunfishcode]

The Reactor concept here is largely motivated by thinking about whether WASI could be used within Web browsers on the main thread, and then, if we design a way to make that work, would it be usable in other contexts as well?

The Web browser main thread environment imposes some constraints, such as the constraint that synchronous I/O is not available (ignoring sync xhr, proxying to Workers, etc.). This is what motivates the suggestion of limiting APIs available to Reactors. And it probably limits the options concerning blocking until an event arrives, or pipelining, or other things.

Maybe having a Reactor concept doesn't mean that all event-based I/O needs to use it, and we still need an epoll-like way for applications to build their own custom event loops. Or perhaps we should have multiple variants of the Reactor concept, to address different use cases.

[kentonv]

@sunfishcode FWIW I had the browser main thread use case in mind when writing my comment. I don't think there's any conflict with pipelining...

FWIW, as someone who has built several custom event loops (using poll, epoll, kqueue, Windows IOCP)... I think I would prefer a built-in event loop. All these interfaces are a PITA and all I really want to do is have the OS call some callback when the event happens (but only one callback at a time).

[PoignardAzur]

@kentonv As much as I like cap'n proto (and I think it has potential as a cross-language layer above wasm, similar to webidl bindings), I don't think Promise pilelining is something the processor should worry about.

Eg an implementation of capn'proto could just have its RPC methods return plain old structs, with both the promise descriptor and the RPC "token" as members. Then wasm would use the promise descriptor, while promise pilelining would use the token.

[titzer]

@kentonv

Designing APIs is hard and the sync/async debate has a long history. It's hard, there are lots of perspectives, so I don't enjoy making overly definitive statements...nevertheless for brevity, I'll state that synchronous I/O is a fantastically straightforward and successful programming model that has been employed in many, many contexts. Straight-line control flow is as simple as it gets. And blocking I/O follows very naturally from straight-line control flow. Of course, it works best when execution stacks are plentiful and cheap, and task switching is similarly cheap, like many threads in C or many goroutines. Callbacks and promises and async functions are all far more complicated than simple blocking I/O APIs. They haven't "won" so much as they are the only choice for the web platform, which historically has lacked plentiful execution stacks and cheap task switching (and is also tied an inherently single-threaded primary programming language). The web's mostly accidental evolution into a programming platform where UI reactivity, I/O, and computation are multiplexed in userland onto a single underlying execution thread, complete with jank galore, should not be emulated here, IMHO.

[kentonv]

@PoignardAzur Yeah honestly I was only half-seriously suggesting it. Promise pipelining could be a win when operating on a remote filesystem, but probably doesn't really benefit anything operating on local resources, which is probably the vast majority of what WASI does. So probably not worth the effort here, but fun to think about.

@titzer Fair enough. But are you saying that the specific design of having async API calls plus a separate await call to block on them is not a good compromise? Or are you only objecting to the philosophical position?

[titzer]

I don't think that having a separate await call that blocks on promises is good to put in the bottom execution layer, since generally engines implement this with the moral equivalent of a CPS transform, which is probably complexity that better belongs in producers and not in engines.