Article summary
Lua is a deceptively simple language. Its designers have done such a good job of keeping it downwardly scalable to simple uses that it’s easy to overlook the advanced parts. One of these is its polished implementation of coroutines.
Why Coroutines Matter
The combination of coroutines, tail-call optimization, and closures means that many sophisticated control structures can be implemented in Lua pretty easily. Rather than turning functions inside-out and nesting them inside an arbitrary primary function, untangling them and giving each its own main loop often simplifies things. Allowing individual closures to suspend and resume means that backtracking, lazy streams, constraint propagation networks, and so on can be expressed cleanly. Also, asynchronous IO can be coordinated with functions like luasocket.select scheduling coroutines; each can still be read linearly, and there’s no need to CPS-transform everything by hand with callbacks.
Lua’s coroutines are similar to Python’s generators and Ruby’s fibers, and lessons learned with them will mostly transfer. (Lua has had them since at least 2003, though. Python is unlikely to ever get proper tail calls, and it sounds like they are still a work in progress for mainstream Ruby implementations.)
For people familiar with Scheme, single-use continuations can be implemented in terms of coroutines, and vice versa, though my experience has consistently been that coroutines are a better fit for most problems. (Even Oleg agrees that full continuations are probably the wrong abstraction!)
The main limitation of Lua coroutines is that, since they are implemented with setjmp(3) and longjmp(3), you cannot use them to call from Lua into C code that calls back into Lua that calls back into C, because the nested longjmp will clobber the C function’s stack frames. (This is detected at runtime, rather than failing silently.) I haven’t found this to be a problem in practice, and I’m not aware of any way to fix it without damaging Lua’s portability, one of my favorite things about Lua — it will run on literally anything with an ANSI C compiler and a modest amount of space. Using Lua means I can travel light. :)
(Also, coroutines allow concurrency, but not parallelism — they won’t take advantage of multiple cores. Multiple Lua VMs can be run in one process without any GIL problems, though, so parallelism via message-passing is still an option.)
How Do you Use Lua Coroutines?
create
coroutine.create(f) returns a new coroutine. f must be a function (possibly anonymous) that takes 0 or more arguments. As per the usual Lua function calling style, arguments not provided will be nil, and extra arguments will be discarded.
resume
coroutine.resume(c, [args]) returns (true, [results from yield or return]) or (false, error_message)
This can be used to pass new values into a coroutine as it is resumed, and the caller will receive any results the callee passes to yield or resume, or any error messages from the coroutine’s execution.
running
coroutine.running() returns a reference to the current coroutine and a boolean for whether execution is currently in the main coroutine. (This is typically useful when the coroutine itself is used as a key in a table.)
status
coroutine.status(c) returns any of the strings “running,” “suspended,” “normal” (live but not running), or “dead” if complete or error’d out.
yield
coroutine.yield(...) suspends the current coroutine, returning its arguments to the caller. If subsequently resumed, yield will return any arguments that were passed to resume, and pick up where it left off.
wrap
coroutine.wrap(f) wraps a function so that it can use coroutine.yield like a coroutine, but called with the normal function call syntax. This is typically used to conceal whether an API is actually using a function or a coroutine internally.
Closing
While the coroutine API is pretty simple, it’s a major boost to the expressiveness of the language. Coroutines make function calls more expressive, and allow many important constructs to be represented cleanly.