Article summary
Developing asynchronous networking applications is an interesting problem with unique challenges and no shortage of solutions for, particularly in Python. But the way you’ve traditionally had to write your code to make it work well could leave you with a codebase that, while solid, could be challenging to read or follow.
I’ll take you through a quick introduction to what asynchronous networking is, show you a callback-based asynchronous implementation using the venerable and excellent Twisted, and then finally show you how Python’s new asyncio library uses coroutines to solve the problem in a whole new way.
Unblocking the Socket
Most networked software uses a blocking model, allowing the programmer to construct the network logic in a clear, imperative beginning-to-end fashion. For this post, we’ll be building a client for a greeting service that we send a name, then get back a personalized greeting. Our client code looks like this:
def get_greeting(name):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(('', 9999))
sock.send(name + b'\r\n')
greeting = b''
while not greeting.endswith(b'\r\n'):
greeting += sock.recv(BUFSIZE)
return greeting
print(get_greeting(b'World'))
It should be fairly clear to most programmers what this code is doing, even if you don’t know Python. We set up a socket, immediately send the name followed by a CR/LF pair, then repeatedly read from the same socket until we have received an entire line. The result is then returned. Very straightforward, very easy to explain.
The problem with doing network programming like this is that, while the networking calls like sock.recv are executing, the program is blocked; nothing else can happen until that code either returns or throws an exception, even though the program itself is otherwise entirely idle.
This is fine for a simple program — you could, for example, run as many copies of it as you need to handle many connections, which is exactly how a number of simple Unix network services are built—but modern network programs often have to deal with coordinating lots of things, not just one connection. Traditionally, this problem has been addressed by threads spawned when new connections are made, but introducing threads means introducing the baggage of threads as well, including both the machine and human cost of dealing with synchronizing access to shared program resources.
Twisting it Inside-Out
Asynchronous networking solves this problem by doing networking in non-blocking mode. In this mode, network calls return immediately whether they have data or not. Programs then collect references to all their open sockets and use system calls like select to block on all of them simultaneously, returning lists of sockets that are ready to be read from or written to, then processing each socket in turn and doing it all over again in an event loop. This flow substantially frees the programmer from having to reason about concurrent access to shared objects like they would with a threaded solution, because only one routine can ever be accessing those objects. Not having to synchronize access can also make the program perform much better.
Libraries like Twisted wrap all this up to make it simpler and more consistent to use. This makes it easy to run not just lots of one kind of server, but many networking connections and protocol implementations all at once. Our greeting-getting implementation looks like this now:
class GreetingGetter(LineOnlyReceiver):
def __init__(self, factory):
self.factory = factory
self.pendingGreeting = None
def connectionMade(self):
self.factory.getterConnected(self)
def getGreeting(self, name):
self.sendLine(name)
self.pendingGreeting = Deferred()
return self.pendingGreeting
def lineReceived(self, line):
if self.pendingGreeting:
self.pendingGreeting.callback(line)
reactor.stop()
class GreetingGetterFactory(ClientFactory):
def buildProtocol(self, addr):
return GreetingGetter(self)
def getterConnected(self, getter):
d = getter.getGreeting(b'World')
d.addCallback(print)
Some explanation is almost certainly needed here. Twisted supports asynchronous behavior through objects called Deferreds, which are promises to eventually return something when we have it, even though that something may not be currently available.Deferred objects are returned in lieu of that something, and the caller then attaches callbacks that will be run when the result is available.
In this case (in GreetingGetter.getGreeting) we do not actually have the greeting back from the server yet, so we track and return a Deferred that we will call back when we do. The factory and protocol (our GreetingGetter) are hooked up to Twisted’s reactor, an event loop that calls methods when network events happen. The reactor will let our protocol know when data has been received; if our protocol has a Deferred reference for returning the greeting, we’ll fire it via the callback method, which in our case calls the built-in Python function print, as we set it up in the factory’s getterConnected method.
There’s a lot going on there, but the key point is that our methods are all constructed to never block execution, which lets Twisted very efficiently juggle lots of network activity. But, of course, there’s also a lot going on there, because in making our code purely driven by method calls via the Twisted reactor, we’ve turned the entire thing effectively inside-out. Could it be made simpler? Absolutely, though it would take some new features of Python that weren’t around when Twisted was first conceived.
Enter the Coroutine
Twisted first made some moves to simplify this workflow with a feature that allows for collapsing callbacks into a single method using generators and Python 2.5’s yield expressions (inlineCallbacks, which I used in a demo for a library I wrote), though a lot of callback-oriented coding is still required. The really impressive gains came only recently with coroutines.
A coroutine is, in a nutshell, a function whose execution can be suspended while it waits for a return value. Coroutines are used in Python 3’s new asyncio library to handle suspending the execution of our code while we wait for network events.
As a result, we can seriously flatten our asynchronous greeting client out, right down to a single coroutine function:
@asyncio.coroutine
def get_greeting(name):
reader, writer = yield from asyncio.open_connection('', 9999)
writer.write(name + b'\r\n')
greeting = yield from reader.readline()
return greeting
loop = asyncio.get_event_loop()
coro = get_greeting(b'World')
greeting = loop.run_until_complete(coro)
print(greeting)
Apart from the new API (and the fact that asyncio gives us a reader object we can use to read a line instead of putting together a line-reading loop ourselves), this is conceptually very similar to our original blocking example. The big new thing introduced here is the yield from expressions — these suspend our current coroutine and allow other coroutines to supply data, which, in turn, are themselves suspended while the networking magic happens and other separate tasks are executed. They’re also used pervasively in asyncio’s API, allowing us to build our entire example without writing a single callback.
It’s important to note that the code in the coroutine functions is not actually run when they are called; instead, a coroutine object is returned that is executed by the event loop when the loop finally gets a hold of it (e.g. loop.run_until_complete). yield from statements inside a coroutine transfer control to the specified coroutine, returning control when the delegate has completed. This means that we’re not actually connecting and running the protocol when we assign the return from the coroutine function get_greeting to coro; we’re actually just preparing to do so. The action only happens when the event loop says so.
asyncio actually has a lot in common with Twisted and can be used in a very similar fashion — even mixing and matching coroutine-based code with asyncio’s own protocols. I’m looking forward to seeing how well it works to build larger networking implementations using asyncio coroutines by default. I can say from experience that it’s a lot of fun learning the ins and outs of an asynchronous system and how all the pieces fit together, but it’s even more fun to find ways to do that same job even better.