Node.js and Asynchronous Programming with Promises

Recently Drew Colthorp and I chose “Node.js”:http://www.nodejs.org as our platform to build a web service. Node utilizes the “V8 JavaScript Runtime”:http://code.google.com/p/v8/ and is considered to be a fast and easily scalable solution for network-based applications.

In node I/O bound operations — such as networking, databases, or file systems — are non-blocking or asynchronous (for good reason). This kind of programming naturally lends itself to callback driven APIs which in turn can lead to unwieldy nested callback structures. Which has affectionally been referred to as the “Pyramid of Doom” in this “article”:http://calculist.org/blog/2011/12/14/why-coroutines-wont-work-on-the-web/.

Promise-based APIs provide an alternative to callback-based APIs. More importantly, they solve the problem that callback APIs introduce: the readability and intent of asynchronous code.

h2. What are promises?

The idea behind Promise-based APIs is that a function will return a _promise_ for an object in the future. Promises can be chained together and as each promise is fulfilled the next promise in line is executed. If a promise can’t be fulfilled it raises an error which can be handled, otherwise the chain of promises stops execution.

So instead of:

I.willBuyMilk (err, milk) ->
  if err
    console.log "I was unable to buy milk"
  else
    I.willMakeAMilkShake (err, milkshake) ->
      if err
        console.log "I was unable to make a milk shake"
      else
        console.log "I drink the #{milkshake} up"

We’d do something like this:

I.promiseToBuyMilk()
  .then I.promiseToMakeAMilkShake
  .then (milkshake) -> console.log "I drink the #{milkshake} up"
  .fail (err) -> console.log "In the end I was unable to make a milk shake"

h2. The library

We went with a library named “Q”:http://github.com/kriskowal/q/ which provides the kind of functionality I demonstrated above. It is a fairly flexible library that has built in support for node style callbacks (function with error and result). However, we noticed the the third party node libraries may have a different set of callback conventions which required us to wrap them.

h3. Node-style example

Q provides a function named ncall which takes a function, an object to execute that function on, and arguments. Q assumes that the first argument of the callback is an _error_ and the second argument is the _result_. If the error is non-null it assumes an error occurred and raises a failure. Otherwise it fulfills the promise with the given result.

Q.ncall(fs.readFile, fs, "./data/seeds.json", "utf-8") 
# or Q.node(fs.readFile, fs)("./data/seeds.json", "utf-8')
  .then (data) ->
    json = JSON.parse(data)
    # Do some interesting stuff
  .then -> console.log "Done!"
  .fail -> console.log "Argh!"

h3. Wrapping non-compliant APIs

In the contrived example below we wrap a library that doesn’t use node-style callbacks. We use Q.defer() to return a promise for a result in the future and wrap the doThis function.

someLibrary =
  doThis: (func) ->
    # Async I/O stuff
    func(result) # Where result is null if it didn't work

someLibrary.qDoThis = ->
  defer = Q.defer()
  @doThis (result) ->
    if result
      defer.resolve result
    else
      defer.reject new Error("Didn't get result")
  defer.promise

someLibrary.qDoThis()
  .then (result) -> console.log result
  .fail (err) -> console.log err

We also wrote this helper to automatically “qify” objects where it wraps methods on an existing object into “Q” style methods.

module.exports =
  qify: (objects...) ->
    for o in objects
      _.extend o,
        q: (name, callback) ->
          callback ?= (arg) -> arg
          (args...) =>
            d = Q.defer()
            @[name].call @, args..., ->
              d.resolve callback.apply(@, arguments)
            d.promise

        node: (name) ->
          Q.node @[name], @

Example:

qify Library

Library.node("method1")(arg1, arg2) 
  .then -> console.log "Yay"

Library.q "method2", (results) -> #map results
  .then (mapped) -> console.log mapped

h2. Alternative to promises

“node-fibers”:https://github.com/laverdet/node-fibers is a library that provides a ‘blocking style’ way of executing asynchronous code. It doesn’t actually block (that would be bad), instead it uses continuations to defer the execution of code until an asynchronous operation has completed.