Node.js and Asynchronous Programming with Promises

Recently Drew Colthorp and I chose Node.js as our platform to build a web service. Node utilizes the V8 JavaScript Runtime and is considered to be a fast and easily scalable solution for network-based applications.

In node I/O bound operations—such as networking, databases, or file systems—are non-blocking or asynchronous (for good reason). This kind of programming naturally lends itself to callback driven APIs which in turn can lead to unwieldy nested callback structures. Which has affectionally been referred to as the “Pyramid of Doom” in this article.

Promise-based APIs provide an alternative to callback-based APIs. More importantly, they solve the problem that callback APIs introduce: the readability and intent of asynchronous code.

What are promises?

The idea behind Promise-based APIs is that a function will return a promise for an object in the future. Promises can be chained together and as each promise is fulfilled the next promise in line is executed. If a promise can’t be fulfilled it raises an error which can be handled, otherwise the chain of promises stops execution.

So instead of:

I.willBuyMilk (err, milk) ->
  if err
    console.log "I was unable to buy milk"
    I.willMakeAMilkShake (err, milkshake) ->
      if err
        console.log "I was unable to make a milk shake"
        console.log "I drink the #{milkshake} up"

We’d do something like this:

  .then I.promiseToMakeAMilkShake
  .then (milkshake) -> console.log "I drink the #{milkshake} up"
  .fail (err) -> console.log "In the end I was unable to make a milk shake"

The library

We went with a library named Q which provides the kind of functionality I demonstrated above. It is a fairly flexible library that has built in support for node style callbacks (function with error and result). However, we noticed the the third party node libraries may have a different set of callback conventions which required us to wrap them.

Node-style example

Q provides a function named ncall which takes a function, an object to execute that function on, and arguments. Q assumes that the first argument of the callback is an error and the second argument is the result. If the error is non-null it assumes an error occurred and raises a failure. Otherwise it fulfills the promise with the given result.

Q.ncall(fs.readFile, fs, "./data/seeds.json", "utf-8") 
# or Q.node(fs.readFile, fs)("./data/seeds.json", "utf-8')
  .then (data) ->
    json = JSON.parse(data)
    # Do some interesting stuff
  .then -> console.log "Done!"
  .fail -> console.log "Argh!"

Wrapping non-compliant APIs

In the contrived example below we wrap a library that doesn’t use node-style callbacks. We use Q.defer() to return a promise for a result in the future and wrap the doThis function.

someLibrary =
  doThis: (func) ->
    # Async I/O stuff
    func(result) # Where result is null if it didn't work

someLibrary.qDoThis = ->
  defer = Q.defer()
  @doThis (result) ->
    if result
      defer.resolve result
      defer.reject new Error("Didn't get result")

  .then (result) -> console.log result
  .fail (err) -> console.log err

We also wrote this helper to automatically “qify” objects where it wraps methods on an existing object into “Q” style methods.

module.exports =
  qify: (objects...) ->
    for o in objects
      _.extend o,
        q: (name, callback) ->
          callback ?= (arg) -> arg
          (args...) =>
            d = Q.defer()
            @[name].call @, args..., ->
              d.resolve callback.apply(@, arguments)

        node: (name) ->
          Q.node @[name], @


qify Library

Library.node("method1")(arg1, arg2) 
  .then -> console.log "Yay"

Library.q "method2", (results) -> #map results
  .then (mapped) -> console.log mapped

Alternative to promises

node-fibers is a library that provides a ‘blocking style’ way of executing asynchronous code. It doesn’t actually block (that would be bad), instead it uses continuations to defer the execution of code until an asynchronous operation has completed.