Extending the Heroku Timeout in Node.js

Heroku will terminate a request connection if no data is sent back to the client within 30 seconds. From Heroku’s Router – HTTP timeouts documentation:

HTTP requests have an initial 30-second window in which the web process must return response data (either the completed response or some amount of response data to indicate that the process is active). Processes that do not send response data within the initial 30-second window will see an H12 error in their logs.

What’s interesting is the following paragraph, which states what happens to that timeout after some data has been sent back to the client:

After the initial response, each byte sent from the server restarts a rolling 55-second window. A similar 55-second window is restarted for every byte sent from the client.

In nearly all cases, if an operation is potentially going to take more than 30 seconds, it should be spun off into a background job. The client can then poll, use websockets, etc. to find out when the work has been finished.

But there are times and situations when setting up that kind of background processing isn’t an option. I wondered if it might be possible to work within Heroku’s limits to allow some API requests to run for longer than 30 seconds. Here are some of the things I discovered.

Node.js

What’s wrong with allowing requests that take longer than 30 seconds? One of the first things that comes to mind is tying up a finite resource (request handlers) for an extended period of time. You wouldn’t want one slow request to prevent other requests from being handled.

But with Node.js, you’re not limited by a number of processes or threads. As long as the requests aren’t blocking the event loop, it can handle many, many concurrent requests.

Whitespace in JSON

As stated above, Heroku will extend the 30-second timeout after sending a byte to the client, and then continue a rolling 55-second timeout window after each additional byte. If the response data isn’t available until after the 30-second timeout expires, what can be sent back to the client to keep the connection open?

Spaces.

According to the JSON RFC 4627:

Insignificant whitespace is allowed before or after any of the six structural characters.

The structural characters are defined as [, {, ], }, :, and ,. And whitespace is defined as a space, horizontal tab, line feed/new line, or carriage return.

This means that it’s perfectly valid to have a bunch of spaces before the first structural character in a JSON file.

Express Middleware

To see if this was going to work, I hacked together an Express middleware function that waits 15 seconds. If no data has been sent back to the client at that point, it writes a space. It keeps waiting and writing spaces every 15 seconds until something else is sent back from the real request handler.


const extendTimeoutMiddleware = (req, res, next) => {
  const space = ' ';
  let isFinished = false;
  let isDataSent = false;

  // Only extend the timeout for API requests
  if (!req.url.includes('/api')) {
    next();
    return;
  }

  res.once('finish', () => {
    isFinished = true;
  });

  res.once('end', () => {
    isFinished = true;
  });

  res.once('close', () => {
    isFinished = true;
  });

  res.on('data', (data) => {
    // Look for something other than our blank space to indicate that real
    // data is now being sent back to the client.
    if (data !== space) {
      isDataSent = true;
    }
  });

  const waitAndSend = () => {
    setTimeout(() => {
      // If the response hasn't finished and hasn't sent any data back....
      if (!isFinished && !isDataSent) {
        // Need to write the status code/headers if they haven't been sent yet.
        if (!res.headersSent) {
          res.writeHead(202);
        }

        res.write(space);

        // Wait another 15 seconds
        waitAndSend();
      }
    }, 15000);
  };

  waitAndSend();
  next();
};

app.use(extendTimeoutMiddleware);

A Proper Library

After doing some of my own experimentation, I came across a well-documented npm package that seems to do everything I’ve been exploring, and quite a bit more: http-delayed-response.

I haven’t tried it out yet, but if you’re interested in actually doing something like this for real, I’d start by looking into this library.

Conclusion

You’re going to be better off using proper background job infrastructure for handling long running requests. But in a pinch, if you’re writing a Node server, it’s definitely possible to go beyond the 30-second Heroku timeout.