Headless Chrome with Testem on VSTS-Hosted Agents

I previously wrote about building a Node app on VSTS Windows agents. Since then, we’ve started using headless Chrome on those agents. Here’s how.


Browser test automation has come a long way over the years. We used to accept having a browser flashing around in an interactive login session. Then we made browsers headless-ish with Xvfb. Then we used truly headless browsers like PhantomJS, even though they tended to be weaker than their desktop counterparts.

Last year, Firefox and Chrome added headless support, giving us the best of both worlds: the performance, stability, and feature set of mass-market browsers with the logistical convenience of command line applications.

The JavaScript ecosystem has embraced them. Ember.js, the front-end framework I’m currently working in, switched to headless Chrome by default last year, and it recently dropped support for Phantom.


At a high level, I need to 1) download Chrome, and 2) make it available to my test framework.

Get Chrome

We’re using VSTS-hosted agents, which don’t yet have Chrome preinstalled. Where should we get it?

There’s a VSTS task and a few npm packages, but nothing seemed widely used or well-maintained. I suppose the JavaScript community doesn’t have a great need for such a package, as competing CI systems offer Chrome out-ofthebox.

Luckily, I stumbled across a blog post by Benjamin Spencer, in which he pulls it out of Puppeteer, a Chrome automation project maintained by Google. We’re not using Puppeteer, but as a source, it seems more likely to stick around (and get updated) than the other, lesser-used projects.

Provide it to Testem

Testem’s approach of looking for specific executables works well for most situations, but it won’t magically find Puppeteer’s chrome.exe deep inside node_modules. I experimented with adding it on the executable path, but then I decided to try Testem’s custom launchers instead.


Here’s how to download and use headless Chrome for an Ember-CLI project on VSTS Windows agents:

  1. Add the Puppeteer package with e.g. yarn add --dev puppeteer. This puts the browser at a path like node_modules/puppeteer/.local-chromium/win64-536395/chrome-win32/chrome.exe.
  2. Wrap Chrome for easy access. We can avoid using this long (and dynamic) path with a short Node script that asks Puppeteer where to find Chrome, then runs it:
    const { executablePath } = require("puppeteer");
    const { execFileSync } = require("child_process");
    let exePath = executablePath();
    let args = process.argv.slice(2);
    execFileSync(exePath, args);

    Now we can run Chrome with e.g. node run-chrome.js google.com.

  3. Add a custom launcher to Testem’s configuration. Here’s my Testem.js:
    /* eslint-env node */
    module.exports = {
      "test_page": "tests/index.html?hidepassed",
      "report_file": "tmp/results.xml",
      "reporter": "xunit",
      "xunit_intermediate_output": true,
      "disable_watching": true,
      "launch_in_ci": [
      "launchers": {
        "LocalChrome": {
          "exe": "node",
          "args": [
          "protocol": "browser"


With Chrome in CI, our builds are faster and more reliable, and we’re able to raise our target language level, which had been held back for PhantomJS. VSTS may soon offer Chrome out-of-the-box, but with a little effort, we can enjoy it today.

  • Jared says:

    Thanks John, worked like a charm. Just as a side note, I found that trying to run ember test as a task in npm would fail in VSTS. I haven’t looked to much into why this is, but I had much more luck using yarn.

    • John Ruble John Ruble says:

      I’m glad it was useful! We’re using yarn, too.

  • Steve Elberger says:

    In case anyone else had trouble with the run-chrome.js file blowing up because executablePath is an instance method and requires an instance as context:

    const puppeteer = require(‘puppeteer’);
    const { execFileSync } = require(‘child_process’);

    const exePath = puppeteer.executablePath();
    const args = process.argv.slice(2);
    execFileSync(exePath, args);

    • John Ruble John Ruble says:

      Good call, Steve. The original version of run-chrome.js in the post broke with a puppeteer update some time ago, and your version is how we fixed ours, too.

  • Comments are closed.