Headless Chrome with Testem on VSTS-Hosted Agents

I previously wrote about building a Node app on VSTS Windows agents. Since then, we’ve started using headless Chrome on those agents. Here’s how.

##Background
Browser test automation has come a long way over the years. We used to accept having a browser flashing around in an interactive login session. Then we made browsers headless-_ish_ with [Xvfb]. Then we used truly headless browsers like [PhantomJS], even though they tended to be weaker than their desktop counterparts.

Last year, Firefox and Chrome added headless support, giving us the best of both worlds: the performance, stability, and feature set of mass-market browsers with the logistical convenience of command line applications.

The JavaScript ecosystem has embraced them. Ember.js, the front-end framework I’m currently working in, switched to headless Chrome by default [last year][ember_blog_215], and it recently [dropped support][ember_blog_3] for Phantom.

##Approaches
At a high level, I need to 1) download Chrome, and 2) make it available to my test framework.

###Get Chrome
We’re using VSTS-hosted agents, which [don’t][vsts_agent_software] yet have Chrome preinstalled. Where should we get it?

There’s a [VSTS task][chromium_installer_task] and a few [npm packages][npm_packages], but nothing seemed widely used or well-maintained. I suppose the JavaScript community doesn’t have a great need for such a package, as competing CI systems offer Chrome out-[of] [circle_chrome] -[the][travis_chrome] -[box][appveyor_chrome].

Luckily, I stumbled across a [blog post][benjaminspencer_post] by Benjamin Spencer, in which he pulls it out of [Puppeteer], a Chrome automation project maintained by Google. We’re not using Puppeteer, but as a source, it seems more likely to stick around (and get updated) than the other, lesser-used projects.

###Provide it to Testem
Testem’s approach of looking for [specific executables][testem_known_browsers] works well for most situations, but it won’t magically find Puppeteer’s chrome.exe deep inside node_modules. I experimented with adding it on the executable path, but then I decided to try Testem’s [custom launchers][testem_launchers] instead.

##Implementation
Here’s how to download and use headless Chrome for an Ember-CLI project on VSTS Windows agents:

1. Add the Puppeteer package with e.g. yarn add --dev puppeteer. This puts the browser at a path like node_modules/puppeteer/.local-chromium/win64-536395/chrome-win32/chrome.exe.
2. Wrap Chrome for easy access. We can avoid using this long (and dynamic) path with a short Node script that asks Puppeteer where to find Chrome, then runs it:

   
   const { executablePath } = require("puppeteer");
   const { execFileSync } = require("child_process");
   
   let exePath = executablePath();
   let args = process.argv.slice(2);
   execFileSync(exePath, args);
   

   Now we can run Chrome with e.g. node run-chrome.js google.com.

3. Add a custom launcher to Testem’s configuration. Here’s my Testem.js:

   
   /* eslint-env node */
   module.exports = {
     "test_page": "tests/index.html?hidepassed",
     "report_file": "tmp/results.xml",
     "reporter": "xunit",
     "xunit_intermediate_output": true,
     "disable_watching": true,
     "launch_in_ci": [
       "LocalChrome"
     ],
     "launchers": {
       "LocalChrome": {
         "exe": "node",
         "args": [
           'run-chrome.js',
           '--disable-gpu',
           '--headless',
           '--remote-debugging-port=9222',
           '--window-size=1440,900'
         ],
         "protocol": "browser"
       },
     }
   };
   

##Conclusion
With Chrome in CI, our builds are faster and more reliable, and we’re able to raise our [target language level][ember_build_targets], which had been held back for PhantomJS. VSTS may [soon][github_issue] offer Chrome out-of-the-box, but with a little effort, we can enjoy it today.

[puppeteer]: https://github.com/GoogleChrome/puppeteer
[vsts_agent_software]: https://docs.microsoft.com/en-us/vsts/build-release/concepts/agents/hosted#software
[github_issue]: https://github.com/Microsoft/vsts-image-generation/issues/47
[ember_build_targets]: https://guides.emberjs.com/v3.0.0/configuring-ember/build-targets/
[benjaminspencer_post]: https://benjaminspencer.me/post/14/headless-chrome-vsts
[headless_chrome]: https://developers.google.com/web/updates/2017/04/headless-chrome

[travis_chrome]: https://docs.travis-ci.com/user/chrome
[circle_chrome]: https://circleci.com/docs/2.0/circleci-images/#language-image-variants
[appveyor_chrome]: https://www.appveyor.com/docs/build-environment/#web-browsers

[ember_blog_215]: https://www.emberjs.com/blog/2017/09/01/ember-2-15-released.html#toc_chrome-by-default
[ember_blog_3]: https://www.emberjs.com/blog/2018/02/14/ember-3-0-released.html#toc_browser-support-in-3-0

[chromium_installer_task]: https://marketplace.visualstudio.com/items?itemName=schlumberger.chromium-build-tasks-Preview

[puppeteer_chromium_updates]: https://github.com/GoogleChrome/puppeteer/commits/master/package.json

[testem_known_browsers]: https://github.com/testem/testem/blob/master/lib/utils/known-browsers.js

[testem_launchers]: https://github.com/testem/testem#launchers

[Xvfb]: https://en.wikipedia.org/wiki/Xvfb
[travis_xvfb]: https://docs.travis-ci.com/user/gui-and-headless-browsers/#Using-xvfb-to-Run-Tests-That-Require-a-GUI
[phantomjs]: http://phantomjs.org/
[npm_packages]: https://www.npmjs.com/search?q=chrome&page=1&ranking=maintenance