I recently found myself starting a project where, for a variety of reasons, it made a lot of sense to use Ember.js to build our interface while using Clojure to power our API.
Before I began setting up the application, I expected that I was in for a large amount of pain. After all, this feels like a somewhat unorthodox approach. While it wasn’t nearly as easy as setting up a new Rails app, for example, I actually found the process wasn’t entirely too painful.
General Setup and Organization
As both the UI (Ember) and the API (Clojure) are intended to form a cohesive product, it makes the most sense that they would be maintained in the same source control repository. I’ve worked on other projects where we had initially made the decision to factor each component out into its own repository; however, due to logistical issues, it was inevitably decided to merge the repositories into one. If you do decide on using separate repositories, you might end up experiencing a fun lesson in smashing two git histories together.
In this case, I opted to place the Ember app in a folder called frontend, while the Clojure API lives in api. Getting those is fairly straightforward: create the Ember app with Ember CLI and your API with Leiningen.
Easy enough so far.
Getting Your Friends to Talk to Each Other
Now we need the Ember.js app to be able to make requests to our Clojure API. In development, our Ember app and Clojure API will run on different ports. Because of this, I was expecting CORS (and Ember CLI’s built-in enhanced security rules) to be a pain point. Worse, I was expecting this pain to be entirely unnecessary because the Ember app will be embedded in the Clojure API when it’s deployed (more on this later).
In reality, I was very pleasantly surprised when I saw that Ember CLI has some built-in support for proxying to your API in development. This is done by passing the --proxy flag to ember server with your API’s URL. Even better, you can have this be the default by editing frontend/.ember-cli:
{
"disableAnalytics": false,
"proxy": "http://localhost:3000"
}
After discovering this, I was still concerned about how we would run system tests. Ember CLI uses Testem and does not respect the same proxy flag that `ember server` does. Fortunately, after doing some research, I realized that I could configure proxies by editing Testem’s configuration in `frontend/testem.json`.
{
"framework": "qunit",
"test_page": "tests/index.html",
"launch_in_ci": [
"PhantomJS"
],
"launch_in_dev": [
"PhantomJS",
"Chrome"
],
"proxies": {
"/api": {
"port": 3020,
"host": "localhost"
}
}
}
It’s important to note the caveat that, unlike Ember CLI, Testem will not proxy unhandled requests to any path. Testem requires that you specify a specific base path, and only paths underneath that will be proxied. This is acceptable, though, as it’s a good idea to scope your API paths anyway.
One more thing worth mentioning from this example is that I’ve configured Testem to proxy to a different port than Ember CLI. This is intentional, to allow for running the servers in development and test mode simultaneously without affecting each other. This brings us to my next topic.
Creating a More Convenient API Configuration
By convention, Clojure services receive their configuration information by looking at environment variables. Contrast this to, say, Rails, where you are provided a built in mechanism for selecting specific sets of configurations (development, production, test).
While I appreciate the merits of Clojure’s conventional approach, I was left very sorely missing the convenience of being able to run rspec without having to first configure my environment to point to a different database.
I was able to come up with a reasonable approach to reclaiming this convenience through a combination of the lein-environ plugin, leiningen profiles, and creating some simple leiningen command aliases.
The lein-environ plugin allows you to define default environment configuration by adding a map to your project.clj under the :env key. (Configuration provided from the unix environment or Java system properties can still override this). When you combine this with leiningen’s profiles support, you can make it easy to swap sets of configuration. Then you can make some command aliases to save yourself some typing.
Take, for example:
(defproject com.atomicobject/some-cool-project "0.1.0-SNAPSHOT"
:min-lein-version "2.0.0"
:dependencies [[org.clojure/clojure "1.6.0"]
[environ "1.0.0"]
[compojure "1.3.1"]
[ring/ring-defaults "0.1.2"]
[log4j "1.2.15" :exclusions [javax.mail/mail
javax.jms/jms
com.sun.jdmk/jmxtools
com.sun.jmx/jmxri]]
[korma "0.4.0"]
[drift "1.5.2"]
[org.postgresql/postgresql "9.3-1102-jdbc41"]]
:plugins [[lein-midje "3.1.3"]
[lein-ring "0.8.13"]
[lein-environ "1.0.0"]
[drift "1.5.2"]]
:ring {:handler some-cool-project.core/app}
:aliases {"server" ["ring" "server-headless"]
"test-server" ["with-profile" "+test" "ring" "server-headless"]
"test-repl" ["with-profile" "+test" "repl"]}
:env {:allow-test-api false}
:profiles {:test {:ring {:port 3020}
:env {:allow-test-api true
:database-name "some-cool-project-test"}}
:dev {:ring {:port 3000}
:env {:database-name "some-cool-project-dev"}
:dependencies [[javax.servlet/servlet-api "2.5"]
[midje "1.6.3"]
[ring-mock "0.1.5"]]}})
With all of this in place, you could simply run lein test-server to spin up the API connected to your test database.
An Approach to Deployment
For development and testing purposes, we’ve now got our Ember app being compiled and served up by Ember CLI. All access to the API is actually proxied through to the Clojure service running in a separate process.
On this particular project, our deployment target is to bundle up both the API and compiled Ember app into a single war file that could be thrown into, for example, Jetty. For this purpose we have some scripts in the root of the repository that compile the Ember app before using Leiningen (and Ring) to build our “uber war”.
To include the Ember app inside of the war, we simply have a relative-path symlink at api/resources/public that points back to ../../frontend/dist.
It’s rather unfortunate that we have a lack of consistency between how the app is run in development and production. I’d love to figure out a better way to accomplish it. Regardless, it does work.
There is just one missing piece you’ll need to get this working: you’ll want the API to serve up your compiled index.html when a client accesses the root path. Without it, everything will still work, but only if your users hit “/index.html”, or if you configure your webserver to rewrite the root to it. We accomplished this by adding a route to Compojure that renders the index using clojure.java.io:
(ns some-cool-app.core
(:require [compojure.core :refer :all]
[compojure.route :as route]
[clojure.java.io :as io]
[ring.middleware.defaults :refer [wrap-defaults api-defaults]]
[ring.middleware.file :refer [wrap-file]]))
(defroutes app-routes
(GET "/" [] (io/resource "public/index.html"))
...)
Our Approach to System Testing
Given that our experience shows that the tests that provide the most value are tests that actually exercise the whole system in concert, it makes absolutely no sense to try to segregate the testing of the UI and API. Instead, we’re running tests against the Ember app while it’s set up to be talking to our real API (running in test mode).
In the past, Atomic Object has made heavy use of tools like Capybara to perform system testing of web apps. These kinds of frameworks allow the testing of a web application by running it in a headless browser and instrumenting crude browser events. When used in combination with a single-page web app (like an Ember app), this model becomes incredibly prone to reporting spurious test failures and having tests take an unnecessarily long time to run. At the heart of the problem is that Capybara has no way to know what’s going on inside the Ember app (even less than it understands a web app based around page refreshes), and the easiest way to compensate is to add lots of sleeps and polling—essentially, opportunities for race conditions.
To get around that, we’re just writing our system tests using ember-testing. Thanks to its in-depth knowledge of what Ember is doing, we can sidestep these problems to write more reliable and fast tests. The tradeoff is that we might have difficulty writing certain kinds of tests that Capybara would not. One example that comes to mind is testing uploading of files. For this project, that is acceptable.
There’s one final piece of this testing puzzle that remains. Typically, it’s convenient to have some control over your server’s state, usually your database, when writing your system tests. In an app that’s using Capybara and Rails, this is usually easily dealt with, as your system tests are written in Ruby and can make use of your ActiveRecord models. In this app, we’ve decided to add some API endpoints (only available in testing) that we can use from our system tests to manipulate that state.
Wrapping Up
In all honesty, I was expecting this to be more of a challenge than it was. In about a day I was able to nail down the infrastructure needed. I’d love to find ways to improve upon this approach, but I think it’s quite acceptable.
While this was more work to put together than a typical Rails + Ember app, and it incurs a little ongoing overhead for instrumentation from system tests, I think that this is a solid foundation that we’ll see a lot of gains from over the course of the project.