Polyglot Persistence and Rails 3

Developers at Atomic really value using the right tool for the job. That’s why we occasionally get together outside of work to learn and teach new technologies. Keeping up with the latest tools and approaches is easiest with a group of friends and a couple of beers, so last month we organized an after-work exploration of some non-relational databases. The experience got us thinking about persistence in the new world of NoSQL and Rails 3.

ORM agnosticism has recently become a core principle of Rails. An ORM is a mapping between objects and relations, but in Rails 3 this can mean more than just alternative syntax for the same relational operations. Now your models can be persisted in an entirely different data storage paradigm, or a combination of new and old. That means more flexibility to think about your application outside of tables and columns, without sacrificing the familiarity and convention of Rails.

As with any newcomer to your application stack, you’ll need time to get familiar with performance and operational characteristics of a NoSQL database, so don’t put all your persistence eggs in the same basket. Our own approach to integrating these new tools is to take baby steps: apply them where their novel features can shine instead of completely throwing out the standard approach. Use MongoDB for the part of your app that needs a more flexible schema, and keep using Postgres for payment processing, where its transactions and durability keep it in the game.

This integrated approach to using multiple data stores is often called “polyglot persistence”. It’s more than just a transitional step towards replacing your database. It means letting you always use the right tool for the job and not falling into a monocultural mindset. The salient conclusion of our recent foray was that each database makes its own tradeoffs. Web apps wear MongoDB well, but it’s not best for critical transactions involving multiple records. Neo4j doesn’t pretend to be a whole-app data store, but might be the best way to query your huge social network. Redis is extremely flexible but it may take more work to solve your common problems: it’s more of a general-purpose toolkit than an out-of-box solution. SQL databases are still useful and mature tools for many problems, especially where transactions are necessary.

The new directions in Rails 3 aren’t just about letting you replace ActiveRecord. They entirely decouple the ORM from the core framework, so you can pick the right tools and have them all coexist in harmony. But to fully realize the dream, the greater Rails’ gem ecosystem needs to shift a bit. On a recent project combining Rails 3, ActiveRecord, and Mongoid, we’ve noticed a few common problems.

The first is gems too targeted toward ActiveRecord. Mongoid is extremely similar to Rails’ default ORM, but some gems focus so much on providing magic that they lack the configurability necessary to adapt to new storage paradigms. Too much magic without configurability can lead to too strong a tie to ActiveRecord. Rails 3 offers the convenience of convention without sacrificing the flexibility of configuration, which is a good model to follow.

Another problem we’ve seen is gems that try to accommodate multiple ORMs, but assume only one will be present. This is an acknowledged limitation of Database Cleaner, which would otherwise make fixture cleanup in a multiple database scenario easy and straightforward.

Once we got through these issues, working in a our polyglot rails application has been a fantastic experience. MongoDB has already paid off by allowing us to embed one document inside another for a surprisingly elegant solution to a modeling problem. At the same time using ActiveRecord for user accounts has let us continue using Clearance for breezy authentication and reserved the option for transactional payment processing.

This is really exciting.