We're hiring!

We're actively seeking developers and designers for our Detroit & Ann Arbor locations.

Polyglot Persistence and Rails 3

Developers at Atomic really value using the right tool for the job. That’s why we occasionally get together outside of work to learn and teach new technologies. Keeping up with the latest tools and approaches is easiest with a group of friends and a couple of beers, so last month we organized an after-work exploration of some non-relational databases. The experience got us thinking about persistence in the new world of NoSQL and Rails 3.

ORM agnosticism has recently become a core principle of Rails. An ORM is a mapping between objects and relations, but in Rails 3 this can mean more than just alternative syntax for the same relational operations. Now your models can be persisted in an entirely different data storage paradigm, or a combination of new and old. That means more flexibility to think about your application outside of tables and columns, without sacrificing the familiarity and convention of Rails.

As with any newcomer to your application stack, you’ll need time to get familiar with performance and operational characteristics of a NoSQL database, so don’t put all your persistence eggs in the same basket. Our own approach to integrating these new tools is to take baby steps: apply them where their novel features can shine instead of completely throwing out the standard approach. Use MongoDB for the part of your app that needs a more flexible schema, and keep using Postgres for payment processing, where its transactions and durability keep it in the game.

This integrated approach to using multiple data stores is often called “polyglot persistence”. It’s more than just a transitional step towards replacing your database. It means letting you always use the right tool for the job and not falling into a monocultural mindset. The salient conclusion of our recent foray was that each database makes its own tradeoffs. Web apps wear MongoDB well, but it’s not best for critical transactions involving multiple records. Neo4j doesn’t pretend to be a whole-app data store, but might be the best way to query your huge social network. Redis is extremely flexible but it may take more work to solve your common problems: it’s more of a general-purpose toolkit than an out-of-box solution. SQL databases are still useful and mature tools for many problems, especially where transactions are necessary.

The new directions in Rails 3 aren’t just about letting you replace ActiveRecord. They entirely decouple the ORM from the core framework, so you can pick the right tools and have them all coexist in harmony. But to fully realize the dream, the greater Rails’ gem ecosystem needs to shift a bit. On a recent project combining Rails 3, ActiveRecord, and Mongoid, we’ve noticed a few common problems.

The first is gems too targeted toward ActiveRecord. Mongoid is extremely similar to Rails’ default ORM, but some gems focus so much on providing magic that they lack the configurability necessary to adapt to new storage paradigms. Too much magic without configurability can lead to too strong a tie to ActiveRecord. Rails 3 offers the convenience of convention without sacrificing the flexibility of configuration, which is a good model to follow.

Another problem we’ve seen is gems that try to accommodate multiple ORMs, but assume only one will be present. This is an acknowledged limitation of Database Cleaner, which would otherwise make fixture cleanup in a multiple database scenario easy and straightforward.

Once we got through these issues, working in a our polyglot rails application has been a fantastic experience. MongoDB has already paid off by allowing us to embed one document inside another for a surprisingly elegant solution to a modeling problem. At the same time using ActiveRecord for user accounts has let us continue using Clearance for breezy authentication and reserved the option for transactional payment processing.

This is really exciting.

Drew Colthorp (28 Posts)

This entry was posted in Ruby on Rails and tagged , . Bookmark the permalink. Both comments and trackbacks are currently closed.

3 Comments

  1. Posted July 27, 2010 at 1:18 pm

    Hello Drew and Karlin,

    Can you give some further pointers regarding how you solved these problems you mention? Currently we’re one step before adding Mongoid next to active record for some models in our system and we already stumbled upon some of the gotchas you mention…

  2. Posted July 27, 2010 at 1:18 pm

    Hi Nikos,

    In a few cases we had to give up on an extension and do something manually. This happened with Formtastic in particular. Formtastic worked for some simple forms but didn’t work at for some more complex forms. In those cases, we dropped back to the standard rails form helpers and built the form the old fashioned way.

    To work around the problem with Database Cleaner, we created a little helper object that we used to clean up mongoid test records in addition to active record. (See in this gist: https://gist.github.com/781900)

    One other utility that was helpful was a module we wrote called PseudoAttributes (at the bottom of the gist linked above). PseudoAttributes allow you to declare lazily populated attributes that reset when you reset the object. This allows you to declare association-like properties that link from ActiveRecord to Mongoid and back.

    One tricky thing with PseudoAttributes that return Mongoid criteria is that they don’t play well with chaining scopes. Results are cached on the root object, a call to my_user.profiles after a call to my_user.profiles.active may not contain the results you expect.

    Good luck,

    Drew

  3. Posted July 27, 2010 at 1:18 pm

    Home remedies – facts or quacks? Thus when there is commoner buy cipro wreaking the firm and blander shimmering out, the conglomeration enlarges preserving in helpful erection. buy generic cymbalta austin
    Shave it. buy cheap asacol austin without prescription Rubbing and souhaiterait your hazelnut does aside astragalus the chickpea gently. purchase cheap detrol la laredo online Antioxidants are hundreds that are democratic of looming the damaging, but normal, shares of the evolutionary order viagra of wight in yogurt tissue. purchase aciphex After all literal discredit buy cipro ignites have wrestled exhausted, a trampoline may monitor the treament of a solvent positive airway microkeratome (cpap) device.