Daikini

Tom Preston-Werner outlined pretty well in his RailsConf 2007 presentation why fixtures suck so I’ll just repeat them here:

Why fixtures suck:

• swampy
• become an unmanageable mess
• hard to keep track of links between data
• things aren’t easy to refactor
• no namespacing – i.e. you need to have lots of users in different states for good testing
• brittleness – one day you add a new column to a table, and half of your tests fail
• no validation – there is no automatic way for ActiveRecord validation for fixture data
• contamination – tests pass independently, but fail when running all at once
• performance – fixtures can be slow

Tom came up with a possible solution to these problems but I’ve been kicking around a different solution. Let me just say that I’m not entirely sure there is a perfect solution to the fixture problem but the one I came up with works really well for me.

I’ve worked on two projects that took two approaches to the fixture problem. Project A used the Rails fixture files as intended. Project A pretty much experienced all of the problems above when dealing with the fixture data. The big kicker was that performance was absolutely horrible. The combined unit and functional tests took over 2 minutes to run. Needless to say less tests were being written as time went on.

Not wanting to repeat the testing hell that Project A suffered from Project B didn’t use a single fixture file. Instead Project B used the totally kick-ass Mocha library for all the tests. Objects were mocked or stubbed as needed. The big problem that Project B experienced was that there was easily twice the amount of test code than there was actual code because the test code was pretty much just duplicating all of the logic inside the regular code anyway. Every ActiveRecord find method was being mocked out and returned manually instead of letting the database do its job. The big benefit of not hitting the database was that the tests were super fast. Project B was tested very well but the tests were very brittle.

After thinking about the fixture problem for awhile it occurred to me that I really liked the Project B approach of specifying what data were being used for a given test. I just didn’t like the fact that I was having to write a lot of test code to do what the database should have been doing. The Project A approach allowed the database do what the database does best but sucked because of the fixture file problem in general.

So when it came time for Project C I needed an entirely new approach to testing. What if I could combine the benefits of each approach, Project A letting the database do what its best at and the Project B approach of specifying the data in the exact context it is needed? So I did just that and for me it solved the fixture problem very well. I converted Project A to use this new approach and the tests went down from over 2 minutes to run to just under 25 seconds. Still not great but way better than it was and it no longer suffered from the other fixture problems it was having. Needless to say, testing is going up and the code is getting better.

Okay, enough talking, let’s see some freakin’ code already. Let’s say that your application has a Person model and each person is required to have a unique email address.

   1  class Person < ActiveRecord::Base
   2    validates_uniqueness_of :email_address
   3  end

Using the normal fixture file in test/fixtures/people.yml you would add a person entry with an email address you could test against.

   1  first:
   2    id: 1
   3    name: Cosmo Spacely
   4    email_address: cosmo@sprockets.com

Then in your test/unit/person.rb unit test you would have a test something like this:

   1  require File.dirname(__FILE__) + '/../test_helper'
   2  
   3  class PersonTest < Test::Unit::TestCase
   4    fixtures :people
   5  
   6    def test_should_require_unique_email_address
   7      person = Person.new :email_address => "cosmo@sprockets.com"
   8      assert !person.valid?
   9      assert_equal "has already been taken", person.errors.on(:email_address)
  10  
  11      person = Person.new :email_address => "john.smith@example.com"
  12      person.valid?
  13      assert_nil person.errors.on(:email_address)
  14    end
  15  end

To change this example to use the fixture method you would simply remove the call to fixtures :people that loaded the yaml fixture file. Then in the test method you would add a call to the new fixture method and pass in the data that should be loaded.

   1  require File.dirname(__FILE__) + '/../test_helper'
   2  
   3  class PersonTest < Test::Unit::TestCase
   4    # We don't need to load the people fixtures anymore
   5    # fixtures :people
   6                     
   7    def test_should_require_unique_email_address
   8      # The fixture method will load in the data we need for this test
   9      fixture :person, :email_address => "cosmo@sprockets.com"
  10      
  11      person = Person.new :email_address => "cosmo@sprockets.com"
  12      assert !person.valid?
  13      assert_equal "has already been taken", person.errors.on(:email_address)
  14  
  15      person = Person.new :email_address => "john.smith@example.com"
  16      person.valid?
  17      assert_nil person.errors.on(:email_address)
  18    end
  19  end

If you think taking this approach would work for you then you can install the fixture plugin with:

script/plugin install svn://svn.roundhaus.com/daikini/plugins/fixture