Daikini

I’ve written a couple test helpers that I’ve found to be pretty useful when testing ActiveRecord validations. Just add them to your RAILS_ROOT/test/test_helper.rb file.

The assert_errors_on method takes an object and a hash of attributes and the validation errors that they should have.

The assert_no_errors_on method takes an object and a list of attributes that should not have any validation errors.

   1  def assert_errors_on(object, errors = {})
   2    assert !object.valid?
   3    errors.each do |attribute, error| 
   4      assert_equal error.to_a.sort, object.errors.on(attribute).to_a.sort, "Error on #{attribute}" 
   5    end
   6  end
   7    
   8  def assert_no_errors_on(object, *attributes)
   9    object.valid?
  10    attributes.each { |attribute| assert_nil object.errors.on(attribute), "Error on #{attribute}" }
  11  end

You can then use them like this:

   1  require File.dirname(__FILE__) + '/../test_helper'
   2  
   3  class PersonTest < Test::Unit::TestCase
   4    def test_should_require_unique_email_address
   5      # Using assert_errors_on method
   6      person = Person.new :email_address => "cosmo@sprockets.com"
   7      assert_errors_on person, :email_address => "has already been taken"
   8  
   9      # Doing the same thing but without the assert_errors_on method
  10      person = Person.new :email_address => "cosmo@sprockets.com"
  11      assert !person.valid?
  12      assert_equal "has already been taken", person.errors.on(:email_address)
  13  
  14      # Using assert_no_errors on method
  15      person = Person.new :email_address => "john.smith@example.com"
  16      assert_no_errors_on person, :email_address
  17      
  18      # Doing the same thing but without the assert_no_errors_on method
  19      person = Person.new :email_address => "john.smith@example.com"
  20      person.valid?
  21      assert_nil person.errors.on(:email_address)
  22    end
  23  
  24    # We can test other attributes independently as well
  25  
  26    def test_should_require_a_first_name
  27      person = Person.new
  28      assert_errors_on person, :first_name => "can't be blank"
  29  
  30      person = Person.new :first_name => "Jonathan"
  31      assert_no_errors_on :person, :first_name
  32    end
  33  
  34    def test_should_require_a_last_name
  35      person = Person.new
  36      assert_errors_on person, :last_name => "can't be blank"
  37  
  38      person = Person.new :last_name => "Younger"
  39      assert_no_errors_on :person, :last_name
  40    end
  41  
  42    def test_should_blah_blah_blah
  43      ...
  44    end
  45  end

Updated 2007-11-28: Added additional attribute tests to show that the attributes can be tested in isolation without regards to ensuring that all of the attributes have to be valid for testing to work.

Updated 2007-09-14: Added code for showing what the tests would look like by not using the test helpers.

Tom Preston-Werner outlined pretty well in his RailsConf 2007 presentation why fixtures suck so I’ll just repeat them here:

Why fixtures suck:

• swampy
• become an unmanageable mess
• hard to keep track of links between data
• things aren’t easy to refactor
• no namespacing – i.e. you need to have lots of users in different states for good testing
• brittleness – one day you add a new column to a table, and half of your tests fail
• no validation – there is no automatic way for ActiveRecord validation for fixture data
• contamination – tests pass independently, but fail when running all at once
• performance – fixtures can be slow

Tom came up with a possible solution to these problems but I’ve been kicking around a different solution. Let me just say that I’m not entirely sure there is a perfect solution to the fixture problem but the one I came up with works really well for me.

I’ve worked on two projects that took two approaches to the fixture problem. Project A used the Rails fixture files as intended. Project A pretty much experienced all of the problems above when dealing with the fixture data. The big kicker was that performance was absolutely horrible. The combined unit and functional tests took over 2 minutes to run. Needless to say less tests were being written as time went on.

Not wanting to repeat the testing hell that Project A suffered from Project B didn’t use a single fixture file. Instead Project B used the totally kick-ass Mocha library for all the tests. Objects were mocked or stubbed as needed. The big problem that Project B experienced was that there was easily twice the amount of test code than there was actual code because the test code was pretty much just duplicating all of the logic inside the regular code anyway. Every ActiveRecord find method was being mocked out and returned manually instead of letting the database do its job. The big benefit of not hitting the database was that the tests were super fast. Project B was tested very well but the tests were very brittle.

After thinking about the fixture problem for awhile it occurred to me that I really liked the Project B approach of specifying what data were being used for a given test. I just didn’t like the fact that I was having to write a lot of test code to do what the database should have been doing. The Project A approach allowed the database do what the database does best but sucked because of the fixture file problem in general.

So when it came time for Project C I needed an entirely new approach to testing. What if I could combine the benefits of each approach, Project A letting the database do what its best at and the Project B approach of specifying the data in the exact context it is needed? So I did just that and for me it solved the fixture problem very well. I converted Project A to use this new approach and the tests went down from over 2 minutes to run to just under 25 seconds. Still not great but way better than it was and it no longer suffered from the other fixture problems it was having. Needless to say, testing is going up and the code is getting better.

Okay, enough talking, let’s see some freakin’ code already. Let’s say that your application has a Person model and each person is required to have a unique email address.

   1  class Person < ActiveRecord::Base
   2    validates_uniqueness_of :email_address
   3  end

Using the normal fixture file in test/fixtures/people.yml you would add a person entry with an email address you could test against.

   1  first:
   2    id: 1
   3    name: Cosmo Spacely
   4    email_address: cosmo@sprockets.com

Then in your test/unit/person.rb unit test you would have a test something like this:

   1  require File.dirname(__FILE__) + '/../test_helper'
   2  
   3  class PersonTest < Test::Unit::TestCase
   4    fixtures :people
   5  
   6    def test_should_require_unique_email_address
   7      person = Person.new :email_address => "cosmo@sprockets.com"
   8      assert !person.valid?
   9      assert_equal "has already been taken", person.errors.on(:email_address)
  10  
  11      person = Person.new :email_address => "john.smith@example.com"
  12      person.valid?
  13      assert_nil person.errors.on(:email_address)
  14    end
  15  end

To change this example to use the fixture method you would simply remove the call to fixtures :people that loaded the yaml fixture file. Then in the test method you would add a call to the new fixture method and pass in the data that should be loaded.

   1  require File.dirname(__FILE__) + '/../test_helper'
   2  
   3  class PersonTest < Test::Unit::TestCase
   4    # We don't need to load the people fixtures anymore
   5    # fixtures :people
   6                     
   7    def test_should_require_unique_email_address
   8      # The fixture method will load in the data we need for this test
   9      fixture :person, :email_address => "cosmo@sprockets.com"
  10      
  11      person = Person.new :email_address => "cosmo@sprockets.com"
  12      assert !person.valid?
  13      assert_equal "has already been taken", person.errors.on(:email_address)
  14  
  15      person = Person.new :email_address => "john.smith@example.com"
  16      person.valid?
  17      assert_nil person.errors.on(:email_address)
  18    end
  19  end

If you think taking this approach would work for you then you can install the fixture plugin with:

script/plugin install svn://svn.roundhaus.com/daikini/plugins/fixture