Testing Gotchas in Rails 6

Posted by Bob Silva Tue, 10 Oct 2006 04:07:00 GMT

I just spent the past several hours tracking down an issue with my tests. I use a combination of autotest from the ZenTest Suite and rails_rcov for my Rails testing. While testing using autotest, all my tests were passing just fine. I then went to check test-code coverage and all of a sudden I have a failing test.

* rake test:functionals:rcov - failed.
* rake test:functionals - failed
* ruby test/functional/clients_controller_test.rb - passed
* autotest - passed

So what gives? Basically, when running tests using the rake tasks, the test databases are recreated each time you run the task via rake db:test:clone. The real problem was that I was failed to load one of my fixtures in my functional test, but since the data existed from a previous test which used those fixtures, the tests passed. But when running the test from the rake task which resets all the tables, no fixture data existed in the database causing the difference in behavior between the two tools.

As always, if your tests are hokie, CHECK YOUR FIXTURES FIRST! Enjoy this tip.

Comments

Leave a response

  1. seb about 3 hours later:
    Damn correct, I spent a long time months ago on this!! Great to share it on your blog
    Seb
  2. Luke Redpath about 9 hours later:
    Its things like that make me avoid Rails fixtures like the plague - a terribly thought out idea if there ever was one. There is a perfectly natural home for test fixtures....setup/teardown. Create fixtures as and when you need them - loading in fixtures in bulk using Rails' fixtures functionality makes your tests slow and brittle (you'd be amazed at the speed increase you get by not using them!). More: http://glu.ttono.us/articles/2006/08/07/why-fixtures-suck-and-how-we-can-fix-them
  3. Rich Waters 23 days later:
    Ran into the same issue not long ago. Btw, are you getting autotest to run rcov? or are you just using them both separately?
  4. Rich Waters 23 days later:
    Ran into the same issue not long ago. Btw, are you getting autotest to run rcov? or are you just using them both separately?
  5. Bob Silva 24 days later:
    I just run them separately.
  6. matt 5 months later:
    I most often run my tests seperatly, during development, so I experience this problem from the other direction. Loading up the fixtures returns the db to an expected state that tests depend on. What ends up happening when you forget to include the fixtures is that an earlier test will run, loading up the db and manipulating the records, then when it gets to the next test, the db isn't in expected state, so the test fails. Yea, first time, it was really confusing why individual tests passed, but the entire test suite failed.
Comments