I.NFECTIO.US: Testing Gotchas in Rails

Testing Gotchas in Rails

Bob Silva — Mon, 09 Oct 2006 21:07:00 -0700

I just spent the past several hours tracking down an issue with my tests. I use a combination of autotest from the ZenTest Suite and rails_rcov for my Rails testing. While testing using autotest, all my tests were passing just fine. I then went to check test-code coverage and all of a sudden I have a failing test.

* rake test:functionals:rcov - failed.
* rake test:functionals - failed
* ruby test/functional/clients_controller_test.rb - passed
* autotest - passed

So what gives? Basically, when running tests using the rake tasks, the test databases are recreated each time you run the task via rake db:test:clone. The real problem was that I was failed to load one of my fixtures in my functional test, but since the data existed from a previous test which used those fixtures, the tests passed. But when running the test from the rake task which resets all the tables, no fixture data existed in the database causing the difference in behavior between the two tools.

As always, if your tests are hokie, CHECK YOUR FIXTURES FIRST! Enjoy this tip.

"Testing Gotchas in Rails" by matt

Thu, 22 Feb 2007 11:15:39 -0800

I most often run my tests seperatly, during development, so I experience this problem from the other direction. Loading up the fixtures returns the db to an expected state that tests depend on. What ends up happening when you forget to include the fixtures is that an earlier test will run, loading up the db and manipulating the records, then when it gets to the next test, the db isn't in expected state, so the test fails. Yea, first time, it was really confusing why individual tests passed, but the entire test suite failed.

"Testing Gotchas in Rails" by Bob Silva

Thu, 02 Nov 2006 11:45:07 -0800

I just run them separately.

"Testing Gotchas in Rails" by Rich Waters

Wed, 01 Nov 2006 11:34:26 -0800

Ran into the same issue not long ago. Btw, are you getting autotest to run rcov? or are you just using them both separately?

"Testing Gotchas in Rails" by Rich Waters

Wed, 01 Nov 2006 11:34:24 -0800

Ran into the same issue not long ago. Btw, are you getting autotest to run rcov? or are you just using them both separately?

"Testing Gotchas in Rails" by Luke Redpath

Tue, 10 Oct 2006 05:54:58 -0700

Its things like that make me avoid Rails fixtures like the plague - a terribly thought out idea if there ever was one. There is a perfectly natural home for test fixtures....setup/teardown. Create fixtures as and when you need them - loading in fixtures in bulk using Rails' fixtures functionality makes your tests slow and brittle (you'd be amazed at the speed increase you get by not using them!). More: http://glu.ttono.us/articles/2006/08/07/why-fixtures-suck-and-how-we-can-fix-them

"Testing Gotchas in Rails" by seb

Mon, 09 Oct 2006 23:45:39 -0700

Damn correct, I spent a long time months ago on this!! Great to share it on your blog
Seb