I.NFECTIO.US

Anyone thats ever grown an application from a couple thousand hits a day to a couple hundred thousand has run into db optimization issues at some point. The best teacher is experience and Bravenet was a great teacher.

During Bravenets prime years, we had six load-balanced clusters, each with 5 webservers and one MySQL database server. Our userbase was partitioned over these 6 clusters. At peak times, our Queries Per Second on the database servers were over 900 on each. Each page made a minimum of 3-5 database queries (this was before caching became common place) and our traffic load was quite high, so high in fact that a poorly optimized PHP page or a non-indexed query would bring Bravenet down instantly requiring us to fix the script, re-commit and restart all the Apaches to come online. At one point, a nasty bug in our ad-serving code has us down for over 24 hours, it turned out that PHP was loading an array of over 10,000 elements on each request. As soon as we fixed it and committed the changes, Bravenet came back to life.

Premature Optimization

Fine concept for code but practicing it with your database is more akin to Immature Optimization. While it's common place to put indexes on your conditions columns and primary/foreign keys, sometimes (especially in Rails Schema) you just forget. If your writing a small application, you may never see the effects of your error, but as your application grows, it will quickly show itself by slowing down the load time of your pages.

MySQL Query Analyzer

With all that said, I wrote a plugin to make it easier to catch those mistakes and stay on top of your database optimizations. This plugin makes use of the EXPLAIN sql statement in MySQL to print out how MySQL formed its execution plan. Basically, for each SELECT query your application runs in the development or testing environments, Rails will also print the query execution plan right after it so you can quickly analyze the queries Rails is making and either add indexes, reorder your joins and remove unneeded or redundant indexes.

To install:
script/plugin install http://svn.nfectio.us/plugins/query_analyzer

Read the README for more information. Any feedback or improvements welcome. Good luck and don't be immature.

Here's a quick way to follow your tests a little more closely in your test.log. In your test setup, add the following line:

def setup
  RAILS_DEFAULT_LOGGER.debug "\n\e[0;31mRUNNING TEST CASE: #{name}\e[m\n"
end

When you run your tests, you will now see something similar to this.

While installing all the pre-requisites for running Rails on my new Mac Pro, I ran across a problem with either the mysql gem or mysql 5.0.24 where it depends on 'ulong' being defined by the OS include files. On OS X it isn't, and this causes the ruby mysql gem to fail compilation which causes Rails to (possibly) throw a Lost Connection to MySQL Server during query error due to the fact that the built-in Ruby driver doesn't work with mysql 4.1+ authentication (edge rails fixes this limitation with the built-in mysql driver). A quick fix for this is to edit the ruby gem mysql.c and add the #define for a ulong so the gem can properly compile.

#define ulong unsigned long

mojombo in #caboose, just released his first version of Chronic, a new Ruby Gem for natural language processing of Dates and Times.

gem install chronic

irb:>
Chronic.parse('tomorrow')
    #=> Mon Aug 28 12:00:00 PDT 2006

Chronic.parse('monday', :context => :past)
    #=> Mon Aug 21 12:00:00 PDT 2006

Chronic.parse('this tuesday 5:00')
    #=> Tue Aug 29 17:00:00 PDT 2006

Chronic.parse('this tuesday 5:00', :ambiguous_time_range => :none)
    #=> Tue Aug 29 05:00:00 PDT 2006

Chronic.parse('may 27th', :now => Time.local(2000, 1, 1))
    #=> Sat May 27 12:00:00 PDT 2000

Chronic.parse('may 27th', :guess => false)
    #=> Sun May 27 00:00:00 PDT 2007..Mon May 28 00:00:00 PDT 2007

The thing I find really nice is that if you pass it a malformed date, it returns nil. This means, in your model validations, you can add:

require 'chronic'

class Meeting < ActiveRecord::Base

  def validation
    errors.add :meeting_date, 'is not a valid date' if Chronic.parse(meeting_date.to_s).nil?
  end

end

The auto loading path which make models in subdirectories of app/models load in Rails has been removed from the lastest trunk. If you are using models in subdirectories of the app/models path, you will now need to add this in your environment.rb:

config.autoload_paths += Dir[RAILS_ROOT + '/app/models/*/']

This won't affect most users, but if you've organized your models into subdirectories similar to the convention used for namespaces controllers, then you will want to update your applications before you move to 1.2.

Specifically, the following path was removed:

#{root_path}/app/models/[_a-z]*"]

Twenty years ago, I learned about sex for the first time. All my friends said they were doing it, although I think most of them were full of it and just wanted to look cool. I didn't know much about sex back then but I'd read about it in The Joy of Sex and watched it on TV. There's something about actually doing it that you don't understand through those other mediums.

Learning about testing has been a very similar experience for me. I've known about it for awhile, all my programmer friends say they're doing it, and I've even read Test Driven Development By Example by Kent Beck. But doing it for the first time is a completely different experience. At first, I didn't know exactly what I was doing, I was aimlessly exploring the inputs and outputs, and before I knew it, I'd had my first testing orgasm. The tests I was writing discovered a serious bug in my code. That's when it clicked for me, really understanding what test driven development is about. If I had written the tests first, the specific bug they revealed wouldn't have existed in the first place.

Now that I've popped my testing cherry, I want to keep doing it, and much like my first sexual partner, Rails makes it easy. And when you get comfortable with your new partner, you can introduce great toys like ZenTest that will improve your experience.

The Rails Developers are rushing to get a 1.2 release out the door before RailsConf Europe. I'll try to list a quick run down of changes you'll may not be aware of in 1.2:

• Access nested attributes in RJS templates. Now you can use syntax like this: page[:foo][:style][:color] which will generate javascript like: $('foo').style.color


• Michael Koziarski is my hero for this change. If you've ever tried to use a 'quote' column in your model, you quickly found out that it conflicted with AR's quote column method. No longer is this an issue. def quote has been changed to def quote_value


• Calling image_tag without the file extension now raises a Deprecation warning. In 2.0, it will no longer append .png to the file name. This is welcome change, unneeded magic is well...unneeded.


• UPDATE: This change has been reverted for the time being. Now you can access the locals hash directly. In your partials, you can check if a local varialbe was passed in by doing:
  if locals[:local_var] then ...


• In 2.0, the default for foreign_keys will be the association name, rather than the class name. So if you specify a :class_name with no :foreign_key on your belongs_to associations, it will throw a Deprecation warning. The tests explain this pretty well:
  
  

  def test_deprecated_inferred_foreign_key  
    assert_not_deprecated { Company.belongs_to :firm }  
    assert_not_deprecated { Company.belongs_to :client, :foreign_key => "firm_id" }  
    assert_not_deprecated { Company.belongs_to :firm, :class_name => "Firm", :foreign_key => "client_of" }  
    assert_deprecated("inferred foreign_key name") { Company.belongs_to :client, :class_name => "Firm" }  
  end 



• The Ruby/MySQL driver that ships with Rails has been updated to work with the new authentication in MySQL 4.1+. Of course, everyone uses the Ruby C bindings to MySQL right?


• My own patch to distance_of_time_in_words was applied. What does this mean to you? Nothing at all, I just wanted to tell you about it.


• Some nice additions to Prototype Element Handling. You can now traverse the DOM using the new Element.up, Element.down, Element.previous and Element.next methods. To make debugging easier, you can use Element.inspect. Check out the changeset for more useful changes.
  
  

  Examples:  
    <div id="sidebar">                   -> $('nav').up() or $('menu').up('div')   
      <ul id="nav">                     -> $('sidebar').down()  or $('sidebar').down('ul') or $('menu').previous()  
        <li>...</li>                    -> $('sidebar').down(1) or $('sidebar').down('li')  
        <li>...</li>                    -> $('sidebar').down(2) or $('sidebar').down('li', 2) or $('sidebar').down('li').next('li')  
         <li class="selected">...</li>    -> $('sidebar').down('li.selected')  
       </ul>                               
       <ul id="menu">                    -> $('sidebar').down('ul').next()  
  
  



• Rails will actually use the 500.html thats been sitting in public for awhile when your application fails to catch an exception not related to routing.


• In the testing department, assert_tag has been deprecated in favor of the new assert_select family of assertions.


• Another one of my patches got applied. radio_button_tag now generates a unique id attribute based on the name_value pattern.


• And plenty of other deprecation warnings throughout Rails...those you can figure out for yourself.

LiteSpeed Web Server is a commercial product being marketed as "the world's best performing Ruby SAPI and easiest Ruby on Rails application setup". Having used LiteSpeed for over 6 months now and after running my own tests, I agree with that statement whole-heartedly. Being a recent Apple convert, to me LiteSpeed is the OS X of the web server world. Not open source, but it looks and works great and the developers actually listen to what the users need.

LiteSpeed offers a Standard Edition which is free for private and commercial use, the only restriction being that your site can't serve content related to pornography, warez or illegal activities. This is an odd restriction, but I guess it's cool that they stand by their principles.

To keep things DRY, I won't repeat LiteSpeeds feature set here. Instead, I'll "show" you why I choose LiteSpeed and why I think you should give it a test drive as well. I've put together two screencasts for your viewing pleasure. The first will demonstrate just how easy it is to get up and running with LiteSpeed. From installing the server to a fully deployed Rails application in under 4 minutes. Live and uncut. It moves along pretty fast, but I felt it was a good demonstration of the effort the LiteSpeed developers have put into supporting Ruby on Rails. In the second screencast, we'll take a more in-depth look at some of the cool features of LiteSpeed Web Server. Hope you enjoy them, I had fun making them.

Up and Running with Rails in Under 4 Minutes - 3:25 runtime - Flash

LiteSpeed In-Depth - 21:35 runtime - Flash

UPDATE: The screencasts show that you have to copy dispatch.lsapi into your public directory, with the 2.2 release, this requirement is lifted. The ruby-lsapi gem still needs to be installed, but LiteSpeed will handle the dispatching to Rails through their LSAPI automatically.

UPDATE: When running the script to restart LiteSpeed, make sure you run it as a user with root priviledges. So for Mac users: sudo lswsctrl restart

Kevin Clark, a fellow #cabooser, and great programmer posted an article on his blog today that for one reason or another sparked some contraversy. He wrote about Rails coding practices that he believes (either through experience or following the core mailing list) should not be used anymore. The pros and cons of scaffolding in production, engines, components, protected instance variables and help vampires are all things that have been discussed openly in various venues and Kevin advocated against them with his reasons why.


What happened next is hard to describe but it's like a pack of wolves came out and shredded him apart. Not sure where these "Rails" programmers came from but I hope they don't stick around. Besides, how can you get upset at a guy that looks like a cute little chia pet?

The guys over at LiteSpeed really stepped it up this time with their support for Ruby on Rails in their newest release of LiteSpeed 2.2. The ability to configure and deploy a Rails site now has its own configuration page in the administration GUI.



Once you define the defaults for RoR applications on your webserver, you can setup and deploy a site in just a few clicks. Prior to the 2.2 release you had several options for getting LiteSpeed to redirect requests to dispatch.lsapi, you could use rewrites or 404 redirects. Now in 2.2 you can just use the new Rails context feature. LiteSpeed will handle the connections to dispatch.lsapi.



Speed plus agility, what a great combination. Check out their Wiki for some quick start instructions on getting up and running with Ruby on Rails on their platform. I'm interested in seeing what performance benefits I will receive with their Enterprise Version running Rails on the new SMP server, hope they offer educational discounts.