Peter Marklund's Home

This weekend I taught a three day Ruby on Rails course here in Rome in Italy. It was a great experience and the people down here have shown the greatest hospitality and have taken very well care of me.

The format of the course was like a workshop with a small group of participants in a private and relaxed setting. I used my course material as a starting point and a road map but then improvised a lot and ended up doing a lot of hands on programming. Basically the course was divided into three parts:

1. Ruby and Rails fundamentals. A lot of time was spent on the programming language itself and we toured the features of Ruby by live coding in TextMate.
2. Writing a demo application. Similar to the AWR book, I built the basics for a store application, including file upload, advanced ActiveRecord associations, and deployment to a VPS with Capistrano.
3. The participants got to work on different features in the application. Time was also spent code reviewing the participants own Rails applications.

Overall I must say the course was quite a success. Every time I teach a course I look for ways to make my courses more interactive, more hands on, and more tailored to the needs of the participants.

I also had a great time outside the course in Rome - a city that I fell in love with immediately.

3 Comments

I've gotten a few error messages lately where a plain vanilla ActiveRecord counter cache update (update_counters_without_lock method) has lead to an error being thrown from Mysql - "Mysql::Error: Deadlock found when trying to get lock; try restarting transaction: UPDATE `events` SET `attendees_count` = COALESCE(`attendees_count`, 0) + 1 WHERE (`id` = 1067)".

It seems someone else has tried to report this as a bug but Mysql is saying that it's a feature and is referring to the lock modes documentation. There is some interesting info on deadlocks in InnoDB over at rubyisms. I haven't had time to dig into the theory though. Has anybody else had this issue? What can be done about it (other than switch to PostgreSQL)?

4 Comments

Over the last couple of years testing has been a controversial topic in Rails. In the beginning though Rails was of course opinionated and gave us model and controller tests along with fixtures for testing our apps. Then integration tests were added by Jamis. Some started using browser testing with tools such as Selenium and Watir. Then the whole RSpec movement swept in with a new terminology, a heavier use of mocking and stubbing, and more isolated testing of the different layers of the MVC stack. One of the most important trends right now seems to be to do "Outside In TDD" with Cucumber and webrat. There are alternatives to RSpec like Shoulda and a move towards simpler tools such as Jeremmy Mcannalys Context and Matchy libraries. There are a number of Factory libraries for replacing fixtures. Maybe the most important controversy over the years has been on whether the database should be stubbed out or not. One of the most common arguments for stubbing out the database is to keep the test execution time low.

Personally I've always been a stubbing sceptic. Given all the changes in the testing landscape it's interesting to read that at Thoughtworks there is a movement away from stubbing and back to where it all started:

"As the teams became familiar with using method stubbing, they used it more and more - falling into the inevitable over-usage where unit tests would stub out every method other than the one being tested. The problem here, as often with using doubles, is brittle tests. As you change the behavior of the application, you also have to change lots of doubles that are mimicking the old behavior. This over-usage has led both teams to move away from stubbed unit tests and to use more rails-style functional tests with direct database access."

2 Comments

I posted over at the Newsdesk developer blog about Rails Development Database Setup Without Migrations.

Make a Comment

I have been doing a bit of blogging over at the Newsdesk developer blog. My two most recent posts are on shelling out to external programs and eager loading of application classes to save STI.

Make a Comment

The ActiveRecord::Callbacks module (that depends on ActiveSupport::Callbacks) defines a multitude of before and after callbacks for the lifecycle of your ActiveRecord objects. Similar to how before filters in controllers work, if a before_validation callback method returns false, the save process will be aborted. Here is what the API documentation says:

# If the returning value of a +before_validation+ callback can be evaluated to +false+,
# the process will be aborted and Base#save< will return +false+.
# If Base#save! is called it will raise a ActiveRecord::RecordInvalid exception.
# Nothing will be appended to the errors object.

# If a before_* callback returns +false+, all the later callbacks and the associated
# action are cancelled. If an after_* callback returns
# +false+, all the later callbacks are cancelled. Callbacks are generally run in the 
# order they are defined, with the exception of callbacks
# defined as methods on the model, which are called last.

What this means is that if a before_validation callback returns false, then save and valid? will both return false but errors.empty? will return true. Yes, you read correctly, there are no validation errors but the record is not valid. Also, note that the valid? method defined in the ActiveRecord::Base class will never even be invoked.

3 Comments

If you want to run your Rails tests with verbose output you can use the TESTOPTS argument like this:

rake test TESTOPTS="-v"

Read More | 4 Comments

I endured a nightmarish debugging session yesterday due to Rails no longer showing proper stack traces from exceptions in plugin code. I've verified this across different Rails apps and it seems to actually be the case that with Rails 2.3.2 you no longer get a stack trace from plugins. Has anybody else had issues with this? Any pointers to where in the Rails code exception handling and stack traces are dealt with and how it could be fixed? I posted about this issue on the Rails core list but haven't received a reply yet.

3 Comments

SlideShare sent me an email saying I'm a rockstar because my Ruby on Rails 101 slides have been getting so popular. I'm really glad that I decided to share my slides and SlideShare has been a great boost in reaching more people. All the great feedback that I get from people makes all the work that went into the slides more than worthwhile. I continue to be fascinated with how rewarding it can be to share information on the web.

2 Comments

I am trying out Rakismet now for this blog to prevent comment spam - an issue that has been bugging me for a long time and that has been getting worse lately. Setting up the Rakismet plugin was a breeze. Very nice! Now, let's hope it actually stops the spammers. We use Rakismet at Newsdesk and I think it's worked out really well there.

7 Comments

I always used logrotate Linux tool to setup log rotation for my Rails apps which has worked fine although it required finding some external config file and understanding its config options and syntax. I never new log rotation could be set up by adding this one line to config/environments/production.rb right in your app:

config.logger = Logger.new(config.log_path, 50, 1.megabyte)

Sweet!

3 Comments

Have you ever tried upgrading Rails from version 1 to version 2? If not, you're in for a treat. Seriously though, it's not that bad, but depending on the size of your app, the number of plugins you use, and how badly you've patched Rails, your mileage may vary.

Here are some notes from an upgrade of this little weblog app just now:

Read More | 13 Comments

At Newsdesk we recently upgraded to Rails 2.3 and ran into a Rails bug triggered by multiple post requests in tests. I was curious to figure out why the bug occured and since I was already familiar with the Rails testing code (in test_process.rb) I was able to nail down the problem pretty quickly. Usually I would probably stopped there, but my colleague Richard encouraged me to submit a Rails patch so I walked the extra mile and wrote a little unit test for my fix. I then followed the contribution instructions to submit an issue in Lighthouse with a patch file followed by the recommended post to the Rails core mailing list. The issue was quickly assigned to Joshua Peek in the Rails core team and a few days later it was applied. It is really encouraging to see how easy it can be to contribute to Rails and that the core team actually picks up a lot of patches and applies them if only they are submitted properly. This really illustrates the power of of open source in general and Git and Rails in particular.

1 Comment

JavaScript test coverage is pretty non-existant in most Rails projects and if your application has a lot of JavaScript code this can become a real problem. But it doesn't have to be that way. One thing you can do just to get some basic syntax and style checking of your JavaScript code is to run it through JavaScript Lint. I've added a simple rake task to run all javascript files through the validator:

namespace :test do
  namespace :javascripts do
    desc "Validate all javascript files with javascript lint - assumes jsl on the command line"
    # Visit http://www.javascriptlint.com to download and install the jsl command line tool
    task :validate do
      total_errors = 0
      Dir[File.join(File.dirname(__FILE__), "..", "..", "public", "javascripts", "*.js")].each do |file_path|
        print "#{file_path} ... "
        result = `jsl -process #{file_path}`
        n_errors, n_warnings = result.match(/(\d+) error\(s\), (\d+) warning\(s\)$/).to_a[1, 2].map(&:to_i)
        
        if n_errors > 0
          puts "FAILED"
          puts result
        else
          puts "OK (#{n_warnings} warnings)"
        end
        total_errors += n_errors
      end

      print "TEST RESULT: "
      if total_errors > 0
        puts "FAILURE - #{total_errors} errors"
      else
        puts "OK"
      end
    end
  end
end

In addition to syntax checking I recommend trying out Dr Nic's javascript_test plugin. To get it to work with the latest prototype I had to download JsUnitTest and use that instead of unittest.js.

3 Comments

UPDATE: Adam Meehan pointed out that my migration didn't work with DST, i.e. different UTC offsets for datetimes at different points of the year. I updated my migration to use a UTC conversion in the database (leading to a variable interval) instead of using a fixed interval adjustment.

If you want to make use of the timezone support in Rails 2.1 and later you'll need to migrate any existing times that you have in your db to UTC. Here is a migration for PostgreSQL I wrote to do that (you'll probably need to adjust it to work on MySQL):

# This migration will work with DST. Because of DST, if you have your datetimes
# spread across the year they will have different offset, i.e. in Stockholm we are UTC+1 usually
# but UTC+2 in the summer.
class MigrateDatabaseToUtc < ActiveRecord::Migration
  COLUMN_TYPES = [:datetime, :timestamp]

  def self.up
    adjust(:up)
  end

  def self.down
    adjust(:down)
  end
  
  def self.adjust(direction)
    connection = ActiveRecord::Base.connection
    connection.tables.each do |table|
      timestamp_columns = connection.columns(table).select do |column|
        COLUMN_TYPES.include?(column.type)
      end

      sign = (direction == :down ? "+" : "-")
      update_clause = timestamp_columns.map do |column|
        "#{column.name} = (#{column.name} #{sign} ((#{column.name} AT TIME ZONE 'UTC')-#{column.name}))"
      end.join(", ")

      execute "UPDATE #{table} SET #{update_clause}" unless update_clause.blank?
    end    
  end
end

When I originally wrote this migration I used a fixed interval adjustment (+1 for Stockholm), i.e. the same approach that Simon Harris uses. However, as Adam Meehan points out in the comments this doesn't work with DST so I adjusted to using a AT TIME ZONE 'UTC' conversion instead that will result in a DST dependent interval. Thanks Adam for pointing this out!

7 Comments

Last week I had the pleasure of teaching a five day introductory course at Tiscali in Cagliari/Itali (Sardinia). The course outline was similar to the course I held back in 2007 but in this case the lectures were half day with the afternoons dedicated to exercises and questions. This format worked out pretty well and I think it's a good idea to have half of the time devoted to hands-on training.

I've updated my old slides for Rails 2.3 and rewritten them in HTML (using S5/Codex). You can view the slides here. The source code for the slides is available on GitHub. Feel free to send feedback and reuse the slides and make any corrections and improvements that you see fit. Thanks!

1 Comment

Today Newsdesk released the Translate plugin that provides a nice web UI for doing translations. The plugin mounts a web UI at /translate where you can list and translate I18n texts. Translations are written directly to YAML files stored at the default location under config/locales. Check out the post at the Newsdesk blog and the README file on Github for more details.

1 Comment

With the timezone support introduced in Rails 2.1 the idea is that all dates in the database are stored in UTC and all dates in Ruby are in a local timezone. The local timezone can be specified by config.timezone in environment.rb or set to the user timezone with Time.zone= in a before filter. Typicaly, when reading/writing from/to the database ActiveRecord will transparently convert time attributes back and forth to UTC for you. However, there is a gotcha with datetimes in ActiveRecord::Base.find conditions. They will only be converted to UTC for you if they are ActiveSupport::TimeWithZone objects, not if they are Time objects. This means that you are fine if you use Time.zone.now, 1.days.ago, or Time.parse("2008-12-23").utc, but not if you use Time.now or Time.parse("2008-12-23"). Example:

  Event.all(:conditions => ["start_time > ?", Time.zone.now])
  Event.all(:conditions => ["start_time > ?", Time.parse("2008-12-23").utc])

Apparently this issue has been reported and marked as invalid. I think it's quite unfortunate that ActiveRecord doesn't do this conversion for us. I suspect other application developers will be bitten by this as well. The difference in behaviour between Time and TimeWithZone objects boils down to the to_s(:db) call:

>> Time.now.to_s(:db)
Time.now.to_s(:db)
=> "2009-01-06 17:52:19"
>> Time.zone.now.to_s(:db)
Time.zone.now.to_s(:db)
=> "2009-01-06 16:52:23"

One way to fix the issue would be to monkey patch the Quoting module in ActiveRecord like this:

module ActiveRecord
  module ConnectionAdapters # :nodoc:
    module Quoting
      # Convert dates and times to UTC so that the following two will be equivalent:
      # Event.all(:conditions => ["start_time > ?", Time.zone.now])
      # Event.all(:conditions => ["start_time > ?", Time.now])
      def quoted_date(value)
        value.respond_to?(:utc) ? value.utc.to_s(:db) : value.to_s(:db)
      end
    end
  end
end

However I'm not sure that this is a good idea and that it won't break anything else. I've at least verified that it doesn't break assignment of ActiveRecord attributes.

18 Comments