Despite the pressure, you knuckle down and fix the bug.

Now it gets interesting: you have to deploy your fix to production. Depending on how your organisation works, this could take anywhere between a couple of minutes and a couple of weeks. You might simply run a single command, or you might spend hours shuffling emails around between different managers trying to get your change signed off. Meanwhile, your customers are still confused and angry. Perhaps they’ve started leaving you for your competitors. Perhaps you feel like leaving your job.

With the bug fixed, you sit down and try to learn some lessons. What can you do to avoid this from happening again?

There are two very different ways you can react to this kind of incident. The choice you make speaks volumes about your organisation.

Make it harder to make mistakes

A typical response to this kind of incident is to go on the defensive. Add more testing phases, hire more testers. Introduce mandatory code review into the development cycle. Add more bureaucracy to the release cycle, to make sure nobody could ever release buggy code into production again.

This is a reaction driven by fear.

Make it easier to fix mistakes

The alternative is to accept the reality. People will always make mistakes, and mistakes will sometimes slip through. From that perspective, it’s clear that what matters is to make it easier to fix your mistakes. This means cutting down on bureaucracy, and trusting developers to have access to their production environments. It means investing in test automation, to allow code to be tested quickly, and building continuous delivery pipelines to make releases happen at the push of a button.

This is a reaction driven by courage.

I know which kind of organisation I’d like to work for.

We’re building it pure test-first, and have kicked off using jsUnit to get us started with something simple, flipping to the browser when we make a change and hitting the ‘run’ button in the jsTest testrunner HTML page.

I’m starting to find this quite unsatisfactory, however.

• Debugging when a test fails seems to be pretty much impossible using the testrunner page. There is a workaround where you put a line to call the failing test function into the fixture html file, then browse to that file, but that all feels like quite a rigmarole when you’re cooking on TDD gas.
• All that alt-tabbing is making my thumb sore! I’m only writing scripts, so I should be able to get rapid feedback like you get with ruby / rails’ autotest plug-in. Shouldn’t I?
• Ultimately, I want to run these tests as part of an automated build process.

It seems I’m not alone in having these goals.

Now jsUnit does have a java server which is built as an Ant task so you can automate the running of tests, so that could solve my automation goal, except we’re a .net shop using NAnt, and I can’t be bothered to get my head around configuring Ant just to run this one task right now.

I also notice from the sourceforge pages that jsUnit hasn’t been touched for a couple of years – version 2.2alpha, the latest release, was uploaded in March 2006.

There are a couple of alternative testing frameworks for javascript that could replace jsTest and make it all feel a bit more sexy.

• Thomas Fuchs has built one that comes with his outstanding script.aculo.us library. The slides from a talk I saw him give at RailsConf London in 2006 are here.
• There’s also JSSpec which looks pretty sweet.

I has a look at JSSpec tonight, and knocked together a quick ruby script which uses the selenium rubygem to run a JSSpec test:

require 'rubygems'
require 'selenium'
require 'webrick'

include WEBrick

def go
    path_to_tests = '/Users/matt/Documents/projects/js-autotest/jsspec'
    port = 8001
    tests_url_base = "http://localhost:#{port}/"

    server = create_web_server_at(path_to_tests, port)
    manager = start_selenium_service
    browser = start_selenium_browser_at(tests_url_base)

    browser.open(tests_url_base + 'demo.html')
    puts "page title is #{browser.get_title}"
    # TODO - check that the specs passed!

    puts 'stopping browser'
    browser.stop

    puts 'stopping selenium service'
    manager.stop

    puts 'stopping web server'
    server.stop    
end

def start_selenium_browser_at(base_url)
    puts 'starting selenium browser'
    browser = Selenium::SeleniumDriver.new("localhost", 4444, "*firefox", base_url, 15000)
    browser.start
    browser
end

def start_selenium_service
    puts 'starting selenium service'
    manager = Selenium::ServerManager.new(Selenium::SeleniumServer.new)
    manager.start
    manager
end

def create_web_server_at(path, port)
    puts "starting web server on port #{port} at path_to_tests #{path}"
    server = HTTPServer.new(
        :Port            => port,
        :DocumentRoot    => path
    )
    trap("INT"){ server.shutdown }
    t = Thread.new { server.start }
    server
end

go

It seems ridiculous to have to write a script that starts a web server, calls a framework that opens a browser that calls the web server that returns a page that runs the harness that runs my tests – surely there must be some shortcuts in there somewhere?!

I’ve considered trying to directly call one of the JavaScript engines from Mozilla, Safari, or the .NET framework but it’s not feeling right, and in any case, most of these test harnesses need use a whole HTML page, not just run javascripts.

I think my next step is to take another look at what Thomas Fuchs has been up to, and also this rails-based effort from Dr Nic. We’re using the prototype.js framework anyway, so this seem like it would be a good fit, and his documentation suggests we should be able to automate it using rake, which should be fun.

Stay tuned.