Automating Javascript Unit Tests / Specs – Part 1

I’m building an Adobe Air application at the moment, which basically means loads of javascript development.

We’re building it pure test-first, and have kicked off using jsUnit to get us started with something simple, flipping to the browser when we make a change and hitting the ‘run’ button in the jsTest testrunner HTML page.

I’m starting to find this quite unsatisfactory, however.

• Debugging when a test fails seems to be pretty much impossible using the testrunner page. There is a workaround where you put a line to call the failing test function into the fixture html file, then browse to that file, but that all feels like quite a rigmarole when you’re cooking on TDD gas.
• All that alt-tabbing is making my thumb sore! I’m only writing scripts, so I should be able to get rapid feedback like you get with ruby / rails’ autotest plug-in. Shouldn’t I?
• Ultimately, I want to run these tests as part of an automated build process.

It seems I’m not alone in having these goals.

Now jsUnit does have a java server which is built as an Ant task so you can automate the running of tests, so that could solve my automation goal, except we’re a .net shop using NAnt, and I can’t be bothered to get my head around configuring Ant just to run this one task right now.

I also notice from the sourceforge pages that jsUnit hasn’t been touched for a couple of years – version 2.2alpha, the latest release, was uploaded in March 2006.

There are a couple of alternative testing frameworks for javascript that could replace jsTest and make it all feel a bit more sexy.

• Thomas Fuchs has built one that comes with his outstanding script.aculo.us library. The slides from a talk I saw him give at RailsConf London in 2006 are here.
• There’s also JSSpec which looks pretty sweet.

I has a look at JSSpec tonight, and knocked together a quick ruby script which uses the selenium rubygem to run a JSSpec test:

require 'rubygems'
require 'selenium'
require 'webrick'

include WEBrick

def go
    path_to_tests = '/Users/matt/Documents/projects/js-autotest/jsspec'
    port = 8001
    tests_url_base = "http://localhost:#{port}/"

    server = create_web_server_at(path_to_tests, port)
    manager = start_selenium_service
    browser = start_selenium_browser_at(tests_url_base)

    browser.open(tests_url_base + 'demo.html')
    puts "page title is #{browser.get_title}"
    # TODO - check that the specs passed!

    puts 'stopping browser'
    browser.stop

    puts 'stopping selenium service'
    manager.stop

    puts 'stopping web server'
    server.stop    
end

def start_selenium_browser_at(base_url)
    puts 'starting selenium browser'
    browser = Selenium::SeleniumDriver.new("localhost", 4444, "*firefox", base_url, 15000)
    browser.start
    browser
end

def start_selenium_service
    puts 'starting selenium service'
    manager = Selenium::ServerManager.new(Selenium::SeleniumServer.new)
    manager.start
    manager
end

def create_web_server_at(path, port)
    puts "starting web server on port #{port} at path_to_tests #{path}"
    server = HTTPServer.new(
        :Port            => port,
        :DocumentRoot    => path
    )
    trap("INT"){ server.shutdown }
    t = Thread.new { server.start }
    server
end

go

It seems ridiculous to have to write a script that starts a web server, calls a framework that opens a browser that calls the web server that returns a page that runs the harness that runs my tests – surely there must be some shortcuts in there somewhere?!

I’ve considered trying to directly call one of the JavaScript engines from Mozilla, Safari, or the .NET framework but it’s not feeling right, and in any case, most of these test harnesses need use a whole HTML page, not just run javascripts.

I think my next step is to take another look at what Thomas Fuchs has been up to, and also this rails-based effort from Dr Nic. We’re using the prototype.js framework anyway, so this seem like it would be a good fit, and his documentation suggests we should be able to automate it using rake, which should be fun.

Stay tuned.