Belly Wants to Eat Your Tests

Ever since I lead the team at Songkick through an Acceptance-Test-Driven re-write of their gorgeous web-ui, I’ve been thinking about problem of scaling a large suite of acceptance tests. By the time I left Songkick for the wilds of Scotland, it would take over 3 hours to run all the Cucumber tests on a single machine.

When things take that long, TDD stops being fun.

Intelligent Selection

In order to make an intelligent selection of which tests to run, you need some knowledge of the past history of each of your tests. Most testing tools are like goldfish: they run the tests and show you what failed, then on the next run they wipe the slate clean and start over. Dumb.

Sir Kent Beck, always ahead of the game, has been building an exciting new product to enable precisely this kind of selective testing for Java projects.

But I don’t work on Java projects.

Enter the Belly

I decided to build a web service that would record the history of each scenario in my Cucumber test suite, so that I could start to make decisions about which ones to run. I see no reason why this service can’t be generic enough to work for any kind of test case, but Cucumber scenarios seem like a good place to get started, since that’s where I do a lot of my testing, and they’re often slow.

Belly works by installing a little hook into your Cucumber test suite. When the tests run, Belly sends a message to the central ‘hub’ web service (currently hosted at reporting the result of the test. Gradually, Belly builds up a picture of your test suite, which you can browse from the website.


The current version of Belly is alpha-ware, proof-of-concept. It works, but I’m sure it won’t scale well to thousands of users with thousands of tests. I’m sure you’ll find bugs. It also looks pretty rough, but don’t let that put you off; there’s huge potential here.


Right now, probably the most useful feature is the belly rerun command, which helps you focus on running just the cukes that you’ve broken. Rather than having to keep track of them in a rerun.txt file, Belly will remember everything you’ve broken and give you the output you need to run it again with Cucumber:

cucumber `belly rerun`

You can see a demonstration of how to get started using Belly in this slick and polished screencast.

How To

If you can’t make out the details on the horribly blurry screencast, here’s the summary:

# install the gem:
gem install belly

# write belly's hook and config files into your project:
belly init

# run your features
cucumber features

# see your test results live on the internets. tada!

# see what's left to do
belly rerun

# tell cucumber to run what's left to do
cucumber `belly rerun`


I can’t stress how rough-and-ready this is, but I think it’s still useful enough to provoke you into giving me some feedback. Use it at your own risk, and let me know your thoughts.


Incredibly, it turns out that Joe Wilk, my old team-mate at Songkick and fellow Cucumber-core hacker, has been working on another solution to exactly the same problem. Living with a 3-hour build can be quite a motivator! I’m hoping Joe and I can figure out a way to combine our efforts into something really beautiful.

Published by Matt

I write software, and love learning how to do it even better.

Join the Conversation


  1. I tried to get round this problem by using manual tagging to run my tests in two batches. I would ensure my 30 minute build ran tests against all key features and recently fixed bugs – and each night my six hour suite would run so that I could see results in the morning.

    However, the effort required to keep updating and fixing the two test suites is almost becoming too much.

    I think tools like Belly and CukeMax are really going to help Cucumber turn the corner and start to alleviate the maintenance burden large test suites often entail.

  2. Hi Matt,

    How does one uninstall belly? I removed the gem but when I try to run my features, I get the following error:

    Using the default profile...
    no such file to load -- belly/for/cucumber (MissingSourceFile)
    C:/Test/Ruby/lib/ruby/siteruby/1.8/rubygems/customrequire.rb:31:in `gem_origin


  3. Hi Arti,

    Thanks for trying Belly. I’d love to hear any feedback you have.

    The belly init command installs a file inside your project in order to hook into belly and send your results during test runs. You’ve uninstalled the gem but that file is still there (and still looking for the gem).

    Try deleting the file features/support/belly.rb – that should be all you need to clean things up.

Leave a comment

Your email address will not be published. Required fields are marked *