Skillsmatter BDD Exchange

Last week I travelled down to London to the BDD Exchange conference. It was a one-day conference organised by Gojko Adzic and I had a great time. I missed Gojko’s talk as I travelled down from my cave in Scotland on the day, but I did arrive in time to see Chris Matt’s excellent lecture on what business analysis really should be about.

I particularly enjoyed the talk from Christian Hassa about teams failing to make BDD work. We can learn the most from failure, and Christian’s thoughtful analysis of what he’s observed in the field as a consultant with TechTalk is useful to any team trying to get the most from these techniques. The message of Christian’s talk very much echoed my own, that the tooling you use is entirely secondary to the collaborative relationship you need to build between the business and technical-facing members of the team. I was interested to learn about the tool, SpecLog, TechTalk are building to help teams with this problem, which seems to have many similar goals to my own Relish. It was nice of Christian to give Relish a name-check in his talk.

My session ran along the same theme as my talk from earlier in the year at Skillsmatter, describing the value of writing acceptance tests at the right level of abstraction, so that they describe business rules rather than implementation details. You can watch the session here.

Agile / Lean Software Development

Comments (2)


Relish Roadmap

I want to give you some news about the future of Relish. A lot has happened since we first started the project just over a year ago. Justin quit the project to concentrate on his new role on the RSpec core team, The Cucumber Book went into beta, and my wife gave birth to our second child. All of those things have meant that Relish hasn’t been able to progress as quickly as I’d have ideally liked, so thanks for sticking with us!

I’m still passionate about the vision for Relish, and as I teach training courses on using Cucumber and BDD, it seems to resonate with a lot of people. With The Cucumber Book pretty much behind us now, Relish will come back into focus and the pace will pick up again. I hope to have it launched by the end of the first quarter of 2012.

Here’s a rough outline of what we plan to do:

Usability & Information Architecture Enhancements

I have an IA specialist on the team now, and you may have noticed the first tweaks we’ve been making this week. There’s a lot to catch up, so we’ll focus on this until the existing features are polished to his (and hopefully your) satisfaction. Please let us know how we’re doing as we progress, and let us know if any of the changes annoy or delight you.

If there’s anything specific you’d like to see done that isn’t already logged in UserVoice, please add it there or reply to this post.

Test Results

Relish can’t deliver on the promise of living documentation until the non-technical readers can tell, when they’re looking at a scenario, whether it’s passing or not. We’ll be building a plugin for Cucumber that allows developers and testers to send their test results to Relish, so each scenario can be rendered with a big green tick if it’s passing. This will also enable us to start building all kinds of exciting dashboards for project managers to get a high-level overview of what’s going on.

Commenting / Feedback

Just being able to read features is not enough. I want Relish to be a collaboration tool, and that means that stakeholders should be able to comment on and give feedback about the documentation they’re reading.

RSS / Activity Feeds

People who want to stay up to date with changes to a project will be able to get a nice high-level summary via RSS, and possibly other means too.

Enterprise Install

As the codebase stabilises, I’m becoming increasingly comfortable with the idea of a stand-alone installer for customers who are not happy about having their features stored outside of their firewall. In the new year, I’ll be looking for a couple of friendly enterprise customers to help me shape this. If you’re interested, please get in touch.

Plans & Pricing

Yes, it had to happen eventually :)

When we come out of private beta (which is still a few months away), it will be with paid accounts for private projects. This is a hard call to make, and I’ll be in working with our beta tester community to get an idea of what they feel is a reasonable price to pay for the service. Public accounts like RSpec and VCR will continue to be free.

As always, I’m keen to hear what you think of these plans. Let me know in the comments.

Agile / Lean Software Development

Comments (0)


BDD Training

Update: This training is now available as a public course, starting October 8th in London.

Would you like to learn how Behaviour-Driven Development can help your company get better at software development?

I’ve helped several teams learn BDD, and I’ve started to formalise the training I’ve been doing into a set of course modules. The modules aim to provide the foundations for a teamʼs successful adoption of BDD.

We start by immersing the whole team in BDD for a day to get everyone enthusiastic about the process. Then I take the programmers and testers and implement their very first scenario, end-to-end, on their own code. Now that we’ve proved it can be done, I work with project managers, product owners, and development leads, to streamline their agile process to get the best from BDD. We practice collaborative scenario-writing sessions, we learn how to use metrics to track progress, and how Kanban and BDD can fit into your existing agile process.

Please take a look at the course prospectus and get in touch to see how I can help.

Agile / Lean Software Development

Comments (11)


Fixing my testing workflow

Okay I’m bored of this. I need to talk about it.

I love to use Ruby, RSpec, Cucumber and Rails to do test-driven development, but my tools for running tests are just infuriatingly dumb. Here’s what I want:

  • When a test fails, it should be kept on a list until it has been seen to pass
  • When more than one test fails:
    • Show me the list, let me choose one
    • Focus on that one until it passes, or I ask to go ‘back up’ to the list
    • When it passes, go back up to the list and let me choose again
    • When the list is empty, I get a free biscuit
  • When a test case is run, a mapping should be stored to the source files that were covered as it ran so that:
    • When a file changes, I can use that mapping to guess which test cases to run. Fuck all this naming convention stuff, it’s full of holes.
    • At any time, I can pipe the git diff though the tool to figure out which test cases to run to cover the entire commit I’m about to make.

When I say test case, I personally mean:

  • An RSpec example
  • A Cucumber scenario

…but it should work for any other testing framework too.

I feel like having a tool like this that I trusted would make a huge difference to me. There are all these various scrappy little pieces of the puzzle around: guard plugins, autotest, cucover, cucumber’s rerun formatter. None of them seem to quite do it, for me. Am I missing something?

Or shall we make one?

Agile / Lean Software Development
Ruby Programming

Comments (8)


Cucumber: Why Bother?

It’s perfectly possible to write automated acceptance tests without using Cucumber. You can just write them in pure Ruby. Take this test for withdrawing cash from an ATM:

Scenario: Attempt withdrawal using stolen card 
  Given I have $100 in my account 
  But my card is invalid
  When I request $50
  Then my card should not be returned 
  And I should be told to contact the bank

We could automate that test using good old Test::Unit, perhaps something like this:

require 'test/unit'
class WithdrawlTests < Test::Unit::TestCase
  def test_attempt_widthrawl_using_stolen_card
    bank =
    account =
    card =
    atm =
    assert atm.card_withheld?, "Expected the card to be withheld by the ATM"
    assert_equal "Please contact the bank.", atm.message_on_screen

The big disadvantage of writing acceptance tests in pure Ruby like this is that it’s unlikely you’ll be able to show this test to your team’s analyst without their eyes glazing over.


Unless your analyst is, or has recently been, a programmer themselves, they won’t be able to see past the noise of Ruby’s syntax, clean as it may be, to understand the actual behaviour that’s being specified. The specification of behaviour and the implementation of the test are all mixed up together, and that’s a problem if we want to get feedback from our stakeholders about whether we’ve specified the right thing before we go ahead and build it.

If we want the benefits of using plain language to write our behaviour specification, then we need a way to translate that into automation code that actually pulls and pokes at our application. Step definitions give you a translation layer between the plain-language specification of behviour and the test automation code, mapping the Gherkin steps of each scenario to Ruby code that Cucumber can execute.

The cost of this extra layer is complexity: Yes, you have more test code to maintain than you would if you stuck to writing your tests in pure Ruby. The benefit is clarity: by separating the what (the features) from the how (the ruby automation code), you keep each part simpler and easier for its target audience to understand.

Agile / Lean Software Development

Comments (11)



I’m going to be speaking at CukeUp!, Cucumber’s very own one-day conference in London on March 24th 2011. It’s going to be a great little conference, I’m really looking forward to hearing talks from people like Gojko Adzic, Dan North, Liz Keough, Capybara’s creator Jonas Nicklas, Joseph Wilk, Chris Matts, Antony Marcano and of course Aslak Hellesoy.

I know it’s only a couple of weeks away, but if you’re in or around the UK and interested or curious about ATDD / BDD, get yourself there: it’s going to be fun.

Agile / Lean Software Development

Comments (1)


Using Capybara with RSpec Outside Cucumber

If you want to try using Capybara for browser automation on it’s own, here’s a simple script to get you started:

require 'rubygems'
require 'capybara'
require 'capybara/dsl'
Capybara.default_driver = :selenium
Capybara.app_host = ""
require "rspec/expectations"
class Google
  include Capybara
  include RSpec::Matchers
  def search_for(text)
    visit "/"
    fill_in "q", :with =&gt; text
    click_button "Search"
  def ensure_results_contain(expected_text)
    page.should have_content(expected_text)
google =
google.search_for("Matt Wynne")

To make this work you’ll need to install the capybara and rspec Ruby gems:

gem install capybara rspec

Agile / Lean Software Development
Ruby Programming

Comments (2)


Features != User Stories

User Stories are a great way to plan your work. You can take a big hairy requirement and break it down into chunks that are small enough to work on without anyone freaking out. When you’ve crumbled up your big hairy requirement into little user story chunks, you can pick and choose which chunk to build first, and even drop some chunks altogether when you realise they’re not that important. Great stuff.

So we like user stories. They’re our friends. Here’s one:

In order to pay for the items in my shopping cart,
As a customer,
I want a credit card checkout

Now, suppose when we do the analysis for this story we discover that certain types of credit cards are much easier to hook up to a checkout than others. We want to ship something as soon as possible, so that people can start giving us money. So we split the story in two.

One for the friendly FooCorp credit card:

In order to pay for the items in my shopping cart,
As a customer with a FooCorp credit card,
I want a credit card checkout that accepts my card

And another for the altogether-much-harder-to-integrate-with BarCorp credit card:

In order to pay for the items in my shopping cart,
As a customer with a BarCorp credit card,
I want a credit card checkout that accepts my card

As everyone knows, BarCorp have some really outdated systems, and connecting to their payment processing is a right old pain. We’ll leave that for later.

So let’s get on with implementing our first story. We’ve done some analysis on it, but let’s make sure we understand the acceptance criteria. We have a little sit-down with our product owner, and come up with this list:

  • when a user attempts to pay with a BarCorp card, we show them a “Sorry” message
  • when a user pays with a FooCorp card, we send them an email with their receipt
  • when a user attempts to pay with a FooCorp card but gets their details wrong, we bounce the page and show them an error message, so they can correct their mistake.
  • when a user attempts to pay with a FooCorp card but the card is declined, we bounce the page and show them an error message, so they can try another card.

So those are the acceptance criteria for this story. We didn’t do anything fancy with them like put them in a Microsoft Word® document or a Jira ticket. We just wrote them down on a piece of paper.

Now we’ll use Cucumber to document these acceptance criteria as Scenarios that we can use to drive out the behaviour we need to get this story done.

Let’s start by creating a new feature:

mate features/pay_with_foocorp_card.feature


Whoops. You made a mistake. Did you spot it?

That’s right, you went and got confused and thought that the user story you were implementing now was the same thing as the feature. It’s not.

Right now, this distinction probably seems a bit nit-picky. But wait until you’ve got around to implementing BarCorp’s credit card check-out, and BazCorp’s too. Your features directory is going to look like this:


Can you see where this is going? Give it a few months, and your features directory is going to look like a history log of the development of your project. Hey, while you’re at it, why not put the jira ticket in the filename, or even use a tool to synchronise the feature files with Pivotal Tracker?

I see lots of people doing this, and trust me, it’s an anti-pattern.

Before there was Cucumber, there was the RSpec Story Runner. There was also this blog post. That was a long time ago. When Aslak created Cucumber, he renamed the files from .story to .feature. This wasn’t an accident or an idle act of whimsy: it’s because there’s a difference.

User Stories are a planning tool. They exist until they’re implemented, and then they disappear, absorbed into the code.

Cucumber features are a communication tool. They describe how the system behaves today, so that if you need to check how it works, you don’t need to read code or go punching buttons on the live system. Organising your features according to the way they were planned and implemented is a distraction from this purpose.

Imagine if the user manual for your washing machine was organised by how the washing machine had been constructed? A nonsense.

Going back to our example, let’s rename our feature:

mv features/pay_with_foocorp_card.feature features/pay_with_credit_card.feature

Now that’s more like it. As we implement the first User Story for FooCorp credit cards, we’ll put some scenarios into this feature. When we come to implement BarCorp support, we can add a couple more scenarios to the same feature. We might even have to modify existing scenarios, perhaps if we add a new widget to the UI to let the user choose their credit card type.

The User Story is absorbed into our features and becomes invisible, leaving the features as a live document of what the software does now, not as a record of how it was constructed.

Thanks to Paul Wilson for reminding me to write this post.

Agile / Lean Software Development

Comments (12)


Outside-In vs Inside Out – Comparing TDD Approaches

At last month’s ScotRUG Brian Swan and I attempted to solve the TDD Avatars problem as a live recital in our chosen style. We each had 35 minutes.

The videos are here:

Brian’s Inside-Out TDD approach

Matt’s Outside-In approach

When Brian had walked us through his approach and solution at the last month’s meeting, he’d built his solution as a Rails application, with web forms for filling out bookings and viewing receipts and so on.

When I came to start practicing and converted the use case from the TDD Avatars paper into a Cucumber feature, it quickly became clear that the value of the system I was building, at least as described by the use case, was to provide printed receipts to customers. I then started to think about the simplest way I could build a system to provide that value.

Here’s the feature I wrote:

Feature: Pay bill
  Background: Prices
    Given the following operations are available:
      | operation        | price |
      | routine check up | 10    |
      | shots            | 5     |
  Scenario: Dave Pays for Fluffy
    Given there is an owner Dave Atkins, let's call him "Dave"
    And Dave brings his pet named Fluffy into the clinic for the following operations:
      | routine check up |
      | shots            |
    When the veterinarian charges him for the visit
    And Dave pays cash
    Then Dave is given a receipt which looks like this:
        $10 (routine check up)
        $5 (shots)
      Total to pay: $15
      Paid cash, received with thanks

Notice that the scenario doesn’t talk about clicking particular buttons or filling in boxes on a form? I’ve used a higher-level declarative style to describe the behaviour I want. In my experience this helps in various ways:

  • more human-readable features
  • features that aren’t coupled to a particular user interface

If you watch the video, you’ll see that the first thing I did, working my way in from the step definitions, was to create a custom step definition DSL for my problem domain. Instead of using a generic DSL like Capybara’s fill_in, click_button etc, I created this one:

module VetsHelper
  def register_operation_price(operation, price)
  def remember_owner(name, nickname)
  def create_visit(owner_nickname, pet_name, operations)
  def charge_for_visit
  def pay_with(payment_type, nickname)
  def receipt

This is arguably unnecessary: my step definitions are already translating from English into Ruby, so why add this extra layer of indirection?

As I worked my way from the outside (the features) into the step definitions, I wasn’t ready to commit myself to how I was going to couple the tests to my new application. By defining this interface, I’ve deferred that commitment a little later. I’ve also given myself a clean view of all the behaviour the new application needs to support.

My first iteration implementation (the one in the video) of VetsHelper drives out a domain model directly from the methods in that module. If that was what we released to our user, they’d only be able to print receipts if they knew how to use an IRB prompt. That might seem ridiculous, but we’ve gone a long way to solving the problem, and we could probably spike a simple script that let them do it from the command-line without much risk.

For our second iteration, we can talk to the customer about that command-line interface, then write a new implementation of VetsHelper, perhaps using some of Aruba’s DSL, which goes through that command-line interface instead of directly to the model. This is the beauty of using a declarative style together with your own domain-specific step definition DSL: it gives you the flexibility to swap in connections to the system that hit it at different levels, using exactly the same acceptance tests.

Did BDD Save Me Time?

When Brian and I were planning this month’s session, I showed him the code I’d written and he decided to do a comparable solution this time, without any UI, so that they were easy to compare. In fact, Brian’s solution looked much simpler, and was certainly quicker to write, because he didn’t have to spend any time writing the acceptance testing layers and he didn’t write any kind of entry-point Practice class. He just went straight into building the Appointment class.

A big difference between the solution we produced this month and the one that Brian had originally built was that we didn’t use Rails, and instead went for a much simpler solution that still provided some immediate value. I like to think that the idea for doing this came from the BDD approach I took—I’m pretty sure I remember the lightbulb going on as I typed out the feature—but we’ll never know now where this idea originated.

I noticed that Brian spent time testing getters on his classes, which I probably wouldn’t have done. I tend to try to avoid using them, except on value object, and I rarely test the behaviour of value objects. I rely on my acceptance tests to tell me if they’re not working.

Focus and Design

Brian’s big take-away was that the difference in our approaches when we needed a collaborator object. When I needed a collaborator for a class, I would just mock out the collaborator and carry on finishing off the class I was building, whereas he would leave the current class broken and go and build the other class first.

I find my (mock-based) approach gives me focus, and also means I can sketch out the design of the collaborator without having to commit myself to that design until I understand how it’s going to be used.

I’m really happy with the design I ended up with. It’s hard to make much of a judgement in such a simple problem, but I’d be interested to hear your thoughts on how the two designs compare. Which one would you have preferred to add a new feature to?

Agile / Lean Software Development

Comments (0)


Fix RubyMine 2.02 Cucumber Integration

If you’re using the latest version of RubyMine (2.0.2) with the latest version of Cucumber (actually anything above 0.7), you’ll probably see this ugly warning when you try to run your cukes from within the IDE:


The bug has been logged, and there’s a published workaround, but I wanted something a bit easier to use.

Try this instead. Close RubyMine, open a terminal, and run this command:

curl | patch -p0

or even

curl -L | patch -p0

It won’t work with Cucumbers older than 0.7, but why would you want to use them?

Update: If you like life on the bleeding edge, you can also try the EAP release (latest development build) of the forthcoming RubyMine 2.5, which contains this fix.

Ruby Programming

Comments (0)