Is Cucumber just a scam?

David Heinemeier Hansson recently wrote on his blog:

Don’t use Cucumber unless you live in the magic kingdom of non-programmers-writing-tests (and send me a bottle of fairy dust if you’re there!)

Well, good news readers! The magic kingdom is real! I’ve been there! Look, I even have a bottle of fairy dust. I keep it right next to the teabags:

Fairy Dust

Admittedly, that fairy dust is pretty good stuff, and I’ve been hitting it hard lately. Maybe it’s been making me hallucinate?

I decided I’d better check with some other people I know who’ve been to the magic kingdom too.

Lisa Clark told me a wonderful story about how her team use BDD:

I live in a magic kingdom and work in a castle (the Rackspace Castle). I’ve been using Cucumber on a RESTful web service project since project inception over a year ago… Currently our BDD sessions include the BA, QA, Dev Lead, and Developer. We will pull in the PO or architects as needed.

Here’s how Lisa’s team have benefitted from using BDD:

We’ve found value in the BDD documentation process and obtaining a shared understanding of what we’re building before it’s actually built. The added benefit of having executable requirements and an automated functional test suite that’s ready when the code is ready is icing.

Hear that readers? Lisa doesn’t see the primary benefit of Cucumber as being the testing, it’s the shared understanding that the team have built from writing the tests together. In Lisa’s magic kingdom, non-programmers don’t sit about writing tests on their own, they collaborate. And look the confidence that the whole team gets from that shared understanding:

The developers on my team have a strong level of confidence when delivering a story that all scenarios have been coded and are working as we said they should be. The QA knows up front exactly what we’re delivering with the story. I have confidence that regardless of which developer owned the story, all expected scenarios are coded and tested.

Of course there are other ways of getting that shared understanding and confidence. Working in small, cross-functional teams helps, and keeping the team together for a long time so that everyone becomes a domain expert is sadly under-appreciated by most big companies. Getting around a whiteboard or a set of design mock-ups to talk through a new feature is also invaluable, and some people find this is enough for them.

It really depends on the complexity of your domain: teams that work with complex, poorly understood business rules and requirements need all the tools they can get their hands on to manage that complexity. Many people I know have found this is the key benefit of using Cucumber: in a strange new domain, having a place to write down what you’re learning can really help you to stay sane.

I’ve never spoken to him about it, but my guess is that this is the reason DHH doesn’t see the need for Cucumber: he works on a team of good communicators who already have a wealth of domain expertise. In that context I might not use Cucumber either.

What I thought was most interesting about Lisa’s story was what happened when the team were under pressure and decided to throw off their BDD shackles:

We realized the value of our process when we bypassed it in order to quickly deliver a number of features for a high profile effort. We absorbed a new BA, QA team, and Devs that were unfamiliar with BDD, had tight timelines, and hoped to quickly knock out features. These features have had a higher number of defects, did not meet their delivery timelines, have a lack of automated testing (from both the developer and QA test fronts), and general hesitation from developers in touching this code in fear of breaking something. After our experience with these non-BDD implemented features the team (with the support of management) has committed to full BDD for all new features.

Higher defects, missed deadlines, hesitation from developers to touch the code in case they break something… Does that sound familiar to you?

It certainly doesn’t sound like much fun. Pass the fairy dust.