BDD

Building software backwards

I am utterly dismayed by the number of so-called Agile teams I meet who are still, after all this time, building software backwards.

Toast

What do I mean by that? Let’s defer to the great W. Edwards Deming as he ridiculed the approach of 1970s American manufacturing to quality:

Let’s make toast the American way! I’ll burn it, you scrape it.

What an incredibly wasteful way to make things: Have one team cobble it together, have another team find all the mistakes, then send all the mistakes back to the first team to be fixed. Still, as long as we’re tracking it all with story points on our burn-down chart, and talking about it in our stand-up meetings, we must be Agile, right?

Wrong. A really Agile team would learn how to prevent those mistakes from happening in the first place.

Yet time-after-time, I come across scrum teams with a separate test team working an iteration behind the developers, finding all their mistakes.

Here’s why I think this is such a bad idea:

  1. The cost of fixing a defect increases exponentially with time. Defects that fester undetected in the code become harder and harder to remove safely. The longer the defective code is there, the more chance there is that other code will be built on top of it. When the original mistake is rectified, that other code may be broken by the change.

  2. The cheapest fix is the one you never had to make. When you write tests up-front, you often spot edge-cases as you write the tests; then you build in support for those edge-cases as you go. If you build software backwards, leaving a team of testers to discover those edge cases, you’ll pay a large penalty for having to go back and introduce the change to existing code.

  3. You discover requirements too late. When you practice test-driven development, you prepare for any change to the software by first defining how it should behave when you’re done. If you build software backwards, you may never discover the behaviour you really want until the testers get their hands on it and start asking all those awkward questions. The toast has already been burned.

  4. Continuous Delivery will always be a pipe-dream. Many companies, perhaps your competitors, deploy to production several times a day, just minutes after a developer checks in their change. If you have to wait for a separate team of testers to scrape all your burned toast, this will only ever be a pipe-dream for you.

When I talk to people about adopting BDD, the most frequent objection I hear is that it must take longer. This is true, in a sense, because it does take longer for a change to get to the stage where a developer is done with it. If you’re used to burning toast you’ll find this frustrating, because you don’t realise yet that the time and effort you’re putting in to write the tests up-front won’t pay off until later, when you hand the change to your testers and they can’t find anything wrong with it.

To do this takes a leap of faith, so hold your nerve. It will be worth it.


Stop building software backwards: come to BDD Kickstart

If you’d like to hear more about these ideas and learn concrete techniques to make them work in your organisation, I’m teaching a course from March 11-13 in Edinburgh, UK just for you. Click here to find out more.

Agile / Lean Software Development
BDD

Comments (0)

Permalink

The problem with solutions

First date. You’re out for dinner and the meal has gone well. Good food, great conversation. The waiter brings the bill, and you ask to pay with your credit card:

May I submit my billing info to your merchant account via your payment gateway?

Your date looks at you askance: nobody talks like that. Do they?

In any software project, there are two main domains: the problem domain and the solution domain. The problem domain is where your customers live. The programmers and other technicians working on solving that problem operate in the solution domain.

Both of those domains contain funny terminology, a dialect of English you can only learn by spending time amongst the people who speak it. This helps you to figure out which domain you’re in by listening to the language being used.

Here are some examples:

Solutions vs Problems

This specialised terminology is inevitable. Both business people and technicians need to use specialist language to communicate effectively with their peers. But what about when those two tribes need to speak to each other?

David West’s Object Thinking eloquently describes the natural divide between these two domains, and the problems it causes. In his view, our job as programmers is to build a model of the problem, and allow a solution to naturally fall out of that model. As soon as we start thinking in solutions, we lose sight of the problem and create ugly solutions. This implies that, as programmers, we need to immerse ourselves in the problem domain.

Using executable acceptance tests written in plain language helps you to keep your language and thinking rooted in the problem domain. This is why I have such an objection to Cucumber scenarios that talk about clicking buttons and filling in fields: they’re jumping into the solution domain too early.

Instead, I like to use Cucumber features as the place where we document our understanding of the problem domain. Writing those features collaboratively with people from both domains helps us to grow that understanding, and increase the overlap between the two domains.

BDD

Comments (3)

Permalink

TDD vs BDD

I regularly find myself explaining to people the difference between TDD (Test-Driven Development) and BDD (Behaviour-Driven Development). There still seems to be a lot of confusion over this, so I wanted to write this up for reference.

Late last year I was interviewed for a virtual panel on InfoQ along with Dan, Gojko, and Liz. Probably the most interesting part of that conversation covered the difference between TDD and BDD. Or rather the lack of any great difference.

We’ll start with some snippets from that discussion.

Both TDD and BDD include acceptance testing

One common misconception is that TDD is what you do when you’re unit-testing, and BDD is what you do when you’re writing customer-facing acceptance tests. Here’s Dan North on that point:

TDD – as originally described – is also about the behaviour of entire systems. Kent [Beck] specifically describes it as operating on multiple levels of abstraction, not just “down in the code”. BDD is equally important in this space, because describing the behaviour of systems is fractal: you can describe different granularities of behaviour from the entire application right down to individual small components, classes or functions.

Extreme Programming has always talked about writing acceptance tests, sometimes also called functional tests to describe what the customer expects to be done at the end of an iteration.

So this is nothing new. What’s new is how we explain it, and therefore how successful teams end up being in making it work for them.

BDD describes TDD done well

When Dan was working as a coach teaching TDD, he found that it was easier to get people to understand the principles of TDD if he stopped using the word ‘test’:

My experiences as a coach told me people were missing the point, with all this talk of unit tests, acceptance tests, functional tests, integration tests… Kent Beck’s style of TDD is a very smart way to develop software, so I tried removing the word “test” when I was coaching it, replacing it with things like behaviour, examples, scenarios etc. The result was very encouraging: People seemed to “get” TDD much quicker when I avoided referring to testing.

When Aslak and I wrote the Cucumber Book, I wrote this description of BDD:

BDD builds upon TDD by formalising the good habits of the best TDD practitioners.

That’s basically all there is to it. We want to re-explain TDD in a way that highlights the habits that successful TDD practitioners having been using for over a decade.

So what are those good habits?

Specifically, I think those good habits are:

  1. Working outside-in, starting from a business or organisational goal
  2. Using examples to clarify requirements
  3. Developing and using a ubiquitous language

Working outside-in seems obvious to habitual TDD practitioners, but many teams seem to limit themselves to doing this at the level of small units of code. Business-level black-box testing is still done manually, or automated as a check after the code has already been implemented.

This misses out of the major benefit of working outside-in, which is having the requirement challenged: if you need to explain to a computer how to check the requirement, you’ll need to be damn sure understand it yourself. If you don’t (and you often don’t) it’s much cheaper to find that out before you write the code.

Examples have always been a great way to make sure you really understand a requirement. What BDD does is formalise this by encouraging you to use scenarios to describe behaviour. These examples provide the perfect bridge between the business-facing and technology-facing sides of a team: they’re just formal enough that you can get a computer to check them, but anyone on the team can read them and make sure they’re describing behaviour that they actually want.

The GOOS Book, written by two of the best TDD practitioners in the business, frequently highlights the importance of domain language in our programs. In software teams, communication is probably the biggest overhead you have, and you make that communication a lot harder when you allow different dialects of terminology to be used by different parts of the team. Developing and then sticking to a consistent language takes deliberate effort, but it’s something that the best TDD practitioners have long learned will give them a significant advantage.

My experience is that BDD’s emphasis on collaboration, and the use of business-readable, executable specifications, means that this shared language develops much more quickly. When everyone is involved in writing documentation that describes what the system should do, they all get a chance to learn the language of the domain together.

So BDD really isn’t all that different to TDD. What BDD adds is a clear emphasis on what it takes to make TDD succeed.

BDD

Comments (5)

Permalink

Is Cucumber just a scam?

David Heinemeier Hansson recently wrote on his blog:

Don’t use Cucumber unless you live in the magic kingdom of non-programmers-writing-tests (and send me a bottle of fairy dust if you’re there!)

Well, good news readers! The magic kingdom is real! I’ve been there! Look, I even have a bottle of fairy dust. I keep it right next to the teabags:

Fairy Dust

Admittedly, that fairy dust is pretty good stuff, and I’ve been hitting it hard lately. Maybe it’s been making me hallucinate?

I decided I’d better check with some other people I know who’ve been to the magic kingdom too.

Lisa Clark told me a wonderful story about how her team use BDD:

I live in a magic kingdom and work in a castle (the Rackspace Castle). I’ve been using Cucumber on a RESTful web service project since project inception over a year ago… Currently our BDD sessions include the BA, QA, Dev Lead, and Developer. We will pull in the PO or architects as needed.

Here’s how Lisa’s team have benefitted from using BDD:

We’ve found value in the BDD documentation process and obtaining a shared understanding of what we’re building before it’s actually built. The added benefit of having executable requirements and an automated functional test suite that’s ready when the code is ready is icing.

Hear that readers? Lisa doesn’t see the primary benefit of Cucumber as being the testing, it’s the shared understanding that the team have built from writing the tests together. In Lisa’s magic kingdom, non-programmers don’t sit about writing tests on their own, they collaborate. And look the confidence that the whole team gets from that shared understanding:

The developers on my team have a strong level of confidence when delivering a story that all scenarios have been coded and are working as we said they should be. The QA knows up front exactly what we’re delivering with the story. I have confidence that regardless of which developer owned the story, all expected scenarios are coded and tested.

Of course there are other ways of getting that shared understanding and confidence. Working in small, cross-functional teams helps, and keeping the team together for a long time so that everyone becomes a domain expert is sadly under-appreciated by most big companies. Getting around a whiteboard or a set of design mock-ups to talk through a new feature is also invaluable, and some people find this is enough for them.

It really depends on the complexity of your domain: teams that work with complex, poorly understood business rules and requirements need all the tools they can get their hands on to manage that complexity. Many people I know have found this is the key benefit of using Cucumber: in a strange new domain, having a place to write down what you’re learning can really help you to stay sane.

I’ve never spoken to him about it, but my guess is that this is the reason DHH doesn’t see the need for Cucumber: he works on a team of good communicators who already have a wealth of domain expertise. In that context I might not use Cucumber either.

What I thought was most interesting about Lisa’s story was what happened when the team were under pressure and decided to throw off their BDD shackles:

We realized the value of our process when we bypassed it in order to quickly deliver a number of features for a high profile effort. We absorbed a new BA, QA team, and Devs that were unfamiliar with BDD, had tight timelines, and hoped to quickly knock out features. These features have had a higher number of defects, did not meet their delivery timelines, have a lack of automated testing (from both the developer and QA test fronts), and general hesitation from developers in touching this code in fear of breaking something. After our experience with these non-BDD implemented features the team (with the support of management) has committed to full BDD for all new features.

Higher defects, missed deadlines, hesitation from developers to touch the code in case they break something… Does that sound familiar to you?

It certainly doesn’t sound like much fun. Pass the fairy dust.

BDD

Comments (15)

Permalink

Interview on QCon: BDD, Cucumber and Hexagonal Architectures

I was interviewed recently by QCon’s Peter Bell about various subjects dear to my heart: BDD, Cucumber, Relish and hexagonal architectures.

Click here to watch the video

Agile / Lean Software Development
BDD
Hexagonal Rails
Relish

Comments (0)

Permalink

Public BDD Training (London, October)

So far all of my BDD training has been done in-house as private courses for companies. That’s about to change.

On October 8th – 11th Chris Parsons and I will be running four one-day training workshops in London focussed on BDD and specification by example. These are the same courses we’ve been successfully running in-house for companies like the BBC, Burberry, and Virgin Atlantic.

There are a range of workshops for everyone involved in software development:

  • experienced testers who want to automate their manual tests
  • product owners and business analysts who want to communicate their requirements more effectively
  • project managers who want less defects and more velocity
  • experienced programmers who want to become better testers
  • experienced Cucumber users who need help with advanced issues like brittle and slow tests

We’re offering a 20% discount to anyone booking before the 1st August. Just use the promotion code super-early-birdy when booking.

So, what are you waiting for?

Agile / Lean Software Development
BDD
Ruby Programming

Comments (1)

Permalink

Matt vs NYC

New York is one of my favourite cities in the world. I’m extremely fortunate to be able to spend a whole week there next week. I’ll be mostly hanging out at QCon:

I’m also going to run a couple of sessions at General Assembly on Thursday evening. If you can’t afford the time or ticket price for QCon, this might be a good option to come along and hear some of my ideas:

  • At 6pm on Thursday I’ll be doing an intro-level talk on BDD. It’s aimed at business people, development managers etc who want to get an idea of the benefits of specification by example and a wider BDD process. Sign up here.
  • at 8pm on Thursday I’ll be doing my ‘BDD as if you meant it’ demonstration, using the example from the middle section of The Cucumber Book. If you’re already using Cucumber, this should give you some fresh perspective and insights about how to use it most effectively. Sign up here.

Finally, I’m honoured to be spending Saturday on the bill with some amazing people at the Gotham Ruby Conference. I’m going to be speaking about hexagonal rails. I’m excited and nervous about this talk, as I feel like I’m still discovering so much about this topic all the time.

As well as all this geekery, I’d love to get some proper culture while I’m in town. Are there any locals who would like to show me around?

Agile / Lean Software Development
BDD
Hexagonal Rails
Ruby Programming

Comments (1)

Permalink

A vision for Cucumber 2.0

At last month’s CukeUp conference, I held a panel discussion between Aslak Hellesoy, Julien Biezemans, Oriol Gual and Jonas Nicklas. I chose these panelists because each of them has written a variation on the original Ruby Cucumber, and I wanted to try to pull these ideas together into a vision for Cucumber 2.0.

This post is a record of that conversation.

Why do we need a new Cucumber?

The current version of Cucumber has been around since 2008, and as Jonas has previously explained the code isn’t much fun to hack on. It’s quite bloated with features, some of which were added as experiments and maybe aren’t used by most users. Carrying this cruft around is holding us back from adding new features that we do want.

Also, the current mechanism for mapping from steps to Ruby code is pretty basic, and doesn’t really encourage the good practices we’ve learned as a community about organising your test code.

What is your vision?

I asked each of the participants what they’d learned from their fork of Cucumber, and what they’d like to see in a new version of Cucumber.

Aslak made three points:

  • It should be possible to use Cucumber as a library, so that other tools can be built on top of it as well as the command-line interface.
  • The way steps are defined in Cucumber-JVM, by simply annotating existing methods, allows you to focus on keeping your test code well organised. We should steal this idea for Cucumber-Ruby.
  • There should be a collaborative web UI for writing Cucumber scenarios with non-technical stakeholders.

Julien added:

  • Integrating the Cucumber-HTML project, which is essentially a web-based client for Cucumber, would be awesome.
  • It would be nice to use the Cucumber-TCK, which is growing into a standard set of specs for any Cucumber implementation.
  • He hasn’t implemented calling steps from step defs in Cucumber-JS, and he’s not sure he wants to.
  • Parallel execution – can we make this possible?

Oriol echoed Aslak’s thoughts about keeping test code better organised: Spinach uses one class per feature, and is generally more object-oriented unlike Cucumber-Ruby’s flat anonymous blocks for step definitions.

Jonas said his goal for Turnip was mainly simplification:

  • He thinks that the reason some people had started to abandon Cucumber was to do with it being perceived as complex.
  • Jumping from plain english steps to Regexps is jarring, and too big a leap in abstraction. 98% of the time you don’t need regular expressions anyway. Turnip uses the concept of placeholders in strings instead to capture arguments.
  • Using RSpec as the runner for Turnip hasn’t worked out so well.

What features should we remove?

It’s always fun making a ‘not’ list, so we brainstormed the current Cucumber features we’d like to kill in a 2.0 release:

Cucumber: Features to Kill

In case you can’t read my scrawl, here’s that list again:

  • Wire protocol
  • Calling steps from step defs
  • Tables (we’re talking about scenario outlines here)
  • DRB / Spork
  • World having everything mixed in, all the time
  • Cucumber.yml
  • Given / When / Then being regarded as the same by Cucumber

As you can see, we got the audience to come up and vote after the session. It looks like nested steps are living on borrowed time!

My view is that with a well-designed core, many of these features can still exist but as plug-ins, thereby allowing people who need them to continue to use them, but without the maintenance burden on the core team.

So what’s next?

Well, I think I have a pretty clear vision for what Cucumber 2.0 would look like. I’d like to start with a simple core library that has no GUI component at all, but can pass the TCK tests. Then we can start building tools around that core, like a console runner, and a websocket server for running cukes from a javascript browser app. I’m personally really excited about where that last idea could lead.

What I’m short on is time. Would it be crazy to ask if there are any companies who would consider sponsoring this project? I can’t imagine a better way to spend the summer than hacking out a brand new shiny Cucumber!

Update: Thanks for the comments, but I don’t find they make a great medium for discussion so I don’t tend to reply to them. If you’d like to discuss this post, please chip in on these threads on the cukes google group.

Agile / Lean Software Development
BDD
Relish
Ruby Programming

Comments (16)

Permalink

Interviewed by RubySource

Last week I had a great chat with Pat Shaughnessy about The Cucumber Book which has been published as an interview on RubySource. Pat really managed to get to the heart of my opinions about how to use Cucumber effectively, so if you’re too busy to read the book, this will give you a good summary of my latest thinking on topics such as:

  • - why Cucumber is pointless unless you involve the whole team
  • - imperative vs declarative style, and why it matters
  • - testing domain objects directly from Cucumber
If you’re using Cucumber or SpecFlow day to day, or considering them and wondering what benefits you’ll get, I think this interview is worth a read.

Agile / Lean Software Development
BDD
Relish

Comments (1)

Permalink

Free BDD Coaching

In this blog post, I will reveal a secret to you. A secret that can get you access to free, hands-on BDD coaching from some of the world’s leading experts like Aslak Hellsøy, Mike Sassak, Greg Hnatiuk, Joseph Wilk and even me, Matt Wynne. Yes, that’s right: absolutely FREE.

Would you like to know how?

I’ll give you step-by-step instructions.

  1. Point your web browser to the Cucumber issues list.

  2. Have a look around. Take your time. Look for issues that don’t already have Code Attached, because we’re looking for an issue that nobody has tried to solve yet. Found an interesting one? Good.

  3. Fork the Cucumber repository to your Github account.

  4. Now comes the hard part. You need to write a failing test for the issue you found. Start by writing a scenario that describes the way the code should behave once the issue has been fixed. Try to not to just create a file called ‘features/fixissue234.feature‘ but instead try to slot your scenario into the existing features as extra detail about that particular aspect of the system. You might need to create a new feature file, but try to name it according to the behaviour it describes, rather than the bug it tests. Look at the other features for examples of how to write your scenario. Try to use a declarative style where possible.

  5. With your scenario written, it’s time to submit a pull request.

  6. This is where your FREE BDD coaching will begin. One of the Cucumber team will see your pull request, and will be delighted to offer you feedback on your scenario. You don’t have to wait for the feedback though, you can go ahead and start to get your scenario running.

  7. Keep pushing. Every time you make progress, getting the steps of your scenario to actually run against Cucumber, push to your fork and the pull request will be updated automatically by Github’s legions of octocats. Your FREE BDD coach will keep encouraging you with comments and feedback. Now comes the hard part.

  8. Once you have the scenario running, you’ve got the bug trapped: it’s time to fix it! Again, your FREE BDD coach will be in your corner. Ask questions about the design of the existing cucumber code, about your code, write specs to isolate behaviour further, and let your FREE BDD coach guide you. Before you know it you’ll have fixed the issue, and your name will be emblazoned in the codebase forever more.

This is a great way to get access to some very experienced and friendly software developers and feel good about giving something back to the community. Many wise software craftspeople have already discovered this secret. Isn’t it time you joined them?

And did I mention that it’s FREE?

Agile / Lean Software Development
BDD
Ruby Programming

Comments (0)

Permalink