Last week I had a great chat with Pat Shaughnessy about The Cucumber BookÂ which has been published as an interview on RubySource. Pat really managed to get to the heart of my opinions about how to use Cucumber effectively, so if you’re too busy to read the book, this will give you a good summary of my latest thinking on topics such as:
– why Cucumber is pointless unless you involve the whole team
– imperative vs declarative style, and why it matters
– testing domain objects directly from Cucumber
If you’re using Cucumber or SpecFlow day to day, or considering them and wondering what benefits you’ll get, I think this interview is worth a read.
A few months ago, development of http://relishapp.comÂ reached what is a familiar but rather unpleasant plateau for me. Up until that point, it had been developed using conventional Ruby on Rails idioms, with controllers that talk directly to ActiveRecord class methods to make queries, then set instance variables to pass the results of those queries to view templates for rendering. It’s all very procedural, and very tightly coupled.
This post is an introduction to a forthcoming series which will explain how I’ve been tackling this.
Let me start by saying that the coupling in my Rails applicationÂ is not necessarily a bad thing. In fact, to get the product off the ground and give RSpec’s amazing living documentation a place to call home, we needed to be able to put Relish together very quickly. This is where Rails absolutely dazzles: the early stages of a project are so easy that adding features is a delight.
Unfortunately, the very forces that make adding features to a new Rails application so easy are the same ones that start to hold you back as the number of features grows. That conventional Rails style is what Kent Beck has called a connected design style, Quoting from that article:
In a connected system, elements are highly available to each other (via global state, for example). Adding the first feature to a connected system is cheap. All the resources you need are available. However, the cost of all those connections is that subsequent features are very likely to interact with previous features, driving up the cost of development over time.
The alternative to a connected design style is what Kent calls a modular design style.Â He contrasts these using this chart:
For the first few features, a connected design style is cheaper than a modular design style. In fact, using what might be referred to by software design snobs as ‘poor design’ could give you a competitive advantage in the early stages of your product’s development. The trick is to recognise when it’s time to switch styles.
So, while I love Rails for what it is, I also recognise its weaknesses. To make the Relish codebase habitable in the long term, I need to move beyond its connected design style and establish a more modular design. But how? It’s a question that many different people are asking at the moment, and I’ve been diving deep into what they are saying. Here’s a sample:
Gary Bernhardt and Corey Haines have been agitating with screencasts and talks that highlight the advantages of decoupling from Rails.
I think it’s easier to use an architecture like this in Java or C#, where packages and interfaces make it easy to see and enforce separation between chunks of the application, than in Ruby where it’s harder to make clear boundaries around areas of the system. But that doesn’t mean it isn’t possible. For the past couple of months I have been experimenting with refactorings in the Relish codebase to move it towards a more hexagonal architectural style. The overall goal is to make the Relish codebase habitable, which for me breaks down into:
1. Fast tests
I never add features without tests, and I want those tests to run fast. The goal of the hexagonal architecture is to have the most interesting (and frequently changing) logic implemented in plain old Ruby objects that have no dependency on any other libraries so they are easy to understand, and can be quickly spun up in a test.
2. Modularity and encapsulation
Ruby is an object-oriented language with great support for functional programming, and I want to make the most of that to keep Relish’s code easy to change.
3. Clean and well-organised
I want a structure that communicates what each part of the system is doing, and makes it easy for new team members to jump in and start hacking on the code.
I’ll start the series by explaining a couple of key concepts from Steve and Nat’s GOOS book, and about hexagonal architectures in general. Then we’ll get down to some practical work refactoring a typical Rails controller. Stay tuned!
In this blog post, I will reveal a secret to you. A secret that can get you access to free, hands-on BDD coaching from some of the world’s leading experts like Aslak HellsÃ¸y, Mike Sassak, Greg Hnatiuk, Joseph Wilk and even me, Matt Wynne. Yes, that’s right: absolutely FREE.
Have a look around. Take your time. Look for issues that don’t already have Code Attached,Â because we’re looking for an issue that nobody has tried to solve yet. Found an interesting one? Good.
Fork the Cucumber repository to your Github account.
Now comes the hard part. You need to write a failing test for the issue you found. Start by writing a scenario that describes the way the code should behave once the issue has been fixed. Try to not to just create a file called ‘features/fix_issue_234.feature‘ but instead try to slot your scenario into the existing features as extra detail about that particular aspect of the system. You might need to create a new feature file, but try to name it according to the behaviour it describes, rather than the bug it tests. Look at the other features for examples of how to write your scenario. Try to use a declarative style where possible.
With your scenario written, it’s time to submit a pull request.
This is where your FREE BDD coaching will begin. One of the Cucumber team will see your pull request, and will be delighted to offer you feedback on your scenario. You don’t have to wait for the feedback though, you can go ahead and start to get your scenario running.
Keep pushing. Every time you make progress, getting the steps of your scenario to actually run against Cucumber, push to your fork and the pull request will be updated automatically by Github’s legions of octocats. Your FREE BDD coach will keep encouraging you with comments and feedback. Now comes the hard part.
Once you have the scenario running, you’ve got the bug trapped: it’s time to fix it! Again, your FREE BDD coach will be in your corner. Ask questions about the design of the existing cucumber code, about your code, write specs to isolate behaviour further, and let your FREE BDD coach guide you. Before you know it you’ll have fixed the issue, and your name will be emblazoned in the codebase forever more.
This is a great way to get access to some very experienced and friendly software developers and feel good about giving something back to the community. Many wise software craftspeople have already discovered this secret. Isn’t it time you joined them?
I sometimes get asked whether it’s possible to use Cucumber to test performance. The way to do it is to specify concrete examples of scenarios that the system will find itself in under stress. For example:
Given there are 100,000 users registered on the system
When I create a new account
Then I should be taken to my dashboard within 5ms
Given 1000 users are hitting the homepage simultaneously
Then each user should get a response within 2ms
Talking through these kinds of scenarios with your stakeholders will help you to understand where the boundary is for what they consider to be acceptable performance. You might find it hard to get them to be this specific at first, but help them understand that what you’re doing is drawing a line in the sand for the minimum acceptable performance – most of the time the application may be much faster than this. Now you have agreement about this, the next step is to work out how to automate these scenarios, by calling your load testing tool of choice from the Cucumber step definitions.
The key thing is to have Cucumber delegate to the stress testing tool, rather than the other way around. A common mistake people make is to simply point JMeter at existing Cucumber scenarios, but this doesn’t give you the benefit of having the parameters of the performance test documented in readable Cucumber scenarios.
These are not normal kinds of Cucumber tests. You would need to run these against an environment that’s representative of your production hardware, whereas normal behaviour scenarios could be run against any environment. It’s useful to create these scenarios early on during the project and run them against every build so they become constraints that every build must meet as the project progresses.
One convention I use is that query methods are idempotent and return a
value, while command methods change state, but they can only return
self. This provides a nice fluent interface when working with the
objects, and feels quite natural (to me at least).
We have an old Kenwood Chef A701 that’s hardly out of use in our kitchen. A few days ago, the gearbox went. I bought a reconditioned one on Ebay for Â£23, and this morning I made the time to do the repair.
As I tinkered away with my screwdriver, I reflected on how grateful I was that the designers of this machine had made sure it could be maintained by an idiot like me. I actually made our A701 by cannibalising two broken machines, and I have always been delighted by how easy it is to work on.
Perhaps it’s my confirmation bias, but it seems that machines like this no longer tend to be designed with maintenance in mind. In our consumer society, we’re encouraged to throw things away rather than repair them, and globalisation ensures that is often the apparently rational economic decision to take: paying a local repair man to fix your DVD player will cost more than popping down to Tesco and buying a new one. Thank goodness for those cheap Chinese factories, eh?
If we don’t design our manufactured goods for maintainability, perhaps it’s no wonder we often feel like we’re swimming against the tide trying to persuade people that it’s important in software.
I particularly enjoyed the talk from Christian Hassa about teams failing to make BDD work. We can learn the most from failure, and Christian’s thoughtful analysis of what he’s observed in the field as a consultant with TechTalk is useful to any team trying to get the most from these techniques. The message of Christian’s talk very much echoed my own, that the tooling you use is entirely secondary to the collaborative relationship you need to build between the business and technical-facing members of the team. I was interested to learn about the tool, SpecLog, TechTalk are building to help teams with this problem, which seems to have many similar goals to my own Relish. It was nice of Christian to give Relish a name-check in his talk.
I want to give you some news about the future of Relish. A lot has happened since we first started the project just over a year ago. Justin quit the project to concentrate on his new role on the RSpec core team, The Cucumber Book went into beta, and my wife gave birth to our second child. All of those things have meant that Relish hasn’t been able to progress as quickly as I’d have ideally liked, so thanks for sticking with us!
I’m still passionate about the vision for Relish, and as I teach training courses on using Cucumber and BDD, it seems to resonate with a lot of people. With The Cucumber Book pretty much behind us now, Relish will come back into focus and the pace will pick up again. I hope to have it launched by the end of the first quarter of 2012.
Here’s a rough outline of what we plan to do:
Usability & Information Architecture Enhancements
I have an IA specialist on the team now, and you may have noticed the first tweaks we’ve been making this week. There’s a lot to catch up, so we’ll focus on this until the existing features are polished to his (and hopefully your) satisfaction. Please let us know how we’re doing as we progress, and let us know if any of the changes annoy or delight you.
If there’s anything specific you’d like to see done that isn’t already logged in UserVoice, please add it there or reply to this post.
Relish can’t deliver on the promise of living documentation until the non-technical readers can tell, when they’re looking at a scenario, whether it’s passing or not. We’ll be building a plugin for Cucumber that allows developers and testers to send their test results to Relish, so each scenario can be rendered with a big green tick if it’s passing. This will also enable us to start building all kinds of exciting dashboards for project managers to get a high-level overview of what’s going on.
Commenting / Feedback
Just being able to read features is not enough. I want Relish to be a collaboration tool, and that means that stakeholders should be able to comment on and give feedback about the documentation they’re reading.
RSS / Activity Feeds
People who want to stay up to date with changes to a project will be able to get a nice high-level summary via RSS, and possibly other means too.
As the codebase stabilises, I’m becoming increasingly comfortable with the idea of a stand-alone installer for customers who are not happy about having their features stored outside of their firewall. In the new year, I’ll be looking for a couple of friendly enterprise customers to help me shape this. If you’re interested, please get in touch.
Plans & Pricing
Yes, it had to happen eventually 🙂
When we come out of private beta (which is still a few months away), it will be with paid accounts for private projects. This is a hard call to make, and I’ll be in working with our beta tester community to get an idea of what they feel is a reasonable price to pay for the service. Public accounts like RSpec and VCR will continue to be free.
As always, I’m keen to hear what you think of these plans. Let me know in the comments.
As requested by JB in the comments to my previous post, here is some data about what happens when a team choose fine-slicing over estimation.
You’re about to see a CFD chart drawn by a team who used BDD to break down every requirement into scenarios before they started hacking on them. The items on the left aren’t actually scenarios in this case, they’re very small user stories which tended to be of a size of about 3-4 scenarios each. The point is, we broke everything down into the smallest pieces of behaviour we could, then re-assembled them into chunks that were meaningful enough to build together.
Rather than using story points to manage the variation in size of stories, we gave each story a value of one point, and used BDD analysis to try to ensure each story was a uniformly small size.
This data was collected over a period of about six months by a team of about eight developers. Their system (a high-volume website) was already live and they were adding features to it.
What strikes me the most is how straight the ‘done’ line is.