As I tinkered away with my screwdriver, I reflected on how grateful I was that the designers of this machine had made sure it could be maintained by an idiot like me. I actually made our A701 by cannibalising two broken machines, and I have always been delighted by how easy it is to work on.

Perhaps it’s my confirmation bias, but it seems that machines like this no longer tend to be designed with maintenance in mind. In our consumer society, we’re encouraged to throw things away rather than repair them, and globalisation ensures that is often the apparently rational economic decision to take: paying a local repair man to fix your DVD player will cost more than popping down to Tesco and buying a new one. Thank goodness for those cheap Chinese factories, eh?

If we don’t design our manufactured goods for maintainability, perhaps it’s no wonder we often feel like we’re swimming against the tide trying to persuade people that it’s important in software.

If you’re using scrum, you’re probably familiar with using a Release Burndown chart to track the team’s progress, iteration by iteration, towards a bigger goal:

Here we can see the team’s velocity appears to be slowing down. They might still release at the end of Sprint #9, but there’s a chance, if the flattening trend in their velocity continues to worsen, that they may never finish the project at all.

We have enough information here to know that something may be wrong. We can use this information to warn stakeholders that our release date may be at risk of slipping, and may also start looking though the backlog of stories for things we can drop.

The problem with these responses is that they’re purely reactive. Assuming there is a systemic cause at the root of this slow-down, we’re not doing anything to deal with it and get the team back on track. We can go and ask the team what they think might be wrong, and try to help them correct any problems they can identify. This can often work, but sometimes the team may not have the experience or perspective to be able to see what’s going wrong.

Another problem is that we’ve had to wait until the end of the iteration to get this information. For an organisation that’s used to going into the darkness of 9-month waterfall projects, getting iteration velocity figures once a fortnight can see pretty decent. An awful lot can happen inside an iteration though, and having that summed up by a single number loses a lot of important detail. Like the signal on an old short-wave radio, this is low-fidelity data.

One crucial piece of data that’s hidden within the burn-down chart is what I call the rate of discovery. The whole point of agile development is that it’s iterative: we build and ship something, get some feedback, learn from that, build and ship some more, and repeat until our stakeholders are happy with it. If we’re doing iterative (as opposed to repetitive) development, we’re going to discover new user stories as we go through the project, perhaps even as we go through the iteration. This is a good thing: it means we’re listening to our customers. We need to make sure our project plans can handle this as a matter of course.

Going back to our release burn-down, we want to separate the rate of discovery from the rate of delivery. A great way to do this is simply to flip the vertical axis of the chart, and use a Release Burn-Up instead. On here we can start tracking two lines instead of one. First we draw the number of completed stories (or story points), and then stacked on top of that, we draw the number of stories still to do. That includes any story not yet done – whether it’s in the backlog or being worked on in the next iteration.

We can add extra fidelity to this chart in two dimensions: We can collect samples more often, and we can collect details about just how done the stories are as they move from ‘not done’ into ‘done’. Let’s start by collecting more detail about how done the stories are. Imagine our task board looks like this at the end of an iteration:

One story (Story A) is done, and 8 other stories are not done. As we track these counts over time, we can draw one line on our Release Burn Up chart for each category of ‘not done’ and stack the lines:

This chart has another name, Cumulative Flow Diagram (CFD). We call it this because as stories flow from ‘not done’ to ‘done’ across the task board, we’re drawing the accumulation of that flow on the diagram. There are lots of things we can gleam from this diagram.

If we look at the example above, we can see that work is stacking up in the design stage of our process. Because our CFD chart highlights this, we can put more directed effort into relieving the bottleneck on the designers, perhaps by adding an extra analyst to the team to run ahead and do some more detailed analysis of the upcoming stories in the backlog, or by helping the designers to break the existing stories up into smaller ones that are easier to understand.

You can wait until the end of the iteration to count these numbers, but why stay ignorant? If you collect this data every day, you’ll get quick feedback about where bottlenecks are appearing in your team, and be able to try small tweaks to correct them.

Estimate: an approximate calculation of quantity or degree or worth

Commitment: the act of binding yourself (intellectually or emotionally) to a course of action

• Estimates are not commitments.
• Plans based only on estimates are bad plans.
• Commitments based only on estimates are at best foolish, at worst dishonest.

Estimates are certainly a useful tool for making realistic commitments, but there are many others too that are equally, if not more, useful. Working at a sustainable pace, measuring data about past performance, and facilitating the genuine consensus of the whole team when committing are the way to make plans that can actually be achieved.

When I got back from the conference, my team were at a point where we had to chose between two competing technologies to build something. Looking at the two options, we really didn’t know which one to go for: they both had advantages, and at that time they were both unknowns to us. Wearing my new ‘options thinking’ hat, I realised that we didn’t need to choose at this point: in fact, making a punt right now would be downright irresponsible: we didn’t have enough information to make the decision properly.

So instead of using my ‘gut feel’ to pick one of the competing options like a real hero would, I did a much more pragmatic, but less intuitive thing: I decided we would do both, until it became clear which one was the right one to pick.

We were lucky that our constraints at this point were time, rather than cost, so we could afford the luxury of having two streams of work going on in parallel, knowing that one would eventually be thrown away. Obviously the hidden deliverable even from that throwaway stream would still be a developer or two who understood the problem domain really well, and with a new technology in their toolbox to use in the future.

A week later it was much clearer which of the options we should pick, and the other workstream was stopped. And everyone lived happily ever after

Update: If you want to know more about real options, read Chris’ article on InfoQ here.

There’s also a great transcript of the talk here on Tom Hume’s blog. Thanks Tom!

The canonical example of this is the ‘log in’ scenario:

Scenario: User logs in
  Given there is a User whose username is "matt"
  And I follow "log in"
  And I enter "matt" in "username"
  And I enter the User's password in "password"
  And I press "Log In"
  Then I should be logged in
  And I should see the text "Hello matt"

Phew. Now obviously I don’t want all this noise in the scenario every time I specify behaviour that requires a logged in user. I want to write something like this:

Scenario: User views their dashboard
  Given I am logged in
  And I follow the "dashboard" link
  Then I should see "This is your dashboard"

Thanks to the fabulous creativity of the Cucumber community, this is now possible. It’s also highly recommended, as it’s a great way to help you keep your step files tidy and DRY of excess duplication.

Given /I am logged in/ do
  Given "there is a User"
  Given "I follow \"log in\""
  Given "I enter \"#{User.first.username}\" in \"username\""
  Given "I enter \"#{User.first.password}\" in \"password\""
  Given "I press \"Log In\""
end

I’m doing this more and more now – writing simple ‘building block’ steps and assembling them to make steps that read nicely and make sense to the stakeholders.

Those of us who are lucky enough to have experienced working on a really effective XP team know about that almost magical thing that starts to happen when the barriers between different members of the team break down, egos and code ownership are cast aside, and the team starts to evolve the codebase as one, mighty, coding machine. It’s truly a joy to be a part of, or even just to watch.

When I worked at the BBC, we tried various experiments with different styles of pairing and retrospected on them regularly to figure out what was working. We went from the extreme of the same pair working together right through a whole story, sometimes staying together for a whole iteration, to switching up pairs several times a day. In the end, of course, we found a sweet spot somewhere in the middle, and seemed to know intuitively when a pair was going stale and needed a fresh injection of energy by changing up the members.

Compare this ideal with this description of “Total Football” (that’s ‘Soccer’ for those of you speak US English!)

“Total Football” is the label for an influential theory of tactical association football in which any player can take over the role of any other player in the team. It was pioneered by Dutch football club Ajax Amsterdam. In Total Football, a player who moves out of his position is replaced by another from his team, thus retaining the team’s intended organizational structure. In this fluid system, no player > is fixed in his nominal role; anyone can be successively an attacker, a midfielder and a defender.

Regularly rotated pairs of programmers are essential in developing this kind of fluid self-organisation within a development team. While specialisation may become important for larger teams, there’s no doubt that keeping the whole team familiar with the whole codebase for as long as possible brings huge benefits.

For me, the biggest, and often the least mentioned of these benefits is that the team develops a kind of shared consciousness about the code and how they conventionally like to work with it. Not only does this mean the team can move extremely quickly, knowing exactly where to find something and how best to hack on it, but it is a stake through duplication’s dark and twisted heart: “Didn’t we work on something a bit like this last week, Brett?”. Once a team starts to think and communicate together this effectively, code re-use goes through the roof as common problems are solved in consistent ways, patterns emerge and are factored out for re-use.

You can probably tell, I’m a massive fan of pair programming, mainly because I absolutely love being this effective in my day-to-day work. How about you, loyal reader? What are your experiences with pair programming? Have you tried it? How was it for you?

Back in the pre-agile days when our projects were planned around component pieces of the architecture, “value” was a meaningless concept. The whole project had a value, but day-to-day that didn’t help us prioritise our work – from inside the plan it was impossible to judge whether my database stored procedure was contributing more value than your pricing component – without one, the other would simply not function.

As we started to plan around small deliverables like user stories, the concept of value came to the fore. Suddenly it was obvious that the “file saving” feature was much more valuable than the “conditional formatting” feature. This helped us immensely – we could prioritise our work around this value, and as a result actually felt the glow from knowing that the work we were doing was worthwhile, valuable even.

Now that value has become a first-class concept to software development teams, what happens when it becomes an obsession? When managers who can now measure value and velocity start to compare teams, or offer them incentives to deliver more, faster? When we as developers get so driven and focussed on the delivery of value that we forget to design, refactor, consolidate, reflect, and sharpen our tools? When we forget to go home on time?

We get burned out.

As ever, it’s the responsibility of the developers (and their leaders) to stand firm and hold on to the space and time that they need to do their work at a sustainable pace. Not all the hours you spend in a day – tidying up the build scripts, writing a code generator, cleaning up a library so you can release it as open source – can or should be directly attributed to the value they will immediately offer to the business. That’s not to say we should spend our days idling, but that we should maintain a level of self-respect in everything that we do.

Last week the mighty Martin Fowler dropped by to talk to us.

I was lucky enough to spend a few hours in Martin’s company, sharing lunch in the canteen with him and a few other senior technical bods, listening to his talk and Q&A, then as he visited the various teams, taking a look at how we do things.

His talk lasted just over an hour, and was delivered with great panache, especially considering it was entirely off the cuff. Beginning with his experiences in software design right through his career, he described key lessons learned, building up to an inspirational tub-thumping call to arms for the British media (and my client in particular) to continue to battle the ‘information wasteland’ coming out of the American commercial media. Fantastic stuff.

I made several notes during his talk and I thought I’d jot them down here.

A lot of good ideas in software get lost though the generations. Part of the responsibility for this lies with the old-timers who don’t take adequate steps to communicate their ideas and lessons they’ve learned to the younger generation. Martin sees it as his main aim nowadays: to look backward, spot patterns, and try to communicate those through his books and talks.

The best ideas and techniques span multiple languages and technologies. By moving back and forth between the C++ and SmallTalk communities in his early days as a programmer, Martin was able to cross-pollinate ideas between the two, and filter out the most powerful concepts – those which applied to both.

Irreversible decisions are the most dangerous kind for any organisation. Lean (and thence Agile) principles try to make as many decisions as possible reversible, and to defer for as long as possible those which appear not to be. Largely, this is achieved by moving rapidly in small, safe steps so that the path backwards is always clear. Thus a big and risky task becomes smaller and more approachable.

Don’t accept that something can’t be changed – just try to figure out how.

If something is awkward and painful (e.g. database refactoring) then do it as often as possible, until it becomes easy.

Collaboration with the customer is key and to that end, the creation of a ubiquitous language is key. Other tools that can help with this collaboration are UML diagrams (as sketched on a whiteboard), CRC cards and tests.

The most important skill for a software developer is to be able to collaborate with people. The traditional view that mathematical skills are the most important is becoming redundant.

When designing code, two main features make for a good design:

• Lack of duplication
• Clear reveal of intent

To enable clarity of intent, naming is extremely important.

When designing a system, one approach is to think of modules as things which keep secrets, then partition your design around the secrets that need to be kept.

With our second major live release looming large on the horizon, I focussed the session on the theme of ‘Success’. My aim was to give the team a blueprint for a successful iteration to keep in mind when things were tough, and to help ensure that we were all pulling in the same direction by agreeing as a team what constitutes success for us.

I started by asking the team to score how successful, in their own mind, the last iteration had been, drawing a histogram on the flipchart of the results. Interestingly, there was quite a variance in the scores, with most people either grading the iteration 2/5 or 4/5, perhaps suggesting that different members of the team have different criteria for success, or that they just experienced the iteration differently.

I then asked them to reflect on the criteria they’d used to make that judgement, again in their own mind, capturing each of the criteria on a separate index card. We then spread the cards out on the floor, and mapped them into headings: teamwork, stability, delivery, happiness & joy… and a couple of others I can’t remember right now.

We then decided that our output would be some kind of vision statement under each heading, of the form “An iteration is a success when…” and worked through the various headings, discussing what it was that made us successful, and distilling that discussion into a bullet point on the chart.

Reflecting on what we’d covered, it seems to me that while the session was fun, motivating and affirming for the team, and thus valuable in it’s own way, it didn’t really enable us to focus on the major issues that are affecting us right now, such as the disruption of the recent office move. It also didn’t leave us with any concrete actions to take away and do in order to make things better. In terms of inspect and adapt, it really didn’t deliver.

When planning your retrospectives always remember your key goals (in priority order):

1. Try to smoke out the most significant areas of waste in the team’s production of quality software
2. Invent concrete actions you can take in order to reduce this waste
3. Build and motivate the team

Whilst building and motivating the team is an important goal for a session like this, it’s often a side-effect of successfully satisfying the first two. Concentrate on facilitating a session that helps reduce waste while having fun, and you’ve hit the nail on the head.