Two Truths I’ve Learned About Writing Clean Rails Apps

In summary:

  • Implement every feature without javascript first, add javascript as a progressive enhancement
  • Stick to REST. Always.

Now I’m as suspicious of absolute rules as you probably are, but these two have served me extremely well in keeping my Rails code clean and easy to maintain. I’m happy to push the boat out and call them truths, my truths anyway.

Let me explain.

Progressive Enhancement

When I first started at Songkick, Cucumber was just emerging and webrat was the new kid on the testing block. Although using Selenium for full-stack javascript testing was possible with webrat, it wasn’t easy and we quickly made a decision: we would build the first iteration of every feature to work without javascript. This was mainly driven by pragmatism: we could easily use webrat to test the app through Rails integration testing stack, and these tests ran quickly rather than waiting for a browser to start up.

What happened surprised us. We would receive fancy designs from our designers which could only be implemented with fairly complex javascript. Because of our rule, we had to figure out how to deliver the same behaviour using basic HTTP posts and gets of forms and URLs. This required us to simplify the designs for our first iteration. We got a little push-back at first but we were a tight team and the designers quickly caught on. Especially when we started shipping these iteration one features quickly and reliably. And funnily enough, it turned out that often these simple HTTP-only versions of the features were actually fine, and the designers decided we should move onto other stuff, instead of building all that complex javascript that had been implied by their earlier designs.

So this rule had helped us to do the simple thing. We shipped features and moved on.

When we did have to add javascript, we added it on top of the existing working HTTP-only implementation, and we got another surprise: it was easy! Nearly every time, because we’d built the server-side infrastructure to a clean, standard pattern, the javascript we needed was clean and simple to write.

Which leads me to my next truth…

Stick to REST. Always.

I’ve always felt a little uncomfortable with Rails’ controllers. They violate the Single Responsibility Principle for me. I’d prefer a pattern where I had a separate Request class for each action, rather than a few largely unrelated methods on the same controller class. But Rails is there and there’s a lot else to like, so let’s not get into that.

We all know it’s a bad idea to let our controllers bloat and something I’ve been enjoying greatly lately is Jose Valim‘s Inherited Resources. By sticking to this pattern, you push all the logic into your domain model, and keep the controllers focussed on what they need to do: handling web requests.

You end up with more controllers, and more models too. You create model objects that — Gosh! — don’t represent database tables, but represent other domain concepts instead. Altogether though, you end up with smaller, more focussed classes that are easier to test and easier to read and understand.

Using Capybara with RSpec Outside Cucumber

If you want to try using Capybara for browser automation on it’s own, here’s a simple script to get you started:

require 'rubygems'

require 'capybara'
require 'capybara/dsl'

Capybara.default_driver = :selenium
Capybara.app_host = "http://www.google.com"

require "rspec/expectations"
class Google
  include Capybara
  include RSpec::Matchers

  def search_for(text)
    visit "/"
    fill_in "q", :with => text
    click_button "Search"
  end

  def ensure_results_contain(expected_text)
    page.should have_content(expected_text)
  end
end

google = Google.new
google.search_for("Matt Wynne")
google.ensure_results_contain("Tea")

To make this work you’ll need to install the capybara and rspec Ruby gems:

gem install capybara rspec

Features != User Stories

User Stories are a great way to plan your work. You can take a big hairy requirement and break it down into chunks that are small enough to work on without anyone freaking out. When you’ve crumbled up your big hairy requirement into little user story chunks, you can pick and choose which chunk to build first, and even drop some chunks altogether when you realise they’re not that important. Great stuff.

So we like user stories. They’re our friends. Here’s one:

In order to pay for the items in my shopping cart,
As a customer,
I want a credit card checkout

Now, suppose when we do the analysis for this story we discover that certain types of credit cards are much easier to hook up to a checkout than others. We want to ship something as soon as possible, so that people can start giving us money. So we split the story in two.

One for the friendly FooCorp credit card:

In order to pay for the items in my shopping cart,
As a customer with a FooCorp credit card,
I want a credit card checkout that accepts my card

And another for the altogether-much-harder-to-integrate-with BarCorp credit card:

In order to pay for the items in my shopping cart,
As a customer with a BarCorp credit card,
I want a credit card checkout that accepts my card

As everyone knows, BarCorp have some really outdated systems, and connecting to their payment processing is a right old pain. We’ll leave that for later.

So let’s get on with implementing our first story. We’ve done some analysis on it, but let’s make sure we understand the acceptance criteria. We have a little sit-down with our product owner, and come up with this list:

  • when a user attempts to pay with a BarCorp card, we show them a “Sorry” message
  • when a user pays with a FooCorp card, we send them an email with their receipt
  • when a user attempts to pay with a FooCorp card but gets their details wrong, we bounce the page and show them an error message, so they can correct their mistake.
  • when a user attempts to pay with a FooCorp card but the card is declined, we bounce the page and show them an error message, so they can try another card.

So those are the acceptance criteria for this story. We didn’t do anything fancy with them like put them in a Microsoft Word® document or a Jira ticket. We just wrote them down on a piece of paper.

Now we’ll use Cucumber to document these acceptance criteria as Scenarios that we can use to drive out the behaviour we need to get this story done.

Let’s start by creating a new feature:

mate features/pay_with_foocorp_card.feature

Wrong

Whoops. You made a mistake. Did you spot it?

That’s right, you went and got confused and thought that the user story you were implementing now was the same thing as the feature. It’s not.

Right now, this distinction probably seems a bit nit-picky. But wait until you’ve got around to implementing BarCorp’s credit card check-out, and BazCorp’s too. Your features directory is going to look like this:

features/
  pay_with_foocorp_card.feature
  pay_with_barcorp_card.feature
  pay_with_bazcorp_card.feature

Can you see where this is going? Give it a few months, and your features directory is going to look like a history log of the development of your project. Hey, while you’re at it, why not put the jira ticket in the filename, or even use a tool to synchronise the feature files with Pivotal Tracker?

I see lots of people doing this, and trust me, it’s an anti-pattern.

Before there was Cucumber, there was the RSpec Story Runner. There was also this blog post. That was a long time ago. When Aslak created Cucumber, he renamed the files from .story to .feature. This wasn’t an accident or an idle act of whimsy: it’s because there’s a difference.

User Stories are a planning tool. They exist until they’re implemented, and then they disappear, absorbed into the code.

Cucumber features are a communication tool. They describe how the system behaves today, so that if you need to check how it works, you don’t need to read code or go punching buttons on the live system. Organising your features according to the way they were planned and implemented is a distraction from this purpose.

Imagine if the user manual for your washing machine was organised by how the washing machine had been constructed? A nonsense.

Going back to our example, let’s rename our feature:

mv features/pay_with_foocorp_card.feature features/pay_with_credit_card.feature

Now that’s more like it. As we implement the first User Story for FooCorp credit cards, we’ll put some scenarios into this feature. When we come to implement BarCorp support, we can add a couple more scenarios to the same feature. We might even have to modify existing scenarios, perhaps if we add a new widget to the UI to let the user choose their credit card type.

The User Story is absorbed into our features and becomes invisible, leaving the features as a live document of what the software does now, not as a record of how it was constructed.

Thanks to Paul Wilson for reminding me to write this post.

Seven Truths Exercise

Recently, I played a game with a team I was training which I called “Seven Truths of Test Automation”.

I got each “truth” and wrote it on an index card and put it in a (blank) envelope. I got to the training room early and hid them around the room, not very well, so that they were quite easy to find.

We did some other stuff first, and people were looking at all these little envelopes sticking out from behind whiteboards and under cushions, wondering what was going to happen.

Then I told them that we were going to play a game. I told them that there are seven truths of test automation, and they were about to discover them. I split them into 6 small groups. I then told them they would each go out and discover a truth. When they found it, they had to ask themselves the following questions:

  • Do we agree with this?
  • What are the implications of acting on this truth?
  • What are we going to do now?

I wrote those questions on a flip-chart. We then played the game out as a group on a single truth to make sure everyone got it. I made sure that we had a full and frank discussion about whether we agreed with the truth or not. We thought through the implications (good and bad) and listed them out. We then talked about some concrete steps we could take. When they offered vague intents “We’ll start getting better at reducing duplication” I urged them to say exactly what they were going to do next week to get better at reducing duplication.

Then they split off and each did their own. They wrote up a poster and we did a gallery at the end of the session where they had a chance to share their learning with the group.

It worked really well, generated great energy and used up a good couple of hours.

For the record, I think the truths I used with this group were:

  • test automation is software development (this is the one I picked to do with the whole group)*
  • duplication destroys maintainability*
  • incidental details destroy maintainability*
  • know when it’s OK to cheat
  • some things just aren’t worth testing
  • work with developers to make their systems testable
  • don’t test other people’s code

It felt a bit egotistical handing them down My Seven Truths, so I made the point that they were just my truths, and they would discover their own as they learned to become better testers. You could obviously vary the truths depending on what you think the that group needs to hear / discuss, and those truths could be about anything, not just test automation.

Installing Ruby Gems with Native Extensions on Windows

If you’re stuck trying to run Ruby on Windows, one barrier you might have encountered is in trying to install a gem like ruby-debug or rdiscount. You’ll have seen an error like this:

%gem install ruby-debug
Building native extensions.  This could take a while...
ERROR:  Error installing ruby-debug:
        ERROR: Failed to build gem native extension.

C:/Ruby187/bin/ruby.exe extconf.rb
creating Makefile

make
'make' is not recognized as an internal or external command, operable program or batch file.


Gem files will remain installed in C:/Ruby187/lib/ruby/gems/1.8/gems/linecache-0.43 for inspection.
Results logged to C:/Ruby187/lib/ruby/gems/1.8/gems/linecache-0.43/ext/gem_make.out

That’s no fun.

The good news is, the lovely guys at Ruby Installer have put together a fix, called the DevKit. This installs the low-level bits and pieces needed to build those pesky native extensions on your Windows machine. There are a couple of manual steps which I didn’t find especially clear, so I’m documenting them here.

  • Download the DevKit self-extracting archive here
  • Run the archive, and when prompted, choose to extract it to C:\DevKit
  • When the archive has finished unpacking, open a command prompt in C:\DevKit and run
    ruby dk.rb init
    ruby dk.rb install
  • That’s it. You can test it using:
    gem install ruby-debug

Hooray!

Photoshop Driven Development, Anyone?

This afternoon I came across a presentation by Michael Tamm on his libary Fighting Layout Bugs. This is an idea that I too, have had bubbling around in my head for some time. Michael, to his credit, has decided to actually do something about it.

Michael’s library takes screenshots of sample pages during a test run and analyses the bitmap, checking for text that’s overflowing a bounding line, for example, and highlighting it. You can even use this to fail the build!

Screen Shot 2010 10 01 at 17.31.34
Screen Shot 2010 10 01 at 17.32.01

I can see these kind of checks being really useful to HTML / CSS hackers to help catch basic mistakes, but I still would love to see a test driven approach become available to front-end coders. When we drive out code from a failing tests, we test the tests as well as the code, because we see the test fail before we see it pass. What if we could do that for our markup and CSS?

I therefore propose: Photoshop-Driven Development.

We can take a PSD design for a page, and compare the design with the rendered page. If the rendered page doesn’t match the PSD, we fail the build until it does.

Of course it could never be as simple as this. Not every pixel in the PSD needs to be in the rendered page: some of it will just be dummy text that you’d never need or want to validate. Also, while a PSD is a fixed size, the way a page behaves when it resizes is an important (and potentially buggy) piece of the puzzle.

Could there be a way to highlight the parts of a PSD that are important? Could the designer use naming conventions for their layers to indicate whether they are just dummy content to be ignored, that they’re bounding boxes that need to flex, or key content that must be there—and precisely there—at all costs?

Crazy? Possible? I know I haven’t got time to play with it right now, but maybe you have?

The Joy of Craftsmanship

I identify a great deal with the label of “Craftsman”, but some of the rhetoric around the current “Software Craftsmanship” movement bothers me. Why?

A little lightbulb went on recently as I read “Let my People Go Surfing”, by Yvon Chouinard, the founder of the outdoor clothing company Patagonia. Patagonia was founded by a group of friends who were social misfits: “dirtbag” climbers and surfers who only worked in order to save cash for their adventures. When they came to work, they wanted to enjoy it:

I wanted to have a company where people came to work on the balls of their feet, taking the stairs two at a time.

Yvon describes in the book how they’ve worked hard to keep this culture alive at Patagonia. Increasingly over, the years, I’ve prioritised enjoying my work over how much I can earn from it, and I believe that this search for joy is a fundamental part of my model of craftsmanship. When I haven’t enjoyed a job, either I’ve worked with people to change things so that I do, or I’ve left. I think it’s impossible to truly craft something when you’re working in an unpleasant environment with people you whose company you don’t enjoy. On the other hand, when I’ve had the privilege to work with great people using great tools, I feel like nothing can stop me.

So for me, craftsmanship is about finding joy in your work. Here’s Brian Marick on Joy:

Now, I could say that a joyful employee is a productive employee, and that lack of joy on a project is like a canary keeling over in a coal mine: a sign that something big is wrong and you better pay attention. Maybe that’s true. I’d certainly like to believe it. But, fundamentally, I don’t care. I think joy is its own excuse. We deserve it. More to the point, those people around us deserve it.

The Importance of Being Idle

It says over there on the left that I like to be idle, but I don’t often take the time to write about it. I’m not going to write much now, really, but here’s a thought for you:

An idle mind is a questioning, sceptical mind. Hence it is a mind not too bound up with ephemeral things, as the minds of workers are. The idler, then, is somebody who separates himself from his occupation: there are many people scarcely conscious of living except in the exercise of some conventional occupation.

— Robert Luis Stevenson

Outside-In vs Inside Out – Comparing TDD Approaches

At last month’s ScotRUG Brian Swan and I attempted to solve the TDD Avatars problem as a live recital in our chosen style. We each had 35 minutes.

The videos are here:

Brian’s Inside-Out TDD approach

Matt’s Outside-In approach

When Brian had walked us through his approach and solution at the last month’s meeting, he’d built his solution as a Rails application, with web forms for filling out bookings and viewing receipts and so on.

When I came to start practicing and converted the use case from the TDD Avatars paper into a Cucumber feature, it quickly became clear that the value of the system I was building, at least as described by the use case, was to provide printed receipts to customers. I then started to think about the simplest way I could build a system to provide that value.

Here’s the feature I wrote:

Feature: Pay bill

  Background: Prices
    Given the following operations are available:
      | operation        | price |
      | routine check up | 10    |
      | shots            | 5     |

  Scenario: Dave Pays for Fluffy
    Given there is an owner Dave Atkins, let's call him "Dave"
    And Dave brings his pet named Fluffy into the clinic for the following operations:
      | routine check up |
      | shots            |
    When the veterinarian charges him for the visit
    And Dave pays cash
    Then Dave is given a receipt which looks like this:
      """
      Operations:
        $10 (routine check up)
        $5 (shots)

      Total to pay: $15

      Paid cash, received with thanks

      """

Notice that the scenario doesn’t talk about clicking particular buttons or filling in boxes on a form? I’ve used a higher-level declarative style to describe the behaviour I want. In my experience this helps in various ways:

  • more human-readable features
  • features that aren’t coupled to a particular user interface

If you watch the video, you’ll see that the first thing I did, working my way in from the step definitions, was to create a custom step definition DSL for my problem domain. Instead of using a generic DSL like Capybara’s fill_in, click_button etc, I created this one:

module VetsHelper
  def register_operation_price(operation, price)
  end

  def remember_owner(name, nickname)
  end

  def create_visit(owner_nickname, pet_name, operations)
  end

  def charge_for_visit
  end

  def pay_with(payment_type, nickname)
  end

  def receipt
    ""
  end
end

This is arguably unnecessary: my step definitions are already translating from English into Ruby, so why add this extra layer of indirection?

As I worked my way from the outside (the features) into the step definitions, I wasn’t ready to commit myself to how I was going to couple the tests to my new application. By defining this interface, I’ve deferred that commitment a little later. I’ve also given myself a clean view of all the behaviour the new application needs to support.

My first iteration implementation (the one in the video) of VetsHelper drives out a domain model directly from the methods in that module. If that was what we released to our user, they’d only be able to print receipts if they knew how to use an IRB prompt. That might seem ridiculous, but we’ve gone a long way to solving the problem, and we could probably spike a simple script that let them do it from the command-line without much risk.

For our second iteration, we can talk to the customer about that command-line interface, then write a new implementation of VetsHelper, perhaps using some of Aruba’s DSL, which goes through that command-line interface instead of directly to the model. This is the beauty of using a declarative style together with your own domain-specific step definition DSL: it gives you the flexibility to swap in connections to the system that hit it at different levels, using exactly the same acceptance tests.

Did BDD Save Me Time?

When Brian and I were planning this month’s session, I showed him the code I’d written and he decided to do a comparable solution this time, without any UI, so that they were easy to compare. In fact, Brian’s solution looked much simpler, and was certainly quicker to write, because he didn’t have to spend any time writing the acceptance testing layers and he didn’t write any kind of entry-point Practice class. He just went straight into building the Appointment class.

A big difference between the solution we produced this month and the one that Brian had originally built was that we didn’t use Rails, and instead went for a much simpler solution that still provided some immediate value. I like to think that the idea for doing this came from the BDD approach I took—I’m pretty sure I remember the lightbulb going on as I typed out the feature—but we’ll never know now where this idea originated.

I noticed that Brian spent time testing getters on his classes, which I probably wouldn’t have done. I tend to try to avoid using them, except on value object, and I rarely test the behaviour of value objects. I rely on my acceptance tests to tell me if they’re not working.

Focus and Design

Brian’s big take-away was that the difference in our approaches when we needed a collaborator object. When I needed a collaborator for a class, I would just mock out the collaborator and carry on finishing off the class I was building, whereas he would leave the current class broken and go and build the other class first.

I find my (mock-based) approach gives me focus, and also means I can sketch out the design of the collaborator without having to commit myself to that design until I understand how it’s going to be used.

I’m really happy with the design I ended up with. It’s hard to make much of a judgement in such a simple problem, but I’d be interested to hear your thoughts on how the two designs compare. Which one would you have preferred to add a new feature to?

The Fable of the User-Centred Designer

Agile software development is not really about burn-down-charts, unit tests, refactoring or code metrics or even pair programming. At it’s heart, it’s about building software that really works for the people who are going to use it. All those practices you read about are just tools that help you to develop software iteratively, so that you can keep moving the software closer and closer to becoming what it’s users actually want.

The goal is not the practices themselves, but the ability they give you to iterate.

If you really want to build software iteratively, you also need to understand about User-Centric Design. I’m finally getting the opportunity to get involved with a project early enough to put this into practice myself. Here’s a great introduction I just found.

In summary:

  • Early and continual focus on user and their tasks
  • Empirical measurement of user behaviour
  • Iterative design