Sandi Metz’s Practial Object Oriented Design comes to London!

I’m delighted to announce that I’ll be joining Sandi Metz to teach two courses on Object-Oriented design this summer in London. We’re nick-naming it POODL.

You have two options:

  1. A two-day class running from July 3-4
  2. A more in-depth three-day class running from June 25-27

Sandi mentoring at a RailsGirls workshop

Sandi is not only one of the most experienced object thinkers I know, she’s also a wonderful teacher. If you’ve read her book, Practial Object Oriented Design in Ruby, you’ll already know her knack of bringing scary-sounding design principles gently down to earth with simple, practical examples.

 Not just for Rubyists

POODL is definitely not just for Rubyists. We’ll teach the examples in Ruby, but any programmer familiar with an Object-Oriented language will be absolutely fine. The challenging stuff will be learning the trade-offs between different design patterns, and how to refactor between them.

The feedback from this course in the US has been amazing, and I’m really looking forward to being a part of it over here in Europe.

Book your ticket now!

Agile / Lean Software Development
BDD
Hexagonal Rails
OOP

Comments (1)

Permalink

How much do you refactor?

Refactoring is probably the main benefit of doing TDD. Without refactoring, your codebase degrades, accumulates technical debt, and eventually has to be thrown away and rewritten. But how much refactoring is enough? How do you know when to stop and get back to adding new features?

TDD loop (image credit: Nat Pryce)

I get asked this question a lot when I’m coaching people who are new to TDD. My answers in the past have been pretty wooly. Refactoring is something I do by feel. I rely on my experience and instincts to tell me when I’m satisfied with the design in the codebase and feel comfortable with adding more complexity again.

Some people rely heavily on metrics to guide their refactoring. I like the signals I get from metrics, alerting me to problems with a design that I might not have noticed, but I’ll never blindly follow their advice. I can’t imagine metrics ever replacing my design intuition.

So how can I give TDD newbies some clear advice to follow? The advice I’ve been giving them up to now has been this:

There are plenty of codebases that suffer from too little refactoring but not many that suffer from too much. If you’re not sure whether you’re doing enough refactoring, you’re probably not.

I think this is a good general rule, but I’d like something more concrete. So today I did some research.

Cucumber’s new Core

This summer my main coding project has been to re-write the guts of Cucumber. Steve Tooke and I have been pairing on a brand new gem, cucumber-core that will become the inner hexagon of Cucumber v2.0. We’ve imported some code from the existing project, but the majority is brand new code. We use spikes sometimes, but all the code in the master branch has been written test-first. We generally make small, frequent, commits and we’ve been refactoring as much as we can.

There are 160 commits in the codebase. How can I look back over those and work out which ones were refactoring commits?

Git log

My first thought was to use git log --dirstat which shows where your commit has changed files. If the commit doesn’t change the tests, it must be a refactoring commit.

Of the 160 commits in the codebase, 58 of them don’t touch the specs. Because we drive all our changes from tests, I’m confident that each of these must be a refactoring commit. So based on this measure alone, at least 36% of all the commits in our codebase are refactorings.

Sometimes though, refactorings (renaming something, for example) will legitimately need to change the tests too. How can we identify those commits?

Commit message

One obvious way is to look at the commit message. It turns out that a further 11 (or 7%) of the commits in our codebase contained the word ‘refactor’. Now we know that at least 43% of our commits are refactorings.

This still didn’t feel like enough. My instinct is that most of our commits are refactorings.

Running tests

One other indication of a refactoring is that the commit doesn’t increase the number of tests. Sure, it’s possible that you change behaviour by swapping one test for another one, but this is pretty unlikely. In the main, adding new features will mean adding new tests.

So to measure this I extended my script to go back over each commit that hadn’t already been identified as a refactoring, check out the code and run the tests. I then did the same for the previous commit, and compared the results. All the tests had to pass, otherwise it didn’t count as a refactoring. If the number of passing tests was unchanged, I counted it as a refactoring.

Here are the results now:

Refactoring vs Feature Adding Commits

Wow. So according to this new rule, less than 25% of the commits to our codebase have added features. The rest have been either improving the design, or perhaps improvements to the build infrastructure. That feels about right from my memory of our work on the code, but it’s still quite amazing to see the chart.

Conclusions

It looks as though in this codebase, there are about three refactoring commits for every one that adds new behaviour.

There will be some errors in how I’ve collected the data, and I may have made some invalid assumptions about what does or does not constitute a refactoring commit. It’s also possible that this number is artificially high because this is a new codebase, but I’m not so sure about that. We know the Cucumber domain pretty well at this stage, but we are being extremely rigorous to pay down technical debt as soon as we spot it.

We have no commercial pressure on us, so we can take our time and do our best to ensure the design is ready before forcing it to absorb more complexity.

If you’re interested, here’s the script I used to analyse my git repo. I realise it’s a cliche to end your blog post with a question, but I’d love to hear how this figure of 3:1 compares to anything you can mine from your own codebases.

Update 28 July 2013: Corrected ratio from 4:1 to 3:1 – thanks Mike for pointing out my poor maths!

Agile / Lean Software Development
BDD

Comments (4)

Permalink

Fixing my testing workflow

Okay I’m bored of this. I need to talk about it.

I love to use Ruby, RSpec, Cucumber and Rails to do test-driven development, but my tools for running tests are just infuriatingly dumb. Here’s what I want:

  • When a test fails, it should be kept on a list until it has been seen to pass
  • When more than one test fails:
    • Show me the list, let me choose one
    • Focus on that one until it passes, or I ask to go ‘back up’ to the list
    • When it passes, go back up to the list and let me choose again
    • When the list is empty, I get a free biscuit
  • When a test case is run, a mapping should be stored to the source files that were covered as it ran so that:
    • When a file changes, I can use that mapping to guess which test cases to run. Fuck all this naming convention stuff, it’s full of holes.
    • At any time, I can pipe the git diff though the tool to figure out which test cases to run to cover the entire commit I’m about to make.

When I say test case, I personally mean:

  • An RSpec example
  • A Cucumber scenario

…but it should work for any other testing framework too.

I feel like having a tool like this that I trusted would make a huge difference to me. There are all these various scrappy little pieces of the puzzle around: guard plugins, autotest, cucover, cucumber’s rerun formatter. None of them seem to quite do it, for me. Am I missing something?

Or shall we make one?

Agile / Lean Software Development
Ruby Programming

Comments (8)

Permalink

Targeting Multiple Platforms (JRuby etc) with a RubyGems .gemspec

Recently we had a user who runs the relish gem on JRuby, and needed jruby-openssl to be loaded. He kindly submitted this patch, which I merged in without really thinking about it too much. Then the problems started.

That’s not the right way to express dependencies for different platforms using RubyGems and Bundler. I’ve done some research and I think I understand the current good practice for this, so I’m going to document it here.

The .gemspec is read at the time you build and release your gem, so any conditional logic in that file will be evaluated once when the gem is built and released on your machine. So a line like this:

s.add_runtime_dependency('jruby-openssl') if RUBY_PLATFORM == 'java'

Will bake-in the dependency based on whatever platform you run gem build on.

What you need instead is to evaluate the platform at runtime. Bundler offers you a way to do this, in your Gemfile:

platforms :jruby do
  gem "jruby-openssl"
end

What a jruby user will now experience is that when your gem is loaded, they’ll see a warning:

→ relish help
JRuby limited openssl loaded. http://jruby.org/openssl
gem install jruby-openssl for full support.
...

It’s now up to the user to manually install the gem themselves. It seems a shame that there isn’t any way to specify this information in the gem’s manifest, so that it can be installed on a platform-specific basis when you gem is installed, but as far as I can tell there’s no way to do that right now.

Agile / Lean Software Development
Ruby Programming

Comments (3)

Permalink

A Puzzle for Polite Ruby Programmers

I really enjoyed Jim Weirich’s session on polite programming at the Scottish Ruby Conference. He covered a problem that’s been vexing me for some time, about avoiding the use of method aliasing, by using inheritance instead. Unfortunately, his suggested solution didn’t tell me anything I hadn’t already tried. I still think this must be possible, but that I just don’t know quite enough about Ruby to be able to achieve it. Maybe you do?

Here’s the puzzle:

https://gist.github.com/912504

Can you solve it?

Ruby Programming

Comments (2)

Permalink

Two Truths I’ve Learned About Writing Clean Rails Apps

In summary:

  • Implement every feature without javascript first, add javascript as a progressive enhancement
  • Stick to REST. Always.

Now I’m as suspicious of absolute rules as you probably are, but these two have served me extremely well in keeping my Rails code clean and easy to maintain. I’m happy to push the boat out and call them truths, my truths anyway.

Let me explain.

Progressive Enhancement

When I first started at Songkick, Cucumber was just emerging and webrat was the new kid on the testing block. Although using Selenium for full-stack javascript testing was possible with webrat, it wasn’t easy and we quickly made a decision: we would build the first iteration of every feature to work without javascript. This was mainly driven by pragmatism: we could easily use webrat to test the app through Rails integration testing stack, and these tests ran quickly rather than waiting for a browser to start up.

What happened surprised us. We would receive fancy designs from our designers which could only be implemented with fairly complex javascript. Because of our rule, we had to figure out how to deliver the same behaviour using basic HTTP posts and gets of forms and URLs. This required us to simplify the designs for our first iteration. We got a little push-back at first but we were a tight team and the designers quickly caught on. Especially when we started shipping these iteration one features quickly and reliably. And funnily enough, it turned out that often these simple HTTP-only versions of the features were actually fine, and the designers decided we should move onto other stuff, instead of building all that complex javascript that had been implied by their earlier designs.

So this rule had helped us to do the simple thing. We shipped features and moved on.

When we did have to add javascript, we added it on top of the existing working HTTP-only implementation, and we got another surprise: it was easy! Nearly every time, because we’d built the server-side infrastructure to a clean, standard pattern, the javascript we needed was clean and simple to write.

Which leads me to my next truth…

Stick to REST. Always.

I’ve always felt a little uncomfortable with Rails’ controllers. They violate the Single Responsibility Principle for me. I’d prefer a pattern where I had a separate Request class for each action, rather than a few largely unrelated methods on the same controller class. But Rails is there and there’s a lot else to like, so let’s not get into that.

We all know it’s a bad idea to let our controllers bloat and something I’ve been enjoying greatly lately is Jose Valim‘s Inherited Resources. By sticking to this pattern, you push all the logic into your domain model, and keep the controllers focussed on what they need to do: handling web requests.

You end up with more controllers, and more models too. You create model objects that — Gosh! — don’t represent database tables, but represent other domain concepts instead. Altogether though, you end up with smaller, more focussed classes that are easier to test and easier to read and understand.

Ruby Programming

Comments (3)

Permalink

Installing Ruby Gems with Native Extensions on Windows

If you’re stuck trying to run Ruby on Windows, one barrier you might have encountered is in trying to install a gem like ruby-debug or rdiscount. You’ll have seen an error like this:

%gem install ruby-debug
Building native extensions.  This could take a while...
ERROR:  Error installing ruby-debug:
        ERROR: Failed to build gem native extension.
 
C:/Ruby187/bin/ruby.exe extconf.rb
creating Makefile
 
make
'make' is not recognized as an internal or external command, operable program or batch file.
 
 
Gem files will remain installed in C:/Ruby187/lib/ruby/gems/1.8/gems/linecache-0.43 for inspection.
Results logged to C:/Ruby187/lib/ruby/gems/1.8/gems/linecache-0.43/ext/gem_make.out

That’s no fun.

The good news is, the lovely guys at Ruby Installer have put together a fix, called the DevKit. This installs the low-level bits and pieces needed to build those pesky native extensions on your Windows machine. There are a couple of manual steps which I didn’t find especially clear, so I’m documenting them here.

  • Download the DevKit self-extracting archive here
  • Run the archive, and when prompted, choose to extract it to C:\DevKit
  • When the archive has finished unpacking, open a command prompt in C:\DevKit and run ruby dk.rb init ruby dk.rb install
  • That’s it. You can test it using: gem install ruby-debug

Hooray!

Ruby Programming

Comments (5)

Permalink

MegaMutex: A Distributed Mutex for Ruby

Sometimes I need to do this:

unless enough_widgets?
  make_more_widgets
end

Which is all well and good, until I start letting two or more of these codes run in parallel. If you’ve never thought about this before, what can happen is something nasty called a race condition, where two or more processes (or threads) simultaneously check #enough_widgets?, and simultaneously both decide that they need to go and #make_more_widgets. With multiple processes now making more widgets, we end up with too many.

The solution is to lock this critical section of code so that only one process could ever run it at once – everyone else has to queue up and wait their turn. That way each check for #enough_widgets? will return an answer that’s accurate. In a single process with threads, this is achieved using the Mutex class, but when you run multiple processes in parallel, across multiple machines, you need something more. You need MegaMutex.

How

Suppose you have a WidgetMaker:

class WidgetMaker
  include MegaMutex
 
  def ensure_just_enough_things  
    with_distributed_mutex("WidgetMaker Mutex ID") do
      unless enough_widgets?
        make_more_widgets
      end
    end
  end
end

Now, thanks to the magic of MegaMutex, you can be sure that all processes trying to run this code will wait their turn, so each one will have the chance to make exactly the right number of widgets without anyone else poking their nose in.

Configuration

MegaMutex uses memcache-client to store the mutex, so your infrastructure must be set up to use memcache servers.

By default, MegaMutex will attempt to connect to a memcache on the local machine, but you can configure any number of servers like so:

MegaMutex.configure do |config|
  config.memcache_servers = ['mc1', 'mc2']
end

Installation

sudo gem install mega_mutex

Hacking

http://github.com/songkick/mega_mutex

Ruby Programming

Comments (2)

Permalink

Rails Tip: Use Polymorphism to Extend your Controllers at Runtime

Metaprogramming in Ruby comes in for quite a bit of stick at times, the accusation being that code which modifies itself at runtime can be hard to understand. As Martin Fowler recently described, there’s a sweet spot where you use just enough to get some of the incredible benefits that Ruby offers, without leaving behind a minefield for future developers who’ll have to maintain your code.

One of my favourite techniques uses the Object#extend method, which allows you to mix in the methods from a module to a specific instance of a class at run-time. In my quest to eliminate as much conditional logic as possible from my code, I’ve seen a common pattern emerge a few times. Here’s an example from a refactoring session I paired on with my colleague craig.

We start with a Rails controller which handles user authentication. Over the passing iterations, it has grown to support not only bog-standard logins from users of the main web application, but a form that’s displayed on a 3rd-party partner site, as well as during the installation of a rich-client GUI app. All these clients need slightly different behaviour – different templates or layout to be rendered, and different destination pages to redirect to when the login has succeded.

Sadly the hackers passing through this controller have not been great boy scouts, and the code has started to get pretty unpleasant. This code is simplified for clarity:

class SessionsController < ApplicationController
 
  def new
    if params[:installer]
      render :layout => 'installer_signup', :action => 'installer_signup')
    else
      render :layout => 'modal'
    end
  end
 
  def create
    if params[:username].blank?
      flash[:error] = "Please enter a username"
      return render_new_action
    end
 
    unless user = User.authenticate(params[:username], params[:password])
      flash[:error] = "Sorry, that username was not recognised"
      return render_new_action
    end
 
    set_logged_in_user(user)
 
    if params[:installer]
      @username = user.username
      return render(:template => 'installer_done', :layout => 'installer_signup' )
    elsif params[:third_party]
      return render(:template => "third_party/#{params[:third_party]}")
    else
      return redirect_to(success_url)
    end
  end
end

Notice how the conditional logic has a similar structure in both actions. Our refactoring starts by introducing a before_filter which works out the necessary extension:

class SessionsController < ApplicationController
 
  before_filter :extend_for_client
 
  ....
 
  private
 
  def extend_for_client
    self.extend(client_exension_module) if client_exension_module
  end
 
  def client_extension_module
    return InstallerClient if params[:installer]
    return ThirdPartyClient if params[:third_party]
  end
 
  module InstallerClient
  end
 
  module ThirdPartyClient
  end
end

Notice that we don’t bother extending the controller for the ‘else’ case of the conditional statements – we’ll leave that behaviour in the base controller, only overriding it where necessary.

Now let’s extract the client-specific code out of the create action into a method that we’ll override in the modules:

class SessionsController < ApplicationController
 
  ...
 
  def create
    if params[:username].blank?
      flash[:error] = "Please enter a username"
      return render_new_action 
    end
 
    unless user = User.authenticate(params[:username], params[:password])
      flash[:error] = "Sorry, that username was not recognised"
      return render_new_action 
    end
 
    set_logged_in_user(user)
 
    handle_successful_login
  end
 
  private 
 
  def handle_successful_login
    if params[:installer]
      @username = user.username
      return render(:template => 'installer_done', :layout => 'installer_signup' )
    elsif params[:third_party]
      return render(:template => "third_party/#{params[:third_party]}")
    else
      return redirect_to(success_url)
    end
  end
 
  ...

Finally, we can the client-specific code into the appropriate module, leaving the default behaviour in the controller:

class SessionsController < ApplicationController
 
  before_filter :extend_for_client
 
  def new
    render :layout => 'modal'
  end
 
  def create
    if params[:username].blank?
      flash[:error] = "Please enter a username"
      return render_new_action 
    end
 
    unless user = User.authenticate(params[:username], params[:password])
      flash[:error] = "Sorry, that username was not recognised"
      return render_new_action 
    end
 
    set_logged_in_user(user)
 
    handle_successful_login
  end
 
  private 
 
  def handle_successful_login
    return redirect_to(success_url)
  end
 
  private
 
  def extend_for_client
    self.extend(client_exension_module) if client_exension_module
  end
 
  def client_extension_module
    return InstallerClient if params[:installer]
    return ThirdPartyClient if params[:third_party]
  end
 
  module InstallerClient
    def new
      render :layout => 'installer_signup', :action => 'installer_signup')
    end
 
    private 
 
    def handle_successful_login
      @username = user.username
      return render(:template => 'installer_done', :layout => 'installer_signup' )
    end
  end
 
  module ThirdPartyClient
    def handle_successful_login
      return render(:template => "third_party/#{params[:third_party]}")
    end
  end
end

Polymorphism is one of the power-features of an object-oriented language, and Ruby’s ability to flex this muscle at run-time opens up some really elegant options.

Ruby Programming

Comments (1)

Permalink

Goodbye CruiseControl.rb, Hello Hudson

Imagine you have a friend who writes a blog. Maybe you actually do. Let’s call him ‘Chump’. One day you’re chatting, and the conversation turns to technology. It turns out that Chump is using Dreamweaver to write his blog entries, and manually uploading them to his site via FTP. You’re appalled.

How do you update the RSS feed?

you enquire, trying to conceal the horror in your voice.

Oh, I just edit the Atom file manually, it’s not that hard.

says Chump.

Maybe nobody ever told Chump about wordpress.

At work, we just switched our build server from CruiseControl.rb to Hudson, and we won’t be looking back.

Ruby people, for some reason, seem distinctly inclined to use build servers made out of Ruby too. That’s nice and everything, but these things are childsplay in comparison to the maturity, usability, and feature-set of hudson.

Here’s why I recommend you switch to hudson for your Ruby / Git projects:

  • open source
  • piss easy to set up, even if you have no idea what java even is
  • solid git support
  • works with CCMenu (or your favourite CruiseControl monitoring desktop widget)
  • kill builds from the GUI
  • in fact, manage everything from the GUI
  • distributed architecture, allowing you to delegate builds to multiple machines
  • huge, active plug-in support
  • you have better things to do with your time than faff around hacking on your build server

The problem is, it doesn’t have a smug website with fancy branding, so you probably overlooked it the first time. Go back and take another look.

Ruby Programming

Comments (26)

Permalink