MegaMutex: A Distributed Mutex for Ruby

Sometimes I need to do this:

unless enough_widgets?
  make_more_widgets
end

Which is all well and good, until I start letting two or more of these codes run in parallel. If you’ve never thought about this before, what can happen is something nasty called a race condition, where two or more processes (or threads) simultaneously check #enough_widgets?, and simultaneously both decide that they need to go and #make_more_widgets. With multiple processes now making more widgets, we end up with too many.

The solution is to lock this critical section of code so that only one process could ever run it at once - everyone else has to queue up and wait their turn. That way each check for #enough_widgets? will return an answer that’s accurate. In a single process with threads, this is achieved using the Mutex class, but when you run multiple processes in parallel, across multiple machines, you need something more. You need MegaMutex.

How

Suppose you have a WidgetMaker:

class WidgetMaker
  include MegaMutex
 
  def ensure_just_enough_things  
    with_distributed_mutex("WidgetMaker Mutex ID") do
      unless enough_widgets?
        make_more_widgets
      end
    end
  end
end

Now, thanks to the magic of MegaMutex, you can be sure that all processes trying to run this code will wait their turn, so each one will have the chance to make exactly the right number of widgets without anyone else poking their nose in.

Configuration

MegaMutex uses memcache-client to store the mutex, so your infrastructure must be set up to use memcache servers.

By default, MegaMutex will attempt to connect to a memcache on the local machine, but you can configure any number of servers like so:

MegaMutex.configure do |config|
  config.memcache_servers = ['mc1', 'mc2']
end

Installation

sudo gem install mega_mutex

Hacking

[http://github.com/songkick/mega_mutex](http://github.com/songkick/mega_mutex)

Ruby Programming

Comments (1)

Permalink

Rails Tip: Use Polymorphism to Extend your Controllers at Runtime

Metaprogramming in Ruby comes in for quite a bit of stick at times, the accusation being that code which modifies itself at runtime can be hard to understand. As Martin Fowler recently described, there’s a sweet spot where you use just enough to get some of the incredible benefits that Ruby offers, without leaving behind a minefield for future developers who’ll have to maintain your code.

One of my favourite techniques uses the Object#extend method, which allows you to mix in the methods from a module to a specific instance of a class at run-time. In my quest to eliminate as much conditional logic as possible from my code, I’ve seen a common pattern emerge a few times. Here’s an example from a refactoring session I paired on with my colleague craig.

We start with a Rails controller which handles user authentication. Over the passing iterations, it has grown to support not only bog-standard logins from users of the main web application, but a form that’s displayed on a 3rd-party partner site, as well as during the installation of a rich-client GUI app. All these clients need slightly different behaviour - different templates or layout to be rendered, and different destination pages to redirect to when the login has succeded.

Sadly the hackers passing through this controller have not been great boy scouts, and the code has started to get pretty unpleasant. This code is simplified for clarity:

class SessionsController < ApplicationController
 
  def new
    if params[:installer]
      render :layout => 'installer_signup', :action => 'installer_signup')
    else
      render :layout => 'modal'
    end
  end
 
  def create
    if params[:username].blank?
      flash[:error] = "Please enter a username"
      return render_new_action
    end
 
    unless user = User.authenticate(params[:username], params[:password])
      flash[:error] = "Sorry, that username was not recognised"
      return render_new_action
    end
 
    set_logged_in_user(user)
 
    if params[:installer]
      @username = user.username
      return render(:template => 'installer_done', :layout => 'installer_signup' )
    elsif params[:third_party]
      return render(:template => "third_party/#{params[:third_party]}")
    else
      return redirect_to(success_url)
    end
  end
end

Notice how the conditional logic has a similar structure in both actions. Our refactoring starts by introducing a before_filter which works out the necessary extension:

class SessionsController < ApplicationController
 
  before_filter :extend_for_client
 
  ....
 
  private
 
  def extend_for_client
    self.extend(client_exension_module) if client_exension_module
  end
 
  def client_extension_module
    return InstallerClient if params[:installer]
    return ThirdPartyClient if params[:third_party]
  end
 
  module InstallerClient
  end
 
  module ThirdPartyClient
  end
end

Notice that we don’t bother extending the controller for the ‘else’ case of the conditional statements - we’ll leave that behaviour in the base controller, only overriding it where necessary.

Now let’s extract the client-specific code out of the create action into a method that we’ll override in the modules:

class SessionsController < ApplicationController
 
  ...
 
  def create
    if params[:username].blank?
      flash[:error] = "Please enter a username"
      return render_new_action 
    end
 
    unless user = User.authenticate(params[:username], params[:password])
      flash[:error] = "Sorry, that username was not recognised"
      return render_new_action 
    end
 
    set_logged_in_user(user)
 
    handle_successful_login
  end
 
  private 
 
  def handle_successful_login
    if params[:installer]
      @username = user.username
      return render(:template => 'installer_done', :layout => 'installer_signup' )
    elsif params[:third_party]
      return render(:template => "third_party/#{params[:third_party]}")
    else
      return redirect_to(success_url)
    end
  end
 
  ...

Finally, we can the client-specific code into the appropriate module, leaving the default behaviour in the controller:

class SessionsController < ApplicationController
 
  before_filter :extend_for_client
 
  def new
    render :layout => 'modal'
  end
 
  def create
    if params[:username].blank?
      flash[:error] = "Please enter a username"
      return render_new_action 
    end
 
    unless user = User.authenticate(params[:username], params[:password])
      flash[:error] = "Sorry, that username was not recognised"
      return render_new_action 
    end
 
    set_logged_in_user(user)
 
    handle_successful_login
  end
 
  private 
 
  def handle_successful_login
    return redirect_to(success_url)
  end
 
  private
 
  def extend_for_client
    self.extend(client_exension_module) if client_exension_module
  end
 
  def client_extension_module
    return InstallerClient if params[:installer]
    return ThirdPartyClient if params[:third_party]
  end
 
  module InstallerClient
    def new
      render :layout => 'installer_signup', :action => 'installer_signup')
    end
 
    private 
 
    def handle_successful_login
      @username = user.username
      return render(:template => 'installer_done', :layout => 'installer_signup' )
    end
  end
 
  module ThirdPartyClient
    def handle_successful_login
      return render(:template => "third_party/#{params[:third_party]}")
    end
  end
end

Polymorphism is one of the power-features of an object-oriented language, and Ruby’s ability to flex this muscle at run-time opens up some really elegant options.

Ruby Programming

Comments (1)

Permalink

Goodbye CruiseControl.rb, Hello Hudson

Imagine you have a friend who writes a blog. Maybe you actually do. Let’s call him ‘Chump’. One day you’re chatting, and the conversation turns to technology. It turns out that Chump is using Dreamweaver to write his blog entries, and manually uploading them to his site via FTP. You’re appalled.

How do you update the RSS feed?

you enquire, trying to conceal the horror in your voice.

Oh, I just edit the Atom file manually, it’s not that hard.

says Chump.

Maybe nobody ever told Chump about wordpress.

At work, we just switched our build server from CruiseControl.rb to Hudson, and we won’t be looking back.

Ruby people, for some reason, seem distinctly inclined to use build servers made out of Ruby too. That’s nice and everything, but these things are childsplay in comparison to the maturity, usability, and feature-set of hudson.

Here’s why I recommend you switch to hudson for your Ruby / Git projects:

  • open source
  • piss easy to set up, even if you have no idea what java even is
  • solid git support
  • works with CCMenu (or your favourite CruiseControl monitoring desktop widget)
  • kill builds from the GUI
  • in fact, manage everything from the GUI
  • distributed architecture, allowing you to delegate builds to multiple machines
  • huge, active plug-in support
  • you have better things to do with your time than faff around hacking on your build server

The problem is, it doesn’t have a smug website with fancy branding, so you probably overlooked it the first time. Go back and take another look.

Ruby Programming

Comments (23)

Permalink

MagLev: Death of the Relational Database?

I just got around to watching Avi Bryant’s talk on MagLev, a new Ruby VM built on top of GemStone’s Smalltalk VM.

http://www.vimeo.com/1147409

Presumably this is the kind of thing Smalltalkers have been able to do for decades, but to me the prospect of having this kind of freedom on a Ruby platform is very exciting. Terrific stuff. I wonder what performance is like.

Agile / Lean Software Development
Ruby Programming

Comments (0)

Permalink

Minesweeper Dojo

This evening I facilitated our first coding dojo at work. I’d spent some time over the holidays putting together a progression of Cucumber acceptance test cases to build up the solution, and had solved the problem once myself.

I used the minesweeper kata from http://codingdojo.org/ which was a really nice easy problem for our first dojo I think.

Format

Executive summary:

  • Randoori format
  • 6 programmers (plus me, facilitating)
  • 2 hours.
  • 10 minute iterations
  • 10 minute retrospective at the end.

I was clear at the start to point out that it was rude and disruptive for the crowd to criticise design when tests were failing, but that once tests were green, we would have a design review and do some re-factoring.

In fact, as we got into it, we began stopping the clock for design discussions (though perhaps these could also be timeboxed to keep the urgency up) so that people got a fair chance at the keyboard, but we also got plenty of time to analyse the design.

Summary

I held a little retrospective at the end where we collected some thoughts, then a few of us went for a pint afterwards and talked about it some more.

Good points:

  • Most people seemed to enjoy working on a ‘toy problem’:
    • everyone had worked on it equally so design discussions were never personal
    • we could really focus on getting the best solution to the problem, because it was so simple
  • Doing it out of hours meant we could really relax and have fun with it
  • It was a great way to get the team to rapidly build consensus about conventions and coding style
  • Having flog stats in the rake task made the final re-factoring stage much more worthwhile and fun

Bad points:

  • Some people found the problem too easy to solve, meaning we spent a fair amount of the session ‘polishing’ a working solution
  • I forgot to reset the stopwatch sometimes, which meant things got stale
  • There was a tendency for people to just rotate in / out in order which meant you kept getting the same combinations of pairs In the end I started drawing them out of a hat
  • We suck at git, particularly moving files from one branch to another
  • We suck at understanding flog’s feedback very well

Ideas for next time:

  • Find a way to visualise the changing Flog scores between commits
  • Try other programming languages
  • Need to mix up the type of kata: some people would prefer doing things they’d find more challenging, like a tough re-factoring
  • Get some food and beer in
  • Try shorter iterations?
    • In fact, try short / long pairs and see which we prefer!

I tend to perfectionism, so I really enjoyed the last half hour where we just re-factored the right-but-ugly first solution to the problem, extracting methods and classes and moving code around, ironing out the wrinkles and getting the flog / reek scores down. I enjoyed the discussions as we traded off different design principles and weighed up the appropriate route to take. Some people definitely found this time less rewarding, but I wonder whether they rather missed the point: this isn’t about solving a tough problem - we do that all day - this is about solving a simple problem the right way so that next time we hit one of those tough problems, we might be able to solve it a little more elegantly.

Legacy

I put the git repository where we worked on the problem up on gihub here:

http://github.com/mattwynne/software-craftsmanship-katas/tree/master/minesweeper

I would be stoked if anyone forked the repo and pushed their own solution. How low can you get your flog scores?

Agile / Lean Software Development

Comments (3)

Permalink

In Defence of class << self

I recently came across this blog post from the venerable Chris Wanstrath (of github / err the blog fame) which took me somewhat aback. Chris suggests that using class << self to contain class methods (C# / Java programmers read ’static methods’) is a harmful habit that should be discouraged.

I dislike class methods generally: they remind me of procedural programming, and can be awkward to test. Yet I actually rather like this ruby idiom and I believe that there are some very good reasons to use it especially when refactoring.

Disadvantages

Following a thorough discussion of the advantages and disadvantages on the RSpec mailing list, I can tell you the reasons why you would not want to use it:

  • It can cause confusion for programmers who are inexperienced, or new to Ruby, who find the unfamiliar syntax and concepts hard to understand.
  • It can cause confusion for any programmer when working on a class file so long that the class definition has scrolled off the top of the screen, so you can’t tell if you’re looking at an instance method or a class method.
  • Your code will run (a bit) slower.
  • Some people might think you’re being overly-clever and showing off. As Mark Wilden puts it:

> look at me, ma, I’m opening the eigenclass!

Advantages

Here is a summary of the reasons why you might:

  • You can do things like make certain class methods private, and define attribute accessors for instance variables of the eigenclass.
  • You might think it looks cleaner.
  • It can help you to think in an object-oriented way about the class methods you’re defining.
  • You might find it aids refactoring.

My Take

In the end, I thought Brian Takita summed it all up very well, after some verbal jousting with Mark Wilden:

> I’m wondering if this is a discussion about taste. I have my > experience with both approaches and you have yours. Just because > something works better for you does not mean its going to work better > for me and vice versa. I know that using class << self has been very > helpful in making my design better and it helps me to understand the > code vs the def self.methodname. > > I see def self.methodname as a useful shortcut in irb or when there > are few (0 to many depending on the person and situation) simple > methods on the class. There is a point to where it, IMO, makes the > code more difficult to understand, though.

So it’s a pretty subtle and subjective thing, and perhaps less risky in a situation where all the people working on the code have a feeling for what makes a good design.

When reading code, I’m generally doing some basic warm-up refactorings to get used to the code, and one of the first things I want to do is group up all the class methods which are often sprinkled around the file together, clearing away the clutter so I can see the instance methods that define the actual behaviour of the class.

At this point, class << self is only marginally useful. I can quite easily group all the class methods just by putting them at the top or bottom of the file. Using a class << self block would allow me to fold them closed in Textmate, but that’s not really all that important to me.

Now let’s imagine that one of these private methods is long and ugly.

As I start to extract smaller methods, I want to be able to mark them as private so it’s clear that they’re not part of the public interface of the class.

This is the point where class << self starts to add value. It’s a safe and painless step to move the methods inside a class << self block, remove the ’self.’ prefix from the method definitions. Now I’m in a position where I can extract smaller private methods from the class methods.

Often, as I do this, I start to see responsibilities in the eigenclass methods that could belong together in another class. Factories often start as a class method, for example. As soon as it becomes clear that a break needs to be made, it’s easy to move the methods into the newborn class. This refactoring can be gently introduced by delegating to the new class from within the eigenclass.

Refactoring Example

Imagine that we have a model where Dogs are born with a temperament (playful, angry, etc.) that cannot be trained out of them during their lifetime:

class Dog
 attrreader :temperament
 def initialize(temperament)
 @temperament = temperament
 end
 end
Now suppose we want to limit those temperaments to just a couple. A valid way to do this might be to create a few factory methods, and mark the initializer method as private:
class Dog
   attrreader :temperament
   def self.newhappy
     new(:happy)
   end
   def self.newangry
     new(:angry)
   end
   private
   def initialize(temperament)
     @temperament = temperament
   end
 end
Although the current example is trivial, I’m going to refactor those class methods inside an eigenclass block, just so you can see what it looks like:
class Dog
   attrreader :temperament
   class << self
     def newhappy
       new(:happy)
     end
     def new_angry
       new(:angry)
     end
   end
   private
   def initialize(temperament)
     @temperament = temperament
   end
 end
Same code, different convention. Whatever.

Now suppose we get a requirement that all Dogs must have an owner. It has do be defined when the Dog is born. Now we’re going to have to take the owner as a parameter on all those factory methods:

class Dog
   attrreader :temperament, :owner
   class << self
     def newhappy(owner)
       new(:happy, owner)
     end
     def newangry(owner)
       new(:angry, owner)
     end
   end
   private
   def initialize(temperament, owner)
     @temperament, @owner = temperament, owner
   end
 end
How boring. Now obviously this is a silly and simplistic example, but with something in the real world, we can imagine a situation where the evidence for factoring out a DogFactory would start to weigh up. Here we go:
class DogFactory
   def initialize(owner)
     @owner = owner
   end
   def newhappy
     Dog.new(:happy, owner)
   end
   def new_angry
     Dog.new(:angry, owner)
   end
   private
   def owner
     @owner
   end
 end
So you can see that our shiny new DogFactory looks a lot like the eigenclass that was hiding inside of the Dog back there.

And what does our Dog class look like now?

class Dog
   attrreader :temperament, :owner
   private
   def initialize(temperament, owner)
     @temperament, @owner = temperament, owner
   end
 end
Nice and tidy. If we need to support clients who are used to the old factory methods on Dog, we can do it like this:
class Dog
  def self.newhappy(owner)
  warn("Please use a DogFactory for creating Dogs.")
end

Note that I deliberately used ’self.’ here to define the method. To me, the refactoring is now complete and this deprecated method is starting to wither away and die, and so it’s OK if it takes an evolutionary step backwards again. Right now I don’t really care whether it’s defined in a class << self block or not.

Wow. If you’re still with me, I hope that gives you an insight into how I think this technique can help you to clean up your Ruby code.

I probably spend at least as much of my time refactoring other people’s code as I do writing new code, so that may affect my perspective on this. Time after time when refactoring, I’ve seen methods move from instance methods on a class, to class methods, then finally off onto a shiny new (testable) class as instance methods. I think that anything that helps you to see the new class that could be formed around a group of methods and responsibilities is a good thing. On the other hand, I’m uncomfortable with the idea of leaving code behind that’s confusing to people less experienced than me.

What do you think?

Ruby Programming

Comments (5)

Permalink

Your Private Methods are None of my Business

A common sloppy mistake I see programmers making is to forget to declare a method as private when it’s only being used inside a class. This seems to happen more in Ruby, or maybe it just bothers me more because we don’t have interfaces. Rails ActiveRecord validation callbacks are a classic example where people often forget to do this.

Continue Reading »

Agile / Lean Software Development

Comments (8)

Permalink

Scrabbling up the Learning Curve

A few months ago I was at the peak of my powers.

I was leading a team of ten C# developers building a huge project on Microsoft’s .NET platform. I had been working on the Windows platform for years, and there was rarely a question from the team I couldn’t answer quickly and confidently, drawing on a deep well of past experience.

Continue Reading »

Agile / Lean Software Development

Comments (1)

Permalink

Behaviour-Driving Routes in Rails with RSpec

One thing that isn’t documented very well for RSpec is how to test your routes.

I came across an old post on the rspec mailing list which described a great way to do this:

describe TasksController "routing" do
 
    it "should route POST request for /tasks to the 'create' action" do
        params_from(:post, "/tasks").should == {:controller =>; "tasks", :action =>; "create"}
    end
 
end

Very nice.

Agile / Lean Software Development

Comments (0)

Permalink

C# => Ruby Culture Shock #1: ‘Private’ Methods Ain’t So Private

This is the first in what I hope to be a series of little posts about the little flashes of ‘culture shock’ that I experience as I start to move from C# to Ruby as my day-to-day programming language.

Here we’re going to look at the subtle yet significant difference in how method access modifiers work in Ruby and C#.

Continue Reading »

Agile / Lean Software Development

Comments (1)

Permalink