{ 2012 03 13 }

Using Cucumber for Load Testing

I sometimes get asked whether it’s possible to use Cucumber to test performance. The way to do it is to specify concrete examples of scenarios that the system will find itself in under stress. For example: 

Given there are 100,000 users registered on the system
When I create a new account
Then I should be taken to my dashboard within 5ms
or 
Given 1000 users are hitting the homepage simultaneously
Then each user should get a response within 2ms
Talking through these kinds of scenarios with your stakeholders will help you to understand where the boundary is for what they consider to be acceptable performance. You might find it hard to get them to be this specific at first, but help them understand that what you’re doing is drawing a line in the sand for the minimum acceptable performance – most of the time the application may be much faster than this. Now you have agreement about this, the next step is to work out how to automate these scenarios, by calling your load testing tool of choice from the Cucumber step definitions.

The key thing is to have Cucumber delegate to the stress testing tool, rather than the other way around. A common mistake people make is to simply point JMeter at existing Cucumber scenarios, but this doesn’t give you the benefit of having the parameters of the performance test documented in readable Cucumber scenarios.

These are not normal kinds of Cucumber tests. You would need to run these against an environment that’s representative of your production hardware, whereas normal behaviour scenarios could be run against any environment. It’s useful to create these scenarios early on during the project and run them against every build so they become constraints that every build must meet as the project progresses.

Posted by Matt on Tuesday, March 13th, 2012, at 3:52 pm, and filed under Agile / Lean Software Development, BDD.

Follow any responses to this entry with the RSS 2.0 feed.

You can post a comment, or trackback from your site.