Comments on: Using Cucumber for Load Testing

By: Lance

Lance — Thu, 20 Jun 2013 12:33:13 +0000

My team is looking for a functional testing tool to overcome a backlog of testing needs and to use during Sprints. Cucumber would be my tool of choice if we can also use it to drive load tests. In essence, we would like the ability to run a Scenario N times using M threads to barrage the server to see how it holds up under load (e.g. "Scenario: Place order for new part" runs 100 times using a pool of 40 threads). Is something like this possible with Cucumber?

Matt if I understand your suggestion above, I would use Cucumber to describe the test, setup fixtures, etc, and then have a step def that calls funkload. Funkload would do it's thing, report results, and Cucumber would evaluate those results. The trouble I'm having with this is I seem to end up writing the same test twice---once as a Scenario and once as a funkload test as funkload doesn't have access to my step defs, etc. Has anyone accomplished this and has an example they can share?

Many thanks.

By: Matt

Matt — Thu, 20 Jun 2013 06:36:17 +0000

@Derek, Cucumber is targeted at business-readable behaviour specification. So-called 'non-functional' requirements like performance can often be described using examples like this. Writing them down helps force the team to make decisions about what they consider acceptable behaviour.

If you have a load testing tool delegating to Cucumber scenarios that just walk the app through basic behaviour, you'd lose that opportunity to discuss the performance constraints. You could still have the conversation of course, but your parameters would be buried away in the scripts for the load generator.

By: Derek

Derek — Wed, 19 Jun 2013 11:13:05 +0000

Considering cucumber seems targeted at functional testing, would you let cucumber to drive the load test (entirely) or get a load generator to drive cucumber tests?

By: Matt

Matt — Fri, 21 Dec 2012 23:22:54 +0000

I’m sorry Agata, I don’t think the performance testing chapter made the cut for the Recipes Book, which is going to the printers any day now.

Keep an eye on the http://cukerecip.es/ blog and maybe we’ll post one up there.

By: Agata Przybyszewska

Agata Przybyszewska — Fri, 21 Dec 2012 22:35:23 +0000

My team has started on bdd, and i am looking into a way of having cucumber rule all our tests. This brought me to a search for how to make cucumber drive performance tests.

I read your blog entry, and my first thought was wow ... i hurried to the pragmatic programmer website and bought the beta book (allready happily owning the cucumber book) ... just to discover the current version doesnt mention anything about performance tests ... now I feel slightly disappointed ...

So, could you give any hints as for how to make cucumber drive a performance test ... like jmeter ?

Agata

By: Matt

Matt — Mon, 24 Sep 2012 22:28:37 +0000

In reply to Ming.

Hi Ming,

Here’s a selection that I turned up in a recent bit of research for Cucumber Recipes:

By: Ming

Ming — Mon, 24 Sep 2012 07:42:01 +0000

"The key thing is to have Cucumber delegate to the stress testing tool."

Hi Matt,

Can you suggest some stress testing tools which Cucumber can delegate to?

Thanks