It’s perfectly possible to write automated acceptance tests without using Cucumber. You can just write them in pure Ruby. Take this test for withdrawing cash from an ATM:
Scenario: Attempt withdrawal using stolen card
Given I have $100 in my account
But my card is invalid
When I request $50
Then my card should not be returned
And I should be told to contact the bank
We could automate that test using good old Test::Unit, perhaps something like this:
{{lang:ruby}}
require 'test/unit'
class WithdrawlTests < Test::Unit::TestCase
def test_attempt_widthrawl_using_stolen_card
bank = Bank.new
account = Account.new(bank)
account.deposit(100)
card = DebitCard.new(account)
bank.lock_out(card)
atm = Atm.new(bank)
atm.insert_card(card)
atm.enter_pin(card.pin)
atm.withdraw(50)
assert atm.card_withheld?, "Expected the card to be withheld by the ATM"
assert_equal "Please contact the bank.", atm.message_on_screen
end
end
The big disadvantage of writing acceptance tests in pure Ruby like this is that it’s unlikely you’ll be able to show this test to your team’s analyst without their eyes glazing over.
Unless your analyst is, or has recently been, a programmer themselves, they won’t be able to see past the noise of Ruby’s syntax, clean as it may be, to understand the actual behaviour that’s being specified. The specification of behaviour and the implementation of the test are all mixed up together, and that’s a problem if we want to get feedback from our stakeholders about whether we’ve specified the right thing before we go ahead and build it.
If we want the benefits of using plain language to write our behaviour specification, then we need a way to translate that into automation code that actually pulls and pokes at our application. Step definitions give you a translation layer between the plain-language specification of behviour and the test automation code, mapping the Gherkin steps of each scenario to Ruby code that Cucumber can execute.
The cost of this extra layer is complexity: Yes, you have more test code to maintain than you would if you stuck to writing your tests in pure Ruby. The benefit is clarity: by separating the what (the features) from the how (the ruby automation code), you keep each part simpler and easier for its target audience to understand.
I’ve found cucumber also very useful for new team members to a project.
As everyone has there own testing style the cucumber spec provide a nice incite into the system before getting down and dirty with how the code is actual implemented.
You don’t have to write your Ruby like that! You could refactor it so that analysts who cannot program can read it as easilly as the cucumber spec. Because you’re not restricted to Cucumber’s stilted style, a refactored Ruby version could be more expressive. I’ve done it in Java, so it should be easy in Ruby.
The real issue is that programmers should NOT be writing these tests. They should be written by BA/QA. I know I’m dreaming. But it’s a good dream.
You could have setup your test to read more better. How about:
Any easier? I think you could simplify this even further, and then even more by using shoulda. I think an analyst could be taught how to read this, most analysts I have worked with could understand this. Also if developers know that an analyst will be reading their test case – they’ll strive to keep it more readable. That has a great knock-on effect as it forces the developer to think about naming conventions – and structure of the the system. It has worked for me in the past, but perhaps I’ve only worked with tech-savvy analysts?
Applying extract method yields:
class WithdrawlTests < Test::Unit::TestCase def test_attempt_widthrawl_using_stolen_card given_I_have_$100_in_my_account given_I_have_a_stolen_card when_I_request_$50 then_my_card_is_not_returned then_I_am_told_to_contact_the_bank end def given_I_have_$100_in_my_account @bank = Bank.new @account = Account.new(@bank) @account.deposit(100) end def given_I_have_a_stolen_card @card = DebitCard.new(account) @bank.lock_out(card) end def when_I_request_$50 @atm = Atm.new(@bank) @atm.insert_card(@card) @atm.enter_pin(@card.pin) @atm.withdraw(50) end def then_my_card_is_not_returned assert @atm.card_withheld?, "Expected the card to be withheld by the ATM" end def then_I_am_told_to_contact_the_bank assert_equal "Please contact the bank.", @atm.message_on_screen end end(Oh dear, formatting fail, sorry.)
I completely agree. Cucumber features make it easy to read what is actually going on. And easy to reuse common step definitions. We use Cucumber and Selenium for testing our PHP applications. Previously, we wrote Selenium tests in PHP and they were so hard to read and maintain that we gave up on Selenium testing altogether! Once we started using Cucumber (a Ruby library for testing our PHP code) we can actually read the tests and track down which features have changed and update our tests accordingly. Much easier to maintain by using this feature/step-definition abstraction than writing tests all in PHP.
I agree, some of QA/BA/client don’t have time to look at the tests, so why do we need to write another extra layer to increase my app’s complexity.
I would say that like many other things, this depends on the context. But for me, Cucumber is a collaboration tool more than its anything else.
“The benefit is clarity: by separating the what (the features) from the how (the ruby automation code), you keep each part simpler and easier for its target audience to understand.”
This said, I think the most important thing you should avoid is making the two target audiences write those things separately. The gherkin features should be the collaborative environment of discussions and decision on what the functionality should be. Only when you discuss those things between BA/QA/Dev you will develop the common understanding you need to build great software.
To sum up, Samnang Chhun, if QA/BA/Client dont have time to look at the cucumber features/tests, they dont know what their job is, and I would teach them the value of being a part of it. If they dont understand that this will make it possible for them to ditch parts of their own tool for test cases/requirements, I would fire them.
@Siggeb, @unclebob I’m absolutely advocating that all the writing is done collaboratively. The features are the place where you document the discussions you’ve had together. I want that documentation to feel as accessible as possible to people how can’t (and don’t want to have to) read code.
@Nat, @ben fair cop, that code sample is a bit of a straw man, but even with a sweet Ruby DSL for expressing your tests, you’re still going to put off many non-technical readers, in my experience.
http://www.software-testing.com.au/blog/2010/08/31/does-cucumber-suck/
I’m yet to be convinced it’s not a tool that has a very narrow area of benefit, being ridiculously overused and adding substantial cost to trivial tests. On the plus side, it’s reminding people that they should pay attention to behaviour, but I suspect a lot of people are really unclear on how much behaviour they should be testing.
One tip for maintainability would be to abstract your data. This would let you write things like
‘Given an account with available funds…’
As it stands, it’s not clear to me whether the dollar amounts for withdrawal are pertinent details of the test, or just noise.