Turnip: trying to solve Cucumber’s problems with ruby BDD

I’ve been playing around with Turnip: a library that was designed to solve Cucumber’s problems in four main areas:

  1. Having a separate test framework is annoying
  2. Mapping steps to regexps is hard
  3. Cucumber has a huge, messy codebase
  4. Steps are always global

I don’t really care about having a Cucumber test framework as I don’t often use RSpec, I actually find regexen quite powerful, I also don’t really care about Cucumber’s code base but I can see some merit in having scoped steps.

I converted my existing demo test suite across to use Turnip and this is what I found.

Turnip follows Spinach in trying to move away from using regular expressions to define steps. Fortunately, unlike Spinach, Turnip allows you to capture values using placeholders in your steps (this was an instant NO to using Spinach):

  step "I should see at least :exp_num_results results" do |exp_num_results|
    on :results do |page|
      page.number_search_results.should >= exp_num_results.to_i
    end
  end

Unfortunately there are a number of limitations to this approach:

  • You can’t capture more than a single word in a placeholder without using double quotes in your feature file – making them less readable; and
  • You can’t capture values like “100,000″ as it doesn’t appear to support special characters in placeholders.

You can also do some magic using custom step placeholders, where you define regular expressions, but to me that kinda defeats the point of not using regexen in the first place.

The scoped steps are quite neat and ruby like, and I can see these being very useful for large projects where multiple people are writing the steps.

Running Turnip files is as simple as using the RSpec test runner, but unfortunately the output of pending steps is less useful than Cucumber in that there are no code snippets, and only the first undefined step is displayed as undefined (unlike the Cucumber which shows you every step that is undefined).

Summary

I like the approach that Turnip takes in being a simple way to write business like tests and run them using RSpec. Unfortunately that simplicity comes at a price, and losing the power of regexen in step definitions is not something I would like to give up hastily. I am hoping that the scoped steps someday make their way into the Cucumber code base.

Writing a CoffeeScript web application using TDD

As part of the Test Automation Bazaar minesweeper challenge, a colleague and I developed a CoffeeScript implementation of minesweeper. We wanted to use a test first approach to writing our CoffeeScript, so we decided to use Jasmine.

Setting up a development environment

Even though we were developing the application in CoffeeScript, which generates JavaScript, we found it easy to use a simple ruby environment to boostrap these tools. Our environment therefore looked something like this:

  • Ruby 1.9.2 (specified through a RVM .rvmrc file)
  • CoffeeScript installed via Homebrew
  • guard, guard-coffeescript, guard-sass gems for automatically generating JavaScript from CoffeeScript (and CSS from Sass)
  • jasmine gem: for writing/running tests
  • Sublime Text 2 editor for writing code

Specifying a Guardfile

A Guardfile consists of a bunch of guards that perform actions whenever a file is modified in that location. Our Guardfile looked like:

guard 'coffeescript', :output => 'javascripts' do
  watch /^coffeescripts\/.*[.]coffee/
end

guard 'coffeescript', :output => 'spec/javascripts' do
  watch /^spec\/coffeescripts\/.*[.]coffee/
end

guard 'sass', :input => 'stylesheets'

Writing a Jasmine specification in CoffeeScript

A Jasmine spec in CoffeeScript looked something like this:

describe 'GameState', ->
  game_state = null
  field = Field.new mineCount: 1, rows: 1, cols: 3

  beforeEach ->
    game_state = GameState.new field

  it 'should initialise lost to false', ->
    expect(game_state.lost()).toEqual false

  it 'should initialise won to false', ->
    expect(game_state.won()).toEqual false

  it 'should initialise remaining_mines to mine_count', ->
    expect(game_state.remaining_mines()).toEqual 1

  it 'should initialise remaining_mines to mine_count', ->
    expect(game_state.remaining_cells()).toEqual 2

Running Jasmine tests

The jasmine gem makes it very easy to run your jasmine tests. There’s a rake task called ‘jasmine’ which you can run to launch a jasmine server locally on port 8888. If you browse to that page, you’ll see something like this:

Test First CoffeeScript development

Now that you have Jasmine running, and Guard generating the CoffeeScript, it’s easy to write a new spec, refresh the Jasmine browser page to run all your tests (in our case in a third of a second) and then write the code to make it pass.

Automatically running Jasmine tests on Travis CI

If you’ve got your CoffeeScript and Jasmine on github, it’s trivial to automatically run all your Jasmine tests using the Jasmine::Ci rake task on Travis CI. All you need is a .travis.yml file like:


language: ruby
rvm:
  - 1.9.2
env:
  - DISPLAY=:99.0
before_install: sh -e /etc/init.d/xvfb start

Once you add the project to travis, it’ll automatically run whenever you push. Magic.

Lessons Learned

  • Use Sublime Text 2 for CoffeeScript joy: TextMate 2 forces you to use tabs (4) for CoffeeScript development, we couldn’t find a way to make it stop, and had to switch to Sublime Text 2 (for the better).
  • You still need to know JavaScript: all CoffeeScript does is generate JavaScript for both your application code and Jasmine specs. We found ourselves often delving into the generated code to see what was actually going on, and when using Firebug to debug.
  • Use the jasmine_content div to test the DOM. Jasmine uses a special div with an id of jasmine_content, which we used to inject and test HTML.
  • CoffeeScript classes are odd: we instead used closures to encapsulate state, using a new method on an object, which has the added benefit of looking like ruby.

I hope you enjoy your CoffeeScript test driven development with Jasmine.

An automated testing journey

I did a presentation this evening in Brisbane on an automated testing journey that one may embark on. The whole thing was centered around this tube map style diagram I came up with recently: (download in PDF)

Here’s a link to my prezi slides and it should appear below (if you have flash enabled that is).  You can also download them in very printable PDF if you so choose.

I feel the presentation was well received, but I really shouldn’t have tried to squeeze three days of thinking into 30 mins. Oh well.

As always, I welcome your feedback.

My thoughts on Brisbane ANZTB SIGIST: September 2010

I went along to the ANZTB SIGIST last night, at the Hilton here in Brisbane. It was probably the best one I’ve been to, both in attendance, and caliber of presenters. There were five presenters all up, which is a lot to squeeze into two hours including drinks and conversation. It’s no surprise then that the last presentation by Craig Aspinall was a little bit rushed, which is a shame because I would have liked to ask more questions than time allowed (apparently the Hilton would kick us out if we stayed longer).

Craig Smith and Rene Masten from Suncorp began with an excellent presentation on Agile Testing. Craig was awesome in both presenting his extensive knowledge (he’s a coach) and his slides were also very cool (minimal text and no bullets!).

Craig’s main theme was about how to make testing cool: ensuring people say “that’s cool” when you tell them about your testing. He talked about what makes up an agile team, the techical divide between devs and testers, ATDD, ensuring you deliver, and how there is currently a great opportunity for testers (who want to be bothered). Rene followed by talking about organizational change, training and coaching, communicating what you’re doing (internally and externally) and building quality in.

Craig finished with a great motivating style of emphasizing it’s all about passion and craft, “who’s awesome!”, and not being afraid of technical challenges.

The other presenters were Ben Sullivan and Brent Acworth, both also from Suncorp, who gave a demo of BDD using JBehave and Hudson. I enjoyed the talk but there wasn’t much I hadn’t seen before or didn’t know.

The final presentation was by Craig Aspinall and was unfortunately squeezed into a small time slot. It was about what Craig dubs “Automated Black Blob Testing”, the rationale being testing is not black box as a box has a predefined shape, it’s more of a blob.

Craig’s approach looked solid, although I was slightly concerned when he mentioned the project being a SaaS solution and how much effort was being put into automated testing.  I’m not criticizing what Craig did tehnically, I am just concerned about the prevalent practice of the onus of testing SaaS solutions being put onto the customer. I believe if you buy a SaaS, you should get a working SaaS, minimal, if any at all, testing required. But that’s just me. Otherwise, a great talk, besides Craig using Java when Ruby and Watir would have done the trick. ;)

A great ANZTB SIGIST, and hopefully more good ones to come!

Craig Smith’s Slides

Some of my photos

This slideshow requires JavaScript.