You probably don’t need a specification framework

I think plain language specification frameworks like SpecFlow and Cucumber are great, but have a lot of overhead and are way overused.

If you don’t have non-technical folk collaborating with you on your specifications, try writing plain automated tests instead. This means using plain NUnit/MSTest over SpecFlow in C# or minitest over Cucumber in Ruby. You’ll avoid the overhead of maintaining a plain language specification framework and be able to focus on developing a great set of tests instead.

It’s easier than you think to add a plain language specification layer to a set of well structured plain tests. So only add the specification layer when you need it, because chances are you ain’t gonna.

Five page object anti-patterns

I’ve observed some page object anti-patterns which commonly arise when starting out designing end-to-end automated tests. Chris McMahon recently asked for some feedback on his initial test spike for Wikipedia, and some of these anti-patterns were present.

Anti-pattern one: frequently opening and closing browsers

I often see both RSpec and Cucumber tests frequently opening and closing browsers. This slows down test execution times, and should be avoided unless absolutely necessary.

You can clear cookies between tests if you’re worried about state.

To open and close the browser only once in Cucumber, specify this in your env.rb file:

browser = Watir::Browser.new

Before do
  @browser = browser
end

at_exit do
  browser.close
end

To open and close the browser only once in RSpec:

browser = Watir::Browser.new

RSpec.configure do |config|
  config.before(:each) { @browser = browser }
  config.after(:suite) { browser.close }
end

Anti-pattern two: hard coding URLs on page classes

Chances are you’ll at some point run your automated tests in different environments, even if it’s just to verify that production has been updated correctly. If you’ve hard coded URLs in page classes, this can be problematic.

Fortunately it’s easy to avoid, by creating a module that contains base URLs which can be accessed by page classes. These base URLs can be stored in YAML files which can be switched for different environments.

module Wikipedia
  BASE_URL = 'https://en.wikipedia.org'
end

class BogusPage
  include PageObject
  page_url "#{Wikipedia::BASE_URL}/wiki/Bogus_page"
end

Anti-pattern three: pages stored as instance variables in steps or rspec specs

I don’t like seeing pages stored as instance variables (those starting with an @) in Cucumber steps or RSpec specs, as it introduces state and thus more room for error.

If you’re using the page-object gem, there are two methods available to access pages directly without using instance variables: visit_page and on_page (also visit or on from 0.6.4+). Both of these can be used as blocks, so you can perform multiple actions within these methods.

visit LoginPage do |page|
  page.login_with('foo', 'badpass')
  page.text.should include "Login error"
  page.text.should include "Secure your account"
end

Anti-pattern four: checking the entire page contains some text somewhere

I often see people checking that the entire web page contains some expected text. Even if the text was at the very bottom of the page hidden in the footer the test would probably pass.

You should check the text is where it should be, using a container that it should belong to. Ideally a span or a div may exist that contains the exact text, but even if it’s in a slightly larger container it is still better than asserting it exists somewhere on the page.

class BogusPage
  include PageObject
  cell :main_text, :class => 'mbox-text'
end

visit_page BogusPage do |page|
  page.main_text.should include 'Wikipedia does not have an article with this exact name'
  page.main_text.should include 'Other reasons this message may be displayed'
end

Anti-pattern five: using RSpec for end-to-end tests

This one is contentious, and I am sure I’ll get lots of opinions to the contrary, but I believe that RSpec is best suited to unit/integration tests, and Cucumber is suited to end-to-end tests.

I find I create duplication when trying to do end-to-end tests in RSpec, which is where Cucumber step definitions come in. Trying to do unit tests in Cucumber seems like too much overhead, and in my opinion is more suited to RSpec.

Turnip: trying to solve Cucumber’s problems with ruby BDD

I’ve been playing around with Turnip: a library that was designed to solve Cucumber’s problems in four main areas:

  1. Having a separate test framework is annoying
  2. Mapping steps to regexps is hard
  3. Cucumber has a huge, messy codebase
  4. Steps are always global

I don’t really care about having a Cucumber test framework as I don’t often use RSpec, I actually find regexen quite powerful, I also don’t really care about Cucumber’s code base but I can see some merit in having scoped steps.

I converted my existing demo test suite across to use Turnip and this is what I found.

Turnip follows Spinach in trying to move away from using regular expressions to define steps. Fortunately, unlike Spinach, Turnip allows you to capture values using placeholders in your steps (this was an instant NO to using Spinach):

  step "I should see at least :exp_num_results results" do |exp_num_results|
    on :results do |page|
      page.number_search_results.should >= exp_num_results.to_i
    end
  end

Unfortunately there are a number of limitations to this approach:

  • You can’t capture more than a single word in a placeholder without using double quotes in your feature file – making them less readable; and
  • You can’t capture values like “100,000” as it doesn’t appear to support special characters in placeholders.

You can also do some magic using custom step placeholders, where you define regular expressions, but to me that kinda defeats the point of not using regexen in the first place.

The scoped steps are quite neat and ruby like, and I can see these being very useful for large projects where multiple people are writing the steps.

Running Turnip files is as simple as using the RSpec test runner, but unfortunately the output of pending steps is less useful than Cucumber in that there are no code snippets, and only the first undefined step is displayed as undefined (unlike the Cucumber which shows you every step that is undefined).

Summary

I like the approach that Turnip takes in being a simple way to write business like tests and run them using RSpec. Unfortunately that simplicity comes at a price, and losing the power of regexen in step definitions is not something I would like to give up hastily. I am hoping that the scoped steps someday make their way into the Cucumber code base.

WatirMelon Spinach

Hello Spinach!

“Spinach is a new awesome BDD framework that features encapsulation and modularity of your step definitions.”

Seems like a good idea, so I thought I’d give it a try, and convert my existing simple Watirmelon Cucumber tests to use Spinach instead (link to my source code). It was very easy to do, here’s some observations:

  • It’s easy to get existing Cukes running using Spinach, but I imagine if you were starting out using Spinach you’d design things a lot differently
  • Steps belong in their own class, but you can include mixins to reuse steps – the ruby way
  • Steps are in a steps directory under features – shorter than step_definitions
  • Goodbye regular expressions in step definitions, which is a bit of a shame, as you can no longer capture values from the step name
  • As you can’t have regular expressions in step names, I find myself repeating steps that are similar but slightly different, this means my example steps have gone from 41 to 57 lines of code
  • Scenario Outlines aren’t supported at the moment, but I have raised this as a feature request
  • Hooks are dramatically improved, so I found them very easy to use and understand
  • There is no cucumber world, so you do a normal include instead, and env.rb is still supported
  • It displays really cool ticks on the command line when you’re running your spins

Well done to Codegram for releasing Spinach. If anything, it creates some great innovations that I imagine may find their way into Cucumber in the future.

Capturing screenshots in Watir-Webdriver & Cucumber

Update: 29 August – Thanks to Rob Hunter for the screenshot embed trick

When you’re running a suite of automated tests and one fails, I find it handy to capture a screenshot of the browser so it’s easier to investigate. Fortunately this is easier than ever before with Watir-Webdriver and it’s inbuilt save_screenshot method.

If you’re using Cucumber, you’ll be looking for something like this:

After do |scenario|
  if scenario.failed?
    Dir::mkdir('screenshots') if not File.directory?('screenshots')
    screenshot = "./screenshots/FAILED_#{scenario.name.gsub(' ','_').gsub(/[^0-9A-Za-z_]/, '')}.png"
    Browser::BROWSER.driver.save_screenshot(screenshot)
    embed screenshot, 'image/png'
  end
end

The neatest thing about this method is that it captures the entire browser page, not just what is displayed without scrolling (see a screenshot of my blog below).

I hope you find this handy, I think it’s awesome.

Removing local page references from Cucumber steps

I’ve been giving some thought about the maintainability of having local page object model references in Cucumber steps. To explain what I mean, here’s some code of mine from an Etsy page model step:

When /^an item is added to the cart$/ do
  @advanced_search_page = EtsyAdvancedSearchPage.new @browser, true
  @search_results_page = @advanced_search_page.search_for 'hat'
  @etsy_item_page = @search_results_page.click_first_result
  @item_title = @etsy_item_page.item_title
  @etsy_cart_page = @etsy_item_page.click_add_to_cart
end

Here I am using instance variables in Cucumber steps to store references to pages that I am interacting with. As you can see, it’s quite easy to lose track of which page is which.

If I want to use one of these pages in another step, I have to refer to the same variable name. The key to this concept is having pages return pages, so you don’t have to initialize pages in your steps (except for the first visit, which normally happens only once per scenario).

A colleague and good friend of mine has been showing me some alternatives. One alternative to this is to actually dynamically refer to pages only when you need them. This means you don’t need to return a page from a page, as they are always initialized when needed. The above method would look like:

When /^an item is added to the cart$/ do
  visit EtsyAdvancedSearchPage do |page| page.search_for 'hat' end
  on EtsySearchResultsPage do |page| page.click_first_result end
  on EtsyItemPage do |page|
    @item_title = page.item_title
    page.click_add_to_cart
  end
end

This introduces two new methods at the Cucumber step level that need to be created: on, and visit. The on method simply creates a new page object so you can call methods from it. As each page initialize checks expected title and expected element, it will raise an error automatically if either are incorrect. The visit method is an extension of on, which actually initalizes the page and visits it.

These are defined as a module in our Cucumber env.rb file, and then mixed into the Cucumber World, so that steps automatically have access to these.

module Browser
  BROWSER = Watir::Browser.new ENV['WEB_DRIVER'] ||:firefox

  def visit page_class, &block
    on page_class, true, &block
  end

  def on page_class, visit=false, &block
    page = page_class.new BROWSER, visit
    block.call page if block
    page
  end
end

World Browser

Summary

By introducing the on and visit methods, it means that we no longer need to have instance variables for page classes in our Cucumber steps. We also no longer need to worry about state transformations in pages, as these can be done in a modular way in the steps themselves. It means that when an error occurs initializing a page, it is more likely to to occur in the correct place. I find the steps more readible, as you only have to initialize the page once using the on block, and then can refer to page.

I have updated my set of Etsy Watir-WebDriver steps if you’d like to take a look.

What do you think? How will this scale?

Update: I have updated the WatirMelonCucumber project to use this style. It’s slightly different in that it supports two sites, and therefore the on method dynamically switches between these.

Cucumber Steps:

When /^I search for a? ?"([^"]*)"$/ do |term|
  on :home do |page|
    page.search_for term
  end
end

The on method:

def on page, visit=false, &block
  page_class = Object.const_get "#{@site}#{page.capitalize}Page"
  page = page_class.new BROWSER, visit
  block.call page if block
  page
end

An automated testing journey

I did a presentation this evening in Brisbane on an automated testing journey that one may embark on. The whole thing was centered around this tube map style diagram I came up with recently: (download in PDF)

Here’s a link to my prezi slides and it should appear below (if you have flash enabled that is).  You can also download them in very printable PDF if you so choose.

I feel the presentation was well received, but I really shouldn’t have tried to squeeze three days of thinking into 30 mins. Oh well.

As always, I welcome your feedback.

C# ATDD on a shoestring (or the complete guide to SpecFlow & NUnit in Visual Studio 2010 Express)

Introduction

When I’m working on something new at work, I like to be able to work on it in my own time at nights and on the weekend to polish my skills through experimentation, and share what I learn. I’ve recently been working on setting up Acceptance Test Driven Development in C# at work, but unfortunately Microsoft makes it really, really hard to polish your skills in C# development at home, unless you’re willing to part with large sums of cash (say from $20,775, and I’d prefer a new car). This is probably one of the reasons why I find the Microsoft .NET testing community (and the dev community too) quite insular, Microsoft just doesn’t encourage their users to share their craft.

Sure, Microsoft provides Visual Studio 2010 (VS2010) Express Edition – which can be used free of charge, even for commercial purposes, but the limitations on its functionality are borderline ridiculous. For example, it doesn’t support Microsoft’s own unit testing framework (Microsoft obviously don’t encourage TDD), and all add-ins are banned (no ReSharper for you). It also doesn’t work with Microsoft’s own version control system (TFS).

So, Microsoft it basically makes it really hard to do anything you want to do, but there are ways to get stuff done, even if they’re convoluted.

ATDD Frameworks in C#

I know of three main open source ATDD/BDD frameworks that work in C#: SpecFlow, Cuke4Nuke and StoryQ. Only two of these use Gherkin (StoryQ uses its own slightly modified DSL), and I like Gherkin, so I’ll ignore StoryQ for now. Cuke4Nuke still requires some Ruby glue, and to keep things pure, I like SpecFlow as it’s a pure .NET implementation of ATDD with full Gherkin syntax support. For this reason, SpecFlow seems to be the most prevalent ATDD framework in the .NET community and this is what I will use for this example.

SpecFlow as an Acceptance Test Driven Development Framework

SpecFlow is an open source framework that is delivered as a Windows installer, and this installs support for feature and step definition files in Visual Studio through an Add-In that automatically generates unit tests for you from the feature files. Unfortunately, as VS2010 Express doesn’t allow add-ins, SpecFlow won’t work without some tinkering.

Install SpecFlow & NUnit

Before we get started, we need to install VS2010 Express, SpecFlow and NUnit.

  1. Install VS2010 C# Express if you don’t already have it
  2. Install the latest SpecFlow version from the SpecFlow GitHub downloads page.
  3. Install the latest NUnit version from the NUnit download page.
  4. Add the Specflow directory (C:\Program Files\TechTalk\SpecFlow or similar) and the NUnit directory (C:\Program Files\NUnit 2.5.9\bin\net-2.0 or similar) to your Windows path.

Adding SpecFlow templates to Visual Studio 2010 Express

When installing SpecFlow to VS2010 (non-express) it creates templates that can be accessed through the “New Item” menu. Fortunately, it is fairly trivial to add these to VS2010 Express. Just download the templates I have uploaded here (2010, or here (2012) unzip this directory and place the .zip files (SpecFlowFeature.zip, SpecFlowStepDefinition.zip etc.) into your Visual Studio 2010 Express C# templates directory (usually: C:\Users\username\Documents\Visual Studio 2010\Templates\ItemTemplates\Visual C# or similar).

This means you now have access to the SpecFlow templates when adding a new item to your C# project.



Generating SpecFlow NUnit tests from Feature Files

When you add a SpecFlow feature to the full version of VS2010, it automatically generates a unit test file behind the feature file, so that it’s always up to date. This doesn’t work in Express edition, so you have to use the specflow.exe file to do this for you. Fortunately, this can be done in a batch file, and then added as an “external tool” to VS2010 Express, so you get the same functionality, but not quite as neat.

  1. Enable Expert Settings in VS2010 Express (if not already enabled), by choosing Tools->Settings->Expert Settings.
  2. Create a batch file in your project directory called something like “generatespecflowtests.bat” with the following content:

echo Starting to generate SpecFlow unit tests
SpecFlow generateAll %1 /force /verbose
echo Finished generating SpecFlow unit tests
  1. Add an external tool command in VS2010 Express, using Tools->External Tools->Add. Make sure you set the starting directory, and “show output”

  1. You can then generate your tests from the Tools menu or, if you like, you can add this command to a new Toolbar by using Tools -> Customize -> Toolbars -> New, then Tools -> Customize -> Commands -> Toolbar -> Add Command.

Running SpecFlow NUnit tests from Feature Files

Simon Whittemore from London wrote an excellent post about how to automatically run Specflow NUnit tests, and capture a pretty html result, on his blog. I’ve included his batch file here, as it’s what we’ll use to run our tests from within VS2010 Express.

  1. Create a batch file in your project directory named runspecflowtests.bat or similar.
  2. Include Simon’s content (below) in your batch file.
 
@echo off
nunit-console %1
specflow.exe nunitexecutionreport %2 /xmlTestResult:%3
if NOT %errorlevel% == 0 (
echo "Error generating report - %errorlevel%"
GOTO :exit
)
if %errorlevel% ==0 TestResult.html
:exit
  1. Create an external tool setting the same as we did for the generate unit tests, but with the parameters defined on this dialog:
    1. Command: Link to your batch file
    2. Arguments: $(TargetName)$(TargetExt)$(ProjectDir)$(ProjectFileName)$(BinDir)TestResult.xml
    3. Initial Directory: $(BinDir)
    4. Use Output Window: checked

  1. Optionally add this to your VS2010 Express Toolbar as above

When you run this command in VS2010, all SpecFlow features are run, and the resulting report is launched in your default web browser.

Summary

What I have shown is that although Microsoft makes it very hard, it is possible to use VS2010 Express and open source tools like SpecFlow so that you can polish your skills in your own time.

In this theme of ATDD using C# and .NET, over the coming days I will demonstrate how you can use SpecFlow to write and run automated web tests against a web browser using the open source tool WebDriver. Stay tuned!

Update: 20 Feb 2011

The original link to the templates was wrong, I have now updated it to the correct link.

Update 29 Sep 2013

Yannick de Kercadio has kindly provided VS2012 templates here.

 

Reducing step explosion by decoupling Cucumber steps from your GUI

Yesterday I wrote about creating a simple page object pattern framework in Cucumber that uses Watir to drive browsers.

Ben Biddington left an excellent comment:

“The thing I like about this pattern is you could run the feature against a completely different search engine by just creating a new page object impl.

This prevents the common “step explosion” problem all too common with cucumber suites.”

Ben highlights an important point, the page object pattern decouples your cucumber steps from your GUI, helping to reduce “step explosion”.

To demonstrate this concept, I very easily modified my feature I wrote yesterday to test both Google and Bing, by using the same step definitions, but using scenario outlines instead of scenarios.

Feature: Internet Search
  As a casual internet user
  I want to find some information about watir, and do a quick conversion
  So that I can be knowledgeable being

  Scenario Outline: Search for Watir
    Given I am on the  Home Page
    When I search for "Watir"
    Then I should see at least  results
    Scenarios:
      | search engine | expected number of  |
      | Google        | 100,000             |
      | Bing          |  85,000             |

  Scenario Outline: Do a unit conversion
    Given I am on the  Home Page
    When I convert 10 cm to inches
    Then I should see the conversion result ""
    Scenarios:
      | search engine | as expected                         |
      | Google        | 10 centimeters = 3.93700787 inches  |
      | Bing          | 10 centimetres = 3.937007874 inches |

  Scenario Outline: Do a search using data specified externally
    Given I am on the  Home Page
    When I search for a ridiculously small number of results
    Then I should see at most 5 results
    Scenarios:
      | search engine |
      | Google        |
      | Bing          |

The way I had written my step definitions before weren’t tied to a particular page (besides the naming, which I simply refactored), so all that was needed was to write two new page.rb files with the same methods as the Google pages.

Bing’s home page:

class BingHomePage

  attr_accessor :search_field, :bing_search_button

  URLS = { :production => "http://www.bing.com/" }

  def initialize(browser)
    @browser = browser
    @search_field         = @browser.text_field(:name => "q")
    @bing_search_button  = @browser.button(:name => "go")
  end

  def method_missing(sym, *args, &block)
    @browser.send sym, *args, &block
  end

  def visit
    @browser.goto URLS[:production]
  end

  def search_for term
    self.search_field.set term
    self.bing_search_button.click
    bing_results_page = BingResultsPage.new(browser)
    bing_results_page.results.wait_until_present if WEBDRIVER
    bing_results_page
  end

and, Bing’s results page:

class BingResultsPage

  attr_accessor :results, :conversion_result

  def initialize(browser)
    @browser = browser
    @results = @browser.span(:id => "count")
    @conversion_result = @browser.span(:class => "sc_bigLine")
  end

  def method_missing(sym, *args, &block)
    @browser.send sym, *args, &block
  end

  def number_search_results
    result = /^[\s\w-]* of ([\d,]+) results$/.match(@results.text)
    raise "Could not determine number of search results from: '#{@results.text}'" if not result
    result.captures[0].gsub(",","").to_i
  end

end

Once I had these pages in place, I didn’t have to require them or anything else, as this is done by Cucumber, so I could now run my cucumber command and get results from two different search engines: Bing and Google. The feature highlighted some subtle differences in how they work: Google had more hits, Bing uses the Australian spelling of centimetres, and adds one more decimal place of precision to conversions.

Summary

As Ben pointed out, and I have demonstrated, by using a page object pattern, you can decouple your steps from your GUI. This allows you to switch GUIs, or indeed testing methods, which in our involved switching search engines, without changing or adding more step definitions, reducing “step explosion”.

Source Code

I have pushed all changes to GitHub: https://github.com/alisterscott/WatirMelonCucumber

Results

My simple Cucumber + Watir page object pattern framework

Introduction

I was very impressed with Jeff Morgan, known as Cheezy, who recently wrote a series of blog posts about how to use Cucumber and Watir, and shared his code on Github.

I love it when people share their ideas this like, so I thought I would share what I have found useful when setting up a very simple Cucumber with Watir (Celerity & Watir-WebDriver) page object pattern framework, and how this compares to what Jeff has proposed.

Show me the code!

Before we begin, I’ll show you my code. It’s all on Github, right now, as we speak, and you can easily fork and clone this repository to play around with it. It uses a simple google search example.

Project Folder Structure

Google Search Feature

Feature: Google Search
  As a casual internet user
  I want to find some information about watir, and do a quick conversion
  So that I can be knowledgeable being

Scenario: Search for Watir
  Given I am on the Google Home Page
  When I search for "Watir"
  Then I should see at least 100,000 results

Scenario: Do a unit conversion
  Given I am on the Google Home Page
  When I convert 10 cm to inches
  Then I should see the conversion result 
            "10 centimeters = 3.93700787 inches"

Scenario: Do a search using data specified externally
  Given I am on the Google Home Page
  When I search for a ridiculously small number of results
  Then I should see at most 5 results

Page Object Pattern

For our example, we have two page objects. The page classes delegate any methods that don’t exist on the page to the Browser object that is passed in from Cucumber. This ensures you can call browser specific methods (.title, .url etc.) at the page object level without duplicating methods.

class GoogleHomePage

  attr_accessor :search_field, :google_search_button

  URLS = { :production => "http://www.google.com/" }

  def initialize(browser)
    @browser = browser
    @search_field         = @browser.text_field(:name => "q")
    @google_search_button = @browser.button(:name => "btnG")
  end

  def method_missing(sym, *args, &block)
    @browser.send sym, *args, &block
  end

  def visit
    @browser.goto URLS[:production]
  end

  def page_title
    @browser.title
  end

  def search_for term
    self.search_field.set term
    self.google_search_button.click
    google_results_page = GoogleResultsPage.new(browser)
    google_results_page.results.wait_until_present if WEBDRIVER
    google_results_page
  end

end

Cucumber Initialization

I have everything to do with initialization in support/env.rb. This initializes a browser once, and then uses the same browser instance for each Cucumber scenario. This is different from Jeff’s hooks.rb file that launches a new browser for every scenario. I have found by reusing the browser, you get much quicker test execution time.

The INDEX_OFFSET is a constant that allows me to use ‘index’ when referring to browser elements across both celerity with its 1-based index and watir-webdriver with its 0-based index.

TEST_DATA_DIR = "./features/test_data"

if ENV["HEADLESS"] then
  require "celerity"
  browser = Celerity::Browser.new
  INDEX_OFFSET = 0
  WEBDRIVER = false
else
  require 'watir-webdriver'
  require 'watir-webdriver/extensions/wait'
  browser = Watir::Browser.new :firefox
  INDEX_OFFSET = -1
  WEBDRIVER = true
end

Before do
  @browser = browser
end

at_exit do
  browser.close
end

Calling the page objects from the steps

As you can see, most of my steps are in a declarative style, so these normally align pretty well to a method on the page object.
I use a given step to create a page object (in our case @google_home_page), and then call methods on this object for when and then steps. I have created a dynamic invocation of the page creation, so it will work for all pages specified in the cucumber step itself. I use the search_for_term method to transition between page objects, as this method returns the new page. As a rule of thumb, I try to keep my step definition code to 1 or 2 lines, and at a fairly high level, which makes it much more readable.

Given /^I am on the (.+)$/ do |page_name|
  @google_home_page = Object.const_get(page_name.gsub(" ","")).new(@browser)
  @google_home_page.visit
end

When /^I search for a? ?"([^"]*)"$/ do |term|
  @google_results_page = @google_home_page.search_for term
end

When /^I search for a?n? ?([^"].+[^"])$/ do |term|
  term = Common.get_search_term_data term
  @google_results_page = @google_home_page.search_for term
end

Then /^I should see at least ([\d,]+) results$/ do |exp_num_results|
  @google_results_page.number_search_results.should >= exp_num_results.gsub(",","").to_i
end

Then /^I should see at most ([\d,]+) results$/ do |exp_num_results|
  @google_results_page.number_search_results.should <= exp_num_results.gsub(",","").to_i
end

When /^I convert (.+)$/ do |conversion_statement|
  @google_results_page = @google_home_page.search_for "convert #{conversion_statement}"
end

Then /^I should see the conversion result "([^"]*)"$/ do |exp_conversion_result|
  @google_results_page.conversion_result.text.should == exp_conversion_result
end

Expressing test data external to the features/pages

Unlike Jeff’s recommendation of using default data on page objects, I prefer to store test data externally to the page objects in YAML files. That way I can describe the test data which is directly used in my Cucumber step. For example:

Scenario: Do a search using data specified externally
  Given I am on the Google Home Page
  When I search for a ridiculously small number of results
  Then I should see at most 5 results

The phrase ridiculously small number of results matches directly to a section of my YAML test data file. This keeps the scenario highly readable, with the detailed data specified in the YAML file.

search terms:
  ridiculously small number of results:
    search term: 'macrocryoglobulinemia marvel'

A common method simply retrieves the appropriate test data, and provides it to the step to use. Note here that, as this step doesn’t contain quotation marks, this implies a description of data, rather than a literal string of data that would contain quotation marks around the string.

When /^I search for a?n? ?([^"].+[^"])$/ do |term|
  term = Common.get_search_term_data term
  @google_results_page = @google_home_page.search_for term
end

Putting it all together
I have set up two profiles in the cucumber.yml file, one is the default that uses Firefox under Watir-Webdriver, and one runs Celerity (headless) and is named ci. It’s then a case of calling "cucumber" or "cucumber -p ci" to run each of these profiles.

I have set up a ./go script that bootstraps the entire process on unix based systems (Mac OSX and Linux).

The final result

Running cucumber provides results as green as a cucumber.

How does this compare to Cheezy’s framework I mentioned?

  • I am using a similar page object pattern to define web application pages, but I haven’t created a helper class to mix into each like he has: I simply use instance variables on a page class to define the GUI elements, instead of creating a series of methods for each object.
  • My page objects also delegate methods to the main Browser object as required, so you can directly call things like .title, .url etc.
  • Instead of using Jeff’s default data pattern, I instead prefer to store this data described in external YAML files instead for maximum flexibility.
  • Jeff talks about using mixins to mix in common objects. I haven’t done this, but it’s easily doable with my framework, and should be done as it scales out.

Summary

Jeff has done an excellent job with his series of posts and code (thank you), and I am hoping here to build on that, and show how it’s easy to set up a simple Cucumber & Watir page object pattern framework.