Your automated acceptance tests needn’t be written in the same language as your system being tested

When selecting which tool to implement for business facing automated acceptance tests, I’ve often heard that it must use the same programming language as the system under test. Whilst this can sometimes work well, often it is better to choose a tool in another language that will ulimately deliver you better success as it can be adopted more passionately. Here are some of the reasons on why I think so.

The simplicity of some other languages mean it’s easier for less technical testers to understand them

Let me show you an example that compares the same test written in C# as in Ruby.

C# WebDriver Example – Inspired by David Burns

using OpenQA.Selenium;
using OpenQA.Selenium.IE;
 
using NUnit.Framework;
 
namespace Selenium.Two.DotNetExample
{
    [TestFixture]
    public class Test_Google
    {
	    IWebDriver driver;
	     
	    [SetUp]
	    public void Setup()
	    {
		    driver = new InternetExplorerDriver();
	    }
	    
	    [TearDown]
	    public void Teardown()
	    {
		    driver.Quit();
	    }
    		
        [Test]
        public void TestSearchGoogleForTheAutomatedTester()
	    {
	        //Navigate to the site
            driver.Navigate().GoToUrl("http://www.google.com");
            //Find the Element and create an object so we can use it
            IWebElement queryBox = driver.FindElement(By.Name("q"));
            //Work with the Element that's on the page
            queryBox.SendKeys("Watir");
			queryBox.SendKeys(Keys.ArrowDown);
            queryBox.Submit();
            //Check that the Title is what we are expecting
            Assert.True(driver.Title.IndexOf("Watir - Google Search") > -1);
        }
    }
}

Ruby Watir Example – written by me

require 'watir-webdriver' #watir, celerity, firewatir
require "test/unit"
class GoogleTest < Test::Unit::TestCase
  def test_google
    b =Watir::Browser.start "www.google.com"
    b.text_field(:name => "q").set "watir"
    b.button(:name, "btnG").click
    assert_equal("Watir - Google Search", b.title)
    b.close
  end
end

I am slightly biased, but I think the ruby example is neater and easier to read, especially to someone with a less technical background. Sure, a C# developer might baulk at this premise, but for most people who don’t know C#, I imagine it’s the case.

Often times developers like learning another programming language to add to their experience
When I’ve been working on automated tests in ruby, I’ve often seen a strong interest in developers wanting to learn ruby and the automated testing framework, as it’s common for developers to specialize in a language (Java or .NET) and then focus solely on that. Don’t get me wrong, this isn’t always the case, but it happens more than you think.

Testers can embrace simpler programming languages such as ruby, and can become the “experts” in that domain

I’ve seen testers pick up a programming language like ruby and polish their skills in it, which helps their morale by becoming experts in it. If the automated acceptance tests were written in the same language as the developers, yes this brings benefits, but the testers might not become experts as quickly, and may lack ownership of the tests.

Automated acceptance test tools provide different mileage across languages
Some languages have particularly good tools for defining business facing automated tests, for example, Cucumber’s support for ruby is superb, whereas support for C# equivalents such as Cuke4Nuke and SpecFlow isn’t quite as good. RSpec is another example of a tool that is awesome in its domain.

Summary

I know this post has ended up sounding like a pro-ruby rant, but it is :) I am trying to point out that there’s simpler, easier languages for testers to pick up than your Javas and C-Sharps. The ruby and python programming languages are two great examples, and as I mentioned, it seems that ruby is quickly becoming the testing language of choice, because of its simplicity, and the wide variety of testing related gems available. Ruby also makes it extremely easy to bootstrap configurations across different environments, and its lack of licensing requirements or need for an IDE make it an excellent choice for testers.

One of the reasons that the Selenium project is so successful is that it supports various programming languages (and browsers), which appeals to the majority of developers who can use their established skills, but it’s common to meet testers who are passionate about Watir, because of its simplicity and focus: a tool for testers by testers. It is for this reason I believe Ruby & Watir & Cucumber is a winning combination, and will bring about many success stories for agile testing teams in the future.

Reducing Cucumber page object element duplication using mixins

The one thing that quickly happens when you start using a page object pattern to develop your Cucumber acceptance tests is you end up with duplicate page elements in multiple page objects. This happens because usually your pages often have elements common to every page and it doesn’t make sense defining these more than once, enter page mixins.

Page mixins are methods in ruby that you can mix into a class, so that each method in your mixin is then available in the instance of your class. We can use a mixin to define our common elements which are then available for every class we define.

For example, in our Google/Bing search example, the search text fields are conveniently given the same id in each of the search engine pages, therefore it makes sense to define this once and mix it in to each page.

We create a mixin file which is essentially a module with a bunch of methods. In our case, we define initialize to define our elements, and method missing as this is a common method we use for Browser object delegation.

module PageMixIn
  attr_accessor :search_field

  def initialize *args, &block
    @search_field = self.text_field(:name => "q")
  end

  def method_missing sym, *args, &block
    @browser.send sym, *args, &block
  end
end

We then simply include this mixin into our page class, and the methods are mixed in. The super statement in the class’s initialize method is needed to ensure the instance variables of the mixin are available in the class.

class GoogleHomePage
  include PageMixIn

  attr_accessor :search_field, :google_search_button

  URLS = { :production => "http://www.google.com/" }

  def initialize(browser)
    @browser = browser
    @google_search_button = @browser.button(:name => "btnG")
    super
  end

  def visit
    @browser.goto URLS[:production]
  end

  def search_for term
    self.search_field.set term
    self.google_search_button.click
    google_results_page = GoogleResultsPage.new(@browser)
    google_results_page.results.wait_until_present if WEBDRIVER
    google_results_page
  end
end

This means we eliminate a lot of duplication of page elements, such as the search field, as well as common methods such as the method_missing.

Summary and a quick tip

I have found using page mixins provides a flexible approach to reducing page object element and method duplication. It makes your pages easy to read and compact. A quick tip though: I thoroughly recommend storing all page objects and page mixins under the support directory in Cucumber, as this ensures they are loaded automatically by Cucumber and are consistently available to reference.

Reducing step explosion by decoupling Cucumber steps from your GUI

Yesterday I wrote about creating a simple page object pattern framework in Cucumber that uses Watir to drive browsers.

Ben Biddington left an excellent comment:

“The thing I like about this pattern is you could run the feature against a completely different search engine by just creating a new page object impl.

This prevents the common “step explosion” problem all too common with cucumber suites.”

Ben highlights an important point, the page object pattern decouples your cucumber steps from your GUI, helping to reduce “step explosion”.

To demonstrate this concept, I very easily modified my feature I wrote yesterday to test both Google and Bing, by using the same step definitions, but using scenario outlines instead of scenarios.

Feature: Internet Search
  As a casual internet user
  I want to find some information about watir, and do a quick conversion
  So that I can be knowledgeable being

  Scenario Outline: Search for Watir
    Given I am on the  Home Page
    When I search for "Watir"
    Then I should see at least  results
    Scenarios:
      | search engine | expected number of  |
      | Google        | 100,000             |
      | Bing          |  85,000             |

  Scenario Outline: Do a unit conversion
    Given I am on the  Home Page
    When I convert 10 cm to inches
    Then I should see the conversion result ""
    Scenarios:
      | search engine | as expected                         |
      | Google        | 10 centimeters = 3.93700787 inches  |
      | Bing          | 10 centimetres = 3.937007874 inches |

  Scenario Outline: Do a search using data specified externally
    Given I am on the  Home Page
    When I search for a ridiculously small number of results
    Then I should see at most 5 results
    Scenarios:
      | search engine |
      | Google        |
      | Bing          |

The way I had written my step definitions before weren’t tied to a particular page (besides the naming, which I simply refactored), so all that was needed was to write two new page.rb files with the same methods as the Google pages.

Bing’s home page:

class BingHomePage

  attr_accessor :search_field, :bing_search_button

  URLS = { :production => "http://www.bing.com/" }

  def initialize(browser)
    @browser = browser
    @search_field         = @browser.text_field(:name => "q")
    @bing_search_button  = @browser.button(:name => "go")
  end

  def method_missing(sym, *args, &block)
    @browser.send sym, *args, &block
  end

  def visit
    @browser.goto URLS[:production]
  end

  def search_for term
    self.search_field.set term
    self.bing_search_button.click
    bing_results_page = BingResultsPage.new(browser)
    bing_results_page.results.wait_until_present if WEBDRIVER
    bing_results_page
  end

and, Bing’s results page:

class BingResultsPage

  attr_accessor :results, :conversion_result

  def initialize(browser)
    @browser = browser
    @results = @browser.span(:id => "count")
    @conversion_result = @browser.span(:class => "sc_bigLine")
  end

  def method_missing(sym, *args, &block)
    @browser.send sym, *args, &block
  end

  def number_search_results
    result = /^[\s\w-]* of ([\d,]+) results$/.match(@results.text)
    raise "Could not determine number of search results from: '#{@results.text}'" if not result
    result.captures[0].gsub(",","").to_i
  end

end

Once I had these pages in place, I didn’t have to require them or anything else, as this is done by Cucumber, so I could now run my cucumber command and get results from two different search engines: Bing and Google. The feature highlighted some subtle differences in how they work: Google had more hits, Bing uses the Australian spelling of centimetres, and adds one more decimal place of precision to conversions.

Summary

As Ben pointed out, and I have demonstrated, by using a page object pattern, you can decouple your steps from your GUI. This allows you to switch GUIs, or indeed testing methods, which in our involved switching search engines, without changing or adding more step definitions, reducing “step explosion”.

Source Code

I have pushed all changes to GitHub: https://github.com/alisterscott/WatirMelonCucumber

Results

My simple Cucumber + Watir page object pattern framework

Introduction

I was very impressed with Jeff Morgan, known as Cheezy, who recently wrote a series of blog posts about how to use Cucumber and Watir, and shared his code on Github.

I love it when people share their ideas this like, so I thought I would share what I have found useful when setting up a very simple Cucumber with Watir (Celerity & Watir-WebDriver) page object pattern framework, and how this compares to what Jeff has proposed.

Show me the code!

Before we begin, I’ll show you my code. It’s all on Github, right now, as we speak, and you can easily fork and clone this repository to play around with it. It uses a simple google search example.

Project Folder Structure

Google Search Feature

Feature: Google Search
  As a casual internet user
  I want to find some information about watir, and do a quick conversion
  So that I can be knowledgeable being

Scenario: Search for Watir
  Given I am on the Google Home Page
  When I search for "Watir"
  Then I should see at least 100,000 results

Scenario: Do a unit conversion
  Given I am on the Google Home Page
  When I convert 10 cm to inches
  Then I should see the conversion result 
            "10 centimeters = 3.93700787 inches"

Scenario: Do a search using data specified externally
  Given I am on the Google Home Page
  When I search for a ridiculously small number of results
  Then I should see at most 5 results

Page Object Pattern

For our example, we have two page objects. The page classes delegate any methods that don’t exist on the page to the Browser object that is passed in from Cucumber. This ensures you can call browser specific methods (.title, .url etc.) at the page object level without duplicating methods.

class GoogleHomePage

  attr_accessor :search_field, :google_search_button

  URLS = { :production => "http://www.google.com/" }

  def initialize(browser)
    @browser = browser
    @search_field         = @browser.text_field(:name => "q")
    @google_search_button = @browser.button(:name => "btnG")
  end

  def method_missing(sym, *args, &block)
    @browser.send sym, *args, &block
  end

  def visit
    @browser.goto URLS[:production]
  end

  def page_title
    @browser.title
  end

  def search_for term
    self.search_field.set term
    self.google_search_button.click
    google_results_page = GoogleResultsPage.new(browser)
    google_results_page.results.wait_until_present if WEBDRIVER
    google_results_page
  end

end

Cucumber Initialization

I have everything to do with initialization in support/env.rb. This initializes a browser once, and then uses the same browser instance for each Cucumber scenario. This is different from Jeff’s hooks.rb file that launches a new browser for every scenario. I have found by reusing the browser, you get much quicker test execution time.

The INDEX_OFFSET is a constant that allows me to use ‘index’ when referring to browser elements across both celerity with its 1-based index and watir-webdriver with its 0-based index.

TEST_DATA_DIR = "./features/test_data"

if ENV["HEADLESS"] then
  require "celerity"
  browser = Celerity::Browser.new
  INDEX_OFFSET = 0
  WEBDRIVER = false
else
  require 'watir-webdriver'
  require 'watir-webdriver/extensions/wait'
  browser = Watir::Browser.new :firefox
  INDEX_OFFSET = -1
  WEBDRIVER = true
end

Before do
  @browser = browser
end

at_exit do
  browser.close
end

Calling the page objects from the steps

As you can see, most of my steps are in a declarative style, so these normally align pretty well to a method on the page object.
I use a given step to create a page object (in our case @google_home_page), and then call methods on this object for when and then steps. I have created a dynamic invocation of the page creation, so it will work for all pages specified in the cucumber step itself. I use the search_for_term method to transition between page objects, as this method returns the new page. As a rule of thumb, I try to keep my step definition code to 1 or 2 lines, and at a fairly high level, which makes it much more readable.

Given /^I am on the (.+)$/ do |page_name|
  @google_home_page = Object.const_get(page_name.gsub(" ","")).new(@browser)
  @google_home_page.visit
end

When /^I search for a? ?"([^"]*)"$/ do |term|
  @google_results_page = @google_home_page.search_for term
end

When /^I search for a?n? ?([^"].+[^"])$/ do |term|
  term = Common.get_search_term_data term
  @google_results_page = @google_home_page.search_for term
end

Then /^I should see at least ([\d,]+) results$/ do |exp_num_results|
  @google_results_page.number_search_results.should >= exp_num_results.gsub(",","").to_i
end

Then /^I should see at most ([\d,]+) results$/ do |exp_num_results|
  @google_results_page.number_search_results.should <= exp_num_results.gsub(",","").to_i
end

When /^I convert (.+)$/ do |conversion_statement|
  @google_results_page = @google_home_page.search_for "convert #{conversion_statement}"
end

Then /^I should see the conversion result "([^"]*)"$/ do |exp_conversion_result|
  @google_results_page.conversion_result.text.should == exp_conversion_result
end

Expressing test data external to the features/pages

Unlike Jeff’s recommendation of using default data on page objects, I prefer to store test data externally to the page objects in YAML files. That way I can describe the test data which is directly used in my Cucumber step. For example:

Scenario: Do a search using data specified externally
  Given I am on the Google Home Page
  When I search for a ridiculously small number of results
  Then I should see at most 5 results

The phrase ridiculously small number of results matches directly to a section of my YAML test data file. This keeps the scenario highly readable, with the detailed data specified in the YAML file.

search terms:
  ridiculously small number of results:
    search term: 'macrocryoglobulinemia marvel'

A common method simply retrieves the appropriate test data, and provides it to the step to use. Note here that, as this step doesn’t contain quotation marks, this implies a description of data, rather than a literal string of data that would contain quotation marks around the string.

When /^I search for a?n? ?([^"].+[^"])$/ do |term|
  term = Common.get_search_term_data term
  @google_results_page = @google_home_page.search_for term
end

Putting it all together
I have set up two profiles in the cucumber.yml file, one is the default that uses Firefox under Watir-Webdriver, and one runs Celerity (headless) and is named ci. It’s then a case of calling "cucumber" or "cucumber -p ci" to run each of these profiles.

I have set up a ./go script that bootstraps the entire process on unix based systems (Mac OSX and Linux).

The final result

Running cucumber provides results as green as a cucumber.

How does this compare to Cheezy’s framework I mentioned?

  • I am using a similar page object pattern to define web application pages, but I haven’t created a helper class to mix into each like he has: I simply use instance variables on a page class to define the GUI elements, instead of creating a series of methods for each object.
  • My page objects also delegate methods to the main Browser object as required, so you can directly call things like .title, .url etc.
  • Instead of using Jeff’s default data pattern, I instead prefer to store this data described in external YAML files instead for maximum flexibility.
  • Jeff talks about using mixins to mix in common objects. I haven’t done this, but it’s easily doable with my framework, and should be done as it scales out.

Summary

Jeff has done an excellent job with his series of posts and code (thank you), and I am hoping here to build on that, and show how it’s easy to set up a simple Cucumber & Watir page object pattern framework.

Watir-WebDriver: A detailed introduction

Update: 22 August 2011: Please see: watirwebdriver.com for a detailed guide to Watir-WebDriver

Update: 22 July 2011: I have updated quite a number of things that have changed since the earlier releases

Watir-WebDriver is a really great tool; Jari Bakken‘s done a really good job of it. There’s not a huge amount on the web about it, how to get it up and running and use it. I’m aiming to fix that here.

For those who don’t know what Watir-WebDriver is, it’s basically a nice Watir (ruby) implementation on WebDriver, so it gives you four browsers (three real, one headless) using one neat API, out of the box. The thing I like about it is you don’t need to use JRuby (like Celerity), which means it plays nice with Cucumber (although Cucumber does work under JRuby).

I’ve written about how Watir, WebDriver and Selenium all fit together before, so this post aims to be a lot more hands-on.

Getting Watir-WebDriver Running

There are essentially two components you need: the Watir-WebDriver ruby gem, and the remote WebDriver server. The remote WebDriver server is only needed if you want to run your tests in headless mode without a real browser (or want to use Opera). You obviously need ruby first but I won’t detail that here. You can find info about ruby versions etc. at watir.com. If you’re on Mac or Linux, I strongly suggest using RVM and bundler.

The Watir-WebDriver ruby gem

It’s a simple matter of opening a command prompt and typing:

  • gem install watir-webdriver (windows); or
  • sudo gem install watir-webdriver (osx or linux – better to use RVM and bundler)

The remote WebDriver Server

This is the slightly tricky part. This is so that WebDriver can run headless without a real browser, and isn’t needed for real browser support (bar Opera). The quickest easiest way to get up and running is to download this java jar file, open a command prompt where you have saved it, and run:

java -jar selenium-server-standalone-2.0b1.jar

You can also specify the startup and max memory allocated, which is handy when running large test suites.

java -Xms1024M -Xmx2048M -jar selenium-server-standalone-2.0a7.jar

You need to have this java server running whenever using WebDriver, but it’s easy enough to bootstrap it.

Update: Jari has pointed out you can run the server programmatically which is even better:

require 'selenium/server'
server = Selenium::Server.new("/path/to/jar", :background => true)
server.start
# run your tests
server.stop

First Impressions

Waiting in Watir-WebDriver doesn’t seem as straightforward as with Watir, probably due to the underlying drivers, but fortunately there’s an inbuilt waiting library to use (also available in Watir 1.6.7+):

  • Watir::Wait.until { ... }: where you can wait for a block to be true
  • object.when_present.set: where you can do something when it’s present
  • object.wait_until_present:; where you just wait until something is present
  • object.wait_while_present:; where you just wait until something disappears

Hello Watir-WebDriver

It seems pertinent to start with a google search example.

Hello Watir-WebDriver in three browsers

These three browsers seem to work very similarly, but obviously Internet Explorer will only run on Microsoft Windows.

require 'rubygems'
require 'watir-webdriver'
b = Watir::Browser.new :chrome
b.goto 'www.google.com'
b.text_field(:name => 'q').set 'Watir-WebDriver'
b.button(:name => 'btnG').click
b.div(:id => 'resultStats').wait_until_present
puts "Displaying page: '#{b.title}' with results: '#{b.div(:id => "resultStats").text}'"
b.close

The only difference for Firefox:

b = Watir::Browser.new :firefox

The only difference for IE:

b = Watir::Browser.new :ie

Hello Watir-WebDriver in Headless (HTML Unit)

I imagine this is the most anticipated feature for Watir users, as it means your tests run much faster, and you can still use ruby (unlike Celerity which uses JRuby). The script is fairly straightforward, you simply specify the hostname for your WebDriver server you started above. You need to specifically enable JavaScript by creating a capabilities profile.

require 'rubygems'
require 'watir-webdriver'
include Selenium
capabilities = WebDriver::Remote::Capabilities.htmlunit(:javascript_enabled => true)
b = Watir::Browser.new(:remote, :url => 'http://127.0.0.1:4444/wd/hub', :desired_capabilities => capabilities)
b = Watir::Browser.new :firefox
b.goto "www.google.com"
b.text_field(:name => "q").set "Watir-WebDriver"
b.button(:name => "btnG").click
b.div(:id => "resultStats").wait_until_present
puts "Displaying page: '#{b.title}' with results: '#{b.div(:id => "resultStats").text}'"
b.close

Sample Timings

As an experiment, I looped the Google Search script above 100 times to measure and compare the execution times.

  • Internet Explorer 8 on Windows 7: 400 seconds
  • Firefox 3.6 on Mac OSX: 277 seconds
  • Headless HTMLUnit on Mac OSX: 269 seconds

Very surprisingly, the headless run wasn’t much quicker at all. This may be due to running the test on Google over the Internet and not a local application.

Caveat Emptor

There are a number of differences between Watir-WebDriver and Watir. The main ones that are important to me are:

Zero based indexing as opposed to 1 based

For example,

In Watir/FireWatir: this finds the first table on a page:

puts b.table(:index => 1).text

But in Watir-WebDriver, it finds the second table on a page.

Attaching to windows
Attaching to new windows (for example pop-ups) has been fairly straightforward in Watir with its attach method, which is missing in Watir-WebDriver. It looks like you can iterate through a collection of windows in Watir-WebDriver using browser.windows, but I haven’t tried this out yet.

Summary

Watir-WebDriver is a very solid testing tool that uses a great browser automation engine (WebDriver) with a clean ruby API (Watir). I believe it will be a popular choice amongst testing teams, particually those using it for ATDD through Cucumber and running it as headless on a continuous integration server.

Further Reading

Cucumber: imperative or declarative? that is the question

I’ve recently been putting some amount of thought into how to write good Cucumber features. One the reasons I love Cucumber so much is its language focus; meaning it makes you think about things like sentence function more than other testing frameworks I have used. One element of sentence function usage in Cucumber that’s important is whether you should write imperative (communicative) or declarative (informative) sentences for your steps. I’ll provide a couple of examples to show the differences.

An imperative (communicative) Cucumber scenario

Scenario: Submit the form with correct data (imperative)
  Given I am on the watir example form
  When I set "What is your name?" to "Alister"
  And I set "What is your story?" to "I like ruby"
  And I set "What testing tool do you like?" to "Watir"
  And I click "Submit"
  Then I should see the message "Thanks! Your response has been recorded."

Each step is a command, or instruction, so it’s pretty clear what is being done.

A declarative (informative) Cucumber scenario

Scenario: Submit the form with correct data (declarative)
  Given I am on the watir example form
  When I submit the form with correct data
  Then I should see the correct message

Each step states an idea, so sometimes it’s not really clear precisely what is being done.

Imperative vs Declarative

There’s some quite good articles about writing Cucumber features/scenarios around. One such article that stood out to me was written in May 2008 about RSpec Story Runner, and whether you should use an imperative or declarative style. The answer is, of course, it depends, but most people tend to lean towards the declarative style, just the way that a lot of programmers prefer declarative programming. I am actually swinging towards imperative style cucumber features and steps; the reasons for which I will explain below.

Running Imperative Features gives you richer output

With the two examples above, I have kinda cheated, in that the step definitions for the declarative style calls imperative steps. For example, this is the declarative step defintion:

When /^I submit the form with correct data$/ do
  When "I set \"What is your name?\" to \"Alister\""
  And "I set \"What is your story?\" to \"I like ruby\""
  And "I set \"What testing tool do you like?\" to \"Watir\""
  And "I click \"Submit\""
end

This is all well and good, as you can now have multiple cucumber steps that call the same thing, it’s DRYish, but the problem is when you run the step “When I submit the form with correct data”, that’s all you get.

Compare this to the imperative step results, which whilst calling exactly the same code, give you a lot more richness.

Writing imperative scenarios allow you to utilise things advanced cucumber functionality

Two table based Cucumber features I like using are scenario outlines, and multiline step arguments, but these don’t make a lot of sense if you’re using a declarative style.
For example, the imperative scenario could be easily adapted to use a scenario outline, whereas the declarative style would mean writing more declarative steps.

Scenario Outline: Submit the form with correct data (imperative - outline)
  Given I am on the watir example form
  When I set "What is your name?" to "<name>"
  And I set "What is your story?" to "<story>"
  And I set "What testing tool do you like?" to "<tool>"
  And I click "Submit"
  Then I should see the message "Thanks! Your response has been recorded."

  Examples:
   | name    | story       | tool     |
   | Alister | I Like ruby | Watir    |
   | Mozart  | I like java | Selenium |

Writing declarative scenarios often means hidden detail

As you move to more and more declarative scenarios, more and more details become hidden in cucumber step definitions and library code. As Rasmus pointed out on Jared Quinert’s blog on does Cucumber suck:

…tests end up looking like the one test to rule them all™:
Given everything is set up
When I submit correct data
Then everything should be OK

Summary

To wrap up, I’m not saying don’t go declarative, what I am saying is that you lose certain things in Cucumber when you do, but whether this is a big deal or not depends on what you’re trying to get out of it. In the meantime, I’ll continue to experiment and see what works and what doesn’t.

Elegantly handling basic browser authentication with Watir

In programming, I find there’s little that compares to the feeling of deleting code; like finding a one or two-line solution that means you can remove a good-sized chunk of code no longer needed.

A fellow ThoughtWorker (who I shall call the Awesome-est Tom™) showed me a solution to handling the basic browser authentication in Internet Explorer, which effectively meant I could delete lines and lines of ruby code that called the AutoIt dll to handle the modal authentication dialog.

Internet Explorer Basic Authentication Dialog

Internet Explorer Basic Authentication Dialog

The code I had was loosely based upon the solutions available on the Watir wiki, all which roughly use the same technique of locating and authenticating the authentication dialog using AutoIt. Because the dialog is modal, the code I had used to run in a separate thread, and this caused me problems with debugging, and I find if I can avoid AutoIt, then my Watir tests run a lot more reliably.

Tom’s Occam’s Razor solution simply embeds the username and password in the URL that usually triggers an authentication dialog.

Instead of using:

Watir::Browser.start "http://yoururl.com"

you use:

Watir::Browser.start "http://username:password@yoururl.com"

This means you don’t see the authentication dialog at all! No more messy AutoIt code. The only tricky part is ensuring that Internet Explorer allows you to do this. Since Internet Explorer 6, this has been disabled by default, but it’s a simple matter of enabling the functionality by setting two registry keys.

This can easily be done running a .reg file you can create (I’ve uploaded mine ‘IESecurityURL.reg’ here), that sets the required values:

Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_HTTP_USERNAME_PASSWORD_DISABLE]
"iexplore.exe"=dword:00000000
"explorer.exe"=dword:00000000

You then run this file from the command line: regedit.exe /s IESecurityURL.reg and Robert is your father’s brother.

I’ve updated the wiki with this solution, so hopefully everyone can now use this and avoid the nastiness that is AutoIt :)

Rubular & RubyMine: makes Cucumber easier

I’ve spent the last couple of months establishing Acceptance Test Driven Development on a medium sized software delivery project using Watir & Cucumber.

Cucumber is premised on reading feature files and matching strings to determine what to do, which is done by using regular expressions in step files. This means constant use of ruby regular expressions, and in polishing up on my regular expression skills, I have found rubular incredidly useful. I love the simple layout and the ‘Regex Quick Reference’ at the bottom, just where you need it. It’s very well done.

Rubular: A Ruby regular expression editor and tester

Rubular: A Ruby regular expression editor and tester

The other tool I have found incredibly useful is RubyMine. Up until now, I’ve never really bothered with an IDE for Watir stuff. I’ve mainly stuck to text editors before, but since I have started using Cucumber, I have found the RubyMine tool almost critical, as its support for Cucumber feature files and step definitions is superb. It features click through linking for feature files so you know exactly what step you’re calling, and the debugger is awesome; no more puts statements for me :)

So, if you’re thinking about implementing Cucumber, or using Cucumber but are annoyed with the lack of efficiency in managing a large suite of step definitions, I would thoroughly recommend these two tools. Rubular is free, and I believe RubyMine is about $99, but less if you need multiple licenses. There’s also an Early Access Program where you can use RubyMine for free as long as you’re happy to test it along the way.

Running Watir reliably on a CI machine

I’ve found that Watir generally works well when running under Windows on a Continuous Integration machine (for example, Hudson or TeamCity) but if you use any of the autoit stuff to handle things like security authentication dialogs, it can fail when the machine is locked or not actively logged in.

I’ve found two ways to ensure Watir reliably runs tests without issues.

Run Caffeine to stop the machine locking or screensaver activating

Your Windows sys-admin might not like this one, but caffeine is a tiny executable that’s perfect for ensuring your Windows machine never locks or activates its screensaver, by simulating a keypress every 59 seconds. It’s a matter of installing it and a coffee cup appears in the tray to show you you’re machine is now on caffeine!

Use a command to release any remote desktop sessions to console

Caffeine works well, but if you use remote desktop to control the machine, when you disconnect, the machine will be locked and this will cause problems. What you need tp do is create a batch file, on your desktop say, that releases the RDP connection to console, so it’s ready to run any watir tests. The command you’ll need in the batch file is:

cmd.exe /Q /C "%windir%\System32\tscon.exe 0 /dest:console"

By using the combination of the above items, it’s easy to set up a reliable suite of Watir tests to run on a CI machine that you can easily remote into to see what is running when needed.

My thoughts on Brisbane ANZTB SIGIST: September 2010

I went along to the ANZTB SIGIST last night, at the Hilton here in Brisbane. It was probably the best one I’ve been to, both in attendance, and caliber of presenters. There were five presenters all up, which is a lot to squeeze into two hours including drinks and conversation. It’s no surprise then that the last presentation by Craig Aspinall was a little bit rushed, which is a shame because I would have liked to ask more questions than time allowed (apparently the Hilton would kick us out if we stayed longer).

Craig Smith and Rene Masten from Suncorp began with an excellent presentation on Agile Testing. Craig was awesome in both presenting his extensive knowledge (he’s a coach) and his slides were also very cool (minimal text and no bullets!).

Craig’s main theme was about how to make testing cool: ensuring people say “that’s cool” when you tell them about your testing. He talked about what makes up an agile team, the techical divide between devs and testers, ATDD, ensuring you deliver, and how there is currently a great opportunity for testers (who want to be bothered). Rene followed by talking about organizational change, training and coaching, communicating what you’re doing (internally and externally) and building quality in.

Craig finished with a great motivating style of emphasizing it’s all about passion and craft, “who’s awesome!”, and not being afraid of technical challenges.

The other presenters were Ben Sullivan and Brent Acworth, both also from Suncorp, who gave a demo of BDD using JBehave and Hudson. I enjoyed the talk but there wasn’t much I hadn’t seen before or didn’t know.

The final presentation was by Craig Aspinall and was unfortunately squeezed into a small time slot. It was about what Craig dubs “Automated Black Blob Testing”, the rationale being testing is not black box as a box has a predefined shape, it’s more of a blob.

Craig’s approach looked solid, although I was slightly concerned when he mentioned the project being a SaaS solution and how much effort was being put into automated testing.  I’m not criticizing what Craig did tehnically, I am just concerned about the prevalent practice of the onus of testing SaaS solutions being put onto the customer. I believe if you buy a SaaS, you should get a working SaaS, minimal, if any at all, testing required. But that’s just me. Otherwise, a great talk, besides Craig using Java when Ruby and Watir would have done the trick. ;)

A great ANZTB SIGIST, and hopefully more good ones to come!

Craig Smith’s Slides

Some of my photos

This slideshow requires JavaScript.