WatirMelon

Background Information

Defining test cases in a wiki has many benefits:

• It’s easy to write, read and update;
• It’s centrally located and easily accessible;
• There’s in-built version control.

I know that FitNesse/Faucets is a wiki built for this purpose, but I have used it before and have a few issues with it:

• It defines the test scripts in wiki tables which means it’s difficult to write logic;
• The FitNesse wiki functionality is basic compared to other wiki software available;
• I couldn’t find a way to store the historical results of test runs; and
• Most organisations already have a enterprise wiki in place so having another wiki is duplicating effort.

Its just as easy to use your enterprise wiki and its API to run your Watir tests, and then write back the results (but that’s another post).

My favourite enterprise wiki software is Confluence, not just because it’s a rock solid product but also because I love Atlassian’s philosophies. This includes providing the software to open source projects, such as Watir, and also providing free personal licenses which I have used to provide this example.

Writing Watir tests

I have already discussed how I usually organise my Watir tests. In this example I am using three methods I have already written for the Depot Rails Application.

Customer.add_book(bookTitle)
Customer.check_out(customerName, customerEmail, customerAddress, customerPaymentMethod)
Customer.empty_cart()

The full code for each method is at the bottom of this post.

Defining test cases in Confluence

I have created a very basic wiki page that defines some test cases in wiki tables. In this example I have three different tests, but in reality I would have more permutations in each table. (Click image for large version)

Note that each test case has a Function, Test and test data associated with it. The reason the test data is on the right hand side of the tables is to support varying amounts of test data for various test cases. Also note that the table supports both positive tests and negative test cases (testing for errors).

Another benefit of using a wiki page is that you can easily add additional information. For example, you may have a manual verification step at the end of your tests that you can also document in the same wiki page: it needn’t be in a different place. You can even insert pictures and attachements.

Writing a basic script to run Watir tests defined in Confluence

After a wiki page has been created it’s a matter of writing some ruby code to parse the page and run the tests. Fortunately there is already an excellent ruby library already written for this purpose: Confluence4r.

So really it is just a matter of reading each line in the wiki page and looking for a line that is a table row (not a header), determining which ruby method to run and running it.

require 'watir'
require './confluence4r.rb' # for wiki
require './Customer.rb'
require './Common.rb'

server = Confluence::Server.new('http://localhost:8080')
server.login('admin', 'password')
wiki_page = server.getPage("Watir","WatirMelon")
page_content = wiki_page['content']
page_content.each do |line|
    if line =~ /^\|{1}[^\|]/ then # if line begins with a | then we have an executable test.
        begin
        line.strip! # remove newline markers.
        cells = line[1..-1].split('|') #split line by | excluding first character.
        raise 'Not enough parameters.' if cells.length < 4
        cells.each { |cell| cell.strip! }
        module_name = cells[0]
        method_name = cells[1].downcase.strip.sub(' ','_') # automatically determine function name based upon method name.
        comments = cells[2]
        expected_outcome = cells[3]
        expected_error = cells[4]
        parameters = cells[5..-1] # the remaining cells.
        required_module = Kernel.const_get(module_name)
        required_method = required_module.method(method_name)
        arity = required_method.arity() # this is how many arguments the method requires, it is negative if a 'catch all' is supplied.
        arity = ((arity * -1) - 1) if arity < 0
        if cells[5..-1].length != arity then
            raise 'Number of parameters supplied do not match method.'
        end
        actual_outcome, actual_output = required_method.call(*parameters)
        # determine the result.
        if (expected_outcome = 'Success') and actual_outcome then
            result = "PASS"
        elsif (expected_outcome = 'Error') and (not actual_outcome) and (expected_error = actual_output) then
            result = "PASS"
        else
            result = "FAIL"
        end
        puts "\nRunning Test: #{method_name} for #{module_name}."
        puts "Expected Outcome: #{expected_outcome}."
        puts "Expected Error: #{expected_error}."
        puts "Actual Outcome: #{actual_outcome}."
        puts "Actual Output: #{actual_output}."
        puts "RESULT: #{result}"
        rescue
            puts "An error occurred: #{$!}"
        end
    end
end

Note that I have dynamically determined the module and method name from the wiki and I have then dynamically called it with the appropriate number of arguments using arity.

When I run it the ruby code the output is:

Running Test: empty_cart for Customer.
Expected Outcome: Success.
Expected Error: .
Actual Outcome: true.
Actual Output: .
RESULT: PASS

Running Test: add_book for Customer.
Expected Outcome: Success.
Expected Error: .
Actual Outcome: true.
Actual Output: .
RESULT: PASS

Running Test: check_out for Customer.
Expected Outcome: Success.
Expected Error: .
Actual Outcome: true.
Actual Output: .
RESULT: PASS

This could be substantially improved but it’s a start. For example, we could easily add timings for each test case.

Next Steps

The next step is to dynamically create a child results page on the Confluence wiki for each time you run a test. That way there is a historical record so you can then schedule your tests to run automatically and check the results later on using the wiki.

I’ll be posting some more information and code in the coming weeks on how to do this.

Click below to see the Watir code I wrote and used.

Watir Source Code used in my example.

Customer.rb

module Customer

    URL = 'http://localhost:3000/store/'

    # Description::        Adds a book named 'bookTitle' to cart
    def Customer.add_book(bookTitle)
        browser = Common.find_or_start_browser('Pragprog Books Online Store', URL)
        # Check if title is already in cart
        browser.link(:text,'Show my cart').click

        prevCountInCart = 0
        prevCartTotal = 0.00
        if not browser.div(:text,'Your cart is currently empty').exist? then
            # We have a non-empty cart
            for row in browser.table(:index,1)
                if row[2].text == bookTitle then
                    prevCountInCart = row[1].text.to_i
                    break
                end
            end
            prevCartTotal = browser.cell(:id, 'totalcell').text[1..-1].to_f #remove $ sign
            browser.link(:text, 'Continue shopping').click
        end

        found = false
        1.upto(browser.divs.length) do |index|
            if (browser.div(:index,index).attribute_value('className') == 'catalogentry') and (browser.div(:index,index).h3(:text,bookTitle).exists?) then
                browser.div(:index,index).link(:class,'addtocart').click
                found = true
                break
            end
        end
        if not found then
            return false,'Could not locate title in store'
        end

        newCountInCart = 0
        newCartTotal = 0.00
        for row in browser.table(:index,1)
            if row[2].text == bookTitle then
                newCountInCart = row[1].text.to_i
                break
            end
        end
        newCartTotal = browser.cell(:id, 'totalcell').text[1..-1].to_f # remove $ sign
        # TODO: Assertions around totals

        browser.link(:text, 'Continue shopping').click
        return true,''
    end

    def Customer.check_out(customerName, customerEmail, customerAddress, customerPaymentMethod)
        browser = Common.find_or_start_browser('Pragprog Books Online Store', URL)
        browser.link(:text,'Show my cart').click
        if browser.div(:text,'Your cart is currently empty').exist? then
            return false,'Your cart is currently empty'
        end
        browser.link(:text,"Checkout").click
        # Assert total value
        #b.cell(:id, 'totalcell').text.should == '$59.90'
        browser.text_field(:id, 'order_name').set(customerName)
        browser.text_field(:id, 'order_email').set(customerEmail)
        browser.text_field(:id, 'order_address').set(customerAddress)
        begin
            browser.select_list(:id, 'order_pay_type').select(customerPaymentMethod)
        rescue Watir::Exception::NoValueFoundException
            puts 'WARNING: could not locate customer payment method in drop down list: '+customerPaymentMethod
        end
        browser.button(:name, 'commit').click
        if browser.div(:id,'errorExplanation').exist? then
            error = browser.div(:id,'errorExplanation').li(:index,1).text
            browser.link(:text,'Continue shopping').click
            return false, error
        end
        return true,''
    end

    def Customer.empty_cart()
        browser = Common.find_or_start_browser('Pragprog Books Online Store', URL)
        puts 'INFO: Found Browser OK'
        browser.link(:text,"Show my cart").click
        if not browser.div(:text,"Your cart is currently empty").exist? then
            browser.link(:text,'Empty cart').click
            puts 'OK: Cart is now empty.'
            #TODO Assert message - cart is now empty
        end
        puts 'OK: Cart was never empty.'
        return true,''
    end
end

Common.rb

module Common
    def Common.find_or_start_browser(title, url)
        begin
            browser = Watir::IE.attach(:title, title)
            browser.goto(url)
        rescue Watir::Exception::NoMatchingWindowFoundException
            browser = Common.start_new_browser(url)
        end
        return browser
    end

    def Common.start_new_browser(url)
        browser = Watir::IE.start(url)
        browser.speed = :fast
        return browser
    end

    def Common.close_browsers(titles)
        for title in titles
            Common.close_browser(title)
        end
        return true
    end

    def Common.close_browser(title)
        begin
            browser = Watir::IE.attach(:title, title)
            browser.close
        rescue Watir::Exception::NoMatchingWindowFoundException
        end
        return true
    end

end