I have spent a bit of time over the last few days setting up Watir.com, hosted here on WordPress.

We were originally aiming to host our own version of Confluence and JIRA and use Confluence to serve the Watir.com homepage, but this ended up being a lot more complicated and expensive than originally planned.

The great thing about WordPress is, although it was originally a blogging platform, its functionality also works as a very neat CMS. Whilst wordpress.com has some limitations over wordpress.org, we can live with these limitations for now as we have a free (as in beer) hosted site that the world can see.

Check it out.

I will be presenting at a SIGIST (Special Interest Group in Software Testing) on Watir on May 26 @ the Hilton in Brisbane. Everyone is welcome to attend.

More details here.

• Watif if you don’t want to learn a new language just so you can test your web app?
• Watif you want to kick it old-skool with punch cards?
• Watif you want a fully supported automated test solution running SAS and with in built notifications of results?

Introducing Watif

Web Application Testing in FORTRAN:
web application testing that punches!

Watif is simple

Tests are created using simple, easy to use coding forms, easily followed by business analysts and end users. No more expensive test automation engineers!

Watif Coding Form

Watif is automated

Code is created automatically on punch cards using state of the art FORTRAN compilers, saving you valuable compilation time.

Sample Watif code (automatically generated)

Watif is fully supported

Watirfort is a new company of 1,000 monkeys, available 24/7 worldwide to commercially support Watif and make it a success in your organization.

Fully supported by Watirfort

Watif is SAS

All your tests are run by Watirfort using state of the art punch card processing systems, just like salesforce.com.

State of the art Watirfort labs

In built notification systems

When you purchase Watif services from Watirfort, you can specifiy how many homing pigeons you would like to lease. These homing pigeons are dedicated to delivering your printed Watif output directly to you! Rapid feedback!

Watirfort Homing Pigeons

Planned Additional Browser Support

While Watif 1.0 only initially supports the WorldWideWeb browser, alternative browsers including Netscape Navigator 1.0 are planned for Watif 2.0.

WorldWideWeb fully supported!

Quotes

“I wanted to run around my office punching my hands in the air.”   -   Bec Ferguson

Watif is available for immediate release.

Order now and get a bonus FORTRAN book.

I’ve been writing a script to monitor our production app that has been playing up a lot lately. The problem is that the service on the server continues to run but when trying to access the main page it just sits there loading indefinitely.

The idea is to write a Watir script that brings up the main page every minute and notifies us if it displays an error, or, as has been happening, sits there loading indefinitely.

One of the things I love about Watir is how it handles browser synchronization: it’s really neat. To quote Bret’s design objective:

“Watir is deterministic. Watir does not wait X seconds. It waits until the page is loaded. Period.”
- Bret Pettichord

So this is great when you’re writing a test script, but as my page sometimes loads indefinitely, so does my Watir script:

require 'watir'
check_url = 'www.google.com'
ie = Watir::IE.new()
ie.goto(check_url)
puts ie.check_for_http_error()

I did some research and found a ruby Timeout class that I figured I could use. My first attempt was to do something like this:

require 'watir'
check_url = 'www.google.com'
ie = Watir::IE.new()
begin</pre>
Timeout::timeout(30) do
 ie.goto(check_url)
 puts ie.check_for_http_error()
 end
rescue
 puts 'timed out'
end

This seemed to work when the page didn’t timeout, but for some reason it wasn’t catching the timeout. So I did a bit of digging and found out a rescue clause in ruby with no following class only catches exceptions of type StandardError, a subclass of Exception. So it wasn’t catching the Timeout::Error exception. The way to catch this exception is to always include the Exception clause, and to pass it to a variable such as ‘e’.

The final script looked something like:

require 'watir'

check_url = 'www.google.com'
ie = Watir::IE.new()
begin
 Timeout::timeout(30) do
 ie.goto(check_url)
 puts ie.check_for_http_error()
 end
rescue Exception => e
 puts 'timed out: '+e
end

In my research on Timeout::timeout I found this article that explains how it’s actually dangerous to use Timeout::timeout, but until I figure out a better solution it seems to work pretty well for me.

I conducted a survey for the Austin Workshop on Test Automation (AWTA) to see what people thought was good about the workshop and what could be improved in the future.  The response was very positive.

Whilst there were twenty-one questions, I believe the following two graphs tell the story:

How much fun did you have at AWTA 2009?

Would you attend another AWTA?

The full results are available here. Bret also did a nice writeup of the AWTA 2009 proceedings here.

It’s been about a week since AWTA 2009 in Texas, and since I had a 30+ hour journey home I think I am excused for not posting this redux until now.

I had an amazing time at both the workshop and also doing things outside the workshop with the people I met there. Everyone was super friendly and make me feel very at home.

At the Workshop

I enjoyed talking about what people expect in a test automation framework. It’s great to see that the list of things is slowly changing as the ways that people work change. For example, continous integration and running ‘headless’ tests was mentioned quite a bit whereas this probably wouldn’t have even been a requirement for people not too long ago.

All the individual presentations were great. I especially enjoyed MarekJ’s on Watirloo and Dave Hoover’s presentation on Cucumber (code here). I like how Opera are going to use Watir to run web tests on any Opera browser through remote control: nice.

I enjoyed many conversations I had, especially those with Hugh McGowan and Gregg Yows about Rasta as well as a great conversation I had with Pete Dignan and Janet Gregory about ethics and choosing who you work for.

I also shared my experiences in running automated tests in live production environments to many people’s amazement. This is why my ears pricked up when I heard mention the possibility of one day replacing Watir’s engine with that of Selenium. The problem I have with selenium is the server side components make it almost impossible for me to use it. I would have lots of trouble convincing the right people to install server side components for automated testing on public facing web applications in production environments: too risky.

On the final day Pete shared some information about a great non-profit aptly named Charity: Water and what they are doing in Africa to improve sanitation. I couldn’t help but share the slightly non-relevant gapminder.org just because it is too cool not to.

Twitter

Using twitter at a meetup is really fun. You can view all the #awta posts. Someone raised the point of why we don’t just use IRC but since I received a fair amount of interest from outside AWTA via twitter, I think it’s best not to use IRC which wouldn’t have generated this interest.

Lightning Talks

My two lightning talks are both available on Google Docs: talk one and talk two.

Podcasts

Zeljko (aka Z*) was busy recording heaps of podcasts at AWTA. The one I did with him is published here.

Outside the workshop

I had one afternoon to spend in sunny Los Angeles and so I caught a bus to Santa Monica to see the Pacific Ocean from the other side of the world. It was surreal.

Santa Monica Beach, Los Angeles

I enjoyed experiencing a few different places to eat in Austin, Texas. Huts did a mean burger downtown.

Hut's Hamburgers, Austin

Rudy’s BBQ and picnic was superb. We got a full tour of the kitchen including the smoke pits by the manager.

Rudy's BBQ, Austin

I enjoyed going to the Broken Spoke on Friday night for some Honkey Tonk action, and then partaking in an entirely different scene downtown on Saturday night. What variety: wow.

The Broken Spoke, Austin

Late night architecture in downtown Austin

I felt special when MarekJ gave me this sticker from the 2008 US election:

US Election Sticker

and also when Bret signed my Lessons Learned in Software Testing Book:

One the flight home I drafted the AWTA attendee survey on the back of two Air Pacific sick bags. It was then straighforward to put these questions into Google Docs and sent out a link. Man I love pen and paper!

AWTA Sick Bag Survey

AWTA Sick Bag Survey

I happened to notice a singer on the Austin music channel on TV that I really liked. She is from Austin and her name is Suzanna Choffel. Here’s a song live from Austin.

I sure hope I can go again next year!

I spent this evening playing with Roo, the ruby library for reading data from spreadsheets and I am very impressed. In a very small amount of time I was able to define tests in four different forms/places and could execute my tests from each of these:

• An Excel file (.xls): stored locally
• An OpenOffice (.ods): stored locally
• An Excel file (.xls) stored in a Confluence wiki page with Confluence Office Connector; and
• A Google Docs spreadsheet.

The great thing about Roo is that you don’t actually need Excel; Roo simply reads the file, unlike the ruby Excel COM WIN32 API I have used previously.

The spreadsheet (embedded in Confluence) looks like this:

The cool thing about embedding it in Confluence is that you can click the title of the spreadsheet to edit it (in OpenOffice in my case).

I made some minor changes to my existing code that executed my depot tests from a wiki page, and it was as easy as that. A data driven Watir solution with four possible ways to define test cases. Cool.

You can find all the code needed below.

require 'watir'
require 'rubygems'
require 'roo'
require './Customer.rb'
require './Common.rb'

case ARGV[0]
when "excel"
	ss = Excel.new("watirmelon.xls")
when "wiki"
	ss = Excel.new("http://localhost:8080/download/attachments/2097153/watirmelon.xls")
when "gdocs"
	ss = Google.new("http://spreadsheets.google.com/ccc?key=pEcLrW3b2djraE8JF_2fJWA")
else
	ss = Openoffice.new("watirmelon.ods")
end

ss.default_sheet = ss.sheets.first
ss.first_row.upto(ss.last_row) do |line|
	if ss.cell(line,1).strip != "Function" then #We have an executable test
		begin
			module_name = ss.cell(line,1).strip
			method_name = ss.cell(line,2).downcase.strip.sub(' ','_') # automatically determine function name based upon method name.
			comments = ss.cell(line,3).strip
			expected_outcome = ss.cell(line,4).strip
			expected_error = ss.cell(line,5).strip
			required_module = Kernel.const_get(module_name)
			required_method = required_module.method(method_name)
			arity = required_method.arity() # this is how many arguments the method requires, it is negative if a 'catch all' is supplied.
			arity = ((arity * -1) - 1) if arity < 0
			parameters = []
			1.upto(arity) do |p|
				parameters.push(ss.cell(line,p+5))
			end
			actual_outcome, actual_output = required_method.call(*parameters)
			# determine the result.
			if (expected_outcome = 'Success') and actual_outcome then
			    result = "PASS"
			elsif (expected_outcome = 'Error') and (not actual_outcome) and (expected_error = actual_output) then
			    result = "PASS"
			else
			    result = "FAIL"
			end
			puts "\nRunning Test: #{method_name} for #{module_name}."
			puts "Expected Outcome: #{expected_outcome}."
			puts "Expected Error: #{expected_error}."
			puts "Actual Outcome: #{actual_outcome}."
			puts "Actual Output: #{actual_output}."
			puts "RESULT: #{result}"
		rescue
			puts "An error occurred: #{$!}"
		end
	end
end

See the full test code below the break.

Common.rb

module Common
    def Common.find_or_start_browser(title, url)
        begin
            browser = Watir::IE.attach(:title, title)
            browser.goto(url)
        rescue Watir::Exception::NoMatchingWindowFoundException
            browser = Common.start_new_browser(url)
        end
        return browser
    end

    def Common.start_new_browser(url)
        browser = Watir::IE.start(url)
        browser.speed = :fast
        return browser
    end

    def Common.close_browsers(titles)
        for title in titles
            Common.close_browser(title)
        end
        return true
    end

    def Common.close_browser(title)
        begin
            browser = Watir::IE.attach(:title, title)
            browser.close
        rescue Watir::Exception::NoMatchingWindowFoundException
        end
        return true
    end

end
 

Customer.rb

module Customer

    URL = 'http://localhost:3000/store/'

    # Description::        Adds a book named 'bookTitle' to cart
    def Customer.add_book(bookTitle)
        browser = Common.find_or_start_browser('Pragprog Books Online Store', URL)
        # Check if title is already in cart
        browser.link(:text,'Show my cart').click

        prevCountInCart = 0
        prevCartTotal = 0.00
        if not browser.div(:text,'Your cart is currently empty').exist? then
            # We have a non-empty cart
            for row in browser.table(:index,1)
                if row[2].text == bookTitle then
                    prevCountInCart = row[1].text.to_i
                    break
                end
            end
            prevCartTotal = browser.cell(:id, 'totalcell').text[1..-1].to_f #remove $ sign
            browser.link(:text, 'Continue shopping').click
        end

        found = false
        1.upto(browser.divs.length) do |index|
            if (browser.div(:index,index).attribute_value('className') == 'catalogentry') and (browser.div(:index,index).h3(:text,bookTitle).exists?) then
                browser.div(:index,index).link(:class,'addtocart').click
                found = true
                break
            end
        end
        if not found then
            return false,'Could not locate title in store'
        end

        newCountInCart = 0       
        newCartTotal = 0.00
        for row in browser.table(:index,1)
            if row[2].text == bookTitle then
                newCountInCart = row[1].text.to_i
                break
            end
        end
        newCartTotal = browser.cell(:id, 'totalcell').text[1..-1].to_f # remove $ sign
        # TODO: Assertions around totals

        browser.link(:text, 'Continue shopping').click
        return true,''
    end

    def Customer.check_out(customerName, customerEmail, customerAddress, customerPaymentMethod)
        browser = Common.find_or_start_browser('Pragprog Books Online Store', URL)
        browser.link(:text,'Show my cart').click
        if browser.div(:text,'Your cart is currently empty').exist? then
            return false,'Your cart is currently empty'
        end
        browser.link(:text,"Checkout").click
        # Assert total value
        #b.cell(:id, 'totalcell').text.should == '$59.90'
        browser.text_field(:id, 'order_name').set(customerName)
        browser.text_field(:id, 'order_email').set(customerEmail)
        browser.text_field(:id, 'order_address').set(customerAddress)
        begin
            browser.select_list(:id, 'order_pay_type').select(customerPaymentMethod)
        rescue Watir::Exception::NoValueFoundException
            puts 'WARNING: could not locate customer payment method in drop down list: '+customerPaymentMethod
        end
        browser.button(:name, 'commit').click
        if browser.div(:id,'errorExplanation').exist? then
            error = browser.div(:id,'errorExplanation').li(:index,1).text
            browser.link(:text,'Continue shopping').click
            return false, error
        end
        return true,''
    end

    def Customer.empty_cart()
        browser = Common.find_or_start_browser('Pragprog Books Online Store', URL)
        puts 'INFO: Found Browser OK'
        browser.link(:text,"Show my cart").click
        if not browser.div(:text,"Your cart is currently empty").exist? then
            browser.link(:text,'Empty cart').click
            puts 'OK: Cart is now empty.'
            #TODO Assert message - cart is now empty
        end
        puts 'OK: Cart was never empty.'   
        return true,''
    end
end

Watircraft are organising the Austin Workshop on Test Automation to be held on 16-18 January 2009 in Austin, Texas.

I have been approved to attend. It means three long flights from Australia (about 25 hours each way) but I am really looking forward to attending and meeting different people who are involved in Watir.

I haven’t been to America before either, so it should be really good.

Episode eight of the Watir Podcast was released today in which Željko Filipin interviews me about my experience in using Watir, and also about my newly announced role as the Watir Wiki Master.

Check it out if you’re interested: http://watirpodcast.com/alister-scott/

I’ve seen a lot of automated testing efforts fail, and have also had to personally deal with the repercussions and expectations that have been set by these failed efforts.

For example, I clearly remember the first day on my new job that I moved 1500km to Brisbane for. I was being introduced to the Project Director whose first words to me were:

“I have never seen automated testing succeed so I will be watching you very closely!”.

Not the best thing to hear on your first day in a job!

I’ve been thinking a fair amount about why automated testing fails to meet expectations. Sure, there is a lot of sales hype generated by test tool vendors and consultants, not a good thing, and there’s also  practitioners out there without the skills or discipline to deliver successful automated testing solutions, but there must be something else.

The problem is, I believe, that the time and effort to deliver a successful automated testing solution is huge. An automated testing framework might be deemed unsuccessful before it has even been given a chance to be successful! This is why I am a strong believer in first identifying some automated testing quick wins, some low hanging fruit, pardon the idiom.

A quick win is that something that requires a small amount of effort, input, for a huge amount of gain, output. These are sometimes hard to find, but almost always deliver a good outcome: some breathing space.

An example I can use is a simple application monitoring script. A place I worked had a problem with the availability of a public facing web app. Server monitoring wasn’t effective, the web/app server could be running fine but no one could log on via the web! There wasn’t a way to know when it was unavailable to users without first getting complaints via email and phone calls from unhappy people.

It only took me a few hours to quickly develop and test a ruby/watir automated testing script that I set to continuously run to monitor the web app availability. If the web app was unavailable, it would send off an email/sms to the people responsible for getting it running again.

The script was hugely successful. The downtime was reduced drastically, and since people saw patterns about when it was going down, it was easier to determine the cause of the availability problem. Since the script used only free/open source software (ruby and watir), there wasn’t any costs or time taken to acquire the software needed. People were like ‘wow: we didn’t know we could do this so easily’.

I recently attended the Test Automation Workshop on the Gold Coast in Australia and one presentation stuck in my mind. It was by a guy called Don Taylor who used to work in my team in Canberra and his presentation was called “Oxygen for Functional Automated Testing“. He told us that quite a few people emailed him when the program for the workshop went out asking: “What’s this tool called Oxygen?” But it wasn’t a tool at all, but rather about oxygen, the breathing space you need for successful automated testing.

And that’s what I consider to be the biggest output from these quick wins. It’s what automated testing needs to be successful. The breathing space generated from short term automated testing quick wins has enabled me to spend time and effort into creating robust automated testing frameworks that have been designed to be maintainable and successful in the long term. A whole heap of oxygen.