Category Archives: FireWatir

Watir-WebDriver: A detailed introduction

Update: 22 August 2011: Please see: watirwebdriver.com for a detailed guide to Watir-WebDriver

Update: 22 July 2011: I have updated quite a number of things that have changed since the earlier releases

Watir-WebDriver is a really great tool; Jari Bakken‘s done a really good job of it. There’s not a huge amount on the web about it, how to get it up and running and use it. I’m aiming to fix that here.

For those who don’t know what Watir-WebDriver is, it’s basically a nice Watir (ruby) implementation on WebDriver, so it gives you four browsers (three real, one headless) using one neat API, out of the box. The thing I like about it is you don’t need to use JRuby (like Celerity), which means it plays nice with Cucumber (although Cucumber does work under JRuby).

I’ve written about how Watir, WebDriver and Selenium all fit together before, so this post aims to be a lot more hands-on.

Getting Watir-WebDriver Running

There are essentially two components you need: the Watir-WebDriver ruby gem, and the remote WebDriver server. The remote WebDriver server is only needed if you want to run your tests in headless mode without a real browser (or want to use Opera). You obviously need ruby first but I won’t detail that here. You can find info about ruby versions etc. at watir.com. If you’re on Mac or Linux, I strongly suggest using RVM and bundler.

The Watir-WebDriver ruby gem

It’s a simple matter of opening a command prompt and typing:

  • gem install watir-webdriver (windows); or
  • sudo gem install watir-webdriver (osx or linux – better to use RVM and bundler)

The remote WebDriver Server

This is the slightly tricky part. This is so that WebDriver can run headless without a real browser, and isn’t needed for real browser support (bar Opera). The quickest easiest way to get up and running is to download this java jar file, open a command prompt where you have saved it, and run:

java -jar selenium-server-standalone-2.0b1.jar

You can also specify the startup and max memory allocated, which is handy when running large test suites.

java -Xms1024M -Xmx2048M -jar selenium-server-standalone-2.0a7.jar

You need to have this java server running whenever using WebDriver, but it’s easy enough to bootstrap it.

Update: Jari has pointed out you can run the server programmatically which is even better:

require 'selenium/server'
server = Selenium::Server.new("/path/to/jar", :background => true)
server.start
# run your tests
server.stop

First Impressions

Waiting in Watir-WebDriver doesn’t seem as straightforward as with Watir, probably due to the underlying drivers, but fortunately there’s an inbuilt waiting library to use (also available in Watir 1.6.7+):

  • Watir::Wait.until { ... }: where you can wait for a block to be true
  • object.when_present.set: where you can do something when it’s present
  • object.wait_until_present:; where you just wait until something is present
  • object.wait_while_present:; where you just wait until something disappears

Hello Watir-WebDriver

It seems pertinent to start with a google search example.

Hello Watir-WebDriver in three browsers

These three browsers seem to work very similarly, but obviously Internet Explorer will only run on Microsoft Windows.

require 'rubygems'
require 'watir-webdriver'
b = Watir::Browser.new :chrome
b.goto 'www.google.com'
b.text_field(:name => 'q').set 'Watir-WebDriver'
b.button(:name => 'btnG').click
b.div(:id => 'resultStats').wait_until_present
puts "Displaying page: '#{b.title}' with results: '#{b.div(:id => "resultStats").text}'"
b.close

The only difference for Firefox:

b = Watir::Browser.new :firefox

The only difference for IE:

b = Watir::Browser.new :ie

Hello Watir-WebDriver in Headless (HTML Unit)

I imagine this is the most anticipated feature for Watir users, as it means your tests run much faster, and you can still use ruby (unlike Celerity which uses JRuby). The script is fairly straightforward, you simply specify the hostname for your WebDriver server you started above. You need to specifically enable JavaScript by creating a capabilities profile.

require 'rubygems'
require 'watir-webdriver'
include Selenium
capabilities = WebDriver::Remote::Capabilities.htmlunit(:javascript_enabled => true)
b = Watir::Browser.new(:remote, :url => 'http://127.0.0.1:4444/wd/hub', :desired_capabilities => capabilities)
b = Watir::Browser.new :firefox
b.goto "www.google.com"
b.text_field(:name => "q").set "Watir-WebDriver"
b.button(:name => "btnG").click
b.div(:id => "resultStats").wait_until_present
puts "Displaying page: '#{b.title}' with results: '#{b.div(:id => "resultStats").text}'"
b.close

Sample Timings

As an experiment, I looped the Google Search script above 100 times to measure and compare the execution times.

  • Internet Explorer 8 on Windows 7: 400 seconds
  • Firefox 3.6 on Mac OSX: 277 seconds
  • Headless HTMLUnit on Mac OSX: 269 seconds

Very surprisingly, the headless run wasn’t much quicker at all. This may be due to running the test on Google over the Internet and not a local application.

Caveat Emptor

There are a number of differences between Watir-WebDriver and Watir. The main ones that are important to me are:

Zero based indexing as opposed to 1 based

For example,

In Watir/FireWatir: this finds the first table on a page:

puts b.table(:index => 1).text

But in Watir-WebDriver, it finds the second table on a page.

Attaching to windows
Attaching to new windows (for example pop-ups) has been fairly straightforward in Watir with its attach method, which is missing in Watir-WebDriver. It looks like you can iterate through a collection of windows in Watir-WebDriver using browser.windows, but I haven’t tried this out yet.

Summary

Watir-WebDriver is a very solid testing tool that uses a great browser automation engine (WebDriver) with a clean ruby API (Watir). I believe it will be a popular choice amongst testing teams, particually those using it for ATDD through Cucumber and running it as headless on a continuous integration server.

Further Reading

Cucumber

How to set up Cucumber and Watir on OSX

These are the steps I had to use to get Cucumber and Watir running on OSX. It’s a shame about step 1, it’s a real pain but I don’t know any way around it.

  1. Install Xcode from OSX installation DVD – this takes about 15 minutes
  2. Open a command prompt and use enter the following commands.
  3. sudo gem update --system
  4. sudo gem install rspec --no-rdoc --no-ri
  5. sudo gem install gherkin --no-rdoc --no-ri
  6. sudo gem install cucumber --no-rdoc --no-ri
  7. sudo gem install firewatir --no-rdoc --no-ri

You should be good to go.

Watir, Selenium & WebDriver

Please also see my new ‘Watir-WebDriver: a detailed introduction‘ post.

Background

Of all the open source automated web testing tools available, Watir and Selenium have been the two most popular ones. Traditionally there has been advantages and disadvantages of each. Selenium’s most useful features have been its support for multiple programming languages, and is support for testing a wide variety of browsers. This is because Selenium uses JavaScript to interact with the browser, and all modern browsers support JavaScript.

Watir

Watir was originally designed to support Internet Explorer only, and was originally designed as a ruby library, meaning you had to use ruby. Watir’s most useful feature was, and is, its neat API, most likely that way because it was designed by testers. Also Watir has been traditionally had more functionality than Selenium, because it was designed to interact directly with IE instead of using JavaScript, as there are limits to what you can do with JavaScript, ie. you can’t script uploading a file.

The limitations of Watir have been addressed over time, for example, by creating new versions that support other browsers (FireWatir, SafariWatir and ChromeWatir for example), but the task of porting Watir to a new browser isn’t an easy one, and the task of keeping every port of Watir in sync, with the same API, is difficult. Because of Watir’s power, there has been lots of interest in it as a tool, particularly from developers who use other languages. Understandably, they have wanted to use the language they are accustomed to, and use for production code, and hence numerous language ports of Watir have also been created: Watij and WatiN to name two.

Selenium & WebDriver

Selenium has had its challenges also. Whilst it has traditionally been flexible in language choice, it has technical limitations because of its JavaScript architecture. It also hasn’t offered a way to control a headless browser. Late last year WebDriver was announced. WebDriver is a common browser automation tool that uses what ever is the most appropriate mechanism to control a browser, but with a common API. WebDriver supports not only real browsers (IE, Chrome & Firefox) but also headless ones (using HtmlUnit). Watir uses have had access to a headless browser by using Celerity. WebDriver was quickly merged with Selenium, to become Selenium 2.0.

Watir & WebDriver

So, what does WebDriver mean to Watir? Some people in the Watir project see an opportunity to leverage the effort put into WebDriver, but to continue to offer the clean neat API that Watir does. Jari Bakken has released an early version of Watir-WebDriver, essentially Watir’s ruby API utilising the WebDriver engine. By using WebDriver, Watir can support any browser that WebDriver does, including a headless HtmlUnit browser, without using Celerity.

What does this mean for the future of Watir?

If you’re a Watir user, it doesn’t really make that much difference. If you think of automated web testing as a car, Watir is the steering wheel and dashboard, which interact with the engine. Allowing Watir users to use WebDriver is like providing an additional engine choice, but keeping the steering wheel and dash the same.

Ultimately, I think that Watir will remain a very popular automated web testing tool, one that has been designed by testers for testers. I can see the usage of WatiN and Watij reducing as more developers move to Selenium 2.0/WebDriver which will offer the same functionality as Watir using a different API and multiple programming languages. If WebDriver can focus on the detail of controlling browsers, ultimately Watir will be a better tool as more effort can be spent on improving the Watir API, upgrading the steering wheel and dash, so to speak.

Happy 5th Birthday to two great open source projects

November 2009 marks the fifth birthday of two great open source projects, both of which I couldn’t live without.

  1. Firefox was first released on November 9th, 2004. I remember first installing it in 2004, loving it and I still use it to this day. I’m actually using it right now to write this post.
  2. Watir made its first public appearance on November 15th, 2004, in a tutorial by Bret and Paul at StarWest 2004.

The two projects became connected in late 2006 when FireWatir was developed by Angrez Singh. Since then Watir has incorporated native Firefox support.

So here’s cheers to these two great open source projects! Hope there are many more great birthdays to come!

Creating a Watir framework using Test::Unit & Roo

One common challenge I see over and over again is people figuring out how to design a logical and maintainable automated testing framework. I have designed quite a few frameworks for various projects, but one thing that has consistently been a win for me is purposely separating test case and test execution design.

It’s therefore logical that the design of my Watir framework deliberately separates test case design and test execution design so that:

  • test case design is done visually in spreadsheets; and
  • test execution design is done in ruby methods, because code is the most efficient and maintainable way.

Since I last published details about my framework on this blog, I have started doing assertions using the Test::Unit ruby library. The reasons I chose Test::Unit are:

  • it is easy to ‘mix-in’ Test::Unit assertions into modules of ruby code using include Test::Unit::Assertions;
  • it is included with ruby;
  • ruby scripts with Test::Unit::TestCase are instantly executable, in my case, from SciTE;
  • its assertions are easy to understand and use.

I have also made some other improvements to my framework code, including:

  • the ability to specify browser types, and spreadsheet sources, as command line arguments (with defaults);
  • logging test output to a file;
  • no longer attaching to an open browser, the same browser instance is used completely for all tests (and elegantly closed at the end).

The main design has been kept the same, in that a spreadsheet (either excel, openoffice or Google Docs) contains tests grouped by functional area, which call a method in a particular module.

The great thing about my framework is that adding a new test is a matter of designing the test case, and then writing the ruby method: as the methods are called dynamically from the spreadsheet, no extra glue is needed!

Enough talk, here’s the code. The Google spreadsheet is here. You can find a .zip file of all the required files to run it here. It runs on the depot app, which you get here. You will need two gems: Watir (oh duh), and Roo.

Test Driver tc_main.rb


$:.unshift File.join(File.dirname(__FILE__), ".", "lib")
require 'watir'
require 'roo'
require 'test/unit'
require 'customer'
require 'admin'
$stdout = File.new('log.txt',File::WRONLY|File::APPEND|File::CREAT)
$stderr = File.new('log.txt',File::WRONLY|File::APPEND|File::CREAT)

class TC_WatirMelon < Test::Unit::TestCase
  @@colmap = {:module_name=>0, :method_name=>1, :comments=>2, :exp_outcome=>3, :exp_error=>4, :first_param=>5}
  @@ss_format = ARGV[0]
  @@specified_browser = ARGV[1]

  def setup
    puts "[Starting at #{Time.now}]\n"
    case @@ss_format
      when "excel"
        @ss = Excel.new("watirmelon.xls")
      when "wiki"
        @ss = Excel.new("http://localhost:8080/download/attachments/2097153/watirmelon.xls")
      when "gdocs"
        @ss = Google.new("0AtL3mPY2rEqmdEY3XzRqUlZKSmM5Z3EtM21UdFdqb1E")
      else
        @ss = Openoffice.new("watirmelon.ods")
      end
    @ss.default_sheet = @ss.sheets.first
    case @@specified_browser
      when "firefox"
        Watir::Browser.default = 'firefox'
        @browser = Watir::Browser.new
      else
        Watir::Browser.default = 'ie'
        @browser = Watir::Browser.new
        @browser.speed = :zippy
        @browser.visible = true
      end
  end

  def test_run_sheet()
    @ss.first_row.upto(@ss.last_row) do |row|
      #Read row into array
      line = Array.new
      @ss.first_column.upto(@ss.last_column) do |column|
        line << @ss.cell(row, column).to_s.strip
      end

      module_name = line[@@colmap[:module_name]]
      if module_name != "Function" then #if not a header
        method_name = line[@@colmap[:method_name]].downcase.gsub(' ','_') #automatically determine ruby method name based upon data sheet
        exp_outcome = line[@@colmap[:exp_outcome]]
        exp_error = line[@@colmap[:exp_error]]
        first_param = @@colmap[:first_param]
        required_module = Kernel.const_get(module_name)
        required_method = required_module.method(method_name)
        arity = required_method.arity() # this is how many arguments the method requires, it is negative if a 'catch all' is supplied.
        arity = ((arity * -1) - 1) if arity < 0 # arity is negative when there is a 'catch all'
        arity = arity-1 # Ignore the first browser parameter
        unless arity == 0
          parameters = line[first_param..first_param+(arity-1)]
        else
          parameters = []
        end
        begin
          act_outcome, act_output = required_method.call(@browser, *parameters)
        rescue Test::Unit::AssertionFailedError => e
          self.send(:add_failure, e.message, e.backtrace)
          act_outcome = false
          act_output = e.message
        end
        if (exp_outcome == 'Success') and act_outcome then
          assert(true, "Expected outcome and actual outcome are the same")
          result = 'PASS'
        elsif (exp_outcome == 'Error') and (not act_outcome) and (exp_error.strip! == act_output.strip!)
          assert(true, "Expected outcome and actual outcome are the same, and error messages match")
          result = 'PASS'
        else
          result = 'FAIL'
          begin
            assert(false,"Row: #{row}: Expected outcome and actual outcome for #{method_name} for #{module_name} do not match, or error messages do not match.")
          rescue Test::Unit::AssertionFailedError => e
            self.send(:add_failure, e.message, e.backtrace)
          end
        end
        puts "###########################################"
        puts "[Running: #{module_name}.#{method_name}]"
        puts "[Expected Outcome: #{exp_outcome}]"
        puts "[Expected Error: #{exp_error}]"
        puts "[Actual Outcome: Success]" if act_outcome
        puts "[Actual Outcome: Error]" if not act_outcome
        puts "[Actual Output: #{act_output}]"
        puts "[RESULT: #{result}]"
        puts "###########################################"
        end
      end
  end

  def teardown
    @browser.close
    puts "[Finishing at #{Time.now}]\n\n"
  end

end

Customer Module customer.rb

require 'test/unit'
include Test::Unit::Assertions

module Customer

  TITLE = 'Pragprog Books Online Store'
  URL = 'http://localhost:3000/store/'

  # Description:: Adds a book named 'book_title' to cart
  def Customer.add_book(browser, book_title)
    browser.goto(URL)
    # Check if title is already in cart - so we can check it was added correctly
    browser.link(:text,'Show my cart').click
    prev_cart_count = 0
    prev_cart_total = 0.00
    if not browser.div(:text,'Your cart is currently empty').exist? then
     # We have a non-empty cart
      for row in browser.table(:index,1)
        if row[2].text == book_title then
          prev_cart_count = row[1].text.to_i
          break
        end
      end
      prev_cart_total = browser.cell(:id, 'totalcell').text[1..-1].to_f #remove $ sign
      browser.link(:text, 'Continue shopping').click
    end

    found = false
    book_price = 0.00
    1.upto(browser.divs.length) do |index|
      if (browser.div(:index,index).attribute_value('className') == 'catalogentry') and (browser.div(:index,index).h3(:text,book_title).exists?) then
        book_price = browser.div(:index,index).span(:class, 'catalogprice').text[1..-1].to_f #remove $ sign
        browser.div(:index,index).link(:class,'addtocart').click
        found = true
        break
      end
    end
    if not found then
      return false,'Could not locate title in store'
    end

    new_cart_count = 0
    for row in browser.table(:index,1)
      if row[2].text == book_title then
        new_cart_count = row[1].text.to_i
        break
      end
    end
    new_cart_total = browser.cell(:id, 'totalcell').text[1..-1].to_f # remove $ sign
    assert_equal(new_cart_count,(prev_cart_count+1), "Ensure that new quantity is now one greater than previously")
    assert_equal(new_cart_total,(prev_cart_total + book_price), "Ensure that new cart total is old cart total plus book price")
    browser.link(:text, 'Continue shopping').click
    return true,new_cart_total
  end

  def Customer.check_out(browser, customerName, customerEmail, customerAddress, customerPaymentMethod)
    browser.goto(URL)
    browser.link(:text,'Show my cart').click
    if browser.div(:text,'Your cart is currently empty').exist? then
      return false,'Your cart is currently empty'
    end
    browser.link(:text,"Checkout").click
    browser.text_field(:id, 'order_name').set(customerName)
    browser.text_field(:id, 'order_email').set(customerEmail)
    browser.text_field(:id, 'order_address').set(customerAddress)
    begin
      browser.select_list(:id, 'order_pay_type').select(customerPaymentMethod)
    rescue Watir::Exception::NoValueFoundException
      flunk('Could not locate customer payment method in drop down list: '+customerPaymentMethod)
    end
    browser.button(:name, 'commit').click
    if browser.div(:id,'errorExplanation').exist? then
      error = ''
      1.upto(browser.div(:id,'errorExplanation').lis.length) do |index|
        error << (browser.div(:id,'errorExplanation').li(:index,index).text + ",")
      end
      browser.link(:text,'Continue shopping').click
      return false, error
    end
    assert_equal(browser.div(:id,'notice').text, 'Thank you for your order.',"Thank you for your order should appear.")
    return true,''
  end

  def Customer.empty_cart(browser)
    browser.goto(URL)
    browser.link(:text,"Show my cart").click
    if browser.div(:text,"Your cart is currently empty").exist? then
      assert('Cart was never empty')
    else
      browser.link(:text,'Empty cart').click
      assert_equal(browser.div(:id, 'notice').text,'Your cart is now empty')
    end
    return true,''
  end

  def Customer.check_cart_total(browser, exp_total)
    browser.goto(URL)
    browser.link(:text,'Show my cart').click
    if browser.div(:text,'Your cart is currently empty').exist? then
      return false,'Your cart is currently empty'
    end
    act_total = browser.cell(:id, 'totalcell').text[1..-1].to_f
    assert_equal(act_total,exp_total.to_f,"Check that cart total is as expected.")
    return true,act_total
  end
end

Admin Module admin.rb


require 'test/unit'
include Test::Unit::Assertions

module Admin
  TITLE = 'ADMINISTER Pragprog Books Online Store'
  URL = 'http://localhost:3000/admin/'

  def Admin.log_on(browser, username, password)
    browser.goto(URL)
    if browser.link(:text,'Log out').exist? then #if already logged in
      browser.link(:text,'Log out').click
    end
    browser.text_field(:id, 'user_name').set username
    browser.text_field(:id, 'user_password').set password
    browser.button(:value, ' LOGIN ').click
    if browser.div(:id, 'notice').exist? then
      return false,browser.div(:id, 'notice').text
    else
      return true,''
    end
  end

  def Admin.ship_items(browser, name)
    browser.goto(URL)
    browser.link(:text, 'Shipping').click
    num_orders = 0
    index = 0
    browser.form(:action,'/admin/ship').divs.each do |div|
      if div.class_name == "olname"
        index+=1
        if div.text == name then
          browser.form(:action,'/admin/ship').checkbox(:index, index).set
          num_orders+=1
        end
      end
    end

    browser.button(:value, ' SHIP CHECKED ITEMS ').click

    if num_orders == 1 then
      assert_equal(browser.div(:id,"notice").text, "One order marked as shipped","Correct notice")
    elsif num_orders > 1 then
      assert_equal(browser.div(:id,"notice").text, "#{num_orders} orders marked as shipped","Correct notice")
    end
    return true, num_orders.to_s
  end

end

Watir.com

I have spent a bit of time over the last few days setting up Watir.com, hosted here on WordPress.

We were originally aiming to host our own version of Confluence and JIRA and use Confluence to serve the Watir.com homepage, but this ended up being a lot more complicated and expensive than originally planned.

The great thing about WordPress is, although it was originally a blogging platform, its functionality also works as a very neat CMS. Whilst wordpress.com has some limitations over wordpress.org, we can live with these limitations for now as we have a free (as in beer) hosted site that the world can see.

Check it out.

watir.com