Celerity: first impressions

Last week at the Test Automation Workshop (TAW) here in Australia, the topic of being able to run automated tests ‘headless’ came up, and I mentioned the Celerity project: a headless Watir port. I decided to have a play with Celerity tonight to see how easy it is to get up and running, and also look at it as a way to quickly run Watir scripts.

Installation

The installation was fairly straightforward. You need a Java 6 JDK, as well as the JRuby binaries. You’ll need to update two environment variables, namely add JRuby to your path, and update your JAVA_HOME variable. Installing Celerity involves a JRuby gem: "jruby -S gem install celerity". You can also install it from github, but I don’t know the difference between the two.

Availability of Gems

Ruby gems are supported in JRuby, but only if they don’t use C libraries. This means that Watir won’t work, nor will Roo. I use Roo to define test cases in spreadsheets, so it means I can’t do a comparison of execution times at the moment, at least until I get these test cases out of spreadsheets.

Using Celerity as a simple load testing tool instead

At TAW, Kelvin Ross thought Celerity sounded promising as a load testing tool, considering it was headless and lightweight. I thought I would give a simple google search script a try, running in both Celerity, and Watir, with 30 concurrent users.

Running under Watir

CPU peaked at 100% for the entire run, and each page varied considerable but took on average 10 seconds to load.

Running under Celerity

CPU use was normal, and each page took just over 1 second to load (minimal variance).

Conclusion

Celerity seems a promising way to execute basic load tests using a headless browser. The benefit of Celerity is support for javascript execution in the browser, the downside at the moment is lack of support for some ruby gems. If you could run Watir under JRuby, I could have used a single script.

Scripts Used
Celerity Script

require 'thread'
require "rubygems"
require "celerity"

def test_google
  browser = Celerity::Browser.new
  browser.goto('http://www.google.com')
  browser.text_field(:name, 'q').value = 'Celerity'
  start_time = Time.now
  browser.button(:name, 'btnG').click
  end_time = Time.now
  puts end_time - start_time
  browser.close
end

threads = []
30.times do
  threads << Thread.new {test_google}
end
threads.each {|x| x.join}

Watir Script
require 'thread'
require 'watir'
require 'watir/ie'

def test_google
  browser = Watir::IE.start('http://www.google.com')
  browser.text_field(:name, 'q').value = 'Celerity'
  start_time = Time.now
  browser.button(:name, 'btnG').click
  end_time = Time.now
  puts end_time - start_time
  browser.close
end

threads = []
30.times do
  threads << Thread.new {test_google}
end
threads.each {|x| x.join}

Easily define Watir tests in excel, OO, wikis and Google docs using Roo

I spent this evening playing with Roo, the ruby library for reading data from spreadsheets and I am very impressed. In a very small amount of time I was able to define tests in four different forms/places and could execute my tests from each of these:

  • An Excel file (.xls): stored locally
  • An OpenOffice (.ods): stored locally
  • An Excel file (.xls) stored in a Confluence wiki page with Confluence Office Connector; and
  • A Google Docs spreadsheet.

The great thing about Roo is that you don’t actually need Excel; Roo simply reads the file, unlike the ruby Excel COM WIN32 API I have used previously.

The spreadsheet (embedded in Confluence) looks like this:

excel-in-confluence

The cool thing about embedding it in Confluence is that you can click the title of the spreadsheet to edit it (in OpenOffice in my case).

I made some minor changes to my existing code that executed my depot tests from a wiki page, and it was as easy as that. A data driven Watir solution with four possible ways to define test cases. Cool.

You can find all the code needed below.


require 'watir'
require 'rubygems'
require 'roo'
require './Customer.rb'
require './Common.rb'

case ARGV[0]
when "excel"
	ss = Excel.new("watirmelon.xls")
when "wiki"
	ss = Excel.new("http://localhost:8080/download/attachments/2097153/watirmelon.xls")
when "gdocs"
	ss = Google.new("http://spreadsheets.google.com/ccc?key=pEcLrW3b2djraE8JF_2fJWA")
else
	ss = Openoffice.new("watirmelon.ods")
end

ss.default_sheet = ss.sheets.first
ss.first_row.upto(ss.last_row) do |line|
	if ss.cell(line,1).strip != "Function" then #We have an executable test
		begin
			module_name = ss.cell(line,1).strip
			method_name = ss.cell(line,2).downcase.strip.sub(' ','_') # automatically determine function name based upon method name.
			comments = ss.cell(line,3).strip
			expected_outcome = ss.cell(line,4).strip
			expected_error = ss.cell(line,5).strip
			required_module = Kernel.const_get(module_name)
			required_method = required_module.method(method_name)
			arity = required_method.arity() # this is how many arguments the method requires, it is negative if a 'catch all' is supplied.
			arity = ((arity * -1) - 1) if arity < 0
			parameters = []
			1.upto(arity) do |p|
				parameters.push(ss.cell(line,p+5))
			end
			actual_outcome, actual_output = required_method.call(*parameters)
			# determine the result.
			if (expected_outcome = 'Success') and actual_outcome then
			    result = "PASS"
			elsif (expected_outcome = 'Error') and (not actual_outcome) and (expected_error = actual_output) then
			    result = "PASS"
			else
			    result = "FAIL"
			end
			puts "\nRunning Test: #{method_name} for #{module_name}."
			puts "Expected Outcome: #{expected_outcome}."
			puts "Expected Error: #{expected_error}."
			puts "Actual Outcome: #{actual_outcome}."
			puts "Actual Output: #{actual_output}."
			puts "RESULT: #{result}"
		rescue
			puts "An error occurred: #{$!}"
		end
	end
end

See the full test code below the break.

Continue reading

Why I like ruby over vbscript

I am glad that Watir uses ruby, rather than something like VBScript. I have been using VBScript a lot in my current job (as it is the only language that QuickTest Professional supports) and when I compare it to ruby, I often find it annoying and inefficient.

A dictionary (a.k.a. hash or associative array) is a good example.

Here is some code I used to create a simple dictionary in VBscript with five items, which then displays one of the item values:


Set dictCheckValues = CreateObject( "Scripting.Dictionary" )
dictCheckValues.Add "a",1
dictCheckValues.Add "b",2
dictCheckValues.Add "c",3
dictCheckValues.Add "d",4
dictCheckValues.Add "e",5
msgbox dictCheckValues.item( "c" )

Here is the same example but this time I used ruby:


check_values = {'a'=>1,'b'=>2,'c'=>3,'d'=>4,'e'=>5}
puts check_values['c']

You can see that the ruby example is not only more concise (2 lines vs 7), but I also find it more readable.

Python is another concise and highly readable language.


check_values = {'a':1,'b':2,'c':3,'d':4,'e':5}
print check_values['c']

I sometimes wish that Watir originally chose to use python over ruby. I find python is more broadly used there are generally a lot more libraries available for it. Although Python’s indentation can be confusing at times, especially across different platforms.