Category Archives: Watir-WebDriver

Running Watir-WebDriver tests on Travis CI: a distributed build system

Posted on by
2

I recently came across Travis CI, a distributed build system that has close links to Github. I’ve seen quite a few projects use it as a CI system, but none that run headless browser tests.

Leveraging off the work I had done recently setting up my own Jenkins server in the cloud to run headless Watir-WebDriver tests, I thought I would have a go at running my WatirMelonCucumber and EtsyWatirWebDriver headless browser tests using Travis CI.

What I didn’t realize is how easy it’d be. The only things I had to do was make sure my Gemfile included rake, and also make sure there was some file existance checking happening for some log files, and it pretty much ran straight away. Wow!

Caveat Emptor

This is pretty new territory, so there’s a few things to watch out for:

  • Every now and, Travis complains about not having Firefox installed. I am not sure why this happens, maybe something to do with the different agents in use;
  • The locale of the build agents seems to be German, so when running my Google tests, the page content is in German, so it fails because it can’t find my expected (English) results; and
  • I can’t seem to capture my test results html file nor screenshot pngs, so it’s a red/green only affair at the moment.

But still, neat distributed free headless CI!

Running your watir-webdriver tests in the cloud, for free!

Posted on by
9
  • What if you could run unlimited Watir WebDriver tests in the cloud? Check.
  • What if the Watir WebDriver tests would run automatically as soon as you pushed a change to github? Check.
  • What if you would have a full visual history of results with embedded screenshots on failure? Check.
  • What if all of this was free?* Checkmate.

I’ve spent a bit of time over the last week working out how to do this. Here are the basics of what you need to do:

And here’s the detailed instructions.

Set up an Amazon EC2 micro instance running Ubuntu.

  1. First you need to sign up for an Amazon AWS account. This means you’re eligible for a free-tier micro instance for a year.
  2. Once you have an account set up, you need to launch a new instance. I found a free tier eligible Ubuntu image (11.04 Natty 64 bit desktop) and launched that.
  3. You will also want to create an elastic IP and associate it to your instance so that if you reboot your machine, you will have the same IP address. This is done through the AWS console under Elastic IPs.
  4. While you’re here, you’ll want to edit your machine’s security group and open up port 22 for SSH, and 80 for HTTP.
  5. This gives you secure shell (SSH) access to this machine using the provided key, and user ‘ubuntu’: 
    ssh -i your-key-name.pem ubuntu@your-ip-address
  6. Everything you will do to configure this machine will be through this SSH session, so polish up your unix command line skills!

Set up Jenkins on your machine

There is a useful page for installing Jenkins on Ubuntu.

wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo aptitude update
sudo aptitude install jenkins

Make Jenkins available on port 80 so that you don’t need to specify port

Jenkins installs by default on port 80. Ubuntu won’t let applications run on port 80 unless they’re running as root, so it’s best to set up an Apache 2 proxy to port 80 to 8080.

sudo aptitude install apache2
sudo a2enmod proxy
sudo a2enmod proxy_http
sudo a2enmod vhost_alias
sudo a2dissite default

Then create a file called jenkins in /etc/apache2/sites-available


	ServerAdmin webmaster@localhost
	ServerName ci.company.com
	ServerAlias ci
	ProxyRequests Off
	
		Order deny,allow
		Allow from all
	
	ProxyPreserveHost on
	ProxyPass / http://localhost:8080/

Then run the following commands:

sudo a2ensite jenkins
sudo apache2ctl restart

Password Protect Jenkins

You should go to your Jenkins site (accessible directly at your instance’s IP address through a web browser), and create an account, and then configure the security of Jenkins.

Install Jenkins Plugins

You will need to install the following Jenkins plugins

  • Github: to integrate to Github SCM
  • Rake: to run ruby rake tasks that run Watir-WebDriver tests
  • Green balls: because blue balls are just plain wrong

Install RVM for the Jenkins user

First we’ll need to install git

sudo apt-get install git

Jenkins will need to be able to run ruby, so we’ll install RVM as the Jenkins user.

To run as the jenkins user, we’ll use the sudo command, with the -Hiu arguments to load the home directory and bash profile:

sudo -Hiu jenkins

Once we are user Jenkins, we’ll install RVM using Git.

bash < <(curl -s https://rvm.beginrescueend.com/install/rvm)

Now we need to work out what Ubuntu packages Ruby needs, which is easily done via RVM.

rvm notes

which gives me something like

For Ruby (MRI, Rubinius, & REE)  you should install the following OS dependencies:
/usr/bin/apt-get install build-essential bison openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake

So, we can log-out as the Jenkins user (control-D) and install the following as ubuntu

sudo apt-get install build-essential bison openssl libreadline6 libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev libyaml-dev libsqlite3-0 libsqlite3-dev sqlite3 libxml2-dev libxslt-dev autoconf libc6-dev ncurses-dev automake

Once we’ve done this, we’ll want to run the following as the Jenkins user (
sudo -Hiu jenkins) to install Ruby 1.9.2.

rvm pkg install zlib
rvm install 1.9.2 --with-zlib-dir=$rvm_path/usr

Running headless Watir-WebDriver tests

I choose a desktop version of Ubuntu, so it’ll already have Firefox installed, but if you don’t, you can install it by:

sudo apt-get install firefox

To run our Watir WebDriver tests headlessly using the headless gem, we’ll need xvfb

sudo apt-get install xvfb

Configuring Jenkins to run tests via Rake

You add a new build in Jenkins where you can specify the github repository location.

As we’ve installed the rake plugin, we can configure a new Jenkins project to use an RVM ruby install (in my case ruby-1.9.2-p290@watirmelon-cucumber).

I simply set up a default task in rake, which runs all my cucumber tests. This generates a results.html file which is captured as an artifact, and also creates and captures junit xml results, which are used to show test summary information.I also capture any file created under the ‘screenshots’ directory.

Summary and Outcome

I have set up both my WatirMelonCucumber and EtsyWatirWebDriver projects on jenkins.watirmelon.com.

My Jenkins Dashboard looks something like this:

Please feel to leave a comment below and let me know what you think.

* Free for one year using an free tier EC2 micro instance

Running headless Watir-WebDriver tests using a real browser

Posted on by
19

I seem to get quite a few questions from people trying to use Watir-WebDriver in headless mode using a WebDriver server and HTMLUnit. It seems to me that it’s problematic, especially when your web app contains JavaScript (who’s app doesn’t?).

The problem stems from HTMLUnit’s JavaScript support. Whilst it does support JavaScript, myself and many others have found the JavsScript support to be pretty poor as it uses its own JavaScript engine (Rhino) that no other ‘real’ browser actually uses, which sorta raises a question: are you really testing your app’s JavaScript?

The other thing about HTMLUnit is the lack of screenshot capability, as it’s purely headless, there’s no way to capture a screenshot when something goes wrong, so you’ll find yourself re-running failed tests using a different browser, which may or may not reproduce your problem. Not ideal!

Finally, running WebDriver headless tests requires you to run a Selenium Server, which is additional overhead for your tests.

But why headless?

Whilst running automated browser tests in a headless form can speed things up, the main reason I see people wanting headless watir-webdriver support is so that tests can be run on headless Linux machines, for example, a Jenkins Server. I’ve found running tests in parallel speeds things up more than headless execution can, so it’s mainly headless server support most people are after.

Enter the headless gem.

The headless gem is a ruby wrapper for Xvfb that makes it super easy to run graphical applications (such as real web browsers) on a headless machine. This gem is perfect for running headless watir-webdriver tests using a real browser.

How? It’s simple.

require 'watir-webdriver'
require 'headless'
headless = Headless.new
headless.start
b = Watir::Browser.start 'www.google.com'
puts b.title
b.close
headless.destroy

You’ll need Xvfb installed, which is as easy as:

sudo apt-get install xvfb

Or if you’re using cucumber like me, you just need the following code in your env.rb file:

if ENV['HEADLESS']
  require 'headless'
  headless = Headless.new
  headless.start
  at_exit do
    headless.destroy
  end
end

This then means you can still launch and use Firefox (or Chrome) on a headless machine.

The best thing about it though is that Watir-WebDriver’s inbuilt screenshots still work perfectly, so it captures a screenshot of the actual page even though it has no display. How awesome is that!

Summary

The headless gem makes it super easy to run real browsers on headless servers. This way you get real JavaScript support without the need to run a Selenium server, and is perfect for running your tests as part of continous integration.

Stay tuned as I’ll be writing a post soon about how to use an EC2 instance to run your headless watir-webdriver tests in the cloud, for free!

Determining your host operating system using ruby

Posted on by
4

I found out this is how you programatically determine your host operating system using ruby:

require 'rbconfig'
RbConfig::CONFIG['host_os']

This came in handy when setting the automatic browser downloads for browsers on Windows, as the location needs to use backslashes on Windows instead of Ruby’s default forward-slashes:

require 'rbconfig'
download_directory = "#{Dir.pwd}/downloads"
download_directory.gsub!("/", "\\") if  RbConfig::CONFIG['host_os'] =~ /mswin|mingw/

Introducing watirwebdriver.com: the most elegant way to use webdriver with ruby

Posted on by
6

I have recently launched watirwebdriver.com: the most elegant way to use webdriver with ruby. It’s a collection of succinct examples of how to use watir-webdriver. I found it very enjoyable piecing together this information from various blog posts I have written, and organizing it in a structured, logical manner. I hope you find it useful.

Telling Watir-WebDriver to always locate elements upon use

Posted on by
6

One of the differences between Watir and Watir-WebDriver is how elements are located. Watir-WebDriver was designe to store references to DOM elements previously used, and this can cause the stale ObseleteElementError exception, as I explained previously.

Watir-WebDriver 0.3.1 has recently been released, and now there is a configurable option to specify that you always want to create a fresh reference when using an element.

 Watir::always_locate = true b = Watir::Browser.new

This will result in a performance hit, but in my experience, it isn’t too bad. Jari Bakken ran WatirSpec using this config which resulted in a 10 second increase in running time (from 145 seconds to 155 seconds) which in my opinion is worth it to have more reliable test results.

Thanks to Jari for working on this. Hope you enjoy.

Update: 15 August 2011: I only just realized this is enabled by default, so you’ll get it automatically when you update your gem. Neat.

Automate entering text into WYSIWYG editors using Watir-WebDriver

Posted on by
10

When writing automated web tests, I find it common to come across WYSIWYG (What-you-see-is-what-you-get) editors, and because these usually are html elements embedded in an iframe, it’s not usually easy to automate entering text into these (nowhere near as easy as a text field or text area say).

I’ve found there are two ways to enter text into a WYSIWYG editor using Watir-WebDriver:

  • locate the iFrame, and use .send_keys to enter text
  • execute javascript on the browser object that sets the value of the WYSIWYG editor

The benefit of using the iFrame is that you can use the same code across the different WYSIWYG editors and don’t need to understand the javascript, but the downside is that when running your tests in Firefox, the browser needs to be in focus or your test will not work. This is a big issue for me, as I often do other things while tests are running, so I avoid this method completely.

The benefit of the method that executes javascript is I have found it more reliable as you don’t need to ensure focus, but the downside is working out what the javascript is that you need to execute.

Setting text on two types of WYSIWIG editors

Here I will show you how easy it is to set these two popular WYSIWYG editors using the two methods I mentioned.

CKEditor

require 'watir-webdriver'
b = Watir::Browser.new :firefox
b.goto 'http://ckeditor.com/demo'
b.execute_script("CKEDITOR.instances['editor1'].setData('hello world');")
b.frame(:title => 'Rich text editor, editor1, press ALT 0 for help.').send_keys 'hello world again'

TinyMCE Editor

require 'watir-webdriver'
b = Watir::Browser.new
b.goto 'http://tinymce.moxiecode.com/tryit/full.php'
b.execute_script("tinyMCE.get('content').execCommand('mceSetContent',false, 'hello world' );")
b.frame(:id => "content_ifr").send_keys 'hello world again'


Summary

I have switched to using the JavaScript method as I find it much more reliable, and once you understand the JavaScript API for common editors, it is reasonably straightforward to do.

Capturing page performance times with my WatirMelonCucumber framework

Posted on by
Reply

I recently wrote about Tim Koopman’s watir-webdriver-performance gem and how it makes it very easy to collect browser performance stats when you’re running automated tests. Unfortunately it’s Chrome (and IE9) only at the moment, but it looks very promising for when Firefox also supports it (version 7 I believe).

One of my ideas when I first saw this gem was to collect page performance statistics when running automated tests using Cucumber and a page object model, and store a summary of response times for each run. The idea is that you can see how your app is performing over time and whether you have any issues being introduced.

Today, I implemented this functionality in both the WatirMelonCucumber framework, and also my suite of EtsyWatirWebDriver tests.

It’s a fairly simple concept. First I have this class that collects the stats:

require 'yaml'

class PageMetrics
  def initialize
    @pages = {}
  end

  def add_page class_name, performance
    @pages[class_name] ||= []
    @pages[class_name] << performance
  end

  def summary
    summary = []
    @pages.each do |page, metrics|
      average = metrics.inject(0) { |sum, metric| sum + metric.summary[:response_time] }/metrics.size/1000
      result = {
        :page => page.to_s,
        :visits => metrics.size,
        :average_time => average
      }
      summary << result
    end
    summary.to_yaml
  end
end

Then I make sure the class is instantiated and accessible in my cucumber env.rb file:

module Metrics
  def self.page_metrics
    @page_metrics ||= PageMetrics.new
  end
end

Then I make my base page class collect the page metrics (if the browser allows). This uses a method called .with_performance which I recently wrote for the watir-webdriver-performance gem, and is in there from 0.1.3.

class BasePageClass
  include WatirPageHelper

  def initialize browser, page_metrics, visit = false
    @browser = browser
    goto if visit
    expected_element if respond_to? :expected_element
    has_expected_title? if respond_to? :has_expected_title?
    @browser.with_performance {|performance| page_metrics.add_page self.class, performance } unless visit
  end
end

Finally, when I finish running my cucumber tests, I have a hook in my cucumber env.rb file that summarises the page performance times.

at_exit do
  puts Metrics.page_metrics.summary
  File.open('pagemetrics.yml', 'w') { |file| file.puts Metrics.page_metrics.summary }
  Browser::BROWSER.close
end

The yaml file produced contains content that looks something like this:

- :page: EtsyAdvancedSearchPage
  :visits: 8
  :average_time: 0.72075
- :page: EtsySearchResultsPage
  :visits: 6
  :average_time: 1.9331666666666667
- :page: EtsyHomePage
  :visits: 1
  :average_time: 0.789
- :page: EtsyBuyPage
  :visits: 1
  :average_time: 0.741
- :page: EtsyTreasuryPage
  :visits: 1
  :average_time: 1.97
- :page: EtsyCartPage
  :visits: 4
  :average_time: 1.23825
- :page: EtsyItemPage
  :visits: 1
  :average_time: 1.61

You can check out the entire solution in either github repository.

Thanks to Tim for releasing the gem and accepting my pull requests.

Let me know what you think.