I recently wrote about Tim Koopman’s watir-webdriver-performance gem and how it makes it very easy to collect browser performance stats when you’re running automated tests. Unfortunately it’s Chrome (and IE9) only at the moment, but it looks very promising for when Firefox also supports it (version 7 I believe).

One of my ideas when I first saw this gem was to collect page performance statistics when running automated tests using Cucumber and a page object model, and store a summary of response times for each run. The idea is that you can see how your app is performing over time and whether you have any issues being introduced.

Today, I implemented this functionality in both the WatirMelonCucumber framework, and also my suite of EtsyWatirWebDriver tests.

It’s a fairly simple concept. First I have this class that collects the stats:

require 'yaml'

class PageMetrics
  def initialize
    @pages = {}
  end

  def add_page class_name, performance
    @pages[class_name] ||= []
    @pages[class_name] << performance
  end

  def summary
    summary = []
    @pages.each do |page, metrics|
      average = metrics.inject(0) { |sum, metric| sum + metric.summary[:response_time] }/metrics.size/1000
      result = {
        :page => page.to_s,
        :visits => metrics.size,
        :average_time => average
      }
      summary << result
    end
    summary.to_yaml
  end
end

Then I make sure the class is instantiated and accessible in my cucumber env.rb file:

module Metrics
  def self.page_metrics
    @page_metrics ||= PageMetrics.new
  end
end

Then I make my base page class collect the page metrics (if the browser allows). This uses a method called .with_performance which I recently wrote for the watir-webdriver-performance gem, and is in there from 0.1.3.

class BasePageClass
  include WatirPageHelper

  def initialize browser, page_metrics, visit = false
    @browser = browser
    goto if visit
    expected_element if respond_to? :expected_element
    has_expected_title? if respond_to? :has_expected_title?
    @browser.with_performance {|performance| page_metrics.add_page self.class, performance } unless visit
  end
end

Finally, when I finish running my cucumber tests, I have a hook in my cucumber env.rb file that summarises the page performance times.

at_exit do
  puts Metrics.page_metrics.summary
  File.open('pagemetrics.yml', 'w') { |file| file.puts Metrics.page_metrics.summary }
  Browser::BROWSER.close
end

The yaml file produced contains content that looks something like this:

- :page: EtsyAdvancedSearchPage
  :visits: 8
  :average_time: 0.72075
- :page: EtsySearchResultsPage
  :visits: 6
  :average_time: 1.9331666666666667
- :page: EtsyHomePage
  :visits: 1
  :average_time: 0.789
- :page: EtsyBuyPage
  :visits: 1
  :average_time: 0.741
- :page: EtsyTreasuryPage
  :visits: 1
  :average_time: 1.97
- :page: EtsyCartPage
  :visits: 4
  :average_time: 1.23825
- :page: EtsyItemPage
  :visits: 1
  :average_time: 1.61

You can check out the entire solution in either github repository.

Thanks to Tim for releasing the gem and accepting my pull requests.

Let me know what you think.