Waiting for AJAX calls in WebDriver C#

I was trying to work out how to wait for AJAX calls to complete in C# WebDriver before continuing a test.

Whilst I believe that your UI should visually indicate that AJAX activity is occurring (such as a spinner) and in this case you should be able to wait until such an indicator changes, if you don’t have a visual indicator and you use JQuery for your AJAX calls, you can use a JavaScript call to jQuery.active to determine if there are any active AJAX requests, and wait until this value is zero.

I wrapped this into a WebDriver extension method on Driver, so you can call it like this:


The actual extension method looks like this:

public static void WaitForAjax(this IWebDriver driver, int timeoutSecs = 10, bool throwException=false)
  for (var i = 0; i < timeoutSecs; i++)
    var ajaxIsComplete = (bool)(driver as IJavaScriptExecutor).ExecuteScript("return jQuery.active == 0");
    if (ajaxIsComplete) return;
  if (throwException)
    throw new Exception("WebDriver timed out waiting for AJAX call to complete");

I hope you find this helpful if you’re ever in the same situation.

Using WebDriver to automatically check for JavaScript errors on every page

Update: please see my newer post on this that doesn’t require changes to your application.

One of the benefits of using a page-object model is that you can perform certain actions on every page in your application that you visit, such as checking for accessibility.

One such check is automatically checking for JavaScript errors on page load. There’s a couple of approaches out there, one involves copying and pasting a small snippet of JavaScript into each page. Since our application we are working on uses a standard template for every page, we simply add some JavaScript to the common page header that is the first thing to load on the page, and catches any JavaScript errors that occur:

define(["amdUtils/string/interpolate"], function(interpolate) {
    window.jsErrors = [];

    window.onerror = function (errorMessage, url, lineNumber) {
        var message = interpolate("Error: [{{0}}], url: [{{1}}], line: [{{2}}]", [errorMessage, url, lineNumber]);
        return false;

It’s then a matter of checking the jsErrors each time we visit a page, this is example C# code we use in the base page class for every page.

var js = driver as IJavaScriptExecutor;
ICollection javascriptErrors = null;
for (var i = 0; i < 20; i++)
  javascriptErrors = js.ExecuteScript("return window.jsErrors") as ICollection;
  if (javascriptErrors != null) break;
Assert.IsNotNull(javascriptErrors, "Can't seem to load JavaScript on the page to find JavaScript errors. Check that JavaScript is enabled.");
var javaScriptErrorsAsString = javascriptErrors.Cast<string>().Aggregate("", (current, error) => current + (error + ", "));
Assert.AreEqual("", javaScriptErrorsAsString, "Found JavaScript errors on page load: " + javaScriptErrorsAsString);

This code waits until it can read the jsErrors on the page. If it can’t, it means that JavaScript didn’t load and this is an error. Once it gets the jsErrors, it checks that there are none.

This code has been very useful for us. It has caught a number of JavaScript errors, and is especially great for finding cross browser JavaScript issues, as we run our acceptance tests in 5 different browsers.

C#: Avoiding the WebDriverException: No response from server for url

When it comes to automated testing, there’s not much worse than intermittent failures, especially when they stem from the driver itself. The current version of the C# WebDriver bindings has such a failure, but I worked out a reasonable way to avoid it happening. Basically it involves creating a WebDriver extension method that I use instead of Driver.FindElement, which tries a number of times to find the element, ignoring the exception that is intermittently raised.

I hope you find this useful if you’re consuming WebDriver in C#.

using OpenQA.Selenium;
using OpenQA.Selenium.Support.UI;
namespace Extensions
    public static class WebDriverExtensions
        public static SelectElement GetSelectElement(this IWebDriver driver, By by)
            return new SelectElement(driver.GetElement(by));
        public static IWebElement GetElement(this IWebDriver driver, By by)
            for (int i = 1; i <= 5; i++ )
                    return driver.FindElement(by);
                catch (Exception e)
                    Console.WriteLine("Exception was raised on locating element: " + e.Message);
            throw new ElementNotVisibleException(by.ToString());

SpecDriver now includes Etsy.com examples

A fellow ThoughtWorker, Paul Hammant, recently announced his intent for QuickTest Professional. And his method? Provide some new technology tutorials for newbies, in the form of a series of open source automated tests for craft site Etsy.com.

He asked me to assist by providing some SpecDriver examples, and since I enjoy doing this stuff I am more than happy to help out.

So, I’ve updated the SpecDriver repository on Github to include some Etsy.com feature files, and working tests. It was a lot of fun to write these tests, because Etsy.com is awesomely testable. Its nicely formatted code with proper ids and class names makes automating against it a real treat.

The benefit of doing this exercise is that it’s meaty enough for me to get some insight into how other people code real automated tests. For example, I’ve had a look into a couple of the code samples and I see a lot of XPath selectors. I personally really dislike XPath stuff, just like I dislike XML as I find it hard to read, and less intuitive than using straight identifiers. I am proud that there isn’t a single XPath used in my Etsy.com examples.

Another thing that stood out to me was how embedded some of the stories are. For example, in the JBehave example the features are located under //etsy-stories/src/main/resources/stories, which took me some time to locate at first, and every time I forget where they are. I much prefer having a features folder in root that makes it pretty obvious what’s in there.

One thing I am very interested in is seeing how people specify tests. I found two of Paul’s scenarios repetitious so I moved them into a single scenario outline. Moving them into a scenario outline made me think about some different ways to test, so I added a new scenario, which didn’t require much more code, but strengthened my feature.

For example, Paul’s two scenarios:

Scenario: Advanced Search for a hat
  Given I am searching on Etsy.com
  When I specify the Knitting sub category
  And I search for hat
  Then there are search results

Scenario: Advanced Search for a ring
  Given I am searching on Etsy.com
  When I specify the Jewelry sub category
  And I search for ring
  Then there are search results

became my:

Scenario Outline: Advanced Search for items in various categories that exist and don't exist
  Given I am searching on Etsy.com
    When I specify the <Sub Category> sub category
    And I search for <Search Term>
    Then I should see <Search Results> search results for <Search Term> 

  | Sub Category | Search Term  | Search Results |
  | Knitting     | 'hat'        | some           |
  | Jewelry      | 'necklace'   | some           |
  | Jewelry      | 'specdriver' | no             | 

While I was at it, I noticed there was a spelling correction feature of the advanced search. I also thought it was a good idea to test that this was working correctly.

Scenario: Misspelling a word corrects search automatically
  Given I am searching on Etsy.com
  When I specify the Knitting sub category
  And I search for 'scalf'
  Then I should see some search results for 'scarf'
  And I should see that the search was for 'scarf' instead of 'scalf'

Now I’ve done this exercise using SpecFlow and WebDriver in C#, I’m keen to do it again using Cucumber & Watir (WebDriver) in Ruby, to compare the implementation. When I am done I will post it on here for you all to see. Enjoy.

SpecDriver: A simple, open-source, page object model framework for C# automated web testing

There doesn’t seem to be a lot of material available in the C# .NET automated testing space, so I thought I would create and share my own page object model centered framework: SpecDriver.

It uses SpecFlow to define features, scenarios and steps, and then WebDriver to actually drive the browser to automate these steps, with a page object model in between to ensure maximum maintainability of the solution.

I have previously documented the steps to getting SpecFlow up and running with Visual Studio C# Express which is free to use for both commercial and non-commercial reasons. You should follow these if you would like to play around with SpecDriver.

You can access all the source code on the github repository, and please feel free to fork/improve it as you see fit.

I will explain the various elements here and how they fit together.

Feature Files for Test Specifications

SpecFlow uses .feature files exactly the same as Cucumber, so it’s pretty easy to create these. I actually used my feature files from my Cucumber framework for this example; the benefits of writing these in a non-technical style!

Feature: Google Search
  As a casual internet user
  I want to find some information about watir, and do a quick conversion
  So that I can be knowledgeable being

Scenario: Search for Watir
  Given I am on the Google Home Page
  When I search for "Watir"
  Then I should see at least 100,000 results

Scenario: Do a unit conversion
  Given I am on the Google Home Page
  When I convert 10 cm to inches
  Then I should see the conversion result "10 centimetres = 3.93700787 inches"

Scenario: Do a search using data specified externally
  Given I am on the Google Home Page
  When I search for a ridiculously small number of results
  Then I should see at most 100 results

Step Definitions that call page objects

The step definitions are small, granular methods that call methods on page objects and do assertions against expected results.

namespace Project1.StepDefinitions
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using TechTalk.SpecFlow;
    using NUnit.Framework;
    using Project1.Pages;

    public class GoogleSearchStepDefinitions : BaseStepDefinitions

        [Given(@"I am on the Google Home Page")]
        public void GivenIAmOnTheGoogleHomePage()
            GoogleHomePage = new GoogleHomePageModel(Driver);
            Assert.AreEqual("Google", GoogleHomePage.Title);

        [When(@"I search for ""(.+)""")]
        public void WhenISearchForSomething(string searchTerm)
            GoogleSearchResultsPage = GoogleHomePage.Search(searchTerm);

        [When(@"I search for a ridiculously small number of results")]
        public void WhenISearchForARidiculouslySmallNumberOfResults()
            GoogleSearchResultsPage = GoogleHomePage.Search("macrocryoglobulinemia marvel");

        [When(@"I convert (.+)")]
        public void WhenIConvertSomething(string ConversionString)
            GoogleSearchResultsPage = GoogleHomePage.Search("convert " + ConversionString);

        [Then(@"I should see at most ([\d,]+) results")]
        public void ThenIShouldSeeAtMostNumberOfResults(string expMaxNumberResults)
            expMaxNumberResults = expMaxNumberResults.Replace(",", "");
            Assert.LessOrEqual(Convert.ToInt32(GoogleSearchResultsPage.NumberOfResults), Convert.ToInt32(expMaxNumberResults));

        [Then(@"I should see at least ([\d,]+) results")]
        public void ThenIShouldSeeAtLeastNumberOfResults(string expMinNumberResults)
            expMinNumberResults = expMinNumberResults.Replace(",", "");
            Assert.GreaterOrEqual(Convert.ToInt32(GoogleSearchResultsPage.NumberOfResults), Convert.ToInt32(expMinNumberResults));

        [Then(@"I should see the conversion result ""(.+)""")]
        public void ThenIShouldSeeTheConversionResult(string expectedConversionResult)
            Assert.AreEqual(expectedConversionResult, GoogleSearchResultsPage.ConversionResult);

Page Object Model

Each page in the application under test is represented by a page class (that inherits from a base page class), and this page class has elements and methods associated with it. The pages are the things that actually use WebDriver to interact with browsers. You can see, like my ruby page object pattern, methods that change pages return an instance of that new page. The other thing to note is that there is a base page class that has a constructor, that requires an known element to instantiate the page. This is a way of knowing where you are in your application and constantly checking it is in the right place. This also ensures consistent syncronization, especially when pages contain dynamic content such as AJAX calls.

namespace Project1.Pages
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using OpenQA.Selenium;

    public class GoogleHomePageModel : BasePageModel
        private static readonly By SearchEditSelector = By.Name("q");
        private static readonly By SearchButtonSelector = By.Name("btnG");

        public GoogleHomePageModel(IWebDriver driver)
            : base(driver, SearchEditSelector)

        private IWebElement SearchEditElement
             get { return Driver.FindElement(SearchEditSelector); }

        private IWebElement SearchButtonElement
            get { return Driver.FindElement(SearchButtonSelector); }

        public GoogleSearchResultsPageModel Search(string term)
            return new GoogleSearchResultsPageModel(Driver);

Putting it all together

I have two batch files that I use in Visual Studio 2010 C# Express, one to generate the SpecFlow NUnit tests, the other to execute them and provide a visual report. I add these as external tools in VS2010 and run them from the menu. Neat.

C# ATDD on a shoestring (or the complete guide to SpecFlow & NUnit in Visual Studio 2010 Express)


When I’m working on something new at work, I like to be able to work on it in my own time at nights and on the weekend to polish my skills through experimentation, and share what I learn. I’ve recently been working on setting up Acceptance Test Driven Development in C# at work, but unfortunately Microsoft makes it really, really hard to polish your skills in C# development at home, unless you’re willing to part with large sums of cash (say from $20,775, and I’d prefer a new car). This is probably one of the reasons why I find the Microsoft .NET testing community (and the dev community too) quite insular, Microsoft just doesn’t encourage their users to share their craft.

Sure, Microsoft provides Visual Studio 2010 (VS2010) Express Edition – which can be used free of charge, even for commercial purposes, but the limitations on its functionality are borderline ridiculous. For example, it doesn’t support Microsoft’s own unit testing framework (Microsoft obviously don’t encourage TDD), and all add-ins are banned (no ReSharper for you). It also doesn’t work with Microsoft’s own version control system (TFS).

So, Microsoft it basically makes it really hard to do anything you want to do, but there are ways to get stuff done, even if they’re convoluted.

ATDD Frameworks in C#

I know of three main open source ATDD/BDD frameworks that work in C#: SpecFlow, Cuke4Nuke and StoryQ. Only two of these use Gherkin (StoryQ uses its own slightly modified DSL), and I like Gherkin, so I’ll ignore StoryQ for now. Cuke4Nuke still requires some Ruby glue, and to keep things pure, I like SpecFlow as it’s a pure .NET implementation of ATDD with full Gherkin syntax support. For this reason, SpecFlow seems to be the most prevalent ATDD framework in the .NET community and this is what I will use for this example.

SpecFlow as an Acceptance Test Driven Development Framework

SpecFlow is an open source framework that is delivered as a Windows installer, and this installs support for feature and step definition files in Visual Studio through an Add-In that automatically generates unit tests for you from the feature files. Unfortunately, as VS2010 Express doesn’t allow add-ins, SpecFlow won’t work without some tinkering.

Install SpecFlow & NUnit

Before we get started, we need to install VS2010 Express, SpecFlow and NUnit.

  1. Install VS2010 C# Express if you don’t already have it
  2. Install the latest SpecFlow version from the SpecFlow GitHub downloads page.
  3. Install the latest NUnit version from the NUnit download page.
  4. Add the Specflow directory (C:\Program Files\TechTalk\SpecFlow or similar) and the NUnit directory (C:\Program Files\NUnit 2.5.9\bin\net-2.0 or similar) to your Windows path.

Adding SpecFlow templates to Visual Studio 2010 Express

When installing SpecFlow to VS2010 (non-express) it creates templates that can be accessed through the “New Item” menu. Fortunately, it is fairly trivial to add these to VS2010 Express. Just download the templates I have uploaded here (2010, or here (2012) unzip this directory and place the .zip files (SpecFlowFeature.zip, SpecFlowStepDefinition.zip etc.) into your Visual Studio 2010 Express C# templates directory (usually: C:\Users\username\Documents\Visual Studio 2010\Templates\ItemTemplates\Visual C# or similar).

This means you now have access to the SpecFlow templates when adding a new item to your C# project.

Generating SpecFlow NUnit tests from Feature Files

When you add a SpecFlow feature to the full version of VS2010, it automatically generates a unit test file behind the feature file, so that it’s always up to date. This doesn’t work in Express edition, so you have to use the specflow.exe file to do this for you. Fortunately, this can be done in a batch file, and then added as an “external tool” to VS2010 Express, so you get the same functionality, but not quite as neat.

  1. Enable Expert Settings in VS2010 Express (if not already enabled), by choosing Tools->Settings->Expert Settings.
  2. Create a batch file in your project directory called something like “generatespecflowtests.bat” with the following content:

echo Starting to generate SpecFlow unit tests
SpecFlow generateAll %1 /force /verbose
echo Finished generating SpecFlow unit tests
  1. Add an external tool command in VS2010 Express, using Tools->External Tools->Add. Make sure you set the starting directory, and “show output”

  1. You can then generate your tests from the Tools menu or, if you like, you can add this command to a new Toolbar by using Tools -> Customize -> Toolbars -> New, then Tools -> Customize -> Commands -> Toolbar -> Add Command.

Running SpecFlow NUnit tests from Feature Files

Simon Whittemore from London wrote an excellent post about how to automatically run Specflow NUnit tests, and capture a pretty html result, on his blog. I’ve included his batch file here, as it’s what we’ll use to run our tests from within VS2010 Express.

  1. Create a batch file in your project directory named runspecflowtests.bat or similar.
  2. Include Simon’s content (below) in your batch file.
@echo off
nunit-console %1
specflow.exe nunitexecutionreport %2 /xmlTestResult:%3
if NOT %errorlevel% == 0 (
echo "Error generating report - %errorlevel%"
GOTO :exit
if %errorlevel% ==0 TestResult.html
  1. Create an external tool setting the same as we did for the generate unit tests, but with the parameters defined on this dialog:
    1. Command: Link to your batch file
    2. Arguments: $(TargetName)$(TargetExt)$(ProjectDir)$(ProjectFileName)$(BinDir)TestResult.xml
    3. Initial Directory: $(BinDir)
    4. Use Output Window: checked

  1. Optionally add this to your VS2010 Express Toolbar as above

When you run this command in VS2010, all SpecFlow features are run, and the resulting report is launched in your default web browser.


What I have shown is that although Microsoft makes it very hard, it is possible to use VS2010 Express and open source tools like SpecFlow so that you can polish your skills in your own time.

In this theme of ATDD using C# and .NET, over the coming days I will demonstrate how you can use SpecFlow to write and run automated web tests against a web browser using the open source tool WebDriver. Stay tuned!

Update: 20 Feb 2011

The original link to the templates was wrong, I have now updated it to the correct link.

Update 29 Sep 2013

Yannick de Kercadio has kindly provided VS2012 templates here.