Your automated acceptance tests needn’t be written in the same language as your system being tested

When selecting which tool to implement for business facing automated acceptance tests, I’ve often heard that it must use the same programming language as the system under test. Whilst this can sometimes work well, often it is better to choose a tool in another language that will ulimately deliver you better success as it can be adopted more passionately. Here are some of the reasons on why I think so.

The simplicity of some other languages mean it’s easier for less technical testers to understand them

Let me show you an example that compares the same test written in C# as in Ruby.

C# WebDriver Example – Inspired by David Burns

using OpenQA.Selenium;
using OpenQA.Selenium.IE;
 
using NUnit.Framework;
 
namespace Selenium.Two.DotNetExample
{
    [TestFixture]
    public class Test_Google
    {
	    IWebDriver driver;
	     
	    [SetUp]
	    public void Setup()
	    {
		    driver = new InternetExplorerDriver();
	    }
	    
	    [TearDown]
	    public void Teardown()
	    {
		    driver.Quit();
	    }
    		
        [Test]
        public void TestSearchGoogleForTheAutomatedTester()
	    {
	        //Navigate to the site
            driver.Navigate().GoToUrl("http://www.google.com");
            //Find the Element and create an object so we can use it
            IWebElement queryBox = driver.FindElement(By.Name("q"));
            //Work with the Element that's on the page
            queryBox.SendKeys("Watir");
			queryBox.SendKeys(Keys.ArrowDown);
            queryBox.Submit();
            //Check that the Title is what we are expecting
            Assert.True(driver.Title.IndexOf("Watir - Google Search") > -1);
        }
    }
}

Ruby Watir Example – written by me

require 'watir-webdriver' #watir, celerity, firewatir
require "test/unit"
class GoogleTest < Test::Unit::TestCase
  def test_google
    b =Watir::Browser.start "www.google.com"
    b.text_field(:name => "q").set "watir"
    b.button(:name, "btnG").click
    assert_equal("Watir - Google Search", b.title)
    b.close
  end
end

I am slightly biased, but I think the ruby example is neater and easier to read, especially to someone with a less technical background. Sure, a C# developer might baulk at this premise, but for most people who don’t know C#, I imagine it’s the case.

Often times developers like learning another programming language to add to their experience
When I’ve been working on automated tests in ruby, I’ve often seen a strong interest in developers wanting to learn ruby and the automated testing framework, as it’s common for developers to specialize in a language (Java or .NET) and then focus solely on that. Don’t get me wrong, this isn’t always the case, but it happens more than you think.

Testers can embrace simpler programming languages such as ruby, and can become the “experts” in that domain

I’ve seen testers pick up a programming language like ruby and polish their skills in it, which helps their morale by becoming experts in it. If the automated acceptance tests were written in the same language as the developers, yes this brings benefits, but the testers might not become experts as quickly, and may lack ownership of the tests.

Automated acceptance test tools provide different mileage across languages
Some languages have particularly good tools for defining business facing automated tests, for example, Cucumber’s support for ruby is superb, whereas support for C# equivalents such as Cuke4Nuke and SpecFlow isn’t quite as good. RSpec is another example of a tool that is awesome in its domain.

Summary

I know this post has ended up sounding like a pro-ruby rant, but it is :) I am trying to point out that there’s simpler, easier languages for testers to pick up than your Javas and C-Sharps. The ruby and python programming languages are two great examples, and as I mentioned, it seems that ruby is quickly becoming the testing language of choice, because of its simplicity, and the wide variety of testing related gems available. Ruby also makes it extremely easy to bootstrap configurations across different environments, and its lack of licensing requirements or need for an IDE make it an excellent choice for testers.

One of the reasons that the Selenium project is so successful is that it supports various programming languages (and browsers), which appeals to the majority of developers who can use their established skills, but it’s common to meet testers who are passionate about Watir, because of its simplicity and focus: a tool for testers by testers. It is for this reason I believe Ruby & Watir & Cucumber is a winning combination, and will bring about many success stories for agile testing teams in the future.

12 thoughts on “Your automated acceptance tests needn’t be written in the same language as your system being tested

  1. Is there an RSpec equivalant to the line:

    class GoogleTest < Test::Unit::TestCase

    So that I can use RSpec assertions instead of the Test::Unit ones?

    1. It’s pretty simple to do:

      require 'rubygems'
      require 'watir-webdriver' #watir, celerity, firewatir
      require "rspec"
      describe "Google Search" do
        it "searches for watir correctly" do
          b =Watir::Browser.start "www.google.com"
          b.text_field(:name => "q").set "watir"
          b.button(:name, "btnG").click
          sleep 2
          b.title.should == "watir - Google Search"
          b.close
        end
      end
      
      1. Yes but this is using the RSpec framework, what I meant was I wanted to use the should be == true assertion type reference for scripts using a cucumber API for acceptance testing.

        So useing step definitions I could call a page class, which uses or better still inherits RSpec type assertions and then I could call the assert method from my tests.

        Does that make any sense or am I barking up the wrong tree?

        Thanks,

        Loga

        1. I’d keep all the RSpec assertions at the Cucumber step level, and make the pages do something, or return something.
          I have been meaning to write a blog post about this, so stay tuned.

  2. The other trick for this is to use the version of Ruby/Python that runs in the same execution environment as the production code. For instance, with clients that have a system written in Java they are scripting against, I coach them to use either Jython or JRuby for a best-ish of both worlds. I had another client be quite successful with IronRuby against an ASP.NET system — including reaching into the ORM layer to validate actions in the front end were reflected in the back.

    Of course, in some systems it makes sense. For instance, if you are using something like Django I would still use Python to be able to access the Modules — regardless of how nice RSpec’s matchers are.

    1. Absolutely! I tested a complex mining application written in Java using Jython and it was fantastic, all the simplicity of Python with excellent integration with Java objects.

  3. I think this makes sense as long as the system under test offers a public interface to arbitrary clients.

    So a web page or REST/SOAP APIs can be tweaked by clients written in any language, but things like fixtures and of course unit tests should probably be kept in the same language as the application code base.

    1. Sure, I agree unit and integration tests should be in the same language as the application code base.

  4. I don’t think your code comparison is fair; the minimal equivalent C# to your Ruby would be:

    
    using OpenQA.Selenium;
    using OpenQA.Selenium.IE;
    using NUnit.Framework;
    public class GoogleTest {
      [Test]
      public void TestGoogle() {
        var b = new InternetExplorerDriver();
        b.Navigate().GoToUrl("http://www.google.com");
        b.FindElement(By.Name("q")).SendKeys("watir");
        b.FindElement(By.Name("btnG")).Click();
        Assert.AreEqual("Watir - Google Search", b.Title);
        b.Quit();
      }
    }
    

    The net differences?
    * We import one more namespace – big deal
    * We specify the browser we’re emulating (“InternetExplorerDriver” rather than “Waitr::Browser” – incidentally, what browser *will* the Ruby run? I don’t know by sight…)
    * We say By.Name(“q”) rather than :name => “q” – pure language idiom, certainly neither is more natural for a non-programmer to write, though FindElement(By.Name(“q”)) I would argue to be a bit more intuitive to read because it’s more natural-language than text_field(:name => “q”)
    * We make it clear we’re typing, rather than setting the value (I presume in the ruby it will clear the text box before setting “watir”? I certainly hope so!)

    Suddenly, comparing like-for-like, they look basically identical.

    I agree that there’s a lot to be gained from being flexible with languages, and choosing the best language for the task is important (and indeed that test code doesn’t have to be the same language as the underlying code it’s testing, particularly for Web stuff), but please don’t claim that by shoving your setup and teardown into the test in one language, or by using obscure asserts in the other (hell, even Assert.That(driver.Title, Is.StringContaining(“Watir – Google Search”)) would be a more sane way of writing the C# example), you’re giving a fair comparison of their traits!

    1. Sure, you make fair points.
      As I mentioned, I used a C# example I found published on a blog by an experienced automated tester, so I was using that as a basis of what a typical C# WebDriver Selenium 2.0 test would look like.
      I am glad we agree on choosing the best language for the task.

    2. Oh, and I always prefer a .setmethod over .typekeys anyday, and yes, it does exactly what you’d expect, set the field. If you want, you can optionally call .clear, but unlike .typekeys, .set doesn’t append: yuck :(

Comments are closed.