Why to avoid t-shirt sizes for user story estimation

The more I work on agile software development teams who use t-shirt sizes (S,M,L,XL etc.) to estimate user stories the more I dislike this approach. Here’s why:

  • In my opinion, the most import thing about user story sizing is relativity, and t-shirt sizes are a subjective measure of relativity: someone in the team might think a large is two times as big as a small, whereas another person might think it’s three times as big. This isn’t helped by the t-shirt analogy where it’s actually hard to determine how much bigger is a large t-shirt than a small one?
  • You can’t create a single measure of team velocity unless you define a scale that converts t-shirt sizes into a numeric size so you can measure t-shirt size relativity and velocity.
  • As soon as create a scale to convert t-shirt sizes into a numeric size you’ve essentially started using story points (in a convoluted way).

TL;DR: Using t-shirt sizes for user story estimation is confusing and ultimately leads the team to using story points so just skip t-shirt sizes and use relative story points instead.

Notes from GTAC 2014 (aka The Flaky Test Conference)

I was lucky enough the attend the Google Test Automation Conference (GTAC) at Google Kirkland in Washington last week. As usual, it was a very well run conference with an interesting mix of talks and attendees.

Whilst there wasn’t an official theme this year, I personally saw two themes emerge throughout the two days: dealing with flaky tests and running automated tests on real mobile devices.

Flaky Tests

There wasn’t too many talks that didn’t mention flaky automated tests (known as ‘flakes’) at some point. Whilst there seemed to be some suggestions for dealing with flaky tests (like Facebook running new tests x times to see if they fail and classify them as flaky and assign to the owner to fix), there didn’t seem to be a lot of solutions for avoiding the creation of flaky tests in the first place which I would have liked to see.

Real Mobile Devices

The obsession of running mobile automated tests on real devices continued from last year’s conference with talks about mobile devices as a service. I personally think we’d be better spending the time and effort on making more realistic mobile emulators that we can scale rather than continuing the real device test obsession.

Key Takeaway

My key takeaway was even highly innovative companies like Google, Facebook and Netflix still struggle balancing software quality and velocity. In Australia, these companies don’t have a strong presence here, and often the IT management of smaller companies here like to say things like “Google does x” or “Facebook does y”. The problem with this is they only know these companies from the outside. Ankit Mehta’s slides at the beginning of his keynote captured this perfectly and hence were my favorite slides of the conference:

Google from the outside (like a jet plane) Google from the inside (lots of people pushing a broken down car)Jonathan Rasmusson also recently wrote about something similar on his blog:

“When we see or admire what one company is doing, we are only seeing it through the lens of the end result. And not the underlying foundations that were setup to get them there.”

Well done to the GTAC committee and Google for putting on another excellent conference. Here’s some photos I took of Seattle/Kirkland during my time there.

Lock down your browser versions if you run WebDriver tests

Recently our WebDriver tests that run in Chrome via a Windows service all suddenly stopped working even though we hadn’t made any changes to our tests. It seems Chrome had automatically updated itself on our WebDriver agents introducing a Chromium 38 bug meaning WebDriver won’t work at all (full details here and here). Getting these tests running again has been very painful, mainly due to Google not having standalone Chrome installers for any previous versions of Chrome publicly available.

If you run any WebDriver tests I highly recommend you lock down your browser versions to stop this happening to you in the future. Here’s how:


Firefox is fantastic in this regard as they make every back version easily accessible as well as a simple way on all platforms to stop automatic upgrades. I tend to lock down to Firefox ESRs (Extended Support Releases) such as versions 24 and 31 which are listed on this comprehensive Wikipedia page.

To stop updates all you do is open preferences, advanced, update and select ‘never’.

Firefox Prevent Updates


Chrome is a P.I.T.A. in both being able to install a previous version or lock down the currently installed one. Google prefer a Chrome web installer which always installs the latest version of Chrome, and if you want a specific version you need the alternate (offline) installer (for all users if you use a Windows service), but they only provide the latest installer. It’s hard if not impossible to find older alternate (offline) installers on the web, even oldapps.com can’t host them.

Once you have a version of Chrome on Windows that you want to keep, you need to download a group policy template, and disable automatic updates before running Chrome (so it doesn’t automatically update before you set the group policy). I won’t go into full details but you should be able to find all details here. Some sites mention using a plugin to stop updates but this doesn’t work so you’ll need to go down the group policy path.


Locking down browser versions avoids having to suddenly work out why your entire WebDriver test suite fails.

An ongoing stream of articles I find interesting

Much like the collection of quotes that I regularly update, I have now created a collection of links/articles that I find interesting at links.watirmelon.com which I will reguarly update. Like articles on this blog, these are automatically published to this follow-only twitter account: @watirmelon

My favorite article I have read lately is the U.S. Digital Services Playbook: a collection of 13 key “plays” about how to deliver great digital services. I was surprised to see this great stuff coming from a government agency!

Digital Service Plays

  1. Understand what people need
  2. Address the whole experience, from start to finish
  3. Make it simple and intuitive
  4. Build the service using agile and iterative practices
  5. Structure budgets and contracts to support delivery
  6. Assign one leader and hold that person accountable
  7. Bring in experienced teams
  8. Choose a modern technology stack
  9. Deploy in a flexible hosting environment
  10. Automate testing and deployments
  11. Manage security and privacy through reusable processes
  12. Use data to drive decisions
  13. Default to open


Faking Geolocation in Selenium WebDriver with Firefox

I recently worked on some Selenium WebDriver tests that needed to provide a Geolocation to a HTML5 site so it could display some results. There’s a couple of things to keep in mind when doing this:

  1. There are two Firefox about:config preferences that you must both set in your WebDriver Firefox profile to always supply the Geolocation information to your test (instead of displaying a prompt). These are:
    1. set ‘geo.prompt.testing’ to true
    2. set ‘geo.prompt.testing.allow’ to true
  2. If you want to use a specific location you need to override an about:config preference to a JSON file containing that location. This is only supported in very recent versions of Firefox (I tested version 31). Whilst this can be manually done using a data:application/json string, Firefox using WebDriver in C# completely ignores this so the workaround is to create a JSON file with the location and reference using using a file:/// prefix
    1. Create C:\Dev\Brussels.json etc containing something like:
          "status": "OK",
          "accuracy": 10.0,
          "location": {
              "lat": 50.850780,
              "lng": 4.358138,
              "latitude": 50.850780,
              "longitude": 4.358138,
              "accuracy": 10.0
    2. set ‘geo.wifi.uri’ to ‘file:///C:/Dev/brussels.json’ (or wherever your file is)

Once you’ve done this you should be able to test Geolocation without seeing the prompt or being able to completely override the location. Some example C# WebDriver scripts are below.

using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.Support.UI;

namespace GeoLocation.Tests.Acceptance
    public class WebDriverGeoLocation
        public void CanOverrideLocationInNewerVersionsOfFirefoxLike31()
            var profile = new FirefoxProfile {EnableNativeEvents = false};
            profile.SetPreference("geo.prompt.testing", true);
            profile.SetPreference("geo.prompt.testing.allow", true);
            profile.SetPreference("geo.wifi.uri", "file:///C:/Dev/brussels.json");
            var driver = new FirefoxDriver(profile);
            new WebDriverWait(driver, TimeSpan.FromSeconds(15)).Until(ExpectedConditions.ElementExists((By.ClassName("success"))));
            Assert.AreEqual("found you!", driver.FindElement(By.Id("status")).Text);

        public void CantOverrideLocationInOlderVersionsOfFirefoxLike24()
            var profile = new FirefoxProfile { EnableNativeEvents = false };
            profile.SetPreference("geo.prompt.testing", true);
            profile.SetPreference("geo.prompt.testing.allow", true);
            var driver = new FirefoxDriver(profile);
            new WebDriverWait(driver, TimeSpan.FromSeconds(15)).Until(ExpectedConditions.ElementExists((By.ClassName("success"))));
            Assert.AreEqual("found you!", driver.FindElement(By.Id("status")).Text);

Enjoy your Geolocation testing!

Connecting to a Windows 8.1 box from Mac OS X

I have a new Windows 8.1 machine for work and couldn’t work out how to connect to it via the Microsoft Remote Desktop Connection client on Mac OS X. No matter what I did I got the following error:

Verify Identity of Computer

When trying to work out the reason for this I stumbled across a link to this page with the solution deeply embedded in another page.

Here’s my summary:

On your Windows 8.1 box:

  1. Open the Management Console (mmc.exe)
  2. Add a Snap-In (File Menu) and choose ‘Group Policy Object Editor’ and then ‘Local Computer’ on the subsequent screen
  3. Navigate using the tree to:
    • Computer Configuration
      • Administrative Templates
        • Windows Components
          • Remote Desktop Services
            • Remote Desktop Session Host
              • Security
  4. Change the following two settings from their default:
    1. ‘Require use of specific for remote desktop (RDP) connections’ from ‘Default’ to ‘Enabled’, then select ‘RDP’ in the options pane
    2. ‘Require user authentication for remote connections by using Network Level Authentications’ to ‘Disabled’
  5. Restart the ‘Remote Desktop Service’ or restart the Windows 8.1 machine

I hope this helps you should you come across this problem yourself.

Deciding to have lots of children and lots of tests is still fun later on

I recently saw a paraphrased quote by James Bach from a testing meetup in Sydney.

Deciding to have lots of (automated) checks [sic: tests] is like deciding to have lots of children. It’s fun at first, but later…

I read it a number of times and each time I read it I disagreed with it a little more.

As a proud father of three beautiful boys, I truly believe having lots of children is fun at first AND fun later on. Sure, having lots of kids is hardest thing you’ll ever do and continues to be hard as each day goes by, but hard and fun aren’t opposites or mutually exclusive whatsoever1; I’ve actually found them to be strongly correlated (think of your funnest job: was it easy?). So don’t let anybody put you off having lots of kids ever, because they are still loads of fun later on (assuming you’re not scared of hard work). I love my boys: they’re the funnest people I know and they get funner every day.

As a developer of software, I also believe having lots of automated tests is fun later on, on the proviso that you’ve put thought into them upfront. I truly believe the only way to make sustainable software that you can change and refactor with confidence is to develop it using self-testing code. Sure, having too many automated e2e tests can be a PITA2 but I’d choose lots of automated tests over no or very few automated tests any day of the week3. Again, don’t let someone put you off having lots of automated tests: just do them right!


I asked James Bach on Twitter about his quote (and how many children he has, the answer is one), and in the typical self-righteous context driven testing ‘community’ style I was called ‘reckless’ for choosing to have three beautiful boys with my lovely wife.

It didn’t end there with other members of the ‘community’ doing what they do4 and taking the opportunity to jump in uninvited, attack me for even wondering how someone with only one child can comment on having lots of children, and try to intimidate me by accusing me of using ‘ad-hominem’ falacies/attacks against James Bach (they like big words).

This entire episode reaffirms my choice to have nothing whatsoever to do with the context driven testing ‘community’ and anyone who associates themselves with it (which started by me deleting my twitter account so they can’t attack me or have anything to do with me).

My final word of warning to those of you who still consider yourself part of that ‘community’, a comment about ‘context-driven testing':

“I chose not to engage in those dogmatic discussions. I once had a job interview where the term context-driven led one of the devs to do some googling. I had to defend myself for affiliating as he’d found some right contentious and dogmatic stuff and wondered if I were some kind of extremist for including that term in my resume. It’s no longer in my resume, FWIW.”



[1] I recently read that happiness and unhappiness aren’t actually the opposite of one another: you can be both happy and unhappy at the same time.

[2] In case you didn’t know: PITA means ‘pain in the ass’, and lots of end to end tests are a pain in the ass. There’s lots of articles on here about why, the most recent one being about Salesforce.com and its 100,000 e2e tests.

[3] FWIW most codebases I have worked on have had zero to little automated tests, so I don’t think having too many automated tests is our common industry problem.

[4] It’s not hard to find examples of where members of this ‘community’ rally against and intimidate a particular person they disagree with on twitter, for examples: here, here, here, here, here, etc. I personally know a fellow tester who had a very similar negative experience to me a couple of years ago and has since distanced herself also.