My thoughts on Brisbane ANZTB SIGIST: September 2010

I went along to the ANZTB SIGIST last night, at the Hilton here in Brisbane. It was probably the best one I’ve been to, both in attendance, and caliber of presenters. There were five presenters all up, which is a lot to squeeze into two hours including drinks and conversation. It’s no surprise then that the last presentation by Craig Aspinall was a little bit rushed, which is a shame because I would have liked to ask more questions than time allowed (apparently the Hilton would kick us out if we stayed longer).

Craig Smith and Rene Masten from Suncorp began with an excellent presentation on Agile Testing. Craig was awesome in both presenting his extensive knowledge (he’s a coach) and his slides were also very cool (minimal text and no bullets!).

Craig’s main theme was about how to make testing cool: ensuring people say “that’s cool” when you tell them about your testing. He talked about what makes up an agile team, the techical divide between devs and testers, ATDD, ensuring you deliver, and how there is currently a great opportunity for testers (who want to be bothered). Rene followed by talking about organizational change, training and coaching, communicating what you’re doing (internally and externally) and building quality in.

Craig finished with a great motivating style of emphasizing it’s all about passion and craft, “who’s awesome!”, and not being afraid of technical challenges.

The other presenters were Ben Sullivan and Brent Acworth, both also from Suncorp, who gave a demo of BDD using JBehave and Hudson. I enjoyed the talk but there wasn’t much I hadn’t seen before or didn’t know.

The final presentation was by Craig Aspinall and was unfortunately squeezed into a small time slot. It was about what Craig dubs “Automated Black Blob Testing”, the rationale being testing is not black box as a box has a predefined shape, it’s more of a blob.

Craig’s approach looked solid, although I was slightly concerned when he mentioned the project being a SaaS solution and how much effort was being put into automated testing.  I’m not criticizing what Craig did tehnically, I am just concerned about the prevalent practice of the onus of testing SaaS solutions being put onto the customer. I believe if you buy a SaaS, you should get a working SaaS, minimal, if any at all, testing required. But that’s just me. Otherwise, a great talk, besides Craig using Java when Ruby and Watir would have done the trick. ;)

A great ANZTB SIGIST, and hopefully more good ones to come!

Craig Smith’s Slides

Some of my photos

This slideshow requires JavaScript.


Reflecting on this blog

I fired up my netbook tonight to read and reflect on some of my old blog posts. Here’s a collection of my favourite blog posts and a comment about each from my current perspective.

Five organisations I would love to work for (geography aside): Amazingly this two+ year old list is still very accurate. In 2010 I’d possibly drop 37signals from number 5 and replace them with either Thoughtworks or Mozilla or Google. Proving how great Atlassian is, Jeffrey Walker commented on this post, but very sadly, he’s no longer with us.

Software Piracy: I still stand by my views on pirated software being unnecessary, and still love my ‘biting the hand that feeds you’ analogy.

Why I do automated testing: This question still comes up (a lot) when attending job interviews. My answer is, unsurprisingly, still the same.

Version control your tests, quickly, easily, today for free: There’s still no excuse not to have version control of your automated tests. Please do it.

Create fancy wiki home pages with Confluence, Lozenge and Nuvola Icons: A dead simple way to create an attractive Confluence home page with free icons.

Weird ways of working, car indicators, and shoshin: The thing that amazes me today about my eight month year old son is his shoshin, and how he contributes to my own.

Running Watir tests from a Confluence wiki page: Some cool stuff I wish I could use more in my day job. One day.

Five reasons starting with F on why I use Watir: Again, two+ years later and all the reasons are still relevant.

Let me know if you have any favourites or would like me to write about something in particular in the future.

Update 20 July 2010:

Somehow I forgot this post I am really proud of: Software Testing Career Development

Upcoming agile testing meetup in Brisbane, Australia

I just got this email from Thoughtworks, and have registered to attend. See you there if you’re in Brisbane.

You are invited to the launch meeting of the Brisbane chapter of the Agile Alliance Australia (AAA). Last year’s inaugural Agile Australia conference established the AAA and it is envisaged that local chapters will provide ongoing education and knowledge-sharing.

ThoughtWorks Global Testing Practice Lead Kristan Vingrys will be our guest speaker discussing the Changing Role of a Tester. Kristan has over 10 years experience in software testing encompassing a wide range of testing practices for both products and applications within a diverse range of industries. He has presented on agile testing and articles – including a chapter in the ThoughtWorks Anthology Book. He has been involved in solution delivery, coaching and mentoring teams with a focus on testing, agile processes and how the two compliment each other.

The Changing Role of a Tester

IT development is getting faster business now expects applications in months not years. Testing has to keep up and can not afford to be seen as a bottleneck. Traditional ways of testing no longer work with the rapid software development methodologies being used; there is no time to do big up-front test case design or multiple cycle test executions. Testers have to become more agile and embrace change while still providing quality information about the application being developed.

Testing on an agile project is different to a waterfall project; it is not just about doing waterfall in smaller iterations. Nor is it focused on finding as many defects as possible, instead the goal is to work as a team delivering quality working software that satisfies customer need. There are some new skills that testers will need to learn, but they do not need to throw away everything they already know. Changing the mindset about how, when and why of testing will help a tester adapt their existing skills to become an invaluable resource on any agile team.

When: 5:30pm, Thursday 13th May
Where: See the meetup site

After the discussion, we will go locally for drinks, food and networking for those interested, location TBA

Please register at the Brisbane Chapter meetup.com site: Brisbane Chapter – First Meeting

Look forward to seeing you there!

Update 19 July 2010:

Link to a great write up by Craig Smith is here, plus slides are here (pdf).

Watir, Selenium & WebDriver

Please also see my new ‘Watir-WebDriver: a detailed introduction‘ post.

Background

Of all the open source automated web testing tools available, Watir and Selenium have been the two most popular ones. Traditionally there has been advantages and disadvantages of each. Selenium’s most useful features have been its support for multiple programming languages, and is support for testing a wide variety of browsers. This is because Selenium uses JavaScript to interact with the browser, and all modern browsers support JavaScript.

Watir

Watir was originally designed to support Internet Explorer only, and was originally designed as a ruby library, meaning you had to use ruby. Watir’s most useful feature was, and is, its neat API, most likely that way because it was designed by testers. Also Watir has been traditionally had more functionality than Selenium, because it was designed to interact directly with IE instead of using JavaScript, as there are limits to what you can do with JavaScript, ie. you can’t script uploading a file.

The limitations of Watir have been addressed over time, for example, by creating new versions that support other browsers (FireWatir, SafariWatir and ChromeWatir for example), but the task of porting Watir to a new browser isn’t an easy one, and the task of keeping every port of Watir in sync, with the same API, is difficult. Because of Watir’s power, there has been lots of interest in it as a tool, particularly from developers who use other languages. Understandably, they have wanted to use the language they are accustomed to, and use for production code, and hence numerous language ports of Watir have also been created: Watij and WatiN to name two.

Selenium & WebDriver

Selenium has had its challenges also. Whilst it has traditionally been flexible in language choice, it has technical limitations because of its JavaScript architecture. It also hasn’t offered a way to control a headless browser. Late last year WebDriver was announced. WebDriver is a common browser automation tool that uses what ever is the most appropriate mechanism to control a browser, but with a common API. WebDriver supports not only real browsers (IE, Chrome & Firefox) but also headless ones (using HtmlUnit). Watir uses have had access to a headless browser by using Celerity. WebDriver was quickly merged with Selenium, to become Selenium 2.0.

Watir & WebDriver

So, what does WebDriver mean to Watir? Some people in the Watir project see an opportunity to leverage the effort put into WebDriver, but to continue to offer the clean neat API that Watir does. Jari Bakken has released an early version of Watir-WebDriver, essentially Watir’s ruby API utilising the WebDriver engine. By using WebDriver, Watir can support any browser that WebDriver does, including a headless HtmlUnit browser, without using Celerity.

What does this mean for the future of Watir?

If you’re a Watir user, it doesn’t really make that much difference. If you think of automated web testing as a car, Watir is the steering wheel and dashboard, which interact with the engine. Allowing Watir users to use WebDriver is like providing an additional engine choice, but keeping the steering wheel and dash the same.

Ultimately, I think that Watir will remain a very popular automated web testing tool, one that has been designed by testers for testers. I can see the usage of WatiN and Watij reducing as more developers move to Selenium 2.0/WebDriver which will offer the same functionality as Watir using a different API and multiple programming languages. If WebDriver can focus on the detail of controlling browsers, ultimately Watir will be a better tool as more effort can be spent on improving the Watir API, upgrading the steering wheel and dash, so to speak.

HP acquires the Watir project, announces Wativ

In a surprise move today, HP, the vendor of numerous Mercury testing tools, has acquired the open source Watir testing product.

Over the last few years, HP has been continually threatened by the Watir open source project, as it provides flexible and efficient automated web testing for zero cost. The only viable commercial option for HP was to acquire the product. The details of how it was acquired are still unclear, but many long term developers have had all rights to code removed overnight in a move that has shocked the once close knit Watir community.

To align the Watir product with the existing HP Mercury toolset, HP have immediately announced Wativ: Web Application Testing in Vendorscript. Whilst it was initially thought the v in Wativ would stand for VBScript, HP have again shocked its user base by forcing Wativ users to use TSL (Test Scripting Language), the C like proprietary vendorscript originally designed to be used in the legacy WinRunner product. It is thought that the Watit name (Web Application testing in TSL) could be misconstrued by its users. HP have announced its reason for using TSL, being they have opened the product up to the large existing WinRunner user base who are too ingrained in TSL to use anything else.

To align Wativ with other HP Mercury products, numerous additional add-ons will be offered, all individually priced, depending on what you wish to test.

Immediately available are the Wativ Web 2.0 web testing add-on, Wativ Flex testing add-on, Wativ Flash testing add-on, Wativ Firefox add-on and the Wativ web services add-on.

Other details are unclear at this stage, whilst the watir.com site has been updated to reflect the change, other details are still in progress so please stay tuned.

2 April 2010: In case you didn’t realize, this is all a joke.

Lessons Learned in Automated Testing Presentation

Yesterday I took time out from my paternity leave to do a quick presentation at the Brisbane ACS Testing SIG on some of my lessons I have learned in automated testing.

You can download the full presentation here, or view it online here.

Not Ruining your Test Automation Strategy

I was very curious yesterday when I read a quote on twitter taken from a blog post by Uncle Bob Martin.

“The net result is that GUI driven automated tests are fragile & the process of maintaining them is expensive & unreliable” @unclebobmartin

I clicked through to the blog post but I wasn’t impressed with what else I saw.

“The bottom line is that automated tests through the GUI are inherently unstable, and will drive you to one or the other of those two undesirable states.”

I immediately thought, I have made automated GUI testing successful, so who’s Uncle Bob Martin to state all automated GUI testing is wrong.

Whilst I understand there are better ways of automated testing without using a GUI, I also understand that sometimes it’s not possible to test anything but the GUI, for example: poorly written, old, legacy systems.

But having to use the GUI doesn’t mean you can’t make a maintainable solution, I have this at many places, it just takes skill. I see what Bob thinks of this:

“Now, clearly, automated tools can be made to be clever enough to avoid simple cosmetic issues like the moving of a button, or a change in the spelling of a menu. But you have to work at making the tests tolerant of such changes. Do you think that outsourced team of test writers care much about that?”

Whether outsourced or not, the people creating automated tests do need to care about this. Somewhat still amazed at what I was reading, I had an aha moment when I read the following (emphasis added):

“What’s more the cost of re-recording the tests is high, and the re-recording process itself is error-prone.”

Aha! Bob might have only ever worked on a project where GUI automated testing has been done through record/playback. No wonder he thinks poorly of it.

So, to conclude, I would like to make the following points clear:

  • Automated testing through non-GUI means is smart, but sometimes you have no choice.
  • I have made automated testing through the GUI reliable and maintainable, but it required skill on my part.
  • Automated GUI tests can be used to deliberately show discrepancies in the GUI, often highlighting unintended GUI changes.
  • It’s generally not a good idea to completely write something off because you may have seen it done poorly yourself. It’s like saying Agile is wrong because you worked somewhere where Agile was done poorly.