I’ve been working on an iOS project and have been looking for a suitable automated functional test library to drive the iOS GUI. I must say that automated iOS testing feels like automated web testing did about 10 years ago: lots of different tools taking different approaches, and all approaches quite flaky!
Through research I came across quite a few different tools that support iOS automated functional testing but need to work out which one is best.
Whilst I often advocate writing functional tests in the same language as your codebase, in the case of iOS and Objective C, I think using a more lightweight language like Ruby has its own advantages (I don’t really like Objective C).
The one thing I really dislike with a lot of these iOS functional test tools is how they are married to Cucumber and encourage users to write tests like “I tap button x” and “I scroll through list y”. These types of tests are much harder to read as they express implementation over intention and should be avoided.
I will also be writing tests for an Android app so being able to use the same tool-set is a great advantage to me.
I also personally prefer an approach where I don’t need to modify the core behavior of my app to run tests against it (designing for testability is completely different and vital). Some approaches embed some sort of server to receive automation commands, which the user must then remove later because it uses undocumented Apple APIs which will lead to app store rejection. Being able to run tests against the app that you submit to the app store means you can be more confident you have tested it right.
Finally, the iOS app I am working on using embedded web views to display dynamic content from the server, which can be interacted with, and therefore it is vital that these can be interacted with. This feature is actually very rare in an iOS automation framework.
Here’s the list of iOS functional automation tools I am aware of and how they stack up:
|Tool||Language||Test Framework||Supports other mobile plaforms||Approach||Deploy same app to store||Supports testing web-views?|
|KIF||Objective C||OCUnit/SenTest||NO||Modifies your app to use undocumented APIs||NO||NO|
|Subliminal||Objective C||OCUnit/SenTest||NO||Embeds into your project but uses UIAutomation instead of undocumented APIs||NO||NO|
|Calabash||Ruby||Cucumber||Also supports Android Apps||Requires you to embed a server in your app to control it||NO||YES|
I’ll let you guess which tool I selected until I write my next blog post about how to get started in that tool.