The lack of a proper iPhone UnitTest Framework
One of the main reasons to do the iRow Project was my eager to try TDD on a real project from the beginning. Before starting any work, I made sure there were tools available to drive such a process. Based on my experiences from the .NET world, I made a list of requirements for my development environment (in no particular order):
- Rich Asserts (Equals, EqualsWithAccuracy, ObjectEquals, CollectionAsserts, throwsAsserts etc.)
- Fast execution speed
- Run tests inside the debugger
- Flexible test deployment (Simulator/Device)
- Isolation Framework available
- Result feedback via GUI tool or XCode integration
The natural choice was to go with the SenTestingKit provided by Apple. I was prepared to accept some shortcomings as I knew that UnitTest Integration for iPhone projects has just been released with iPhoneOS 3.0 SDK. It is a ported version of SenTe’s OCUnit and extends it with XCode integration. I followed this guide to set up my first test project. I really liked the tests beeing run as a part of my build process and getting instant feedback on test errors inside XCode.
SenTestingKit worked fine for Unit tests, however it has severe shortcomings in Integration testing. The most blocking problem was that I couldn’t get my IntegrationTests running inside the debugger. A real showstopper for me. To make things worse, Unit tests can only be executed when compiled for x86 via the “octest” test runner – no testing inside the Simulator. Integration tests only work on a real device, further complicating the process.
That’s why I started looking for alternatives. The first I came across was the Google Toolbox for Mac, that also offers a test framework for iPhone based on OCUnit. I learned it was the first test framework available for iPhone before the 3.0 SDK. Compared to SenTestingKit it can run tests on the device and inside the simulator and projects are build as normal CocoaTouch Applications, which in theory makes debugging easy. However, it seems test output can only be captured form the debugger console, which is not really what I think is comfortable.
The next project I looked into was GHUnit by Gabriel Handford. It is based on GTM, so it has all the GTM goodies plus it provides a nice GUI (iPhone and Mac). That’s why I didn’t even try GTM but decided to go with GHUnit directly. There were other reasons that made GHUnit attractive to me.
- Well documented source
- Example project available, showing off the test GUI and running tests from buildscripts
- Sourcecode on github, making it easy for me to fork off fixing bugs and adapting to my needs
GHUnit also has its shortcommings, most of them beeing the same as they are for GTM. One of the them is missing XCode Integration but since GHUnit provides a nice GUI, I can live with that. None of the frameworks provides Isolation Framework integration. The only one I know of is OCMock.I will evaluate what to do about that in a future post and also provide some detail on fixes I do already have provided for GHUnit.