Archive for the ‘GHUnit’ Category

GHUnit: Writing Custom Assert Macros

October 24, 2009 Leave a comment

When I evaluated unit testing frameworks for iPhone development, One of the reasons why I chose GHUnit was that it has more sophisticated Assert Macros than other available frameworks. Despite this fact, there are still some Asserts that I missed, so I simply took the time to write my own.

Unlike test frameworks in the .NET or Java ecosystem, all Objective-C Frameworks provide preprocessor macros to realize assertions instead of providing a static class with Assert methods. A typical assert macro looks like the following:

#define GHAssertEquals(a1, a2, description, ...) \
do { \
	@try {\
		if (@encode(__typeof__(a1)) != @encode(__typeof__(a2))) { \
			[self failWithException:[NSException ghu_failureInFile:@"Type mismatch"...]; \
		} else { \
			if (![a1encoded isEqualToValue:a2encoded]) { \
				[self failWithException:[NSException ghu_failureInEqualityBetweenValue...]; \
			} \
		} \
	} \
	@catch (id anException) {\
		[self failWithException:[NSException ghu_failureInRaise...]; \
} while(0)

The body of the macro consists of a do{} while(false) loop, which is used to provide local scope for variables needed to implement the assertion. It is clear that code executed only once, even though a loop construct is used. The macro first checks necessary preconditions, in this case argument types. This is necessary due to the nature of a macro being a simple text substitution rather than a true method call that the compiler checks argument types for (that’s why I don’t like assertions being implemented as macros but would rather like to see assert methods). Next is the actual assertion. The type check and the actual assertion are both wrapped in a try{} catch(){} block, so any errors occurring in the macro code let the test fail also (a real macro would have a lot of code for preparing exception descriptions etc.).

I consider the GHUnit macros as a very useful set of primitve’s that can be combined to construct more complicated assertions:

#define GHFileAssertNotEmpty(file) \
do { \
	GHAssertTrue([[NSFileManager defaultManager] fileExistsAtPath:file], nil); \
	NSString* written = [NSString stringWithContentsOfFile:file]; \
	GHAssertNotNil(written, nil); \
	GHAssertGreaterThan((int)[written length], 0, nil); \
} while (0)

Note that I don’t need to take care of all the nasty details that are needed to write a proper primitive macro as outlined above. The only disadvantage with a macro like the one above is localizing the failed assertion, as the exception thrown might not be directly obvious from the code using the macro. It is not a real disadvantage of the method itself but rather inherent to all macros. XCode right-click Jump to Definition comes to the rescue here.

Categories: GHUnit, iPhone, Open Source, Testing

GHUnit: Parallel test execution performance implications

October 23, 2009 Leave a comment

As my unit test suite for the iRow project starts to grow, I am running into issues regarding test execution speed. I have maintained a clear distinction between integration and unit tests, so there are no external (possibly slow) resources such as disc i/o (including nib’s) or Sqlite databases involved.

I usually run my unit test suite in the Simulator. Having set GHUnit to automatically run my tests on startup this makes it as simple as hitting cmd-r (Xcode Build&Run). It takes some time to update the app in the Simulator, usually around 1-3 secs but I haven’t found this to be an issue as I usually take the time to do some formatting on the code I am currently working on. GHUnit makes it also very convenient to select a subset of tests  that shall be run and persists these settings between builds, so I don’t have to browse through a hundred of tests if one was failing.

Even though only running a subset of tests, it clearly took to long for me (measured 4-5 secs with stopwatch from app startup). This number also had no coincidence with what GHUnit reported as test execution time (around 0.2 secs). Browsing the GHUnit code to see where the time is wasted, I noticed that GHTestCase default implementation of the

- (BOOL)shouldRunOnMainThread

method , which GHUnit uses to determine if the runner needs to spawn off a child thread for executing this testcase, always returns false. Creating a thread is a costly operation in terms of overhead, the necessary synchronization to retrieve test results another. That’s why I suggest deriving all your testcases from a baseclass (which inherits GHTestCase) to have a central point of control about unit test execution (via shouldRunOnMainThread). This yields another positive effect for integration testing. My integration tests often need to be run on the main thread because they require certain input dispatched to the main threads runLoop only.

This is how my implementation of the shouldRunOnMainThread method looks like:

- (BOOL)shouldRunOnMainThread



return TRUE;



return TRUE;



The IROW_INTEGRATION_TESTING symbol is defined in  the integration test project’s prefix header. I think it is a pretty simple but effective solution to control test execution.

Executing all tests on the main thread brought astonishing results: Test time is down to 0.1.secs (measured with stopwatch).  However, it might be interesting to run tests on different threads from time to time to detect possible unintended side effects regarding global state. If tests seem to fail randomly if run multiple times in a row, this is a good indicator for such problems.

GHUnit: New Buildsystem

October 12, 2009 Leave a comment

Managing a project’s source is always challenging, especially when external dependencies are involved and even more when they aren’t binary but source dependencies. That’s how it is with GHUnit.

In my attempt to integrate OCMock, I therefore decided to create a new build system for GHUnit. GHUnit needs to be built for two different platforms: Mac OS X and iPhone OS (Simulator and Device). The first is built as a framework, while the second is built as a single static library combined out of a static simulator and a static device library. Originally, GHUnit made use of the following structure:


  • Classes  // common classes
  • Classes-IPhone // iPhone classes
  • Classes-MacOSX // Mac OS classess

There was an initial structure, not bad. The problem was with the XCode Projects. They were contained in subfolders called Project and ProjectIPhone. To make things worse, the iPhone project contained two different targets for building device and simulator versions with otherwise identical settings. A third target was used to combine them and create a zip archive with the static libraries and the headers that need to be redistributed. Thus, adding a new file to GHUnit would require adding it two three targets in two different solutions. Not nice.

I therefore decided to reorganize the source code and unify the different projects and targets into a single project. Releasing should be a one command action and is therefore done via a simple makefile. The project structure looks now the following:


  • src
    • Classes  // common classes
    • Classes-IPhone // iPhone classes
    • Classes-MacOSX // Mac OS classes
  • test // tests
  • build // build output
  • // buildscript for iPhone, builds simulator and device versions and combines them in a single binarie
  • // buildscript for mac
  • Makefile // invokes redist targets (see below) / runs unit tests

The single project now has a two main targets (for iPhone and mac) and a redist target for each of  these versions. The redist target is necessary to configure the build environment. Afterwards it invokes the appropriate build script.

External dependencies can never be binary becuase iPhone OS only allows static linking. It is far easier to include the sources into GHUnit and compile them into its binary. For GTM, GHKit and BWToolkit dependencies are only on a very small subset of the code. Considering the slow pace of these projects, including the whole projects would be too much overhead. Because GHUnit doesn’t attempt modification of OCMock but a full inclusion the whole OCMock svn trunk is included in the lib/OCMock folder. Some small adjustments on the OCMock Source were nescessary to make it build on the iPhone, I hope these will get included in the official trunk of OCMock.

Categories: GHUnit, iPhone, Testing

GHUnit: OCMock Integration

October 11, 2009 1 comment

As I lined out in a recent post, there is currently no satisfying Testing framework solution available for iPhone projects. I decided to go with GHUnit as it is the most flexible framework and combines the advantages of GTM and SenTestingKit.

Every good Testing framework needs to be integratable with an Isolation Framework. OCMock seems to be the only framework available for Objective-C. Colin Barrett has a nice introduction on how to use OCMock with GTM. His approach is based on the fact that Apple does neither allow linking dynamic libraries, while binaries compiled for the Simulator can without a problem.

GHUnit on the other hand, is compiled as a static library and can therefore be used in both environments, iPhone and Simulator. I think this is one of its major strengths and using plain OCMock would limit me on running Unit tests in the Simulator (what I’m doing most of the time), but from time to time I like to run them on the device just to make sure everything works. Another reason is Integration Testing. In general, there is no need for an Isolation Framework during Integration Testing, but I don’t feel very confident about this at the moment. However, the most important disadvantage would be a further complication of my build process, which I’d like to be clean and easy (GHUnit setup is complicated enough IMHO).

That’s why I decided to integrate OCMock into the GHUnit binaries. I am considering the following usage pattern to be the one with least friction:

  • setup your project to link against static GHUnit library
  • include “GHUnit.h” for unit testing
  • include “OCMock.h” for mocking
  • include “GHUIKitMocks for specialized mocks

The last one is currently in a brainstorm phase. Gabriel Handford has made some efforts to provide Mocks (or are they better called Fakes?) for the CLLocationManager and some network related classes. I have written a UIAccelerometer Fake myself for iRow and that’s why I think there is a general need for specialized UIKit Mocks.

You can find my changes committed here on github.

Categories: GHUnit, iPhone, Open Source, Testing

Get every new post delivered to your Inbox.

%d bloggers like this: