Archive

Archive for October, 2009

GHUnit: Writing Custom Assert Macros

October 24, 2009 Leave a comment

When I evaluated unit testing frameworks for iPhone development, One of the reasons why I chose GHUnit was that it has more sophisticated Assert Macros than other available frameworks. Despite this fact, there are still some Asserts that I missed, so I simply took the time to write my own.

Unlike test frameworks in the .NET or Java ecosystem, all Objective-C Frameworks provide preprocessor macros to realize assertions instead of providing a static class with Assert methods. A typical assert macro looks like the following:


#define GHAssertEquals(a1, a2, description, ...) \
do { \
	@try {\
		if (@encode(__typeof__(a1)) != @encode(__typeof__(a2))) { \
			[self failWithException:[NSException ghu_failureInFile:@"Type mismatch"...]; \
		} else { \
			if (![a1encoded isEqualToValue:a2encoded]) { \
				[self failWithException:[NSException ghu_failureInEqualityBetweenValue...]; \
			} \
		} \
	} \
	@catch (id anException) {\
		[self failWithException:[NSException ghu_failureInRaise...]; \
	}\
} while(0)

The body of the macro consists of a do{} while(false) loop, which is used to provide local scope for variables needed to implement the assertion. It is clear that code executed only once, even though a loop construct is used. The macro first checks necessary preconditions, in this case argument types. This is necessary due to the nature of a macro being a simple text substitution rather than a true method call that the compiler checks argument types for (that’s why I don’t like assertions being implemented as macros but would rather like to see assert methods). Next is the actual assertion. The type check and the actual assertion are both wrapped in a try{} catch(){} block, so any errors occurring in the macro code let the test fail also (a real macro would have a lot of code for preparing exception descriptions etc.).

I consider the GHUnit macros as a very useful set of primitve’s that can be combined to construct more complicated assertions:


#define GHFileAssertNotEmpty(file) \
do { \
	GHAssertTrue([[NSFileManager defaultManager] fileExistsAtPath:file], nil); \
	NSString* written = [NSString stringWithContentsOfFile:file]; \
	GHAssertNotNil(written, nil); \
	GHAssertGreaterThan((int)[written length], 0, nil); \
} while (0)

Note that I don’t need to take care of all the nasty details that are needed to write a proper primitive macro as outlined above. The only disadvantage with a macro like the one above is localizing the failed assertion, as the exception thrown might not be directly obvious from the code using the macro. It is not a real disadvantage of the method itself but rather inherent to all macros. XCode right-click Jump to Definition comes to the rescue here.

Categories: GHUnit, iPhone, Open Source, Testing

GHUnit: Parallel test execution performance implications

October 23, 2009 Leave a comment

As my unit test suite for the iRow project starts to grow, I am running into issues regarding test execution speed. I have maintained a clear distinction between integration and unit tests, so there are no external (possibly slow) resources such as disc i/o (including nib’s) or Sqlite databases involved.

I usually run my unit test suite in the Simulator. Having set GHUnit to automatically run my tests on startup this makes it as simple as hitting cmd-r (Xcode Build&Run). It takes some time to update the app in the Simulator, usually around 1-3 secs but I haven’t found this to be an issue as I usually take the time to do some formatting on the code I am currently working on. GHUnit makes it also very convenient to select a subset of tests  that shall be run and persists these settings between builds, so I don’t have to browse through a hundred of tests if one was failing.

Even though only running a subset of tests, it clearly took to long for me (measured 4-5 secs with stopwatch from app startup). This number also had no coincidence with what GHUnit reported as test execution time (around 0.2 secs). Browsing the GHUnit code to see where the time is wasted, I noticed that GHTestCase default implementation of the

- (BOOL)shouldRunOnMainThread

method , which GHUnit uses to determine if the runner needs to spawn off a child thread for executing this testcase, always returns false. Creating a thread is a costly operation in terms of overhead, the necessary synchronization to retrieve test results another. That’s why I suggest deriving all your testcases from a baseclass (which inherits GHTestCase) to have a central point of control about unit test execution (via shouldRunOnMainThread). This yields another positive effect for integration testing. My integration tests often need to be run on the main thread because they require certain input dispatched to the main threads runLoop only.

This is how my implementation of the shouldRunOnMainThread method looks like:

- (BOOL)shouldRunOnMainThread

{

#ifdef IROW_INTEGRATION_TESTING

return TRUE;

#endif

#ifndef IROW_INTEGRATION_TESTING

return TRUE;

#endif

}

The IROW_INTEGRATION_TESTING symbol is defined in  the integration test project’s prefix header. I think it is a pretty simple but effective solution to control test execution.

Executing all tests on the main thread brought astonishing results: Test time is down to 0.1.secs (measured with stopwatch).  However, it might be interesting to run tests on different threads from time to time to detect possible unintended side effects regarding global state. If tests seem to fail randomly if run multiple times in a row, this is a good indicator for such problems.

Cocoa Memory management conventions

October 15, 2009 Leave a comment

Apple uses reference counting as the default memory management technique in its frameworks. Augmented with an autorelease pool that keeps track of the current stack frame and releases objects when the current stack frame exits, it is a reasonable balance between the performance of manual memory management and the comfort of garbage collection I am usually used to working on .NET projects.

However, reference counting establishes the need for ownership conventions that define who is responsible for releasing an object and under what circumstances I can assume a reference to be valid. Unfortunately, I have found these conventions hard to remember every time I created an object, because there are many of them (at least if you have just started Objective-C programming). Thought as a memo to myself, here follows the (hopefully complete) list of conventions:

Creating Objects:

  • Objects retrieved via methods whose name begins with “alloc” or “new” or contains “copy” are retained. The caller is responsible for releasing the object.
  • CoreFoundation Objects retrieved via C functions whose name contains “Create” or “Copy” are retained. The caller is responsible for releasing the object.
  • In any other case, the caller is not responsible for releasing the object e.g. receiving a string from [NSString stringWithFormat:] returns an autoreleased object.

Loading Objects from NIB Files:

  • Objects in the nib file are created with a retain count of 1 and then autoreleased.
  • UIKit reestablishes connections between the objects using setValue:forKey:, which uses the available setter method or retains the object by default if no setter method is available
  • If there are any top-level objects you not stored in outlets, the caller must retain either the array returned by the loadNibNamed:owner:options: method or the objects inside the array to prevent those objects from being released prematurely.

Receiving Objects

  • The conventions governing receiving objects follow from the rules governing object creation
  • A received object can be assumed valid throughout the scope of the caller
  • If a caller wants to retain an object in another scope (e.g. storing it in a private field) he must call -retain and is responsible for releasing the object when done with it

This guide is probably not complete, however it contains the most important rules. For additional detail please see the resources below.

Sources:

Categories: Design, iPhone

GHUnit: New Buildsystem

October 12, 2009 Leave a comment

Managing a project’s source is always challenging, especially when external dependencies are involved and even more when they aren’t binary but source dependencies. That’s how it is with GHUnit.

In my attempt to integrate OCMock, I therefore decided to create a new build system for GHUnit. GHUnit needs to be built for two different platforms: Mac OS X and iPhone OS (Simulator and Device). The first is built as a framework, while the second is built as a single static library combined out of a static simulator and a static device library. Originally, GHUnit made use of the following structure:

GHUnit/

  • Classes  // common classes
  • Classes-IPhone // iPhone classes
  • Classes-MacOSX // Mac OS classess

There was an initial structure, not bad. The problem was with the XCode Projects. They were contained in subfolders called Project and ProjectIPhone. To make things worse, the iPhone project contained two different targets for building device and simulator versions with otherwise identical settings. A third target was used to combine them and create a zip archive with the static libraries and the headers that need to be redistributed. Thus, adding a new file to GHUnit would require adding it two three targets in two different solutions. Not nice.

I therefore decided to reorganize the source code and unify the different projects and targets into a single project. Releasing should be a one command action and is therefore done via a simple makefile. The project structure looks now the following:

GHUnit/

  • src
    • Classes  // common classes
    • Classes-IPhone // iPhone classes
    • Classes-MacOSX // Mac OS classes
  • test // tests
  • build // build output
  • buildIPhone.sh // buildscript for iPhone, builds simulator and device versions and combines them in a single binarie
  • buildMac.sh // buildscript for mac
  • Makefile // invokes redist targets (see below) / runs unit tests

The single project now has a two main targets (for iPhone and mac) and a redist target for each of  these versions. The redist target is necessary to configure the build environment. Afterwards it invokes the appropriate build script.

External dependencies can never be binary becuase iPhone OS only allows static linking. It is far easier to include the sources into GHUnit and compile them into its binary. For GTM, GHKit and BWToolkit dependencies are only on a very small subset of the code. Considering the slow pace of these projects, including the whole projects would be too much overhead. Because GHUnit doesn’t attempt modification of OCMock but a full inclusion the whole OCMock svn trunk is included in the lib/OCMock folder. Some small adjustments on the OCMock Source were nescessary to make it build on the iPhone, I hope these will get included in the official trunk of OCMock.

Categories: GHUnit, iPhone, Testing

GHUnit: OCMock Integration

October 11, 2009 1 comment

As I lined out in a recent post, there is currently no satisfying Testing framework solution available for iPhone projects. I decided to go with GHUnit as it is the most flexible framework and combines the advantages of GTM and SenTestingKit.

Every good Testing framework needs to be integratable with an Isolation Framework. OCMock seems to be the only framework available for Objective-C. Colin Barrett has a nice introduction on how to use OCMock with GTM. His approach is based on the fact that Apple does neither allow linking dynamic libraries, while binaries compiled for the Simulator can without a problem.

GHUnit on the other hand, is compiled as a static library and can therefore be used in both environments, iPhone and Simulator. I think this is one of its major strengths and using plain OCMock would limit me on running Unit tests in the Simulator (what I’m doing most of the time), but from time to time I like to run them on the device just to make sure everything works. Another reason is Integration Testing. In general, there is no need for an Isolation Framework during Integration Testing, but I don’t feel very confident about this at the moment. However, the most important disadvantage would be a further complication of my build process, which I’d like to be clean and easy (GHUnit setup is complicated enough IMHO).

That’s why I decided to integrate OCMock into the GHUnit binaries. I am considering the following usage pattern to be the one with least friction:

  • setup your project to link against static GHUnit library
  • include “GHUnit.h” for unit testing
  • include “OCMock.h” for mocking
  • include “GHUIKitMocks for specialized mocks

The last one is currently in a brainstorm phase. Gabriel Handford has made some efforts to provide Mocks (or are they better called Fakes?) for the CLLocationManager and some network related classes. I have written a UIAccelerometer Fake myself for iRow and that’s why I think there is a general need for specialized UIKit Mocks.

You can find my changes committed here on github.

Categories: GHUnit, iPhone, Open Source, Testing

The lack of a proper iPhone UnitTest Framework

October 10, 2009 9 comments

One of the main reasons to do the iRow Project was my eager to try TDD on a real project from the beginning. Before starting any work, I made sure there were tools available to drive such a process. Based on my experiences from the .NET world, I made a list of requirements for my development environment (in no particular order):

  • Rich Asserts (Equals, EqualsWithAccuracy, ObjectEquals, CollectionAsserts, throwsAsserts etc.)
  • Reliable/Stable
  • Fast execution speed
  • Run tests inside the debugger
  • Flexible test deployment (Simulator/Device)
  • Isolation Framework available
  • Result feedback via GUI tool or XCode integration

The natural choice was to go with the SenTestingKit provided by Apple. I was prepared to accept some shortcomings as I knew that UnitTest Integration for iPhone projects has just been released with iPhoneOS 3.0 SDK. It is a ported version of SenTe’s OCUnit and extends it with XCode integration. I followed this guide to set up my first test project. I really liked the tests beeing run as a part of my build process and getting instant feedback on test errors inside XCode.

SenTestingKit worked fine for Unit tests, however it has severe shortcomings in Integration testing. The most blocking problem was that I couldn’t get my IntegrationTests running inside the debugger. A real showstopper for me. To make things worse, Unit tests can only be executed when compiled for x86 via the “octest” test runner – no testing inside the Simulator. Integration tests only work on a real device, further complicating the process.

That’s why I started looking for alternatives. The first I came across was the Google Toolbox for Mac, that also offers a test framework for iPhone based on OCUnit. I learned it was the first test framework available for iPhone before the 3.0 SDK. Compared to SenTestingKit it can run tests on the device and inside the simulator and projects are build as normal CocoaTouch Applications, which in theory makes debugging easy. However, it seems test output can only be captured form the debugger console, which is not really what I think is comfortable.

The next project I looked into was GHUnit by Gabriel Handford. It is based on GTM, so it has all the GTM goodies plus it provides a nice GUI (iPhone and Mac). That’s why I didn’t even try GTM but decided to go with GHUnit directly. There were other reasons that made GHUnit attractive to me.

  • Well documented source
  • Example project available, showing off the test GUI and running tests from buildscripts
  • Sourcecode on github, making it easy for me to fork off fixing bugs and adapting to my needs

GHUnit also has its shortcommings, most of them beeing the same as they are for GTM. One of the them is missing XCode Integration but since GHUnit provides a nice GUI, I can live with that. None of the frameworks provides Isolation Framework integration. The only one I know of is OCMock.I will evaluate what to do about that in a future post and also provide some detail on fixes I do already have provided for GHUnit.

Categories: iPhone, Open Source, Testing

Eric Evans: Domain Driven Design

October 9, 2009 Leave a comment

Domain-Driven Design: Tackling Complexity in the Heart of Software

Author : Eric J. EvansEvans - DDD
Pages: 560
Publisher: Addison-Wesley Longman, Amsterdam; Edition: 1. A. (4. September 2003)
Language: English
ISBN-10: 0321125215
ISBN-13: 978-0321125217

Personal Summary:

Eric Evans introduces a new approach to Software Modeling: Domain Driven Design (DDD). Just as the title says it deals with techniques and best practices for managing domain complexity. At the core of it lies the concept of “unavoidable complexity” that is, the complexity of the domain. It is all about creating a model of your domain in software that captures all the knowledge necessary to solve the problems the software is intended to.

Evans starts with describing the development process necessary to make this goal achievable. Knowledge Crunching is all about invoking the Domain Experts and making the Developers understand the Domain. A Ubiquitos Language with a sharp description of all its elements must evolve, to make communication efficient and less prone to errors. An agile, feedback driven process is necessary for Binding Model and Implementation that is, feeding back insight gained during model implementation into the modeling process.

This takes us to the next important concept: Insight. Insight describes the process at a human level. It is the core challenge of building software that is intended to solve real-world problems. Software Developers mustn’t be specialists – they need to be generalists: engineers. A good developer bridges the gap between technology and domain because he is an expert in both. Eric Evans acknowledges it is impossible to be an expert in every domain without having experience with it.He therfore appreciates Insight and advocates refining the model when new Insight is gained. Through continuous Knowledge Crunching and Distillation a Deep Model eventually emerges.

A process like this requires proactive structuring of the code base to accomodate change – Supple Design. Something which I think is a very essential concept. It is necessary for a Test Driven Design Process or in areas of the domain that are likely to evolve in the future. For this purpose he describes a set of patterns that help abstracting domain object responsibilities into explicit concepts, which is another core tenet of Supple Design: Making Implicit Concepts Explicit. This applies not only to the structure of the code base but also to the concepts of the domain.

Opinion and Conclusion:

I think this book is a must read for everyone that considers software development to be a craft. The parallels to engineering are astonishing. Eric Evans has a very explicit style of writing, which accounts a lot to the value of this book. The sharp definitions and clear distinctions make it possible to convey a lot of information in a very dense manner. I wonder if Evans is a mathematician :-)

Links and Accompanying Books:

Categories: Book Reviews Tags: ,

Introducing the ‘My Bookshelf’ Series

October 7, 2009 Leave a comment

In the following weeks I want to write reviews about the programming and technology related books sitting on my bookshelf. You can find an up to date list of these books here, where I will also link the corresponding Blog Post.

There are several goals I try to accomplish with this series:

  • Recapitulate gained knowledge, focusing on the most crucial aspects
  • Keeping track of what books I do have but didn’t have time to read yet
  • A Wishlist of Books I plan to buy, mostly focused on those areas I want to enrich my knowledge
  • Provide a ‘get started on a topic’ list for others

I think the format will be something like:

  • Title, Author, Pages, ISBN, Picture, Amazon link
  • Short Summary of Content
  • Most important thing i learned
  • What did I like? What not?
  • Conclusion, good accompanying books
Categories: Book Reviews Tags:

Hello world!

October 5, 2009 Leave a comment

Today’s heroic words:

Hello World!

Categories: General
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: