Kiwi as a static framework or Universal Library

A problem commonly encountered when using open-source iOS frameworks is the lack of a fully-functional framework facility in xCode. Part of the issue is that Apple does not allow dynamic linking on iOS devices, the other is that there are two different architectures that need to be supported by libraries targeting both armv6 (up to iPhone 3G) and armv7 devices (iPhone 3GS and later). On top of that, we also need a binary that will run on the simulator (x86).

The easiest solution to the library problem in XCode is using project dependencies to build libraries in the configuration you need them. When taking a source dependency is not desirable, you are pretty much left to your own if the OSS project doesn’t provide binaries.

Fortunately enough, it’s not too difficult to build your own universal frameworks. Below are the steps I use for building a version of Kiwi:

1. Grab the Universal Framework XCode templates from https://github.com/kstenerud/iOS-Universal-Framework
2. Install the Fake framework flavor (although the Real framework flavor should work as well)
3. Create a new xCode project with the Fake framework template
4. Add all source files of Kiwi (make sure to check the Copy to destination group folder box)
5. Select the Kiwi static library target, project editor, build phases, Copy Headers, select all headers in the Project Group, right click and select move to Public
6. Select the Kiwi static library target, project editor, build phases, link binary with libraries and add SenTestingKit.framework
7. Build
8. Go to the  Project Navigator (Cmd-1) and select Products Kiwi.framework. Right-Click and select “Show in Finder”
9. You should see two folders: Kiwi.framework and Kiwi.embeddedframework – Kiwi.framework is the one we need
10. Copy the Kiwi.framework folder into your lib folder
11. Open the project you want to use Kiwi.framework in and select your target, project editor, build phases, link binary with libraries, click + and add Kiwi.framework from your lib folder

That’s it. Takes less than two minutes once you know the trick.

Continous Deployment for Apps via testflightapp

The benefits of continous integration are widely known. By extending the ideas of continous integration to the full software lifecycle, continous delivery becomes an inevitable practice. Especially in the context of managing a beta program for mobile devices, to most of which I as a developer have no physical access, the ability to have fully automated deployments is crucial.

Testflightapp.com provides a great service to iOS developers by managing app provisioning and deployments. They do also provide easy to use instrumentation facilities.

Continous deployment with testflightapp.com is a breaze, all you need your build-server to do is interact with a straightforward web-api to upload your ipa packages.
Here’s the script I’m using for RowMotion:

#!/bin/bash
# 

# testflightapp.com tokens
API_TOKEN="YOUR_API_TOKEN"
TEAM_TOKEN="YOUR_TEAM_TOKEN"

PRODUCT_NAME="RowMotion"
ARTEFACTS="$PWD/Artefacts"

SIGNING_IDENTITY="iPhone Distribution"
PROVISIONING_PROFILE="$PWD/id/RowMotionAdHoc.mobileprovision"

# calculated vars
OUT_IPA="${ARTEFACTS}/${PRODUCT_NAME}.ipa"
OUT_DSYM="${ARTEFACTS}/${PRODUCT_NAME}.dSYM.zip"

# kill artefacts directory
rm -rf $ARTEFACTS
mkdir $ARTEFACTS

# compile
echo "##teamcity[compilationStarted compiler='xcodebuild']"
xcodebuild -workspace RowMotion.xcworkspace -scheme RowMotion -sdk iphoneos5.0 -configuration Release build archive
buildSucess=$?

if [[ $buildSucess != 0 ]] ; then
echo "##teamcity[message text='compiler error' status='ERROR']"
echo "##teamcity[compilationFinished compiler='xcodebuild']"
exit $buildSucess
fi

echo "##teamcity[compilationFinished compiler='xcodebuild']"

#ipa
echo "##teamcity[progressMessage 'Creating .ipa for ${PRODUCT_NAME}']"

DATE=$( /bin/date +"%Y-%m-%d" )
ARCHIVE=$( /bin/ls -t "${HOME}/Library/Developer/Xcode/Archives/${DATE}" | /usr/bin/grep xcarchive | /usr/bin/sed -n 1p )
DSYM="${HOME}/Library/Developer/Xcode/Archives/${DATE}/${ARCHIVE}/dSYMs/${PRODUCT_NAME}.app.dSYM"
APP="${HOME}/Library/Developer/Xcode/Archives/${DATE}/${ARCHIVE}/Products/Applications/${PRODUCT_NAME}.app"

/usr/bin/xcrun -sdk iphoneos PackageApplication -v "${APP}" -o "${OUT_IPA}" --sign "${SIGNING_IDENTITY}" --embed "${PROVISIONING_PROFILE}"

#symbols
echo "##teamcity[progressMessage 'Zipping .dSYM for ${PRODUCT_NAME}']"
/usr/bin/zip -r "${OUT_DSYM}" "${DSYM}"

# prepare build notes
NOTES=`hg tip`

#upload
echo "##teamcity[progressMessage 'Uploading ${PRODUCT_NAME} to TestFlight']"

/usr/bin/curl "http://testflightapp.com/api/builds.json" \
-F file=@"${OUT_IPA}" \
-F dsym=@"${OUT_DSYM}" \
-F api_token="${API_TOKEN}" \
-F team_token="${TEAM_TOKEN}" \
-F notes="${NOTES}" \
-F notify="True" \
-F distribution_lists="Private"

Make sure to adapt the script to your requirements. One trick I’m fond of is automatically providing SCM information in the build notes (the step with executing hg tip does just that).
For deployments I’m using two lists, a private one to which all builds will be published, and a public one to which I can selectively deploy. What’s so great about testflightapp.com is that it will automatically send emails to notify my testers about the new build and will then allow me to monitor installs.

Retrieving Coverage Information – LLVM, CoverStory and Teamcity

Abstract

This time, we will set up automated code coverage metrics retrieval from our unit and integration test suites. When I started writing my iOS Continous Integration Series, we needed to use GCC and GCov to generate coverage information. Fortunately, this has changed and starting with iOS 5 Beta 4 XCode ships a version of LLVM that is capable of generating coverage information.

We will use CoverStory to create a (not-so-pretty but useful) HTML report of our coverage information and teamcity will pick this up and display it in the Build Result tab.

Collecting Metrics

Setting up LLVM to generate coverage information is easy. The following is an augmented version of the instructions found at the CoverStory Wiki:

1. Edit all test targets, add -fprofile-arcs and -ftest-coverage to Other C Flags
2. Edit all test targets, select the “Build Phases” tab and add /Developer/usr/lib/libprofile_rt.dylib to the stage “Link Binary WIth Libraries”
3. Build and run, you should see *.gcda and *.gcno files generated in your bin directory

HTML Reports with CoverStory

Once the Build Script compiled the product and executed all the tests, we need to pick up the coverage information generated. From this data, we need to generate a report that shows the number of lines covered. I’m using CoverStory to do just this. Unfortunately, it doesn’t come with a command line interface, so I had to resort to some hacked AppleScript to drive the process (note that I checked in CoverStory in my source tree under tool/CoverStory.app). The script takes two command line parameters, the first being the directory to search for coverage information, the second being the output directory for the generated html report.

on run argv
	
	tell application "tool/CoverStory.app"
	    quit
		activate
		set x to open (item 1 of argv)
		tell x to export to HTML in (item 2 of argv)
		quit
	end tell
	
	return item 1 of argv & "|" & item 2 of argv
	
end run

Ok, so that’s all nice and sweet. However, there is another issue I hit with CoverStory (which has already been filed as a bug with the maintainers). Basically, CoverStory would startup and freeze upon opening your coverage information. The Link also has a patch fixing it, that did not make it’s way into an official release yet. I do have a private build of CoverStory with the fix applied, If you are feel unable to build CoverStory yourself email me and I’ll be happy to provide you with my build. I hope this becomes obsolete very soon.

Alright, where did we left off? We have an apple script for collecting coverage information. This needs to be included in our build script. So our Shell based build script looks like this now:

# build here

# test here

# collect coverage results
echo "##teamcity[progressStart 'Collecting Coverage results']"
osascript coverStory.applescript $PWD $PWD/Artefacts/Coverage
echo "##teamcity[progressFinish 'Collecting Coverage results']"

Note that $PWD denotes the current working directory, i.e. the directory the build script is executed from. CoverStory will generate  a HTML based report and put it in our Artefacts directory in a subdirectory called Coverage. Next, we’re going to make Teamcity pick up this report.

Integrating Coverage Reports with Teamcity

Teamcity does an admirable job at managing artefacts. As a first step, we need to edit our build configuration and add an Artifact Path: “%system.teamcity.build.checkoutDir%/Artefacts/Coverage => coverage.zip”. This will cause TeamCity to take the Coverage Report and put it into a coverage.zip archive that it sucks in and stores along the build results.

Next, we edit the Teamcity Server Configuration, chose “Report Tabs” and hit “Create new report tab” with the following settings:

• Tab Title: Code Coverage
• Base path: coverage.zip
• Start page: index.html

Teamcity will display an iframe with the contents of index.html of the coverage.zip artifact we created in the previous step. If everything went well, your build results should look like this now:

iOS: Detect Personal Hotspot

When you want to detect the type of available connections on an iPhone, the best resource you can find on the web is the sample code from Erica Sadun’s excellent iPhone Cookbook book (which I can wholeheartedly recommend). The sample code can be found on github (look into 02 and 03): https://github.com/erica/iphone-3.0-cookbook-/tree/master/C13-Networking

While the solution presented is great, it fails to work on an iPhone 4 that has the Personal Hotspot feature enabled. In this scenario, the iPhone will create a network interface called “ap0″ that bridges through to “en0″ (WiFi) and “pdp_ip0″ (3G) . Since “en0″ will not be marked as AF_INET interface in this scenario, the approach Erica outlined will fail here. Here’s a dump of the available interfaces, their loopback and AF_INET status and their assigned address:

2011-07-22 12:59:07.120 RowMotion[286:707] name: lo0, inet: 0, loopback: 0, adress: 24.3.0.0
2011-07-22 12:59:07.126 RowMotion[286:707] name: lo0, inet: 0, loopback: 0, adress: 0.0.0.0
2011-07-22 12:59:07.129 RowMotion[286:707] name: lo0, inet: 1, loopback: 0, adress: 127.0.0.1
2011-07-22 12:59:07.134 RowMotion[286:707] name: lo0, inet: 0, loopback: 0, adress: 0.0.0.0
2011-07-22 12:59:07.137 RowMotion[286:707] name: en0, inet: 0, loopback: 1, adress: 6.3.6.0
2011-07-22 12:59:07.141 RowMotion[286:707] name: ap0, inet: 0, loopback: 1, adress: 6.3.6.0
2011-07-22 12:59:07.145 RowMotion[286:707] name: pdp_ip0, inet: 0, loopback: 1, adress: 255.7.0.0
2011-07-22 12:59:07.149 RowMotion[286:707] name: pdp_ip0, inet: 1, loopback: 1, adress: 10.217.22.129
2011-07-22 12:59:07.154 RowMotion[286:707] name: pdp_ip1, inet: 0, loopback: 1, adress: 255.7.0.0
2011-07-22 12:59:07.157 RowMotion[286:707] name: pdp_ip2, inet: 0, loopback: 1, adress: 255.7.0.0
2011-07-22 12:59:07.161 RowMotion[286:707] name: pdp_ip3, inet: 0, loopback: 1, adress: 255.7.0.0
2011-07-22 12:59:07.165 RowMotion[286:707] name: en1, inet: 0, loopback: 1, adress: 6.3.6.0
2011-07-22 12:59:07.168 RowMotion[286:707] name: bridge0, inet: 0, loopback: 1, adress: 6.7.6.0
2011-07-22 12:59:07.172 RowMotion[286:707] name: bridge0, inet: 1, loopback: 1, adress: 172.20.10.1

See that last line? Yep, that’s the bridge interface we need to use to communicate with other devices on our “personal hotspot”. Here’s how to ammend Erica’s code to make personal hotspots transparent:

// Matt Brown's get WiFi IP addy solution
// http://mattbsoftware.blogspot.com/2009/04/how-to-get-ip-address-of-iphone-os-v221.html
+ (NSString *) localWiFiIPAddress
{
    BOOL success;
    struct ifaddrs * addrs;
    const struct ifaddrs * cursor;
    
    success = getifaddrs(&addrs) == 0;
    if (success) {
        cursor = addrs;
        while (cursor != NULL) {
            
            NSString *name = [NSString stringWithUTF8String:cursor->ifa_name];
            
            NSLog(@"available network interfaces: name: %@, inet: %d, loopback: %d, adress: %@", name, cursor->ifa_addr->sa_family == AF_INET, (cursor->ifa_flags & IFF_LOOPBACK) == 0, [NSString stringWithUTF8String:inet_ntoa(((struct sockaddr_in *)cursor->ifa_addr)->sin_addr)]);
            
            // the second test keeps from picking up the loopback address
            if (cursor->ifa_addr->sa_family == AF_INET && (cursor->ifa_flags & IFF_LOOPBACK) == 0) 
            {
                if ([name isEqualToString:@"en0"] || [name isEqualToString:@"bridge0"])  //  Wi-Fi adapter, or iPhone 4 Personal hotspot bridge adapter
                    return [NSString stringWithUTF8String:inet_ntoa(((struct sockaddr_in *)cursor->ifa_addr)->sin_addr)];
            }
            cursor = cursor->ifa_next;
        }
        freeifaddrs(addrs);
    }
    return nil;
}

+ (BOOL) activeWLAN
{
    return ([self localWiFiIPAddress] != nil);
}

+ (BOOL) activePersonalHotspot
{

    // Personal hotspot is fixed to 172.20.10
    return ([self activeWLAN] && [ hasPrefix:@"172.20.10"]);
}

+ (BOOL) activeWLAN
{
    return ([self localWiFiIPAddress] != nil);
}

+ (BOOL) activePersonalHotspot
{
    // Personal hotspot is fixed to 172.20.10
    NSString* localWifiAddress = [self localWiFiIPAddress];
    return (localWifiAddress != nil && [localWifiAddress hasPrefix:@"172.20.10"]);
}

I hope this will find it’s way into the sample code soon. Pull request is pending.

LLVM/Clang Code Coverage about to come

According to information from this LLVM bug report, Nick Lewycky recently implemented support for generating gcov compatible coverage files from LLVM/Clang. I’m not keen to replace my local LLVM with a svn build, but I’m really looking forward to finally ditch gcc.

Joomla backups made easy

This post sums up the backup strategy I’m using for the website of my next project RowMotion. RowMotion is hosted on a Joomla installation. My hoster is nice enough to provide a decent pre-built mysqldump based backup script, which can be found here. Did I mention it does email notifications too? All the necessary instructions to set it up are mentioned there too.

Okay, so now we have a nice backup script for our Joomla database that needs to be triggered and the resulting backup file downloaded? Since the web-request needs to be authenticated, I figured it would be easiest to use some PowerShell magic to leverage the .NET WebClient. Here’s the full script:

$username = "un"
$password = "pw"
$backupDir = "C:\backup\"

$web_client = New-Object System.Net.WebClient;
$web_client.Credentials = new-object System.Net.NetworkCredential($username, $password)

$response = $web_client.DownloadString("http://yourDomain.com/backup/databaseBackup.phpx");
echo "Response:\n"
echo $response

$regex = "http://yourDomain.com.com/backup/.*?.gz";

$response -match $regex
$dumpUrl = $matches[0]

echo "URL:\n"
echo $dumpUrl

$fn = [System.IO.Path]::GetFileName($dumpUrl);

echo "Filename:\n"
echo $fn

$target = [System.IO.Path]::Combine($backupDir, $fn);

echo "Target:\n"
echo $target

$web_client.DownloadFile($dumpUrl, $target);

Sure enough, this is neither pretty nor the most robust, but it’s the simplest thing that could possibly work (and it does). Next, we need to schedule this task with the Windows Task Scheduler. I’m running this on my server together with all the other backup tasks.

Be sure to enter “powershell” as the command and “-noprofile -command “C:\backupJobs\yourps.ps1″ “as the argument.

Mercurial Server using hgweb.cgi on Ubuntu

In a previous post, we set up a virtual machine template for an ubuntu server. Now that we have set up a clone of this machine, it is time to set up a Mercurial repository server

Abstract

Mercurial provides an easy to use repository server via a python cgi script. Mercurials protocol facilitates fast transfers over http, making it superior over an ssh solution based (such as git) when considering minimal protocol overhead vs. ease of use. As my webserver of choice, we will use lighttpd. This guide will follow the instructions published here http://mercurial.selenic.com/wiki/PublishingRepositories#multiple

Installing Python

This one is simple:

ubuntu@localhost: sudo apt-get install python

Installing Lighttpd

To install lighttpd, run:

ubuntu@localhost: sudo apt-get install lighttpd

Next, we need to create a specific configuration for our mercurial cgi script. We need to redirect all incoming requests to the cgi script, and then we apply some url rewrite magic to remove the ugly hgweb.cgi from our URLs. The hgweb.cgi script will be served from /var/www/hgweb.cgi. If you use a different location, make sure to chown it to www-data and chmod+x it (all described in the mercurial wiki). I created my config like this:

ubuntu@localhost:~$ sudo vi /etc/lighttpd/hg.conf
ubuntu@localhost:~$ cat /etc/lighttpd/hg.conf
url.rewrite-once = (
  "^([/?].*)?$" => "/hgweb.cgi$1",
   "^/([/?].*)?$" => "/hgweb.cgi$1"
)

$HTTP["url"] =~ "^/hgweb.cgi([/?].*)?$" {
             server.document-root = "/var/www/"
             cgi.assign = ( ".cgi" => "/usr/bin/python" )
}

Next is the lighttpd config, that will need to reference our hg.conf and enable mod_cgi:

ubuntu@localhost:~$ cat /etc/lighttpd/lighttpd.conf
include "hg.conf"

server.modules = (
	"mod_access",
	"mod_alias",
	"mod_compress",
 	"mod_redirect",
        "mod_rewrite",
        "mod_cgi"
)

Further configuration Tricks

You should force hgweb.cgi to serve UTF-8 content. Fortunately enough, this is as simple as adding (or uncommenting) the following lines to hgweb.cgi:

import os
os.environ["HGENCODING"] = "UTF-8"

You will also need a hgweb.conf right next to hgweb.cgi and reference it from there (again, described in the mercurial wiki). For reference, my configuration includes all repos sourced under /var/hg/repos (and subdirectories) and allows anonymous push (I’m authenticated via VPN policy):

ubuntu@localhost:~$ cat /var/www/hgweb.config
[paths]
/ = /var/hg/repos/**

[web]
baseurl = /
allow_push = *
push_ssl = false

Final Words

That’s all there is to it. To make the server available via DNS, you need to make sure the servers hostname is registered with your local DNS. In my case, I simply added a static record to it.

Objective-C Pitfall: Synthesized Properties without backing field

This is just a quick and short post about an Objective-C pitfall I have encountered today. When using synthesized properties, you do normally supply a backing field:

@property (nonatomic, readwrite, retain) Message* message = message_;

This will synthesize a getter and setter, that will use message_ as its backing field. Since I found out one can go clever and ommit the backing field, so a simple line like this will work too:

@property (nonatomic, readwrite, retain) Message* message;

However, now we get into a bit of trouble when accessing the property. Contrary to the behavior in Java or C#, you now get something different when accessing 

self.message

 vs.

message

. While the former will use the synthesized getter, the latter will use the synthesized backing field directly. This is a bit unexpected (I thought the backing field would be anonymous). So, my general advice would be to always use backing fields in your synthesized backing fields, so you don’t accidentally forget a “self.”. (This is water on the mills of people that advocate _not_ using the dot syntax for properties).

 

 

TeamCity Server on Ubuntu

Last time, we set up a virtual machine template for an ubuntu server. Now that we have set up a clone of this machine, it is time to set up Teamcity on it.

Abstract

Teamcity on linux is meant to be run from its integrated Tomcat server. We will use the default Teamcity installation procedure in combination with the lightweight lighttpd to act as a front end server listening on port 80 and forwarding requests to Teamcity’s Tomcat installation. This setup is both, easier than configuring Tomcat on Port 80 (remember it requires root permissions to allocate) and we could add authentication or https access more easily later (though I will not do that for now).

Installing Teamcity

To install Teamcity, follow the instructions from JetBrains, which can be found here (takes less than 10 minutes). I chose to install mine at /var/TeamCity:
http://confluence.jetbrains.net/display/TCD65/Installing+and+Configuring+the+TeamCity+Server#InstallingandConfiguringtheTeamCityServer-installingWithTomcat 

Follow the instructions to set up your external database (recommended approach). I am using a SQL Server 2008 Installation that is already present and regularly backed up in my “private cloud”.  Edit your server.conf to configure a port for the Tomcat Server :

ubuntu@localhost: sudo vi /var/TeamCity/conf/server.xml

Permissions are a chore, but we don’t want the Teamcity Server directory to be owned by our admin user, so we change the owner of our Teamcity install directory to the default www-data user.

ubuntu@localhost: sudo chown -R www-data /var/TeamCity

Next, we want Teamcity to start automatically when the server is booted, so we add a small init script. Be sure to adjust the TEAMCITY_DATA_PATH environment variable to a static directory of your choice, otherwise TCs default will make end up in www-data’s home directory, which is, frankly, a very inconvenient location.

ubuntu@localhost:/var/TeamCity$ cat /etc/init.d/teamcity
#!/bin/sh
# /etc/init.d/teamcity -  startup script for teamcity
export TEAMCITY_DATA_PATH="/var/TeamCity/.BuildServer"

case $1 in
start)
 start-stop-daemon --start  -c www-data --exec /var/TeamCity/bin/teamcity-server.sh start
;;

stop)
 start-stop-daemon --start -c www-data  --exec  /var/TeamCity/bin/teamcity-server.sh stop
;;

esac

exit 0

Now we need to register the startup script to run automatically:

ubuntu@localhost: sudo update-rc.d teamcity defaults

Next, we start the server manually (you can reboot too):

ubuntu@localhost: sudo /etc/init.d/teamcity start

Installing Lighttpd

Now we need to install lighttpd:

ubuntu@localhost: sudo apt-get install lighttpd

And configure it to forward requests from port 80 to the port we configured for Tomcat (8080 in my case).

ubuntu@localhost: sudo vi /etc/lighttpd/lighttpd.conf
server.modules = (
        "mod_access",
        "mod_alias",
        "mod_compress",
        "mod_redirect",
#       "mod_rewrite",
        "mod_proxy"
)

$HTTP["host"] =~ "teamcity.yourdomain.com" {
        proxy.server = (
                "" => (
                        "tomcat" => (
                                "host" => "127.0.0.1",
                                "port" => 8080,
                                "fix-redirects" => 1
                        )
                )
        )
}

Final Words

That’s it. By now you should have a running teamcity server. If something goes wrong, be sure to check the logs which can be found at /var/TeamCity/logs. To make the server available via DNS, you need to make sure the servers hostname is registered with your local DNS. In my case, I simply added a static record to it.

Ubuntu Server on HyperV

As my Linux distro of choice for a set of lightweight virtualized Servers, Ubuntu Sever provides several advantages that made me go for it:

* Driver support for HyperV
* Active Community, number of HowTo’s available considered superior to Debian
* Packages available are cutting edge
* Well documented
* Good experience with Ubuntu Desktop

At the time of writing this, there are two choices of Ubuntu Server: Ubuntu 10 with LTS or cutting edge Ubuntu 11. LTS stands for LongTermSupport and Cannonical guarantees there will be updates for at least 3 years. It’s a matter of preference, but I chose Ubuntu 11.

To install it in HyperV, I recommend you follow this guide: http://social.technet.microsoft.com/wiki/contents/articles/how-to-install-ubuntu-server-10-04-in-hyper-v.aspx

Of course, you should adapt your network configuration to your requirements. Before templating this machine, I installed openssh because I consider it a core part of my server administration toolkit.

By this point, we should have a core installation of Ubuntu Server template that is ready to be cloned. Depending on your virtualization solution of choice, different steps apply here. (In HyperV it is as simple as exporting and re-importing the machine.) Make sure you create a unique copy of the machine, so that its network adapter gets assigned a new MAC Adress (everything else is calling for trouble).

After instantiating your template, we now need to customize that template:

1. Change hostname
 sudo hostname "NewHostName"

2. Configure Networking (it is likely your adapter will now show up as eth1 instead of eth0, rember that instantiating a VM tempate involves changing the MAC adress of the server)

sudo vi /etc/network/interfaces

3. Change user name/password

sudo passwd

4. Reboot

sudo shutdown -r now