Home > .NET, Testing, Tools > Running MSTest 9 on a CI Server without installing Visual Studio

Running MSTest 9 on a CI Server without installing Visual Studio

Disclaimer: I would have loved to migrate to a different framework (and I would strongly advice you do so if you’re not a full stack TeamSystem shop), however I have a couple of consultants on that project who are not very test experienced and having built-in MSTest has compelling advantages. Having said that, I know that Gallio has nice VS integration that you can use to run any frameworks’ tests inside Visual Studios Test windows, however that would require each developer to install gallio on their machine (which is bad too).

Without reiterating the tirades of hate Microsoft has earned for making it impossible to run MSTest on a build server without installing Visual Studio, I want to present what I have compiled from several sources to get it working for me:

  1. See this post on Stackoverflow for an overview of the issue and possible solutions
  2. Mark Kharitonov has compiled a basic set of instructions that allow installing MSTest on a Build Server

My setup consists of a Teamcity Build Agent running on Windows Server 2008R2 x64, so I needed to change all registry keys in the reg file to point at HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\ instead of HKEY_LOCAL_MACHINE\SOFTWARE\.

Next, I am using Gallio to run the tests instead of executing them directly using MSTest. Even though Gallio is considerably slower than native MSTest, which you can also use with a built-in Teamcity buildstep, there are a couple of advantages:

  1. Pretty Reports
  2. No need to deal with test run configurations and test metadata (I’ve got no idea what they are and why I would need them)
  3. Teamcity picks up the test resulty properly
  4. I can use a MSBuild script to pick up my Test dlls via wildcards, no need to have extra MSTest build tasks.

As a reference, here’s my MSBuild script for running the tests using Gallio:

<Project DefaultTargets="Test" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
	<!-- This is needed by MSBuild to locate the Gallio task -->
    <UsingTask AssemblyFile="tools\Gallio\Gallio.MSBuildTasks.dll" TaskName="Gallio" />
	<!-- Specify the tests assemblies -->
        <TestAssemblies Include="src\test\**\bin\$(Configuration)\*Tests.dll" />
	<Target Name="Test">
            <!-- This tells MSBuild to store the output value of the task's ExitCode property
                 into the project's ExitCode property -->
            <Output TaskParameter="ExitCode" PropertyName="ExitCode"/>
		<Error Text="Tests execution failed" Condition="'$(ExitCode)' != 0" />
Categories: .NET, Testing, Tools
  1. August 8, 2011 at 23:24

    Did you use the 3.2.3 version of Gallio? Any chance you could list what DLLs are located in your “tools\Gallio” folder?

  2. August 9, 2011 at 14:44

    Hi Robert,
    my Dlls identify themselves as (AssemblyVersion) and 3.2.744 (FileVersion) repsectively. Hope that helps.

  3. August 9, 2011 at 15:14

    Looks like we are using the same version of Gallio (3.2.3). Besides Gallio.MSBuildTasks.dll, what do you have in your tools\Gallio folder?

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: