I was on a project which was using TFS 2008 to manage source control and builds but not using MSTest for unit testing. Instead we were using NUnit which is kind of a shame since MSTest is nicely integrated into Visual Studio and you don't have to jump through hoops to get unit test reports from a TFS build.
If you find yourself in this situation, my primary advice would be to ignore the rest of this article and just use MSTest with TFS 2008 (why wouldn't you? I won't go into why we didn't but it wasn't a good reason). But if you have to jump through those hoops, here's how I did it. The results were reasonably good but still not as good as MSTest would have given.
Firstly, credit goes to the following articles whose ideas were merged into my final result:
The build instructions below do the following:
-
Integrate running of NUnit unit tests in automated build
-
Run the unit tests using NCover to give a report on unit testing code coverage
-
Translate the unit test results into a format that can be interpreted by Visual Studio so they can be viewed within the VS IDE
Instructions
1. Install Visual Studio Team Server Dev edition on the build server to allow code analysis and unit test reporting within VS IDE.
2. Install NUnit, NCover and NCoverExplorer on the build server.
3. Install the "MSBuild.Community.Tasks.Targets" file on the build server. The installation file and instructions are available from http://msbuildtasks.tigris.org/.
4. Copy the "NCoverExplorer.MSBuildTasks.dll" file to folder "c:\program files\msbuild\customtargets".
The dll is available from www.kiwidude.com/dotnet/downloadpage.html in NCoverExplorerExtras 1.4.0.5.
5. Install the following XSLT command line tool on the build server:
NXSLT.exe - http://www.xmllab.net/Downloads/tabid/61/Default.aspx (free command line tool for performing XML transformations).
6. Install NUnit for Team Build on the build server:
NUnit for Team Build - http://www.codeplex.com/nunit4teambuild (a set of XSLT templates which can be used for transforming NUnit test results into Microsoft Test results).
7. Create a targets file to reside on the build server as "c:\program files\msbuild\customtargets\NUnit.Targets". Contents listed below.
8. Edit the TFSBuild.proj file to import the "MSBuild.Community.Tasks.Targets" community targets file and the NUnit.Targets file as follows:
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets"/>
<Import Project="$(MSBuildExtensionsPath)\customtargets\NUnit.Targets"/>
9. In the TFSBuild.proj file, override the AfterTest target to call the unit tests target.
<Target Name="AfterTest" DependsOnTargets="RunUnitTests"></Target>
Note that the normal AfterTest target is the one that runs MSTest unit tests so it is safe to override if you have no MSTest tests.
The content of the NUnit.Targets file is:
<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" >
<!-- NOTE: any build agent machine must have the latest custom targets files in "C:\Program Files\MSBuild\CustomTargets" prior to build -->
<!-- NOTE: NCoverExplorer.MSBuildTasks.dll must also be present in the same folder -->
<UsingTask TaskName="NCoverExplorer.MSBuildTasks.NCoverExplorer" AssemblyFile="NCoverExplorer.MSBuildTasks.dll"/>
<UsingTask TaskName="NCoverExplorer.MSBuildTasks.NCover" AssemblyFile="NCoverExplorer.MSBuildTasks.dll"/>
<UsingTask TaskName="NCoverExplorer.MSBuildTasks.NUnitProject" AssemblyFile="NCoverExplorer.MSBuildTasks.dll"/>
<Target Name="RunUnitTests" DependsOnTargets="$(RunUnitTestsDependsOn)" />
<!-- RunUnitTests dependencies -->
<PropertyGroup>
<RunUnitTestsDependsOn>
SetUnitTestTargetProperties;
CreateUnitTestsBuildStep;
NCover;
CreateNCoverReport;
MergeNUnitResultsIntoBuild;
SetUnitTestsBuildStepResultSuccess;
</RunUnitTestsDependsOn>
</PropertyGroup>
<!-- Sets property values for unit test target -->
<Target Name="SetUnitTestTargetProperties">
<!-- Create a property representing the type of build (eg release, debug) -->
<CreateProperty Value="%(ConfigurationToBuild.FlavorToBuild)">
<Output PropertyName="FlavorToBuildValue" TaskParameter="Value" />
</CreateProperty>
<ItemGroup>
<TestAssemblies Include="$(OutDir)\**\*Tests.dll" />
</ItemGroup>
<PropertyGroup>
<CoverageResultsOutputFileName>CoverageResults.xml</CoverageResultsOutputFileName>
<NUnitResultsOutputFileName>NUnitResults.xml</NUnitResultsOutputFileName>
<NCoverToolPath>C:\Program Files\TestDriven.NET 2.0\NCover\1.5.8</NCoverToolPath>
<NUnitConsoleExeFileInfo>C:\Program Files\NUnit 2.4.8\bin\nunit-console.exe</NUnitConsoleExeFileInfo>
<NCoverLogFileName>coverage.log</NCoverLogFileName>
<NCoverExplorerToolPath>C:\Program Files\TestDriven.NET 2.0\NCoverExplorer\</NCoverExplorerToolPath>
<NCoverXmlReportFileName>ncoverreport.xml</NCoverXmlReportFileName>
<NCoverHtmlReportFileName>ncoverreport.html</NCoverHtmlReportFileName>
<NXslt2FileInfo>nxslt-2.3-bin\nxslt2.exe</NXslt2FileInfo>
<NUnitTransformTemplateFileInfo>MSBuild\NUnitForTeamBuild\nunit transform.xslt</NUnitTransformTemplateFileInfo>
<NUnitResultsTrxFileName>nunit_results.trx</NUnitResultsTrxFileName>
</PropertyGroup>
</Target>
<!-- Create a Custom Build Step for reporting on Unit Test actions -->
<Target Name="CreateUnitTestsBuildStep">
<BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" Name="NUnitTestStep" Message="Running NUnit Tests">
<Output TaskParameter="Id" PropertyName="NUnitStepId" />
</BuildStep>
</Target>
<!-- Run Unit Tests via NCover to get unit test results and coverage report -->
<Target Name="NCover">
<!-- Use NCover to run unit tests and produce coverage report -->
<NCover ToolPath="$(NCoverToolPath)"
WorkingDirectory="$(BinariesRoot)\$(FlavorToBuildValue)"
CommandLineExe="$(NUnitConsoleExeFileInfo)"
CommandLineArgs="@(TestAssemblies->'%(Filename)%(Extension)',' ') /xml=$(TestResultsRoot)\$(NUnitResultsOutputFileName) /labels /nologo"
CoverageFile="$(TestResultsRoot)\$(CoverageResultsOutputFileName)" LogFile="$(TestResultsRoot)\$(NCoverLogFileName)">
<Output TaskParameter="ExitCode" PropertyName="NUnitResult" />
</NCover>
<Message Text="NCover target executed." />
<OnError ExecuteTargets="NCoverErrorHandler" />
</Target>
<!-- Create unit test coverage report -->
<Target Name="CreateNCoverReport">
<NCoverExplorer ToolPath="$(NCoverExplorerToolPath)"
ProjectName="$(TeamProject)" ReportType="ModuleClassFunctionSummary"
OutputDir="$(TestResultsRoot)"
XmlReportName="$(NCoverXmlReportFileName)" HtmlReportName="$(NCoverHtmlReportFileName)"
ShowExcluded="False" SatisfactoryCoverage="80"
CoverageFiles="$(TestResultsRoot)\$(CoverageResultsOutputFileName)" Exclusions="Assembly=*Tests" />
<Message Text="CreateNCoverReport target executed." />
</Target>
<!-- Update the unit test build step status for success -->
<Target Name="SetUnitTestsBuildStepResultSuccess">
<BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" Id="$(NUnitStepId)" Status="Succeeded" />
<Message Text="SetUnitTestsBuildStepResult target executed." />
</Target>
<!-- Use a command line xslt transformation tool and xsl file to create a trx file that reports unit test results in the same format as if MSTest were used and publish to report server for availability in VS IDE -->
<Target Name="MergeNUnitResultsIntoBuild">
<!-- Regardless of NUnit success/failure merge results into the build -->
<Exec Command=""$(ProgramFiles)\$(NXslt2FileInfo)" "$(TestResultsRoot)\$(NUnitResultsOutputFileName)" "$(ProgramFiles)\$(NUnitTransformTemplateFileInfo)" -o "$(TestResultsRoot)\$(NUnitResultsTrxFileName)"" />
<Exec Command=""$(ProgramFiles)\Microsoft Visual Studio 9.0\Common7\IDE\mstest.exe" /publish:$(TeamFoundationServerUrl) /publishbuild:"$(BuildNumber)" /publishresultsfile:"$(TestResultsRoot)\$(NUnitResultsTrxFileName)" /teamproject:"$(TeamProject)" /platform:"%(ConfigurationToBuild.PlatformToBuild)" /flavor:"%(ConfigurationToBuild.FlavorToBuild)"" IgnoreExitCode="true" />
<Message Text="MergeNUnitResultsIntoBuild target executed." />
</Target>
<!-- Overriding AfterDropBuild target to copy over NUnit test results. This means that we can copy in our test results without fear of overwriting -->
<Target Name="AfterDropBuild" DependsOnTargets="CopyUnitTestResultFiles">
</Target>
<!-- Copy the test result files to the final output folder on the build server (where they are archived along with the binaries that were produced in the build) -->
<Target Name="CopyUnitTestResultFiles">
<!-- Create a list (at runtime of this target) of the files in the TestResults folder -->
<ItemGroup>
<TestResultFiles Include="$(TestResultsRoot)\*.*"/>
</ItemGroup>
<Copy SourceFiles="@(TestResultFiles)" DestinationFolder="$(DropLocation)\$(BuildNumber)\TestResults" />
<Message Text="CopyUnitTestResultFiles target executed." />
</Target>
<Target Name="NCoverErrorHandler" DependsOnTargets="$(NCoverErrorHandlerDependsOn)" />
<!-- NCoverErrorHandler dependencies -->
<PropertyGroup>
<NCoverErrorHandlerDependsOn>
CreateNCoverReport;
MergeNUnitResultsIntoBuild;
SetUnitTestsBuildStepResultFailure;
</NCoverErrorHandlerDependsOn>
</PropertyGroup>
<!-- Update the unit test build step status for failure -->
<Target Name="SetUnitTestsBuildStepResultFailure">
<!-- If deploy failed, error out -->
<Message Text="Error in Unit Tests target." />
<BuildStep TeamFoundationServerUrl="$(TeamFoundationServerUrl)" BuildUri="$(BuildUri)" Id="$(NUnitStepId)" Status="Failed" />
</Target>
</Project>