Saturday, January 26, 2013

Sitecore NUnit Testing Simplified

There's been a lot of interest paid in the last couple of years on how to run NUnit tests against Sitecore.

  • Back in 2010, Alistair Deneys introduced the CodeFlood test runner to run tests inside a browser window, so that an HttpContext was present.  
  • Later that year, Mike Edwards showed that it was possible to run unit tests directly in Sitecoreby copying Web.Config to the test project as App.Config, copying the App_Config folder structure to the project root, and then editing the App_Config references (changing "/App_Config" to ".\App_Config" throughout) and remove the contents of the <log4net> node, you could run NUnit tests directly in the NUnit GUI.
  • In 2011, Deneyes wrote a blog post describing how to automate the copying of configuration files to the test project using the AfterBuild target in the project file, so that the test project would stay in sync with changes to the Website configuration file. He pointed out that this method does not access App_Config\Include patches unless they are copied to the root directory of the operating system (i.e., "C:\App_Config\Include\", etc.)
Last week I posted a solution to the App_Config\Include issue.  By including this line in the SetUp method of the unit test class, Sitecore's patching logic will look in the test project root, instead of using the HttpRuntime. 
public void SetUp()
  Sitecore.Context.IsUnitTesting = true;  
But it was still necessary to manipulate all the App_Config file references, a somewhat cumbersome process, and difficult to automate.  It turns out that this step is unnecessary, as long as you copy the App_Config hierarchy to two locations, the project root and the output directory ("bin\Debug" usually).  The copy in the project root will be used by Sitecore's config patching process, and the one in the bin\Debug directory is used to resolve .Net include files, such as the reference to ConnectionStrings.config.

Building on Alistair Deneys' post, this can be automated by editing the project file. First, add the path to the site root in first <PropertyGroup> section:

Then uncomment the AfterBuild target, and add the following:

<Target Name="AfterBuild">
    <Copy SourceFiles="$(SitecorePath)\Web.config" 
    <CreateItem Include="$(SitecorePath)\App_Config\**\*.*">
      <Output ItemName="configFiles" TaskParameter="Include" />
    <Copy SourceFiles="@(configFiles)" 
        DestinationFolder="$(OutputPath)\App_Config\%(RecursiveDir)" />
    <Copy SourceFiles="@(configFiles)" 
        DestinationFolder="$(MSBuildProjectDirectory)\App_Config\%(RecursiveDir)" />
    <CreateItem Include="$(SitecorePath)\bin\*.dll">
      <Output ItemName="binaryFiles" TaskParameter="Include" />
    <Copy SourceFiles="@(binaryFiles)" DestinationFolder="$(OutputPath)" />  
($OutputPath) resolves to "bin\Debug", and ($MSBuildProjectDirectory) resolves to the project root, so this script copies the App_Config directory to both locations. It also copies out all project DLLs, and writes Web.config to bin\Debug\projectName.dll.config.

And that's all there is to it. No manipulation of paths, no rewriting of the <log4net> node, and no copying of patch files to your system root. If you have your license file location explicitly stated in a patch file (as SitecoreInstaller does, placing it in zLicense.config), you don't even need to copy your license to the test project.

I have tested this approach with Sitecore 6.4.0 (rev 101124) and 6.5.0 (rev. 120706).

Update (1/29/2013): Corrected capitalization in AfterBuild task.
Update (1/30/2013): Removed try/catch code from SetUp method. Further testing showed this is not necessary.
Update (9/27/2015): The IsUnitTesting technique no longer works with Sitecore 8. However, Dan Cruickshank has posted an alternative:

State.HttpRuntime.AppDomainAppPath = Directory.GetCurrentDirectory();


  1. Great post Dan! It was very helpful. I followed these steps and was able to solve the auto include issue. However, I'm running into an unusual issue.

    I created a test project with two unit tests.
    I can successfully run the unit tests when I run them in the debugger (Test->Debug->All Tests option in the Visual Studio). When I run these tests in the Test Explorer, they fail to run and the following exception is thrown:
    Result Message:
    SetUp : System.TypeInitializationException : The type initializer for 'Sitecore.Diagnostics.LoggerFactory' threw an exception.
    ----> System.NullReferenceException : Object reference not set to an instance of an object.
    Result StackTrace:

    This exception is thrown while executing the following in the test method.

    var db = Sitecore.Configuration.Factory.GetDatabase("master");

    Any ideas why would the tests run fine in the debugger but not in the test explorer?


    1. Are you using MSTEST rather than NUnit? MSTEST runs outside of the output directory, either in the TestResults folder or in the %temp% location, so it won't have access to the files copied to the output directory. You can prove this out by including this line in your tests:
      Console.WriteLine("Working directory: " + System.IO.Directory.GetCurrentDirectory());

      This behavior seems to have changed with Visual Studio 2012, which appears to run tests from the output directory.

      You should be able to resolve by replacing [TestClass] with [TestFixture], and [TestCase] with [Test], and adding an assembly reference to NUnit.framework. You would then have to use a different test runner, such as the NUnit Gui.

      If you must stick with MSTest and cannot move to Visual Studio 2012, you may want to look into the [DeploymentItem] attribute to move the resources over that you need. See this on Stack Overflow:

      Good luck, and thanks for the feedback!

    2. I have the same problem. I am running the tests inside Visual Studio 2012 with Resharper. When I use debug the test goes through. If I run it I am getting the above error when setting the IsUnitTesting to true.

    3. @Kai Are you using NUnit or MSTEST?

    4. Sorry, I meant to writethat it is NUnit. Somehow got lost somewhere. ;)

    5. OK, I found the problem. For me it was an empty config transformation file which got somehow copied to the Include folder of the website and therefore also to the test project. After I removed it from the include folder of the Test project, the test went green.
      It is interesting that the files are also in the Debug (/App_Config/Debug) folder but this doesn't appear to have an impact on the debug process.