Customising Your Build Process

TL;DR

This article got much longer than I had originally intended, so here’s a summary for those with short attention spans, wondering whether they should bother reading further:

  • You should automate manual pre and post build tasks by customising your MSBuild script.
  • Customising your builds with MSBuild can actually be quite easy, and changes can be made rapidly using a technique I demonstrate.
  • The best way I’ve discovered to structure build script customisations is to store your custom tasks in a .targets file, and import the .targets file into your project file.
  • This technique means you don’t need to modify your project file each time you want to make a change to your custom tasks.
  • You can also write code directly in your build scripts using inline tasks!

Introduction

I’ve known for years that you could customise how your .NET applications are compiled by MSBuild, but for a number of reasons (which I’ll mention shortly) I’ve avoided doing so up until now, tending to specify commands as pre or post build events instead.  However, I’ve recently discovered what I think is a much better way to customise MSBuild scripts, which I’ll share with you in this article.  Before we do so, I’ll provide a bit of background info for those of you who are new to MSBuild.

What is MSBuild?

For those that don’t know, MSBuild is the short name for the Microsoft Build Engine.  When you compile your .NET applications in Visual Studio, TFS Build, or on the command-line, it’s MSBuild that’s doing all the work.

MSBuild follows a build script, which specifies what it should build and how.  If you’ve ever taken the time to look at the source of your project file (.csproj, .vbproj, etc), you’ll notice that it’s all XML.  You may or may not have realised it, but it’s actually a MSBuild script!  MSBuild parses the XML in your project file (along with any other files that it imports), and compiles your project accordingly.

The great thing is that you can customise this build script and add your own tasks to it to suit your project’s specific needs.  The possibilities this provides are almost limitless!

It’s beyond the scope of this article to detail the structure and features of MSBuild scripts in any sort of depth.  MSDN is the place to go for that sort of information.  In summary though, a build script may consist of:

Properties – key/value pairs used to configure builds.
Items – items (usually file references) that are to be compiled or used as inputs to tasks.
Item Groups – all items must be contained within one or more item group elements
Tasks – actually perform work within the build process
Targets – used to sequence tasks within the build process

More information on the can be found on MSDN here: MSBuild Concepts

Manual Processes are Bad!

So when should you customise your builds, and why?  If you find that you need to manually perform any task before or after your application has built, regardless of whether it needs to be done every time you press F5 in Visual Studio or only when your build server is doing a release build, then you should automate this task.  Being a developer, you should naturally always be on the lookout for repetitive tasks that can be eliminated with automation.  MSBuild is the tool you should be using for this job.  Or Nant, or custom build templates in TFS Build, etc, but you get the drift.  This article is about MSBuild though, so let’s stay focused.

For example, if you’re doing any of these things manually before or after a build (a few examples off the top of my head), then you should be automating the process immediately!

  • Incrementing a version number
  • Zipping up outputs
  • Transforming XML files
  • Installing an assembly into the GAC
  • Generating a NuGet package
  • Deploying the outputs
  • Modifying outputs in any way (such as renaming them, etc)
  • Copying outputs between projects
  • Generating documentation files
  • Obfuscating assemblies
  • Executing SQL statements
  • Excluding files or folders from being published

The vast majority of build script customisations will typically only run when performing a release build (which is hopefully being done by a build server!).  We’ll look at how your custom build script can check for this later in the article.  A nice thing about custom MSBuild scripts though is that they’ll also run when building the project in Visual Studio, which can make developing, testing, and debugging your customisations a lot easier.

Misconceptions

As I mentioned at the start of the article, I’ve known that you can modify project files in order to customise the build process, but I’ve avoided doing so due to a few misconceptions I had:

  • I thought each time I wanted to change something in the build process I’d have to edit the project file.  Modifying a project file in Visual Studio involves unloading the project, opening the project file, finding the right spot for the change, making the change, and then reloading the project it again.  This can be a frustrating and time consuming process, especially when the changes you make don’t work and you need to repeat the process.
  • I thought that if you wanted to write custom code in C# as part of the build process, you’d have to write it as a task, compile it as an assembly, and make it available somehow such that MSBuild could find it.  This seemed a bit too much effort for what was generally a minor task.
  • I was worried that modifications to project files are not easily visible to future maintainers of the application, and can lead to confusion and frustration due to “unexpected behaviour” (good programmers should always consider the impact that their customisations will have on the maintainability of the applications they’re working on).

None of these points are true when you use the technique I’m about to describe for customising the build process.

Why not Simply use the Build Events Tab in the Project Properties?

Previously, my alternative to customising the MSBuild script was to specify pre and post build event commands in the Build Events tab in the Project Properties window.  For cases which involve simply executing a command on the command-line, this does the job fine.  It’s certainly worked for me in the past.  However, a lot more flexibility can be gained by customising the MSBuild script, and it opens many further possibilities.

I also personally hate the Build Events tab with a vengeance.  I’m pretty sure that no-one has touched it since VS2002.  Commands are usually quite long, yet it insists on having a fixed width of about 390px for the text boxes used to configure the commands, regardless of the width of the document pane.

CustomBuild1

A Better Way

I was using the AjaxMin to bundle and minimise JavaScript files recently (more on that in a future blog post), and noticed how the documentation for its build task demonstrated using a separate targets file to define its custom build actions, which could then be imported into the main build script (i.e. the project file).  It suddenly struck me that this was the perfect mechanism to use when customising the build process of my projects.  Rather than having to add custom build tasks directly to a project file, I could simply maintain them in a separate targets file, which the project file could then import.  The benefits of this would be:

  • Modifying my project’s build configuration becomes very easy and rapid, and
  • The presence of the .targets makes it much more obvious to future maintainers that the build process has been customised, and it’s easy to see exactly what those customisations are.

Sure, I still needed to modify the project file to get it to import the targets file, but then no further changes would be required.  As you’ll see shortly, when I discuss inline tasks, it can be a huge benefit not having to modify your project file each time you want to make a change to your build script.

Configuring a Targets File

So let’s look at how you go about customising your build script using this technique.  The first step in the process is to create a targets file, and import it into your project file:

  1. Add a new XML file to the root of your project named CustomBuildProcess.targets.
  2. Add the following XML to it:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

</Project>

  1. Right-click on your project file in the Solution Explorer window, and select Unload Project from the context menu.
  2. Right-click on your project file again in the Solution Explorer, and select Edit [Project File Name] from the context menu.
  3. Find a line that imports a targets file.  For example:

<Import Project="$(MSBuildToolsPath)\Microsoft.CSharp.targets" />

  1. Add a new line after this, and add the following XML element:

<Import Project="$(MSBuildProjectDirectory)\CustomBuildProcess.targets" />

  1. Save the changes you’ve made to the project file, then reload the project by right-clicking on the project in the Solution Explorer window, and selecting Reload Project from the context menu.
  2. Try to build your project – it should build successfully.

Customising Your Build Process

You’ve now got a targets file, in which you can specify the customisations to your project’s build process.  How you write MSBuild scripts is really beyond the scope of this article, as that’s a whole topic in itself.  There’s plenty of information available on doing so on MSDN and the wider web though – I’d probably recommend this MSDN article as a good first port of call: Walkthrough: Using MSBuild

That said, I don’t think this article would be complete without a simple example of customising the build script.  So for this example, we’re just going to log a message to the output console.  The key concern here is when will your custom tasks execute?  There’s actually a number of ways that you could use to specify when your custom tasks should run, such as overriding the default targets defined in Microsoft.Common.CurrentVersion.targets (not recommended – see the comment from Chris Walters at the end of the post), or adding your custom targets to the existing default target definitions (which is not ideal, as you have to modify the .csproj file). For this example we’ll actually use the simplest way, which is by specifying which target our target should run before or after.  There’s a number of targets you can “target”, including Compile, Build, Rebuild, Clean, Publish, ResolveReferences, and ResGen, amongst the many others defined in the Microsoft.Common.CurrentVersion.targets file.  As you can see, there’s no shortage of extensibility points.  In this example, we’ll create a target that runs after the Compile target, and get MSBuild to write Hello World! to the Output window.  All you need to do is add the Target element below to your CustomBuildProcess.targets file:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Target Name="SayHello" AfterTargets="Compile">
    <Message Text="Hello World!" Importance="High" />
  </Target>
</Project>

Note that it’s important that you set the Importance attribute to High.  Otherwise the message won’t display in the Output window unless you increase the output verbosity of MSBuild, a topic we’ll discuss further when it comes to debugging your customisations.

Now when you compile your project, the message is displayed in the Output window after the project has successfully built.

CustomBuild2

This was an extremely simple example, but it should help you at least get started customising your build.  There’s numerous tasks that are built into MSBuild which you can use, such as Copy, Delete, MakeDir, Exec, to name a few.  A full list of these with descriptions and examples can be found here: MSBuild Task Reference.  However, they are all very basic tasks, which is where the MSBuild Extension Pack and MSBuild Community Tasks projects come in.  These provide much more sophisticated build tasks that you can use, enabling you to work with FTP, the GAC, the registry, Active Directory, IIS, source control, XML, and much more.

Debugging Your Customisations

I thought I’d just touch on two tips to debug your build script customisations, and help you solve any issues you might have.

The first tip is to write messages to the Output Window, as demonstrated in the previous section.  The second is to increase the output verbosity of MSBuild in order to give you more of an insight into what’s going on.  You can change this in Visual Studio’s option dialog, under Projects and Solutions > Build and Run.

CustomBuild3

Using Inline Tasks

The build tasks provided out of the box with MSBuild, along with the community build tasks, will usually cover most of your requirements when customising your build process.  However, sometimes what you need done warrants writing your own custom build task, and even if it doesn’t it can often be easier to write just a bit of code to do what you need rather than mucking around with modifying the MSBuild script itself.

Traditionally, you had to create an assembly, write your custom build tasks in that, and then ensure it’s available to MSBuild when it runs.  But this incurred quite an overhead in configuration and management, so to solve this issue MSBuild 4.0 introduced inline tasks.  Essentially, inline tasks enable you to write code directly in the build script using your favourite .NET language (or even other languages if someone has developed a custom task factory, such as those for Powershell and the DLR provided by the MSBuild Extension Pack), and have it execute as part of your build process.  This is an awesome feature, and makes it so much easier to develop and maintain build customisations.  You can easily write code in a language that’s familiar to you, you’ve got the full .NET Framework at your disposal, and you’ve basically got almost limitless possibilities.

It’s when developing inline tasks that you’ll find locating your custom build tasks separately from your project file makes a huge difference.  You can iterate rapidly during the development of these inline tasks, and it’s very easy for future maintainers of the project to see exactly what’s going on.

I don’t want to get too deeply into how to write inline tasks as there’s pretty good documentation on MSDN already on this topic, however given that the targets file structure I’ve demonstrated is perfect for use when writing inline tasks, I think a simple example is in order.  Again, we’re just going to keep things simple and write a message to the Output window.

  1. The first step is to create a UsingTask element in your targets file.  You need to give the task a name, and specify the task factory and assembly file.  Since we’re writing .NET code for our inline task, the task factory will be CodeTaskFactory, and the assembly will be the MSBuild tasks assembly:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <UsingTask TaskName="MyCustomInlineTask" TaskFactory="CodeTaskFactory"
             AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
  </UsingTask>
</Project>

Note that these values will always be the same, assuming you’re just writing .NET code in your inline task.  Task factories essentially interpret and execute the code you write in your build script, so we’re pointing to the task factory class that manages this, and the assembly that it can be found in.  If you’d prefer to execute PowerShell script in your inline task, then you can use the PowerShellTaskFactory available in the MSBuild Extension Pack instead.

  1. The next step is to define your task (within a Task element), and write the code for it within a Code element, remembering to specify the language you’re using with the Language element:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <UsingTask TaskName="MyCustomInlineTask" TaskFactory="CodeTaskFactory"
             AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll
    <Task>
      <Code Type="Fragment" Language="cs">
        <![CDATA[
	  Log.LogMessage("Hello from an inline build task!", MessageImportance.High);
	]]>
      </Code>
    </Task>
  </UsingTask>
</Project>

Note that the CData elements are not required if your code doesn’t use XML characters such as (i.e. XML reserved characters), but it’s strongly recommended.

You can specify assembly references, using statements, and input parameters as required, which is documented on MSDN.

  1. The final step is to reference your task (using the name you gave it on the UsingTask element) in a target such that it will be executed:
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <UsingTask TaskName="MyCustomInlineTask" TaskFactory="CodeTaskFactory"
             AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll>
    <Task>
      <Code Type="Fragment" Language="cs">
        <![CDATA[
	  Log.LogMessage("Hello from an inline build task!", MessageImportance.High);
	]]>
      </Code>
    </Task>
  </UsingTask>

  <Target Name="RunMyInlineTask" AfterTargets="Compile">
    <MyCustomInlineTask />
  </Target>
</Project>

Now when you build your project you should see the message appear in the Output window.

CustomBuild4

If the message doesn’t appear, try changing your MSBuild output verbosity option to Normal in Visual Studio’s options window, as described in the Debugging Your Customisations section of this article.  For some reason the Log.LogMessage method doesn’t seem to work exactly like the Message build task used earlier.  Even though the importance is set to High when using them both, the Message build task shows the message when the build output verbosity is set to Minimal, but the Log.Message method only shows the message if the build output verbosity is ramped up to Normal or higher.

This was a very simple example, but I have an upcoming blog post in which I’ll demonstrate implementing an inline task which is much more sophisticated.

Running Build Customisations on a Build Server

To be honest, I think most customisations to your build script will only need to be run when it’s being built on a build server.  From my experience, it’s rare to need to perform custom actions when doing a “desktop” build.  Many of the possible customisations I listed earlier in the article are things you’d only want done when creating a release build, and it’s not out of place to say that most teams should have a build server that does this for them.

Common build servers include TeamCity, CruiseControl.NET, and TFS Build. The technique outlined in this blog post works with all of these.

If this is the case and you’re using TFS Build, it may be worth investigating whether it would be better to customise your build definition’s template instead (build templates use Windows Workflow to define a sequence of activities to execute during the build, and can easily be reused across build definitions).  That said, I’ve done a lot of build template customisation work, and it’s a right royal PITA.  It’s not fun at all.  Testing and debugging customisations is a seriously slow and painful process, requiring checking in of the template to source control each time you make a change that you want to test.  It’s also hard to see what customisations have been made, making build templates a nightmare to maintain.  Anyway, that’s just my experience.

I did lots of work around customising TFS Build templates last year, and I might try and share some tips and recipes in future blog posts. Custom TFS Build templates very much have their place, particularly when implementing customisations that apply to multiple projects being built by your build server.  It may be a painful process, but is often the most appropriate choice over developing custom MSBuild scripts when implementing custom processes that only ever run on the build server.

If you do go down the path of developing custom MSBuild scripts but your custom build processes are to run only on the server, you can create a condition on your targets that check the value of the $(BuildingInsideVisualStudio) property.  It’s true when the project is being built from Visual Studio , and empty when hosted elsewhere (i.e. from the command line or a build server).  For example, the following target will only run when the project is built outside of Visual Studio:

  <Target Name="DoStuff" AfterTargets="Compile" Condition="'$(BuildingInsideVisualStudio)' == ''">
    <!-- My tasks -->
  </Target>

Conclusion

So we’re done!  In this article you’ve seen how you can implement MSBuild script customisations with reasonable ease.  I use this technique in many of my projects – hopefully you’ll agree that this is a pretty nice technique for customising your build scripts, and you’ll use it in your own projects too!

The best place to get more information on customising your build process is MSDN.  The section on MSBuild can be found here: MSBuild.

Advertisements