Category: TFS

The build process template and custom activity described in this post is available here:
http://cid-ee034c9f620cd58d.office.live.com/self.aspx/BlogSamples/Inmeta%20TFS%20Build%20Sample.zip

Running code metrics has been available since VS 2008, but only from inside the IDE. Yesterday Microsoft finally released Visual Studio Code Metrics Power Tool 10.0, a command line tool that lets you run code metrics on your applications.  This means that it is now possible to perform code metrics analysis on the build server as part of your nightly/QA builds. In this post I will show how you can run the metrics command line tool from a build, and also a custom activity that reads the output and appends the results to the build log, and fails the build if the metric values exceeds certain (configurable) treshold values.

The code metrics tool analyzes all the methods in the assemblies, measuring cyclomatic complexity, class coupling, depth of inheritance and lines of code. Then it calculates a Maintainability Index from these values that is a measure of how maintanable this method is, between 0 (worst) and 100 (best). For information on how this value is calculated, see http://blogs.msdn.com/b/codeanalysis/archive/2007/11/20/maintainability-index-range-and-meaning.aspx. After this it aggregates the information and present it at the class, namespace and module level as well.

Running Metrics.exe in a build definition
Running the actual tool is easy, just use a InvokeProcess activity last in the Compile the Project sequence, reference the metrics.exe file and pass the correct arguments and you will end up with a result XML file in the drop directory. Here is how it is done in the attached build process template:

In the above sequence I first assign the path to the code metrics result file ([BinariesDirectory]result.xml) to a variable called MetricsResultFile, which is then sent to the InvokeProcess activity in the Arguments property.
Here are the arguments for the InvokeProcess activity:

Note that we tell metrics.exe to analyze all assemblies located in the Binaries folder. You might want to do some more intelligent filtering here, you probably don’t want to analyze all 3rd party assemblies for example.
Note also the path to the metrics.exe, this is the default location when you install the Code Metrics power tool. You must of course install the power tool on all build servers.

Using the standard output logging (in the Handle Standard Output/Handle Error Output sections), we get the following output when running the build:

Integrating Code Metrics into the build
Having the results available next to the build result is nice, but we want to have results integrated in the build result itself, and also to affect the outcome of the build. The point of having QA builds that measure, for example, code metrics is to make it very clear how the code being built measures up to the standards of the project/company. Just having a XML file available in the drop location will not cause the developers to improve their code, but a (partially) failing build will!

To do this, we need to write a custom activity that parses the metrics result file, logs it to the build log and fails the build if the values frfom the metrics is below/above some predefined treshold values.

The custom activity performs the following steps

1. Parses the XML. I’m using Linq 2 XSD for this. Since the XML schema for the result file is available with the power tool, it is vey easy to generate code that lets you query the structure using standard Linq operators.
2. Runs through the metric result hierarchy and logs the metrics for each level and also verifies maintainability index and the cyclomatic complexity against the treshold values. The treshold values are defined in the build process template and are sent in as arguments to the custom activity
   
3. If the treshold limits are exceeded, the activity either fails or partially fails the current build.

For more information about the structure of the code metrics result file, read Cameron Skinner’s post about it. It is very simple and easy to understand. I won’t go through the code of the custom activity here, since there is nothing special about it and it is available for download so you can look at it and play with it yourself.

The treshold values for Maintainability Index and Cyclomatic Complexity is defined in the build process template, and can be modified per build definition:

I have chosen the default values for these settings based on a post from my colleague Terje Sandström – Code Metrics – suggestions for approriate limits. When you think about it, this is quite an improvement compared to using code metrics inside the IDE, where the Red/Yellow/Green limits are fixed (and the default values are somewhat strange, see Terjes post for a discussion on this)

This is the first version of the code metrics integration with TFS 2010 Build, I will probably enhance the functionality and the logging (the “tree view” structure in the log becomes quite hard to read) soon.
I will also consider adding it to the Community TFS Build Extensions site when it becomes a bit more mature.

Another obvious improvement is to extend the data warehouse of TFS and push the metric results back to the warehouse and make it visible in the reports.

*** The sample build process template discussed in this post is available for download from here: http://cid-ee034c9f620cd58d.office.live.com/self.aspx/BlogSamples/ILMerge.xaml ***

In my previous post I talked about library builds that we use to build and replicate dependencies between applications in TFS. This is typically used for common libraries and tools that several other application need to reference.

When the libraries grow in size over time, so does the number of assemblies. So all solutions that uses the common library must reference all the necessary assemblies that they need, and if we for example do a refactoring and extract some
code into a new assembly, all the clients must update their references to reflect these changes, otherwise it won’t compile.

To improve on this, we use a tool from Microsoft Research called ILMerge (Download from here). It can be used to merge several assemblies into one assembly that contains all types. If you haven’t used this tool before, you should check it out.

Previously I have implemented this in builds using a simple batch file that contains the full command, something like this:

“%ProgramFiles(x86)%microsoftilmergeilmerge.exe” /target:library /attr:ClassLibrary1.bl.dll /out:MyNewLibrary.dll ClassLibrary1.dll ClassLibrar2.dll ClassLibrary3.dll

This merges 3 assemblies (ClassLibrary1, 2 and 3) into a new assembly called MyNewLibrary.dll. It will copy the attributes (file version, product version etc..) from ClassLibrary1.dll, using the /attr switch. For more info on ILMerge command line tool, see the above link.

This approach works, but requires a little bit too much knowledge for the developers creating builds, therefor I have implemented a custom activity that wraps the use of ILMerge. This makes it much simpler to setup a new build definition and have the build automatically do the merging.
The usage of the activity is then implemented as part of the Library Build process template mentioned in the previous post. For this article I have just created a simple build process template that only performs the ILMerge operation.

Below is the code for the custom activity. To make it compile, you need to reference the ILMerge.exe assembly.

    /// <summary>
    /// Activity for merging a list of assembies into one, using ILMerge
    /// </summary>
    public sealed class ILMergeActivity : BaseCodeActivity
    {
        /// <summary>
        /// A list of file paths to the assemblies that should be merged
        /// </summary>
        [RequiredArgument]
        public InArgument<IEnumerable<string>> InputAssemblies { get; set; }
        /// <summary>
        /// Full path to the generated assembly
        /// </summary>
        [RequiredArgument]
        public InArgument<string> OutputFile { get; set; }
        /// <summary>
        /// Which input assembly that the attibutes for the generated assembly should be copied from.
        /// Optional. If not specified, the first input assembly will be used
        /// </summary>
        public InArgument<string> AttributeFile { get; set; }
        /// <summary>
        /// Kind of assembly to generate, dll or exe
        /// </summary>
        public InArgument<TargetKindEnum> TargetKind { get; set; }

        // If your activity returns a value, derive from CodeActivity<TResult>
        // and return the value from the Execute method.
        protected override void Execute(CodeActivityContext context)
        {
            string message = InputAssemblies.Get(context).Aggregate("", (current, assembly) => current + (assembly + " "));
            TrackMessage(context, "Merging " + message + " into " + OutputFile.Get(context));
            ILMerge m = new ILMerge();
            m.SetInputAssemblies(InputAssemblies.Get(context).ToArray());
            m.TargetKind = TargetKind.Get(context) == TargetKindEnum.Dll ? ILMerge.Kind.Dll : ILMerge.Kind.Exe;
            m.OutputFile = OutputFile.Get(context);
            m.AttributeFile = !String.IsNullOrEmpty(AttributeFile.Get(context)) ? AttributeFile.Get(context) : InputAssemblies.Get(context).First();
            m.SetTargetPlatform(RuntimeEnvironment.GetSystemVersion().Substring(0,2), RuntimeEnvironment.GetRuntimeDirectory());
            m.Merge();

            TrackMessage(context, "Generated " + m.OutputFile);
        }
    }
    [Browsable(true)]
    public enum TargetKindEnum
    {
        Dll,
        Exe
    }

NB: The activity inherits from a BaseCodeActivity class which is an internal helper class which contains some methods and properties useful for moste custom activities. In this case, it uses the TrackeMessage method for writing

to the build log. You either need to remove the TrackMessage method calls, or implement this yourself (which is not very hard… )

The custom activity has the following input arguments:

Of course, there are more switches to the ILMerge.exe, and these can be exposed as input arguments as well if you need it.

To show how the custom activity can be used, I have attached a build process template (see link at the top of this post) that merges the output of the projects being built (CommonLibrary.dll and CommonLibrary2.dll) into a merged assembly (NewLibrary.dll).

The build process template has the following custom process parameters:

The Assemblies To Merge argument is passed into a FindMatchingFiles activity to located all assemblies that are located in the BinariesDirectory folder after the compilation has been performed by Team Build.

Here is the complete sequence of activities that performs the merge operation. It is located at the end of the Try, Compile, Test and Associate… sequence:

It splits the AssembliesToMerge parameter and appends the full path (using the BinariesDirectory variable) and then enumerates the matching files using the FindMatchingFiles activity.

When running the build, you can see that it merges two assemblies into a new one:

And the merged assembly (and associated pdb file) is copied to the drop location together with the rest of the assemblies:

Update 15.03.2014 – Fixed broken link to download

*** The custom activity is availabe for download here: https://onedrive.live.com/redir?resid=EE034C9F620CD58D%21168 ***

Often when creating different types of release builds (e.g. where you build something that should be installed or consumed by other applications) there is a need to check in the results of the build back
to source control. A common build type for us at Inmeta is Library Builds, which we use for common libraries that are shared among several applications. We have created a special build process template
for this scenario that handles versioning, copying of the resulting binaries to a pre-defined folder, optionally merges the assemblies using ILMerge, and finally checking that binaries back to TFS as part of the build.

When it comes to communicating with TFS source control during a build, I find that the most flexible appraoch is to wrap the command line tool tf.exe. This tool has all the functionality you need, and if you choose
to implement custom activities for the same functionality you would need to expose a lot of functionality that is very simple to call using tf.exe. There are two major drawbacks of this approach however:

• You need tf.exe on the build server, e.g. you must install Team Explorer. Generally it is a very good principle to keep the build server as minimal as possible.
• It is not that easy to invoke command line tools in TFS 2010 Build, it is hard to get the syntax and the paths correct.

If you can live with the first drawback, then we can at least reduce the problems of the other drawback by creating a simple custom activity that wraps some of the nitty gritty details of calling tf.exe.
This is a simple activity, but I have found it useful and hopefully some of you will too.

Here is an example of how it looks when it is used in a build process template:

Note:

• The property Command is an enum type which just enumerates all possible commands to tf.exe. This makes it easy to select the correct command and stop you from getting syntax errors.
  The enum is available as well in the download packages
• You don’t need to know about how tf.exe is called
• There is an IgnoreError boolean property that will ignore any errors if true (default is false)
• The Arguments property is the argument to the tf.exe command. In this case, the resulting command will be tf.exe checkin *.dll /recursive
  
• WorkingDirectory is mapped to the InvokeProcess.WorkingDirectory. Generally when performing source control operations using tf.exe, you should point the working directory somewhere inside the workspace.
  By doing so, you don’t need to bother about spefcifying workspace or collection URL:s.

The custom activity itself is pretty simple, it just wraps the call to InvokeProcess, with the standard output/error logging and error checking:

This activity is very simple, and can definitely be exended. Let me know if you find it useful and if you have any suggestions for improvements.

In TFS 2010, branching and merging have been greatly improved with support for branch visualization and tracking of changesets and work items across branches. A simple example of this looks like this:

Here we track Work Item nr 3 which was originally resovled in the Test branch (with changeset 35). We can also see that the work item has been merged into the Production branch (as changeset 37), back to Main (62) and finally to FeatureC (140).
If we switch to the Timeline view, we get a nice view of the order of these merges, together with the dates.

Unfortunately, this does not solve one of the bigger problems when it comes to branches and work items. When you merge your changes into another branch, you must remember to asociated the corresponding work item again, otherwise that information is lost and the work item will not show up in the build report from the builds running off the target branches. A typical example is that we have 3 bugs in the Test branch, and we perform (at least) 3 changesets and each changeset is associated to the corresponding work item. When it is time to merge the bug fixes to the Production branch, we must manually associated the merge with the work item again. If we peform one merge operation that brings all changes from Test to Main, we must associate all 3 work items. There is really no support for “merging” work items across branches in TFS, only changesets can be merged.

What you really want is to have the tool automatically assign the work items that were associated with the changesets that you are merging. One way to implement this is with a checkin policy, which we have done at our company. The reason for choosing a checkin policy as the tool of choice is because it is executed on the client at the time of the checkin, and we can display the work items to the developers before they check in.

So, how does it work? Lets look at an example:

In my development branch, I have 3 bugs (Bug1, Bug2 and Bug3) that I need to fix. I fix each one in a changeset that gets checked in and associated with the corresponding work item. Then it is time to merge the fixes to the Main branch (trunk).
I perform a merge by just using the normal Merge operation from source control. This leaves me with the following pending merge:

Now, I would normally go to the Work Items tab and link to the work items that I know were resolved by the changes that I am currently merging. But now I don’t need to do this but instead I just click on the Check In button. This will evaluate all checkin policies, and one of them is the Merge Work Items policy that pops up the following dialog:

This dialog shows me a list of the work items that were associated with the changesets that I am currently merging in. The checkin policy uses the TFS Version Control API to locate the merge sources of each pending merge item and basically shows the union of these work items (several changesets can be associated with the same work item). If I check in now, the changeset will automatically be associated with these 3 work items! The beauty of this comes when running the builds off the Main branch, the build summary show me that these 3 work items have been resolved in this build:

In another post I will show the interesteing parts of the implementation

By default, TFS Team Build creates a new folder in the drop location for every build. I have seen request from people that wonder how to always have team build put the output in the same folder every time, effectively overwriting the results from the last build. This is easy to accomplish by adding an activity that copies the drop folder to a fixed location.

To copy the result of the build to a fixed location, you need to modify the build process template:

1. Open the build process template XAML file in the workflow designer.
2. Click on the Collapse All link in the upper right corner so that only the top level activities are shown
3. Open the Toolbox window and locate the CopyDirectory activity (it is located in the Team Foundation Build Activities tab)
4. Drag the CopyDirectory activity onto the design surface and drop it between the Run On Agent and the Check In Gated Changes for CheckInShelveset Builds activity:

     

5. Right click on the CopyDirectory activity and select Properties. Fill out the properties, you will of course need to modify the path for the destination accordingly:

   

6. Save the build definition and check it in. NB: Remember to check in the build process template file after modifying it, a lot of times people forget this step!
7. Queue a new build and, after the build has succeeded, verify that the build output has been copied to the corresponding output path

To make this build process template more generic, you proabably want to create a process parameter that lets you define the path either when you create a new build definition, or when you queue the build (or both). This will let you you reuse the build process template for builds with different output paths

Recently I’ve come across this error a couple of times when running builds that exeucte unit tests using Test containers:

API restriction: The assembly ‘file:///C:Builds<path>myassembly.dll’ has already loaded from a different location. It cannot be loaded from a new location within the same appdomain.

Every time I’ve got this error, the project has been a web application, and the path to the assembly points down to the _PublishedWebsites directory that is created beneath the Binaries folder during a team build.

The error description really says it all (although slightly cryptic), when using test containers, MSTest needs to load all assemblies and see if they contain any unit tests. During this serach, it finds the ‘myassembly.dll’ in two different locations. First it is found directly beneth the Binaries folder, and then it is alos found beneath the _PublishedWebsitesProjectbin folder. The reason is that the default setting for test containers in a TFS 2010 build definition is ***test*.dll:

This pattern means that MSTest will search recursively for all assemblies beneath the Binaries folder, and during the search it will find the MyAssembly.dll twice.
The solution is simple, set the Test assembly file specification property to *test*.dll instead, this will disable the recursive search: