Category: TFS Build

Disclaimer: This blog post discusses features in the TFS 2010 Beta 1 release. Some of these  features might be changed in the RTM release.

In TFS 2010, Microsoft has changed the build orchestration language in Team Build from MSBuild to Windows Workflow 4.0. Aaron Hallberg has written a post on how to implement a custom workflow activity using either the workflow designer or using a code activity that composes an activity. In this post, I will show how to implement a “pure” code activity, i.e. no workflow elements,and how to add this activity to a build process definition. Note, this is still Beta 1, and some things will definitely change when Beta2 and RTM arrives but this will get you started with customizing your builds in TFS 2010.

Here is a very simple custom activity that has one input variable, CurrentBuild of type IBuildDetail, and has a string result value. All it does is return the build number of the IBuildDetail object as a string result. This is of course quite useless, but never the less it shows you how to send in variables from your build process workflow and return result back:

public class WriteBuildNumberActivity : CodeActivity<String>

{

    [Browsable(true)]

    [DefaultValue(null)]

    public InArgument<IBuildDetail> CurrentBuild { get; set; }

    protected override void Execute(CodeActivityContext context)

    {

        string buildNumber = "BuildNumber: " + this.CurrentBuild.Get(context).BuildNumber;

        context.SetValue(Result, buildNumber);

    }

}

Note that the class inherits from CodeActivity<String>, which basically gives it a string return value (OutArgument) called Result.

Now, to add this activity to a build process, you need to open the build process designer and drag your activity and configure it. This might be a Beta 1 issue, but the only way I got this to work is to include the build process XAML template in the same project that contains the custom activities. This is of course far from ideal, but I am sure that this will resolved in Beta 2.

So, first of all create a new build process. Check out my previous blog post on how to do this. Note that since you will customize your build process, you’ll want to create a copy of the standard DefaultTemplate XAML process file. You should never modify the DefaultTemplate.xaml project file. Too make it easy for the sample, you can just place the xaml file in the same folder as your custom activity project.

Next, include the build process XAML in your library project  and then double click the XAML file. This opens up the designer, and of you open the toolbox you should see your custom activity in a separate toolbox tab:

Next step is to add the activity to your build process. Drag the activity from the toolbox and place it after the UpdateBuildNumber actity:

Since the activity has an input variable of type IBuildDetail, we need to pass the current BuildDetail object into this variable. To do this, select the activity and edit the CurrentBuildproperty in the properties window to contain the value BuildDetail:

The BuildDetail is a variable that is initialized previously in the Get the Build activity at the start of the build process.

Now save the build process and check it into source control. To be able to run this build on your build agent, the library must of course be available to the build agent. A new approach in TFS 2010 is that you can specify a version control path for custom assemblies on each build controller. This path is where you would store the assemblies that contain custom activities that you want to use in your builds.

Unfortunately, there seem to be a caching issue in Beta 1 which complicates development. If you modify your activity library and check it in, the old version seem to be cached on the build controller. The only work around that I have found is to clear the version control path field, close the dialog and the reopen it and put the old value back. This seem to cause the build controller to reload the custom assemblies.

Ok, if you have checked in everything, including the custom activity library and configured the build controller, you can now run the build and verify that your activity is being called. In the default log view, you will only see the name of the activity in the list. Too view more info, click on the Show Property Values. This will show all input and output variables for each activity. Note though that it only shows variables that is of standard value types, so it won’t show the CurrentBuild variable:

As you can see, the result property contains the string”BuildNumber: “ plus the generated build number that was created previously in the UpdateBuildNumber activity.

Ok, this was a rather contrived and crude example, but it shows how you can create custom code activities and incorporate them into your builds in TFS 2010 Team Build Beta 1. I know many people are interested in how to write “pure” code activites in TFS 2010 for different scenarios, hopefully this post is helpful to get you started!

For more on working with custom code activities in WF4.0, read this walkthrough from Guy Burstein: http://blogs.msdn.com/bursteg/archive/2009/05/19/wf-4-0-code-only-custom-activities-for-atomic-actions-codeactivity-codeactivity-t.aspx

A really cool new feature in VSTS 2010 is Test Impact Analysis which let developers view what tests that are affected by the current code changes. Pieter Gheysens wrote a blog post on how to set this up in the CTP, but things have changed a bit in Beta 1 so I thought that I would show how it is done. Since it still is a bit unintuitive to enable it, it might change once again in the RTM. The reason that it is a bit unintuitive to set it up, is because you need to have the following things:

• You must use Test Metadata files when running your tests. You can’t use Test Assemblies
• You must have code coverage enabled in your test settings. VSTS use the code coverage information from a test run to determine which tests that are impacted by a code change.
• You must setup a team build in TFS with test impact analysis enabled. The build will publish the test results including the code coverage information and VSTS will read information from this build.

So, lets set it up:

1. First you will (obviously) need a solution containing some tests. Note that I don’t explicitly write unit tests here, because it might as well for example web tests. Check in your solution.
2. Enable Code Coverage for you current test settings. See my previous post on how to do this
3. Create a new team build and select your solution. Then set the Analyze Test Impacts parameter to true

   

4. Select the Test Container TestSettings File and make sure that it is the one in which you have enabled code coverage. In the figure above, I have selected $/Demo/LibraryWithTests/TraceAndTestImpact.testsettings

5. Save the build definition and queue a new build. Make sure it finishes successfully and that the tests were executed
6. Now, change some code that you know is called by one or more tests.
7. Switch to the Test Impact View, that is located in the Test –> Windows submenu. First off you can select the button on the top left, called  Show Impacted Tests, that will show all tests that are impacted by all your current code changes.In this case, one test was impacted (ImportedMethodTest). When you select the test, you can see in the bottom part of view what code changes that caused the impact

   

   The next button  is called Show Code Changes and shows the opposite information, e.g. what code changes that has been done, and for each code change it lists the affected test.
   Note that there is Run Tests link in the view. This is also available as a button. This lets you run all the tests that are affected by your code change. This is a very nice feature, that will speed up your development considerably (at least if you have many tests….)

8. Check in your code change and queue a new build. When the build finishes, open the build summary. This will show you, in addition to the test and code coverage information, what tests that were impacted.

   

   If you click the 1 code change(s) link next to the test, a dialog that lists all the methods that had impact on that test is shown

   

There are some changes and improvements in the area of executing unit tests in Team Build 2010. Mostly the changes make it easier to define which unit tests you want to execute as part of the build. In this post I will go through the different options that you have when it comes to running unit tests and enabling code coverage.

To configure test settings for a Team Build, you select Edit Build Definition in Team Explorer, and then go to the Build Process tab. In the Build process parameters box there is a section for the testing parameters. To change the parameters, just edit them and hit the save button. There is no need to check anything in or out to change the parameters. Only changes to the build process require you to check in the build process file (.xaml)

Running Unit Tests from Test List(s)
If you like to manage your tests using test lists (I don’t!), this option lets you run all the tests from one or more test lists. Here I have created two test lists, ImportantTests and LessImportantTests. Each test list contains two unit tests from two different assemblies.

To run all the tests from these test lists in a team build, locate the Test Metadata Files parameter and press the browse button to the right. This brings up a dialog that lets you choose which test metadata files (.vsmdi files) that you want to execute tests from. By default, all the tests will be executed. To filter this, click the Specify Lists button and you can select one or more test lists instead:

Running Unit Tests from Test Assemblies (a.k.a. Test Containers)
Since managing test lists in VSTS doesn’t scale very well, a common approach is to use Test Assemblies instead. In previous versions this was called Test Containers. So instead of creating different test lists to group your tests, you create several unit test assemblies and group your unit tests by adding them to the corresponding unit test assembly. For example you can have one (or more) assembly that contain pure unit tests, another set of assemblies that contain integration tests that you might only want to run in your nightly builds. To specify which test assemblies you want to execute, you use the Test Assembly Filespec parameter. This parameters contains a search pattern that should match the names of your test assemblies.

To use this approach in your company, you need to define a naming scheme for your unit test assemblies, such as Project.UnitTests.dll, Project.IntegrationTests.dll and so on.

Running Unit Tests by Priority
In addition to selecting which unit tests to execute, you can now further filter the tests by using the Priority attribute on your test methods. This is an attribute that has been around since .NET 2.0, but strangely enough there was no support of using it when running tests, neither in Visual Studio or in Team Build. In TFS 2010 however you can define a minimum and maximum test priority for your team build, meaning that all tests with a priority within that range will be executed as part of the build.

Here is how you decorate your test method with a priority.

/// <summary>

 ///A test for ImportantMethod

 ///</summary>

 [TestMethod()]

 [DeploymentItem("LibraryWithTests.dll")]

 [Priority(1)]

 public void ImportantMethodTest()

 {

     Class1 target = new Class1(); // TODO: Initialize to an appropriate value

     string arg = "42"; // TODO: Initialize to an appropriate value

     int expected = 42; // TODO: Initialize to an appropriate value

     int actual;

     actual = target.ImportantMethod(arg);

     Assert.AreEqual(expected, actual);

 }

Then, set the priority range in your build definition

Note that this is an additional filter that is applied to, in this case, all unit tests in all test assemblies that end with UnitTests.dll.

Running Unit Tests by Category
In addition to Priority, you can also filter your tests by using categories. This is very similar to priorities, but instead of working with numeric ranges, you define categories with meaningful names and apply them to your test methods.

To use this approach, decorate your test methods with the TestCategory attribute and then specify one or more test categories in your build definition.

According to the tooltip you should be able to construct the filter by using logical operators such as & and | , but this doesn’t seem to work in Beta 1.

Enabling Code Coverage in a Team Build

When running unit tests you normally want to know how much of your code is actually tested, a.k.a. code coverage. The way you enable this for your tests and in your team build has changed a bit. First, the previous *.testrunconfig files has been renamed into *.testsettings. To enable code coverage, double click on your .TestSettings file and select the Execution Criteria tab. Here you will see a totally new Collectors section that contains information about what data you want to collect when executing test. One of them is code coverage. The GUI is a bit weird in Beta 1, since you are supposed to first select the Code Coverage checkbox, and then click the Advanced button to specify which assemblies that should be instrumented for code coverage.

Next, in your team build definition you must specify the name of the test settings file you want to use. This is done using the Test Container TestSettings File parameter. Save the build definition, check in the test settings file and queue a new build. When it has finished, open the build summary that will show you the numbers on number of executed/passed/failed tests, and also the overall code coverage.

Very nice summary view indeed! If you want to look at the test results in detail, you click the “View Test Results” link which will download the test run to your local machine and then open it in the Test Results window.

Disclaimer: This blog post discusses features in the TFS 2010 Beta 1 release. Some of these  features might be changed in the RTM release.

In my last post I talked about the new major features of Team Build in TFS 2010. This time, I will go into more detail on how you work with build definitions. In TFS 2010, the whole build process is now implemented on top of Windows Workflow Foundation 4.0 (WF4). This means that everything that has to do with creating and customizing builds in TFS 2010 is now done using a workflow designer UI. This means that you no longer have to remember all the different MSBuild targets when you want to insert some custom logic in your build. On the other hand, you obviously need to understand how a default team build process is implemented, which activities does what, what WF properties and variables that exist. And eventually you might also have to learn how to implement custom workflow activites when you need more functionality than what is included in the standard team build activities.

Note that MSBuild is still used to actually compile all the projects. The output from the compilations are available in a separate log file that is available from the build summary view.

So, lets create a new build definition. When you select the New Build Definition menu item, you get a dialog that looks very much like the one in TFS 2008.

General
This tab just contains the name and the description of your build. There is also a checkbox that lets you disable the build definition, in case you want to work on it more before making it enabled.

Trigger
Here you define how this build should be queued. The only new option here in 2010 is Gated Check-in, which is a very cool feature that will stop you from check in in anything that breaks the build.

Workspace
This tab has not changed since 2008. Here you define the workspace for the build, i.e. what part of the source control tree that should be downloaded as part of the build. Here I set the $/Demo/WpfApplication1 as my workspace root. You always want to make your workspace as small as possible to speed up build time.

Build Defaults
In the previous version of Team Build you select which build agent that should run the build. In 2010, you now select a Build Controller. The build controller manages a pool of build agents that will be selected by an algorithm that takes into account the queue length on each build agent, in a round-robin fashion (although this algorithm is not yet documented, and it is not clear if you can implement your own algorithm)

In addition to must enter the drop location for the build.

Process

Now we come to the interesting part! Here you select the Build process file, which is a Windows Workflow XAML file that must be located somewhere in your TFS source control repository. By default for all new team projects, there are two build process files created automatically, DefaultTemplate and UpgradeTemplate. The default template is the standard Team Build process, with the get, label, compile etc.. The UpgradeTemplate process file can be used to execute legacy builds, i.e. TFSBuild.proj files.  

This functionality, e.g. selecting a build process template from a list, is in itself a nice improvement from earlier versions where you always had to create a standard build process and the modify the TFSBuild.proj accordingly. (Lots of people instead wrote applications that create TFSBuild.proj programattically to simplify the process).

However, you should not use the default template as the process file for your builds. Instead you should create a new template from the default template and use this one instead. You do this by clicking on the New button:

This mechanism lets you create a set of build process templates (for example you can have one template for CI builds, one for nightly builds, one for relase builds etc… These templates can be stored in a dedicated location in source control and any changes to them should only be allowed for the build managers. Application developers can then setup new builds from the existing templates and should only need to modify the parameters (see below) which are not part of the template but stored together with the build definition.

You can view and/or edit the build process file by clicking the link which takes you to the source control explorer, then double-click the xaml file to open it up in the workflow designer. The following (slightly MSPaint hacked) screen shot show you the top level process of the DefaultTemplate build process:

You can drill-down into the different activities to see how the process is designed. In my next post I will show how to customize the build process by adding new activities to it.

When you have selected the build process template, you then go through the parameters of the build. The properties are defined in the build process as arguments to the workflow and corresponds to the MSBuild properties in the previous versions. If you have used team build before, you’ll definitely recognize many of the properties. The most important ones are:

Retention Policy
In this tab you select how builds should be retained. Note that you now can select a different configuration for manual/triggered builds and private build.Private here means builds with the Gated Check-in trigger enabled. You will typically want to retain fewer private builds compared with the manual/triggered builds:

Ok, you are done! Save the build definition and queue a build in the team explorer. When the build finishes, double click it to see the Build summary view:

For a detailed view of the build, click the View Log link:

A nice feature here is the Show Property Values link. This show the log, but in addiotn it shows each in/out property for each activity. This is very useful when trying to troubleshoot a failing build:

OK, this was a quick walkthrough of how to create a basic build definition in Team Build 2010. In my next post, I will show how to customize the build process using the workflow designer!

VSTS 2010 Beta 1 is finally available. Beta 1 is a huge release a contains tons of new functionality in almost all areas of Team System. We at Osiris Data have been using Team System since the first beta, and we also help our customers how to adopt TFS in their organization. So for us VSTS 2010 is a very exciting release.

I am planning to post several blog entries about the new functionality of VSTS 2010, in particular I will focus on Team Foundation Server (TFS) and Team Build. I have been working a lot with Team Build 2005 and 2008 and have been looking forward to the 2010 release. VSTS 2008 was a minor release for TFS, so a lot of missing features and change requests haven’t been implemented until now.

So, in this blog post I will run through the new functionality in Team Build 2010, and then I will follow up with several blog posts that go into more detail on the different areas.

Build Controllers lets you pool your builds
In TFS 2005/2008 you could only assign a team build to one build agent. In 2010, you assign a team build to a Build Controller instead. The Build Controller will now be responsible for managing a custom pool of Build Agents and will select a Build Agent to run the Team Build The following screenshot show the Manage Build Controllers dialog which in this case show one build controller that contains two build agents.

Often you need to have several different build agents that have different configurations. For example, you might need .NET 3.5 SP1 on one build agent to be able to build applications that rely on SP1, but then you have apps that still need to be built without SP1 installed. In 2010, you can use tags to tell Team Build which build agent to choose. You create the tags in the Build Agent Properties dialog:

To assign a build definition to one or more tags, you use the Agent Requirements property in the process tab of the build definition:

Build Process are defined using Windows Workflow 4.0
Team Build 2005 and 2008 relies solely upon MSBuild to drive the build process. MSBuild is a powerful language designed for building applications. It is also lets you extend the build process by using tasks which are .NET classes that implement a certian interface.

The problem with MSBuild is that the learning curve is rather steep, it is quite unintuitive for most people. In addition, Team Build 2005/2008 hides the entire build process in a separate targets file. Although this does hide all the hairy details from the user from , it also makes customizing a team build very complex. The order in which the team build targets are executed is very hard to visualize by just looking at the MSBuild file. One of the most common questions from team build users is I want to run my things after my project is built/tested/dropped, how do I do it?

2010 to the resuce! The build process of a team build is now completely driven by Windows Workflow 4.0. This means a whole new UI for editing your build process, and you need to learn how to write/use workflow activities when you need to extend your builds instead of MSBuild. Here is a snapshot of parts of a build process in the workflow designer in VSTS 2010, and also the Toolbos with some of the available team build activities:

You should recognize most of the activities from the corresponding team build tasks in Team Build 2005/2008. The default build process is still the same, get the sources, label the sources, build the projects, test the projects etc.

So now the entire build process is visible when extending a team build. For most times, you don’t have to change the build process itself, but just modify the build properties to fit your needs. There are several new build properties that lets you customize the build that previously was quite difficult/cumbersome:

I will go into more detail on these properties in an upcoming blog post.

Reusing Build Definitions using Build Process Templates
When creating builds for a larger system, you often find yourself defining a set of builds (CI, Nightly, Release etc) for the different applications. These builds are often defined using the exact same (or very similar) build definition process. In TFS 2005/2008, there is no way to easily resuse build definitions as templates for new builds.

TFS 2010 supports this by adding a feature called Build Process Templates. When creating a new build, you select from a list of build process templates that contain the actual build process. These templates are workflow XAML files that are located in source control  in the TeamBuildProcessTemplates folder:

By default, there are two build process templates when you create a new team project, DefaultTemplate and UpgradeTemplate. DefaultTemplate is the standard build process that performs a complete build of your app. The UpgradeTemplate is a sort of placeholder build template that is used to execute legacy TFSBuild.proj files. This means that you can still use your existing team build definitions when upgrading to TFS 2010.

In a later post, I will show how you can create new build process templates for your particular scenarios, and how to share them between team projects.

The Build Summary and Log is now readable!
The Build summary screen has also been completely rewritten and lets you find the problem with the build much faster than in previous versions of Team Build. It also contains much more information, such as build times compared to the 9 previous builds and the list of properties that were sent into the build. Here is a screenshot of the Build Summary and the Activity Log:

For more info about these screens, check out Jason Prickett’s blog: http://blogs.msdn.com/jpricket/archive/2009/05/12/tfs-2010-beta1-build-details-view-summary-section.aspx, http://blogs.msdn.com/jpricket/archive/2009/05/18/tfs-2010-beta1-build-details-view-log-view-section.aspx

Gated Checkins, a.k.a No More Broken Builds
The gated checkin is a feature that has been requested for a long time and exisst in other build labs. The idea is very simple, when a checkin occurs, you want to ensure that that checkin does not break the build. At certain stages in a project, such as in the stabilization phase, broken builds cause lots of grief with people trying to fix the build to be able to push out a new version of the application to the testers. Team Build 2010 implements this by performing whats known as a Private Build in isolation from the source control system. This means that if the private build is succesful, the checkin will be executed. But if the build fails, the checkin will not be executed.

You turn on Gated checkin for a build by modifying the trigger of the build definition:

When a user tries to check in something that will trigger this build (e.g. inside the workspace of the build), the following dialog is shown:

If the checkin affects several builds with the Gated Checkin trigger turned on, the user have to choose one of these builds.

Btw, there is an open source project on CodePlex that implements a variation of this for TFS 2008 called Buddy Build: http://www.codeplex.com/BuddyBuild

———————————-

Alright, I think that is enough for a summary post of the new features of Team Build 2010. As I mentioned previously, I will write several posts where I go into more detail on these new features and how they compare to the existing functionality in TFS 2005/2008.

Happy building!