Deploying an Azure Web Site using TFS Build vNext

TFS 2015 is around the corner, and with it comes a whole new build system. All the biggest pain points from the existing build system (now called “XAML builds”) are gone and instead we get a light weight build system with a web UI that makes it very easy to customize our build processes and that doesn’t perform a lot of magic such as redirecting your build output for example.

In this post, I will show you just how easy it is to setup a build that builds a standard ASP.NET web application, generates a web deploy package as part of the build and then pick up this package and deploys it to a Azure web site.
As part of this, you will also see how smooth the integration with Azure is when it comes to connecting your subscription.

 
Sample Application

For this blog post I will use the common FabrikamFiber sample application, the full source for this is available at https://fabrikam.codeplex.com/

I have created a new team project called FabrikamFiber on my local TFS 2015 instance, and pushed the source to the default Git repo in this team project:

image

So, let’s see how we can build, test and deploy this application to Azure.

Register your Azure subscription in TFS

First of all, you need to add your Azure Subscription to TFS. You only need to this once of course (or at least one per subscription in case you have several).

This is done on the collection level, by using the Services tab:

image

Click the Add new Azure Subscription link on the top left. Now you need to enter the Subscription Id and the subscription certificate (you can use credential as well, but using the certificate option is more secure). 
To get this information, follow these steps:

  • Open a Windows PowerShell windows in administrative mode
  • Type  Get-AzurePublishSettingsFile
  • This will open a web browser and automatically download your subscription file
  • Open the subscription file (named <SubscriptionName>-<Date>-credentials.publishsettings
  • In this file, locate the Subscription Id and the ManagementCertificate fields

Now, copy these values into the Add Azure Subscription dialog:

image

Press OK. After it is saved, you should see your subscription to the left and some general information:

image

That’s it, now we can crate a build definition for our application.

 

Create a Build Definition

Go to the team project where the source is located and click on the Build Preview tab. Click on the + button to create a new vNext definition:
Now you can select a definition template for your build definition. Go to the Deployment tab and select the Azure Website template:

 

image

 

This will create a build definition with three steps, one for build the solution, one for running all the tests and one for deploying a web site to Azure.
Note: For this post, I disabled the test step since not all tests pass in the default FabrikamFiber solution.

 

image

 

If you take a look at the Visual Studio Build step, you can see the arguments that are passed to MSBuild

/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation=”$(build.stagingDirectory)”

These are standard MSBuild parameters for triggering and configuration web deploy during compilation. These specific settings will create a web deploy package and put it in the staging directory of the build definition.

 

Now, over to the deployment step. Here you can see that you can select your Azure subscription that we registered before. In addition we give the web site a unique name, this name is what the public web site will be called,in this case
it will be http://fabrikamfibervnext.azurewebsites.net.  We also need to specify the Web Site Location for this web site.

The final parameter that we need to specify is the Package. By default it will fetch all zip files located in the staging directory. I want to deploy the FabrikamFiber.Web application, so I change this to $(build.stagingDirectory)FabrikamFiber.Web.zip.

image

That’s it! Save the build definition and queue a new build.After it has completed, you should see a log that looks something like this and you should have a web sites published in Azure.
As you can see, all steps in the process are shown to the left and you can easily see the output from each step.

 

image

 

Deploying Multiple Web Sites

If you’re familiar with the FabrikamFiber sample solution, you know that it actually has two web applications in the same solution, FabrikamFiber.Web and Fabrikamfiber.Extranet.Web.

So, how do we go about to deploy both web sites as part of our build? Well, it couldn’t be easier really, just add a new Azure Web Site Deployment build step and reference the other web deploy package:

 

image

This will now first build the solution and then publish both web sites to Azure.

 

Summary

Now you have seen how easy it is to build and deploy web applications using TFS Build vNext.

Speaking at DevSum 2015

Image result for devsumVSO

 

This year I’ll be presenting at DevSum, which is the largest .NET conference in Sweden. I’ll be talking about Visual Studio Online, Microsoft cloud offering for your development projects. I will go through all the major features of VSO, such as source code hosting, agile planning tooling , automated builds, test management and integration with other 3rd party service such as Trello and AppVeyor. I will also talk about how you can migrate your existing projects to VSO

 

I’ve inserted the abstract of the session below, you can find the full information here

http://www.devsum.se/speaker/jakob-ehn/ 

Hope to see you there!

 

Moving your development to the Cloud using Visual Studio Online

Did you know that Visual Studio Online supports all aspects of your development process? In addition to keeping track of your source code (using Git or TFVC), you
can manage your product- and sprint backlogs, run automated builds in the cloud, define and execute test plans and create and manage release pipelines using
Visual Studio Release Management. On top of this, VSO integrates with several other popular services such as Jenkins, AppVeyor, Trello and CampFire, which means
that you can move to VSO and still use other services in the ecosystem.

In this session we will do a complete tour of what Visual Studio Online has to offer, and discuss the differences and pros/cons compared with on premise.
We will also talk about how you can go about to migrate your development to VSO.

Introducing GitFlow for Visual Studio

Update: The exension is now also available for Visual Studio 2015 Preview: https://visualstudiogallery.msdn.microsoft.com/f5ae0a1d-005f-4a09-a19c-3f46ff30400a

 

GitFlow is a popular workflow that provides a consistent naming convention to your branches as well as clear guidance on how your code should flow through these branches.
GitFlow was introduced by Vincent Driessen in this post back in 2010, and quickly caught a lot of attention in the community. Since GitFlow by nature is very prescriptive it made a lot of
sense to implement tooling support for the workflow, which Vincent added shortly after. His repo is available at https://github.com/nvie/gitflow, although it hasn’t been updated since 2012.
However, several forks has been made, one of the most active is being developed by Peter van der Does at https://github.com/petervanderdoes/gitflow

To make GitFlow more approachable I decided to integrate the GitFlow toolset into Visual Studio, by extending Team Explorer. This makes it very easy to access the commands and lowers the learning
curve a bit by making it available as a UI. Note that the extension includes the GitFlow scripts from Peter van der Does fork of GitFlow and uses them for every command, so it provides the exact same
functionality as the GitFlow scripts does.

 

Installation

Note: The extension requires Visual Studio 2013 Update 3 or higher

You’ll find the extension over at the Visual Studio Gallery, just search for GitFlow in the Extension and Updates Window.
Or, download it from https://visualstudiogallery.msdn.microsoft.com/27f6d087-9b6f-46b0-b236-d72907b54683:

image

Install it and restart Visual Studio, as usual.

 

Using the extension

When you connect to a Git repo in Visual Studio (either local or remote), you will see a new icon show up on the home page in Team Explorer:

image

 
Now, if GitFlow is not installed on your machine you will be presented with the following message:

image

By clicking Install, the extension will copy the necessary files into the Git for Windows directory, and run the install script for GitFlow.
You can check the installation details for GitFlow here https://github.com/nvie/gitflow/wiki/Windows

(Note: Since copying files into the %ProgramFiles(x86)%Gitbin directory requires elevated priveledges, this is done by running this as a elevated Powershell script.
You will see this flash by during the installation
)

 

Initialize

Now, you are ready to start use the extension! The first thing you will have to do is to initialize the repo for GitFlow. What this means is that you
should create your permanent development and master branches, and set the naming conventions for future feature, release and hotfix branches:

 

image

You can also set the Tag prefix, which will be used when you tag a release or hotfix branch as part of finishing up those branches.

Note that all output from all GitFlow operations are sent to a separate output window pane in Visual Studio, which is activated when the command start.
Here you can which gitflow command that were used and the output from it:

 

image

 

Working with features

From now on, the extension will show the recommended actions based on which branch you are currently in. After initializing the repo, you will be in the develop
branch, so from here you would typically either start a new feature, release or hotfix branch.

 

image

Clicking Start Feature will let you define a name for the branch. GitFlow will add the feature branch prefix for you so don’t include that.

Here I have created a feature branch called SingleSignOn:

image

 

As you can see, the extension will now suggest Finish Feature as the recommended action:

image

Note that all other actions are still available from the Other menu.

 

In the GitFlow world, you are allowed to have multiple feature branches but only one release and hotfix branch at any single time.
In fact, if you try to create multiple release branches, you will get an error.

Keeping track of multiple feature branches can be cumbersome, so the extension lists all active feature branches is a separate section:

image       image

As you can see, if you hover over a feature will get some more details on it, and if you right-click on it you can (depending on the state), checkout, track or publish the feature branch.
I will be looking at adding more functionality here in the future to make is even easier to use this workflow.

 

I hope you will find this extension useful. Please report any bugs or feedback over at the GitHub site for this extenstion, over at  https://github.com/jakobehn/GitFlow.VS

Trigger Releases in Visual Studio Release Management

In my previous post, I showed how you can trigger a release in Visual Studio Release Management from a TeamCity build step.

When Visual Studio Release Management 2013 RTM’ed, it came with customized TFS build templates that made it easy to trigger a release from a TFVC or Git build in TFS. These build templates relied on the ReleaseManagementBuild command line client, so it required the VSRM client being installed on the build server.

Then, with the update train of Visual Studio 2013, new functionality such as vNext deployments, support for Powershell DSC etc was added. Together with this a new REST API was introduced that removes the need for a command line application to trigger a release. However, this REST API _only_ works for vNext deployments, and it will most likely never be implemented for agent based deployments. Basically all new functionality in the release management area only works for vNext deployements, so you should be looking to move in that direction.

Also, in VS 2013 Update 4 support was added for connecting to the Release Management service running in Visual Studio Online. With this option, another way of triggering release was added, in which the Release Management service listens for build completion events in TFS which will kick off a release. It is still possible to use the API however, in cases where you don’t run TFS Build.

 

So, since this is rather confusing at the moment and causes a lot of questions, I decided to summarize the currently available combinations and the options that you have when it comes to triggering a release in VSRM:

 

Source

Release Management

Deployment Type

Options

VSO Build

VSO

vNext

Automatically (through build completed event)

VSO Build

VSO

Agent-Based

N/A

TFS Build

On-Premise (Update 3)

vNext

REST API

TFS Build

On-Premise (Update 3)

Agent-Based

ReleaseBuildTemplate

3rd party/share

VSO

vNext

REST API

3rd party/share

VSO

Agent-Based

N/A

3rd party/share

On-Premise (Update 3)

vNext

REST API

3rd party/share

On-Premise (Update 3)

Agent-Based

ReleaseManagementBuild.exe

Note that some of these combinations are not supported at the moment, those are marked with N/A.

Hope that helps

TFS Build vNext – A Preview

Currently, Microsoft is working hard on a complete rewrite of the TFS Build system. They announced this, among other things, at the Connect event back in November and did a short demo of it. It is not yet available, but as part of the MVP program, a few of us has now been fortunate enough to get access to an early preview of the new build system.

Note: This version is in pre-alpha state, meaning that a lot of things can and will change until the final version!

Microsoft will enable a beta version to a broader public on Visual Studio Online later on this year, and it will RTM in TFS 2015. I will write a series of blog posts about Build vNext as it evolves. In this first post I will take a quick look at how to create and customize build definitions and running them in the new Web UI.

Key Principals

The key principals of TFS Build vNext are:

  • Customization should be dead simple. Users should not have to learn a new language (Windows Workflow anyone?) in order to just run some tool or script as part of the build process
  • Provide a simple Web UI for creating and customizing build definitions. No need for Visual Studio to customize builds anymore.
  • Support cross platform builds (Linux, MacOS…)  As in most other areas right now, Microsoft is serious about supporting other platforms than Windows, and TFS Build won’t be an exception.
  • Sharing build infrastructure. The concept of a build controller tied to a collection is gone, instead we will create agent pools at the deployment level and connect agents to those.
  • Don’t mess with the build output and keep my logs clean. One of the most frustrating things about the current version of TFS Build is that by default it doesn’t preserve the output structure of the compiled projects, but instead places them beneath a common Binaries directory. This causes all sort of problems, such as post build events that are dependent on relative path etc.

    CI build == Dev build !

  • Work side by side with the existing build system. All your existing XAML build definitions will continue to work just like before, and you will be able to create new ones. But don’t expect anything to be added in terms of functionality to the XAML builds

 

Creating a build definition in vNext

In this post, I will show how easy it is to create a  new build definition in Build vNext, and customize it to update the version info in all the assembly files using a PowerShell task with a custom script.

Note: There will probably be a versioning task out of the box in the final release, but currently there isn’t.

Let’s get started:

  1. From the dropdown, you can select from a list of build definition templates. These templates are created by saving an existing build definition as a template. Currently, they are scoped to the team project level: 

    image

    Here I’m going to select the Visual Studio definition template.

  2. This results in a build definition with two tasks, one that compiles the solution and one that runs all the unit tests using the Visual Studio test runner task

    image 

  3. Selecting a task shows the properties of that task to the right, the properties of the Visual Studio Build task is shown above. Some of these properties are typed, meaning that for example the Solution property let’s you browse the current repository to select a solution. By default, it will locate all solution files and compile them (using the ***.sln pattern)

    You can also see that the Platform and the Configuration property uses the variables $(platform) and $(config). We will come back to them shortly.

  4. The first thing we want to do typically is specifying which repository that the build will download the source from. To do this, select the Repository tab:

    image

    Looking at the Repository type field, since this is a Git team project it will let me choose either Git (which means a git repo in the current team project) or GitHub, which actually lets me choose any Git repo. In that case I will need to enter credentials as well. When selecting the Git repo type, i can then select the repo (Visual Studio ALM in this case) from the current team project, and then I can select the default branch in a drop down.

  5. If you want to add more tasks to the definition, go back to the Build tab and click on the Add new task link. This will display a list of all available tasks, shown below:

    image

    The list of tasks here are of course not complete, but as you can see there are a lot of cross platform tasks there already. To add a task to the build definition,  select it and press Add.
    Here, I will select the PowerShell task that I will use to version all my assemblies:

    image 

    When adding a new task it will end up at bottom of the task list, but you can easily reorder the tasks by using drag and drop. Since I must stamp the version information before i compile my solution, I have dragged the PowerShell task to the top. This task then requires me to select a PowerShell file to execute, and I also specified a working folder since the script I am using assumes this. All paths here are relative to the repository.

    Note: Although currently not possible, it will be possible to rename the build tasks

  6. On the Triggers tab, the only option available right now is to trigger the build on every check-in (or push in the case of Git). I can also select to batch the changes, which is the same thing as the Rolling build trigger in the existing version of TFS build.
    This means that all changes that are checked in during a running build will be batched up and processed in the next build as soon as the current build has completed.

    In addition we can specify one or more branches that should trigger this build.

     image

    Tip: The filter strings can include * as a filter, which lets me for example specify refs/heads/feature* as a filter that will trigger on any change on any branch that starts with the string feature.

  7. On the Variables tab, we can specify and add new variables for this build definition. Here you can see the config and platform variables that were referenced in the Build tab:

    image

  8. Last, let’s take a look at the Options tab. The MultiConfiguration option enables us to build the selected solution(s) multiple times, each time for every combination of the variables that are specified here. By default, it will list the config and platform variables here. So, this means that the build will first build my solution using Debug | Any CPU, and the it will compile it using Release | Any CPU.

    If I check the Parallel checkbox, it will run these combinations in parallel, if there are multiple agents available of course.

    image

    Also, we can enable copy to Staging and Drop location here. The build agent will copy everything that matches the Search pattern and place it in the staging folder

  9. Finally, let’s save the new build definition. Note that you can also add a comment every time you save a build definition:

    image

    These comments are visible on the History tab, where you can view the history of all the changes to the build definition and view the changes between any of the changes. Nice!

 

Running a Build

The build definition is complete, let’s kick off a build. Click on the Queue build.. button at the top to do this. This will show the live output of the build, both in a console output window and in an aggregated view where the status of each task is shown:

image

 

Here you can note several nice features of build vNext:

  • The console output shoes the exact output from each task, just like it would if you ran the same command on your local machine
  • The aggregated view to the left shows the status and progress of every task which makes it very clear how far in the process the build is
  • This view also show parallel builds, in this case I have enabled the MultiConfiguration option and have two build agents configured, meaning that both Debug|AnyCPU and Release|AnyCPU are being processed in parallel on separate agents.
  • If you select one of the tasks to the left, you will see the build output for this particular task

 
When the build completes, you will see a build summary and you can also view a timeline of the build where the duration of each task is shown:

 

image

Trigger Visual Studio Release Management vNext from TeamCity

The last couple of updates to Visual Studio has included a lot of new functionality for Visual Studio Release Management. The biggest one is the introduction of so called vNext releases, that leverages Powershell DSC for carrying out the provisioning and deployments of environments and applications.

Also included in Visual Studio 2013 Update 3 was the introduction of a REST API that allows us to both trigger new releases and read back information about them. This API is only available for vNext release templates, and will probably not be implemented for the “old” agentbased deployments.

This REST API opens up a lot of integration possibilities, for example we don’t have to use TFS Build to automatically trigger a release. In this post, I will show how we can use the popular TeamCity build server from JetBrains to trigger a release as part of the build.

 
First of all, we need to setup a release template in Visual Studio Release Management. I won’t go through all the details here, the most important once is that the components that we define must use the UNC Path as the source parameter:

image

Here I have specified a shared folder, \localhostbuildoutput. Beneath this path, VSRM will look for a folder that typically correspond to a build number that we will pass in using the API, as you will see later on.

Next, we create our release template. To be able to trigger a build using the API, make sure that you tick the “Can trigger a release from a Build?” checkbox:

 image

 

Then we define our deployment sequence, in this case I just have one action for deploying my web site using a PowerShell script:

SNAGHTML5c6e9ead

There is nothing really nothing special here, I refere to the DeployWebSite.ps1 which is a PowerShell DSC script that installs my web site. I have also a DestinationPath variable that is referenced in the DSC script and specifies where the web site should be installed on the server.

 

Now, to trigger a release of this release template we are going to use a PowerShell script, that we can execute using TeamCity. I have used a sample from Microsoft and modified it to use parameters.

Here is the full script:

 

param( [string]$rmServer, [string]$rmUser, [string]$rmPassword, [string]$rmDomain, [string]$releaseDefinition, [string]$deploymentPropertyBag ) $deploymentPropertyBag = $propertyBag = [System.Uri]::EscapeDataString($deploymentPropertyBag) $exitCode = 0 trap { $e = $error[0].Exception $e.Message $e.StackTrace if ($exitCode -eq 0) { $exitCode = 1 } } $scriptName = $MyInvocation.MyCommand.Name $scriptPath = Split-Path -Parent (Get-Variable MyInvocation -Scope Script).Value.MyCommand.Path Push-Location $scriptPath $orchestratorService = "http://$rmServer/account/releaseManagementService/_apis/releaseManagement/OrchestratorService" $status = @{ "2" = "InProgress"; "3" = "Released"; "4" = "Stopped"; "5" = "Rejected"; "6" = "Abandoned"; } #For Update3 use api-version=2.0 for Update4 use api-version=3.0. $uri = "$orchestratorService/InitiateRelease?releaseTemplateName=" + $releaseDefinition + "&deploymentPropertyBag=" + $propertyBag + "&api-version=3.0" $wc = New-Object System.Net.WebClient #$wc.UseDefaultCredentials = $true # rmuser should be part rm users list and he should have permission to trigger the release. $wc.Credentials = new-object System.Net.NetworkCredential("$rmUser", "$rmPassword", "$rmDomain") try { $releaseId = $wc.UploadString($uri,"") $url = "$orchestratorService/ReleaseStatus?releaseId=$releaseId" $releaseStatus = $wc.DownloadString($url) Write-Host -NoNewline "`nReleasing ..." while($status[$releaseStatus] -eq "InProgress") { Start-Sleep -s 5 $releaseStatus = $wc.DownloadString($url) Write-Host -NoNewline "." } " done.`n`nRelease completed with {0} status." -f $status[$releaseStatus] } catch [System.Exception] { if ($exitCode -eq 0) { $exitCode = 1 } Write-Host "`n$_`n" -ForegroundColor Red } if ($exitCode -eq 0) { "`nThe script completed successfully.`n" } else { $err = "Exiting with error: " + $exitCode + "`n" Write-Host $err -ForegroundColor Red } Pop-Location exit $exitCode

Basically, this script calls the following REST API endpoint:

http://RMSERVER/account/releaseManagementService/_apis/releaseManagement/OrchestratorService/InitiateRelease?releaseTemplateName=RELEASEDEFINITION&deploymentPropertyBag=PROPERTYBAG&api-version=3.0 

 

This REST endpoint returns a release id, which we can use to read the status of the release. The script loops until the status is not “In Progress” and the quits and returns an exit code depending on how the release went.

 

The parameters of the above endpoint are:

  • RMSERVER

    The URL including port to the release management server, typically somthing like contoso.com:1000 

  • RELEASEDEFINITION

    The name of the release template that we want to trigger.

  • PROPERTYBAG

    Now this one is a bit special. This is an array of key/value which is not yet documented but it typically looks like this:

  • { "Component1:Build" : "Component1Build_20140814.1", "Component2:Build" : "Component2Build_20140815.1", "ReleaseName" : "$releaseName" }

    The first two lines references two different components in RM. Here I have just called them Component1 and Component2. The :Build is a keyword and must be there. The value part is the build number, which RM will append to the UNC path that we defined earlier. Note that we can use different build numbers per component here if we want to.

    The last line is a predefined keyword that allows us to specify the name of the release, something that we actually can’t do using the RM client or the standard RM TFS Build release template so this is a nice feature.

 

So, to execute this script in TeamCity, add the above script to repository and then add a PowerShell build step at the end of your build in TeamCity that looks something like this:

image 

 

The Script Arguments property is the tricky part, since we have to escape the quotes for the propertyBag parameter, and we will also use the %env:BUILD_NUMBER% variable from TeamCity for the build number. Here is the full string as an example:

-rmServer SERVER:1000 -rmUser USER -rmPassword PASSWORD -rmDomain DOMAIN -releaseDefinition VSGallery -deploymentPropertyBag “{“VSGallery Web Application:Build” : “%env.BUILD_NUMBER%”,”ReleaseName” : “%env.BUILD_NUMBER%”}”

 

Running this build in TeamCity will now trigger a release in Release Management:

image

Using Log4Net for Application Insights

Getting started with Application Insights (AI) in a new or existing application is very easy. From Visual Studio 2013 Update 3 it is even integrated right into the New Web Project dialog:

 

image

 

When you select this option, Visual Studio will automatically create a new instrumentation key for you that identifies the web application, and insert the necessary Javascript into your master layout page that takes care of sending usage information to Application Insights. Try running your application, click around a few times, and you will see information showing up in the Azure portal within, literally, a few seconds.

NB: you can also perform this operation later on, by select XXXX from the context menu in Solution Explorer.

This will give you a lot of information such as page views, response times, user information like browser version and geographic location. If you want to add custom tracing, you can do this using the Application Insights Telemtry SDK It exists both for .NET code and JavaScript, so you can add tracing both on the backend and the frontend.

Now, this is very nice for a new project but what if you have an existing application that already contains a lot of code for writing trace and debug information? Perhaps you are using Log4Net or the TraceListener class to emit diagnostic information in some way. Well, the good news is that the nice fellows over at Microsoft thought about this. They have created a set of public NuGet packages that makes it very easy to forward the information that you are logging to Application Insights. This means that you won’t have to rewrite any of the existing code and still have the tracing information show up in the portal.

For example, if you are using Log4Net you can use the Microsoft.ApplicationInsights.Log4NetAppender package that will send the information you log using the Log4Net API to AI. It is implemented as a standard Log4Net Appender class, which makes it very easy to use. Just add the package to the projects that performs logging, and the following information will be added to your configuration file:

<log4net> <root> <level value="ALL"/> <appender-ref ref="aiAppender"/> </root> <appender name="aiAppender" type="Microsoft.ApplicationInsights.Log4NetAppender.ApplicationInsightsAppender, Microsoft.ApplicationInsights.Log4NetAppender"> <layout type="log4net.Layout.PatternLayout"> <conversionPattern value="%message%newline"/> </layout> </appender> </log4net>

Now, you existing logging Log4Net code..

log4net.Config.XmlConfigurator.Configure(); var logger = log4net.LogManager.GetLogger(this.GetType()); logger.Info("Some information message"); logger.Warn("A warning message"); logger.Error("An error message");

   

Will end up as Trace information in the Azure Portal;

 

image

 
Since this is a regular log4net appender, you can apply the standard filters, to include or exclude certain types of information.

Microsoft has also implemented NuGet packages for NLog and also a TraceListener class, for use when you do standard .NET tracing. In addition, there are some 3rd party packages that covers other logging frameworks such as Serilog:

 

image

 

Happy logging!

Copy TFS Build Definitions between Projects and Collections

The last couple of years it has become apparent that using multiple team projects in TFS is generally a bad idea. There are of course exceptions to this, but there are a lot ot things that becomes much easier to do when you put all of your projects and team in the same team project.

Fellow ALM MVP Martin Hinshelwood has blogged about this several times, as well as other people in the community. In particular, using the backlog and portfolio management tools makes much more sense when everything is located in the same team project.

Consolidating multiple team projects into one is not that easy unfortunately, it involves migrating source code, work items, reports etc.  Another thing that also need to be migrated is build definitions. It is possible to clone build definitions within the same team project using the TFS power tools.

The Community TFS Build Manager also lets you clone build definitions to other team projects. But there is no tool that allows you to clone/copy a build definition to another collection. So, I whipped up a simple console application that let you do this.

The tool can be downloaded from

https://onedrive.live.com/redir?resid=EE034C9F620CD58D!8162&authkey=!ACTr56v1QVowzuE&ithint=file%2c.zip

 

Using CopyTFSBuildDefinitions

You use the tool like this:

CopyTFSBuildDefinitions  SourceCollectionUrl  SourceTeamProject  BuildDefinitionName  DestinationCollectionUrl  DestinationTeamProject [NewDefinitionName]


Arguments

  • SourceCollectionUrl
    The URL to the TFS collection that contains the team project with the build definition that you want to copy
  • SourceTeamProject
    The name of the team project that contains the build definition
  • BuildDefinitionName
    Name of the build definition
  • DestinationCollectionUrl
    The URL to the TFS collection that contains the team project that you want to copy your build definition to
  • DestinationTeamProject
    The name of the team project in the destination collection
  • NewDefinitionName (Optional)
    Use this to override the name of the new build definition. If you don’t specify this, the name will the same as the original one
  • Example:

    CopyTFSBuildDefinitions  https://jakob.visualstudio.com DemoProject  WebApplication.CI https://anotheraccount.visualstudio.com
     
     

    Notes

    • Since we are (potentially) create a build definition in a new collection, there is no guarantee that the various paths that are defined in the build definition exist in the new collection. For example, a build definition refers to server paths in TFVC or repos + branches in TFGit. It also refers to build controllers that definitely don’t exist in the new collection. So there will be some cleanup to do after you copy your build definitions. You can fix some of these using the Community TFS Build Manager, for example it is very easy to apply the correct build controller to a set of build definitions

    • The problem stated above also applies to build process templates. However, the tool tries to find a build process template in the new team project with the same file name as the one that existed in the old team project. If it finds one, it will be used for the new build definition. Otherwise is will use the default build template

    • If you want to run the tool for many build definitions, you can use this SQL scripts, compliments of Mr. Scrum/ALM MVP Richard Hundhausen to generate the necessary commands:

      USE Tfs_Collection

      GO

      SELECT ‘CopyTFSBuildDefinitions.exe http://SERVER:8080/tfs/collection “‘ + P.ProjectName + ‘” “‘ + REPLACE(BD.DefinitionName,”,”) + ‘” http://NEWSERVER:8080/tfs/COLLECTION TEAMPROJECT’

        FROM tbl_Project P

             INNER JOIN tbl_BuildGroup BG on BG.TeamProject = P.ProjectUri

             INNER JOIN tbl_BuildDefinition BD on BD.GroupId = BG.GroupId

        ORDER BY P.ProjectName, BD.DefinitionName

     

    Hope that helps, let me know if you have any problems with the tool or if you find it useful

TFS Build: NuGet.exe was not found in the expected location

Problem

One of our customers recently had a problem with NuGet restore when they created a new build template, based on the standard TfvcTemplate.12.xaml template. In TFS 2013, package restore is done automatically by the default build templates.

It is configured as part of the RunMSBuild activity, where you can enable and disable this by setting the EnableNuGetPackageRestore property:

image

 

However, when we were executing the builds we got the following warning in the build log:

Unable to restore NuGet packages. Details: NuGet.exe was not found in the expected location: C:UsersBuildAppDataLocalTempBuildAgent18Assembliesnuget.exe

This error puzzled us quite a bit. NuGet.exe is installed as part of TFS Build and is located in the %ProgramFiles%/Microsoft Team Foundation Server 12.0/Tools folder. Why was Team Build looking in the build agent custom assembly folder?

It turns out that the reason for this was that the customer had not only checked in the custom activity assemblies in TFVC, they had also checked in all the references TFS assemblies (such as Microsoft.TeamFoundation.Build.Workflow.dll for example). This is not necessary, but had until now never caused any problems. But now, since this assembly was used during the build, it looked in the current path of the assembly for NuGet.exe which resolved to the path from the error message above.

Solution

After removing all the TFS assemblies from version control NuGet package restore started working again.

Creating a Build Definition using the TFS 2013 API

Almost four years ago I wrote a post on how to create a build definition programmatically using the TFS 2010 API. The code is partyl still valid, but since a lot of the information in a build definition is dependent on the build process template, the code in the blog
does not work properly for the TFS 2013 default build templates. In addition, since the introduction of Git in TFS 2013, there are some other differences in how you create a build definition for a Git team project compared to a TFVC team project.

So, in this post I will show an updated version of the code for creating a build defintion. I will actually show two samples, one for the GitTemplate.12.xaml and one for the TfvcTemplate.xaml which are the default template used in TFS 2013.

 

Creating a Git Build definition (GitTemplate.12.xaml)

Here is the code for creating a build definition using the GitTemplate.12.xaml build process template
//Create build definition and give it a name and desription
IBuildDefinition buildDef = buildServer.CreateBuildDefinition(tp);
buildDef.Name = "ATestBuild";
buildDef.Description = "A description for this build defintion";
buildDef.ContinuousIntegrationType = ContinuousIntegrationType.Individual; //CI

//Controller and default build process template
buildDef.BuildController = buildServer.GetBuildController("Hosted Build Controller");
var defaultTemplate = buildServer.QueryProcessTemplates(tp).First(p => p.TemplateType == ProcessTemplateType.Default);
buildDef.Process = defaultTemplate;

//Drop location
buildDef.DefaultDropLocation = "#/";    //set to server drop

//Source Settings
var provider = buildDef.CreateInitialSourceProvider("TFGIT");
provider.Fields["RepositoryName"] = "Git";
provider.Fields["DefaultBranch"] = "refs/heads/master";
provider.Fields["CIBranches"] = "refs/heads/master";
provider.Fields["RepositoryUrl"] = url + "/_git/Git";
buildDef.SetSourceProvider(provider);

//Process params
var process = WorkflowHelpers.DeserializeProcessParameters(buildDef.ProcessParameters);

//What to build
process.Add("ProjectsToBuild", new[]{"Test.sln"});
process.Add("ConfigurationsToBuild", new[]{"Mixed Platforms|Debug"});

//Advanced build settings
var buildParams = new Dictionary<string, string>();
buildParams.Add("PreActionScriptPath", "/prebuild.ps1");
buildParams.Add("PostActionScriptPath", "/postbuild.ps1");
var param = new BuildParameter(buildParams);
process.Add("AdvancedBuildSettings", param);

//test settings
var testParams = new Dictionary<string, object>
                                 {
                                     { "AssemblyFileSpec", "*.exe" },
                                     { "HasRunSettingsFile", true },
                                     { "ExecutionPlatform", "X86" },
                                     { "FailBuildOnFailure", true },
                                     { "RunName", "MyTestRunName" },
                                     { "HasTestCaseFilter", false },
                                     { "TestCaseFilter", null }
                                 };

var runSettingsForTestRun = new Dictionary<string, object>
                                            {
                                                { "HasRunSettingsFile", true },
                                                { "ServerRunSettingsFile", "" },
                                                { "TypeRunSettings", "CodeCoverageEnabled" }
                                            };
testParams.Add("RunSettingsForTestRun", runSettingsForTestRun);

process.Add("AutomatedTests", new[]{ new BuildParameter(testParams)});
            
//Symbol settings
process.Add("SymbolStorePath", @"\serversymbolssomepath");

buildDef.ProcessParameters = WorkflowHelpers.SerializeProcessParameters(process);

//Retention policy
buildDef.RetentionPolicyList.Clear();
buildDef.AddRetentionPolicy(BuildReason.Triggered, BuildStatus.Succeeded, 10, DeleteOptions.All);
buildDef.AddRetentionPolicy(BuildReason.Triggered, BuildStatus.Failed, 10, DeleteOptions.All);
buildDef.AddRetentionPolicy(BuildReason.Triggered, BuildStatus.Stopped, 1, DeleteOptions.All);
buildDef.AddRetentionPolicy(BuildReason.Triggered, BuildStatus.PartiallySucceeded, 10, DeleteOptions.All);

//Lets save it
buildDef.Save();

 

Some things to note here:

  • The IBuildDefinitionSourceProvider interface is new in TFS 2013, and the reason for it is of course to abstract the differences between TFVC and Git source control. As you can see, we use the “TFGIT”

    to select the correct provider, and then we use the Fields property to populate it with information

  • The process parameters are created by using dictionaires, with the corresponding key and values. If you are familiar with the GitTemplate.12.xaml, you will recognize the name of these parameters.
  • As for drop locations, in TFS 2013 you can select between no drop location, a drop folder (share) or a server drop, which means the output is stored inside TFS and accessible from the web access.

    In the sample above, we specify #/ which (not that obvious) means a server drop. If you want to use a share drop location, just specify the server path for the DefaultDropLocation

 

Creating a TFVC Build definition (TfvcTemplate.12.xaml)

AS it turns out, creating a TFVC build definition using the TfvcTemplate.12.xaml is almost identical, since the build team went to great effort and abstracted away most differences. The only difference in fact, at least when it comes to the most common settings is how you define the workspace mappings. And this code is the same as it was in TFS 2010/2012. In addition, you don’t need to create a source provider, because there is nothing that needs to be configured other than the workspace.

 

Here is the full sample for TFVC:

//Create build definition and give it a name and desription
IBuildDefinition buildDef = buildServer.CreateBuildDefinition(tp);
buildDef.Name = "ATestBuild";
buildDef.Description = "A description for this build defintion";
buildDef.ContinuousIntegrationType = ContinuousIntegrationType.Individual; //CI

//Controller and default build process template
buildDef.BuildController = buildServer.GetBuildController("Hosted Build Controller");
var defaultTemplate = buildServer.QueryProcessTemplates(tp).First(p => p.TemplateType == ProcessTemplateType.Default);
buildDef.Process = defaultTemplate;

//Drop location
buildDef.DefaultDropLocation = "#/";    //set to server drop

//Source Settings
buildDef.Workspace.AddMapping("$/Path/project.sln", "$(SourceDir)", WorkspaceMappingType.Map);
buildDef.Workspace.AddMapping("$/OtherPath/", "", WorkspaceMappingType.Cloak); 

//Process params
var process = WorkflowHelpers.DeserializeProcessParameters(buildDef.ProcessParameters);

//What to build
process.Add("ProjectsToBuild", new[]{"Test.sln"});
process.Add("ConfigurationsToBuild", new[]{"Mixed Platforms|Debug"});

//Advanced build settings
var buildParams = new Dictionary<string, string>();
buildParams.Add("PreActionScriptPath", "/prebuild.ps1");
buildParams.Add("PostActionScriptPath", "/postbuild.ps1");
var param = new BuildParameter(buildParams);
process.Add("AdvancedBuildSettings", param);

//test settings
var testParams = new Dictionary<string, object>
                                 {
                                     { "AssemblyFileSpec", "*.exe" },
                                     { "HasRunSettingsFile", true },
                                     { "ExecutionPlatform", "X86" },
                                     { "FailBuildOnFailure", true },
                                     { "RunName", "MyTestRunName" },
                                     { "HasTestCaseFilter", false },
                                     { "TestCaseFilter", null }
                                 };

var runSettingsForTestRun = new Dictionary<string, object>
                                            {
                                                { "HasRunSettingsFile", true },
                                                { "ServerRunSettingsFile", "" },
                                                { "TypeRunSettings", "CodeCoverageEnabled" }
                                            };
testParams.Add("RunSettingsForTestRun", runSettingsForTestRun);

process.Add("AutomatedTests", new[]{ new BuildParameter(testParams)});
            
//Symbol settings
process.Add("SymbolStorePath", @"\serversymbolssomepath");

buildDef.ProcessParameters = WorkflowHelpers.SerializeProcessParameters(process);

//Retention policy
buildDef.RetentionPolicyList.Clear();
buildDef.AddRetentionPolicy(BuildReason.Triggered, BuildStatus.Succeeded, 10, DeleteOptions.All);
buildDef.AddRetentionPolicy(BuildReason.Triggered, BuildStatus.Failed, 10, DeleteOptions.All);
buildDef.AddRetentionPolicy(BuildReason.Triggered, BuildStatus.Stopped, 1, DeleteOptions.All);
buildDef.AddRetentionPolicy(BuildReason.Triggered, BuildStatus.PartiallySucceeded, 10, DeleteOptions.All);

//Lets save it
buildDef.Save();

 

Hope you find this useful!