Crawler-Lib Build-Tools Samples

The Crawler-Lib Build-Tools are PowerShell based build automation tools that can be integrated in a continuous integration process or used stand alone even direct in Visual Studio via the NuGet console.

XML Configuration

The Build-Tools are configured with a XML file with the default name “BuildConfig.xml”. Here is the basic structure:

<?xml version="1.0" encoding="utf-8"?>
<!-- Build configuration -->
<BuildConfig >
  <!-- Definition of the tool that creates a label for the new build -->
  <Label Tool="BuildRevisionTimestamp" />
  <Credentials>
    <ProviderTool="Plain">
      NuGetApiKey=1234
      FtpServer=user:password
    </Provider>
 </Credentials>
  <!-- The solutions are build one by one -->
  <Solutions>
    <!-- A Solution to build. -->
    <Solution Name="Test">
      <!-- Definition of the tool that creates a version for the solutuon for new builds -->
      <Version Tool="AppendLabel">2.1</Version>
      <!-- The build sequences are executable by the appopriate commands -->
      <BuildSequences>
        <!-- A build sequence for a new build, the steps are executed one by one -->
        <New-Build>
          <Step Tool="Autover" Path="Test.sln" />
          <StepTool="MSBuild"Path="$($solution.Name).sln"Configuration="Release"Platform="Any CPU" />
          <StepTool="MSBuild"Path="$($solution.Name).sln"Configuration="Release"Platform="x86" />
          <StepTool="MSBuild"Path="$($solution.Name).sln"Configuration="Release"Platform="x64" />
          <Step Tool="NUnit" Path="UnitTestsbinreleaseunitTests.exe" Output="NUnitTestResult.xml"/>
          <Step Tool="FxCop" Path="UnitTestsbinreleaseunitTests.exe" Output="FxCopTestResult.xml"/>
        </New-Build>
        <!-- A build sequence to process an exiting build (here to publish the build)-->
        <Publish-Build>
        </Publish-Build>
      </BuildSequences>
    </Solution>
  </Solutions>
</BuildConfig> 

 

Powershell in the XML Configuration

Nearly all XML attributes in the configuration can use PowerShell. The strings are treated as powershell strings. As a result the strings in the XML configuration must be properly escaped if no PowerShell is meant. The Powershell escape character is “`”. All powershell commands and statements have full access to the surrounding build process which is also powershell.

In this example the path name is constructed from the solution name using powershell:

<Step Tool="Autover" Path="$($solution.Name).sln" />

All tools labled as Powershell have PowerShell commands in the XML Element. This build step will execute the powershell stements inside the element:

<Step Tool="Powershell">
 write-host "New-Build for" $solution.Name
</Step>

PowerShell is very close and seamless integrated in the XML configuration.

Build Labels

Each build is labeled to identify the build.

BuildRevisionTimestamp Label

The build revision timestamp creates a build.revision label accourding to the microsoft assembly naming specifications. But in contrast to visual studio UTC time is used so that all intraday builds can be orderded in the meaning of “newer” regardless in which timezone they where build. 

<Label Tool="BuildRevisionTimestamp" />

 

Credentials

The build tools are designed to use credentials that are not directy sored in the build steps because of maintainability and security reasons. Build steps can access credentials via the $context.Credentials hashtable. E.g if a NuGetApiKey cedential exists, it cam be used for a ApiKey attribute in XML in this way: ApiKey="$($context.Credentials.NuGetApiKey)"

Credentials are provided in the <Credentials> section of the configuration. In this section multiple <Provider> elements can be specified. The resulting hashtable contains all values of them.

 

Plain Credentials Provider

 The plain provider specifies the credentials direct as name/value pairs:

<Provider Tool="Plain">
  NuGetApiKey=xxx
  UploadServer=user:password
</Provider>

 

File Credentials Provider

The file provider uses the same format as the Plain provider, but reads the text form a file

<Provider Tool="File"path="Credentials.txt" />

 

PowerShell Credentials Provider

The powershell provider executes powershell statements which must return a [Hashtable] with the name/value paris 

<Provider Tool="Powershell">
  return @{"ApiKey"="xxx"; "UploadKey"="xxx"}
</Provider>

 

Solution Versions

Every solution gets a version during the build. At first this is declarative only, but the build steps can use the version to update the assembly and file versions and to verify the generated files. Versions are generated with the Version element in the solution. The generation of a solution version depends on the used tool.

Plain Solution Version

The plain version tool parses the given string to produce a [Version] object. 

<Version Tool="Plain">2.0.1.1</Version>

PowerShell Solution Version

The powershell version tool executes powershell statements which must return a [Version] object. 

<Version Tool="Powershell">
  return [Version]::Parse("2.0.2.3")
</Version>

AppendLabel Solution Version

 

The AppendLabel version tool appends the build label to the given text and parses the result to produce a [Version] object. 

<Version Tool="AppendLabel">2.0</Version>

Build Steps

The building block of a build is a sequence of build steps. These build steps are executed one by one on a build. 

Autover Step

Autover ist the command line untility of Versioning Controlled Build, a Visual Studio addin to automate versioning of .NET and VC++ projects. It can be used to patch the assembly versions of the current build:

<Step Tool="Autover" Path="$($solution.Name).sln" />

BuildSequence Step

The BuildSequence step allows to execute the steps of a build sequence in another build sequence. It is like a call of the build sequence:

<Step Tool="BuildSequence" Name="SubSequence" />

 

Download Step

Downloads the specified remote file to the local location using the WebClient class:

<StepTool="Download" Local="Dictionaries.zip" Remote=ftp://***:***@buildserver/Dictionaries/Dictionaries.zip />

 

MSBuild Step

MSBuild is the build system for Microsoft Visual Studio. It is able to build XML based project files, which include all Visual Studio project files and Visual Studio solutions. It is possible to specify the configuration and platform of the build.

o the build process variables:

<Step Tool="MSBuild" Path="$($solution.Name).sln" Configuration="Release" Platform="Any CPU" />

PowerShell Step

PowerShell can be executed directly. The PowerShell statements have full access to the build process variables:

<Step Tool="Powershell">
 write-host "New-Build for" $solution.Name
</Step>

It is also possible to use an external PowerShell file for the step:

<Step Tool="Powershell" Path="PowerShellFile.ps1" />

Upload Step

Uploads the specified local file to the remote location using the WebClient class:

<StepTool="Upload"Local="UnitTestsbinreleaseunitTests.exe"Remote="ftp://***:***@buildserver/unitTests.exe" />

 

 

RemoveFile

Removes a file if it exists. To ensure that the file exists, perform a VeryfyFile step before.

<Step Tool="RemoveFile" Path="Release.zip" />

 

VerifyFile Step

Veryfy File checks if the given file exist on a specific point of a build. It can also be checked if the file has the buid specific file version and product version. And it can be checked if the file was created (last write timestamp) during the current build. If one of this conditions fail, the build is aborted.

<Step Tool="VerifyFile" Path="TestApplicationbinreleaseTestApplication.exe"  FileVersion="true"  ProductVersion="true"  New="true"/>
<Step Tool="VerifyFile" Path="TestApplicationbinreleaseTestApplication.pdb"  FileVersion="false" ProductVersion="false" New="true"/>

 

Zip Step

Appends the specified file to the zip archieve.

<Step Tool="Zip" Path="TestApplicationbinreleaseTestApplication.exe" Target="binTestApplication.exe" ZipFile="Release.zip"/>
<Step Tool="Zip" Path="TestApplicationbinreleaseTestApplication.pdb" Target="binTestApplication.pdb" ZipFile="Release."/>

 

Extendability

The Build-Tools can be extended by providing build steps, build labelers and solution version providers. A build step for a tool called YourCommand is integrated by defining a function with the name Invoke-BuildStep_YourCommand_Tool:

function Invoke-BuildStep_YourCommand_Tool($context)
{
 $path = Expand-ParameterString $context.Step.Path
 $option1 Expand-ParameterString $context.Step.Option1
 YourCommand $path $option1
}

It can be used in the XML configuration like this:

<Step Tool="YourCommand" Path="YourFile" Option1="Value1"/>

The passed $context variable is a PowerShell hashtable which contains the following data:

• Config
  The configuration of the solution as XML.
• CurrentBuild
  The current build data of the solution as XML. This data is stored after the build sequence.    
• BuildCreatedUtc
  A UTC timestamp when the build was created
• BuildCreatedLocal
  A local timestamp when the build was created according to the settings of the actual computer.
• BuildSequenceName
  The name of the build sequence currently executiong
• Step
  The data of the current build step as XML.
• StepTool
  The step tool of the current step.
• StepNumber
  The number of the current step. Step numbers start by 1 and increment on every step.
• Version
  The solution version for this build. This is generated by the version tool during a new build.
• FileVersion
  The file version for this build, initially the same as the solution version but can be changed during the build process.
• ProductVersion
  the product version for the build, initially the same as the solution version but can be changed during the build process.
• PackageVersion
  the package version for the build, initially the same as the solution version but can be changed during the build process.

 

The build label generation and the solution version generation are also extendable. To provide tools for this you must define functions with this names:

function New-LabelForBuild_MyBuildLabel_Tool( [DateTime] $buildTimeUtc)
function New-VersionForSolution_MyVersion_Tool([System.Xml.XmlElement] $solution, $buildLabel)

And you have your build label and solution version tools integrated. Then they can be used in the configuration with these declarations:

 

<Label Tool="MyBuildLabel" />

<Version Tool="MyVersion">2.1</Version>

 

Debugging the Build Process

The Crawler-Lib Build-Tools are pure PowerShell. There is no magic executable. To debug the build process you just start the PowerShell ISE and set a breakpoint in the Invoke-Build function:

Set-PSBreakpoint -Command Invoke-Build

When you run your next build, the ISE will halt on the breakpoint and you can step in every single build step and see what is going on. This is even true when you execute the Build_tools on a continuous integration server. You just go to the checkout dirtectory and test your stuff.