Automatisch updaten van Nuget Packages in jouw projecten

We all have shared code which we use in multiple projects. Just as you do with third party libraries you can also use NuGet packages to distribute your own internal shared code using for example the build-in NuGet feed in VSTS (or in TFS if you’re hosting it on-premise). You can automate the creation of the packages using a build definition. Every change to your shared code will produce a new version of the packages. Over time, the amount of shared code grows and more and more projects are using it. Every time there’s a new version you need to update the packages in the projects that are using them.

You will typically right-press your solution, select ‘Manages NuGet packages for solution...' and update the packages. Once you’ve done that you will build your solutions and run your tests locally. If everything is green you will probably ask your colleagues to look at your pull-request you’ll create and have it complete. This all takes quite some time and is very boring stuff to do. There's another problem. Since it’s boring you’ll probably only update the packages in the project that really need the update now. Projects that you haven't touched on for a while will be left unattended. This means that you will never know if the change in your shared code broke something in these older projects up until the moment you have to fix a minor bug or something. By that time months might have gone by and it gets harder to fix the problem. If we could only automate this process...

Packages upgrade build

The rest of this blog will explain how we automated this process. First, we need to create a build which will do the manual stuff we had to do in Visual Studio. So, open up Azure DevOps or TFS Server and create a new empty build and name it Package Upgrade Build [SolutionName]. Choose any project that uses your shared packages as a source repository. Add four PowerShell steps and select the first one. We use pull-requests for every change that will be merged to master. Colleagues will review the change and we’ve added a build validation to the pull-request which runs a build and runs the unit-tests. By doing so we always make sure master is in a deployable state. This means that our first step in this build will be the creation of a new branch. This is done by the following line:

git checkout -b packagesupgrade/Shared-$(Build.BuildId) -q

This will create the branch with for example the name Packagesupgrade/Shared-1234 Now that we have a branch it’s time to do the actual upgrade work. I’ve you’ve ever looked at the changes to your code when doing a package upgrade in Visual Studio you’ll see to type of changes; the versions in the packages.config file have been changed and the versions of the references in the .csproj files have changed. I’ve you’re using the newer Visual Studio 2017 file structure then there will only be changes in the .csproj files. To update the packages in a project based on the full .Net framework we have to use nuget.exe, projects based on .Net Core use the dotnet command. Here’s the script for the full framework solutions:

$version = "4.7.1"

$url = "https://dist.nuget.org/win-x86-commandline/v$version/nuget.exe"

$nugetFolder = "$env:Agent_ToolsDirectory\NuGet\$version\x64"
if(!(Test-Path -Path $nugetFolder)){
    New-Item -ItemType directory -Path "$env:Agent_ToolsDirectory\NuGet\$version\x64"
}

$nugetExe = "$nugetFolder\nuget.exe"
if(!(Test-Path -Path $nugetExe)){
    Invoke-WebRequest -Uri $url -OutFile $nugetExe
}

write-host $nugetExe update "$(PathToSolution)" $(NugetFeeds)

& $nugetExe update "$(PathToSolution)" $(NugetFeeds)

Most of this script is there to download a recent version of Nuget.exe. The last line will do the actual work. The following script can be used for .Net Core solutions:

[uri] $PackagesUri = "https://[url]/DefaultCollection/_packaging/[feed-id]/nuget/v3/query2/"

$Base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $User,$env:SYSTEM_ACCESSTOKEN))) 

$PackageIdStartsWith = "[YourOrganization]Shared."

foreach($childItem in (Get-ChildItem -Path $SolutionRootDirFullPath -Force -Recurse -Include "*.csproj"))
   {
      Write-Host "Updating $childItem"
      [xml]$XmlDocument = Get-Content -Path $childItem

      foreach($reference in $XmlDocument.Project.ItemGroup.PackageReference)
      {
         if($reference -eq $null)
         {
            continue
         }
         
         if($reference.Include -ne $null -and $reference.Include.StartsWith($PackageIdStartsWith))
         {
$packageQuery = "$($PackagesUri)?q=$($reference.Include)"

 $Packages = Invoke-RestMethod -Uri $packageQuery `
 -Method Get `
 -ContentType "application/json" `
 -Headers @{Authorization=("Basic {0}" -f $Base64AuthInfo)}
            $package =  $Packages.data | where { $_.id -eq $reference.Include }

             write-host "Found version " $package.version " for " $reference.Include

            dotnet remove $childItem package $reference.Include
            write-host dotnet add $childItem package -v $package.version $reference.Include
            dotnet add $childItem package -v $package.version $reference.Include
         }
      }
   }

In this script we use the dotnet command to do the update. We will have to remove and re-add the packages to do the upgrade. Since the add command sometimes did and sometimes didn’t correctly use the latest versions (caching perhaps?) I decided to use the nuget package feed to query the latest version and pass that to the add command. If you’re on Azure DevOps you can also use the api, this is not available yet on-premises. Now that the files have been updated we need to commit and push the changes. Use the following script:

git config --global core.safecrlf false 

git add . > $null 

git commit -m 'Automatic PR creation for upgrading shared package files' 

git push origin Packagesupgrade/Shared-$(Build.BuildId) -q

The last step in this build is to create a new pull request. Use the following script to do so.

[CmdletBinding()]
param()

Trace-VstsEnteringInvocation $MyInvocation

try
{
   $script:Repository = Get-VstsInput -Name Repository -Require
   $script:SourceRefName = Get-VstsInput -Name SourceRefName -Require
   $script:TargetRefName = Get-VstsInput -Name TargetRefName -Require
   $script:APIVersion = Get-VstsInput -Name APIVersion -Require

   <#
    .SYNOPSIS
        Uses the VSTS REST API to create pull request   
     
    .DESCRIPTION
        This script uses the VSTS REST API to create a Pull Request in the specified
        repository, source and target branches. Intended to run via VSTS Build using a build step for each repository.
        https://www.visualstudio.com/en-us/docs/integrate/api/git/pull-requests/pull-requests
    .NOTES
        Existing branch policies are automatically applied.
    .PARAMETER Repository
        Repository to create PR in
    
    .PARAMETER SourceRefName
        The name of the source branch without ref.
    
    .PARAMETER TargetRefName
        The name of the target branch without ref.
    
    .PARAMETER APIVersion
        API versions are in the format {major}.{minor}[-{stage}[.{resource-version}]] - For example, 1.0, 1.1, 1.2-preview, 2.0.
    
    .PARAMETER PAT
        Personal Access token. It's recommended to use a service account and pass via encrypted build definition variable.
    
    .PARAMETER ReviewerGUID
        ID(s) of the initial reviewer(s). Not mandadory. 
        Can be found in existing PR by using GET https://{instance}/DefaultCollection/{project}/_apis/git/repositories/{repository}/pullRequests/{pullrequestid}?api-version=3.0
   #>

   Function CreatePullRequest     
   {       
      # Contruct Uri for Pull Requests: https://{instance}/DefaultCollection/{project}/_apis/git/repositories/{repository}/pullRequests?api-version={version}
      # Note: /DefaultCollection/ is required for all VSTS accounts
      # Environment variables are populated when running via VSTS build
      [uri] $CommitsUri = $env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI + $env:SYSTEM_TEAMPROJECT + "/_apis/git/repositories/$Repository/commits?searchCriteria.itemVersion.version=$SourceRefName&api-version=$APIVersion"
      [uri] $PRUri = $env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI + $env:SYSTEM_TEAMPROJECT + "/_apis/git/repositories/$Repository/pullRequests?api-version=$APIVersion"
      Write-Host "Posting to $PRUri"
      Write-Host "SourceRefName: $SourceRefName"
      Write-Host "TargetRefName: $TargetRefName"

      # Base64-encodes the Personal Access Token (PAT) appropriately
      # This is required to pass PAT through HTTP header in Invoke-RestMethod bellow
      $User = "" # Not needed when using PAT, can be set to anything
      $Base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $User,$env:SYSTEM_ACCESSTOKEN)))     

      # Prepend refs/heads/ to branches so shortened version can be used in title
      $Ref = "refs/heads/"
      $SourceBranch = "$Ref" + "$SourceRefName"
      $TargetBranch = "$Ref" + "$TargetRefName"

      Write-Host "SourceBranch: $SourceBranch"
      Write-Host "TargetBranch: $TargetBranch"

      $commitsResponse = Invoke-RestMethod -Uri $CommitsUri `
         -Method Get `
         -ContentType "application/json" `
         -Headers @{Authorization=("Basic {0}" -f $Base64AuthInfo)}

      $prDescription = "PR Created automatically through the REST API"
      if($commitsResponse.value[0] -ne $null)
      {
            $prDescription = $commitsResponse.value[0].comment
            Write-Host "PR Description: $prDescription"
      }

      Write-Host "PR Desc before modification: $prDescription"
      $prDescription = $prDescription -replace '"', '\"'
      Write-Host "PR Desc after modification: $prDescription"

      $JSONBody = '
            {
               "sourceRefName": "' + $SourceBranch + '",
               "targetRefName": "' + $TargetBranch + '",
               "title": "Merge ' + $sourceRefName + ' to ' + $targetRefName + '",
               "description": "' + $prDescription + '",
            }'

      Write-Host "JSON Body: $JSONBody"

       # Use URI and JSON above to invoke the REST call and capture the response.
      $Response = Invoke-RestMethod -Uri $PRUri `
                                    -Method Post `
                                    -ContentType "application/json" `
                                    -Headers @{Authorization=("Basic {0}" -f $Base64AuthInfo)} `
                                    -Body $JSONBody

       # Get new PR info from response
       $script:NewPRID = $Response.pullRequestId
       $script:NewPRURL = $Response.url
   }

   Try
   {
       "Creating PR in $Repository repository: Source branch $SourceRefName Target Branch: $TargetRefName"
       CreatePullRequest
       "Created PR $NewPRID`: $NewPRURL"
   }
   Catch
   {
       $result = $_.Exception.Response.GetResponseStream()
       $reader = New-Object System.IO.StreamReader($result)
       $reader.BaseStream.Position = 0
       $reader.DiscardBufferedData()
       $responseBody = $reader.ReadToEnd();
       $responseBody
       Exit 1 # Fail build if errors
   }
}
finally
{
    Trace-VstsLeavingInvocation $MyInvocation
}

Trigger upgrade build

The build we’ve just created, the package upgrade build, needs to be triggered as soon as anything has changed to our shared code and new packages have been created. I’m using a task from the VSTS Marketplace to do this. Install this task and add it to your shared codes build. Now let’s configure the Trigger Build task. Fill the ‘Name of the Build Definitions that shall be triggered’-field with the name of the build that should be triggered. You’ve created this build in the previous paragraph. Click on ‘Authentication’ and select ‘OAuth Token’ under ‘Authentication Method’. To allow this task to access the token click on the Agent phase and select ‘Allow scripts to access OAuth token’ under ‘Additional options’ Now trigger this build and let Azure DevOps work for you!

Automatically update NuGet packages in your projects

Packages upgrade build

Trigger upgrade build

Verder lezen