Usage of libgit2sharp to get updates from remote repository - libgit2sharp

I need my application to do two simple operations with Git.
1) Clone a remote repository;
2) Perform updates from the remote repository on regular basis.
I am absolutely at a lost how to do it with libgit2sharp. The API is quite complicated.
Can anybody help me please?

1) Clone a branch from remote repository;
// Url of the remote repository to clone
string url = "https://github.com/path/to_repo.git";
// Location on the disk where the local repository should be cloned
string workingDirectory = "D:\\projects\to_repo";
// Perform the initial clone
string repoPath = Repository.Clone(url, workingDirectory);
using (var repo = new Repository(repoPath))
{
... do stuff ...
}
2) Perform updates from the remote branch on regular basis
// "origin" is the default name given by a Clone operation
// to the created remote
var remote = repo.Network.Remotes["origin");
// Retrieve the changes from the remote repository
// (eg. new commits that have been pushed by other contributors)
repo.Network.Fetch(remote);
Update
...files are not updating. May it be because Fetch downloads changes, but not applies them to working copy?
Indeed, a Fetch() call doesn't update the working directory. It "downloads" the changes from upstream and update the references of the remote tracking branches.
If you want to update the content of the working directory, then you'll have to either
Replace the content of your working directory by checking out (ie. repo.Checkout()) the remote tracking branch that has been fetched
Merge the remote tracking branch you're interested in (once it's been fetched) into your current HEAD (ie. repo.Merge()`)
Replace the call to Fetch() with a call to repo.Network.Pull() which will perform the fetch and the merge
Note: You'll have to cope with conflicts during the merge is some changes you've locally performed prevent the merge from applying cleanly.
You can peek at the test suites to get a better understanding of those methods and their options:
CheckoutFixture.cs
MergeFixture.cs
NetworkFixture.cs

Related

PullRequest Build Validation with Jenkins and OnPrem Az-Devops

First off the setup in question:
A Jenkins Instance with several build nodes and on prem Azure-Devops server containing the Git Repositories.
The Repo in question is too large to always build on push for all branches and all devs, so a small workaround was done:
The production branches have a polling enabled twice a day (because of testing duration which is handled downstream more builds would not help with quality)
All other branches have their automated building suppressed. They still can start it manually for Builds/Deployments/Unittests if they so choose.
The jenkinsfile has parameterization for which platforms to build, on prod* all the platforms are true, on all other branches false.
This helps because else the initial build of a feature branch would always build/deploy locally all platforms which would take too much of a load on the server infrastructure.
I added a service endpoint for Jenkins in the Azure Devops, added a Buildvalidation .yml - this basically works because when I call the sourcebranch of the pull request with the merge commitID i added a parameter
isPullRequestBuild which contains the ID of the PR.
snippet of the yml:
- task: JenkinsQueueJob#2
inputs:
serverEndpoint: 'MyServerEndpoint'
jobName: 'MyJob'
isMultibranchJob: true
captureConsole: true
capturePipeline: true
isParameterizedJob: true
multibranchPipelineBranch: $(System.PullRequest.SourceBranch)
jobParameters: |
stepsToPerform=Build
runUnittest=true
pullRequestID=$(System.PullRequest.PullRequestId)
Snippet of the Jenkinsfile:
def isPullRequest = false
if ( params.pullRequestID?.trim() )
{
isPullRequest = true
//do stuff to change how the pipeline should react.
}
In the jenkinsfile I look whether the parameter is not empty and reset the platforms to build to basically all and to run the unittests.
The problem is: if the branch has never run, Jenkins does not already know the parameter in the first run, so it is ignored, building nothing, and returning with 0 because "nothing had to be done".
Is there any way to only run the jenkins build if it hasnt run already?
Or is it possible to get information from the remote call if this was the build with ID 1?
The only other thing would be to Call the Jenkins via web api and check for the last successful build, but in that case I would have have the token somewhere stored in source control.
Am I missing something obvious here? I dont want to trigger the feature branch builds to do nothing more than once, because Devs could lose useful information about their started builds/deployments.
Any ideas appreciated
To whom it may concern with similar problems:
In the end I used the following workaround:
The Jenkins Endpoint is called via a user that only is used for automated builds. So, in case that this user triggered the build, I set everything to run a Pull Request Validation, even if it is the first build. Along the lines of
def causes = currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause')
if (causes != null)
{
def buildCauses= readJSON text: currentBuild.getBuildCauses('hudson.model.Cause$UserIdCause').toString()
buildCauses.each
{
buildCause ->
if (buildCause['userId'] == "theNameOfMyBuildUser")
{
triggeredByAzureDevops = true
}
}
}
getBuildcauses must be allowed to run by a Jenkins Admin for that to work.

BuildHTTPClient not able to get Build Definition Steps?

We are using the BuildHTTPClient to programmatically create a copy of a build definition, update the variables in memory and then save the updated object as a new definition.
I'm using Microsoft.TeamFoundation.Build2.WebApi.BuildHTTPClient 16.141. The TFS version is 17 update 3 (rest api 3.x)
This is a similar question to https://serverfault.com/questions/799607/tfs-buildhttpclient-updatedefinition-c-example but I'm trying to stay within using the BuildHttpClient libraries and not go directly to the RestAPIs.
The problem is the Steps list is always null along with other properties even though we have them in the build definition.
UPDATE Posted as an answer below
After looking at #Daniel Frosts attempt below we started looking at using older versions of the NuGet package. Surprisingly the supported version 15.131.1 does not support this but we have found out that the version="15.112.0-preview" does.
After rolling back all of our Dlls to match that version the steps were cloned when saving the new copy of the build.
All of the code examples we used work when you are using this package. We were unable to get Daniel's example working but the version of the Dll was the issue.
We need to create a GitHub issue and report it to MS
First Attempt - GetDefinitionAsync:
VssConnection connection = new VssConnection(DefinitionTypesDTO.serverUrl, new VssCredentials());
BuildHttpClient bdClient = connection.GetClient<BuildHttpClient>();
Task <BuildDefinition> resultDef = bdClient.GetDefinitionAsync(DefinitionTypesDTO.teamProjectName, buildID);
resultDef.Wait();
BuildDefinition updatedDefinition = UpdateBuildDefinitionValues(resultDef.Result, dr, defName);
updatedTask = bdClient.CreateDefinitionAsync(updatedDefinition, DefinitionTypesDTO.teamProjectName);
The update works on the variables and we can save the updated definition back to TFS but there are not any tasks in the newly created build definition. When we look at the object that is returned from GetDefinitionAsync we see that the Steps list is empty. It looks like GetDefinitionAsync just doesn't get the full object.
Second Attempt - Specific Revision:
int rev = 9;
Task <BuildDefinition> resultDef = bdClient.GetDefinitionAsync(DefinitionTypesDTO.teamProjectName, buildID, revision: rev);
resultDef.Wait();
BuildDefinition updatedDefinition = UpdateBuildDefinitionValues(resultDef.Result, dr, defName);
Based on SteveSims post we were thinking we are not getting the correct revision. So we added revision to the request. I see the same issue with the correct revision. Similarly to SteveSims post I can open the DefinitionURL in a browser and I see that the tasks are in the JSON in the browser but the BuildDefinition object is not populated with them.
Third Attempt - GetFullDefinition:
So then I thought to try getFullDefinition, maybe that's that "Full" means of course with out any documentation on these libraries I have no idea.
var task2 = bdClient.GetFullDefinitionsAsync(DefinitionTypesDTO.teamProjectName, "MyBuildDefName","$/","TfsVersionControl");
task2.Wait();
Still no luck, the Steps list is always null even though we have steps in the build definition.
Fourth Attempt - Save As Template
var task2 = bdClient.GetTemplateAsync DefinitionTypesDTO.teamProjectName, "1_Batch_Dev");
task2.Wait();
I tried saving the Build Definition off as a template. So in the Web UI I chose "Save as Template", still no steps.
Fifth Attempt: Using the URL as mentioned in SteveSims post:
Finally i said ok, i'll try the solution SteveSims used, using the webclient to get the object from the URL.
var client = new WebClient();
client.UseDefaultCredentials = true;
var json = client.DownloadString(LastDefinitionUrl);
//Convert the JSON to an actual builddefinition
BuildDefinition result = JsonConvert.DeserializeObject<BuildDefinition>(json);
This also didn't work. The build definition steps are null. Even when looking at the Json object (var json) i see the steps. But the object is not loaded with them.
I've seen this post which seems to add the Steps to the base definition, i've tried this but honestly I'm having an issue understanding how he has modified the BuildDefinition Object when referencing that via NuGet?
https://dennisdel.com/blog/getting-build-steps-with-visual-studio-team-services-.net-api/
After looking at #Daniel Frosts attempt below we started looking at using older versions of the NuGet package. Surprisingly the supported version 15.131.1 does not support this but we have found out that the version="15.112.0-preview" does.
After rolling back all of our Dlls to match that version the steps were cloned when saving the new copy of the build.
All of the code examples above work when you are using this package. We were unable to get Daniel's example working but we didn't try hard as we had working code.
We need to create a GitHub issue for this.
Found this in my code, which works.
Use this package, not sure if it could have an impact (joke).
...packages\Microsoft.TeamFoundationServer.Client.15.112.1\lib\net45\Microsoft.TeamFoundation.Build2.WebApi.dll
private Microsoft.TeamFoundation.Build.WebApi.BuildDefinition GetBuildDefinition(string projectName, string buildDefinitionName)
{
var buildDefinitionReferences = _buildHttpClient.GetFullDefinitionsAsync(projectName, "*", null, null, DefinitionQueryOrder.DefinitionNameAscending, top: 1000).Result;
return buildDefinitionReferences.SingleOrDefault(x => x.Name == buildDefinitionName && x.DefinitionQuality != DefinitionQuality.Draft);
}
With the newer clients Steps will always be empty. In newer api-versions (which are used by the newer clients) the steps have moved to Phases. If you use GetDefinitions or GetFullDefinitions and look in
definition.Process.Phases[0].Steps
you'll find them. (GetDefinitions gets shallow references so the process won't be included.)
The Steps collection still exists for compatibility reasons (we don't want apps to crash with stuff like MethodNotFoundExceptions) but it won't be populated.
I was having this problem, although I able to get Phases[0] information at runtime, but could not get it at design time. I solved this problem using dynamic type.
dynamic process = buildDefTemplate.Process;
foreach (BuildDefinitionStep tempStep in process.Phases[0].Steps)
{
// do some work here
}
Not, it is working!
Microsoft.TeamFoundationServer.Client version 16.170.0 I can get build steps through process.Phases[0].Steps only with process and step being dynamic as #whitecore above stated
var definitions = buildClient.GetFullDefinitionsAsync(project: project.Name);
foreach (var definition in definitions.Result)
{
Console.WriteLine(string.Format("\n {0} - {1}:", definition.Id, definition.Name));
dynamic process = definition.Process;
foreach (dynamic step in process.Phases[0].Steps)
{
Console.WriteLine(step.DisplayName);
}
}

Cake Build auto merge functionality

I'm using cakebuid as my build tool for TFS 2017 Update 2 and trying to implement the traditional Git Flow. In this flow, there are a few automatic merges that happen every time changes get into master, those changes need to be propagated to the develop branch.
Using cake I can run a PowerShell script or use LibGit2Sharp to accomplish the automatic merge for the best case scenarios. But, what about when the merge has conflicts? Do I need to fail the whole build because the merge process fail?
We have certainly something to deal with merges in TFS, this is no other than the Pull Request.
Question
Is there any tool or add-in for cake that allows me to create
Pull Request during the execution of a build step?
I don't think there is any add-in available for you to create a pull request but since you can run PowerShell, you can easily use the TFS rest api to create pull request
https://www.visualstudio.com/en-us/docs/integrate/api/git/pull-requests/pull-requests
It was recently announced that there is a VSTS CLI:
https://blogs.msdn.microsoft.com/devops/2017/11/15/introducing-the-new-cli-for-vsts/
Which includes the ability to create a pull request:
https://learn.microsoft.com/en-gb/cli/vsts/get-started?view=vsts-cli-latest#create-a-pull-request
I don't think it would be particularly hard to create a Cake Addin which wraps this tool, and exposes the functionality through a set of addin's.
In the mean time, you could shell out to this tool using the Process aliases that currently exist in Cake.
Finally, I spend sometimes creating the package:
Nuget: https://www.nuget.org/packages/Cake.Tfs.AutoMerge
GitHub: https://github.com/mabreuortega/Cake.Tfs
The way you can use it now is similar to this:
Task("Merge")
.Does(c => {
CreateAutoMergePullRequest(
new AutoMergeSettings
{
// Required
CollectionUri = "https://{instance}/{collection-name}",
ProjectName = "project-name",
RepositoryName = "repository-name",
SourceBranch = "refs/heads/release/1.0.0",
TargetBranch = "refs/heads/develop",
Title = "[Auto Merge from Release]",
// Optional
Description = "Brief description of the changes about to get merge",
// Control
DeleteSourceBranch = false,
SquashMerge = false,
OverridePolicies = true,
AutoComplete = true,
AutoApprove = true
});
});
Any suggestions, please use the GitHub issue tracker.
Hope this help!

tfs checkout / checkin programmatically

I am building a VS2010 addin. This addin will work only for our custom project types and create a menu item that will copy the output assembly from the current solution to another solution. Both are under TFS control.
I have the following code:
var tfs = new TeamFoundationServer(address);
var version = (VersionControlServer)tfs.GetService(typeof(VersionControlServer));
var workspace = version.GetWorkspace(System.Net.Dns.GetHostName().ToString(), version.AuthorizedUser);
workspace.PendEdit(dest);
System.IO.File.Copy(source, dest, true);
Now I want to checkin the change. The problem is that I don't know how to select only that file I checked out just now? I have other pending changes in the same project and also in other projects. Will this checkin EVERYTHING I have checked out?
Can I be more selective?
PendingChange[] pendingChange = workSpace.GetPendingChanges(dest);
workSpace.CheckIn(pendingChange, comments);
Workspace.GetPendingChanges Method (String)
http://msdn.microsoft.com/en-us/library/bb139277(v=vs.100).aspx
Parameters
item: The path, local or server, to the item that is being queried.
And
Workspace.CheckIn Method
http://msdn.microsoft.com/en-us/library/microsoft.teamfoundation.versioncontrol.client.workspace.checkin(v=vs.100).aspx
Parameters
changes
The set of pending changes to check in. If you do not specify this parameter, all changes in the workspace are checked in.
comment
The comment to be associated with this check-in. May be null.

TFS 2010 Issue with tracking Changesets in Builds that are a result of Gated Checkins

In order to retrieve the information which Changeset was included in which Build, we use "Label Sidekick" of Team Foundation Sidekicks, where we place the Label of the Build & expect to find the newly built Changeset.
Our development process in TFS 2010 is making use of 'Gated' checkins, so we are faced with the situation that the latest checkins are not presented in Sidekicks (we actually receive the changeset of the previous build). This is explainable, since at the time the labeling takes place, the latest changes have not yet been committed.
The BuildLog does report the associated Changeset correctly.
I 've made several experiments in our Build Process Template but can't seem to get what we need.
Placing, for example, the Labeling activity out of the "Run On Agent" scope, lead me to a build that fails at the very start with an "Object reference not set to an instance of an object." (I suppose this is related with fact I had to widen the scope for 'Label' & 'Workspace' variables to get the second part running).
The 'before' state of the build process template for this attempt is here (this works), the 'after' state ("Object ref not set..") is here.
So, to summarize, two different types of input could help me out:
How should I change our build process template so that the labeling happens after the Gated checkins have been committed? (-- This would rationalize the display in Sidekicks)
or
How can I programmatically retrieve the associated Changeset of each Build? (-- This would enable me to write a small app that could obsolete the Sidekicks angle)
You can use the TFS API to get this done.
public static void GetBuild()
{
var tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(new Uri("http://tfsdevlonuk:8080/tfs/gazprom.mt"), new UICredentialsProvider());
tfs.EnsureAuthenticated();
var buildServer = tfs.GetService<IBuildServer>();
// Get Project Name
var versionControl = tfs.GetService<VersionControlServer>();
var teamProjects = versionControl.GetAllTeamProjects(true);
// Get Builds for a team project
var buildDetails = buildServer.QueryBuilds(teamProjects[0].Name);
// For each build
foreach (IBuildDetail buildDetail in buildDetails)
{
// Get the build details
var buildInfor = buildDetail.Information;
// More build infor like shelveset, etc
Debug.Write(buildDetail.LabelName + buildDetail.ShelvesetName);
}
The above code will help you get the build details programatically. I have some blog posts on how to connect to tfs programmatically and use the tfs api. http://geekswithblogs.net/TarunArora/archive/2011/06/18/tfs-2010-sdk-connecting-to-tfs-2010-programmaticallyndashpart-1.aspx

Resources