How to download files/subfolders from Git using libgit2sharp - libgit2sharp

From ASP.NET application I want to download specific files and subfolders that are present in Git remote repository using LibGit2Sharp.
For Example :
Main Remote Repository URL : https://github.com/macmillanhighered/TestApplication
Sub folder URL : https://github.com/macmillanhighered/TestApplication/tree/develop/Scripts/PRODUCTS
Instead of getting all files/folders i want to get specific folder only using libgit2sharp API.
I am using below sample code to do this, but it is not working. Please help me out as soon as possible
// Url of the remote repository to clone
string url = "https://github.com/macmillanhighered/TestApplication/tree/develop/Scripts/PRODUCTS";
// Location on the disk where the local repository should be cloned
string workingDirectory = "C:\\TestGit";
var credential = new UsernamePasswordCredentials() { Username = "nitin", Password = "test" };
CloneOptions cloneOptions = new CloneOptions();
cloneOptions.Credentials = credential;
// Perform the initial clone
string repoPath = Repository.Clone(url, workingDirectory, cloneOptions);
using (var repo = new Repository(repoPath))
{
// "origin" is the default name given by a Clone operation
// to the created remote
var remote = repo.Network.Remotes["origin"];
// Retrieve the changes from the remote repository
// (eg. new commits that have been pushed by other contributors)
Signature sign = new Signature("Test", "", DateTime.Now);
FetchOptions fetchOptions = new FetchOptions();
fetchOptions.Credentials = credential;
MergeOptions mergeOptions = new MergeOptions();
PullOptions pullOptions = new PullOptions();
pullOptions.FetchOptions = fetchOptions;
pullOptions.MergeOptions = mergeOptions;
repo.Network.Pull(sign, pullOptions);
}
I am getting below error
Request failed with status code: 404

I am getting below error
Request failed with status code: 404
Clone() expects an url pointing to a git repository, not a file or folder with it.
Something like this should work better: string url = "https://github.com/macmillanhighered/TestApplication"
Instead of getting all files/folders i want to get specific folder only using libgit2sharp API.
The Git protocol doesn't work like this. You should fetch the whole repository and then locally work with the selected files you're interested in.

Related

Azure DevOps Client Library - Download Earlier Repository Files

I am trying to download earlier repository versions (.csproj files) from the commit records that I am obtaining from the Azure DevOps NuGet client library. I want to do this so I can access the Assembly Version information in the .csproj file. The GetFile() function I am using to get the current version of the file works fine but I want to download the older versions of the file from the commit records.
This is the GetFile function.
public string GetFile(string projectName, string repoName, string fileName)
{
try
{
var items = ListItems(projectName, repoName);
var projectPath = items.FirstOrDefault(i => i.Path.Contains(fileName))?.Path ?? "";
GitHttpClient gitClient = GetGitHttpClient();
GitRepository repo = GetRepositoryAsync(projectName, repoName);
var stream = gitClient.GetItemTextAsync(repo.Id, projectPath).Result;
var reader = new StreamReader(stream);
return reader.ReadToEnd();
}
catch (Exception e)
{
return "";
}
}
And this function gets the commits.
public async Task<List<GitCommitRef>> GetCommitsAsync(string projectName, string repoName)
{
var client = GetGitHttpClient();
var repo = GetRepositoryAsync(projectName, repoName);
var gitQueryCommitsCriteria = new GitQueryCommitsCriteria();
return await client.GetCommitsAsync(repo.Id, gitQueryCommitsCriteria);
}
Now I want to download each version of the .csproj that relates to each of these commits.
Any help anyone can give me with this would be greatly appreciated.
Kind regards, Stuart
The GetFile() function I am using to get the current version of the
file works fine but I want to download the older versions of the file
from the commit records.
I think you're in the right direction. I once used Azure Devops Rest Api Items-Get to get content of one specific version of file successfully with the help of versionDescriptor parameter.
https://dev.azure.com/MyOrgName/MyProjectName/_apis/git/repositories/MyReposName/Items?path=/README.md&versionDescriptor%5BversionOptions%5D=0&versionDescriptor%5BversionType%5D=2&versionDescriptor%5Bversion%5D={Commit ID}&download=true&resolveLfs=true&%24format=octetStream&api-version=5.0-preview.1
Since functions available in client library are corresponding to that in Rest Api, so client library must have corresponding parameters to do that.
And here's what I found:
Most of the overloads of gitClient.GetItemTextAsync() functions have GitVersionDescriptor versionDescriptor = null as input, and I think this is what you need.
Hope it makes some help.

Response Header issue on Azure Web Application

I am not sure what is happening here.
When I run my web application locally and click a button to download a file, the file is downloaded fine and Response header as you can see in the attached screenshot where it says local.
But when I publish the application to azure web app. Somehow the download button stops working. I checked the Response Header and you can see the difference.
What would cause this problem? The code is the same? Is there any settings that I should be setting in azure web app in azure portal?
Updated to add code
I have debugged remotely to figure out what is going on as #Amor suggested.
It is so strange that When I debug on my local machine first ExportTo action gets hit which prepares the TempData then Download action gets called once the first action completed with ajax call.
However, this is not the case when I debug remotely. Somehow the ExportTo action never gets called. It directly calls the Download action. As a result the TempData null checking is always null.
But why? Why on earth and how that is possible? Is there something cached somewhere?
I have wiped the content of web application on the remote and re-publish evertyhing to ensure everything is updated. But still no success.
here is the code:
[HttpPost]
public virtual ActionResult ExportTo(SearchVm searchVm)
{
var data = _companyService.GetCompanieBySearchTerm(searchVm).Take(150).ToList();
string handle = Guid.NewGuid().ToString();
TempData[handle] = data;
var fileName = $"C-{handle}.xlsx";
var locationUrl = Url.Action("Download", new { fileGuid = handle, fileName });
var downloadUrl = Url.Action("Download");
return Json(new { success = true, locationUrl, guid = handle, downloadUrl }, JsonRequestBehavior.AllowGet);
}
[HttpGet]
public ActionResult Download(string fileGuid, string fileName)
{
if (TempData[fileGuid] != null)
{
var fileNameSafe = $"C-{fileGuid}.xlsx";
var data = TempData[fileGuid] as List<Company>;
using (MemoryStream ms = new MemoryStream())
{
GridViewExtension.WriteXlsx(GetGridSettings(fileNameSafe), data, ms);
MVCxSpreadsheet mySpreadsheet = new MVCxSpreadsheet();
ms.Position = 0;
mySpreadsheet.Open("myDoc", DocumentFormat.Xlsx, () =>
{
return ms;
});
mySpreadsheet.Document.Worksheets.Insert(0);
var image = Server.MapPath("~/images/logo.png");
var worksheet = mySpreadsheet.Document.Worksheets[0];
worksheet.Name = "Logo";
worksheet.Pictures.AddPicture(image, worksheet.Cells[0, 0]);
byte[] result = mySpreadsheet.SaveCopy(DocumentFormat.Xlsx);
DocumentManager.CloseDocument("myDoc");
Response.Clear();
//Response.AppendHeader("Set-Cookie", "fileDownload=true; path=/");
Response.ContentType = "application/force-download";
Response.AddHeader("content-disposition", $"attachment; filename={fileNameSafe}");
Response.BinaryWrite(result);
Response.End();
}
}
return new EmptyResult();
}
here is the javascript:
var exportData = function (urlExport) {
console.log('Export to link in searchController: ' + urlExport);
ExportButton.SetEnabled(false);
var objData = new Object();
var filterData = companyFilterData(objData);
console.log(filterData);
$.post(urlExport, filterData)
.done(function (data) {
console.log(data.locationUrl);
window.location.href = data.locationUrl;
});
};
When Export button is clicked exportData function is called:
var exportToLink = '#Url.Action("ExportTo")';
console.log('Export to link in index: '+exportToLink);
SearchController.exportData(exportToLink);
As I mentioned that this code works perfectly on the local machine. something weird is happening on azure webapp that ExportTo action breakpoint is never gets hit.
I am not sure what else I could change to get the ExportTo action hit?
Based on the Response Header of Azure Web App, we find that the value of Content-Length is 0. It means that no data has been sent from web app server side.
In ASP.NET MVC, we can response file using following ways.
The first way, send the file which hosted on server. For this way, please check whether the excel file has been uploaded to Azure Web App. You could use Kudu or FTP to the folder to check whether the file is exist.
string fileLocation = Server.MapPath("~/Content/myfile.xlsx");
string contentType = System.Net.Mime.MediaTypeNames.Application.Octet;
string fileName = "file.xlsx";
return File(fileLocation, contentType, fileName);
The second way, we can read the file from any location(database, server or azure storage) and send the file content to client side. For this way, please check whether the file has been read successfully. You can remote debug your azure web app to check whether the file content hasn't been read in the right way.
byte[] fileContent = GetFileContent();
string contentType = System.Net.Mime.MediaTypeNames.Application.Octet;
string fileName = "file.xlsx";
return File(fileContent, contentType, fileName);
5/27/2017 Update
Somehow the ExportTo action never gets called. It directly calls the Download action. As a result the TempData null checking is always null.
How many instances does your Web App assigned? If your Web App have multi instances, the ExportTo request is handled by one instance and the Download request is handled by another instance. Since the TempData is store in memory of dedicated instance, it can't be got from another instance. According to the remote debug document. I find out the reason why the ExportTo action never gets called.
If you do have multiple web server instances, when you attach to the debugger you'll get a random instance, and you have no way to ensure that subsequent browser requests will go to that instance.
To solve this issue, I suggest you response the data directly from the ExportTo action or save the temp data in Azure blob storage which can't be accessed from multi instances.

clone a git repository with SSH and libgit2sharp

I'm trying to use the library "libgit2sharp" to clone a repository via a SSH key and... I can't find anything... I can clone it via "https" but what I'd like to do is using an SSH key. It's really unclear if it is supported or not.
As of now, there is a SSH implementation using libssh2 library. You can find it here LibGit2Sharp - SSH
You should add libgit2sharp-ssh dependency on you Project to be able to use it. It is available as a nugget: https://www.nuget.org/packages/LibGit2Sharp-SSH
Disclaimer: I haven't found a formal usage guide yet, what I know is from putting together bits and pieces from other user questions through LibGit2 forums.
From what I understood, you would need to create a new credential using eitherSshUserKeyCredentials OR SshAgentCredentials to authenticate using SSH, and pass it as part of CloneOptions.
In the sample code I use "git" as user, simply because the remote would be something like git#bitbucket.org:project/reponame.git , in which case "git" is the correct user, otherwise you will get an error saying
$exception {"username does not match previous request"}LibGit2Sharp.LibGit2SharpException
The code to clone a repo with SSH should be something like that:
public CloneOptions cloningSSHAuthentication(string username, string path_to_public_key_file, string path_to_private_key_file)
{
CloneOptions options = new CloneOptions();
SshUserKeyCredentials credentials = new SshUserKeyCredentials();
credentials.Username = username;
credentials.PublicKey = path_to_public_key_file;
credentials.PrivateKey = path_to_private_key_file;
credentials.Passphrase = "ssh_key_password";
options.CredentialsProvider = new LibGit2Sharp.Handlers.CredentialsHandler((url, usernameFromUrl, types) => credentials) ;
return options;
}
public CloneOptions cloneSSHAgent(string username){
CloneOptions options = new CloneOptions();
SshAgentCredentials credentials = new SshAgentCredentials();
credentials.Username = username;
var handler = new LibGit2Sharp.Handlers.CredentialsHandler((url, usernameFromUrl, types) => credentials);
options.CredentialsProvider = handler;
return options;
}
public void CloneRepo(string remotePath, string localPath){
CloneOptions options = cloningSSHAuthentication("git", "C:\\folder\\id_rsa.pub", "C:\\folder\\id_rsa");
Repository.Clone(remotePath, localPath, options);
}

Using GitLab API to set external issues tracker settings?

I'm using GitLab with an external issue tracker (JIRA), and it works well.
My problem is when I create a new GitLab project (using API), I have to go the GitLab's project settings and manually select the issue tracker I want to use and manually enter the project's id of my external issue tracker.
This screen will be more eloquent:
(source: bayimg.com)
(The two fields I am talking about are "Issue tracker" and "Project name or id in issues tracker")
So here is my question: is there any way to set up this two fields automatically, using API or other ? Currently, GitLab API does not mention anything about external issues tracker settings.
This code helped me to automatically set the GitLab's external issues-tracker settings, using Apache HttpClient and Jsoup.
This code is absolutely not 100% good, but it shows the main idea, wich is to recreate the corresponding POST request that the web form sends.
// 1 - Prepare the HttpClient object :
BasicCookieStore cookieStore = new BasicCookieStore();
LaxRedirectStrategy redirectStrategy = new LaxRedirectStrategy();
CloseableHttpClient httpclient = HttpClients.custom()
.setDefaultCookieStore(cookieStore)
.setRedirectStrategy(redirectStrategy)
.build();
try {
// 2 - Second you need to get the "CSRF Token", from a <meta> tag in the edit page :
HttpUriRequest getCsrfToken = RequestBuilder.get()
.setUri(new URI("http://localhost/_NAMESPACE_/_PROJECT_NAME_/edit"))
.build();
CloseableHttpResponse responseCsrf = httpclient.execute(getCsrfToken);
try {
HttpEntity entity = responseCsrf.getEntity();
Document doc = Jsoup.parse(EntityUtils.toString(entity));
String csrf_token = doc.getElementsByAttributeValue("name", "csrf-token").get(0).attr("content");
// 3 - Fill and submit the "edit" form with new values :
HttpUriRequest updateIssueTracker = RequestBuilder
.post()
.setUri(new URI("http://localhost/_NAMESPACE_/_PROJECT_NAME_"))
.addParameter("authenticity_token", csrf_token)
.addParameter("private_token", "_MY_PRIVATE_TOKEN_")
.addParameter("_method", "patch")
.addParameter("commit", "Save changes")
.addParameter("utf8", "✓")
.addParameter("project[issues_tracker]", "jira")
.addParameter("project[issues_tracker_id]", "_MY_JIRA_PROJECT_NAME_")
.addParameter("project[name]", "...")
...
.build();
CloseableHttpResponse responseSubmit = httpclient.execute(updateIssueTracker, httpContext);
} finally {
responseCsrf.close();
}
} finally {
httpclient.close();
}
Change _NAMESPACE_/_PROJECT_NAME_ to make it corresponds to your project URL, change _MY_PRIVATE_TOKEN_ with your admin account's token, and change _MY_JIRA_PROJECT_NAME_ with ... your jira project's name.

Breeze is not working with some project settings

I have a problem using breeze on a project based on John Papa's HotTowel. I configured breeze like:
var mgr = new breeze.EntityManager('breeze/Breeze');
everything is ok but in the case I change the Project properties Start Action from Current Page to Specific Page: HotTowel/Index and breeze doesn't work properly.
I've checked the requests using firebug. It seems in this case application sends a GET request like this:
http://localhost:53180/HotTowel/Index/breeze/Breeze/Metadata
instead of
http://localhost:53180/breeze/Breeze/Metadata
I've also checked this part of breeze.js which is going to send get request.
The url parameter is set to breeze/Breeze/Metadata in both cases which seems correct.
ctor.prototype.fetchMetadata = function (metadataStore, dataService) {
var serviceName = dataService.serviceName;
var url = dataService.makeUrl("Metadata");
var deferred = Q.defer();
var that = this;
ajaxImpl.ajax({
url: url,
dataType: 'json',...
I've also tried ~/breeze/Breeze but it didn't work as remote service name.
As I'm new to web, probably it's not related to breeze.
The question is why the ajax call (or breeze) depends on how the project activates?
Add a / character to your configuration to execute the request relative to the base directory:
var mgr = new breeze.EntityManager('/breeze/Breeze');
The reason why this happens is because you specified a relative path for the EntityManager and if your url is localhost:53180/HotTowel/Index then the relative url for the EntityManager is localhost:53180/HotTowel/Index + /breeze/Breeze.
To correct the issue, change your EntityManager path to the following:
var mgr = new breeze.EntityManager('breeze/Breeze');

Resources