Authentication Steps to BigQuery via Windows Service - windows-services

I've created a project in Big Query and within API Access pane, I've created Service Account so that i can make Big Query API accessible through windows application on behalf of user interaction. Since i am new to using and accessing Google API, I want to know the basic steps so that i can access Big Query via Windows Service.

There are some .NET code samples here using web-based OAuth flows:
Google BigQuery with .NET documentation/ samples
For an example from another dev using service accounts in .NET, see this:
Google OAuth2 Service Account Access Token Request gives 'Invalid Request' Response
These manually implement service accounts on top of the .NET library. The .NET library has recently added native support for service accounts, though I haven't yet found an official example.
Here's an unofficial sample working with the Analytics API. It should be directly analogous to using service accounts with BigQuery:
How do I use a Service Account to Access the Google Analytics API V3 with .NET C#?

Finally here's the working code for Authentication process to Big Query API and retrieving records from Big Query table:-
I've created a class having a method of return type OAuth2Authenticator<AssertionFlowClient>.
internal class clsGetOAuth2Authentication
{
public OAuth2Authenticator<AssertionFlowClient> objGetOAuth2(string strPrivateFilePath, string strPrivateFilePassword, string strServiceAccEmailId,string strScope)
{
AuthorizationServerDescription objAuthServerDesc;
X509Certificate2 objKey;
AssertionFlowClient objClient;
OAuth2Authenticator<AssertionFlowClient> objAuth = null;
string ScopeUrl = "https://www.googleapis.com/auth/" + strScope;
string strSrvAccEmailId = strServiceAccEmailId;
string strKeyFile = strPrivateFilePath; //KeyFile: This is the physical path to the key file you downloaded when you created your Service Account.
string strKeyPassword = (strPrivateFilePassword != "") ? strPrivateFilePassword : "notasecret"; //key_pass: This is probably the password for all key files, but if you're given a different one, use that.
objAuthServerDesc = GoogleAuthenticationServer.Description; //objAuthServerDesc: Description of the server that will grant Authentiation.
objKey = new X509Certificate2(strKeyFile, strKeyPassword, X509KeyStorageFlags.Exportable); //objkey: Load up and decrypt the key.
objClient = new AssertionFlowClient(objAuthServerDesc, objKey) { ServiceAccountId = strSrvAccEmailId, Scope = ScopeUrl }; //objClient: Using the AssertionFlowClient, because we're logging in with our certificate.
objAuth = new OAuth2Authenticator<AssertionFlowClient>(objClient, AssertionFlowClient.GetState); //objAuth: Requesting Authentication.
return objAuth;
}
}
Now, another class calls the above method:-
clsGetOAuth2Authentication objOAuth2 = new clsGetOAuth2Authentication();
try
{
var objAuth = objOAuth2.objGetOAuth2(strKeyFile, strKeyPassword, strSrvAccEmailId, strScope); //Authentication data returned
objBQServ = new BigqueryService(objAuth); //Instantiate BigQueryService with credentials(objAuth) as its parameter
#region Retrieving Records:-
JobsResource j = objBQServ.Jobs;
QueryRequest qr = new QueryRequest();
qr.Query = strQuery;
DateTime dtmBegin = DateTime.UtcNow;
QueryResponse response = j.Query(qr, strProjId).Fetch();
DateTime dtmEnd = DateTime.UtcNow;
string strColHead = "";
foreach (var colHeaders in response.Schema.Fields)
{
strColHead += colHeaders.Name.ToString() + "\t";
}
Console.WriteLine(strColHead);
int intCount = 0;
foreach (TableRow row in response.Rows)
{
intCount += 1;
List<string> list = new List<string>();
foreach (var field in row.F)
{
list.Add(field.V);
}
Console.WriteLine(String.Join("\t", list));
}
TimeSpan tsElapsed = dtmEnd - dtmBegin;
Console.WriteLine("\n" + "Total no. of records:- " + intCount + ". Time taken:- " + tsElapsed);
#endregion
}
catch (Exception ex)
{
Console.WriteLine("Error Occured!" + "\n\n" + "Statement:- " + ex.Message.ToString() + "\n\n" + "Description:- " + ex.ToString() + "\n\n");
Console.WriteLine("\nPress enter to exit");
Console.ReadLine();
}

Related

copying google doc file in shared drive using c#

We have been using google drive for a while now to select an existing doc, owned by one of our employees, copy it and then merge data into placeholder fields to then download a pdf version of the doc. It's been working fine except now other employees want access to the created docs. So when we give them a link, the original employee has to give them access. We want to move the docs to a shared drive where all employees can see anything in the shared drive. From what I could find in google on this, it looks like we need to set the SupportsAllDrives property to true on the request. However, I can't find that property on any of the objects that we're creating when copying the base document. Because of this I keep getting a 404 from google when trying to copy the file. Can anyone suggest how to get this working?
var secrets = new ClientSecrets
{
ClientId = GoogleCredentials.accesskey,
ClientSecret = GoogleCredentials.secretkey
};
var refreshToken = _credService.GetRefreshToken();
if (string.IsNullOrEmpty(refreshToken))
{
throw new Exception("Missing google refresh token for google doc processor task.");
}
var token = new TokenResponse { RefreshToken = refreshToken };
var credentials = new UserCredential(new GoogleAuthorizationCodeFlow(
new GoogleAuthorizationCodeFlow.Initializer
{
ClientSecrets = secrets
}),
"user",
token);
var docService = new DocsService(new BaseClientService.Initializer
{
HttpClientInitializer = credentials,
ApplicationName = "Contract Merge"
});
var driveService = new DriveService(new BaseClientService.Initializer
{
HttpClientInitializer = credentials,
ApplicationName = "Contract Merge"
});
var newTitle = "Agreement for " + contract.FirstName + " " + contract.LastName + " " + DateTime.Now.Month.ToString() + "-" + DateTime.Now.Day.ToString() + "-" + DateTime.Now.Year.ToString();
var newFile = new google.Apis.Drive.v2.Data.File { Title = newTitle };
var documentCopyFile = driveService.Files.Copy(newFile, GoogleConstants.TemplateDocId).Execute();
As you can see in the C# library reference, an optional query parameter like supportsAllDrives (see query parameters on the API docs) is handled by a property of the CopyRequest class.
Therefore, after building the CopyRequest, but before executing it, you have to set the property SupportsAllDrives to true, as shown here:
FilesResource.CopyRequest copyRequest = driveService.Files.Copy(newFile, GoogleConstants.TemplateDocId);
copyRequest.SupportsAllDrives = true;
copyRequest.Execute();
Reference:
Class FilesResource.CopyRequest

Project Server In premise 2016 CSOM code with Claim based Authentication

One of our clients has enabled ADFS on his Project server 2016 In premise environment . We are using CSOM operation in our custom application and CSOM operations have failed due to this change.
For ADFS Claims based authentication we need to pass Authenticated cookie . Can someone help us with how to add authenticated Cookie with CSOM code.
Existing CSOM Code for getting list of projects:
public static void GetProjectListInpremise()
{
NetworkCredential net = null;
Console.WriteLine("Read Project Online Started ..");
string PWAOnlineUrl = ConfigurationManager.AppSettings ["pwaInpremiseUrl"];
string userName = ConfigurationManager.AppSettings["pwaInpremiseUser"];
string password = ConfigurationManager.AppSettings["pwaInpremiseUserPwd"];
string domain = ConfigurationManager.AppSettings["pwaInpremiseDomian"];
net = new NetworkCredential(userName, password, domain);
ProjectContext projContext = null;
projContext = new ProjectContext(PWAOnlineUrl);
// projContext.AuthenticationMode = ClientAuthenticationMode.FormsAuthentication;
projContext.Credentials = net;
projContext.Load(projContext.Projects);
projContext.ExecuteQuery();
Console.WriteLine("Read Project Execute Query Successful..");
foreach (PublishedProject pubProj in projContext.Projects)
{
Console.WriteLine("Project Name :" + pubProj.Name);
}
Console.ReadLine();
}

Sample Webapi to trigger TFS rest services & post HTTP calls

I have been working on a requirement, i.e. when a bug is created/inprogress in TFS post a HTTP call to Slack (third party collaboration tool).
When a bug is closed post one more HTTP call to Slack.
I had implemented TFS server side plugin, unfortunately we don't have complete access to TFS and cannot implement. So, planning to implement Webapi and host it (say in Docker container) and whenever bug created / closed event happens in TFS it should post HTTP call.
I have created a simple console app with a method and it's working fine.
any sample code or thoughts to convert it to web api?
if I host, can it monitor TFS events and posts some HTTP calls?
public class GetWI
{
static void Main(string[] args)
{
GetWI ex = new GetWI();
ex.GetWorkItemsByWiql();
}
public void GetWorkItemsByWiql()
{
string _personalAccessToken = "xxxx";
string _credentials = Convert.ToBase64String(System.Text.ASCIIEncoding.ASCII.GetBytes(string.Format("{0}:{1}", "", _personalAccessToken)));
//this is needed because we want to create a project scoped query
string project = "Agileportfolio";
//create wiql object
var wiql = new
{
query = "Select [State], [Title] " +
"From WorkItems " +
"Where [Work Item Type] = 'Bug' " +
"And [System.TeamProject] = '" + project + "' " +
"And [System.State] = 'New' " +
"Order By [State] Asc, [Changed Date] Desc"
};
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("https://test.visualstudio.com");
client.DefaultRequestHeaders.Accept.Clear();
client.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic", _credentials);
//serialize the wiql object into a json string
var postValue = new StringContent(JsonConvert.SerializeObject(wiql), Encoding.UTF8, "application/json"); //mediaType needs to be application/json for a post call
var method = new HttpMethod("POST");
var httpRequestMessage = new HttpRequestMessage(method, "https://abrahamdhanyaraj.visualstudio.com/_apis/wit/wiql?api-version=2.2") { Content = postValue };
var httpResponseMessage = client.SendAsync(httpRequestMessage).Result;
if (httpResponseMessage.IsSuccessStatusCode)
{
WorkItemQueryResult workItemQueryResult = httpResponseMessage.Content.ReadAsAsync<WorkItemQueryResult>().Result;
//now that we have a bunch of work items, build a list of id's so we can get details
var builder = new System.Text.StringBuilder();
foreach (var item in workItemQueryResult.WorkItems)
{
builder.Append(item.Id.ToString()).Append(",");
}
//clean up string of id's
string ids = builder.ToString().TrimEnd(new char[] { ',' });
HttpResponseMessage getWorkItemsHttpResponse = client.GetAsync("_apis/wit/workitems?ids=" + ids + "&fields=System.Id,System.Title,System.State&asOf=" + workItemQueryResult.AsOf + "&api-version=2.2").Result;
if (getWorkItemsHttpResponse.IsSuccessStatusCode)
{
var result = getWorkItemsHttpResponse.Content.ReadAsStringAsync().Result;
//Read title
}
}
// Create Channel
string name = "xyzz3";
var payload = new
{
token = "xoxp-291239704800-292962676087-297314229698-a80e720d98e443c8afb0c4cb2c09e745",
name = "xyzz3",
};
var serializedPayload = JsonConvert.SerializeObject(payload);
var response = client.PostAsync("https://slack.com/api/channels.create" + "?token=test&name=" + name + "&pretty=1",
new StringContent(serializedPayload, Encoding.UTF8, "application/json")).Result;
if (response.IsSuccessStatusCode)
{
dynamic content = JsonConvert.DeserializeObject(
response.Content.ReadAsStringAsync()
.Result);
}
}
}
I use wcf service to listen events from TFS. You may find my project here: https://github.com/ashamrai/tfevents
For wcf service:
Update your ServiceName.svc file and add:
Factory="System.ServiceModel.Activation.WebServiceHostFactory"
Create web method to use json:
[OperationContract]
[WebInvoke(Method = "POST", RequestFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Bare)]
void WorkItemChangedEvent(Stream EventData);
Convert stream with Newtonsoft.Json to get information about event and work item:
StreamReader _reader = new StreamReader(pEventData, Encoding.UTF8);
string _eventStr = _reader.ReadToEnd();
WorkItemEventCore _wieventcorre = JsonConvert.DeserializeObject(_eventStr);
Then you have to create the subscription with url "http://host:port/service.svc/webmethod": https://learn.microsoft.com/en-us/vsts/service-hooks/services/webhooks
Instead of using a query and manually polling Visual Studio Team Services (VSTS), you can use a concept called WebHooks. You configure a WebHook in VSTS to listen for events and send these to a public endpoint. One event type is for Work Items. The endpoint can be any type of public endpoint, for example an Azure Function.
If the only thing you want to do is post the events to Slack, it's even easier because that's a standard integration point: Slack with VSTS.
This is much easier then using a server side plugin or writing your own Web API.

Why do I encounter a 401 when attempting to upload a document to SharePoint Online using CSOM and a token?

Hello Office / SharePoint Developers,
I am working on a project based on the Office Developer Patterns and Practices Sample where a console application accesses a WebAPI which then access SharePoint Online as the logged in user:  The sample is here: https://github.com/SharePoint/PnP/tree/master/Samples/AzureAD.WebAPI.SPOnline
Question:
When I attempt to upload a file to the document library, I get an error 401 "The remote server returned an error: (401) Unauthorized".
The file read options such as listing the documents and querying for documents works fine.
The user credentials I supply are of a user that is the site collection admin, owner, and global admin on the tenant.
I get an access token from SharePoint online based on the token I get in the native client.
public string GetAccessToken(string accessToken)
{
string clientID = _clientId;           
string clientSecret = _clientSecret;
var appCred = new ClientCredential(clientID, clientSecret);
var authContext = new Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext("https://login.windows.net/common");
AuthenticationResult authResult = authContext.AcquireToken(new Uri(_spoUrl).GetLeftPart(UriPartial.Authority), appCred, new UserAssertion(accessToken));
return authResult.AccessToken;
}
This is the CSOM that uploads the file.  I know it works as I can paste it into a console app and using (SharePointOnlineCredentails) it works fine.
string newToken = _tokenSvc.GetAccessToken(accessToken);
using (ClientContext cli = new ClientContext(_spoUrl))
{
cli.ExecutingWebRequest += (s, e) => e.WebRequestExecutor.WebRequest.Headers.Add("Authorization", "Bearer " + newToken);
cli.AuthenticationMode = ClientAuthenticationMode.Default;
using (var fs = new FileStream(#"c:\test.txt", FileMode.Open))
{
var fi = new FileInfo("test.txt");
var list = cli.Web.Lists.GetByTitle("documents");
cli.Load(list.RootFolder);
cli.ExecuteQuery();
var fileUrl = String.Format("{0}/{1}", list.RootFolder.ServerRelativeUrl, fi.Name);
Microsoft.SharePoint.Client.File.SaveBinaryDirect(cli, fileUrl, fs, true);
Web web = cli.Web;
Microsoft.SharePoint.Client.File newFile = web.GetFileByServerRelativeUrl(fileUrl);
cli.Load(newFile);
cli.ExecuteQuery();
ListItem item = newFile.ListItemAllFields;
item["CRUID"] = "CRU_1337";
item.Update();
cli.ExecuteQuery();
}
}...
TLDR:  I get 401 on file upload.  Reads work.  I am using CSOM with an access token that is supposed to be a webAPI on behalf of the logged in user.
I look forward to hearing your advice!
Chris
I am not sure whether we could upload/download files from SP using access tokens with CSOM now , see discussion here two years ago . But we could use sharepoint online rest api to upload files to sharepoint online , i tried below code and it works fine in the code sample AzureAD.WebAPI.SPOnline :
string sharePointUrl = ConfigurationManager.AppSettings["SharePointURL"];
string newToken = GetSharePointAccessToken(sharePointUrl, this.Request.Headers.Authorization.Parameter);
using (ClientContext cli = new ClientContext(sharePointUrl))
{
cli.AuthenticationMode = ClientAuthenticationMode.Default;
cli.ExecutingWebRequest += (s, e) => e.WebRequestExecutor.WebRequest.Headers.Add("Authorization", "Bearer " + newToken);
cli.AuthenticationMode = ClientAuthenticationMode.Default;
byte[] bytefile = System.IO.File.ReadAllBytes(#"e:\log.txt");
HttpWebRequest endpointRequest = (HttpWebRequest)HttpWebRequest.Create("https://xxx.sharepoint.com/sites/xxx/" + "/_api/web/GetFolderByServerRelativeUrl('Shared%20Documents')/Files/add(url='log.txt',overwrite=true)");
endpointRequest.Method = "POST";
endpointRequest.Headers.Add("binaryStringRequestBody", "true");
endpointRequest.Headers.Add("Authorization", "Bearer " + newToken);
endpointRequest.GetRequestStream().Write(bytefile, 0, bytefile.Length);
HttpWebResponse endpointresponse = (HttpWebResponse)endpointRequest.GetResponse();
}
The code below is ended up being the solution to my question:
/* Beginning CSOM Magic */
using (ClientContext cli = new ClientContext(_spoUrl))
{
/* Adding authorization header */
cli.ExecutingWebRequest += (s, e) => e.WebRequestExecutor.WebRequest.Headers.Add("Authorization", "Bearer " + newToken);
cli.AuthenticationMode = ClientAuthenticationMode.Default;
//Get Document List
List documentsList = cli.Web.Lists.GetByTitle(_libraryName);
var fileCreationInformation = new FileCreationInformation();
//Assign to content byte[] i.e. documentStream
var data = System.IO.File.ReadAllBytes(#"c:\test.txt");
fileCreationInformation.Content = data;
//Allow owerwrite of document
fileCreationInformation.Overwrite = true;
//var siteURL = _spoUrl;
var documentListURL = "shared documents";
//var documentName = "/test.txt";
//Upload URL
fileCreationInformation.Url = string.Concat(_spoUrl,"/",documentListURL,"/",documentName);
Microsoft.SharePoint.Client.File uploadFile = documentsList.RootFolder.Files.Add(
fileCreationInformation);
//Update the metadata for a field having name "DocType"
uploadFile.ListItemAllFields["CRUID"] = cruId;
uploadFile.ListItemAllFields.Update();
cli.ExecuteQuery();
}

Mendeley Implicit auth type not returning full data

This question relates to the Mendeley API.
http://dev.mendeley.com/
When using the implicit auth type: http://dev.mendeley.com/reference/topics/authorization_overview.html
I seem to only receive a subset of data for a given document. For example, the 'websites' field seems to not come through even when it is populated.
I am only experiencing this issue using the implicit auth type and not other auth types.
Are any other Mendeley API users experiencing this? It seems like a bug.
Certain fields get returned depending on the document view that you specify. This was implemented to be able to support the needs of multiple clients e.g. mobile clients require smaller datasets than larger web clients
Please read - http://dev.mendeley.com/methods/#document-views
You need to specify 'view=bib' on your endpoint call.
Here is a very crude worked example just using Java
#Test
public void testImplicitGrantFlow() {
String random = RandomStringUtils.random(5);
String query = String.format(
"?client_id=%s&redirect_uri=%s&response_type=token&scope=all&state=%s", IMPLICIT_GRANT_FLOW_CLIENT_ID, "http://localhost:5000/callback", random);
ClientResponse authorise = jerseyClient.resource(AUTH_URL + query)
.accept(MediaType.APPLICATION_JSON)
.get(ClientResponse.class);
assertThat(authorise.getStatus()).isEqualTo(200);
ClientResponse postFormDataResponse = jerseyClient.resource(AUTH_URL + query)
.entity("username=joyce.stack#mendeley.com&password=spuds", MediaType.APPLICATION_FORM_URLENCODED_TYPE)
.accept(MediaType.APPLICATION_JSON)
.post(ClientResponse.class);
assertThat(postFormDataResponse.getStatus()).isEqualTo(302);
String queryString = postFormDataResponse.getHeaders().get("Location").get(0);
Matcher matcher = ACCESS_TOKEN_REGEX.matcher(queryString);
matcher.find();
String accessToken = matcher.group(1);
matcher = STATE_REGEX.matcher(queryString);
matcher.find();
String state = matcher.group(1);
assertNotNull(accessToken);
assertThat(queryString).contains(accessToken);
assertNotNull(state);
assertThat(queryString).contains(state);
ClientResponse response = jerseyClient.resource(OAuthBaseClass.DOCUMENTS_URL)
.header("Authorization", "Bearer " + accessToken)
.get(ClientResponse.class);
assertThat(response.getStatus()).isEqualTo(200);
List<Document> documents = response.getEntity(new GenericType<List<Document>>() {
});
assertThat(documents.size()).isGreaterThan(0);
ListIterator<Document> documentListIterator = documents.listIterator();
while (documentListIterator.hasNext()) {
Document next = documentListIterator.next();
System.out.println(next.getTitle());
System.out.println(next.getWebsites());
}
}

Resources