Here I have written code for Gmail API to fetch mail with date filter
I am able to fetch MessageId and ThreadId using the First API. On the basis of MessageId, I put that messageId parameter in a List object and I have sent this parameter in foreach loop from List to the next API to fetch email body on basis of messageID. But the process is very slow for fetching messages from Gmail
public async Task<ActionResult> DisplayEmailWithFilter (string fromDate, string toDate) {
Message messageObj = new Message ();
Example exampleObj = new Example ();
List<GmailMessage> gmailMessagesList = new List<GmailMessage> ();
GmailMessage gmailMessage = new GmailMessage ();
var responseData = "";
//dateFilter string parameter Created with Date Values
string dateFilter = "in:Inbox after:" + fromDate + " before:" + toDate;
try {
// calling Gmail API to get MessageID Details by Date Filter
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue (scheme: "Bearer",
parameter : Session["Token"].ToString ());
HttpResponseMessage responseMessage = await client.GetAsync ("https://www.googleapis.com/gmail/v1/users/me/messages?q=" + dateFilter);
if (responseMessage.IsSuccessStatusCode) {
var data = responseMessage.Content;
}
try {
responseData = responseMessage.Content.ReadAsStringAsync ().Result;
//This Json Data Converted into List Object
var msgList = JsonConvert.DeserializeObject<Root1> (responseData);
//loop for Fetching EmailMessageData by MessageID
if (msgList.resultSizeEstimate != 0) {
foreach (var msgItem in msgList.messages) {
messageObj.id = msgItem.id;
//Calling API with MessageID Parameter to fetch Respective Message Data
HttpResponseMessage responseMessageList = await client.GetAsync ("https://www.googleapis.com/gmail/v1/users/userId/messages/id?id=" + messageObj.id.ToString () + "&userId=me&format=full");
if (responseMessageList.IsSuccessStatusCode) {
var dataNew = responseMessageList.Content;
var responseDataNew = responseMessageList.Content.ReadAsStringAsync ().Result;
//Converting json string in Object
exampleObj = JsonConvert.DeserializeObject<Example> (responseDataNew);
gmailMessage.Body = exampleObj.snippet;
//fetching Header Values comparing with string to get Data
for (int i = 1; i < exampleObj.payload.headers.Count; i++) {
if (exampleObj.payload.headers[i].name.ToString () == "Date") {
gmailMessage.RecievedDate = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "Subject") {
gmailMessage.Subject = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "Message-ID") {
gmailMessage.SenderEmailID = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "From") {
gmailMessage.SenderName = exampleObj.payload.headers[i].value;
}
}
//Adding This Object Values in GmailMessgage List Object
gmailMessagesList.Add (
new GmailMessage {
Body = exampleObj.snippet,
SenderEmailID = gmailMessage.SenderEmailID,
RecievedDate = gmailMessage.RecievedDate,
SenderName = gmailMessage.SenderName,
Subject = gmailMessage.Subject,
});
}
}
}
} catch (Exception e) {
string errorMgs = e.Message.ToString ();
throw;
}
} catch (Exception e) {
string errorMgs = e.Message.ToString ();
throw;
}
return View (gmailMessagesList);
}
I can fetch Gmail email datewise but it took so much time to fetch. how can I improve my code and performance faster?
The query seems like the most you can do. If you know more information about those emails, like a specific subjects or there always come from the same sender you can try to filter that too, like you would in the Gmail interface.
Other way you would be kind of out of luck. You are limited by the files retrieved from User.messages.list.
If you need to escape from the API limitations maybe trying to retrieve the message other way would be the correct way to go. Considerate creating a small code to retrieve message by the IMAP protocol. Several questions in this topic may help you:
Reading Gmail messages using Python IMAP
Reading Gmail Email in Python
How can I get an email message's text content using Python?
When I run my web api method using Postman passing in my URL, it works fine - it returns the value of '5' which I expect since the call returns just a single integer. Also at the very bottom I include another method of my web api that I run using Postman and it too works just fine.
http://localhost:56224/api/profileandblog/validatelogin/DemoUser1/DemoUser1Password/169.254.102.60/
However, in the client - an Asp.Net MVC method, when building the URL, it is DROPPING the "/api/profileandblog" part. Note: I'm using "attribute routing" in the web api.
Here is the Asp.Net MVC method to call the web api:
I stop it on this line so I can see the error details: if (result1.IsSuccessStatusCode)
It's INCORRECTLY building the URL as: http://localhost:56224/validatelogin/DemoUser1/DemoUser1Password/169.254.102.60/
It's dropping the: "/api/profileandblog" part that should follow 56224.
So it give's me the Not found.
Why does it drop it? It has the localhost:56224 correct.
[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult SignIn(SignInViewModel signInViewModel)
{
int returnedApiValue = 0;
User returnedApiUser = new User();
DateTime currentDateTime = DateTime.Now;
string hostName = Dns.GetHostName();
string myIpAddress = Dns.GetHostEntry(hostName).AddressList[2].ToString();
try
{
if (!this.IsCaptchaValid("Captcha is not valid"))
{
ViewBag.errormessage = "Error: captcha entered is not valid.";
}
else
{
if (!string.IsNullOrEmpty(signInViewModel.Username) && !string.IsNullOrEmpty(signInViewModel.Password))
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost:56224/api/profileandblog");
string restOfUrl = "/validatelogin/" + signInViewModel.Username + "/" + signInViewModel.Password + "/" + myIpAddress + "/";
// Call the web api to validate the sign in.
// Sends back a -1(failure), -2(validation issue) or the UserId(success) via an OUTPUT parameter.
var responseTask1 = client.GetAsync(restOfUrl);
responseTask1.Wait();
var result1 = responseTask1.Result;
if (result1.IsSuccessStatusCode)
{
var readTask1 = result1.Content.ReadAsAsync<string>();
readTask1.Wait();
returnedApiValue = Convert.ToInt32(readTask1.Result);
if (returnedApiValue == -2)
{
ViewBag.errormessage = "You entered an invalid user name and/or password";
}
else
{
// I have the 'user id'.
// Continue processing...
}
}
else
{
ModelState.AddModelError(string.Empty, "Server error on signing in. 'validatelogin'. Please contact the administrator.");
}
}
}
}
return View(signInViewModel);
}
catch (Exception)
{
throw;
}
}
Per the suggestion about not having headers, I used another tutorial (https://www.c-sharpcorner.com/article/consuming-asp-net-web-api-rest-service-in-asp-net-mvc-using-http-client/) and it has the code for defining the headers. But it is coded slightly different - using async Task<> on the method definition. I was not using async in my prior version.
[HttpPost]
[ValidateAntiForgeryToken]
public async Task<ActionResult> SignIn(SignInViewModel signInViewModel)
{
int returnedApiValue = 0;
User returnedApiUser = new User();
DateTime currentDateTime = DateTime.Now;
string hostName = Dns.GetHostName();
string myIpAddress = Dns.GetHostEntry(hostName).AddressList[2].ToString();
try
{
if (!this.IsCaptchaValid("Captcha is not valid"))
{
ViewBag.errormessage = "Error: captcha entered is not valid.";
}
else
{
if (!string.IsNullOrEmpty(signInViewModel.Username) && !string.IsNullOrEmpty(signInViewModel.Password))
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost:56224/api/profileandblog");
string restOfUrl = "/validatelogin/" + signInViewModel.Username + "/" + signInViewModel.Password + "/" + myIpAddress + "/";
client.DefaultRequestHeaders.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
// Call the web api to validate the sign in.
// Sends back a -1(failure), -2(validation issue) or the UserId(success) via an OUTPUT parameter.
HttpResponseMessage result1 = await client.GetAsync(restOfUrl);
if (result1.IsSuccessStatusCode)
{
var readTask1 = result1.Content.ReadAsAsync<string>();
readTask1.Wait();
returnedApiValue = Convert.ToInt32(readTask1.Result);
if (returnedApiValue == -2)
{
ViewBag.errormessage = "You entered an invalid user name and/or password";
}
else
{
// I have the 'user id'.
// Do other processing....
}
}
else
{
ModelState.AddModelError(string.Empty, "Server error on signing in. 'validatelogin'. Please contact the administrator.");
}
}
}
}
return View(signInViewModel);
}
catch (Exception)
{
throw;
}
}
It now has a header but still NOT building the URL properly as it is not including the "/api/profileandblog" part.
Here is the web api and the method being called:
namespace GbngWebApi2.Controllers
{
[RoutePrefix("api/profileandblog")]
public class WebApi2Controller : ApiController
{
[HttpGet]
[Route("validatelogin/{userName}/{userPassword}/{ipAddress}/")]
public IHttpActionResult ValidateLogin(string userName, string userPassword, string ipAddress)
{
try
{
IHttpActionResult httpActionResult;
HttpResponseMessage httpResponseMessage;
int returnValue = 0;
// Will either be a valid 'user id" or a -2 indicating a validation issue.
returnValue = dataaccesslayer.ValidateLogin(userName, userPassword, ipAddress);
httpResponseMessage = Request.CreateResponse(HttpStatusCode.OK, returnValue);
httpActionResult = ResponseMessage(httpResponseMessage);
return httpActionResult;
}
catch (Exception)
{
throw;
}
}
}
}
Here's the network tab of the client browser before I hit the button to fire of the Asp.Net MVC method.
The network tab of the client browser after I hit the button to fire of the Asp.Net MVC method and it fails.
Here's another example of Postman executing another method of my api just fine.
I got it to work by setting this as: client.BaseAddress = new Uri("localhost:56224"); and setting the string restOfUrl = "/api/profileandblog/validatesignin/" + signInViewModel.Username + "/" + signInViewModel.Password + "/" + myIpAddress + "/";
I have an ASP.NET Core 3.0 Web API endpoint that I have set up to allow me to post large audio files. I have followed the following directions from MS docs to set up the endpoint.
https://learn.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-3.0#kestrel-maximum-request-body-size
When an audio file is uploaded to the endpoint, it is streamed to an Azure Blob Storage container.
My code works as expected locally.
When I push it to my production server in Azure App Service on Linux, the code does not work and errors with
Unhandled exception in request pipeline: System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Request body too large.
Per advice from the above article, I have configured incrementally updated Kesterl with the following:
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseKestrel((ctx, options) =>
{
var config = ctx.Configuration;
options.Limits.MaxRequestBodySize = 6000000000;
options.Limits.MinRequestBodyDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.MinResponseDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.RequestHeadersTimeout =
TimeSpan.FromMinutes(2);
}).UseStartup<Startup>();
Also configured FormOptions to accept files up to 6000000000
services.Configure<FormOptions>(options =>
{
options.MultipartBodyLengthLimit = 6000000000;
});
And also set up the API controller with the following attributes, per advice from the article
[HttpPost("audio", Name="UploadAudio")]
[DisableFormValueModelBinding]
[GenerateAntiforgeryTokenCookie]
[RequestSizeLimit(6000000000)]
[RequestFormLimits(MultipartBodyLengthLimit = 6000000000)]
Finally, here is the action itself. This giant block of code is not indicative of how I want the code to be written but I have merged it into one method as part of the debugging exercise.
public async Task<IActionResult> Audio()
{
if (!MultipartRequestHelper.IsMultipartContentType(Request.ContentType))
{
throw new ArgumentException("The media file could not be processed.");
}
string mediaId = string.Empty;
string instructorId = string.Empty;
try
{
// process file first
KeyValueAccumulator formAccumulator = new KeyValueAccumulator();
var streamedFileContent = new byte[0];
var boundary = MultipartRequestHelper.GetBoundary(
MediaTypeHeaderValue.Parse(Request.ContentType),
_defaultFormOptions.MultipartBoundaryLengthLimit
);
var reader = new MultipartReader(boundary, Request.Body);
var section = await reader.ReadNextSectionAsync();
while (section != null)
{
var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(
section.ContentDisposition, out var contentDisposition);
if (hasContentDispositionHeader)
{
if (MultipartRequestHelper
.HasFileContentDisposition(contentDisposition))
{
streamedFileContent =
await FileHelpers.ProcessStreamedFile(section, contentDisposition,
_permittedExtensions, _fileSizeLimit);
}
else if (MultipartRequestHelper
.HasFormDataContentDisposition(contentDisposition))
{
var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name).Value;
var encoding = FileHelpers.GetEncoding(section);
if (encoding == null)
{
return BadRequest($"The request could not be processed: Bad Encoding");
}
using (var streamReader = new StreamReader(
section.Body,
encoding,
detectEncodingFromByteOrderMarks: true,
bufferSize: 1024,
leaveOpen: true))
{
// The value length limit is enforced by
// MultipartBodyLengthLimit
var value = await streamReader.ReadToEndAsync();
if (string.Equals(value, "undefined",
StringComparison.OrdinalIgnoreCase))
{
value = string.Empty;
}
formAccumulator.Append(key, value);
if (formAccumulator.ValueCount >
_defaultFormOptions.ValueCountLimit)
{
return BadRequest($"The request could not be processed: Key Count limit exceeded.");
}
}
}
}
// Drain any remaining section body that hasn't been consumed and
// read the headers for the next section.
section = await reader.ReadNextSectionAsync();
}
var form = formAccumulator;
var file = streamedFileContent;
var results = form.GetResults();
instructorId = results["instructorId"];
string title = results["title"];
string firstName = results["firstName"];
string lastName = results["lastName"];
string durationInMinutes = results["durationInMinutes"];
//mediaId = await AddInstructorAudioMedia(instructorId, firstName, lastName, title, Convert.ToInt32(duration), DateTime.UtcNow, DateTime.UtcNow, file);
string fileExtension = "m4a";
// Generate Container Name - InstructorSpecific
string containerName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{instructorId}";
string contentType = "audio/mp4";
FileType fileType = FileType.audio;
string authorName = $"{firstName} {lastName}";
string authorShortName = $"{firstName[0]}{lastName}";
string description = $"{authorShortName} - {title}";
long duration = (Convert.ToInt32(durationInMinutes) * 60000);
// Generate new filename
string fileName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{Guid.NewGuid()}";
DateTime recordingDate = DateTime.UtcNow;
DateTime uploadDate = DateTime.UtcNow;
long blobSize = long.MinValue;
try
{
// Update file properties in storage
Dictionary<string, string> fileProperties = new Dictionary<string, string>();
fileProperties.Add("ContentType", contentType);
// update file metadata in storage
Dictionary<string, string> metadata = new Dictionary<string, string>();
metadata.Add("author", authorShortName);
metadata.Add("tite", title);
metadata.Add("description", description);
metadata.Add("duration", duration.ToString());
metadata.Add("recordingDate", recordingDate.ToString());
metadata.Add("uploadDate", uploadDate.ToString());
var fileNameWExt = $"{fileName}.{fileExtension}";
var blobContainer = await _cloudStorageService.CreateBlob(containerName, fileNameWExt, "audio");
try
{
MemoryStream fileContent = new MemoryStream(streamedFileContent);
fileContent.Position = 0;
using (fileContent)
{
await blobContainer.UploadFromStreamAsync(fileContent);
}
}
catch (StorageException e)
{
if (e.RequestInformation.HttpStatusCode == 403)
{
return BadRequest(e.Message);
}
else
{
return BadRequest(e.Message);
}
}
try
{
foreach (var key in metadata.Keys.ToList())
{
blobContainer.Metadata.Add(key, metadata[key]);
}
await blobContainer.SetMetadataAsync();
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
blobSize = await StorageUtils.GetBlobSize(blobContainer);
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
Media media = Media.Create(string.Empty, instructorId, authorName, fileName, fileType, fileExtension, recordingDate, uploadDate, ContentDetails.Create(title, description, duration, blobSize, 0, new List<string>()), StateDetails.Create(StatusType.STAGED, DateTime.MinValue, DateTime.UtcNow, DateTime.MaxValue), Manifest.Create(new Dictionary<string, string>()));
// upload to MongoDB
if (media != null)
{
var mapper = new Mapper(_mapperConfiguration);
var dao = mapper.Map<ContentDAO>(media);
try
{
await _db.Content.InsertOneAsync(dao);
}
catch (Exception)
{
mediaId = string.Empty;
}
mediaId = dao.Id.ToString();
}
else
{
// metadata wasn't stored, remove blob
await _cloudStorageService.DeleteBlob(containerName, fileName, "audio");
return BadRequest($"An issue occurred during media upload: rolling back storage change");
}
if (string.IsNullOrEmpty(mediaId))
{
return BadRequest($"Could not add instructor media");
}
}
catch (Exception ex)
{
return BadRequest(ex.Message);
}
var result = new { MediaId = mediaId, InstructorId = instructorId };
return Ok(result);
}
I reiterate, this all works great locally. I do not run it in IISExpress, I run it as a console app.
I submit large audio files via my SPA app and Postman and it works perfectly.
I am deploying this code to an Azure App Service on Linux (as a Basic B1).
Since the code works in my local development environment, I am at a loss of what my next steps are. I have refactored this code a few times but I suspect that it's environment related.
I cannot find anywhere that mentions that the level of App Service Plan is the culprit so before I go out spending more money I wanted to see if anyone here had encountered this challenge and could provide advice.
UPDATE: I attempted upgrading to a Production App Service Plan to see if there was an undocumented gate for incoming traffic. Upgrading didn't work either.
Thanks in advance.
-A
Currently, as of 11/2019, there is a limitation with the Azure App Service for Linux. It's CORS functionality is enabled by default and cannot be disabled AND it has a file size limitation that doesn't appear to get overridden by any of the published Kestrel configurations. The solution is to move the Web API app to a Azure App Service for Windows and it works as expected.
I am sure there is some way to get around it if you know the magic combination of configurations, server settings, and CLI commands but I need to move on with development.
have the following which works on Win10 phone in a pcl.
But i cannot get the same code to return OK on samsung s7 with android 7.0
project is xamarin forms.
nuget for system.net.http is 2.2.29.
I've include the same nuget in my UWP for the win10 phone and android projects.
i've also changed the user to include be "domain\user", "domain#user", "user#domain"
var httpClientHandler = new System.Net.Http.HttpClientHandler()
{
Credentials = credentials.GetCredential(new Uri(location), "NTLM")
};
I've tried and alternative to setting the httpClientHandler.Credentials.
var credentials = new NetworkCredentials("user", "pass", "domain");
var location = "http://apps.mysite.com/api#/doit";
var httpClientHandler = new HttpClientHandler{
Credentials = credentials
}
using (var httpClient = new HttpClient(httpClientHandler, true))
{
httpClient.DefaultRequestHeaders.Accept.Clear();
httpClient.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
httpClient.DefaultRequestHeaders.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f");
try
{
var httpResponseMessage = await httpClient.GetAsync(location);
if (httpResponseMessage.StatusCode != System.Net.HttpStatusCode.OK)
{
//handle error
}
else
{
//do something
}
}
catch (Exception)
{}
finally
{}
}
another strange thing. when i run this on android, the code hits the await httpclient.getasync(location);
and the immediately jumps to the finally.
I hava a simple form with username & password Entry Fields, plus an OK button.
all three controls are bound to a viewmodel. the OK button via an ICommand.
this code and the view live in the PCL. which has a reference to Microsoft.Net.Http.
I have Android and Universal Windows Xamarin forms builds that consume the PCL.
Android Properties. Default httpClient, SSL/TLS Default. supported arch armeabi, armeabi-v7a;x86
Android Manifest: Camera, flashlight and internet
private bool calcEnabled = false;
private ICommand okCommand;
private string message = string.Empty;
private string validatingMessage = "Validating!";
private string unauthorizedMessage = "Invalid Credentials!";
private string authenticatedMessage = "Validated";
private bool validating = false;
public ICommand OkCommand => okCommand ?? (okCommand = new Command<object>((o) => clicked(o), (o) => CalcEnabled));
protected async void clicked(object state)
{
try
{
Validating = true;
Message = validatingMessage;
var credentials = new
System.Net.NetworkCredential(Helpers.Settings.UserName, Helpers.Settings.Password, "www.domain.com");
var location = "http://apps.wwwoodproducts.com/wwlocator#/information";
var httpClientHandler = new System.Net.Http.HttpClientHandler()
{
Credentials = credentials.GetCredential(new Uri(location), "NTLM") };
using (var httpClient = new System.Net.Http.HttpClient(httpClientHandler))
{
httpClient.DefaultRequestHeaders.Accept.Clear();
httpClient.DefaultRequestHeaders.Accept.Add(new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue("application/json"));
httpClient.DefaultRequestHeaders.Add("X-FORMS_BASED_AUTH_ACCEPTED", "f");
try
{
var httpResponseMessage = await
httpClient.GetAsync(location);
if (httpResponseMessage.StatusCode != System.Net.HttpStatusCode.OK)
{
Message = unauthorizedMessage;
}
else
{
Message = authenticatedMessage;
Messenger.Default.Send<bool>(true);
}
}
catch (Exception)
{
Message = unauthorizedMessage;
}
finally
{
Validating = false;
}
}
}
catch (Exception)
{
throw;
}
}
I am using very simple code to post message in twitter. The code is as given below:
public void UpdateStatus(string username, string password, string tweetMsg)
{
byte[] bytes = System.Text.Encoding.ASCII.GetBytes("status=" + tweetMsg);
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://twitter.com/statuses/update.xml");
request.Credentials = new NetworkCredential(username, password);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ServicePoint.Expect100Continue = false;
string statusUpdate = string.Empty;
using (StreamWriter sw = new StreamWriter(request.GetRequestStream()))
{
statusUpdate = "status=" + System.Web.HttpUtility.UrlEncode(tweetMsg);
sw.Write(statusUpdate);
}
using (HttpWebResponse resp = (HttpWebResponse) request.GetResponse())
{
// Handle response here
using (StreamReader sr = new StreamReader(resp.GetResponseStream()))
{
Console.WriteLine(sr.ReadToEnd());
}
}
}
catch (WebException ex)
{
Console.WriteLine("Web Error: {0}", ex.Message);
if (ex.Response != null)
{
HttpWebResponse r = (HttpWebResponse)ex.Response;
using (StreamReader esr = new StreamReader(r.GetResponseStream()))
{
Console.WriteLine("Error: {0}", esr.ReadToEnd());
}
}
}
catch (IOException ex)
{
Console.WriteLine("I/O Error: {0}", ex.Message);
}
}
I am not sure why this code is not working. I am getting the exception
The remote server returned an error: (401) Unauthorized.
But, whichever site I am referring, everywhere I find that people are able to post their messages using the same code.
Can anybody please tell me what I am missing in this code? The site that I referred for this code is http://www.dreamincode.net/forums/topic/120468-twitter-api-posting-status/
Thanks in advance....
Hari
The Twitter API no longer supports Basic Auth for authentication. You need to switch to OAuth which uses tokens instead of passwords.
Twitter's guide to transitioning from Basic Auth to OAuth
OAuth homepage