Here I have written code for Gmail API to fetch mail with date filter
I am able to fetch MessageId and ThreadId using the First API. On the basis of MessageId, I put that messageId parameter in a List object and I have sent this parameter in foreach loop from List to the next API to fetch email body on basis of messageID. But the process is very slow for fetching messages from Gmail
public async Task<ActionResult> DisplayEmailWithFilter (string fromDate, string toDate) {
Message messageObj = new Message ();
Example exampleObj = new Example ();
List<GmailMessage> gmailMessagesList = new List<GmailMessage> ();
GmailMessage gmailMessage = new GmailMessage ();
var responseData = "";
//dateFilter string parameter Created with Date Values
string dateFilter = "in:Inbox after:" + fromDate + " before:" + toDate;
try {
// calling Gmail API to get MessageID Details by Date Filter
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue (scheme: "Bearer",
parameter : Session["Token"].ToString ());
HttpResponseMessage responseMessage = await client.GetAsync ("https://www.googleapis.com/gmail/v1/users/me/messages?q=" + dateFilter);
if (responseMessage.IsSuccessStatusCode) {
var data = responseMessage.Content;
}
try {
responseData = responseMessage.Content.ReadAsStringAsync ().Result;
//This Json Data Converted into List Object
var msgList = JsonConvert.DeserializeObject<Root1> (responseData);
//loop for Fetching EmailMessageData by MessageID
if (msgList.resultSizeEstimate != 0) {
foreach (var msgItem in msgList.messages) {
messageObj.id = msgItem.id;
//Calling API with MessageID Parameter to fetch Respective Message Data
HttpResponseMessage responseMessageList = await client.GetAsync ("https://www.googleapis.com/gmail/v1/users/userId/messages/id?id=" + messageObj.id.ToString () + "&userId=me&format=full");
if (responseMessageList.IsSuccessStatusCode) {
var dataNew = responseMessageList.Content;
var responseDataNew = responseMessageList.Content.ReadAsStringAsync ().Result;
//Converting json string in Object
exampleObj = JsonConvert.DeserializeObject<Example> (responseDataNew);
gmailMessage.Body = exampleObj.snippet;
//fetching Header Values comparing with string to get Data
for (int i = 1; i < exampleObj.payload.headers.Count; i++) {
if (exampleObj.payload.headers[i].name.ToString () == "Date") {
gmailMessage.RecievedDate = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "Subject") {
gmailMessage.Subject = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "Message-ID") {
gmailMessage.SenderEmailID = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "From") {
gmailMessage.SenderName = exampleObj.payload.headers[i].value;
}
}
//Adding This Object Values in GmailMessgage List Object
gmailMessagesList.Add (
new GmailMessage {
Body = exampleObj.snippet,
SenderEmailID = gmailMessage.SenderEmailID,
RecievedDate = gmailMessage.RecievedDate,
SenderName = gmailMessage.SenderName,
Subject = gmailMessage.Subject,
});
}
}
}
} catch (Exception e) {
string errorMgs = e.Message.ToString ();
throw;
}
} catch (Exception e) {
string errorMgs = e.Message.ToString ();
throw;
}
return View (gmailMessagesList);
}
I can fetch Gmail email datewise but it took so much time to fetch. how can I improve my code and performance faster?
The query seems like the most you can do. If you know more information about those emails, like a specific subjects or there always come from the same sender you can try to filter that too, like you would in the Gmail interface.
Other way you would be kind of out of luck. You are limited by the files retrieved from User.messages.list.
If you need to escape from the API limitations maybe trying to retrieve the message other way would be the correct way to go. Considerate creating a small code to retrieve message by the IMAP protocol. Several questions in this topic may help you:
Reading Gmail messages using Python IMAP
Reading Gmail Email in Python
How can I get an email message's text content using Python?
I have an ASP.NET Core 3.0 Web API endpoint that I have set up to allow me to post large audio files. I have followed the following directions from MS docs to set up the endpoint.
https://learn.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-3.0#kestrel-maximum-request-body-size
When an audio file is uploaded to the endpoint, it is streamed to an Azure Blob Storage container.
My code works as expected locally.
When I push it to my production server in Azure App Service on Linux, the code does not work and errors with
Unhandled exception in request pipeline: System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Request body too large.
Per advice from the above article, I have configured incrementally updated Kesterl with the following:
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseKestrel((ctx, options) =>
{
var config = ctx.Configuration;
options.Limits.MaxRequestBodySize = 6000000000;
options.Limits.MinRequestBodyDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.MinResponseDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.RequestHeadersTimeout =
TimeSpan.FromMinutes(2);
}).UseStartup<Startup>();
Also configured FormOptions to accept files up to 6000000000
services.Configure<FormOptions>(options =>
{
options.MultipartBodyLengthLimit = 6000000000;
});
And also set up the API controller with the following attributes, per advice from the article
[HttpPost("audio", Name="UploadAudio")]
[DisableFormValueModelBinding]
[GenerateAntiforgeryTokenCookie]
[RequestSizeLimit(6000000000)]
[RequestFormLimits(MultipartBodyLengthLimit = 6000000000)]
Finally, here is the action itself. This giant block of code is not indicative of how I want the code to be written but I have merged it into one method as part of the debugging exercise.
public async Task<IActionResult> Audio()
{
if (!MultipartRequestHelper.IsMultipartContentType(Request.ContentType))
{
throw new ArgumentException("The media file could not be processed.");
}
string mediaId = string.Empty;
string instructorId = string.Empty;
try
{
// process file first
KeyValueAccumulator formAccumulator = new KeyValueAccumulator();
var streamedFileContent = new byte[0];
var boundary = MultipartRequestHelper.GetBoundary(
MediaTypeHeaderValue.Parse(Request.ContentType),
_defaultFormOptions.MultipartBoundaryLengthLimit
);
var reader = new MultipartReader(boundary, Request.Body);
var section = await reader.ReadNextSectionAsync();
while (section != null)
{
var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(
section.ContentDisposition, out var contentDisposition);
if (hasContentDispositionHeader)
{
if (MultipartRequestHelper
.HasFileContentDisposition(contentDisposition))
{
streamedFileContent =
await FileHelpers.ProcessStreamedFile(section, contentDisposition,
_permittedExtensions, _fileSizeLimit);
}
else if (MultipartRequestHelper
.HasFormDataContentDisposition(contentDisposition))
{
var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name).Value;
var encoding = FileHelpers.GetEncoding(section);
if (encoding == null)
{
return BadRequest($"The request could not be processed: Bad Encoding");
}
using (var streamReader = new StreamReader(
section.Body,
encoding,
detectEncodingFromByteOrderMarks: true,
bufferSize: 1024,
leaveOpen: true))
{
// The value length limit is enforced by
// MultipartBodyLengthLimit
var value = await streamReader.ReadToEndAsync();
if (string.Equals(value, "undefined",
StringComparison.OrdinalIgnoreCase))
{
value = string.Empty;
}
formAccumulator.Append(key, value);
if (formAccumulator.ValueCount >
_defaultFormOptions.ValueCountLimit)
{
return BadRequest($"The request could not be processed: Key Count limit exceeded.");
}
}
}
}
// Drain any remaining section body that hasn't been consumed and
// read the headers for the next section.
section = await reader.ReadNextSectionAsync();
}
var form = formAccumulator;
var file = streamedFileContent;
var results = form.GetResults();
instructorId = results["instructorId"];
string title = results["title"];
string firstName = results["firstName"];
string lastName = results["lastName"];
string durationInMinutes = results["durationInMinutes"];
//mediaId = await AddInstructorAudioMedia(instructorId, firstName, lastName, title, Convert.ToInt32(duration), DateTime.UtcNow, DateTime.UtcNow, file);
string fileExtension = "m4a";
// Generate Container Name - InstructorSpecific
string containerName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{instructorId}";
string contentType = "audio/mp4";
FileType fileType = FileType.audio;
string authorName = $"{firstName} {lastName}";
string authorShortName = $"{firstName[0]}{lastName}";
string description = $"{authorShortName} - {title}";
long duration = (Convert.ToInt32(durationInMinutes) * 60000);
// Generate new filename
string fileName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{Guid.NewGuid()}";
DateTime recordingDate = DateTime.UtcNow;
DateTime uploadDate = DateTime.UtcNow;
long blobSize = long.MinValue;
try
{
// Update file properties in storage
Dictionary<string, string> fileProperties = new Dictionary<string, string>();
fileProperties.Add("ContentType", contentType);
// update file metadata in storage
Dictionary<string, string> metadata = new Dictionary<string, string>();
metadata.Add("author", authorShortName);
metadata.Add("tite", title);
metadata.Add("description", description);
metadata.Add("duration", duration.ToString());
metadata.Add("recordingDate", recordingDate.ToString());
metadata.Add("uploadDate", uploadDate.ToString());
var fileNameWExt = $"{fileName}.{fileExtension}";
var blobContainer = await _cloudStorageService.CreateBlob(containerName, fileNameWExt, "audio");
try
{
MemoryStream fileContent = new MemoryStream(streamedFileContent);
fileContent.Position = 0;
using (fileContent)
{
await blobContainer.UploadFromStreamAsync(fileContent);
}
}
catch (StorageException e)
{
if (e.RequestInformation.HttpStatusCode == 403)
{
return BadRequest(e.Message);
}
else
{
return BadRequest(e.Message);
}
}
try
{
foreach (var key in metadata.Keys.ToList())
{
blobContainer.Metadata.Add(key, metadata[key]);
}
await blobContainer.SetMetadataAsync();
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
blobSize = await StorageUtils.GetBlobSize(blobContainer);
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
Media media = Media.Create(string.Empty, instructorId, authorName, fileName, fileType, fileExtension, recordingDate, uploadDate, ContentDetails.Create(title, description, duration, blobSize, 0, new List<string>()), StateDetails.Create(StatusType.STAGED, DateTime.MinValue, DateTime.UtcNow, DateTime.MaxValue), Manifest.Create(new Dictionary<string, string>()));
// upload to MongoDB
if (media != null)
{
var mapper = new Mapper(_mapperConfiguration);
var dao = mapper.Map<ContentDAO>(media);
try
{
await _db.Content.InsertOneAsync(dao);
}
catch (Exception)
{
mediaId = string.Empty;
}
mediaId = dao.Id.ToString();
}
else
{
// metadata wasn't stored, remove blob
await _cloudStorageService.DeleteBlob(containerName, fileName, "audio");
return BadRequest($"An issue occurred during media upload: rolling back storage change");
}
if (string.IsNullOrEmpty(mediaId))
{
return BadRequest($"Could not add instructor media");
}
}
catch (Exception ex)
{
return BadRequest(ex.Message);
}
var result = new { MediaId = mediaId, InstructorId = instructorId };
return Ok(result);
}
I reiterate, this all works great locally. I do not run it in IISExpress, I run it as a console app.
I submit large audio files via my SPA app and Postman and it works perfectly.
I am deploying this code to an Azure App Service on Linux (as a Basic B1).
Since the code works in my local development environment, I am at a loss of what my next steps are. I have refactored this code a few times but I suspect that it's environment related.
I cannot find anywhere that mentions that the level of App Service Plan is the culprit so before I go out spending more money I wanted to see if anyone here had encountered this challenge and could provide advice.
UPDATE: I attempted upgrading to a Production App Service Plan to see if there was an undocumented gate for incoming traffic. Upgrading didn't work either.
Thanks in advance.
-A
Currently, as of 11/2019, there is a limitation with the Azure App Service for Linux. It's CORS functionality is enabled by default and cannot be disabled AND it has a file size limitation that doesn't appear to get overridden by any of the published Kestrel configurations. The solution is to move the Web API app to a Azure App Service for Windows and it works as expected.
I am sure there is some way to get around it if you know the magic combination of configurations, server settings, and CLI commands but I need to move on with development.
I am using Xamarin iOS on windows Visual studio to extract metadata from a photo taken by the iphone camera.
private void GetMetaData(NSUrl url)
{
CGImageSource myImageSource;
myImageSource = CGImageSource.FromUrl(url, null);
var ns = new NSDictionary();
var imageProperties = myImageSource.CopyProperties(ns, 0);
var gps = imageProperties.ObjectForKey(CGImageProperties.GPSDictionary) as NSDictionary;
var lat = gps[CGImageProperties.GPSLatitude];
var latref = gps[CGImageProperties.GPSLatitudeRef];
var lon = gps[CGImageProperties.GPSLongitude];
var lonref = gps[CGImageProperties.GPSLongitudeRef];
var loc = String.Format("GPS: {0} {1}, {2} {3}", lat, latref, lon, lonref);
Console.WriteLine(loc);
}
the url being passed into the method is as:- {file:///var/mobile/Media/DCIM/100APPLE/IMG_0006.JPG}
The CGImageSource.FromUrl(url, null) returns null and my app crashes... Can anyone explain to me how I need to fix this?
Edit This is how I am getting the URL for the image.
protected void Handle_FinishedPickingMedia(object sender, UIImagePickerMediaPickedEventArgs e)
{
NSUrl url = null;
try
{
void ImageData(PHAsset asset)
{
if (asset == null) throw new Exception("PHAsset is null");
PHImageManager.DefaultManager.RequestImageData(asset, null, (data, dataUti, orientation, info) =>
{
//Console.WriteLine(data);
Console.WriteLine(info);
url = info.ValueForKey(new NSString("PHImageFileURLKey")) as NSUrl;
// Call method to get MetaData from Image Url //
GetMetaData(url);
});
}
As I said in your last post: Xamarin iOS camera and photos: If you capture photos from camera, the event will not return a ReferenceUrl. But you can get the metadata info with another key:
protected void Handle_FinishedPickingMedia(object sender, UIImagePickerMediaPickedEventArgs e)
{
var metadataInfo = e.Info["UIImagePickerControllerMediaMetadata"]; //or e.MediaMetadata;
}
This will contain some basic information which may be helpful to you. But this NSDictionary doesn't contain the GPS information. Because this picture is created by yourself, you should get the GPS by using CLLocationManager manually:
CLLocationManager manager;
manager = new CLLocationManager();
manager.RequestWhenInUseAuthorization();
manager.LocationsUpdated += (locationSender, e) =>
{
//get current locations
manager.StopUpdatingLocation();
};
manager.StartUpdatingLocation();
Please notice that add the keys in info.plist: NSLocationWhenInUseUsageDescription and NSLocationAlwaysAndWhenInUsageDescription on iOS11, if you want to deploy iOS10 add NSLocationAlwaysUsageDescription.
In this way, there's no need to get metadata from CGImageSource when you pick photos from camera.
In my application i have to integrate API. I am not getting code for how to check whether internet is available or not. How to send JSON Data to server. Please help me out. As in android we call API in AsyncTask class. In blackberry i did not found like this.
Suggest me some link or ideas so that i can integrate code. I am googling. But did not getting result .
What I have tried is here:
JSONObject postData = new JSONObject();
postData.put("userId", "24");
postData.put("messageTime","06:00:00");
postData.put("language", language[lang_Ocf.getSelectedIndex()]);
System.out.println("********json********"+postData);
ConnectionFactory conFactory = new ConnectionFactory();
ConnectionDescriptor conDesc = null;
try
{
conDesc = conFactory.getConnection(url+";deviceside=true");
}
catch(Exception e)
{
System.out.println(e.toString()+":"+e.getMessage());
}
String response = ""; // this variable used for the server response
// if we can get the connection descriptor from ConnectionFactory
if(null != conDesc)
{
try
{
HttpConnection connection = (HttpConnection)conDesc.getConnection();
//set the header property
connection.setRequestMethod(HttpConnection.POST);
connection.setRequestProperty("Content-Length", Integer.toString(postData.length()));
connection.setRequestProperty("Content-Type","application/json");
OutputStream out = connection.openOutputStream();
out.write(postData.get);
out.flush();
out.close();
int responseCode = connection.getResponseCode();
if(responseCode == HttpConnection.HTTP_OK){
InputStream in = connection.openInputStream();
StringBuffer buf = new StringBuffer();
int read = -1;
while((read = in.read())!= -1)
buf.append((char)read);
response = buf.toString();
}
Dialog.inform(response);
connection.close();
} catch(Exception e) {
}
}
return response;
Thanks
I solved this problem
Error:
Error: Cannot run program "jar": CreateProcess error=2, The system cannot find the file specified Packaging project HelaBibleWhereUR failed (took 10.715 seconds)
I simply put jar.exe that is under java bin folder in the jre bin folder.
I am working on an BB app in which I need to maintain a HTTP connection and with a name of image which is stored on server to get the text written in that image document.
I am getting the response in RTF format.
When I directly hit the server on open browser Chrome, I RTF file get downloaded.
Now I needs to perform that programetically,
1) Either convert the bytes which are coming in response in a simple string format so that I can read that.
or
2) Download the file as its happening on the browser manually so that by reading that file I read the information written in the document.
please suggest me how can I read the data from server by hitting any URL?
Currently I am working with this code:
try {
byte []b = send("new_image.JPG");
String s = new String(b, "UTF-8");
System.out.println(s);
} catch (Exception e) {
e.printStackTrace();
}
public byte[] send(String Imagename) throws Exception
{
HttpConnection hc = null;
String imageName = "BasicExp_1345619462234.jpg";
InputStream is = null;
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] res = null;
try
{
hc = (HttpConnection) Connector.open("http://webservice.tvdevphp.com/basisexpdemo/webservices/ocr.php?imgname="+imageName);
hc.setRequestProperty("Content-Type", "multipart/form-data;");
hc.setRequestMethod(HttpConnection.GET);
int ch;
StringBuffer sb= new StringBuffer();
is = hc.openInputStream();
while ((ch = is.read()) != -1)
{
bos.write(ch);
sb.append(ch);
}
System.out.println(sb.toString());
res = bos.toByteArray();
}
catch(Exception e){
e.printStackTrace();
}
finally
{
try
{
if(bos != null)
bos.close();
if(is != null)
is.close();
if(hc != null)
hc.close();
}
catch(Exception e2)
{
e2.printStackTrace();
}
}
return res;
}
The response is like:
{\rtf1\ansi\ansicpg1252\uc1\deflang1033\adeflang1033...................
I can read the data but its not formatted, so that i can read that programetically too.
I have done with this task....
Actually the mistake was on server side.
When they were performing OCR, the format parameter was not corrected that was reason.