I've created a public Data Entity in dynamics with the following fields:
I keep getting a bad request response, but I'm not sure why.
I've tried to make a POST request in two ways:
1.
HireAction hireAction = new HireAction() { CompanyName = "DEMF", MovieId = "DEMF-000000014", HireActionStatus = "Created" };
string jsonMessage = JsonConvert.SerializeObject(hireAction);
using (HttpClient client = new HttpClient())
{
HttpRequestMessage requestMessage = new
HttpRequestMessage(HttpMethod.Post, "MyDynamicsEnvironmentName/data/HireActions?cross-company=true");
requestMessage.Content = new StringContent(jsonMessage, Encoding.UTF8, "application/json");
requestMessage.Headers.Add("Authorization", AuthResult.AuthorizationHeader);
HttpResponseMessage response = client.SendAsync(requestMessage).Result;
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
//Logic
}
}
var url = "MyDynamicsEnvironmentName/data/HireActions?cross-company=true";
var req = HttpWebRequest.Create(url);
req.Method = "POST";
req.ContentType = "application/json";
req.Headers["Authorization"] = AuthResult.AuthorizationHeader;
HireAction hireAction = new HireAction() { CompanyName = "DEMF", MovieId = "DEMF-000000014", HireActionId = "12345", HireActionStatus = "Created" };
var jsonSettings = new JsonSerializerSettings
{
DateTimeZoneHandling = DateTimeZoneHandling.Local
};
var postString = "CompanyName='DEMF'" + "&MovieId='DEMF-000000014'" + "&HireActionId=132&HireActionStatus='Created'";
var data = JsonConvert.SerializeObject(postString, jsonSettings);
var bytes = Encoding.Default.GetBytes(postString);
var newStream = req.GetRequestStream();
newStream.Write(bytes, 0, bytes.Length);
newStream.Close();
using (var resp = req.GetResponse())
{
var results = new StreamReader(resp.GetResponseStream()).ReadToEnd();
}
Some keypoints:
-Of course you'd replace MyDynamicsEnvironmentName with the URL for the environment. The URL is correct and verified however, by the fact that GET requests do work
-The Authresult.AuthorizationHeader contains a valid token, also validated by working GET requests
As said before, both of these result in a bad request. Does someone know what is wrong or missing?
Here I have written code for Gmail API to fetch mail with date filter
I am able to fetch MessageId and ThreadId using the First API. On the basis of MessageId, I put that messageId parameter in a List object and I have sent this parameter in foreach loop from List to the next API to fetch email body on basis of messageID. But the process is very slow for fetching messages from Gmail
public async Task<ActionResult> DisplayEmailWithFilter (string fromDate, string toDate) {
Message messageObj = new Message ();
Example exampleObj = new Example ();
List<GmailMessage> gmailMessagesList = new List<GmailMessage> ();
GmailMessage gmailMessage = new GmailMessage ();
var responseData = "";
//dateFilter string parameter Created with Date Values
string dateFilter = "in:Inbox after:" + fromDate + " before:" + toDate;
try {
// calling Gmail API to get MessageID Details by Date Filter
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue (scheme: "Bearer",
parameter : Session["Token"].ToString ());
HttpResponseMessage responseMessage = await client.GetAsync ("https://www.googleapis.com/gmail/v1/users/me/messages?q=" + dateFilter);
if (responseMessage.IsSuccessStatusCode) {
var data = responseMessage.Content;
}
try {
responseData = responseMessage.Content.ReadAsStringAsync ().Result;
//This Json Data Converted into List Object
var msgList = JsonConvert.DeserializeObject<Root1> (responseData);
//loop for Fetching EmailMessageData by MessageID
if (msgList.resultSizeEstimate != 0) {
foreach (var msgItem in msgList.messages) {
messageObj.id = msgItem.id;
//Calling API with MessageID Parameter to fetch Respective Message Data
HttpResponseMessage responseMessageList = await client.GetAsync ("https://www.googleapis.com/gmail/v1/users/userId/messages/id?id=" + messageObj.id.ToString () + "&userId=me&format=full");
if (responseMessageList.IsSuccessStatusCode) {
var dataNew = responseMessageList.Content;
var responseDataNew = responseMessageList.Content.ReadAsStringAsync ().Result;
//Converting json string in Object
exampleObj = JsonConvert.DeserializeObject<Example> (responseDataNew);
gmailMessage.Body = exampleObj.snippet;
//fetching Header Values comparing with string to get Data
for (int i = 1; i < exampleObj.payload.headers.Count; i++) {
if (exampleObj.payload.headers[i].name.ToString () == "Date") {
gmailMessage.RecievedDate = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "Subject") {
gmailMessage.Subject = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "Message-ID") {
gmailMessage.SenderEmailID = exampleObj.payload.headers[i].value;
}
if (exampleObj.payload.headers[i].name.ToString () == "From") {
gmailMessage.SenderName = exampleObj.payload.headers[i].value;
}
}
//Adding This Object Values in GmailMessgage List Object
gmailMessagesList.Add (
new GmailMessage {
Body = exampleObj.snippet,
SenderEmailID = gmailMessage.SenderEmailID,
RecievedDate = gmailMessage.RecievedDate,
SenderName = gmailMessage.SenderName,
Subject = gmailMessage.Subject,
});
}
}
}
} catch (Exception e) {
string errorMgs = e.Message.ToString ();
throw;
}
} catch (Exception e) {
string errorMgs = e.Message.ToString ();
throw;
}
return View (gmailMessagesList);
}
I can fetch Gmail email datewise but it took so much time to fetch. how can I improve my code and performance faster?
The query seems like the most you can do. If you know more information about those emails, like a specific subjects or there always come from the same sender you can try to filter that too, like you would in the Gmail interface.
Other way you would be kind of out of luck. You are limited by the files retrieved from User.messages.list.
If you need to escape from the API limitations maybe trying to retrieve the message other way would be the correct way to go. Considerate creating a small code to retrieve message by the IMAP protocol. Several questions in this topic may help you:
Reading Gmail messages using Python IMAP
Reading Gmail Email in Python
How can I get an email message's text content using Python?
I have an ASP.NET Core 3.0 Web API endpoint that I have set up to allow me to post large audio files. I have followed the following directions from MS docs to set up the endpoint.
https://learn.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-3.0#kestrel-maximum-request-body-size
When an audio file is uploaded to the endpoint, it is streamed to an Azure Blob Storage container.
My code works as expected locally.
When I push it to my production server in Azure App Service on Linux, the code does not work and errors with
Unhandled exception in request pipeline: System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Request body too large.
Per advice from the above article, I have configured incrementally updated Kesterl with the following:
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseKestrel((ctx, options) =>
{
var config = ctx.Configuration;
options.Limits.MaxRequestBodySize = 6000000000;
options.Limits.MinRequestBodyDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.MinResponseDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.RequestHeadersTimeout =
TimeSpan.FromMinutes(2);
}).UseStartup<Startup>();
Also configured FormOptions to accept files up to 6000000000
services.Configure<FormOptions>(options =>
{
options.MultipartBodyLengthLimit = 6000000000;
});
And also set up the API controller with the following attributes, per advice from the article
[HttpPost("audio", Name="UploadAudio")]
[DisableFormValueModelBinding]
[GenerateAntiforgeryTokenCookie]
[RequestSizeLimit(6000000000)]
[RequestFormLimits(MultipartBodyLengthLimit = 6000000000)]
Finally, here is the action itself. This giant block of code is not indicative of how I want the code to be written but I have merged it into one method as part of the debugging exercise.
public async Task<IActionResult> Audio()
{
if (!MultipartRequestHelper.IsMultipartContentType(Request.ContentType))
{
throw new ArgumentException("The media file could not be processed.");
}
string mediaId = string.Empty;
string instructorId = string.Empty;
try
{
// process file first
KeyValueAccumulator formAccumulator = new KeyValueAccumulator();
var streamedFileContent = new byte[0];
var boundary = MultipartRequestHelper.GetBoundary(
MediaTypeHeaderValue.Parse(Request.ContentType),
_defaultFormOptions.MultipartBoundaryLengthLimit
);
var reader = new MultipartReader(boundary, Request.Body);
var section = await reader.ReadNextSectionAsync();
while (section != null)
{
var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(
section.ContentDisposition, out var contentDisposition);
if (hasContentDispositionHeader)
{
if (MultipartRequestHelper
.HasFileContentDisposition(contentDisposition))
{
streamedFileContent =
await FileHelpers.ProcessStreamedFile(section, contentDisposition,
_permittedExtensions, _fileSizeLimit);
}
else if (MultipartRequestHelper
.HasFormDataContentDisposition(contentDisposition))
{
var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name).Value;
var encoding = FileHelpers.GetEncoding(section);
if (encoding == null)
{
return BadRequest($"The request could not be processed: Bad Encoding");
}
using (var streamReader = new StreamReader(
section.Body,
encoding,
detectEncodingFromByteOrderMarks: true,
bufferSize: 1024,
leaveOpen: true))
{
// The value length limit is enforced by
// MultipartBodyLengthLimit
var value = await streamReader.ReadToEndAsync();
if (string.Equals(value, "undefined",
StringComparison.OrdinalIgnoreCase))
{
value = string.Empty;
}
formAccumulator.Append(key, value);
if (formAccumulator.ValueCount >
_defaultFormOptions.ValueCountLimit)
{
return BadRequest($"The request could not be processed: Key Count limit exceeded.");
}
}
}
}
// Drain any remaining section body that hasn't been consumed and
// read the headers for the next section.
section = await reader.ReadNextSectionAsync();
}
var form = formAccumulator;
var file = streamedFileContent;
var results = form.GetResults();
instructorId = results["instructorId"];
string title = results["title"];
string firstName = results["firstName"];
string lastName = results["lastName"];
string durationInMinutes = results["durationInMinutes"];
//mediaId = await AddInstructorAudioMedia(instructorId, firstName, lastName, title, Convert.ToInt32(duration), DateTime.UtcNow, DateTime.UtcNow, file);
string fileExtension = "m4a";
// Generate Container Name - InstructorSpecific
string containerName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{instructorId}";
string contentType = "audio/mp4";
FileType fileType = FileType.audio;
string authorName = $"{firstName} {lastName}";
string authorShortName = $"{firstName[0]}{lastName}";
string description = $"{authorShortName} - {title}";
long duration = (Convert.ToInt32(durationInMinutes) * 60000);
// Generate new filename
string fileName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{Guid.NewGuid()}";
DateTime recordingDate = DateTime.UtcNow;
DateTime uploadDate = DateTime.UtcNow;
long blobSize = long.MinValue;
try
{
// Update file properties in storage
Dictionary<string, string> fileProperties = new Dictionary<string, string>();
fileProperties.Add("ContentType", contentType);
// update file metadata in storage
Dictionary<string, string> metadata = new Dictionary<string, string>();
metadata.Add("author", authorShortName);
metadata.Add("tite", title);
metadata.Add("description", description);
metadata.Add("duration", duration.ToString());
metadata.Add("recordingDate", recordingDate.ToString());
metadata.Add("uploadDate", uploadDate.ToString());
var fileNameWExt = $"{fileName}.{fileExtension}";
var blobContainer = await _cloudStorageService.CreateBlob(containerName, fileNameWExt, "audio");
try
{
MemoryStream fileContent = new MemoryStream(streamedFileContent);
fileContent.Position = 0;
using (fileContent)
{
await blobContainer.UploadFromStreamAsync(fileContent);
}
}
catch (StorageException e)
{
if (e.RequestInformation.HttpStatusCode == 403)
{
return BadRequest(e.Message);
}
else
{
return BadRequest(e.Message);
}
}
try
{
foreach (var key in metadata.Keys.ToList())
{
blobContainer.Metadata.Add(key, metadata[key]);
}
await blobContainer.SetMetadataAsync();
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
blobSize = await StorageUtils.GetBlobSize(blobContainer);
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
Media media = Media.Create(string.Empty, instructorId, authorName, fileName, fileType, fileExtension, recordingDate, uploadDate, ContentDetails.Create(title, description, duration, blobSize, 0, new List<string>()), StateDetails.Create(StatusType.STAGED, DateTime.MinValue, DateTime.UtcNow, DateTime.MaxValue), Manifest.Create(new Dictionary<string, string>()));
// upload to MongoDB
if (media != null)
{
var mapper = new Mapper(_mapperConfiguration);
var dao = mapper.Map<ContentDAO>(media);
try
{
await _db.Content.InsertOneAsync(dao);
}
catch (Exception)
{
mediaId = string.Empty;
}
mediaId = dao.Id.ToString();
}
else
{
// metadata wasn't stored, remove blob
await _cloudStorageService.DeleteBlob(containerName, fileName, "audio");
return BadRequest($"An issue occurred during media upload: rolling back storage change");
}
if (string.IsNullOrEmpty(mediaId))
{
return BadRequest($"Could not add instructor media");
}
}
catch (Exception ex)
{
return BadRequest(ex.Message);
}
var result = new { MediaId = mediaId, InstructorId = instructorId };
return Ok(result);
}
I reiterate, this all works great locally. I do not run it in IISExpress, I run it as a console app.
I submit large audio files via my SPA app and Postman and it works perfectly.
I am deploying this code to an Azure App Service on Linux (as a Basic B1).
Since the code works in my local development environment, I am at a loss of what my next steps are. I have refactored this code a few times but I suspect that it's environment related.
I cannot find anywhere that mentions that the level of App Service Plan is the culprit so before I go out spending more money I wanted to see if anyone here had encountered this challenge and could provide advice.
UPDATE: I attempted upgrading to a Production App Service Plan to see if there was an undocumented gate for incoming traffic. Upgrading didn't work either.
Thanks in advance.
-A
Currently, as of 11/2019, there is a limitation with the Azure App Service for Linux. It's CORS functionality is enabled by default and cannot be disabled AND it has a file size limitation that doesn't appear to get overridden by any of the published Kestrel configurations. The solution is to move the Web API app to a Azure App Service for Windows and it works as expected.
I am sure there is some way to get around it if you know the magic combination of configurations, server settings, and CLI commands but I need to move on with development.
CommentThreads amount changes by order
Hi I'm trying to fetch all comments of a video. For testing purpose I'm using this video Id U55NGD9Jm7M.
When I order by time I get 1538 comments the last wrote on the 02.05.2015.
If I’m using the relevance I only receive 1353 comment and the last was wrote on the 29.04.2015
This doesn’t seem right to me. I expected to receive the same comments but in a different order and not different comments.
I also tried this on a different video and the results were the same.
My code cut down to minimum
Thank you for your help
public class foo
{
public void bar(string videoId)
{
var allTopLevelComments = new List<CommentThread>();
var searchListResponse = getThread(videoId);
allTopLevelComments.AddRange(searchListResponse.Items);
string nextPage = searchListResponse.NextPageToken;
while (!String.IsNullOrEmpty(nextPage))
{
searchListResponse = getThread(videoId, searchListResponse.NextPageToken);
nextPage = searchListResponse.NextPageToken;
allTopLevelComments.AddRange(searchListResponse.Items);
}
var first = allTopLevelComments.OrderBy(c => c.Snippet.TopLevelComment.Snippet.PublishedAt).First();
}
private CommentThreadListResponse getThread(string videoId, string nextPageToken = "")
{
var youtubeService = new YouTubeService(new BaseClientService.Initializer
{
ApiKey = "my key",
ApplicationName = "my app"
});
var searchListRequest = youtubeService.CommentThreads.List("id, replies, snippet");
searchListRequest.VideoId = videoId;
searchListRequest.MaxResults = 100;
searchListRequest.Order = CommentThreadsResource.ListRequest.OrderEnum.Time;
searchListRequest.TextFormat = CommentThreadsResource.ListRequest.TextFormatEnum.PlainText;
if (!String.IsNullOrEmpty(nextPageToken))
{
searchListRequest.PageToken = nextPageToken;
}
return searchListRequest.Execute();
}
}
How to ensure UploadStringCompletedEventHandler event has been executed successfully ? in following code you can see i am calling function UploadMyPOST with my lastreads parameter having some data. Now you can see i am saving a variable named response into the MyClassXYZ varialbe. in the extreme last you can see there is a event which invoked by the method UploadMyPost() is filling the server response into the response variable. Now here issue is UploadMyPost(lastreads) executes successfully but its invoked event does not executes. Even cursor do not go on that event by which i am not able to fill server response into the response variable. So Anyone know any approach by which i can wait until that event successfully execute and i could able to save server response ?
private async void MyMethod(MyClassXYZ lastreads)
{
await UploadMyPOST(lastreads);
MyClassXYZ serverResponse = response;
if (serverResponse.Book == null)
{
//Do Something.
}
}
private void UploadMyPOST(MyClassXYZ lastreads)
{
apiData = new MyClassXYZApi()
{
AccessToken = thisApp.currentUser.AccessToken,
Book = lastreads.Book,
Page = lastreads.Page,
Device = lastreads.Device
};
//jsondata is my global variable of MyClassXYZ class.
jsondata = Newtonsoft.Json.JsonConvert.SerializeObject(apiData);
MyClassXYZ responsedData = new MyClassXYZ();
Uri lastread_url = new Uri(string.Format("{0}lastread", url_rootPath));
WebClient wc = new WebClient();
wc.Headers["Content-Type"] = "application/json;charset=utf-8";
wc.UploadStringCompleted += new UploadStringCompletedEventHandler(MyUploadStringCompleted);
wc.UploadStringAsync(lastread_url, "POST", jsondata);
}
private void MyUploadStringCompleted(object sender, UploadStringCompletedEventArgs e)
{
try
{
if (e.Error == null)
{
string resutls = e.Result;
DataContractJsonSerializer json = new DataContractJsonSerializer(typeof(MyClassXYZ));
MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(resutls));
response = (MyClassXYZ)json.ReadObject(ms);
}
else
{
string sx = e.Error.ToString();
}
}
catch(Exception exe)
{
}
}
//After Stephen suggession i used the HttpClient so i have written new code with the help of HttpClient. Code is building successfully but at run time cursor goes out from this method to the parent method where from its calling.
private async Task<string> UploadMyPOST(MyClassXYZ lastreads)
{
string value = "";
try
{
apiData = new LastReadAPI()
{
AccessToken = thisApp.currentUser.AccessToken,
Book = lastreads.Book,
Page = lastreads.Page,
Device = lastreads.Device
};
jsondata = Newtonsoft.Json.JsonConvert.SerializeObject(apiData);
LastRead responsedData = new LastRead();
Uri lastread_url = new Uri(string.Format("{0}lastread", url_rootPath));
HttpClient hc = new HttpClient();
//After following line cursor go back to main Method.
var res = await hc.PostAsync(lastread_url, new StringContent(jsondata));
res.EnsureSuccessStatusCode();
Stream content = await res.Content.ReadAsStreamAsync();
return await Task.Run(() => Newtonsoft.Json.JsonConvert.SerializeObject(content));
value = "kd";
}
catch
{ }
return value;
}
I recommend that you use HttpClient or wrap the UploadStringAsync/UploadStringCompleted pair into a Task-based method. Then you can use await like you want to in MyMethod.
Thank you Stephen Clear you leaded me in a right direction and i did POST my request successfully using HttpClient.
HttpClient hc = new HttpClient();
hc.BaseAddress = new Uri(annotation_url.ToString());
HttpRequestMessage req = new HttpRequestMessage(HttpMethod.Post, myUrl);
HttpContent myContent = req.Content = new StringContent(myJsonData, Encoding.UTF8, "application/json");
var response = await hc.PostAsync(myUrl, myContent);
//Following line for pull out the value of content key value which has the actual resposne.
string resutlContetnt = response.Content.ReadAsStringAsync().Result;
DataContractJsonSerializer deserializer_Json = new DataContractJsonSerializer(typeof(MyWrapperClass));
MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(resutlContetnt.ToString()));
AnnotateResponse = deserializer_Json.ReadObject(ms) as Annotation;