ASPNET CORE Middleware strange exception - middleware

I've created a middleware delegate in a brand new blazor-server project, to transform my html responses. See code below.
app.Use(async (context, next) =>
{
var originalStream = context.Response.Body;// currently holds the original stream
var newStream = new MemoryStream();
context.Response.Body = newStream;
await next();
string contentType1 = context.Response.ContentType?.Split(';', StringSplitOptions.RemoveEmptyEntries)[0];
string contentType2 = "text/html";
if (contentType1 == "text/html") // EXCEPTION THROWN, SEE IMAGE 1
//if (contentType2 == "text/html") // EXCEPTION NOT THROWN, SEE IMAGE 2
{
newStream.Seek(0, SeekOrigin.Begin);
var originalContent = new StreamReader(newStream).ReadToEnd();
var updatedContent = originalContent.Replace("Hello", "HOLA");
context.Response.Body = originalStream;
await context.Response.WriteAsync(updatedContent);
}
});
But when I compare a string extracted from the Response.ContentType, I'm getting an exception (image 1). But when comparing another string with the same value, I'm not getting it (image 2) In both cases the page is rendered with the replacement. Any help on this one?
Image 1
Image 2

Actually, the Exception occurs when the response content type is NOT "text/html" (of course, in other requests)
The real cause is that I'm assigning the body a new stream but never restoring the originalStream, which is wrong.
But even assigning it afterwards does not work. This is the final solution I've managed to work, please, see the comments en the IF statements.
app.Use(async (context, next) =>
{
var originalStream = context.Response.Body;
var newStream = new MemoryStream();
context.Response.Body = newStream;
await next();
string contentType = context.Response.ContentType?.Split(';', StringSplitOptions.RemoveEmptyEntries)[0];
//if (contentType == "text/html") //THIS IF -at this level- is still throwing the exceptions for NO text/hteml content type.
//{
newStream.Seek(0, SeekOrigin.Begin);
var originalContent = new StreamReader(newStream).ReadToEnd();
if (contentType == "text/html") //THIS IF at these level, makes the replacement if needed, but it does not avoid the "Streams operations"
{
originalContent = originalContent.Replace("Hello", "HOLA");
}
var memoryStreamModified = GenerateStreamFromString(originalContent);
await memoryStreamModified.CopyToAsync(originalStream).ConfigureAwait(false);
//}
context.Response.Body = originalStream;
});

Related

Headers already sent in .net 5

We have middleware in a web API, which we use to filter the resposne body from a controller
After updating our service to .net 5, replacing the output fails with
System.InvalidOperationException: Headers already sent.
at Microsoft.AspNetCore.Server.HttpSys.Response.CheckResponseStarted()
at Microsoft.AspNetCore.Server.HttpSys.FeatureContext.ConsiderEnablingResponseCache()
at Microsoft.AspNetCore.Server.HttpSys.FeatureContext.OnResponseStart()
at Microsoft.AspNetCore.Server.HttpSys.FeatureContext.CompleteAsync()
at Microsoft.AspNetCore.Server.HttpSys.RequestContext.Execute()
at Microsoft.AspNetCore.Server.HttpSys.RequestContext.Execute()
Our middleware to filter the output looks something like this
internal class FilterOutput : IMiddleware
{
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
var originalBodyStream = context.Response.Body;
var tempResponseBody = new MemoryStream();
context.Response.Body = tempResponseBody;
context.Response.OnStarting(async state =>
{
await FilterResponse(context, tempResponseBody, originalBodyStream);
}, context);
await next(context);
}
private async Task FilterResponse(HttpContext context, MemoryStream tempResponseBody, Stream originalBodyStream)
{
if (context.Response.StatusCode == 200)
{
var output = Encoding.UTF8.GetString(tempResponseBody.GetBuffer());
var newOutput = output.Filter(null);
var updatedStream = GenerateStreamFromString(newOutput);
await updatedStream.CopyToAsync(originalBodyStream);
context.Response.Body = originalBodyStream;
return;
}
await tempResponseBody.CopyToAsync(originalBodyStream);
}
public static Stream GenerateStreamFromString(string s)
{
var stream = new MemoryStream();
var writer = new StreamWriter(stream);
writer.Write(s);
writer.Flush();
stream.Position = 0;
return stream;
}
}
I realize net 5 is propably more asynchronous and sends headers earlier
Is there a way to stop that, so I can modify the output body in middleware?

Large File upload to ASP.NET Core 3.0 Web API fails due to Request Body to Large

I have an ASP.NET Core 3.0 Web API endpoint that I have set up to allow me to post large audio files. I have followed the following directions from MS docs to set up the endpoint.
https://learn.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-3.0#kestrel-maximum-request-body-size
When an audio file is uploaded to the endpoint, it is streamed to an Azure Blob Storage container.
My code works as expected locally.
When I push it to my production server in Azure App Service on Linux, the code does not work and errors with
Unhandled exception in request pipeline: System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Request body too large.
Per advice from the above article, I have configured incrementally updated Kesterl with the following:
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseKestrel((ctx, options) =>
{
var config = ctx.Configuration;
options.Limits.MaxRequestBodySize = 6000000000;
options.Limits.MinRequestBodyDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.MinResponseDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.RequestHeadersTimeout =
TimeSpan.FromMinutes(2);
}).UseStartup<Startup>();
Also configured FormOptions to accept files up to 6000000000
services.Configure<FormOptions>(options =>
{
options.MultipartBodyLengthLimit = 6000000000;
});
And also set up the API controller with the following attributes, per advice from the article
[HttpPost("audio", Name="UploadAudio")]
[DisableFormValueModelBinding]
[GenerateAntiforgeryTokenCookie]
[RequestSizeLimit(6000000000)]
[RequestFormLimits(MultipartBodyLengthLimit = 6000000000)]
Finally, here is the action itself. This giant block of code is not indicative of how I want the code to be written but I have merged it into one method as part of the debugging exercise.
public async Task<IActionResult> Audio()
{
if (!MultipartRequestHelper.IsMultipartContentType(Request.ContentType))
{
throw new ArgumentException("The media file could not be processed.");
}
string mediaId = string.Empty;
string instructorId = string.Empty;
try
{
// process file first
KeyValueAccumulator formAccumulator = new KeyValueAccumulator();
var streamedFileContent = new byte[0];
var boundary = MultipartRequestHelper.GetBoundary(
MediaTypeHeaderValue.Parse(Request.ContentType),
_defaultFormOptions.MultipartBoundaryLengthLimit
);
var reader = new MultipartReader(boundary, Request.Body);
var section = await reader.ReadNextSectionAsync();
while (section != null)
{
var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(
section.ContentDisposition, out var contentDisposition);
if (hasContentDispositionHeader)
{
if (MultipartRequestHelper
.HasFileContentDisposition(contentDisposition))
{
streamedFileContent =
await FileHelpers.ProcessStreamedFile(section, contentDisposition,
_permittedExtensions, _fileSizeLimit);
}
else if (MultipartRequestHelper
.HasFormDataContentDisposition(contentDisposition))
{
var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name).Value;
var encoding = FileHelpers.GetEncoding(section);
if (encoding == null)
{
return BadRequest($"The request could not be processed: Bad Encoding");
}
using (var streamReader = new StreamReader(
section.Body,
encoding,
detectEncodingFromByteOrderMarks: true,
bufferSize: 1024,
leaveOpen: true))
{
// The value length limit is enforced by
// MultipartBodyLengthLimit
var value = await streamReader.ReadToEndAsync();
if (string.Equals(value, "undefined",
StringComparison.OrdinalIgnoreCase))
{
value = string.Empty;
}
formAccumulator.Append(key, value);
if (formAccumulator.ValueCount >
_defaultFormOptions.ValueCountLimit)
{
return BadRequest($"The request could not be processed: Key Count limit exceeded.");
}
}
}
}
// Drain any remaining section body that hasn't been consumed and
// read the headers for the next section.
section = await reader.ReadNextSectionAsync();
}
var form = formAccumulator;
var file = streamedFileContent;
var results = form.GetResults();
instructorId = results["instructorId"];
string title = results["title"];
string firstName = results["firstName"];
string lastName = results["lastName"];
string durationInMinutes = results["durationInMinutes"];
//mediaId = await AddInstructorAudioMedia(instructorId, firstName, lastName, title, Convert.ToInt32(duration), DateTime.UtcNow, DateTime.UtcNow, file);
string fileExtension = "m4a";
// Generate Container Name - InstructorSpecific
string containerName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{instructorId}";
string contentType = "audio/mp4";
FileType fileType = FileType.audio;
string authorName = $"{firstName} {lastName}";
string authorShortName = $"{firstName[0]}{lastName}";
string description = $"{authorShortName} - {title}";
long duration = (Convert.ToInt32(durationInMinutes) * 60000);
// Generate new filename
string fileName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{Guid.NewGuid()}";
DateTime recordingDate = DateTime.UtcNow;
DateTime uploadDate = DateTime.UtcNow;
long blobSize = long.MinValue;
try
{
// Update file properties in storage
Dictionary<string, string> fileProperties = new Dictionary<string, string>();
fileProperties.Add("ContentType", contentType);
// update file metadata in storage
Dictionary<string, string> metadata = new Dictionary<string, string>();
metadata.Add("author", authorShortName);
metadata.Add("tite", title);
metadata.Add("description", description);
metadata.Add("duration", duration.ToString());
metadata.Add("recordingDate", recordingDate.ToString());
metadata.Add("uploadDate", uploadDate.ToString());
var fileNameWExt = $"{fileName}.{fileExtension}";
var blobContainer = await _cloudStorageService.CreateBlob(containerName, fileNameWExt, "audio");
try
{
MemoryStream fileContent = new MemoryStream(streamedFileContent);
fileContent.Position = 0;
using (fileContent)
{
await blobContainer.UploadFromStreamAsync(fileContent);
}
}
catch (StorageException e)
{
if (e.RequestInformation.HttpStatusCode == 403)
{
return BadRequest(e.Message);
}
else
{
return BadRequest(e.Message);
}
}
try
{
foreach (var key in metadata.Keys.ToList())
{
blobContainer.Metadata.Add(key, metadata[key]);
}
await blobContainer.SetMetadataAsync();
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
blobSize = await StorageUtils.GetBlobSize(blobContainer);
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
Media media = Media.Create(string.Empty, instructorId, authorName, fileName, fileType, fileExtension, recordingDate, uploadDate, ContentDetails.Create(title, description, duration, blobSize, 0, new List<string>()), StateDetails.Create(StatusType.STAGED, DateTime.MinValue, DateTime.UtcNow, DateTime.MaxValue), Manifest.Create(new Dictionary<string, string>()));
// upload to MongoDB
if (media != null)
{
var mapper = new Mapper(_mapperConfiguration);
var dao = mapper.Map<ContentDAO>(media);
try
{
await _db.Content.InsertOneAsync(dao);
}
catch (Exception)
{
mediaId = string.Empty;
}
mediaId = dao.Id.ToString();
}
else
{
// metadata wasn't stored, remove blob
await _cloudStorageService.DeleteBlob(containerName, fileName, "audio");
return BadRequest($"An issue occurred during media upload: rolling back storage change");
}
if (string.IsNullOrEmpty(mediaId))
{
return BadRequest($"Could not add instructor media");
}
}
catch (Exception ex)
{
return BadRequest(ex.Message);
}
var result = new { MediaId = mediaId, InstructorId = instructorId };
return Ok(result);
}
I reiterate, this all works great locally. I do not run it in IISExpress, I run it as a console app.
I submit large audio files via my SPA app and Postman and it works perfectly.
I am deploying this code to an Azure App Service on Linux (as a Basic B1).
Since the code works in my local development environment, I am at a loss of what my next steps are. I have refactored this code a few times but I suspect that it's environment related.
I cannot find anywhere that mentions that the level of App Service Plan is the culprit so before I go out spending more money I wanted to see if anyone here had encountered this challenge and could provide advice.
UPDATE: I attempted upgrading to a Production App Service Plan to see if there was an undocumented gate for incoming traffic. Upgrading didn't work either.
Thanks in advance.
-A
Currently, as of 11/2019, there is a limitation with the Azure App Service for Linux. It's CORS functionality is enabled by default and cannot be disabled AND it has a file size limitation that doesn't appear to get overridden by any of the published Kestrel configurations. The solution is to move the Web API app to a Azure App Service for Windows and it works as expected.
I am sure there is some way to get around it if you know the magic combination of configurations, server settings, and CLI commands but I need to move on with development.

Cannot refresh listview with observablecollection

I am trying to refresh my listview when an item is removed from it, but every time it gives me this error:
System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values.
Parameter name: index.
Before updating the ObservableCollection, I do this:
Groups = new ObservableCollection<RequestGroups>();
And then I fill it with this:
var temp = (JArray)resultJson["data"];
JArray jarr = temp;
foreach (JObject contents in jarr.Children<JObject>())
{
Requests obj = new Requests();
obj.Id = (int)contents["id"];
Client c = new Client();
c.address = contents["address"].ToString();
c.phone = contents["phone"].ToString();
c.name = contents["user"].ToString();
obj.Client = c;
obj.Date = contents["date"].ToString();
obj.Duration = contents["duration"].ToString();
obj.DurationText = "Duración: "+contents["duration"].ToString()+"h";
obj.Price = "$" + contents["price"].ToString();
String[] cDate = obj.Date.Split(' ');
String cHour = cDate[1]+" "+cDate[2];
obj.Hour = cHour;
String[] date = cDate[0].Split('-');
String title = months[date[1]] + " " + date[0];
obj.Title = title;
bool flag = false;
foreach(RequestGroups rqG in Groups){
if(rqG.Title.Equals(title)){
rqG.Add(obj);
flag = true;
}
}
if(!flag){
RequestGroups rq = new RequestGroups(title, date[1] + "-" + date[0]);
rq.Add(obj);
Device.BeginInvokeOnMainThread(() =>
{
Groups.Add(rq);
});
}
}
This is where I remove the items:
private async Task UpdateRequest(int status,int idEvent)
{
HttpClient hTTPClient = new HttpClient();
var client = new HttpClient();
client.Timeout = TimeSpan.FromSeconds(60);
client.BaseAddress = new Uri(Utils.baseUrl);
Dictionary<string, string> dataToSend = new Dictionary<string, string>();
dataToSend.Add("session", Utils.loginKey);
dataToSend.Add("eventId", idEvent+"");
dataToSend.Add("status", status.ToString());
string jsonData = Newtonsoft.Json.JsonConvert.SerializeObject(dataToSend, new KeyValuePairConverter());
var contentVar = new StringContent(jsonData, Encoding.UTF8, "application/json");
try
{
HttpResponseMessage response = await client.PostAsync("/UpdateEvent", contentVar);
var result = await response.Content.ReadAsStringAsync();
if (response.IsSuccessStatusCode)
{
string contents = await response.Content.ReadAsStringAsync();
var resultJson = JObject.Parse(result);
if ((int)resultJson["status"] == 0)
{
await base.DisplayAlert((string)resultJson["msg"], "", "OK");
return;
}
else if ((int)resultJson["status"] == 1)
{
//I'm currently trying to reload the whole view, before this was calling the method above.
await this.mainPage.Navigation.PushAsync(new NavigationPage(new MasterMenu.MainPage()));
await getRequests();
}
else
{
await base.DisplayAlert("Error procesando la solicitud, intente más tarde", "", "Ok");
return;
}
}
}
catch (Exception ex)
{
Debug.WriteLine("Error update request: {0}", ex);
}
await Task.Delay(1);
}
If I leave it like that, UI will not update. Please help me, as I've been struggling with this issue for 2 days now. It happens exclusively on iOS.
The issue was fixed after updating iOS. The problem was caused because of a buggy iOS version that had problems indexing objects. After updating, everything ran as smoothly as usual. If anyone runs into this issue (exclusively on iOS), try updating both iOS and Xamarin Forms.

How to ensure UploadStringCompletedEventHandler event has been executed successfully?

How to ensure UploadStringCompletedEventHandler event has been executed successfully ? in following code you can see i am calling function UploadMyPOST with my lastreads parameter having some data. Now you can see i am saving a variable named response into the MyClassXYZ varialbe. in the extreme last you can see there is a event which invoked by the method UploadMyPost() is filling the server response into the response variable. Now here issue is UploadMyPost(lastreads) executes successfully but its invoked event does not executes. Even cursor do not go on that event by which i am not able to fill server response into the response variable. So Anyone know any approach by which i can wait until that event successfully execute and i could able to save server response ?
private async void MyMethod(MyClassXYZ lastreads)
{
await UploadMyPOST(lastreads);
MyClassXYZ serverResponse = response;
if (serverResponse.Book == null)
{
//Do Something.
}
}
private void UploadMyPOST(MyClassXYZ lastreads)
{
apiData = new MyClassXYZApi()
{
AccessToken = thisApp.currentUser.AccessToken,
Book = lastreads.Book,
Page = lastreads.Page,
Device = lastreads.Device
};
//jsondata is my global variable of MyClassXYZ class.
jsondata = Newtonsoft.Json.JsonConvert.SerializeObject(apiData);
MyClassXYZ responsedData = new MyClassXYZ();
Uri lastread_url = new Uri(string.Format("{0}lastread", url_rootPath));
WebClient wc = new WebClient();
wc.Headers["Content-Type"] = "application/json;charset=utf-8";
wc.UploadStringCompleted += new UploadStringCompletedEventHandler(MyUploadStringCompleted);
wc.UploadStringAsync(lastread_url, "POST", jsondata);
}
private void MyUploadStringCompleted(object sender, UploadStringCompletedEventArgs e)
{
try
{
if (e.Error == null)
{
string resutls = e.Result;
DataContractJsonSerializer json = new DataContractJsonSerializer(typeof(MyClassXYZ));
MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(resutls));
response = (MyClassXYZ)json.ReadObject(ms);
}
else
{
string sx = e.Error.ToString();
}
}
catch(Exception exe)
{
}
}
//After Stephen suggession i used the HttpClient so i have written new code with the help of HttpClient. Code is building successfully but at run time cursor goes out from this method to the parent method where from its calling.
private async Task<string> UploadMyPOST(MyClassXYZ lastreads)
{
string value = "";
try
{
apiData = new LastReadAPI()
{
AccessToken = thisApp.currentUser.AccessToken,
Book = lastreads.Book,
Page = lastreads.Page,
Device = lastreads.Device
};
jsondata = Newtonsoft.Json.JsonConvert.SerializeObject(apiData);
LastRead responsedData = new LastRead();
Uri lastread_url = new Uri(string.Format("{0}lastread", url_rootPath));
HttpClient hc = new HttpClient();
//After following line cursor go back to main Method.
var res = await hc.PostAsync(lastread_url, new StringContent(jsondata));
res.EnsureSuccessStatusCode();
Stream content = await res.Content.ReadAsStreamAsync();
return await Task.Run(() => Newtonsoft.Json.JsonConvert.SerializeObject(content));
value = "kd";
}
catch
{ }
return value;
}
I recommend that you use HttpClient or wrap the UploadStringAsync/UploadStringCompleted pair into a Task-based method. Then you can use await like you want to in MyMethod.
Thank you Stephen Clear you leaded me in a right direction and i did POST my request successfully using HttpClient.
HttpClient hc = new HttpClient();
hc.BaseAddress = new Uri(annotation_url.ToString());
HttpRequestMessage req = new HttpRequestMessage(HttpMethod.Post, myUrl);
HttpContent myContent = req.Content = new StringContent(myJsonData, Encoding.UTF8, "application/json");
var response = await hc.PostAsync(myUrl, myContent);
//Following line for pull out the value of content key value which has the actual resposne.
string resutlContetnt = response.Content.ReadAsStringAsync().Result;
DataContractJsonSerializer deserializer_Json = new DataContractJsonSerializer(typeof(MyWrapperClass));
MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(resutlContetnt.ToString()));
AnnotateResponse = deserializer_Json.ReadObject(ms) as Annotation;

Receive, send file over Web Api

I'm trying to write a WebApi service that receives a file, does a trivial manipulation, and sends the file back. I'm having issues on sending and/or receiving the file from the service.
The issue I'm having is that the file returned from the service is ~1.5x larger than the manipulated file, e.g. when the file is returned it's like 300kb instead of the 200kb it should be.
I assume its being wrapped and or manipulated somehow, and I'm unsure of how to receive it properly. The code for the WebAPI service and the method that calls the web service are included below
In, the WebApi service, when I hit the line return Ok(bufferResult), the file is a byte[253312]
In the method that calls the web service, after the file is manipulated and returned, following the line var content = stream.Result;, the stream has a length of 337754 bytes.
Web API service code
public ConversionController: APIController{
public async Task<IHttpActionResult> TransformImage()
{
if (!Request.Content.IsMimeMultipartContent())
throw new Exception();
var provider = new MultipartMemoryStreamProvider();
await Request.Content.ReadAsMultipartAsync(provider);
var file = provider.Contents.First();
var filename = file.Headers.ContentDisposition.FileName.Trim('\"');
var buffer = await file.ReadAsByteArrayAsync();
var stream = new MemoryStream(buffer);
// [file manipulations omitted;]
// [the result is populated into a MemoryStream named response ]
//debug : save memory stream to disk to make sure tranformation is successfull
/*response.Position = 0;
path = #"C:\temp\file.ext";
using (var fileStream = System.IO.File.Create(path))
{
saveStream.CopyTo(fileStream);
}*/
var bufferResult = response.GetBuffer();
return Ok(bufferResult);
}
}
Method Calling the Service
public async Task<ActionResult> AsyncConvert()
{
var url = "http://localhost:49246/api/conversion/transformImage";
var filepath = "drive/file/path.ext";
HttpContent fileContent = new ByteArrayContent(System.IO.File.ReadAllBytes(filepath));
using (var client = new HttpClient())
{
using (var formData = new MultipartFormDataContent())
{
formData.Add(fileContent, "file", "fileName");
//call service
var response = client.PostAsync(url, formData).Result;
if (!response.IsSuccessStatusCode)
{
throw new Exception();
}
else
{
if (response.Content.GetType() != typeof(System.Net.Http.StreamContent))
throw new Exception();
var stream = response.Content.ReadAsStreamAsync();
var content = stream.Result;
var path = #"drive\completed\name.ext";
using (var fileStream = System.IO.File.Create(path))
{
content.CopyTo(fileStream);
}
}
}
}
return null;
}
I'm still new to streams and WebApi, so I may be missing something quite obvious. Why are the file streams different sizes? (eg. is it wrapped and how do I unwrap and/or receive the stream)
okay, to receive the file correctly, I needed to replace the line
var stream = response.Content.ReadAsStreamAsync();
with
var contents = await response.Content.ReadAsAsync<Byte[]>();
to provide the correct type for the binding
so, the later part of the methods that calls the service looks something like
var content = await response.Content.ReadAsAsync<Byte[]>();
var saveStream = new MemoryStream(content);
saveStream.Position = 0;
//Debug: save converted file to disk
/*
var path = #"drive\completed\name.ext";
using (var fileStream = System.IO.File.Create(path))
{
saveStream.CopyTo(fileStream);
}*/

Resources