We just started using SignalR in an MVC application and now we're getting a bunch of alerts due to high average response time. I suspect this to be misleading as the application isn't experiencing performance degradation. It appears that SignalR uses this URL to make a connection. This url not a controller/action of the project and just the built in SignalR code in the js file. jquery.signalR-2.2.1.js is the file. I suspect that it is just leaving the websocket connection open while they are on this page and it's skewing our numbers. Is this accurate? If so, is there a way to filter it out of the application insights?
Here is the counter. Is this the expected behavior?
Here is the signalR jquery code where it builds it's url:
// BUG #2953: The url needs to be same otherwise it will cause a memory leak
getUrl: function (connection, transport, reconnecting, poll, ajaxPost) {
/// <summary>Gets the url for making a GET based connect request</summary>
var baseUrl = transport === "webSockets" ? "" : connection.baseUrl,
url = baseUrl + connection.appRelativeUrl,
qs = "transport=" + transport;
if (!ajaxPost && connection.groupsToken) {
qs += "&groupsToken=" + window.encodeURIComponent(connection.groupsToken);
}
if (!reconnecting) {
url += "/connect";
} else {
if (poll) {
// longPolling transport specific
url += "/poll";
} else {
url += "/reconnect";
}
if (!ajaxPost && connection.messageId) {
qs += "&messageId=" + window.encodeURIComponent(connection.messageId);
}
}
url += "?" + qs;
url = transportLogic.prepareQueryString(connection, url);
if (!ajaxPost) {
url += "&tid=" + Math.floor(Math.random() * 11);
}
return url;
},
I fixed this by following the instructions on https://learn.microsoft.com/en-us/azure/application-insights/app-insights-api-filtering-sampling:
Update your ApplicationInsights Nuget package to 2.0.0 or later.
Create a class implementing ITelemetryProcessor:
public class UnwantedTelemetryFilter : ITelemetryProcessor
{
private ITelemetryProcessor Next { get; set; }
public UnwantedTelemetryFilter(ITelemetryProcessor next)
{
this.Next = next;
}
public void Process(ITelemetry item)
{
var request = item as RequestTelemetry;
if (request != null && request.Name != null)
if (request.Name.Contains("signalr"))
return;
// Send everything else:
this.Next.Process(item);
}
}
Add the processor to your Application_Start() in Global.asax.cs:
var builder = TelemetryConfiguration.Active.TelemetryProcessorChainBuilder;
builder.Use((next) => new UnwantedTelemetryFilter(next));
builder.Build();
if the calls are coming from the C# part of the app, the easiest way is to write a custom telemetry processor:
https://learn.microsoft.com/en-us/azure/application-insights/app-insights-api-filtering-sampling
public void Process(ITelemetry item)
{
var request = item as RequestTelemetry;
if (request != null && request.[some field here].Equals("[some signalr specific check here]", StringComparison.OrdinalIgnoreCase))
{
// To filter out an item, just terminate the chain:
return;
}
// Send everything else:
this.Next.Process(item);
}
and use that to explicitly filter out the signalr calls from being sent
or if the calls are coming from JS, then the telemetry initializer there does a similar thing to filter out telemetry if you return false in the initializer.
Related
When I run my web api method using Postman passing in my URL, it works fine - it returns the value of '5' which I expect since the call returns just a single integer. Also at the very bottom I include another method of my web api that I run using Postman and it too works just fine.
http://localhost:56224/api/profileandblog/validatelogin/DemoUser1/DemoUser1Password/169.254.102.60/
However, in the client - an Asp.Net MVC method, when building the URL, it is DROPPING the "/api/profileandblog" part. Note: I'm using "attribute routing" in the web api.
Here is the Asp.Net MVC method to call the web api:
I stop it on this line so I can see the error details: if (result1.IsSuccessStatusCode)
It's INCORRECTLY building the URL as: http://localhost:56224/validatelogin/DemoUser1/DemoUser1Password/169.254.102.60/
It's dropping the: "/api/profileandblog" part that should follow 56224.
So it give's me the Not found.
Why does it drop it? It has the localhost:56224 correct.
[HttpPost]
[ValidateAntiForgeryToken]
public ActionResult SignIn(SignInViewModel signInViewModel)
{
int returnedApiValue = 0;
User returnedApiUser = new User();
DateTime currentDateTime = DateTime.Now;
string hostName = Dns.GetHostName();
string myIpAddress = Dns.GetHostEntry(hostName).AddressList[2].ToString();
try
{
if (!this.IsCaptchaValid("Captcha is not valid"))
{
ViewBag.errormessage = "Error: captcha entered is not valid.";
}
else
{
if (!string.IsNullOrEmpty(signInViewModel.Username) && !string.IsNullOrEmpty(signInViewModel.Password))
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost:56224/api/profileandblog");
string restOfUrl = "/validatelogin/" + signInViewModel.Username + "/" + signInViewModel.Password + "/" + myIpAddress + "/";
// Call the web api to validate the sign in.
// Sends back a -1(failure), -2(validation issue) or the UserId(success) via an OUTPUT parameter.
var responseTask1 = client.GetAsync(restOfUrl);
responseTask1.Wait();
var result1 = responseTask1.Result;
if (result1.IsSuccessStatusCode)
{
var readTask1 = result1.Content.ReadAsAsync<string>();
readTask1.Wait();
returnedApiValue = Convert.ToInt32(readTask1.Result);
if (returnedApiValue == -2)
{
ViewBag.errormessage = "You entered an invalid user name and/or password";
}
else
{
// I have the 'user id'.
// Continue processing...
}
}
else
{
ModelState.AddModelError(string.Empty, "Server error on signing in. 'validatelogin'. Please contact the administrator.");
}
}
}
}
return View(signInViewModel);
}
catch (Exception)
{
throw;
}
}
Per the suggestion about not having headers, I used another tutorial (https://www.c-sharpcorner.com/article/consuming-asp-net-web-api-rest-service-in-asp-net-mvc-using-http-client/) and it has the code for defining the headers. But it is coded slightly different - using async Task<> on the method definition. I was not using async in my prior version.
[HttpPost]
[ValidateAntiForgeryToken]
public async Task<ActionResult> SignIn(SignInViewModel signInViewModel)
{
int returnedApiValue = 0;
User returnedApiUser = new User();
DateTime currentDateTime = DateTime.Now;
string hostName = Dns.GetHostName();
string myIpAddress = Dns.GetHostEntry(hostName).AddressList[2].ToString();
try
{
if (!this.IsCaptchaValid("Captcha is not valid"))
{
ViewBag.errormessage = "Error: captcha entered is not valid.";
}
else
{
if (!string.IsNullOrEmpty(signInViewModel.Username) && !string.IsNullOrEmpty(signInViewModel.Password))
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("http://localhost:56224/api/profileandblog");
string restOfUrl = "/validatelogin/" + signInViewModel.Username + "/" + signInViewModel.Password + "/" + myIpAddress + "/";
client.DefaultRequestHeaders.Clear();
client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
// Call the web api to validate the sign in.
// Sends back a -1(failure), -2(validation issue) or the UserId(success) via an OUTPUT parameter.
HttpResponseMessage result1 = await client.GetAsync(restOfUrl);
if (result1.IsSuccessStatusCode)
{
var readTask1 = result1.Content.ReadAsAsync<string>();
readTask1.Wait();
returnedApiValue = Convert.ToInt32(readTask1.Result);
if (returnedApiValue == -2)
{
ViewBag.errormessage = "You entered an invalid user name and/or password";
}
else
{
// I have the 'user id'.
// Do other processing....
}
}
else
{
ModelState.AddModelError(string.Empty, "Server error on signing in. 'validatelogin'. Please contact the administrator.");
}
}
}
}
return View(signInViewModel);
}
catch (Exception)
{
throw;
}
}
It now has a header but still NOT building the URL properly as it is not including the "/api/profileandblog" part.
Here is the web api and the method being called:
namespace GbngWebApi2.Controllers
{
[RoutePrefix("api/profileandblog")]
public class WebApi2Controller : ApiController
{
[HttpGet]
[Route("validatelogin/{userName}/{userPassword}/{ipAddress}/")]
public IHttpActionResult ValidateLogin(string userName, string userPassword, string ipAddress)
{
try
{
IHttpActionResult httpActionResult;
HttpResponseMessage httpResponseMessage;
int returnValue = 0;
// Will either be a valid 'user id" or a -2 indicating a validation issue.
returnValue = dataaccesslayer.ValidateLogin(userName, userPassword, ipAddress);
httpResponseMessage = Request.CreateResponse(HttpStatusCode.OK, returnValue);
httpActionResult = ResponseMessage(httpResponseMessage);
return httpActionResult;
}
catch (Exception)
{
throw;
}
}
}
}
Here's the network tab of the client browser before I hit the button to fire of the Asp.Net MVC method.
The network tab of the client browser after I hit the button to fire of the Asp.Net MVC method and it fails.
Here's another example of Postman executing another method of my api just fine.
I got it to work by setting this as: client.BaseAddress = new Uri("localhost:56224"); and setting the string restOfUrl = "/api/profileandblog/validatesignin/" + signInViewModel.Username + "/" + signInViewModel.Password + "/" + myIpAddress + "/";
I have an ASP.NET Core 3.0 Web API endpoint that I have set up to allow me to post large audio files. I have followed the following directions from MS docs to set up the endpoint.
https://learn.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-3.0#kestrel-maximum-request-body-size
When an audio file is uploaded to the endpoint, it is streamed to an Azure Blob Storage container.
My code works as expected locally.
When I push it to my production server in Azure App Service on Linux, the code does not work and errors with
Unhandled exception in request pipeline: System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Request body too large.
Per advice from the above article, I have configured incrementally updated Kesterl with the following:
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseKestrel((ctx, options) =>
{
var config = ctx.Configuration;
options.Limits.MaxRequestBodySize = 6000000000;
options.Limits.MinRequestBodyDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.MinResponseDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.RequestHeadersTimeout =
TimeSpan.FromMinutes(2);
}).UseStartup<Startup>();
Also configured FormOptions to accept files up to 6000000000
services.Configure<FormOptions>(options =>
{
options.MultipartBodyLengthLimit = 6000000000;
});
And also set up the API controller with the following attributes, per advice from the article
[HttpPost("audio", Name="UploadAudio")]
[DisableFormValueModelBinding]
[GenerateAntiforgeryTokenCookie]
[RequestSizeLimit(6000000000)]
[RequestFormLimits(MultipartBodyLengthLimit = 6000000000)]
Finally, here is the action itself. This giant block of code is not indicative of how I want the code to be written but I have merged it into one method as part of the debugging exercise.
public async Task<IActionResult> Audio()
{
if (!MultipartRequestHelper.IsMultipartContentType(Request.ContentType))
{
throw new ArgumentException("The media file could not be processed.");
}
string mediaId = string.Empty;
string instructorId = string.Empty;
try
{
// process file first
KeyValueAccumulator formAccumulator = new KeyValueAccumulator();
var streamedFileContent = new byte[0];
var boundary = MultipartRequestHelper.GetBoundary(
MediaTypeHeaderValue.Parse(Request.ContentType),
_defaultFormOptions.MultipartBoundaryLengthLimit
);
var reader = new MultipartReader(boundary, Request.Body);
var section = await reader.ReadNextSectionAsync();
while (section != null)
{
var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(
section.ContentDisposition, out var contentDisposition);
if (hasContentDispositionHeader)
{
if (MultipartRequestHelper
.HasFileContentDisposition(contentDisposition))
{
streamedFileContent =
await FileHelpers.ProcessStreamedFile(section, contentDisposition,
_permittedExtensions, _fileSizeLimit);
}
else if (MultipartRequestHelper
.HasFormDataContentDisposition(contentDisposition))
{
var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name).Value;
var encoding = FileHelpers.GetEncoding(section);
if (encoding == null)
{
return BadRequest($"The request could not be processed: Bad Encoding");
}
using (var streamReader = new StreamReader(
section.Body,
encoding,
detectEncodingFromByteOrderMarks: true,
bufferSize: 1024,
leaveOpen: true))
{
// The value length limit is enforced by
// MultipartBodyLengthLimit
var value = await streamReader.ReadToEndAsync();
if (string.Equals(value, "undefined",
StringComparison.OrdinalIgnoreCase))
{
value = string.Empty;
}
formAccumulator.Append(key, value);
if (formAccumulator.ValueCount >
_defaultFormOptions.ValueCountLimit)
{
return BadRequest($"The request could not be processed: Key Count limit exceeded.");
}
}
}
}
// Drain any remaining section body that hasn't been consumed and
// read the headers for the next section.
section = await reader.ReadNextSectionAsync();
}
var form = formAccumulator;
var file = streamedFileContent;
var results = form.GetResults();
instructorId = results["instructorId"];
string title = results["title"];
string firstName = results["firstName"];
string lastName = results["lastName"];
string durationInMinutes = results["durationInMinutes"];
//mediaId = await AddInstructorAudioMedia(instructorId, firstName, lastName, title, Convert.ToInt32(duration), DateTime.UtcNow, DateTime.UtcNow, file);
string fileExtension = "m4a";
// Generate Container Name - InstructorSpecific
string containerName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{instructorId}";
string contentType = "audio/mp4";
FileType fileType = FileType.audio;
string authorName = $"{firstName} {lastName}";
string authorShortName = $"{firstName[0]}{lastName}";
string description = $"{authorShortName} - {title}";
long duration = (Convert.ToInt32(durationInMinutes) * 60000);
// Generate new filename
string fileName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{Guid.NewGuid()}";
DateTime recordingDate = DateTime.UtcNow;
DateTime uploadDate = DateTime.UtcNow;
long blobSize = long.MinValue;
try
{
// Update file properties in storage
Dictionary<string, string> fileProperties = new Dictionary<string, string>();
fileProperties.Add("ContentType", contentType);
// update file metadata in storage
Dictionary<string, string> metadata = new Dictionary<string, string>();
metadata.Add("author", authorShortName);
metadata.Add("tite", title);
metadata.Add("description", description);
metadata.Add("duration", duration.ToString());
metadata.Add("recordingDate", recordingDate.ToString());
metadata.Add("uploadDate", uploadDate.ToString());
var fileNameWExt = $"{fileName}.{fileExtension}";
var blobContainer = await _cloudStorageService.CreateBlob(containerName, fileNameWExt, "audio");
try
{
MemoryStream fileContent = new MemoryStream(streamedFileContent);
fileContent.Position = 0;
using (fileContent)
{
await blobContainer.UploadFromStreamAsync(fileContent);
}
}
catch (StorageException e)
{
if (e.RequestInformation.HttpStatusCode == 403)
{
return BadRequest(e.Message);
}
else
{
return BadRequest(e.Message);
}
}
try
{
foreach (var key in metadata.Keys.ToList())
{
blobContainer.Metadata.Add(key, metadata[key]);
}
await blobContainer.SetMetadataAsync();
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
blobSize = await StorageUtils.GetBlobSize(blobContainer);
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
Media media = Media.Create(string.Empty, instructorId, authorName, fileName, fileType, fileExtension, recordingDate, uploadDate, ContentDetails.Create(title, description, duration, blobSize, 0, new List<string>()), StateDetails.Create(StatusType.STAGED, DateTime.MinValue, DateTime.UtcNow, DateTime.MaxValue), Manifest.Create(new Dictionary<string, string>()));
// upload to MongoDB
if (media != null)
{
var mapper = new Mapper(_mapperConfiguration);
var dao = mapper.Map<ContentDAO>(media);
try
{
await _db.Content.InsertOneAsync(dao);
}
catch (Exception)
{
mediaId = string.Empty;
}
mediaId = dao.Id.ToString();
}
else
{
// metadata wasn't stored, remove blob
await _cloudStorageService.DeleteBlob(containerName, fileName, "audio");
return BadRequest($"An issue occurred during media upload: rolling back storage change");
}
if (string.IsNullOrEmpty(mediaId))
{
return BadRequest($"Could not add instructor media");
}
}
catch (Exception ex)
{
return BadRequest(ex.Message);
}
var result = new { MediaId = mediaId, InstructorId = instructorId };
return Ok(result);
}
I reiterate, this all works great locally. I do not run it in IISExpress, I run it as a console app.
I submit large audio files via my SPA app and Postman and it works perfectly.
I am deploying this code to an Azure App Service on Linux (as a Basic B1).
Since the code works in my local development environment, I am at a loss of what my next steps are. I have refactored this code a few times but I suspect that it's environment related.
I cannot find anywhere that mentions that the level of App Service Plan is the culprit so before I go out spending more money I wanted to see if anyone here had encountered this challenge and could provide advice.
UPDATE: I attempted upgrading to a Production App Service Plan to see if there was an undocumented gate for incoming traffic. Upgrading didn't work either.
Thanks in advance.
-A
Currently, as of 11/2019, there is a limitation with the Azure App Service for Linux. It's CORS functionality is enabled by default and cannot be disabled AND it has a file size limitation that doesn't appear to get overridden by any of the published Kestrel configurations. The solution is to move the Web API app to a Azure App Service for Windows and it works as expected.
I am sure there is some way to get around it if you know the magic combination of configurations, server settings, and CLI commands but I need to move on with development.
I was able to run the FiddlerCore demo (that comes with the package) without issue. I see both http and https traffic being logged on my PC.
My goal now is to do the same for my iOS traffic but I can't figure out what I am missing. I can see my https traffic fine when I use the desktop Fiddler app, by following the instructions at ConfigureForiOS.
I run the console FiddlerCore demo, hit 't' to trust the root certificate and then try to follow the same steps on my iPhone as I did for the Fidder app, namely setting my proxy to the Fiddler instance (my machine's IP and port 7777 as that is what it looks like the demo is using) and trusting the Fiddler cert that I had already installed on my phone when setting it up to work with the desktop Fiddler app. Then when I try to start an app on my phone that goes over https (for example a game) it just hangs. I don't see any errors being logged in the console app. It works ok when just running the desktop Fiddler app.
My SSL/cert/Fiddler knowledge is weak so I am hoping I am just missing a simple step or two.
Questions:
How can I capture iOS HTTPS traffic using the FiddlerCore demo app?
Do I need to trust the root certificate each time I start the demo
app (hitting 't')?
Thanks.
P.S. I added the demo app here, which can be found in the FiddlerCore package, for reference.
using Fiddler;
using System;
using System.Collections.Generic;
using System.Threading;
namespace FiddlerCoreDemo
{
class Program
{
static Proxy oSecureEndpoint;
static string sSecureEndpointHostname = "localhost";
static int iSecureEndpointPort = 7777;
public static void WriteCommandResponse(string s)
{
ConsoleColor oldColor = Console.ForegroundColor;
Console.ForegroundColor = ConsoleColor.Yellow;
Console.WriteLine(s);
Console.ForegroundColor = oldColor;
}
public static void DoQuit()
{
WriteCommandResponse("Shutting down...");
if (null != oSecureEndpoint) oSecureEndpoint.Dispose();
Fiddler.FiddlerApplication.Shutdown();
Thread.Sleep(500);
}
private static string Ellipsize(string s, int iLen)
{
if (s.Length <= iLen) return s;
return s.Substring(0, iLen - 3) + "...";
}
#if SAZ_SUPPORT
private static void ReadSessions(List<Fiddler.Session> oAllSessions)
{
Session[] oLoaded = Utilities.ReadSessionArchive(Environment.GetFolderPath(Environment.SpecialFolder.Desktop)
+ Path.DirectorySeparatorChar + "ToLoad.saz", false);
if ((oLoaded != null) && (oLoaded.Length > 0))
{
oAllSessions.AddRange(oLoaded);
WriteCommandResponse("Loaded: " + oLoaded.Length + " sessions.");
}
}
private static void SaveSessionsToDesktop(List<Fiddler.Session> oAllSessions)
{
bool bSuccess = false;
string sFilename = Environment.GetFolderPath(Environment.SpecialFolder.DesktopDirectory)
+ Path.DirectorySeparatorChar + DateTime.Now.ToString("hh-mm-ss") + ".saz";
try
{
try
{
Monitor.Enter(oAllSessions);
string sPassword = null;
Console.WriteLine("Password Protect this Archive (Y/N)?");
ConsoleKeyInfo oCKI = Console.ReadKey();
if ((oCKI.KeyChar == 'y') || (oCKI.KeyChar == 'Y'))
{
Console.WriteLine("\nEnter the password:");
sPassword = Console.ReadLine();
Console.WriteLine(String.Format("\nEncrypting with Password: '{0}'", sPassword));
}
Console.WriteLine();
bSuccess = Utilities.WriteSessionArchive(sFilename, oAllSessions.ToArray(), sPassword, false);
}
finally
{
Monitor.Exit(oAllSessions);
}
WriteCommandResponse( bSuccess ? ("Wrote: " + sFilename) : ("Failed to save: " + sFilename) );
}
catch (Exception eX)
{
Console.WriteLine("Save failed: " + eX.Message);
}
}
#endif
private static void WriteSessionList(List<Fiddler.Session> oAllSessions)
{
ConsoleColor oldColor = Console.ForegroundColor;
Console.ForegroundColor = ConsoleColor.White;
Console.WriteLine("Session list contains...");
try
{
Monitor.Enter(oAllSessions);
foreach (Session oS in oAllSessions)
{
Console.Write(String.Format("{0} {1} {2}\n{3} {4}\n\n", oS.id, oS.oRequest.headers.HTTPMethod, Ellipsize(oS.fullUrl, 60), oS.responseCode, oS.oResponse.MIMEType));
}
}
finally
{
Monitor.Exit(oAllSessions);
}
Console.WriteLine();
Console.ForegroundColor = oldColor;
}
static void Main(string[] args)
{
List<Fiddler.Session> oAllSessions = new List<Fiddler.Session>();
// <-- Personalize for your Application, 64 chars or fewer
Fiddler.FiddlerApplication.SetAppDisplayName("FiddlerCoreDemoApp");
#region AttachEventListeners
//
// It is important to understand that FiddlerCore calls event handlers on session-handling
// background threads. If you need to properly synchronize to the UI-thread (say, because
// you're adding the sessions to a list view) you must call .Invoke on a delegate on the
// window handle.
//
// If you are writing to a non-threadsafe data structure (e.g. List<t>) you must
// use a Monitor or other mechanism to ensure safety.
//
// Simply echo notifications to the console. Because Fiddler.CONFIG.QuietMode=true
// by default, we must handle notifying the user ourselves.
Fiddler.FiddlerApplication.OnNotification += delegate (object sender, NotificationEventArgs oNEA) { Console.WriteLine("** NotifyUser: " + oNEA.NotifyString); };
Fiddler.FiddlerApplication.Log.OnLogString += delegate (object sender, LogEventArgs oLEA) { Console.WriteLine("** LogString: " + oLEA.LogString); };
Fiddler.FiddlerApplication.BeforeRequest += delegate (Fiddler.Session oS)
{
// Console.WriteLine("Before request for:\t" + oS.fullUrl);
// In order to enable response tampering, buffering mode MUST
// be enabled; this allows FiddlerCore to permit modification of
// the response in the BeforeResponse handler rather than streaming
// the response to the client as the response comes in.
oS.bBufferResponse = false;
Monitor.Enter(oAllSessions);
oAllSessions.Add(oS);
Monitor.Exit(oAllSessions);
// Set this property if you want FiddlerCore to automatically authenticate by
// answering Digest/Negotiate/NTLM/Kerberos challenges itself
// oS["X-AutoAuth"] = "(default)";
/* If the request is going to our secure endpoint, we'll echo back the response.
Note: This BeforeRequest is getting called for both our main proxy tunnel AND our secure endpoint,
so we have to look at which Fiddler port the client connected to (pipeClient.LocalPort) to determine whether this request
was sent to secure endpoint, or was merely sent to the main proxy tunnel (e.g. a CONNECT) in order to *reach* the secure endpoint.
As a result of this, if you run the demo and visit https://localhost:7777 in your browser, you'll see
Session list contains...
1 CONNECT http://localhost:7777
200 <-- CONNECT tunnel sent to the main proxy tunnel, port 8877
2 GET https://localhost:7777/
200 text/html <-- GET request decrypted on the main proxy tunnel, port 8877
3 GET https://localhost:7777/
200 text/html <-- GET request received by the secure endpoint, port 7777
*/
if ((oS.oRequest.pipeClient.LocalPort == iSecureEndpointPort) && (oS.hostname == sSecureEndpointHostname))
{
oS.utilCreateResponseAndBypassServer();
oS.oResponse.headers.SetStatus(200, "Ok");
oS.oResponse["Content-Type"] = "text/html; charset=UTF-8";
oS.oResponse["Cache-Control"] = "private, max-age=0";
oS.utilSetResponseBody("<html><body>Request for httpS://" + sSecureEndpointHostname + ":" + iSecureEndpointPort.ToString() + " received. Your request was:<br /><plaintext>" + oS.oRequest.headers.ToString());
}
};
/*
// The following event allows you to examine every response buffer read by Fiddler. Note that this isn't useful for the vast majority of
// applications because the raw buffer is nearly useless; it's not decompressed, it includes both headers and body bytes, etc.
//
// This event is only useful for a handful of applications which need access to a raw, unprocessed byte-stream
Fiddler.FiddlerApplication.OnReadResponseBuffer += new EventHandler<RawReadEventArgs>(FiddlerApplication_OnReadResponseBuffer);
*/
/*
Fiddler.FiddlerApplication.BeforeResponse += delegate(Fiddler.Session oS) {
// Console.WriteLine("{0}:HTTP {1} for {2}", oS.id, oS.responseCode, oS.fullUrl);
// Uncomment the following two statements to decompress/unchunk the
// HTTP response and subsequently modify any HTTP responses to replace
// instances of the word "Microsoft" with "Bayden". You MUST also
// set bBufferResponse = true inside the beforeREQUEST method above.
//
//oS.utilDecodeResponse(); oS.utilReplaceInResponse("Microsoft", "Bayden");
};*/
Fiddler.FiddlerApplication.AfterSessionComplete += delegate (Fiddler.Session oS)
{
//Console.WriteLine("Finished session:\t" + oS.fullUrl);
Console.Title = ("Session list contains: " + oAllSessions.Count.ToString() + " sessions");
};
// Tell the system console to handle CTRL+C by calling our method that
// gracefully shuts down the FiddlerCore.
//
// Note, this doesn't handle the case where the user closes the window with the close button.
// See http://geekswithblogs.net/mrnat/archive/2004/09/23/11594.aspx for info on that...
//
Console.CancelKeyPress += new ConsoleCancelEventHandler(Console_CancelKeyPress);
#endregion AttachEventListeners
string sSAZInfo = "NoSAZ";
#if SAZ_SUPPORT
sSAZInfo = Assembly.GetAssembly(typeof(Ionic.Zip.ZipFile)).FullName;
// You can load Transcoders from any different assembly if you'd like, using the ImportTranscoders(string AssemblyPath)
// overload.
//
//if (!FiddlerApplication.oTranscoders.ImportTranscoders(Assembly.GetExecutingAssembly()))
//{
// Console.WriteLine("This assembly was not compiled with a SAZ-exporter");
//}
DNZSAZProvider.fnObtainPwd = () =>
{
Console.WriteLine("Enter the password (or just hit Enter to cancel):");
string sResult = Console.ReadLine();
Console.WriteLine();
return sResult;
};
FiddlerApplication.oSAZProvider = new DNZSAZProvider();
#endif
Console.WriteLine(String.Format("Starting {0} ({1})...", Fiddler.FiddlerApplication.GetVersionString(), sSAZInfo));
// For the purposes of this demo, we'll forbid connections to HTTPS
// sites that use invalid certificates. Change this from the default only
// if you know EXACTLY what that implies.
Fiddler.CONFIG.IgnoreServerCertErrors = false;
// ... but you can allow a specific (even invalid) certificate by implementing and assigning a callback...
// FiddlerApplication.OnValidateServerCertificate += new System.EventHandler<ValidateServerCertificateEventArgs>(CheckCert);
FiddlerApplication.Prefs.SetBoolPref("fiddler.network.streaming.abortifclientaborts", true);
// For forward-compatibility with updated FiddlerCore libraries, it is strongly recommended that you
// start with the DEFAULT options and manually disable specific unwanted options.
FiddlerCoreStartupFlags oFCSF = FiddlerCoreStartupFlags.Default;
// E.g. If you want to add a flag, start with the .Default and "OR" the new flag on:
// oFCSF = (oFCSF | FiddlerCoreStartupFlags.CaptureFTP);
// ... or if you don't want a flag in the defaults, "and not" it out:
// Uncomment the next line if you don't want FiddlerCore to act as the system proxy
// oFCSF = (oFCSF & ~FiddlerCoreStartupFlags.RegisterAsSystemProxy);
// *******************************
// Important HTTPS Decryption Info
// *******************************
// When FiddlerCoreStartupFlags.DecryptSSL is enabled, you must include either
//
// MakeCert.exe
//
// *or*
//
// CertMaker.dll
// BCMakeCert.dll
//
// ... in the folder where your executable and FiddlerCore.dll live. These files
// are needed to generate the self-signed certificates used to man-in-the-middle
// secure traffic. MakeCert.exe uses Windows APIs to generate certificates which
// are stored in the user's \Personal\ Certificates store. These certificates are
// NOT compatible with iOS devices which require specific fields in the certificate
// which are not set by MakeCert.exe.
//
// In contrast, CertMaker.dll uses the BouncyCastle C# library (BCMakeCert.dll) to
// generate new certificates from scratch. These certificates are stored in memory
// only, and are compatible with iOS devices.
// Uncomment the next line if you don't want to decrypt SSL traffic.
// oFCSF = (oFCSF & ~FiddlerCoreStartupFlags.DecryptSSL);
// NOTE: In the next line, you can pass 0 for the port (instead of 8877) to have FiddlerCore auto-select an available port
int iPort = 8877;
Fiddler.FiddlerApplication.Startup(iPort, oFCSF);
FiddlerApplication.Log.LogFormat("Created endpoint listening on port {0}", iPort);
FiddlerApplication.Log.LogFormat("Starting with settings: [{0}]", oFCSF);
FiddlerApplication.Log.LogFormat("Gateway: {0}", CONFIG.UpstreamGateway.ToString());
Console.WriteLine("Hit CTRL+C to end session.");
// We'll also create a HTTPS listener, useful for when FiddlerCore is masquerading as a HTTPS server
// instead of acting as a normal CERN-style proxy server.
oSecureEndpoint = FiddlerApplication.CreateProxyEndpoint(iSecureEndpointPort, true, sSecureEndpointHostname);
if (null != oSecureEndpoint)
{
FiddlerApplication.Log.LogFormat("Created secure endpoint listening on port {0}, using a HTTPS certificate for '{1}'", iSecureEndpointPort, sSecureEndpointHostname);
}
bool bDone = false;
do
{
Console.WriteLine("\nEnter a command [C=Clear; L=List; G=Collect Garbage; W=write SAZ; R=read SAZ;\n\tS=Toggle Forgetful Streaming; T=Trust Root Certificate; Q=Quit]:");
Console.Write(">");
ConsoleKeyInfo cki = Console.ReadKey();
Console.WriteLine();
switch (Char.ToLower(cki.KeyChar))
{
case 'c':
Monitor.Enter(oAllSessions);
oAllSessions.Clear();
Monitor.Exit(oAllSessions);
WriteCommandResponse("Clear...");
FiddlerApplication.Log.LogString("Cleared session list.");
break;
case 'd':
FiddlerApplication.Log.LogString("FiddlerApplication::Shutdown.");
FiddlerApplication.Shutdown();
break;
case 'l':
WriteSessionList(oAllSessions);
break;
case 'g':
Console.WriteLine("Working Set:\t" + Environment.WorkingSet.ToString("n0"));
Console.WriteLine("Begin GC...");
GC.Collect();
Console.WriteLine("GC Done.\nWorking Set:\t" + Environment.WorkingSet.ToString("n0"));
break;
case 'q':
bDone = true;
DoQuit();
break;
case 'r':
#if SAZ_SUPPORT
ReadSessions(oAllSessions);
#else
WriteCommandResponse("This demo was compiled without SAZ_SUPPORT defined");
#endif
break;
case 'w':
#if SAZ_SUPPORT
if (oAllSessions.Count > 0)
{
SaveSessionsToDesktop(oAllSessions);
}
else
{
WriteCommandResponse("No sessions have been captured");
}
#else
WriteCommandResponse("This demo was compiled without SAZ_SUPPORT defined");
#endif
break;
case 't':
try
{
WriteCommandResponse("Result: " + Fiddler.CertMaker.trustRootCert().ToString());
}
catch (Exception eX)
{
WriteCommandResponse("Failed: " + eX.ToString());
}
break;
// Forgetful streaming
case 's':
bool bForgetful = !FiddlerApplication.Prefs.GetBoolPref("fiddler.network.streaming.ForgetStreamedData", false);
FiddlerApplication.Prefs.SetBoolPref("fiddler.network.streaming.ForgetStreamedData", bForgetful);
Console.WriteLine(bForgetful ? "FiddlerCore will immediately dump streaming response data." : "FiddlerCore will keep a copy of streamed response data.");
break;
}
} while (!bDone);
}
/*
/// <summary>
/// This callback allows your code to evaluate the certificate for a site and optionally override default validation behavior for that certificate.
/// You should not implement this method unless you understand why it is a security risk.
/// </summary>
static void CheckCert(object sender, ValidateServerCertificateEventArgs e)
{
if (null != e.ServerCertificate)
{
Console.WriteLine("Certificate for " + e.ExpectedCN + " was for site " + e.ServerCertificate.Subject + " and errors were " + e.CertificatePolicyErrors.ToString());
if (e.ServerCertificate.Subject.Contains("fiddler2.com"))
{
Console.WriteLine("Got a certificate for fiddler2.com. We'll say this is also good for any other site, like https://fiddlertool.com.");
e.ValidityState = CertificateValidity.ForceValid;
}
}
}
*/
/*
// This event handler is called on every socket read for the HTTP Response. You almost certainly don't want
// to add a handler for this event, but the code below shows how you can use it to mess up your HTTP traffic.
static void FiddlerApplication_OnReadResponseBuffer(object sender, RawReadEventArgs e)
{
// NOTE: arrDataBuffer is a fixed-size array. Only bytes 0 to iCountOfBytes should be read/manipulated.
//
// Just for kicks, lowercase every byte. Note that this will obviously break any binary content.
for (int i = 0; i < e.iCountOfBytes; i++)
{
if ((e.arrDataBuffer[i] > 0x40) && (e.arrDataBuffer[i] < 0x5b))
{
e.arrDataBuffer[i] = (byte)(e.arrDataBuffer[i] + (byte)0x20);
}
}
Console.WriteLine(String.Format("Read {0} response bytes for session {1}", e.iCountOfBytes, e.sessionOwner.id));
}
*/
/// <summary>
/// When the user hits CTRL+C, this event fires. We use this to shut down and unregister our FiddlerCore.
/// </summary>
/// <param name="sender"></param>
/// <param name="e"></param>
static void Console_CancelKeyPress(object sender, ConsoleCancelEventArgs e)
{
DoQuit();
}
}
}
You can grant the certificate full trust at:
Settings > General > About > Trust Cert
I have created a ASP.Net Web API project and using this link. NServiceBus is integrated with web api. Here is my configuration at web api as a client.
Configure.Serialization.Xml();
Configure.Transactions.Enable();
Configure.With()
.DefineEndpointName(Constants.ClientName)
.DefaultBuilder()
.ForWebApi()
.Log4Net()
.UseTransport<Msmq>()
.PurgeOnStartup(false)
.UnicastBus()
.ImpersonateSender(false)
.CreateBus()
.Start();
This is how I'm sending message to Server
var response = await Bus.Send(Constants.ServerName, request)
.Register<ResponseModel>((NServiceBus.CompletionResult completionResult) =>
{
ResponseModel responseMessage = null;
if (completionResult != null && completionResult.Messages.Length > 0)
{
var status = completionResult.Messages[0] as RequestStatus?;
if (status == RequestStatus.Successful)
{
responseMessage = TransactionManager.TransactionDictionary[request.RequestId].ResponseModel;
}
}
return responseMessage;
});
This is how I'm sending response from Server. I have commented some lines to show what I have already tried.
public void Handle(RequestModel message)
{
ProcessRequest(message).RunSynchronously();
}
private async Task ProcessRequest(RequestModel message)
{
....
ResponseModel response = new ResponseModel();
response.RequestId = message.RequestId;
response.Result = await responseMessage.Content.ReadAsStringAsync();
//Bus.Send(Util.Constants.ClientName, response);
//Bus.Reply(response);
//Bus.Reply<ResponseModel>((ResponseModel response) =>
//{
// response = Bus.CreateInstance<ResponseModel>(r =>
// {
// r.RequestId = message.RequestId;
// r.Result = responseMessage.Content.ReadAsStringAsync().Result;
// });
//});
await Bus.Send(Util.Constants.ClientName, response).Register((NServiceBus.CompletionResult completionResult) =>
{
if (completionResult != null && completionResult.Messages.Length > 0)
{
var msg = completionResult.Messages[0];
if (msg != null)
{
var status = (RequestStatus)msg;
return status;
}
}
return RequestStatus.Error;
});
....
}
From any of the above response methods ultimately all messages end up in error queue.
Previously I was getting 'Could not enlist message' error. Now it is not throwing that error. But Server could not send message to Client.
I could not get what I'm doing wrong. Please also suggest if you see any scope for improvements.
I'm not sure if TransactionScope work correctly with async/await in C#. According to this question (Get TransactionScope to work with async / await) in .NET 4.5.1 there was introduced option for TransactionScope that enable mixing it with async/await. Unfortunately NServiceBus doesn't support .NET 4.5/4.5.1 so try just remove async/await.
I am trying to figure out how to use the AWS .NET SDK to confirm a subscription to a SNS Topic.
The subscription is via HTTP
The endpoint will be in a .net mvc website.
I can't find any .net examples anywhere?
A working example would be fantastic.
I'm trying something like this
Dim snsclient As New Amazon.SimpleNotificationService.AmazonSimpleNotificationServiceClient(ConfigurationSettings.AppSettings("AWSAccessKey"), ConfigurationSettings.AppSettings("AWSSecretKey"))
Dim TopicArn As String = "arn:aws:sns:us-east-1:991924819628:post-delivery"
If Request.Headers("x-amz-sns-message-type") = "SubscriptionConfirmation" Then
Request.InputStream.Seek(0, 0)
Dim reader As New System.IO.StreamReader(Request.InputStream)
Dim inputString As String = reader.ReadToEnd()
Dim jsSerializer As New System.Web.Script.Serialization.JavaScriptSerializer
Dim message As Dictionary(Of String, String) = jsSerializer.Deserialize(Of Dictionary(Of String, String))(inputString)
snsclient.ConfirmSubscription(New Amazon.SimpleNotificationService.Model.ConfirmSubscriptionRequest With {.AuthenticateOnUnsubscribe = False, .Token = message("Token"), .TopicArn = TopicArn})
End If
Here is a working example using MVC WebApi 2 and the latest AWS .NET SDK.
var jsonData = Request.Content.ReadAsStringAsync().Result;
var snsMessage = Amazon.SimpleNotificationService.Util.Message.ParseMessage(jsonData);
//verify the signaure using AWS method
if(!snsMessage.IsMessageSignatureValid())
throw new Exception("Invalid signature");
if(snsMessage.Type == Amazon.SimpleNotificationService.Util.Message.MESSAGE_TYPE_SUBSCRIPTION_CONFIRMATION)
{
var subscribeUrl = snsMessage.SubscribeURL;
var webClient = new WebClient();
webClient.DownloadString(subscribeUrl);
return "Successfully subscribed to: " + subscribeUrl;
}
Building on #Craig's answer above (which helped me greatly), the below is an ASP.NET MVC WebAPI controller for consuming and auto-subscribing to SNS topics. #WebHooksFTW
using RestSharp;
using System;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Web.Http;
using System.Web.Http.Description;
namespace sb.web.Controllers.api {
[System.Web.Mvc.HandleError]
[AllowAnonymous]
[ApiExplorerSettings(IgnoreApi = true)]
public class SnsController : ApiController {
private static string className = MethodBase.GetCurrentMethod().DeclaringType.Name;
[HttpPost]
public HttpResponseMessage Post(string id = "") {
try {
var jsonData = Request.Content.ReadAsStringAsync().Result;
var sm = Amazon.SimpleNotificationService.Util.Message.ParseMessage(jsonData);
//LogIt.D(jsonData);
//LogIt.D(sm);
if (!string.IsNullOrEmpty(sm.SubscribeURL)) {
var uri = new Uri(sm.SubscribeURL);
var baseUrl = uri.GetLeftPart(System.UriPartial.Authority);
var resource = sm.SubscribeURL.Replace(baseUrl, "");
var response = new RestClient {
BaseUrl = new Uri(baseUrl),
}.Execute(new RestRequest {
Resource = resource,
Method = Method.GET,
RequestFormat = RestSharp.DataFormat.Xml
});
if (response.StatusCode != System.Net.HttpStatusCode.OK) {
//LogIt.W(response.StatusCode);
} else {
//LogIt.I(response.Content);
}
}
//read for topic: sm.TopicArn
//read for data: dynamic json = JObject.Parse(sm.MessageText);
//extract value: var s3OrigUrlSnippet = json.input.key.Value as string;
//do stuff
return Request.CreateResponse(HttpStatusCode.OK, new { });
} catch (Exception ex) {
//LogIt.E(ex);
return Request.CreateResponse(HttpStatusCode.InternalServerError, new { status = "unexpected error" });
}
}
}
}
I don't know how recently this has changed, but I've found that AWS SNS now provides a very simply method for subscribing that doesn't involve extracting urls or building requests using RESTSharp.....Here's the simplified WebApi POST method:
[HttpPost]
public HttpResponseMessage Post(string id = "")
{
try
{
var jsonData = Request.Content.ReadAsStringAsync().Result;
var sm = Amazon.SimpleNotificationService.Util.Message.ParseMessage(jsonData);
if (sm.IsSubscriptionType)
{
sm.SubscribeToTopic(); // CONFIRM THE SUBSCRIPTION
}
if (sm.IsNotificationType) // PROCESS NOTIFICATIONS
{
//read for topic: sm.TopicArn
//read for data: dynamic json = JObject.Parse(sm.MessageText);
//extract value: var s3OrigUrlSnippet = json.input.key.Value as string;
}
//do stuff
return Request.CreateResponse(HttpStatusCode.OK, new { });
}
catch (Exception ex)
{
//LogIt.E(ex);
return Request.CreateResponse(HttpStatusCode.InternalServerError, new { status = "unexpected error" });
}
}
The following example helped me work with SNS. It goes through all the steps to work with Topics. The subscribe request in this case is an email address, however that can be changed to HTTP.
Pavel's SNS Example
Documentation
I ended up getting it working using the code shown. I was having trouble capturing the exception on the development server which turned out was telling me the server's time didn't match the timestamp in the SNS message.
Once the server's time was fixed up (an Amazon server BTW), the confirmation worked.