Just upgraded from MassTransit 6.3.2 to 7.2.4. I'm using .Net Core 5.
The following unit test works fine before the upgrade but fails after.
using var harness = new InMemoryTestHarness();
harness.Consumer(() => new MockedxxxService(), xxxEndPoint);
harness.Start().Wait();
IBus endpoint = harness.Bus;
var tasks = new System.Collections.Concurrent.ConcurrentBag<Task<string>>();
var result = Parallel.For(0, 100, index =>
{
var sut = new xxxRetriever(..., endpoint, ...);
tasks.Add(sut.Getxxx(paramx));
});
Assert.True(result.IsCompleted);
await Task.WhenAll(tasks);
The code in the xxxRetriever class looks like this.
public async Task<string> Getxxx(string paramx)
{
try
{
var serviceAddress = new Uri("queue:" + ...);
var xxxService = _busService.CreateRequestClient<xxxContract>(serviceAddress, TimeSpan.FromMilliseconds(xxx));
var xxxResponse = await xxxService.GetResponse<xxxResultContract>(new
{
...
}).ConfigureAwait(false);
....
}
}
The endpoint is injected into the class as an IBus.
The mocked service looks like this.
public class MockedxxxService : IConsumer<xxxContract>
{
public async Task Consume(ConsumeContext<xxxContract> context)
{
await context.RespondAsync<xxxResultContract>(new { ... } } });
}
}
The tests run fine when we limit the number of tasks to about 30. But above that it fails consistently with the message "Timeout waiting for response, RequestId: ...".
Any help would be appreciated.
It might be related to contention in the TPL, but that would only be a guess. You might be able to configure increase the concurrency limit on the bus and see if that helps:
harness.OnConfigureInMemoryBus += c =>
{
c.Host(h => h.TransportConcurrencyLimit = 100);
};
That's a guess, but it might be related since it's load specific.
Obviously, this should be called prior to calling Start on the hardness (which should be awaited, instead of using .Wait() by the way).
After much investigation, we found that this was caused by the way .Net 5 handles thread creation. See ThreadPool.SetMinThreads does not create any new threads
We primed the thread pool before the test and that fixed the issue. Not sure why this worked correctly in .Net 5 under MassTransit 6, but it's working now.
Related
Hi Im new to MVC and still trying to understand the possible problem of using async.
I have the code below and as you can see i have placed async for all database calls (Not sure whether this is a good practice).
As you might notice this is a simple payment process. I am updating the CustomerPayment entity using the attribute of Customer and also once I added the new transaction, I am updating the balance attribute of the customer.
Using the colde below, will there be any risk of using async in regards to database calls ?
Also, I could see the _context.Database.BeginTransactionAsync() method for transaction, what will be the difference of using BeginTransactionAsync and normal BeginTransaction ?
Any chance this code can be re-written to properly use async ?
using (var dbContextTransaction = _context.Database.BeginTransaction())
{
try
{
CustomerPayment cp = vm;
Customer c = await _context.Customer.SingleOrDefaultAsync(m => m.CustomerId == vm.SelectedCustomerID);
decimal? updatedOutstandingAmount = c.CurrentOutStandingBalance - cp.Payment.PaymentAmount;
cp.OutstandingAmount = updatedOutstandingAmount;
c.CurrentOutStandingBalance = updatedOutstandingAmount;
_context.Add(cp);
_context.Update(c);
await _context.SaveChangesAsync();
dbContextTransaction.Commit();
TempData["status"] = "CustomerPaymentAdded";
return RedirectToAction("Payment", "Customer");
}
catch(Exception ex)
{
dbContextTransaction.Rollback();
}
};
I've got a .NET Core ASP.NET MVC 6 application and what I'm convinced is a bug in Visual Studio. If I place a break point after an await statement, the object does not show up in Locals, and I cannot mouse over to inspect. But if I use the variable, it still works fine, and it's definitely populated.
Something as simple as this:
public async Task<IActionResult> Index()
{
var location = await _listingLocationService.GetLocationByAddress("123 Fake Street");
return Content(location.Latitude.ToString() + " " +location.Longitude.ToString());
}
If I place the break point on the return statement, I cannot inspect location. It doesn't show up anywhere. I can even remove the await & place a .Result at the end, and still nothing shows. But when I continue, the view displays location.latitude, and location.longitude fine. So I know it's being populated.
For the sake of completeness, I'll include the GetLocationByAddress function as well, which does the same thing, if I place a breakpoint anywhere after the awaits, I can't inspect the variables (even the deserialized list!).
public async Task<Geolocation> GetLocationByAddress(string address)
{
using (var client = new HttpClient())
{
client.BaseAddress = new Uri("https://maps.googleapis.com/maps/api/geocode/json");
var request = new HttpRequestMessage(HttpMethod.Get, "?address=" + WebUtility.UrlEncode(address) + "&key=...");
var response = await client.SendAsync(request);
var contents = await response.Content.ReadAsStringAsync();
var locationResult = JsonConvert.DeserializeObject<GoogleLocationResult>(contents);
if (locationResult.status == "OK")
{
var result = locationResult.results.First().geometry.location;
return new Geolocation
{
Latitude = result.lat,
Longitude = result.lng
};
}
else
{
return null;
}
}
}
Well, after loosing all day with this issue I've finally found the source of the problem and also the solution...
In my case, the problem has started after updating my project to use the .net core 1.1 Preview 1 following the steps shown here: Link to MSFT .NET Blog
The problem was caused by this line of code in the project.json file:
"buildOptions": {
"debugType": "portable", <---- THIS
"preserveCompilationContext": true,
"emitEntryPoint": true
}
after setting "debugType" to "full", awaited variables started to show again while debugging.
Hope it helps to somebody outthere!
It's not a bug. When the debugger hits the return line, it's because the previous line immediately returned a Task object. Your GetLocation method isn't done running yet (which makes sense since it's making an outbound HTTP call that will absolutely take longer than the milliseconds it took to return a Task). When you hit the return line, your debugger has stopped there, but the incomplete task means that location won't be ready. A better place to put a breakpoint is probably in the return line of your GetLocation method.
And by the way, you'll get a null ref error in your action method if the call made to Google Maps fails.
I am using Quartz.Net and we regularly see misfires during development and live. Whilst this is not a problem as such we would like to enable some sort of tracing so in development it is possible to see when a misfire occurs.
Are there any events we can hook into for this purpose? Ideally I am after something like...
var factory = new StdSchedulerFactory();
var scheduler = factory.GetScheduler();
scheduler.Start();
scheduler.OnMisfire += (e) => {
Console.Out.WriteLine(e);
}
You can use a trigger listener to handle this, see Lesson 7: TriggerListeners and JobListeners.
You can use the history plugin as a reference for building your own logging.
Example
class MisfireLogger : TriggerListenerSupport
{
private readonly ILog log = LogManager.GetLogger (typeof (MisfireLogger));
public override void TriggerMisfired (ITrigger trigger)
{
log.WarnFormat("Trigger {0} misfired", trigger.Key);
}
}
scheduler.ListenerManager.AddTriggerListener (new MisfireLogger ());
So I'm trying to get my head around this new 'async' stuff in .net 4.5. I previously played a bit with async controllers and the Task Parallel Library and wound up with this piece of code:
Take this model:
public class TestOutput
{
public string One { get; set; }
public string Two { get; set; }
public string Three { get; set; }
public static string DoWork(string input)
{
Thread.Sleep(2000);
return input;
}
}
Which is used in a controller like this:
public void IndexAsync()
{
AsyncManager.OutstandingOperations.Increment(3);
Task.Factory.StartNew(() =>
{
return TestOutput.DoWork("1");
})
.ContinueWith(t =>
{
AsyncManager.OutstandingOperations.Decrement();
AsyncManager.Parameters["one"] = t.Result;
});
Task.Factory.StartNew(() =>
{
return TestOutput.DoWork("2");
})
.ContinueWith(t =>
{
AsyncManager.OutstandingOperations.Decrement();
AsyncManager.Parameters["two"] = t.Result;
});
Task.Factory.StartNew(() =>
{
return TestOutput.DoWork("3");
})
.ContinueWith(t =>
{
AsyncManager.OutstandingOperations.Decrement();
AsyncManager.Parameters["three"] = t.Result;
});
}
public ActionResult IndexCompleted(string one, string two, string three)
{
return View(new TestOutput { One = one, Two = two, Three = three });
}
This controller renders the view in 2 seconds, thanks to the magic of the TPL.
Now I expected (rather naively) that the code above would translate into the following, using the new 'async' and 'await' features of C# 5:
public async Task<ActionResult> Index()
{
return View(new TestOutput
{
One = await Task.Run(() =>TestOutput.DoWork("one")),
Two = await Task.Run(() =>TestOutput.DoWork("two")),
Three = await Task.Run(() =>TestOutput.DoWork("three"))
});
}
This controller renders the view in 6 seconds. Somewhere in the translation the code became no longer parallel. I know async and parallel are two different concepts, but somehow I thought the code would work the same. Could someone point out what is happening here and how it can be fixed?
Somewhere in the translation the code became no longer parallel.
Precisely. await will (asynchronously) wait for a single operation to complete.
Parallel asynchronous operations can be done by starting the actual Tasks but not awaiting them until later:
public async Task<ActionResult> Index()
{
// Start all three operations.
var tasks = new[]
{
Task.Run(() =>TestOutput.DoWork("one")),
Task.Run(() =>TestOutput.DoWork("two")),
Task.Run(() =>TestOutput.DoWork("three"))
};
// Asynchronously wait for them all to complete.
var results = await Task.WhenAll(tasks);
// Retrieve the results.
return View(new TestOutput
{
One = results[0],
Two = results[1],
Three = results[2]
});
}
P.S. There's also a Task.WhenAny.
No, you stated the reason that this is different already. Parallel and Async are two different things.
The Task version works in 2 seconds because it runs the three operations at the same time (as long as you have 3+ processors).
The await is actually what it sounds like, the code will await the execution of the Task.Run before continuing to the next line of code.
So, the big difference between the TPL version and the async version are that the TPL version runs in any order because all of the tasks are independent of each other. Whereas, the async version runs in the order that the code is written. So, if you want parallel, use the TPL, and if you want async, use async.
The point of async is the ability to write synchronous looking code that will not lock up a UI while a long running action is happening. However, this is typically an action that all the processor is doing is waiting for a response. The async/await makes it so that the code that called the async method will not wait for the async method to return, that is all. So, if you really wanted to emulate your first model using async/await (which I would NOT suggest), you could do something like this:
MainMethod()
{
RunTask1();
RunTask2();
RunTask3();
}
async RunTask1()
{
var one = await Task.Factory.StartNew(()=>TestOutput.DoWork("one"));
//do stuff with one
}
async RunTask2()
{
var two= await Task.Factory.StartNew(()=>TestOutput.DoWork("two"));
//do stuff with two
}
async RunTask3()
{
var three= await Task.Factory.StartNew(()=>TestOutput.DoWork("three"));
//do stuff with three
}
The code path will go something like this (if the tasks are long running)
main call to RunTask1
RunTask1 awaits and returns
main call to RunTask2
RunTask2 awaits and returns
main call to RunTask3
RunTask3 awaits and returns
main is now done
RunTask1/2/3 returns and continues doing something with one/two/three
Same as 7, except less the one that already completed
Same as 7, except less the two that already completed
****A big disclaimer about this, though. Await will run synchronously if the task is already completed by the time that the await is hit. This saves the runtime from having to perform its vudu :) since it is not needed. This will make the code flow above incorrect as the flow is now synchronous****
Eric Lippert's blog post on this explains things much better than I am doing :)
http://blogs.msdn.com/b/ericlippert/archive/2010/10/29/asynchronous-programming-in-c-5-0-part-two-whence-await.aspx
Hopefully, that helps dispel some of your questions about async versus TPL? The biggest thing to take away is that async is NOT parallel.
I using a wrapper to the WCL BT library. Since the app is in .NET 4 and the license we have to the wrapper is in .NET 2, this is a sloppy workaround.
Using this wrapper with a .NET 4 WPF application works fine as long as the useLegacyV2RuntimeActivationPolicy is on. More on this here. It takes around 22 seconds firing the OnDiscoveryComplete event.
But when using the same wrapper with an ASP NET MVC 3 application, the OnDiscoveryComplete event on the library is never fired. Anyone knows why?
The wrapper is called on buttonClickedEvent on the WPF app and on a SearchAsync action on an AsyncController on the MVC app.
The relevant code is here:
Calling the wrapper:
var wrapper = new Wrapper();
wrapper.Search();
Wrapper:
public Wrapper() {
_wclApi = new wclAPI();
_wclApi.Load();
_btDiscovery = new wclBluetoothDiscovery();
_btDiscovery.OnDiscoveryStarted += BtDiscoveryOnDiscoveryStarted;
_btDiscovery.OnDiscoveryComplete += BtDiscoveryOnDiscoveryComplete;
}
public void Search() {
var radios = new wclBluetoothRadios();
var ret = _btDiscovery.EnumRadios(radios);
if (ret == 0) {
wclBluetoothRadio radio = radios[0];
_btDiscovery.Discovery(radio, 0x15);
}
}
private void BtDiscoveryOnDiscoveryComplete(object sender, wclBluetoothDiscoveryCompleteEventArgs e) {
// handle devices found
}
There is probably an exception that the wclBluetoothDiscovery is encountering but not properly exposing to you. It's a common bug in async APIs to not inform callers when bad things happen... the proper API design would be to raise the *Complete event but then raise the exception to you when you query the completed event args.