I'm calling HttpClient via Async.AwaitTask, being called from within an agent (MailboxProcessor). I was wanting to catch errors during the HTTP call so used a try...with in the async workflow, but it completely misses catching client-side timeout exceptions which then cause the agent to crash.
Minimal reproduction:
#r "System.Net.Http"
open System
open System.Net.Http
let client = new HttpClient()
client.Timeout <- TimeSpan.FromSeconds(1.)
async {
try
let! content = Async.AwaitTask <| client.GetStringAsync("http://fake-response.appspot.com/?sleep=30")
return content
with ex ->
// Does not catch client-side timeout exception
return "Caught it!"
}
|> Async.RunSynchronously
// Throws System.OperationCanceledException: The operation was canceled
I can fix it by making it completely syncronous, but would prefer to keep the whole stack async as might be running a lot of these in parallel:
#r "System.Net.Http"
open System
open System.Net.Http
let client = new HttpClient()
client.Timeout <- TimeSpan.FromSeconds(1.)
try
Async.AwaitTask <| client.GetStringAsync("http://fake-response.appspot.com/?sleep=30")
|> Async.RunSynchronously
with ex ->
"Caught it!"
// Returns "Caught it!"
Is there an effective way of catching the OperationCanceledException within an async context?
This happens because the HttpClient.GetStringAsync task will be cancelled, instead of failing with a TimeoutException, thus prompting the async mechanism to fire its cancellation continuations, which cannot be handled. A simple way to solve this problem is the following:
async {
try
let! content =
client.GetStringAsync("http://fake-response.appspot.com/?sleep=30")
.ContinueWith(fun (t:Task<string>) -> t.Result)
|> Async.AwaitTask
return content
with ex ->
// Does not catch client-side timeout exception
return "Caught it!"
}
Related
I have integration test with docker using test containers. On container I run jms. In test I am putting message on queue.
How I can wait in test to make it populated on jms?
On local machine it works, but on jenkins it fails, so I have to add
Thread.sleep(3000);
but this is nasty.
org.awaitility seems to be missed usage:
await().atMost(2, TimeUnit.SECONDS).until(() -> return true));
I just need to do a pause to make jms propagate (put on jms queue) and wait for listener to act, which is putting message to database.
Then I have to call get rest endpoint to see it worked.
With topic it would be easier, because I would create test listener on topic.
But it is queue, there can be on listener that will get message.
Use org.awaitility with a JMS QueueBrowser, e.g.:
#Test
public void myTest() throws Exception {
...
await().atMost(2, TimeUnit.SECONDS).until(() -> return queueIsEmpty(queueName)));
...
}
private boolean queueIsEmpty(String queueName) {
ConnectionFactory cf = new MyBrokersConnectionFactory();
Connection connection = cf.createConnection();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
QueueBrowser browser = session.createBrowser(session.createQueue(queueName));
Enumeration enumeration = senderBrowser.getEnumeration();
while (enumeration.hasMoreElements()) {
return false;
}
return true;
}
A QueueBrowser is read only so there is no danger that it will actually consume the message.
Another potential option would be to create a consumer with a transacted session and then try to receive the message. If you actually did receive a message you could rollback the transaction and close the consumer.
Use retries (e.g. Spring RetryTemplate or Failsafe Retry Policy) to improve integration test execution time:
Retry the SQL query until record is present
Retry the REST endpoint until it is successful
Here an example to wait for a DB record; tweak the policies to your needs:
RetryTemplate retryTemplate = new RetryTemplate();
retryTemplate.setBackOffPolicy(new FixedBackOffPolicy());
retryTemplate.setRetryPolicy(new SimpleRetryPolicy(
10, Collections.singletonMap(AssertionError.class, true)));
retryTemplate.execute(retryContext -> {
List<MyRecord> records = jdbcTemplate.query("select ...");
Assert.assertEquals(1, records.size());
return null;
});
My solution is to use org.awaitility lib and replace asserts with return statement:
await().atMost(30, TimeUnit.SECONDS).until(
() -> {
//
// assertTrue(condition);
return condition == true;
}
I've been working with Serilog.Sinks.Email, and I notice that (other than the fact that I don't get an email), there is no exception or anything that indicates a failure when I execute the Log statement. Even if I put in junk for the MailServer, the Log statement executes as though it were successful.
I've tried the SelfLog statement, but see nothing in my Output window from that.
Serilog.Debugging.SelfLog.Enable(msg => Debug.WriteLine(msg));
I've added SymbolSource (http://www.symbolsource.org/Public/Wiki/Using), and am able to step through Serilog's code, but once again, no exceptions are caught.
Does Serilog really give no indication of error, or am I missing something?
Using: .NET Framework 4.6.2, Serilog 2.8, Serilog.Sinks.Email 2.3
Sample Code:
Log.Logger = new LoggerConfiguration()
.WriteTo.Email(new EmailConnectionInfo
{
FromEmail = "xxx",
ToEmail = "xxx",
MailServer = "smtp.xxx.com",
EmailSubject = "My Test"
}
).CreateLogger();
Log.ForContext<Program>().Error("Test Number {Parm}", "1");
Writing to a Serilog logger is supposed to be a safe operation and never throw exceptions, by design, thus any exceptions that happen when sending the e-mail would only appear in the SelfLog - in your case, it would write to the Debug console.
Now, the reason you're not even seeing exceptions in the SelfLog is because e-mails sent via Serilog.Sinks.Email are sent asynchronously, and your program is terminating before the the Sink had the opportunity to send the e-mail.
Before your program terminates, you need to call Log.CloseAndFlush() so that all logs can be sent, before the app closes.
static void Main(string[] args)
{
Serilog.Debugging.SelfLog.Enable(msg => Console.WriteLine(msg));
Log.Logger = new LoggerConfiguration()
.WriteTo.Email(...)
.CreateLogger();
try
{
Log.ForContext<Program>().Error("Test Number {Parm}", "1");
// ...
}
finally
{
Log.CloseAndFlush();
}
}
That will allow you to see the error messages you're writing via SelfLog.
You can read more about that in the documentation: Lifecycle of Loggers.
ps: If the operation you're logging is important enough that you'd want to guarantee it succeeds (or throw exception if it doesn't) then you should use Audit Logging, i.e. Use .AuditTo.Email(...) instead of .WriteTo.Email(...)
I am trying to get answer from Isolate as list. I wrote next code. The problem That it's not working. It's simply wait.
main() async
{
ReceivePort receivePort = ReceivePort();
Isolate.spawn(echo, receivePort.sendPort);
var d = await receivePort.toList();
receivePort.close();
print(d);
}
echo(SendPort sendPort) async
{
ReceivePort receivePort = ReceivePort();
sendPort.send("message");
}
The receivePort is a stream. When you call toList on the stream, it waits for the stream to complete. That never happens, so the toList call stalls forever.
If you know that the other end of the communication only sends one message, then you can instead do var d = await receivePort.first;. This only waits for the first message.
In general, when doing isolate communication, the sending isolate should send a special message when they are done, because the receiving isolate has no other way to know that. That is: You want a communication protocol, so the receiver can know whether there are going to be more messages. Maybe each message is an object which contains an isLast boolean, or the last message is null, or you wrap all messages in subclasses of a Message class that defines your protocol. What to do depends on the actual use-case.
The only rule is: You have to say when you are done.
You should use .listen() instead of await for, if you want to see output of the list before is stream closed. listen will register the handler and execution keeps continue.
instead of
var d = await receivePort.toList();
receivePort.close();
print(d);
try
receivePort.listen((data) => {print(data)});
You can find differences with listen and await here
I am using HttpClient to make a request to an api. This code is located in a class library project shared with two aditional projects, a Console and a Asp.Net Mvc project. When I make a request from the Console project it works great, but in the asp project it blocks in the line
using(Stream responseStream = await response.Content.ReadAsStreamAsync()
this is my request code
private async Task<dynamic> ReadJson(string url)
{
HttpResponseMessage response = await httpClient.GetAsync(url);
if (response.StatusCode == System.Net.HttpStatusCode.NoContent)
throw new RateLimitException();
if (response.StatusCode == System.Net.HttpStatusCode.Forbidden)
throw new AccessDeniedException();
if (response.StatusCode != System.Net.HttpStatusCode.OK)
throw new Exception("Error: " + response.StatusCode);
using (Stream responseStream = await response.Content.ReadAsStreamAsync())
using (StreamReader sr = new StreamReader(responseStream, System.Text.Encoding.UTF8))
{
string json = sr.ReadToEnd();
return JObject.Parse(json);
}
}
I am making the same call to the method from the Console and the Asp.Net project. From the console works but the asp .net project blocks in the line when reads the response content
Most probably this deadlock occurs because the controller action that calls ReadJson function is synchronous. You need to make the action async. You can find an excellent explanation of this deadlock here. (All the credits go to Stephen Cleary)
Quick summary is:
/ My "library" method.
public static async Task<JObject> GetJsonAsync(Uri uri)
{
using (var client = new HttpClient())
{
var jsonString = await client.GetStringAsync(uri);
return JObject.Parse(jsonString);
}
}
// My "top-level" method.
public class MyController : ApiController
{
public string Get()
{
var jsonTask = GetJsonAsync(...);
return jsonTask.Result.ToString();
}
}
What Causes the Deadlock
The top-level method calls GetJsonAsync (within the UI/ASP.NET context). GetJsonAsync starts the REST request by calling
HttpClient.GetStringAsync (still within the context).
GetStringAsync returns an uncompleted Task, indicating the REST request is not complete.
GetJsonAsync awaits the Task returned by GetStringAsync. The context is captured and will be used to continue running the
GetJsonAsync method later. GetJsonAsync returns an uncompleted
Task, indicating that the GetJsonAsync method is not complete.
The top-level method synchronously blocks on the Task returned by GetJsonAsync. This blocks the context thread.
… Eventually, the REST request will complete. This completes the Task that was returned by GetStringAsync.
The continuation for GetJsonAsync is now ready to run, and it waits for the context to be available so it can execute in the context.
Deadlock. The top-level method is blocking the context thread, waiting for GetJsonAsync to complete, and GetJsonAsync is waiting for
the context to be free so it can complete.
Preventing the Deadlock
There are two best practices that avoid this situation:
In your “library” async methods, use ConfigureAwait(false) wherever possible.
Don’t block on Tasks; use async all the way down.
With C# I can use the System.Net.Http Library to post a stream directly like so:
private async Task UploadFileAsync(Uri uri, string filename)
{
using (var stream = File.OpenRead(filename))
using (var client = new HttpClient())
{
await client.PostAsync(uri, new StreamContent(stream));
}
}
Is there a way to do this in F# in a functionally idiomatic way?
FSharp.Data.Http.Request lets you post Binary data but requires a byte array to do so, which means reading the stream into memory. I'd like to avoid this as I am posting > 20Mb.
Doing this in a functionally idiomatic way would involve some kind of IO monad. As F# is a mixed paradigm language e.g.
let uploadFile (uri: Uri) fileName =
async {
use stream = File.OpenRead fileName
use client = new HttpClient()
return! client.PostAsync(uri, new StreamContent(stream)) |> Async.AwaitTask
}
is perfectly acceptable IMHO.