I'm trying to make an async workflow, where there's a main async loop, which executes an async sub-block in each loop. And I want this async sub-block to be cancellable, but when it cancels then I don't want the main loop to cancel. I want it to continue, at the line after the do! subBlock.
The only method I see in Async that even has an acceptable signature (takes CancellationToken, returns something that can be converted to async) is Async.StartAsTask, but that seems to hang when canceled; in the below, it prints "cancelled" and then nothing else.
open System
open System.Threading
open System.Threading.Tasks
// runs until cancelled
let subBlock =
async {
try
while true do
printfn "doing it"
do! Async.Sleep 1000
printfn "did it"
finally
printfn "cancelled!"
}
[<EntryPoint>]
let main argv =
let ctsRef = ref <| new CancellationTokenSource()
let mainBlock =
//calls subBlock in a loop
async {
while true do
ctsRef := new CancellationTokenSource()
do! Async.StartAsTask(subBlock, TaskCreationOptions.None, (!ctsRef).Token)
|> Async.AwaitTask
printfn "restarting"
}
Async.Start mainBlock
//loop to cancel CTS at each keypress
while true do
Console.ReadLine() |> ignore
(!ctsRef).Cancel()
0
Is there any way to do this?
Whether the caller that starts and cancels the worker is an async too doesn't really affect this problem, since the worker is managed via its explicitly specified cancellation token.
Asyncs have three continutations: the normal one, which can return a value, one for exceptions, and one for cancellation. There are multiple ways to add a cancellation continuation to an async, such as Async.OnCancel, Async.TryCancelled, or the general Async.FromContinuations, which includes the exception case. Here's a program that has the desired output:
let rec doBlocks () =
async { printfn "doing it"
do! Async.Sleep 1000
printfn "did it"
do! doBlocks () }
let rec runMain () =
use cts = new CancellationTokenSource()
let worker = Async.TryCancelled(doBlocks (), fun _ -> printfn "Cancelled")
Async.Start(worker, cts.Token)
let k = Console.ReadKey(true)
cts.Cancel()
if k.Key <> ConsoleKey.Q then runMain ()
This works just as well if runMain is an async. In this simple case, you could also just have it print the "cancelled" message itself.
I hope this helps. I don't think there is a general answer to how to structure the program; that depends on the concrete use case.
What happens here is that when your child task is cancelled, the OperationCanceledException brings down your mainBlock as well. I was able to get it to work by using this:
let rec mainBlock =
async {
ctsRef := new CancellationTokenSource()
let task = Async.StartAsTask(subBlock, TaskCreationOptions.None, (!ctsRef).Token) |> Async.AwaitTask
do! Async.TryCancelled(task, fun e ->
(!ctsRef).Dispose()
printfn "restarting"
Async.Start mainBlock)
}
When the task is cancelled, mainBlock is explicitly restarted in the cancelation handler. You need to add #nowarn "40" for it since mainBlock is used inside its definition. Also note the dispose on token source.
You can find more information on this problem (and perhaps a nicer solution in the form of StartCatchCancellation) in these two threads.
Related
If we have a function f that returns Async<something>, we can ignore the return value with do! f() |> Async.Ignore. Is there a similar construct for ignoring Tasks? Right now, I'm having to do something like this:
task {
let! result = SomeOperation()
result |> ignore
}
Technically I could do |> Async.AwaitTask |> Async.Ignore, but I'm not sure how wasteful it is to convert a task to async just to ignore it...
If it matters, I'm using task{} computation expressions from Ply.
You can wait for a task to complete using the Result property:
let t =
task {
let! _ = SomeOperation()
()
}
let _ = t.Result
If you don't care for the return value, then you don't have to wait for the task to finish, you can just ignore the whole task:
SomeOperation() |> ignore
If your function returns Async, then simply ignoring it won't work because unlike Tasks, an Async operation does not execute unless you explicitly start it.
If you just want to make it run, but don't care about waiting until it is done you can do
f() |> Async.RunAsTask |> ignore
if you do want to wait for the result, you can do
f() |> Async.RunSynchronously |> ignore
Background.
I am trying to figure out MailboxProcessor. The idea is to use it as a some kind of state machine and pass arguments around between the states and then quit. Some parts are going to have async communication so I made a Sleep there.
It's a console application, making a Post does nothing because main thread quits and kills everything behind it. I am making a PostAndReply in main.
Also, I have tried without
let sleepWorkflow = async
, doesn't make any difference.
Questions.
(I am probably doing something wrong)
Go24 is not async. Changing RunSynchronously to StartImmediate makes no visible difference. The end should be somewhere below GetMe instead. At the same time Done is printed after Fetch. Isn't the control supposed t be returned to the main thread on sleep?
Go24, wait
go24 1, end
Fetch 1
Done
GetMe
...
Run time is terrible slow. Without delay in Fetch it's about 10s (stopwatch). I thought F# threads are lightweight and should use threadpool.
According to debugger it takes appr 1s to create every and it looks like real threads.
Also, changing to [1..100] will "pause" the program for 100s, according to ProcessExplorer 100 threads are created during that time and only then everything is printed. I would actually prefer fewer threads and slow increase.
Code.
Program.fs
[<EntryPoint>]
let main argv =
let a = Mailbox.MessageBasedCounter.DoGo24 1
let a = Mailbox.MessageBasedCounter.DoFetch 1
let b = Mailbox.MessageBasedCounter.GetMe
let task i = async {
//Mailbox.MessageBasedCounter.DoGo24 1
let a = Mailbox.MessageBasedCounter.DoFetch i
return a
}
let stopWatch = System.Diagnostics.Stopwatch.StartNew()
let x =
[1..10]
|> Seq.map task
|> Async.Parallel
|> Async.RunSynchronously
stopWatch.Stop()
printfn "%f" stopWatch.Elapsed.TotalMilliseconds
printfn "a: %A" a
printfn "b: %A" b
printfn "x: %A" x
0 // return an integer exit code
Mailbox.fs
module Mailbox
#nowarn "40"
type parserMsg =
| Go24 of int
| Done
| Fetch of int * AsyncReplyChannel<string>
| GetMe of AsyncReplyChannel<string>
type MessageBasedCounter () =
/// Create the agent
static let agent = MailboxProcessor.Start(fun inbox ->
// the message processing function
let rec messageLoop() = async{
let! msg = inbox.Receive()
match msg with
| Go24 n ->
let sleepWorkflow = async{
printfn "Go24, wait"
do! Async.Sleep 4000
MessageBasedCounter.DoDone() // POST Done.
printfn "go24 %d, end" n
return! messageLoop()
}
Async.RunSynchronously sleepWorkflow
| Fetch (i, repl) ->
let sync = async{
printfn "Fetch %d" i
do! Async.Sleep 1000
repl.Reply( "Reply Fetch " + i.ToString() ) // Reply to the caller
return! messageLoop()
}
Async.RunSynchronously sync
| GetMe (repl) ->
let sync = async{
printfn "GetMe"
repl.Reply( "GetMe" ) // Reply to the caller
return! messageLoop()
}
Async.RunSynchronously sync
| Done ->
let sync = async{
printfn "Done"
return! messageLoop()
}
Async.RunSynchronously sync
}
// start the loop
messageLoop()
)
// public interface to hide the implementation
static member DoDone () = agent.Post( Done )
static member DoGo24 (i:int) = agent.Post( Go24(i) )
static member DoFetch (i:int) = agent.PostAndReply( fun reply -> Fetch(i, reply) )
static member GetMe = agent.PostAndReply( GetMe )
I'm not necessarily sure that this is the main problem, but the nested asyncs and Async.RunSynchrously in the agent code look suspicious.
You do not need to create a nested async - you can just call asynchronous operations in the body of the match clauses directly:
// the message processing function
let rec messageLoop() = async{
let! msg = inbox.Receive()
match msg with
| Go24 n ->
printfn "Go24, wait"
do! Async.Sleep 4000
MessageBasedCounter.DoDone()
printfn "go24 %d, end" n
return! messageLoop()
| Fetch (i, repl) ->
(...)
Aside from that, it is important to understand that the agent has exactly one instance of the body computation running. So, if you block the body of the agent, all other operations will be queued.
If you want to start some task (like the synchronous operations) in the background and resume the agent immediately, you can use Async.Start inside the body (but be sure to call the main loop recursively in the main part of the body):
| Go24 n ->
// Create work item that will run in the background
let work = async {
printfn "Go24, wait"
do! Async.Sleep 4000
MessageBasedCounter.DoDone()
printfn "go24 %d, end" n }
// Queue the work in a thread pool to be processed
Async.Start(work)
// Continue the message loop, waiting for other messages
return! messageLoop()
Experimenting some with F# async taught me that I can StartImmediate on the current running thread. This allows me, it seems, to run an async expression which can still pass out control, whenever getting inside of it to some async operation (e.g. do!), to the code outside of the async expression. Please see the simple example below:
open System.Threading
let workThenWait() = async {
printfn "async start"
do! Async.Sleep(1000)
printfn "async end"
}
let demo() =
workThenWait() |> Async.StartImmediate
printfn "main started"
// here I want to wait to the async expression in case it has passed control
printfn "main end"
demo()
The result we get is:
async start
main started
main end
async end
On the other hand, if I execute the same async expression (in this case workThenWait) using StartAsTask (inside demo), I can potentially wait on it at the end.
MY QUESTION is:
using the previous example using StartImmediate, can I run on the same thread, but ALSO wait at the end for the async expression in case some async operation (such as do!) is called and passes control forward?
I think You need Async.RunSynchronously (http://msdn.microsoft.com/en-us/library/ee370262.aspx)
update:
Ok, now I understand better what You want, and I was able to achieve this with Async.StartWithContinuations method.
Here's the code:
open System.Threading
let f() =
printfn "main thread: %A" Thread.CurrentThread.ManagedThreadId
let c1 =
async {
printfn "c1 async thread: %A" Thread.CurrentThread.ManagedThreadId
do! Async.Sleep(1000)
return "some result"
}
let continuation s =
printfn "continuation thread: %A" Thread.CurrentThread.ManagedThreadId
printfn "now the code You want after waiting and the result %s" s
Async.StartWithContinuations(
c1,
continuation,
(fun _ -> ()),
(fun _ -> ())
)
printfn "Code that runs during async computation"
Now this is definitely not very readable as the flow of the code is not obvious. I couldn't find any better solution.
You can do this with Hopac libary:
let workThenWait() = job {
printfn "async start"
do! Hopac.Timer.Global.sleep (TimeSpan.FromMilliseconds 1000.)
printfn "async end"
}
let demo() =
let promise = workThenWait() |> Promise.start |> run
printfn "main started"
// here I want to wait to the async expression in case it has passed control
let result = run promise
printfn "main end"
demo()
Hopac is both more performant and functional than async and is little known compared to how good it is. I highly recommend it.
I have problems with seemingly inconsistent behavior when cancelling different kinds of Asyncs.
To reproduce the problem, let's says there is a function that takes a list of "jobs" (Async<_> list), waits for them to complete and prints their results. The function also gets a cancellation token so it can be cancelled:
let processJobs jobs cancel =
Async.Start(async {
try
let! results = jobs |> Async.Parallel
printfn "%A" results
finally
printfn "stopped"
}, cancel)
The function is called like that:
let jobs = [job1(); job2(); job3(); job4(); job5()]
use cancel = new CancellationTokenSource()
processJobs jobs cancel.Token
And somewhat later it is cancelled:
Thread.Sleep(1000)
printfn "cancelling..."
cancel.Cancel()
When the cancellation token source is cancelled, the function should execute the finally-block and print "stopped".
That works fine for job1, 2 and 3, but doesn't work when there is a job4 or job5 in the list.
Job1 just Async.Sleeps:
let job1() = async {
do! Async.Sleep 1000000
return 10
}
Job2 starts some async childs and waits for them:
let job2() = async {
let! child1 = Async.StartChild(async {
do! Async.Sleep 1000000
return 10
})
let! child2 = Async.StartChild(async {
do! Async.Sleep 1000000
return 10
})
let! results = [child1; child2] |> Async.Parallel
return results |> Seq.sum
}
Job3 waits for some ugly wait handle that's set by some even uglier thread:
let job3() = async {
use doneevent = new ManualResetEvent(false)
let thread = Thread(fun () -> Thread.Sleep(1000000); doneevent.Set() |> ignore)
thread.Start()
do! Async.AwaitWaitHandle(doneevent :> WaitHandle) |> Async.Ignore
return 30
}
Job4 posts to and waits for a reply from a MailboxProcessor:
let job4() = async {
let worker = MailboxProcessor.Start(fun inbox -> async {
let! (msg:AsyncReplyChannel<int>) = inbox.Receive()
do! Async.Sleep 1000000
msg.Reply 40
})
return! worker.PostAndAsyncReply (fun reply -> reply) // <- cannot cancel this
}
Job5 waits for a Task (or TaskCompletionSource):
let job5() = async {
let tcs = TaskCompletionSource<int>()
Async.Start(async {
do! Async.Sleep 1000000
tcs.SetResult 50
})
return! (Async.AwaitTask tcs.Task) // <- cannot cancel this
}
Why can Job1, 2 and 3 be cancelled ("stopped" gets printed), while Job4 and 5 make the function hang "forever"?
So far I always relied on F# to handle cancellation behind the scenes - as long as I'm in async-blocks and use !s (let!, do!, return!,...) everything should be fine.. but that doesn't seem to be the case all the time.
Quote:
In F# asynchronous workflows, the CancellationToken object is passed
around automatically under the cover. This means that we don't have to
do anything special to support cancellation. When running asynchronous
workflow, we can give it cancellation token and everything will work
automatically.
Complete code is available here: http://codepad.org/euVO3xgP
EDIT
I noticed that piping an async through Async.StartAsTask followed by Async.AwaitTask makes it cancelable in all cases.
i.e. for Job4 that means changing the line:
return! worker.PostAndAsyncReply (fun reply -> reply)
to:
return! cancelable <| worker.PostAndAsyncReply (fun reply -> reply)
With cancelable being:
let cancelable (x:Async<_>) = async {
let! cancel = Async.CancellationToken
return! Async.StartAsTask(x, cancellationToken = cancel) |> Async.AwaitTask
}
The same works for making Job5 cancelable.
But.. that's just a workaround and I can hardly put that around each call to an unknown Async<_>.
Only the Async. methods handle using the default CancellationToken themselves.
In your MailboxProcessor example the cancel should go on the Start method
let! ct= Async.CancellationToken
use worker := MailboxProcessor.Start( theWork, ct)
In the TaskCompletionSource example, you are going to have to register a callback to cancel it.
let! ct = Async.CancellationToken
use canceler = ct.Register( fun () -> tcs.TrySetCanceled() |> ignore )
Consider the following definition
let test =
Async.FromContinuations(
fun (cont,econt,ccont) ->
let rec inner () =
async {
do printfn "looping..."
do! Async.Sleep 1000
return! inner ()
}
Async.Start(inner ())
cont ())
Suppose I want to try the computation like so
let cts = new CancellationTokenSource ()
Async.Start(test, cts.Token)
cts.Cancel()
This will naturally not make the inner loop stop, since I have not passed the suitable cancellation token. Is there any way I can obtain the outer cancellation token through Async.FromContinuations? I could rewrite this using the async builder and Async.CancellationToken, but then I would lose the ability to pass continuations to the inner expression.
Can you describe what are you trying to do? If I understand your code correctly, you want to start the inner loop function in the background and then, in parallel, continue running the rest of the workflow (using the cont() call).
To do this, you do not need Async.FromContinuations. There is a function that does exactly this and it also takes care of handling exceptions, cancellation tokens etc.
I think you could rewrite your program like this:
let test =
// The inner loop function from your example
let rec inner () = async {
do printfn "looping..."
do! Async.Sleep 1000
return! inner () }
async {
// Start the inner loop as a child async
let! _ = Async.StartChild(inner())
// ... continue doing other things in parllel if you wish
do printfn "main body running..." }
Starting and cancelling of the computation looks as before:
let cts = new CancellationTokenSource ()
Async.Start(test, cts.Token)
// This will cancel the 'inner' loop as well
cts.Cancel()
If you call Async.StartChild using let! then it will start the inner task, pass it the cancellation token etc. It returns a token that you can use later to wait until the child task completes, but since you're not doing that, I used the _ pattern.
smth like this?
let test =
async {
let! ct = Async.CancellationToken
return! Async.FromContinuations(
fun (cont,econt,ccont) ->
let rec inner () =
async {
do printfn "looping..."
do! Async.Sleep 1000
return! inner ()
}
Async.Start(inner (), cancellationToken = ct)
cont ())
}
let cts = new CancellationTokenSource ()
Async.Start(test, cts.Token)
cts.CancelAfter(1000)