A ZMQ subscriber socket keeps only the last message in queue when the CONFLATE option is set to true. (zmq_docs) However, it does not seem to be working for me. Typically in python i would do something like the following and it would work:
context = zmq.Context()
subscriber = context.socket(zmq.SUB)
subscriber.setsockopt(zmq.CONFLATE, 1)
subscriber.connect("tcp://localhost:5555")
The subscriber in pub/sub pattern below simply ignores the CONFLATE option set to 1. You can observe by the minute and second being displayed on the subscriber that the program is tied up as it is processing old messages (fib 42). If fib is set to a trivial value, you can see that the subscriber is indeed receiving messages from the publisher.
Here is the publisher:
open System
open fszmq
open fszmq.Context
open fszmq.Socket
let funcPublish () =
use context = new Context()
use publisher = pub context
"tcp://*:5563" |> bind publisher
while true do
let tm = System.DateTime.Now
let t = String.Concat([tm.Minute.ToString(); " "; tm.Second.ToString()])
t |> s_send publisher
sleep 1
EXIT_SUCCESS
[<EntryPoint>]
let main argv =
funcPublish ()
0
And here is the subscriber:
open fszmq
open fszmq.Context
open fszmq.Socket
let rec fib n =
match n with
| 1 | 2 -> 1
| n -> fib(n-1) + fib(n-2)
let funcSubscribe () =
use context = new Context()
use subscriber = sub context
"tcp://localhost:5563" |> connect subscriber
Socket.setOption subscriber (ZMQ.CONFLATE, 1)
[ ""B ] |> subscribe subscriber
while true do
let contents = s_recv subscriber
fib 42
contents |> printfn "%A"
EXIT_SUCCESS
[<EntryPoint>]
let main argv =
funcSubscribe ()
0
Thanks.
I noticed one difference between your Python code and your F# code. In Python, you set the CONFLATE option before you connect to the socket, but in F#, you set the CONFLATE option after you connect.
If you move the Socket.setOption line to before your connect subscriber line in your F# code, I suspect that should solve your problem.
In other words, instead of this:
"tcp://localhost:5563" |> connect subscriber
Socket.setOption subscriber (ZMQ.CONFLATE, 1)
you should probably have this:
Socket.setOption subscriber (ZMQ.CONFLATE, 1)
"tcp://localhost:5563" |> connect subscriber
Related
How do I implement Failover within an Akka.NET cluster using the Akka.FSharp API?
I have the following cluster node that serves as a seed:
open Akka
open Akka.FSharp
open Akka.Cluster
open System
open System.Configuration
let systemName = "script-cluster"
let nodeName = sprintf "cluster-node-%s" Environment.MachineName
let akkaConfig = Configuration.parse("""akka {
actor {
provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
}
remote {
log-remote-lifecycle-events = off
helios.tcp {
hostname = "127.0.0.1"
port = 2551
}
}
cluster {
roles = ["seed"] # custom node roles
seed-nodes = ["akka.tcp://script-cluster#127.0.0.1:2551"]
# when node cannot be reached within 10 sec, mark is as down
auto-down-unreachable-after = 10s
}
}""")
let actorSystem = akkaConfig |> System.create systemName
let clusterHostActor =
spawn actorSystem nodeName (fun (inbox: Actor<ClusterEvent.IClusterDomainEvent>) ->
let cluster = Cluster.Get actorSystem
cluster.Subscribe(inbox.Self, [| typeof<ClusterEvent.IClusterDomainEvent> |])
inbox.Defer(fun () -> cluster.Unsubscribe(inbox.Self))
let rec messageLoop () =
actor {
let! message = inbox.Receive()
// TODO: Handle messages
match message with
| :? ClusterEvent.MemberJoined as event -> printfn "Member %s Joined the Cluster at %O" event.Member.Address.Host DateTime.Now
| :? ClusterEvent.MemberLeft as event -> printfn "Member %s Left the Cluster at %O" event.Member.Address.Host DateTime.Now
| other -> printfn "Cluster Received event %O at %O" other DateTime.Now
return! messageLoop()
}
messageLoop())
I then have an arbitrary node that could die:
open Akka
open Akka.FSharp
open Akka.Cluster
open System
open System.Configuration
let systemName = "script-cluster"
let nodeName = sprintf "cluster-node-%s" Environment.MachineName
let akkaConfig = Configuration.parse("""akka {
actor {
provider = "Akka.Cluster.ClusterActorRefProvider, Akka.Cluster"
}
remote {
log-remote-lifecycle-events = off
helios.tcp {
hostname = "127.0.0.1"
port = 0
}
}
cluster {
roles = ["role-a"] # custom node roles
seed-nodes = ["akka.tcp://script-cluster#127.0.0.1:2551"]
# when node cannot be reached within 10 sec, mark is as down
auto-down-unreachable-after = 10s
}
}""")
let actorSystem = akkaConfig |> System.create systemName
let listenerRef =
spawn actorSystem "temp2"
<| fun mailbox ->
let cluster = Cluster.Get (mailbox.Context.System)
cluster.Subscribe (mailbox.Self, [| typeof<ClusterEvent.IMemberEvent>|])
mailbox.Defer <| fun () -> cluster.Unsubscribe (mailbox.Self)
printfn "Created an actor on node [%A] with roles [%s]" cluster.SelfAddress (String.Join(",", cluster.SelfRoles))
let rec seed () =
actor {
let! (msg: obj) = mailbox.Receive ()
match msg with
| :? ClusterEvent.MemberRemoved as actor -> printfn "Actor removed %A" msg
| :? ClusterEvent.IMemberEvent -> printfn "Cluster event %A" msg
| _ -> printfn "Received: %A" msg
return! seed () }
seed ()
What is the recommended practice for implementing failover within a cluster?
Specifically, is there a code example of how a cluster should behave when one of its nodes is no longer available?
Should my cluster node spin-up a replacement or is there a different behavior?
Is there a configuration that automatically handles this that I can just set without having to write code?
What code would I have to implement and where?
First of all it's a better idea to rely on MemberUp and MemberRemoved events (both implementing ClusterEvent.IMemberEvent interface, so subscribe for it), as they mark phases, when node joining/leaving procedure has been completed. Joined and left events doesn't necessarily ensure that node is fully operable at signaled point in time.
Regarding failover scenario:
Automatic spinning of the replacements can be done via Akka.Cluster.Sharding plugin (read articles 1 and 2 to get more info about how does it work). There's no equivalent in Akka.FSharp for it, but you may use Akkling.Cluster.Sharding plugin instead: see example code.
Another way is to create replacement actors up-front on each of the nodes. You can route messages to them by using clustered routers or distributed publish/subscribe. This is however more a case in situation, when you have stateless scenarios, so that each actor is perfectly able to pick up work of another actor at any time. This is more general solution for distributing work among many actors living on many different nodes.
You may also set watchers over processing actors. By using monitor function, you may order your actor to watch over another actor (no matter where does it live). In case of node failure, info about dying actor will be send in form of Terminated message to all of its watchers. This way you may implement your own logic i.e. recreating actor on another node. This is actually the most generic way, as it doesn't use any extra plugins or configuration, but the behavior needs to be described by yourself.
Issue:
I am struggling to understand why my Reporter actor is not receiving messages based on the following statement that's in my Generator actor:
reporter <! Message input
My reporter actor is the following:
let reporterActor (mailbox:Actor<_>) =
let rec loop() = actor { let! msg = mailbox.Receive()
match msg |> box :?> Command with
| Start -> ()
| Message v -> printf "%s" v
| Exit -> mailbox.Context.System.Terminate() |> ignore }
loop() |> ignore
Basically, a console is launched that accepts input from the user. My Generator actor forwards that input to my Reporter actor. However, the code above never gets executed.
The code is the following:
module Main
open System
open Akka.FSharp
open Akka.Actor
open Actors
type Command =
| Message of string
| Start | Exit
let reporterActor (mailbox:Actor<_>) =
let rec loop() = actor { let! msg = mailbox.Receive()
match msg |> box :?> Command with
| Start -> ()
| Message v -> printf "%s" v
| Exit -> mailbox.Context.System.Terminate() |> ignore }
loop() |> ignore
let generatorActor (reporter:IActorRef) (mailbox:Actor<_>) message =
let handle input = match input with
| "exit" -> mailbox.Context.System.Terminate |> ignore
| _ -> reporter <! Message input
handle (Console.ReadLine().ToLower())
[<EntryPoint>]
let main argv =
let system = System.create "system" (Configuration.load())
let reporterActor = spawn system "reporterActor" (actorOf(reporterActor))
let generatorActor = spawn system "generatorActor" (actorOf2(generatorActor reporterActor))
generatorActor <! Start
system.AwaitTermination ()
0
Update:
I learned that I could trigger my Reporter actor by replacing the mailbox parameter with an arbitrary message parameter:
let reporterActor message =
match message |> box :?> Command with
| Start -> ()
| Message v -> printf "Reporting: %s" v
| Exit -> failwith "Kill this!"
I still don't understand when I should use a mailbox parameter versus when I should rely on a message parameter.
The difference is in how actorOf and actorOf2 work.
actorOf in conjunction with spawn creates an actor as a child of the root of the system which will handle messages with the function 'Message -> unit that was passed to it.
actorOf2 in in conjunction with spawn creates an actor as a child of the actor that you passed in and the child will handle the messages with the function 'Message -> unit that was passed.
Your original function signature for reporter actor was:
Actor<'Message> -> unit
and you used spawn system "reporterActor" (actorOf(reporterActor))
In this case you were saying that the message type that the new actor that was created would receive would be of type Actor<'Message> . This compiled because actorof just expects a function that takes a 'Message, and a 'Message is generic therefore Actor<'Message> satisfied the 'Message parameter.
When you updated the signature of reporterActor you change the signature to 'Message -> unit which is what actorOf is actually intended to accept.
In short generics allowed your code to compile since 'Message isn't really restricted, nor should it really be.
From: http://getakka.net/docs/FSharp%20API
actorOf (fn : 'Message -> unit) (mailbox : Actor<'Message>) :
Cont<'Message, 'Returned> - uses a function, which takes a message as
the only parameter. Mailbox parameter is injected by spawning
functions.
actorOf2 (fn : Actor<'Message> -> 'Message -> unit) (mailbox :
Actor<'Message>) : Cont<'Message, 'Returned> - uses a function, which
takes both the message and an Actor instance as the parameters.
Mailbox parameter is injected by spawning functions. Example:
> let handleMessage (mailbox: Actor<'a>) msg =
> match msg with
> | Some x -> printf "%A" x
> | None -> ()
>
> let aref = spawn system "my-actor" (actorOf2 handleMessage) let
> blackHole = spawn system "black-hole" (actorOf (fun msg -> ()))
spawn (actorFactory : IActorRefFactory) (name : string) (f :
Actor<'Message> -> Cont<'Message, 'Returned>) : IActorRef - spawns an
actor using a specified actor computation expression. The actor can
only be used locally.
All of these functions may be used with either the actor system or the
actor itself. In the first case the spawned actor will be placed under
/user root guardian of the current actor system hierarchy. In the
second option the spawned actor will become a child of the actor used
as the actorFactory parameter of the spawning function.
I am a little curious about the code example below and what people think.
The idea was to read from a NetworkStream (~20 msg/s) and instead of working in the main, pass things to MainboxProcessor to handle and get things back for bindings when done.
The usual way is to use PostAndReply, but I want to bind to ListView or other control in C#. Must do magic with LastN items and filtering anyway.
Plus, Rx has some error handling.
The example below observes numbers from 2..10 and returns "hello X". On 8 it stops like it was EOF. Made it to ToEnumerable because other thread finishes before otherwise, but it works with Subscribe as well.
What bothers me:
passing Subject(obj) around in recursion. I don't see any problems having around 3-4 of those. Good idea?
Lifetime of Subject.
open System
open System.Threading
open System.Reactive.Subjects
open System.Reactive.Linq // NuGet, take System.Reactive.Core also.
open System.Reactive.Concurrency
type SerializedLogger() =
let _letters = new Subject<string>()
// create the mailbox processor
let agent = MailboxProcessor.Start(fun inbox ->
// the message processing function
let rec messageLoop (letters:Subject<string>) = async{
// read a message
let! msg = inbox.Receive()
printfn "mailbox: %d in Thread: %d" msg Thread.CurrentThread.ManagedThreadId
do! Async.Sleep 100
// write it to the log
match msg with
| 8 -> letters.OnCompleted() // like EOF.
| x -> letters.OnNext(sprintf "hello %d" x)
// loop to top
return! messageLoop letters
}
// start the loop
messageLoop _letters
)
// public interface
member this.Log msg = agent.Post msg
member this.Getletters() = _letters.AsObservable()
/// Print line with prefix 1.
let myPrint1 x = printfn "onNext - %s, Thread: %d" x Thread.CurrentThread.ManagedThreadId
// Actions
let onNext = new Action<string>(myPrint1)
let onCompleted = new Action(fun _ -> printfn "Complete")
[<EntryPoint>]
let main argv =
async{
printfn "Main is on: %d" Thread.CurrentThread.ManagedThreadId
// test
let logger = SerializedLogger()
logger.Log 1 // ignored?
let xObs = logger
.Getletters() //.Where( fun x -> x <> "hello 5")
.SubscribeOn(Scheduler.CurrentThread)
.ObserveOn(Scheduler.CurrentThread)
.ToEnumerable() // this
//.Subscribe(onNext, onCompleted) // or with Dispose()
[2..10] |> Seq.iter (logger.Log)
xObs |> Seq.iter myPrint1
while true
do
printfn "waiting"
System.Threading.Thread.Sleep(1000)
return 0
} |> Async.RunSynchronously // return an integer exit code
I have done similar things, but using the plain F# Event type rather than Subject. It basically lets you create IObservable and trigger its subscribes - much like your use of more complex Subject. The event-based version would be:
type SerializedLogger() =
let letterProduced = new Event<string>()
let lettersEnded = new Event<unit>()
let agent = MailboxProcessor.Start(fun inbox ->
let rec messageLoop (letters:Subject<string>) = async {
// Some code omitted
match msg with
| 8 -> lettersEnded.Trigger()
| x -> letterProduced.Trigger(sprintf "hello %d" x)
// ...
member this.Log msg = agent.Post msg
member this.LetterProduced = letterProduced.Publish
member this.LettersEnded = lettersEnded.Publish
The important differences are:
Event cannot trigger OnCompleted, so I instead exposed two separate events. This is quite unfortunate! Given that Subject is very similar to events in all other aspects, this might be a good reason for using subject instead of plain event.
The nice aspect of using Event is that it is a standard F# type, so you do not need any external dependencies in the agent.
I noticed your comment noting that the first call to Log was ignored. That's because you subscribe to the event handler only after this call happens. I think you could use ReplaySubject variation on the Subject idea here - it replays all events when you subscribe to it, so the one that happened earlier would not be lost (but there is a cost to caching).
In summary, I think using Subject is probably a good idea - it is essentially the same pattern as using Event (which I think is quite standard way of exposing notifications from agents), but it lets you trigger OnCompleted. I would probably not use ReplaySubject, because of the caching cost - you just have to make sure to subscribe before triggering any events.
I can't work out why the following code is hanging at the call to GetTotal. I don't seem to be able to debug inside the MailboxProcessor, so it's hard to see what's going on.
module Aggregator
open System
type Message<'T, 'TState> =
| Aggregate of 'T
| GetTotal of AsyncReplyChannel<'TState>
type Aggregator<'T, 'TState>(initialState, f) =
let myAgent = new MailboxProcessor<Message<'T, 'TState>>(fun inbox ->
let rec loop agg =
async {
let! message = inbox.Receive()
match message with
| Aggregate x -> return! loop (f agg x)
| GetTotal replyChannel ->
replyChannel.Reply(agg)
return! loop agg
}
loop initialState
)
member m.Aggregate x = myAgent.Post(Aggregate(x))
member m.GetTotal = myAgent.PostAndReply(fun replyChannel -> GetTotal(replyChannel))
let myAggregator = new Aggregator<int, int>(0, (+))
myAggregator.Aggregate(3)
myAggregator.Aggregate(4)
myAggregator.Aggregate(5)
let totalSoFar = myAggregator.GetTotal
printfn "%d" totalSoFar
Console.ReadLine() |> ignore
It seems to work fine when using an identical MailboxProcessor directly, rather than wrapping in the Aggregator class.
The problem is that you did not start the agent. You can either call Start after you create the agent:
let myAgent = (...)
do myAgent.Start()
Alternatively, you can create the agent using MailboxProcessor<'T>.Start instead of calling the constructor (I usually prefer this option, because it looks more functional):
let myAgent = MailboxProcessor<Message<'T, 'TState>>.Start(fun inbox -> (...) )
I suppose that you couldn't debug the agent, because the code inside agent wasn't actually running. I tried adding printfn "Msg: %A" message right after the call to Receive inside the agent (to print incoming messages for debugging) and I noticed that, after calling Aggregate, no messages were actually received by the agent... (It only blocked after calling GetTotal, which avaits reply)
As a side-note, I would probably turn GetTotal into a method, so you'd call GetTotal(). Properties are re-evaluated each time you access them, so your code does the same thing, but best practices don't recommend using properties that do complex work.
You forgot to start the mailbox:
open System
type Message<'T, 'TState> =
| Aggregate of 'T
| GetTotal of AsyncReplyChannel<'TState>
type Aggregator<'T, 'TState>(initialState, f) =
let myAgent = new MailboxProcessor<Message<'T, 'TState>>(fun inbox ->
let rec loop agg =
async {
let! message = inbox.Receive()
match message with
| Aggregate x -> return! loop (f agg x)
| GetTotal replyChannel ->
replyChannel.Reply(agg)
return! loop agg
}
loop initialState
)
member m.Aggregate x = myAgent.Post(Aggregate(x))
member m.GetTotal = myAgent.PostAndReply(fun replyChannel -> GetTotal(replyChannel))
member m.Start() = myAgent.Start()
let myAggregator = new Aggregator<int, int>(0, (+))
myAggregator.Start()
myAggregator.Aggregate(3)
myAggregator.Aggregate(4)
myAggregator.Aggregate(5)
let totalSoFar = myAggregator.GetTotal
printfn "%d" totalSoFar
Console.ReadLine() |> ignore
I'd like to write some code that runs a sequence of F# scripts (.fsx). The thing is that I could have literally hundreds of scripts and if I do that:
let shellExecute program args =
let startInfo = new ProcessStartInfo()
do startInfo.FileName <- program
do startInfo.Arguments <- args
do startInfo.UseShellExecute <- true
do startInfo.WindowStyle <- ProcessWindowStyle.Hidden
//do printfn "%s" startInfo.Arguments
let proc = Process.Start(startInfo)
()
scripts
|> Seq.iter (shellExecute "fsi")
it could stress too much my 2GB system. Anyway, I'd like to run scripts by batch of n, which seems also a good exercise for learning Async (I guess it's the way to go).
I have started to write some code for that but unfortunately it doesn't work:
open System.Diagnostics
let p = shellExecute "fsi" #"C:\Users\Stringer\foo.fsx"
async {
let! exit = Async.AwaitEvent p.Exited
do printfn "process has exited"
}
|> Async.StartImmediate
foo.fsx is just a hello world script.
What would be the most idiomatic way of solving this problem?
I'd like also to figure out if it's doable to retrieve a return code for each executing script and if not, find another way. Thanks!
EDIT:
Thanks a lot for your insights and links! I've learned a lot.
I just want to add some code for running batchs in parallel using Async.Parallel as Tomas suggested it. Please comment if there is a better implementation for my cut function.
module Seq =
/// Returns a sequence of sequences of N elements from the source sequence.
/// If the length of the source sequence is not a multiple
/// of N, last element of the returned sequence will have a length
/// included between 1 and N-1.
let cut (count : int) (source : seq<´T>) =
let rec aux s length = seq {
if (length < count) then yield s
else
yield Seq.take count s
if (length <> count) then
yield! aux (Seq.skip count s) (length - count)
}
aux source (Seq.length source)
let batchCount = 2
let filesPerBatch =
let q = (scripts.Length / batchCount)
q + if scripts.Length % batchCount = 0 then 0 else 1
let batchs =
scripts
|> Seq.cut filesPerBatch
|> Seq.map Seq.toList
|> Seq.map loop
Async.RunSynchronously (Async.Parallel batchs) |> ignore
EDIT2:
So I had some troubles to get Tomas's guard code working. I guess the f function had to be called in AddHandler method, otherwise we loose the event for ever... Here's the code:
module Event =
let guard f (e:IEvent<´Del, ´Args>) =
let e = Event.map id e
{ new IEvent<´Args> with
member this.AddHandler(d) = e.AddHandler(d); f() //must call f here!
member this.RemoveHandler(d) = e.RemoveHandler(d); f()
member this.Subscribe(observer) =
let rm = e.Subscribe(observer) in f(); rm }
The interesting thing (as mentioned by Tomas) is that it looks like the Exited event is stored somewhere when the process terminates, even though the process has not started with EnableRaisingEvents set to true.
When this property is finally set to true, the event is fired up.
Since I'm not sure that this is the official specification (and also a bit paranoid), I found another solution that consists in starting the process in the guard function, so we ensure that the code will work on whichever situation:
let createStartInfo program args =
new ProcessStartInfo
(FileName = program, Arguments = args, UseShellExecute = false,
WindowStyle = ProcessWindowStyle.Normal,
RedirectStandardOutput = true)
let createProcess info =
let p = new Process()
do p.StartInfo <- info
do p.EnableRaisingEvents <- true
p
let rec loop scripts = async {
match scripts with
| [] -> printfn "FINISHED"
| script::scripts ->
let args = sprintf "\"%s\"" script
let p = createStartInfo "notepad" args |> createProcess
let! exit =
p.Exited
|> Event.guard (fun () -> p.Start() |> ignore)
|> Async.AwaitEvent
let output = p.StandardOutput.ReadToEnd()
do printfn "\nPROCESSED: %s, CODE: %d, OUTPUT: %A"script p.ExitCode output
return! loop scripts
}
Notice I've replaced fsi.exe by notepad.exe so I can replay different scenarios step by step in the debugger and control explicitly the exit of the process myself.
I did some experiments and here is one way to deal with the problem discussed in the comments below my post and in the answer from Joel (which I think doesn't work currently, but could be fixed).
I think the specification of Process is that it can trigger the Exited event after we set the EnableRaisingEvents property to true (and will trigger the event even if the process has already completed before we set the property). To handle this case correctly, we need to enable raising of events after we attach handler to the Exited event.
This is a problme, because if we use AwaitEvent it will block the workflow until the event fires. We cannot do anything after calling AwaitEvent from the workflow (and if we set the property before calling AwaitEvent, then we get a race....). Vladimir's approach is correct, but I think there is a simpler way to deal with this.
I'll create a function Event.guard taking an event and returning an event, which allows us to specify some function that will be executed after a handler is attached to the event. This means that if we do some operation (which in turn triggers the event) inside this function, the event will be handled.
To use it for the problem discussed here, we need to change my original solution as follows. Firstly, the shellExecute function must not set the EnableRaisingEvents property (otherwise, we could lose the event!). Secondly, the waiting code should look like this:
let rec loop scripts = async {
match scripts with
| [] -> printf "FINISHED"
| script::scripts ->
let p = shellExecute fsi script
let! exit =
p.Exited
|> Event.guard (fun () -> p.EnableRaisingEvents <- true)
|> Async.AwaitEvent
let output = p.StandardOutput.ReadToEnd()
return! loop scripts }
Note the use of the Event.guard function. Roughly, it says that after the workflow attaches handler to the p.Exited event, the provided lambda function will run (and will enable raising of events). However, we already attached the handler to the event, so if this causes the event immediately, we're fine!
The implementation (for both Event and Observable) looks like this:
module Event =
let guard f (e:IEvent<'Del, 'Args>) =
let e = Event.map id e
{ new IEvent<'Args> with
member x.AddHandler(d) = e.AddHandler(d)
member x.RemoveHandler(d) = e.RemoveHandler(d); f()
member x.Subscribe(observer) =
let rm = e.Subscribe(observer) in f(); rm }
module Observable =
let guard f (e:IObservable<'Args>) =
{ new IObservable<'Args> with
member x.Subscribe(observer) =
let rm = e.Subscribe(observer) in f(); rm }
Nice thing is that this code is very straightforward.
Your approach looks great to me, I really like the idea of embedding process execution into asynchronous workflows using AwaitEvent!
The likely reason why it didn't work is that you need to set EnableRisingEvents property of the Process to true if you want it to ever trigger the Exited event (don't ask my why you have to do that, it sounds pretty silly to me!) Anyway, I did a couple of other changes to your code when testing it, so here is a version that worked for me:
open System
open System.Diagnostics
let shellExecute program args =
// Configure process to redirect output (so that we can read it)
let startInfo =
new ProcessStartInfo
(FileName = program, Arguments = args, UseShellExecute = false,
WindowStyle = ProcessWindowStyle.Hidden,
RedirectStandardOutput = true)
// Start the process
// Note: We must enable rising events explicitly here!
Process.Start(startInfo, EnableRaisingEvents = true)
Most importantly, the code now sets EnableRaisingEvents to true. I also changed the code to use a syntax where you specify properties of an object when constructing it (to make the code a bit more succinct) and I changed a few properties, so that I can read the output (RedirectStandardOutput).
Now, we can use the AwaitEvent method to wait until a process completes. I'll assume that fsi contains the path to fsi.exe and that scripts is a list of FSX scripts. If you want to run them sequentially, you could use a loop implemented using recursion:
let rec loop scripts = async {
match scripts with
| [] -> printf "FINISHED"
| script::scripts ->
// Start the proces in background
let p = shellExecute fsi script
// Wait until the process completes
let! exit = Async.AwaitEvent p.Exited
// Read the output produced by the process, the exit code
// is available in the `ExitCode` property of `Process`
let output = p.StandardOutput.ReadToEnd()
printfn "\nPROCESSED: %s, CODE: %d\n%A" script p.ExitCode output
// Process the rest of the scripts
return! loop scripts }
// This starts the workflow on background thread, so that we can
// do other things in the meantime. You need to add `ReadLine`, so that
// the console application doesn't quit immedeiately
loop scripts |> Async.Start
Console.ReadLine() |> ignore
Of course, you could also run the processes in parallel (or for example run 2 groups of them in parallel etc.) To do that you would use Async.Parallel (in the usual way).
Anyway, this is a really nice example of using asynchronous workflows in an area where I haven't seen them used so far. Very interesting :-)
In response to Tomas's answer, would this be a workable solution to the race condition involved in starting the process, and then subscribing to its Exited event?
type Process with
static member AsyncStart psi =
let proc = new Process(StartInfo = psi, EnableRaisingEvents = true)
let asyncExit = Async.AwaitEvent proc.Exited
async {
proc.Start() |> ignore
let! args = asyncExit
return proc
}
Unless I'm mistaken, this would subscribe to the event prior to starting the process, and package it all up as an Async<Process> result.
This would allow you to rewrite the rest of the code like this:
let shellExecute program args =
// Configure process to redirect output (so that we can read it)
let startInfo =
new ProcessStartInfo(FileName = program, Arguments = args,
UseShellExecute = false,
WindowStyle = ProcessWindowStyle.Hidden,
RedirectStandardOutput = true)
// Start the process
Process.AsyncStart(startInfo)
let fsi = "PATH TO FSI.EXE"
let rec loop scripts = async {
match scripts with
| [] -> printf "FINISHED"
| script::scripts ->
// Start the proces in background
use! p = shellExecute fsi script
// Read the output produced by the process, the exit code
// is available in the `ExitCode` property of `Process`
let output = p.StandardOutput.ReadToEnd()
printfn "\nPROCESSED: %s, CODE: %d\n%A" script p.ExitCode output
// Process the rest of the scripts
return! loop scripts
}
If that does the job, it's certainly a lot less code to worry about than Vladimir's Async.GetSubject.
What about a mailboxprocessor?
It is possible to simplify version of Subject from blogpost. instead of returning imitation of event, getSubject can return workflow.
Result workflow itself is state machine with two states
1. Event wasn't triggered yet: all pending listeners should be registered
2. Value is already set, listener is served immediately
In code it will appear like this:
type SubjectState<'T> = Listen of ('T -> unit) list | Value of 'T
getSubject implementation is trivial
let getSubject (e : IEvent<_, _>) =
let state = ref (Listen [])
let switchState v =
let listeners =
lock state (fun () ->
match !state with
| Listen ls ->
state := Value v
ls
| _ -> failwith "Value is set twice"
)
for l in listeners do l v
Async.StartWithContinuations(
Async.AwaitEvent e,
switchState,
ignore,
ignore
)
Async.FromContinuations(fun (cont, _, _) ->
let ok, v = lock state (fun () ->
match !state with
| Listen ls ->
state := Listen (cont::ls)
false, Unchecked.defaultof<_>
| Value v ->
true, v
)
if ok then cont v
)