Enforcing one Async Observable at a time in F# - f#

I have an observable sequence of things that need to be mapped to
a C# task. These C# tasks should not run concurrently, but one after
another. Basically, I need to achieve the F# equivalent of this C#
question:
Enforcing one async observable at a time
Naively translating this C# code gives something like the following:
let run (idx:int) (delay:int) =
async {
sprintf "start: %i (%i)" idx delay |> System.Diagnostics.Trace.WriteLine
let! t = System.Threading.Tasks.Task.Delay(delay) |> Async.AwaitTask
sprintf "finish: %i" idx |> System.Diagnostics.Trace.WriteLine
t
}
Observable.generate (new Random()) (fun _ -> true) id (fun s -> s.Next(250, 500))
|> Observable.take 20
|> Observable.mapi(fun idx delay -> idx, delay)
|> Observable.bind(fun (idx, delay) -> Observable.ofAsync (run idx delay))
|> Observable.subscribe ignore
|> ignore
Which does not work as expected because I don't wait for a result anywhere. Is there even
a way to do this in F# without blocking a thread, just like C#'s await would?

There's a handy library that exists in F# called AsyncSeq:
https://www.nuget.org/packages/FSharp.Control.AsyncSeq/
It's very similar to IAsyncEnumerable<T> that was added to C# 8.0, this gives you a nice solution to what you are after.
Solution:
open System
open FSharp.Control
open FSharp.Control.Reactive
[<EntryPoint>]
let main _ =
let run (idx:int) (delay:int) =
async {
sprintf "start: %i (%i)" idx delay |> Console.WriteLine
do! Async.Sleep delay
sprintf "finish: %i" idx |> Console.WriteLine
}
Observable.generate (new Random()) (fun _ -> true) id (fun s -> s.Next(250, 500))
|> Observable.take 20
|> Observable.mapi(fun idx delay -> idx, delay)
|> AsyncSeq.ofObservableBuffered
|> AsyncSeq.iterAsync (fun (idx,delay) -> run idx delay)
|> Async.RunSynchronously
0
AsyncSeq.ofObservableBuffered does the work of subscribing to your Observable and acts as AsyncSeq source which you can pipeline on top of. Finally we call Async.RunSynchronously to kick it off and wait on the entrypoint thread.
Note: I also update run as it was returning Async<Async<unit>> and I assume that wasn't intended.

Related

Seq.iter fails to iterate the sequence created with seq.collect and sequence expression

I've had a bug in F# code that I have reduced to the following minimal reproduction sequence, but now I don't understand why it works that way.
let duplicate element =
[ element; element ]
let passThrough (sq: seq<_>) =
use it = sq.GetEnumerator ()
seq {
while (it.MoveNext ()) do
yield it.Current
}
[<EntryPoint>]
let main _ =
[0; 1]
|> Seq.collect (duplicate)
(* |> Seq.toArray // When uncommented - works as expected. *)
|> passThrough
|> Seq.iter (fun i -> printf $"{i} ")
0
When the Seq.toArray call is uncommented, it produces the result I expect, i.e. iterates the sequence pipeline and prints 0 0 1 1. However with that line commented out, the code just finishes without printing anything.
I conferred with one of our experts at F# Slack (thanks R. C.), and was advised that a proper implementation of passThrough should look like this. The enumerator is then properly disposed when the while loop exits. The problem with the original implementation is that the enumerator is disposed at a point in time before the while loop exits, if ever.
let passThrough (sq: seq<_>) =
seq {
use it = sq.GetEnumerator ()
while (it.MoveNext ()) do
yield it.Current
}

why does Seq.isEmpty say not enough elements?

nums is indeed seq of int when I mouse over. Any idea what's going on?
This function line is intended to be the equivalent of C#'s DefaultIfEmpty Linq function.
The general idea is take a space delimited line of strings and write out which ones occur count number of times.
code:
open System
[<EntryPoint>]
let main argv =
let tests = Console.ReadLine() |> int
for i in [0..tests] do
let (length, count) = Console.ReadLine()
|> (fun s -> s.Split [|' '|])
|> (fun split -> Int32.Parse(split.[0]), Int32.Parse(split.[1]))
Console.ReadLine()
|> (fun s -> s.Split [|' '|])
|> Seq.map int
|> Seq.take length
|> Seq.groupBy (fun x -> x)
|> Seq.map (fun (key, group) -> key, Seq.sum group)
|> Seq.where (fun (_, countx) -> countx = count)
|> Seq.map (fun (n, _) -> n)
|> (fun nums -> if Seq.isEmpty nums then "-1" else String.Join(" ", nums))
|> Console.WriteLine
0 // return an integer exit code
Sample input:
3
9 2
4 5 2 5 4 3 1 3 4
So, sequences in F# use lazy evaluation. That means that when you use functions such as map, where, take etc, the results are not evaluated immediately.
The results are only evaluated when the sequence is actually enumerated. When you call Seq.isEmpty you trigger a call to MoveNext() which results in the first element of the result sequence being evaluated - in your case this results in a large chain of functions being evaluated.
In this case, the InvalidOperationException is actually being triggered by Seq.take which throws if the sequence doesn't have sufficient elements. This might surprise you coming from C# where Enumerable.Take will take up to the requested number of elements but could take fewer if you reach the end of the sequence.
If you want this behaviour in F#, you need to replace Seq.take with Seq.truncate.

http download to disk with fsharp.data.dll and async workflows stalls

The following .fsx file is supposed to download and save to disk binary table base files which are posted as links in a html page on the internet, using Fsharp.Data.dll.
What happens, is that the whole thing stalls after a while and way before it is done, not even throwing an exception or alike.
I am pretty sure, I kind of mis-handle the CopyToAsync() thingy in my async workflow. As this is supposed to run while I go for a nap, it would be nice if someone could tell me how it is supposed to be done correctly. (In more general terms - how to handle a System.Threading.Task thingy in an async workflow thingy?)
#r #"E:\R\playground\DataTypeProviderStuff\packages\FSharp.Data.2.2.3\lib\net40\FSharp.Data.dll"
open FSharp.Data
open Microsoft.FSharp.Control.CommonExtensions
let document = HtmlDocument.Load("http://www.olympuschess.com/egtb/gaviota/")
let links =
document.Descendants ["a"] |> Seq.choose (fun x -> x.TryGetAttribute("href") |> Option.map (fun a -> a.Value()))
|> Seq.filter (fun v -> v.EndsWith(".cp4"))
|> List.ofSeq
let targetFolder = #"E:\temp\tablebases\"
let downloadUrls =
links |> List.map (fun name -> "http://www.olympuschess.com/egtb/gaviota/" + name, targetFolder + name )
let awaitTask = Async.AwaitIAsyncResult >> Async.Ignore
let fetchAndSave (s,t) =
async {
printfn "Starting with %s..." s
let! result = Http.AsyncRequestStream(s)
use fileStream = new System.IO.FileStream(t,System.IO.FileMode.Create)
do! awaitTask (result.ResponseStream.CopyToAsync(fileStream))
printfn "Done with %s." s
}
let makeBatches n jobs =
let rec collect i jl acc =
match i,jl with
| 0, _ -> acc,jl
| _, [] -> acc,jl
| _, x::xs -> collect (i-1) (xs) (acc # [x])
let rec loop remaining acc =
match remaining with
| [] -> acc
| x::xs ->
let r,rest = collect n remaining []
loop rest (acc # [r])
loop jobs []
let download () =
downloadUrls
|> List.map fetchAndSave
|> makeBatches 2
|> List.iter (fun l -> l |> Async.Parallel |> Async.RunSynchronously |> ignore )
|> ignore
download()
Note Updated code so it creates batches of 2 downloads at a time and only the first batch works. Also added the awaitTask from the first answer as this seems the right way to do it.
News What is also funny: If I interrupt the stalled script and then #load it again into the same instance of fsi.exe, it stalls right away. I start to think it is a bug in the library I use or something like that.
Thanks, in advance!
Here fetchAndSave has been modified to handle the Task returned from CopyToAsync asynchronously. In your version you are waiting on the Task synchronously. Your script will appear to lock up as you are using Async.RunSynchronously to run the whole workflow. However the files do download as expected in the background.
let awaitTask = Async.AwaitIAsyncResult >> Async.Ignore
let fetchAndSave (s,t) = async {
let! result = Http.AsyncRequestStream(s)
use fileStream = new System.IO.FileStream(t,System.IO.FileMode.Create)
do! awaitTask (result.ResponseStream.CopyToAsync(fileStream))
}
Of course you also need to call
do download()
on the last line of your script to kick things into motion.

Can I intercept the F# sequence generator?

trying to work around a problem in outside library - is there a way to try-catch the generator itself item by item (probably not, but just to be sure...)?
let myTest() =
let mySeq = seq { for i in -3 .. 3 -> 1 / i }
// how to keep the line above intact, but modify the code below to try-catch-ignore the bad one?
mySeq |> Seq.iter (fun i -> printfn "%d" i)
()
You can't.
Once the exception happens, the state of the source enumerator is screwed up. If you can't get into the source enumerator to "fix" its state, you can't make it keep producing values.
You can, however, make the whole process "stop" after the exception, but you'll have to go a level below and work with IEnumerator<T>:
let takeUntilError (sq: seq<_>) = seq {
use enm = sq.GetEnumerator()
let next () = try enm.MoveNext() with _ -> false
let cur () = try Some enm.Current with _ -> None
while next() do
match cur() with
| Some c -> yield c
| None -> ()
}
mySeq |> takeUntilError |> Seq.iter (printf "%d")

printfn in pipeline

So I have a function SolveEquasion that returns a pair float*float[]. What is the best way to print the number and the array and continue working with the array? I made the following code but it seems there is a better way
...
|> SolveEquasion
|> (fun (det, solution) -> printfn "Determinant = %f\nSolution = %A" det (Array.toList solution), solution )
|> snd
I don't think your solution can improved if you want to do this in a pipeline. Another approach is to use a let binding, along with splitting up the pipelined operations, to avoid having a function that acts like the love child of map and iter.
let (det, solution) = SolveEquasion
printfn "Determinant = %f\nSolution = %A" det (Array.toList solution)
//do something else with solution
I think the original solution is fine, and we can improve its clarity by giving your anonymous function the name I've seen it given in some other libraries based around pipelining higher-order functions: tap.
let tap f x =
f x
x
(1.0, [| 2.0; 3.0 |])
|> tap (fun (s, a) -> printfn "%A %A" s a)
|> snd
Well, for one thing you can skip the use of snd by returning a single value rather than a tuple from the previous function:
...
|> SolveEquasion
|> (fun (det, solution) ->
printfn "Determinant = %f\nSolution = %A" det (Array.toList solution)
solution )
I'd probably use Daniel's approach and just assign the value you want to print to a symbol using let. Alternatively, you could define a variant of printf that takes some arguments and returns one of them. I'm not sure if there is a general scheme how this should be done - for your example it would take a two-element tuple:
let mprintf fmt (a, b) =
Printf.kprintf (fun s -> printf "%s" s; (a, b)) fmt a b
Then you can write:
...
|> SolveEquasion
|> mprintfn "Determinant = %f\nSolution = %A"
|> snd |> // ... more stuff with solution

Resources