I am learning F# and I have problem with using WriteLineAsync.
With synchronous code it works perfectly:
using (new StreamWriter(new FileStream(#"c:\work\f2.txt", FileMode.Create, FileAccess.Write, FileShare.None, bufferSize= 4096, useAsync= true)))
(fun sw -> File.ReadLines #"c:\Work\f1.txt" |> Seq.iter(fun x-> sw.WriteLine(x) ))
When I try to user WriteLineAsync:
using (new StreamWriter(new FileStream(#"c:\work\f2.txt", FileMode.Create, FileAccess.Write, FileShare.None, bufferSize= 4096, useAsync= true)))
(fun sw -> File.ReadLines #"c:\Work\f1.txt" |> Seq.iter(fun x-> sw.WriteLineAsync(x) |> Async.AwaitTask |> ignore ))
I got error InvalidOperationException:The stream writer is currently in use by a previous write operation.
What I am doing wrong ?
All AwaitTask does is that it converts from Task<T> to Async<T>, it doesn't actually await anything. That means you can't just ignore the result.
What you can do is to create an async workflow and await WriteLineAsync() by using do!:
async {
use sw = new StreamWriter(new FileStream(#"c:\work\f2.txt", FileMode.Create, FileAccess.Write, FileShare.None, bufferSize= 4096, useAsync= true))
for line in File.ReadLines #"c:\Work\f1.txt" do
do! sw.WriteLineAsync(line) |> Async.AwaitTask
}
Related
The fsi blocks when I try to combine these two transformers, without any warning or error message.
open FSharpPlus
open FSharpPlus.Data
type MyError = | Err of string
let f : WriterT<ResultT<Async<Result<_, MyError>>>> = monad {
do! liftAsync <| Async.Sleep 30
do! tell ["abc"]
let! r0 = lift<| ResultT.hoist (Ok 25)
return r0 + 5
}
let g = monad {
let! r1 = f
do! tell ["def"]
do! liftAsync <| Async.Sleep 50
let! r2 = lift<| ResultT.hoist (Ok 2)
return r1 + r2
}
let runStackT stack = stack |> WriterT.run |> ResultT.run |> Async.RunSynchronously
#time "on"
let finalResult = runStackT g
#time "off"
What is the problem and how to make this code work?
As stated in the comments, this is an F# bug.
Your code is correct, and even if it wasn't there's no reason why the F# compiler should hang.
I've found a workaround, if you change these lines:
...
let g: WriterT<ResultT<Async<Result<_, MyError>>>> = monad {
let! r1 = f
let! _ = tell ["def"]
let! _ = liftAsync <| Async.Sleep 50
let! r2 = WriterT.Lift <| ResultT.hoist (Ok 2)
...
it will work.
Note that changing a do! for a let! _ = is a trick I use frequently to improve type inference.
I have an observable sequence of things that need to be mapped to
a C# task. These C# tasks should not run concurrently, but one after
another. Basically, I need to achieve the F# equivalent of this C#
question:
Enforcing one async observable at a time
Naively translating this C# code gives something like the following:
let run (idx:int) (delay:int) =
async {
sprintf "start: %i (%i)" idx delay |> System.Diagnostics.Trace.WriteLine
let! t = System.Threading.Tasks.Task.Delay(delay) |> Async.AwaitTask
sprintf "finish: %i" idx |> System.Diagnostics.Trace.WriteLine
t
}
Observable.generate (new Random()) (fun _ -> true) id (fun s -> s.Next(250, 500))
|> Observable.take 20
|> Observable.mapi(fun idx delay -> idx, delay)
|> Observable.bind(fun (idx, delay) -> Observable.ofAsync (run idx delay))
|> Observable.subscribe ignore
|> ignore
Which does not work as expected because I don't wait for a result anywhere. Is there even
a way to do this in F# without blocking a thread, just like C#'s await would?
There's a handy library that exists in F# called AsyncSeq:
https://www.nuget.org/packages/FSharp.Control.AsyncSeq/
It's very similar to IAsyncEnumerable<T> that was added to C# 8.0, this gives you a nice solution to what you are after.
Solution:
open System
open FSharp.Control
open FSharp.Control.Reactive
[<EntryPoint>]
let main _ =
let run (idx:int) (delay:int) =
async {
sprintf "start: %i (%i)" idx delay |> Console.WriteLine
do! Async.Sleep delay
sprintf "finish: %i" idx |> Console.WriteLine
}
Observable.generate (new Random()) (fun _ -> true) id (fun s -> s.Next(250, 500))
|> Observable.take 20
|> Observable.mapi(fun idx delay -> idx, delay)
|> AsyncSeq.ofObservableBuffered
|> AsyncSeq.iterAsync (fun (idx,delay) -> run idx delay)
|> Async.RunSynchronously
0
AsyncSeq.ofObservableBuffered does the work of subscribing to your Observable and acts as AsyncSeq source which you can pipeline on top of. Finally we call Async.RunSynchronously to kick it off and wait on the entrypoint thread.
Note: I also update run as it was returning Async<Async<unit>> and I assume that wasn't intended.
I have the following code
let rec consume() : Async<unit> = async {
.....
listA
|> Seq.iter(fun i ->
.....
let listB : seq<...> option =
let c = getListB a b
match c with
| Some d -> Seq.filter(....) |> Some
| None -> None
match listB with .....
....
Now the function getListB is converted to return async<Seq<B>> instead of Seq<B>. So the code was converted to the following. However, the getListB blocked the execution. How to rewrite it nonblocking? Simply convert the line to let! c = getListB a b won't work because the code is in an inner lambda? The error message is "This construct may only be used within computation expressions".
let rec consume() : Async<unit> = async {
.....
listA
|> Seq.iter(fun i ->
.....
let listB : seq<...> option =
let c = getListB a b |> Async.RunSynchronously
match c with
| Some d -> Seq.filter(....) |> Some
| None -> None
I believe the problem you are describing boils down to how to convert an seq<Async> to an Async<seq>. This is described comprehensively in this post by Scott Wlaschin.
This is a poor man's implementation of the concepts described in his post which are far more powerful and generic. The general idea is that we want to delay the creation of the sequence until we have the values promised by the instance of Async<_>
let traverseSequence ( seqAsync : seq<Async<'a>>) =
let promiseOfAnEmptySequence = async { return Seq.empty }
let delayedCalculation (asyncHead : Async<'a>) (asyncTail : Async<seq<'a>>) =
async {
let! calculatedHead = asyncHead
return!
async {
let! calculatedTail = asyncTail
return calculatedHead |> Seq.singleton |> Seq.append(calculatedTail)
}
}
Seq.foldBack delayedCalculation seqAsync promiseOfAnEmptySequence
The answer depends on whether you want to run each element of the sequence sequentially or in parallel.
In both cases, start by using Seq.map instead of Seq.iter, then you can put another async block inside the lambda such that the result of the map is seq<Async<'a>>.
Sequential
For this, you need define some extra functions in an extra Async module.
module Async =
let map f x =
async{
let! x = x
return f x
}
let lift2 f x1 x2 =
async{
let! x1 = x1
let! x2 = x2
return f x1 x2
}
let return' x = async { return x }
let mapM mFunc sequ =
let consF x ys = lift2 (fun h t -> h::t) (mFunc x) ys
Seq.foldBack(consF) sequ (return' [])
|> map (Seq.ofList)
let sequence sequ = mapM id sequ
You might have seen mapM called traverse elsewhere, they are basically just different names for the same concept.
The sequence function is just a special case of mapM where the supplied binding function is just the identity (id) function. It has type seq<Async<'a>> -> Async<seq<'a>>, i.e. it flips the Async from being inside the Seq to being outside.
You then simply pipe the result of your Seq.map to the sequence function, which gives you an async value.
Your example code isn't complete so I made up some example code to use this:
let sleep = Async.Sleep 100
let sleeps = Seq.init 15 (fun _ -> sleep)
let sequencedSleeps = Async.sequence sleeps
Async.RunSynchronously sequencedSleeps
Real: 00:00:01.632, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0
val it : seq<unit> =
[null; null; null; null; null; null; null; null; null; null; null; null;
null; null; null]
Parallel
To execute each element of the sequence in parallel, instead of sequentially, you could do:
let pSequence sequ = Async.Parallel sequ |> Async.map (Seq.ofArray)
Example test code:
let pSleeps = pSequence sleeps;;
Async.RunSynchronously pSleeps;;
Real: 00:00:00.104, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0
val it : seq<unit> = seq [null; null; null; null; ...]
Note how the execution time depends on the chosen approach.
For the cases where you're getting back a seq<unit> and so want to ignore the result it can be useful to define some extra helper functions, such as:
let sequenceIgnore sequ = sequ |> Async.sequence |> Async.map (ignore)
let pSequenceIgnore sequ = sequ |> pSequence |> Async.map (ignore)
That lets you return a single unit rather than a superfluous sequence of them.
Trying to learning F# and I tried to reimplement the following function in F#
private string[] GetSynonyms(string synonyms)
{
var items = Enumerable.Repeat(synonyms, 1)
.Where(s => s != null)
.Select(XDocument.Parse)
.Select(doc => doc.Root)
.Where(root => root != null)
.SelectMany(e => e.Elements(SynonymsNamespace + "synonym"))
.Select(e => e.Value)
.ToArray();
return items;
}
I got this far by myself
let xname = XNamespace.Get "http://localuri"
let syn = "<synonyms xmlns=\"http://localuri\"><synonym>a word</synonym><synonym>another word</synonym></synonyms>"
let synonyms str =
let items = [str]
items
|> List.map System.Xml.Linq.XDocument.Parse
|> List.map (fun x -> x.Root)
|> List.map (fun x -> x.Elements(xname + "synonym") |> Seq.cast<System.Xml.Linq.XElement>)
|> Seq.collect (fun x -> x)
|> Seq.map (fun x -> x.Value)
let a = synonyms syn
Dump a
Now I'm wondering if there is a more-functional way to write the same code.
By extracting the access to the properties to standalone functions I got this version
let xname = XNamespace.Get "http://localuri"
let syn = "<synonyms xmlns=\"http://localuri\"><synonym>a word</synonym><synonym>another word</synonym></synonyms>"
let getRoot (doc:System.Xml.Linq.XDocument) = doc.Root
let getValue (element:System.Xml.Linq.XElement) = element.Value
let getElements (element:System.Xml.Linq.XElement) =
element.Elements(xname + "synonym")
|> Seq.cast<System.Xml.Linq.XElement>
let synonyms str =
let items = [str]
items
|> List.map System.Xml.Linq.XDocument.Parse
|> List.map getRoot
|> List.map getElements
|> Seq.collect (fun x -> x)
|> Seq.map getValue
let a = synonyms syn
Dump a
But I still have a few concerns
Can I rewrite that Seq.collect (fun x -> x) in another way? It sounds redundant
Can I remove all those (fun x -> x.Property) without creating new functions?
How to actually return an array and not a Seq<'a> (which I understand is an IEnumerable<'a>)
Thanks
Seq.collect (fun x -> x) can be rewritten with the predefined id function to Seq.collect id
In F# 4.0 it can be removed for constructors only.
use Seq.toArray or Seq.toList
Would it be very wrong to drop the C#-code and go all in with the XML-provider in F#? In my world its always wrong to parse XML when there exists other solutions (unless Im trying to create octogonal wheels or moist gun powders other have made better before me).
In this regard I would even have used some transformation (XSLT) or selection (XPATH/XQUERY) unless I could use the XML-provider or some XSD (c#) for generating code.
If for some reason the XML is so unstructured that you actually need parsing, then the XML is arguably wrong...
If using the XmlProvider you get namespacing, types etc for free...
#r #"..\correct\this\path\to\packages\FSharp.Data.2.2.5\lib\net40\FSharp.Data.dll"
#r "System.Xml.Linq"
open FSharp.Data
[<Literal>]
let syn = "<synonyms xmlns=\"http://localuri\"><synonym>a word</synonym><synonym>another word</synonym></synonyms>"
type Synonyms = XmlProvider<syn>
let a = Synonyms.GetSample()
a.Synonyms |> Seq.iter (printfn "%A")
Mind that the XmlProvider also can take files or url as examples for inferring types etc, and that you also can have this code as the example and then use
let a = Synonyms.Load(stuff)
where stuff is a read from stream, textreader or URI and inferred according to your example. The sample and the stuff might even point to same file/Uri if this is some standard placing of data.
See also: http://fsharp.github.io/FSharp.Data/library/XmlProvider.html
The following .fsx file is supposed to download and save to disk binary table base files which are posted as links in a html page on the internet, using Fsharp.Data.dll.
What happens, is that the whole thing stalls after a while and way before it is done, not even throwing an exception or alike.
I am pretty sure, I kind of mis-handle the CopyToAsync() thingy in my async workflow. As this is supposed to run while I go for a nap, it would be nice if someone could tell me how it is supposed to be done correctly. (In more general terms - how to handle a System.Threading.Task thingy in an async workflow thingy?)
#r #"E:\R\playground\DataTypeProviderStuff\packages\FSharp.Data.2.2.3\lib\net40\FSharp.Data.dll"
open FSharp.Data
open Microsoft.FSharp.Control.CommonExtensions
let document = HtmlDocument.Load("http://www.olympuschess.com/egtb/gaviota/")
let links =
document.Descendants ["a"] |> Seq.choose (fun x -> x.TryGetAttribute("href") |> Option.map (fun a -> a.Value()))
|> Seq.filter (fun v -> v.EndsWith(".cp4"))
|> List.ofSeq
let targetFolder = #"E:\temp\tablebases\"
let downloadUrls =
links |> List.map (fun name -> "http://www.olympuschess.com/egtb/gaviota/" + name, targetFolder + name )
let awaitTask = Async.AwaitIAsyncResult >> Async.Ignore
let fetchAndSave (s,t) =
async {
printfn "Starting with %s..." s
let! result = Http.AsyncRequestStream(s)
use fileStream = new System.IO.FileStream(t,System.IO.FileMode.Create)
do! awaitTask (result.ResponseStream.CopyToAsync(fileStream))
printfn "Done with %s." s
}
let makeBatches n jobs =
let rec collect i jl acc =
match i,jl with
| 0, _ -> acc,jl
| _, [] -> acc,jl
| _, x::xs -> collect (i-1) (xs) (acc # [x])
let rec loop remaining acc =
match remaining with
| [] -> acc
| x::xs ->
let r,rest = collect n remaining []
loop rest (acc # [r])
loop jobs []
let download () =
downloadUrls
|> List.map fetchAndSave
|> makeBatches 2
|> List.iter (fun l -> l |> Async.Parallel |> Async.RunSynchronously |> ignore )
|> ignore
download()
Note Updated code so it creates batches of 2 downloads at a time and only the first batch works. Also added the awaitTask from the first answer as this seems the right way to do it.
News What is also funny: If I interrupt the stalled script and then #load it again into the same instance of fsi.exe, it stalls right away. I start to think it is a bug in the library I use or something like that.
Thanks, in advance!
Here fetchAndSave has been modified to handle the Task returned from CopyToAsync asynchronously. In your version you are waiting on the Task synchronously. Your script will appear to lock up as you are using Async.RunSynchronously to run the whole workflow. However the files do download as expected in the background.
let awaitTask = Async.AwaitIAsyncResult >> Async.Ignore
let fetchAndSave (s,t) = async {
let! result = Http.AsyncRequestStream(s)
use fileStream = new System.IO.FileStream(t,System.IO.FileMode.Create)
do! awaitTask (result.ResponseStream.CopyToAsync(fileStream))
}
Of course you also need to call
do download()
on the last line of your script to kick things into motion.