I am new to f# and I have a question about timeout in asynchornization operations, this is simple communication using serial com ports, so I have a method/function:
let SendMessenge(port : Ports.SerialPort, messange: string) =
async {
do! port.AsyncWriteLine messange
let! response = port.AsyncReadLine() // this returns a string
return response
}
|> Async.RunSynchronously // the place i fiddled with
All communication (messengering) is controlled in this module:
type SerialPort with
member this.AsyncWriteLine(messange : string) = this.BaseStream.AsyncWrite(this.Encoding.GetBytes(messange + "\n"))
member this.AsyncReadLine() =
async {
let messange_builder = StringBuilder()
let buffer_ref = ref (Array.zeroCreate<byte> this.ReadBufferSize)
let buffer = !buffer_ref
let last_char = ref 0uy
while !last_char <> byte '\n' do
let! readCount = this.BaseStream.AsyncRead buffer
last_char := buffer.[readCount-1]
messange_builder.Append (this.Encoding.GetString(buffer.[0 .. readCount-1])) |> ignore
messange_builder.Length <- messange_builder.Length-1
let response : string = messange_builder.ToString()
printfn "Response: %s" response
return response
}
Basically this works fine, it sends a message and receives response, But now I want to add a timeout, in case i am connect. I tried to fiddle with
|> Async.RunSynchronously(???, timeout, cancel_token)
but with no luck. As i see in documentation it takes timeout and cancellation token and Async, What would be this T0 generic parameter in my case?
What happens is that Async.RunSynchronously is a static method, rather than a method function, so it takes its arguments with tuple syntax. So you can't partially apply it and pipe the last argument into it.
You can do this:
let response = async {
// ...
}
Async.RunSynchronously(response, timeout, cancel_token)
or if you really want to pipe:
async {
// ...
}
|> fun response -> Async.RunSynchronously(response, timeout, cancel_token)
Related
I'm being a little adventurous with my code for the amount of experience I have with F# and I am a little worried about cross threading issues.
Background:
I have a number of orders where I need to validate the address. Some of the orders can be validated against google maps geocoding API which allows 50/ second. the rest are Australian PO Boxes which we don't have many of - but I need to validate them against a different API that only allows 1 call per second.
I have switched over most of my code from async{} functions to task{} functions and I am assuming to get something on several threads at the same time it needs to be in an async{} function or block and be piped to Async.Parallel
Question: Is this the right way to do this or will it fall over? I am wondering if I am fundamentally thinking about this the wrong way.
Notes:
I am passing a database context into the async function and updating the database within that function
I will call this from a C# ( WPF ) Application and report the progress
Am I going to have cross threading issues?
let validateOrder
(
order: artooProvider.dataContext.``dbo.OrdersEntity``,
httpClient: HttpClient,
ctx: artooProvider.dataContext,
isAuPoBox: bool
) =
async {
// Validate Address
let! addressExceptions = ValidateAddress.validateAddress (order, httpClient, ctx, isAuPoBox) |> Async.AwaitTask
// SaveExceptions
do! ctx.SubmitUpdatesAsync()
// return Exception count
return ""
}
let validateGMapOrders(httpClient: HttpClient, ctx: artooProvider.dataContext, orders: artooProvider.dataContext.``dbo.OrdersEntity`` list) =
async {
let ordersChunked = orders |> List.chunkBySize 50
for fiftyOrders in ordersChunked do
let! tasks =
fiftyOrders
|> List.map (fun (order) -> validateOrder (order, httpClient, ctx, false) )
|> Async.Parallel
do! Async.Sleep(2000)
}
let validateOrders (ctx: artooProvider.dataContext, progress: IProgress<DownloadProgressModel>) =
task {
let unvalidatedOrders =
query {
for orders in ctx.Dbo.Orders do
where (orders.IsValidated.IsNone)
select (orders)
}
|> Seq.toList
let auPoBoxOrders =
unvalidatedOrders
|> List.filter (fun order -> isAUPoBox(order) = true )
let gMapOrders =
unvalidatedOrders
|> List.filter (fun order -> isAUPoBox(order) = false )
let googleHttpClient = new HttpClient()
let auspostHttpclient = Auspost.AuspostApi.getApiClient ()
// Google maps validations
do! validateGMapOrders(googleHttpClient,ctx,gMapOrders)
// PO Box Validations
for position in 0 .. auPoBoxOrders.Length - 1 do
let! result = validateOrder (gMapOrders[position], auspostHttpclient, ctx, true)
do! Task.Delay(1000)
return true
}
When I have had to deal with rate-limited API problems I hide that API behind a MailboxProcessor that maintains an internal time to comply with the rate limit but appears as a normal async API from the outside.
Since you have two API's with different rate limits I'd parameterise the time delay and processing action then create one object for each API.
open System
type Request = string
type Response = string
type RateLimitedProcessor() =
// Initialise 1s in past so ready to start immediately.
let mutable lastCall = DateTime.Now - TimeSpan(0, 0, 1)
let mbox = new MailboxProcessor<Request * AsyncReplyChannel<Response>>((fun mbox ->
let rec f () =
async {
let! (req, reply) = mbox.Receive()
let msSinceCall = (DateTime.Now - lastCall).Milliseconds
// wait 1s between requests
if msSinceCall < 1000 then
do! Async.Sleep (1000 - msSinceCall)
lastCall <- DateTime.Now
reply.Reply "Response"
// Call self recursively to process the next incoming message
return! f()
}
f()
))
do mbox.Start()
member __.Process(req:Request): Async<Response> =
async {
return! mbox.PostAndAsyncReply(fun reply -> req, reply)
}
interface IDisposable with
member this.Dispose() = (mbox :> IDisposable).Dispose()
I am creating UDP receiver in f# using .Net UdpCLient class and it looks simple:
let Start (ip: IPAddress, port : int32) : Async<unit> =
async {
try
let endpoint = IPEndPoint(ip, port)
use receivingClient = new UdpClient();
receivingClient.Client.Bind(endpoint)
let! receiveResult = receivingClient.ReceiveAsync() |> Async.AwaitTask
let receiveBytes = receiveResult.Buffer
printfn "%A" receiveBytes
with | ex -> raise (ex)
}
And to keep it alive I am using another property that uses rec function in it and it looks like:
let Watcher (ip: IPAddress, port : int32) : unit =
let rec listenerWatcher () =
async {
try
do! Start (ip, port)
return! listenerWatcher()
with | :? UdpClientDisposedException ->
return ()
}
listenerWatcher() |> Async.Start
and call with type is simple:
UdpReceiver.Watcher (ip, port) (* where UdpReceiver is module name *)
My problem is that I am only receiving first incoming package, like listener is shutting down after receiving first one, what could be the problem?
Maybe your problem is that you are sending packages too fast. After receiving the first package, it takes time to start again the receiver, but in the meanwhile the sender is still sending the next packages.
Not sure what is your exact intention, but I think you should start (setup) the receiver only one time, then repeatedly receive the incoming packages, and only restart the receiver when something wrong happens (exceptions are thrown).
By the way, your code is not really idiomatic in F#, you should:
Prefer separated parameters over tuples, it increases the chance of using currying.
Use type annotation only when needed, it makes code shorter.
Name functions so that they are verbs not nouns, and use camelCase style.
I would rewrite your code as below:
let start (ip: IPAddress) port =
let endpoint = IPEndPoint (ip, port)
let receivingClient = new UdpClient ()
receivingClient.Client.Bind endpoint
let rec loop () = async {
printfn "Waiting..."
let! receiveResult = receivingClient.ReceiveAsync () |> Async.AwaitTask
let receiveBytes = receiveResult.Buffer
printfn "Receive: %A" receiveBytes
return! loop ()
}
loop ()
let watch ip port =
let rec loop () = async {
try
return! start ip port
with ex ->
printfn "Error: %s" ex.Message
return! loop ()
}
loop ()
// in main function or somewhere:
watch ... ... |> Async.Start...
If you have lots of URLs to call get on, are there any queue patterns in F# with some kind of limit, like let's say 5 or 10 calls at a time before proceeding to the next batch.
let urls = [
"http://example.com/1",
"http://example.com/2",
"http://example.com/3",
....
"http://example.com/100"]
While passing a function to call
let getAsync (url:string) =
async {
let httpClient = new System.Net.Http.HttpClient()
let! response = httpClient.GetAsync(url) |> Async.AwaitTask
response.EnsureSuccessStatusCode () |> ignore
let! content = response.Content.ReadAsStringAsync() |> Async.AwaitTask
return content
}
and then receive a list of all results and any errors since some calls may fail due to any reason like 404 or 500 errors.
First, make your function actually return the error information instead of throwing it as an exception:
let getAsync url : Result<_,_> =
async {
try
...
return (Ok content)
with ex ->
return (Error ex)
}
Then use Async.Parallel to execute them in parallel. This function takes an list<Async> and returns an Async<list>:
let allResults = urls |> List.map getAsync |> Async.Parallel
Suppose I have a stream which only allows one request/response at a time but is used in several threads.
Requests/commands should be throttled such that a new request can only occur once
the previous request has been sent and a reply has been received.
The user would be able to do this
let! res = getResponse("longResp")
let! res2 = getResponse("shortResp")
and not really know or care about the throttle.
I have tried with a modified version of Tomas Petricek's Throttling Agent that allows async with return values, but this requires the user to call getResponse("..") |> Enqueue |> w.Post which is a recipe for disaster (in case they forget to do so).
Is there a good/idiomatic way of doing this in F#?
Then make it explicit in your type system that the returned type needs to be unwrapped with another function. So instead of returning an Async<'T> which as you pointed out can be called directly with Async.Start, rather return something like:
type Queuable<'T> = Queuable of Async<'T>
Then getResponse changes to return a Queueable:
let getResponse (s:string) =
let r =
async{
do! write to your stream
return! read from your stream
}
Queuable r
Provide a function that unwraps the Queuable:
let enqueue (Queuable q) = async{
return! processor.PostAndAsyncReply(fun replyChannel -> replyChannel,q)
}
The processor is an agent that simply runs the Async workflow. Something like this:
let processor = new MailboxProcessor<_>(fun inbox ->
let rec Loop() = async {
let! (r:AsyncReplyChannel<_>,job) = inbox.Receive()
let! res = job
r.Reply res
return! Loop()}
Loop())
Considering the following
type MyClass () =
member x.ReadStreamAsync(stream:Stream) =
async {
let tcs = new TaskCompletionSource<int>()
let buffer = Array.create 2048 0uy
let! bytesReadCount = stream.ReadAsync(buffer, 0, buffer.Length) |> Async.AwaitTask
if bytesReadCount > 0 then
for i in 0..bytesReadCount do
if buffer.[i] = 10uy then
tcs.SetResult(i)
// Omitted more code to handle the case if 10uy is not found..
return tcs.Task
}
The code reads from a stream until in meets a certain character (represented by a byte value) at which point the task returned by the method completes.
The function signature of DoSomethingAsync is unit -> Async<Task<int>>, but I would like it to be unit -> Task<int> such that it can be used more generally in .NET.
Can this be done in F# using an asynchronous expression, or do I can to rely more on the Task constructs of .NET?
Given that you don't actually use the async workflow for anything in your example, the easiest solution would be to forgo it entirely:
member x.DoSomethingAsync() =
let tcs = new TaskCompletionSource<int>()
Task.Delay(100).Wait()
tcs.SetResult(10)
tcs.Task
This implementation of DoSomethingAsync has the type unit -> Task<int>.
It's not clear to me exactly what you're trying to do, but why don't you just do the following?
member x.DoSomethingAsync() =
async {
do! Async.Sleep 100
return 10 } |> Async.StartAsTask
This implementation also has the type unit -> Task<int>.
Based on the updated question, here's a way to do it:
member x.DoSomethingAsync(stream:Stream) =
async {
let buffer = Array.create 2048 0uy
let! bytesReadCount =
stream.ReadAsync(buffer, 0, buffer.Length) |> Async.AwaitTask
if bytesReadCount > 0
then
let res =
[0..bytesReadCount]
|> List.tryFind (fun i -> buffer.[i] = 10uy)
return defaultArg res -1
else return -1
}
|> Async.StartAsTask
The DoSomethingAsync function has the type Stream -> System.Task<int>. I didn't know what to do in the else case, so I just put -1, but I'm sure you can replace it with something more correct.