I have this function to read data from a cache:
let private tryLoadFromCacheAsync filename =
async {
let filespec = makePath filename
match File.Exists filespec with
| true ->
let! bytes = File.ReadAllBytesAsync(filespec) |> Async.AwaitTask
use pBytes = fixed bytes
let sourceSpan = Span<byte>(NativePtr.toVoidPtr pBytes, bytes.Length)
return Some (MemoryMarshal.Cast<byte, ShortTradeData>(sourceSpan).ToArray())
| false ->
return None
}
the line of interest is here:
use pBytes = fixed bytes
it will fail to compile with:
The type 'nativeptr<'a>' is not compatible with the type 'IDisposable'
but the line will compile as:
let pBytes = fixed bytes
and this is happening in an async block.
Why is that?
The issue is less about the use part and more about the fixed part. The fixed keyword ensures that a value is kept on stack. The issue with combining this with async is that local variables used in computation expressions are not always kept on stack. If you have let! in between their uses, they need to be stored as fields of an object, so fixed cannot work in such context.
You could fix this by limiting the scope of the fixed variable to a non-async code block:
let private tryLoadFromCacheAsync filename =
async {
let filespec = makePath filename
match File.Exists filespec with
| true ->
let! bytes = File.ReadAllBytesAsync(filespec) |> Async.AwaitTask
let res =
use pBytes = fixed bytes
let sourceSpan = Span<byte>(NativePtr.toVoidPtr pBytes, bytes.Length)
MemoryMarshal.Cast<byte, ShortTradeData>(sourceSpan).ToArray()
return Some res
| false ->
return None
}
Related
I need to request data from several URLs and then use the results.
I am using plain Fable 3 with the Fable-Promise and Fable-Fetch libraries.
I have worked out how to fetch from multiple URLs and combine the results into a single Promise that I can then use to update the UI (the multiple results need to be drawn only once).
But if one of the fetch errors then the whole thing falls over. Ideally I'd like to use tryFetch and then propagate the Result<TermData, None | Exception> but nothing I do seems to compile.
How exactly do I use tryFetch and then unwrap the result with a second let! in the CE? (The comments explain more)
module App
open Browser.Dom
open App
open System.Collections.Generic
open System.Text.RegularExpressions
open Fetch
open System
type TermData =
abstract counts : int []
abstract scores : int []
abstract term : string
abstract allWords : bool
type QueryTerm =
{ mutable Term: string
mutable AllWords: bool }
let loadSingleSeries (term: QueryTerm) =
promise {
let url =
$"/api/plot/{term.Term}?allWords={term.AllWords}"
// Works but doesn't handle errors.
let! plotData = fetch url [] // type of plotData: Response
// let plotDataResult = tryFetch url []
// This ideally becomes Promise<Result<TermData, None>>
// let unwrapped = match plotDataResult with
// | Ok res -> Ok (res.json<TermData>()) // type: Promise<TermData>
// | Error err -> ??? // tried Error (Promise.create(fun resolve reject -> resolve None)) among others
let! result = plotData.json<TermData>() // type of result: TermData
return result
}
let dataArrays =
parsed // type Dictionary<int, TermData>
|> Seq.map (fun term -> loadSingleSeries term.Value)
|> Promise.Parallel
|> Promise.map (fun allData -> console.log(allData))
// Here we will handle None when we have it
I don't have much Fable experience, but if I understand your question correctly, I think the following should work:
let loadSingleSeries (term: QueryTerm) =
promise {
let url =
$"/api/plot/{term.Term}?allWords={term.AllWords}"
let! plotDataResult = tryFetch url []
match plotDataResult with
| Ok resp ->
let! termData = resp.json<TermData>()
return Ok termData
| Error ex ->
return Error ex
}
The idea here is that if you get an error, you simply propagate that error in the new Result value. This returns a Promise<Result<TermData, Exception>>, as you requested.
Update: Fixed return type using a second let!.
I haven't run this code but looking at the docs it looks like you need to use Promise.catch
let loadSingleSeries (term: QueryTerm) =
promise {
let url =
$"/api/plot/{term.Term}?allWords={term.AllWords}"
let! plotDataResult =
fetch url []
|> Promise.map Ok // Wraps the "happy path" in a Result.Ok
|> Promise.catch (fun err ->
//handle the error
Error err)
return
match plotDataResult with
| Ok res -> ...
| Error err -> ...
}
I ended up having to use the pipeline rather than CE approach for this as follows:
let loadSingleSeries (term: QueryTerm) =
let url =
$"/api/plot/{term.Term}?allWords={term.AllWords}"
let resultPromise =
fetch url []
|> Promise.bind (fun response ->
let arr = response.json<TermData> ()
arr)
|> Promise.map (Ok)
|> Promise.catch (Error)
resultPromise
The key was using Promise.bind to convert the first promise to get the Response to the promise of Promise<TermData>. The map and catch then convert to a Promise<Result<TermData, exn>>.
I am new to f# and I have a question about timeout in asynchornization operations, this is simple communication using serial com ports, so I have a method/function:
let SendMessenge(port : Ports.SerialPort, messange: string) =
async {
do! port.AsyncWriteLine messange
let! response = port.AsyncReadLine() // this returns a string
return response
}
|> Async.RunSynchronously // the place i fiddled with
All communication (messengering) is controlled in this module:
type SerialPort with
member this.AsyncWriteLine(messange : string) = this.BaseStream.AsyncWrite(this.Encoding.GetBytes(messange + "\n"))
member this.AsyncReadLine() =
async {
let messange_builder = StringBuilder()
let buffer_ref = ref (Array.zeroCreate<byte> this.ReadBufferSize)
let buffer = !buffer_ref
let last_char = ref 0uy
while !last_char <> byte '\n' do
let! readCount = this.BaseStream.AsyncRead buffer
last_char := buffer.[readCount-1]
messange_builder.Append (this.Encoding.GetString(buffer.[0 .. readCount-1])) |> ignore
messange_builder.Length <- messange_builder.Length-1
let response : string = messange_builder.ToString()
printfn "Response: %s" response
return response
}
Basically this works fine, it sends a message and receives response, But now I want to add a timeout, in case i am connect. I tried to fiddle with
|> Async.RunSynchronously(???, timeout, cancel_token)
but with no luck. As i see in documentation it takes timeout and cancellation token and Async, What would be this T0 generic parameter in my case?
What happens is that Async.RunSynchronously is a static method, rather than a method function, so it takes its arguments with tuple syntax. So you can't partially apply it and pipe the last argument into it.
You can do this:
let response = async {
// ...
}
Async.RunSynchronously(response, timeout, cancel_token)
or if you really want to pipe:
async {
// ...
}
|> fun response -> Async.RunSynchronously(response, timeout, cancel_token)
Considering the following
type MyClass () =
member x.ReadStreamAsync(stream:Stream) =
async {
let tcs = new TaskCompletionSource<int>()
let buffer = Array.create 2048 0uy
let! bytesReadCount = stream.ReadAsync(buffer, 0, buffer.Length) |> Async.AwaitTask
if bytesReadCount > 0 then
for i in 0..bytesReadCount do
if buffer.[i] = 10uy then
tcs.SetResult(i)
// Omitted more code to handle the case if 10uy is not found..
return tcs.Task
}
The code reads from a stream until in meets a certain character (represented by a byte value) at which point the task returned by the method completes.
The function signature of DoSomethingAsync is unit -> Async<Task<int>>, but I would like it to be unit -> Task<int> such that it can be used more generally in .NET.
Can this be done in F# using an asynchronous expression, or do I can to rely more on the Task constructs of .NET?
Given that you don't actually use the async workflow for anything in your example, the easiest solution would be to forgo it entirely:
member x.DoSomethingAsync() =
let tcs = new TaskCompletionSource<int>()
Task.Delay(100).Wait()
tcs.SetResult(10)
tcs.Task
This implementation of DoSomethingAsync has the type unit -> Task<int>.
It's not clear to me exactly what you're trying to do, but why don't you just do the following?
member x.DoSomethingAsync() =
async {
do! Async.Sleep 100
return 10 } |> Async.StartAsTask
This implementation also has the type unit -> Task<int>.
Based on the updated question, here's a way to do it:
member x.DoSomethingAsync(stream:Stream) =
async {
let buffer = Array.create 2048 0uy
let! bytesReadCount =
stream.ReadAsync(buffer, 0, buffer.Length) |> Async.AwaitTask
if bytesReadCount > 0
then
let res =
[0..bytesReadCount]
|> List.tryFind (fun i -> buffer.[i] = 10uy)
return defaultArg res -1
else return -1
}
|> Async.StartAsTask
The DoSomethingAsync function has the type Stream -> System.Task<int>. I didn't know what to do in the else case, so I just put -1, but I'm sure you can replace it with something more correct.
Not sure if I got this right or whether there's a better way or an existing library solving this problem already.
In particular I'm not sure if the CAS would need a memory fence... I think not but better ask.
I also tried with an agent and mutable dictionary but my intuition that it would be slower was confirmed and the implementation was more involved.
module CAS =
open System.Threading
let create (value: 'T) =
let cell = ref value
let get () = !cell
let rec swap f =
let before = get()
let newValue = f before
match Interlocked.CompareExchange<'T>(cell, newValue, before) with
| result when obj.ReferenceEquals(before, result) ->
newValue
| _ ->
swap f
get, swap
module Memoization =
let timeToLive milis f =
let get, swap = CAS.create Map.empty
let evict key =
async {
do! Async.Sleep milis
swap (Map.remove key) |> ignore
} |> Async.Start
fun key ->
let data = get()
match data.TryFind key with
| Some v -> v
| None ->
let v = f key
swap (Map.add key v) |> ignore
evict key
v
If you are willing to limit what to memoize to functions that take a string input, you can reuse the functionality from System.Runtime.Caching.
This should be reasonably robust as part of the core library (you would hope...) but the string limitation is a pretty heavy one and you'd have to benchmark against your current implementation if you want to do a comparison on performance.
open System
open System.Runtime.Caching
type Cached<'a>(func : string -> 'a, cache : IDisposable) =
member x.Func : string -> 'a = func
interface IDisposable with
member x.Dispose () =
cache.Dispose ()
let cache timespan (func : string -> 'a) =
let cache = new MemoryCache(typeof<'a>.FullName)
let newFunc parameter =
match cache.Get(parameter) with
| null ->
let result = func parameter
let ci = CacheItem(parameter, result :> obj)
let cip = CacheItemPolicy()
cip.AbsoluteExpiration <- DateTimeOffset(DateTime.UtcNow + timespan)
cip.SlidingExpiration <- TimeSpan.Zero
cache.Add(ci, cip) |> ignore
result
| result ->
(result :?> 'a)
new Cached<'a>(newFunc, cache)
let cacheAsync timespan (func : string -> Async<'a>) =
let cache = new MemoryCache(typeof<'a>.FullName)
let newFunc parameter =
match cache.Get(parameter) with
| null ->
async {
let! result = func parameter
let ci = CacheItem(parameter, result :> obj)
let cip = CacheItemPolicy()
cip.AbsoluteExpiration <- DateTimeOffset(DateTime.UtcNow + timespan)
cip.SlidingExpiration <- TimeSpan.Zero
cache.Add(ci, cip) |> ignore
return result
}
| result ->
async { return (result :?> 'a) }
new Cached<Async<'a>>(newFunc, cache)
Usage:
let getStuff =
let cached = cacheAsync (TimeSpan(0, 0, 5)) uncachedGetStuff
// deal with the fact that the cache is IDisposable here
// however is appropriate...
cached.Func
If you're never interested in accessing the underlying cache directly you can obviously just return a new function with the same signature of the old - but given the cache is IDisposable, that seemed unwise.
I think in many ways I prefer your solution, but when I faced a similar problem I had a perverse thought that I should really use the built in stuff if I could.
This is C# version:
public static IEnumerable<string> ReadLinesEnumerable(string path) {
using ( var reader = new StreamReader(path) ) {
var line = reader.ReadLine();
while ( line != null ) {
yield return line;
line = reader.ReadLine();
}
}
}
But directly translating needs a mutable variable.
If you're using .NET 4.0, you can just use File.ReadLines.
> let readLines filePath = System.IO.File.ReadLines(filePath);;
val readLines : string -> seq<string>
open System.IO
let readLines (filePath:string) = seq {
use sr = new StreamReader (filePath)
while not sr.EndOfStream do
yield sr.ReadLine ()
}
To answer the question whether there is a library function for encapsulating this pattern - there isn't a function exactly for this, but there is a function that allows you to generate sequence from some state called Seq.unfold. You can use it to implement the functionality above like this:
new StreamReader(filePath) |> Seq.unfold (fun sr ->
match sr.ReadLine() with
| null -> sr.Dispose(); None
| str -> Some(str, sr))
The sr value represents the stream reader and is passed as the state. As long as it gives you non-null values, you can return Some containing an element to generate and the state (which could change if you wanted). When it reads null, we dispose it and return None to end the sequence. This isn't a direct equivalent, because it doesn't properly dispose StreamReader when an exception is thrown.
In this case, I would definitely use sequence expression (which is more elegant and more readable in most of the cases), but it's useful to know that it could be also written using a higher-order function.
let lines = File.ReadLines(path)
// To check
lines |> Seq.iter(fun x -> printfn "%s" x)
On .NET 2/3 you can do:
let readLines filePath = File.ReadAllLines(filePath) |> Seq.cast<string>
and on .NET 4:
let readLines filePath = File.ReadLines(filePath);;
In order to avoid the "System.ObjectDisposedException: Cannot read from a closed TextReader." exception, use:
let lines = seq { yield! System.IO.File.ReadLines "/path/to/file.txt" }