async programming, async's in asycns - f#

I have a DU (don't worry about the specific types, but 2 normal options, or some errors)
type Content =
| Episode of EpisodeJSON.Root
| Program of string
| Errors of List<exn>
I have 2 functions (again the specifics don't matter, just the types):
let getEpisode : _ -> _ -> Async<Content> = ...
let getProgram : _ -> _ -> Async<Content> = ...
I want to write
let getContent : _ -> _ -> Async<Content> =
fun (contentBAPIClient: ContentBAPI.Client) id -> ...
such that, it tries to get the data from getEpisode, if there's an error, it tries to get it from getProgram, and if there is an error again, it returns both errors as the Error DU.
so, this works
let getContent : _ -> _ -> Async<Content> =
fun (contentBAPIClient: ContentBAPI.Client) id ->
async {
let! episodeMaybe = getEpisode contentBAPIClient id
let! programMaybe = getProgram contentBAPIClient id
return
match episodeMaybe with
| Errors xs ->
match programMaybe with
| Errors ys ->
Errors (List.append xs ys)
| program ->
program
| episode ->
episode
}
but I notice that getprogram is being executed even when the data is found via getepisode.
How do i structure this simple function to try getEpisode first, then only try getProgram if episode 'fails'
this works, still feels a bit clunky
let getContent : _ -> _ -> Async<Content> =
fun (contentBAPIClient: ContentBAPI.Client) id ->
async {
let! episodeMaybe = getEpisode contentBAPIClient id
match episodeMaybe with
| Errors xs ->
let! programMaybe = getProgram contentBAPIClient id
match programMaybe with
| Errors ys ->
return Errors (List.append xs ys)
| program ->
return program
| episode ->
return episode
}

This will get a lot easier if you use some library that lets you work with values representing asynchronous computations that may fail, i.e. type Async<Result<'TOk, 'TErrror>>.
A good choice would be something like FsToolkit.ErrorHandling. This defines the asyncResult computation expression as well as a number of primitives that you may find useful.
To get this to compile, I started with this:
#r "nuget: FsToolkit.ErrorHandling"
open FsToolkit.ErrorHandling
type Client = class end
let getEpisode contentBAPIClient id =
async { return Error ["no episode"] }
let getProgram contentBAPIClient id =
async { return Ok "fine" }
Now, to do what (I think) you are doing, you can use:
let getContent (contentBAPIClient: Client) id =
getEpisode contentBAPIClient id
|> AsyncResult.orElseWith (fun e1 ->
getProgram contentBAPIClient id
|> AsyncResult.mapError (fun e2 -> e1 # e2) )
Here, we try to return the episode, but if there is not one, we try to get program (using orElseWith). The only tricky thing is that we need to pass the errors from the first call, which are then appended to (potential) errors of the second call using mapError.
Note that you can also write:
let getContent (contentBAPIClient: Client) id =
asyncResult {
let! ep = getEpisode contentBAPIClient id
and! prog = getProgram contentBAPIClient id
return ep, prog }
This succeeds only if both calls succeed, so it can be quite useful in many scenarios, but does not exactly do the thing that you are trying to do.

Related

how can I combine / compose computation expressions, in F#?

This is not for a practical need, but rather to try to learn something.
I am using FSToolKit's asyncResult expression which is very handy and I would like to know if there is a way to 'combine' expressions, such as async and result here, or does a custom expression have to be written?
Here is an example of my function to set the ip to a subdomain, with CloudFlare:
let setSubdomainToIpAsync zoneName url ip =
let decodeResult (r: CloudFlareResult<'a>) =
match r.Success with
| true -> Ok r.Result
| false -> Error r.Errors.[0].Message
let getZoneAsync (client: CloudFlareClient) =
asyncResult {
let! r = client.Zones.GetAsync()
let! d = decodeResult r
return!
match d |> Seq.filter (fun x -> x.Name = zoneName) |> Seq.toList with
| z::_ -> Ok z // take the first one
| _ -> Error $"zone '{zoneName}' not found"
}
let getRecordsAsync (client: CloudFlareClient) zoneId =
asyncResult {
let! r = client.Zones.DnsRecords.GetAsync(zoneId)
return! decodeResult r
}
let updateRecordAsync (client: CloudFlareClient) zoneId (records: DnsRecord seq) =
asyncResult {
return!
match records |> Seq.filter (fun x -> x.Name = url) |> Seq.toList with
| r::_ -> client.Zones.DnsRecords.UpdateAsync(zoneId, r.Id, ModifiedDnsRecord(Name = url, Content = ip, Type = DnsRecordType.A, Proxied = true))
| [] -> client.Zones.DnsRecords.AddAsync(zoneId, NewDnsRecord(Name = url, Content = ip, Proxied = true))
}
asyncResult {
use client = new CloudFlareClient(Credentials.CloudFlare.Email, Credentials.CloudFlare.Key)
let! zone = getZoneAsync client
let! records = getRecordsAsync client zone.Id
let! update = updateRecordAsync client zone.Id records
return! decodeResult update
}
It is interfacing with a C# lib that handles all the calls to the CloudFlare API and returns a CloudFlareResult object which has a success flag, a result and an error.
I remapped that type to a Result<'a, string> type:
let decodeResult (r: CloudFlareResult<'a>) =
match r.Success with
| true -> Ok r.Result
| false -> Error r.Errors.[0].Message
And I could write an expression for it (hypothetically since I've been using them but haven't written my own yet), but then I would be happy to have an asyncCloudFlareResult expression, or even an asyncCloudFlareResultOrResult expression, if that makes sense.
I am wondering if there is a mechanism to combine expressions together, the same way FSToolKit does (although I suspect it's just custom code there).
Again, this is a question to learn something, not about the practicality since it would probably add more code than it's worth.
Following Gus' comment, I realized it would be good to illustrate the point with some simpler code:
function DoA : int -> Async<AWSCallResult<int, string>>
function DoB : int -> Async<Result<int, string>>
AWSCallResultAndResult {
let! a = DoA 3
let! b = DoB a
return b
}
in this example I would end up with two types that can take an int and return an error string, but they are different. Both have their expressions so I can chain them as needed.
And the original question is about how these can be combined together.
It's possible to extend CEs with overloads.
The example below makes it possible to use the CustomResult type with a usual result builder.
open FsToolkit.ErrorHandling
type CustomResult<'T, 'TError> =
{ IsError: bool
Error: 'TError
Value: 'T }
type ResultBuilder with
member inline _.Source(result : CustomResult<'T, 'TError>) =
if result.IsError then
Error result.Error
else
Ok result.Value
let computeA () = Ok 42
let computeB () = Ok 23
let computeC () =
{ CustomResult.Error = "oops. This went wrong"
CustomResult.IsError = true
CustomResult.Value = 64 }
let computedResult =
result {
let! a = computeA ()
let! b = computeB ()
let! c = computeC ()
return a + b + c
}

FsCheck lazy generators

I have issues with generation of data within my tests.
testProperty "calculate Operation against different operations should increase major" <| fun operationIdApi operationIdClient summaryApi summaryClient descriptionApi descriptionClient ->
( notAllEqual [
fun () -> assessEquality <| StringEquals(operationIdApi, operationIdClient)
fun () -> assessEquality <| StringEquals(summaryApi , summaryClient)
fun () -> assessEquality <| StringEquals(descriptionApi, descriptionClient)
]) ==> lazy (
let operationClient = createOpenApiOperation operationIdClient summaryClient descriptionClient
let operationAPI = createOpenApiOperation operationIdApi summaryApi descriptionApi
let actual = calculate operationAPI operationClient
Expect.equal actual (Fact.Semver.IncreaseMajor) "return IncreaseMajor"
)
The code that is actually tested is :
semver {
if operationAPI.OperationId<> operationClient.OperationId then yield! IncreaseMajor
if operationAPI.Summary <> operationClient.Summary then yield! IncreaseMajor
}
The test should fail when the data produced is same OperationId, same summary and different description.
But it does not and it led me to create my own generator or at least try to do so:
I wanted my test to be written like this :
testProperty "calculate Operation against different operations should increase major" <| fun (operationId:ElementSet<string>) (summary:ElementSet<string>) ->
Therefore I create a type accordingly:
type ElementSet<'a> =
| Same of 'a
| Different
and a generator for this type :
let setGen<'a> =
Gen.oneof [
gen {
let! v = Arb.generate<'a>
return Same(v)
}
gen { return Different}
]
type ElementSetGenerator =
static member ElementSet() =
Arb.fromGen setGen<'a>
do Arb.register<ElementSetGenerator>() |> ignore
I was then trying to extract the data to construct my object :
let createOpenApiOperation operationId summary=
let pi = OpenApiOperation(OperationId=operationId.Get, Summary=summary.Get)
pi
The Get method did not exist yet so I was about to implement it by adding a member to my ElementSet<'a>:
type ElementSet<'a> =
| Same of 'a
| Different
with member this.Get =
match this with
| Same s -> s
| Different -> Arb.generate<'a>// some random generation here
And this is where I am stuck. I would love to get some randomness here when I extract data. I wonder if this is the correct way to do so, or if I should have answered the problem earlier?
Thanks for your inputs.
I think I found it, the answer was to handle it at the beginning :
let setGen<'a when 'a:equality> =
Gen.oneof [
gen {
let! v = Arb.generate<'a>
return Same(v)
}
gen {
let! x,y =
Arb.generate<'a>
|> Gen.two
|> Gen.filter (fun (a,b)-> a <> b)
return Different(x,y)
}
]
and then to use two getter to access the values :
type ElementSet<'a> when 'a:equality=
| Same of 'a
| Different of 'a*'a
with member this.Fst = match this with | Same s -> s | Different (a, b)-> a
member this.Snd = match this with | Same s -> s | Different (a, b)-> b
this way I can access values within my test:
testProperty "calculate Operation against different operations should increase major" <| fun (operationId:ElementSet<NonWhiteSpaceString>) (summary:ElementSet<NonWhiteSpaceString>) (description:ElementSet<NonWhiteSpaceString>) ->
let operationClient = createOpenApiOperation operationId.Fst summary.Fst description.Fst
let operationAPI = createOpenApiOperation operationId.Snd summary.Snd description.Snd
let actual = calculate operationAPI operationClient
Expect.equal actual (Fact.Semver.IncreaseMajor) "return IncreaseMajor"
for the record I then have the creation of my stub as follows :
let createOpenApiOperation (operationId:NonWhiteSpaceString) (summary:NonWhiteSpaceString) (description:NonWhiteSpaceString)=
let pi = OpenApiOperation(OperationId=operationId.Get, Summary=summary.Get, Description=description.Get)
pi

How to implement Task.Map

Have I correctly implemented map for Task?
let map continuation (t: Task<'A>) =
t.ContinueWith(fun (antecedent: Task<'A>) ->
if antecedent.Status <> TaskStatus.Canceled &&
antecedent.Status <> TaskStatus.Faulted then
continuation antecedent.Result
else
raise antecedent.Exception // must I?
)
I got the TaskStatus checks from the docs. I feel most uncertain about raise antecedent.Exception, but I can't think of another way to handle it.
As background, yes I'm aware of Async, but my current stack uses Entity Framework and Blazor, so I have a backend that uses things like .ToListAsync() and a front end in C#, so I'd rather just not deal with converting from Task to Async then back again.
I would suggest implementing your solution in terms of the interfaces behind the concept of awaitable in the TPL, namely INotifyCompletion and ICriticalNotifyCompletion. Also, to implement map correctly, you should really do it in terms of bind. This is something that there are already some existing solutions for in F#, such as the TaskBuilder library. Personally, I have been using the following in a library for years without any issues:
open System.Runtime.CompilerServices
open System.Threading.Tasks
type TaskStep<'result> =
| Value of 'result
| AsyncValue of 'result Task
| Continuation of ICriticalNotifyCompletion * (unit -> 'result TaskStep)
and StateMachine<'a>(firstStep) as this =
let methodBuilder = AsyncTaskMethodBuilder<'a Task>()
let mutable continuation = fun () -> firstStep
let nextAwaitable() =
try
match continuation() with
| Value r ->
methodBuilder.SetResult(Task.FromResult(r))
null
| AsyncValue t ->
methodBuilder.SetResult(t)
null
| Continuation (await, next) ->
continuation <- next
await
with
| exn ->
methodBuilder.SetException(exn)
null
let mutable self = this
member __.Run() =
methodBuilder.Start(&self)
methodBuilder.Task
interface IAsyncStateMachine with
member __.MoveNext() =
let mutable await = nextAwaitable()
if not (isNull await) then
methodBuilder.AwaitUnsafeOnCompleted(&await, &self)
member __.SetStateMachine(_) =
()
type Binder<'out> =
static member inline GenericAwait< ^abl, ^awt, ^inp
when ^abl : (member GetAwaiter : unit -> ^awt)
and ^awt :> ICriticalNotifyCompletion
and ^awt : (member get_IsCompleted : unit -> bool)
and ^awt : (member GetResult : unit -> ^inp) >
(abl : ^abl, continuation : ^inp -> 'out TaskStep) : 'out TaskStep =
let awt = (^abl : (member GetAwaiter : unit -> ^awt)(abl))
if (^awt : (member get_IsCompleted : unit -> bool)(awt))
then continuation (^awt : (member GetResult : unit -> ^inp)(awt))
else Continuation (awt, fun () -> continuation (^awt : (member GetResult : unit -> ^inp)(awt)))
module TaskStep =
let inline bind f step : TaskStep<'a> =
Binder<'a>.GenericAwait(step, f)
let inline toTask (step: TaskStep<'a>) =
try
match step with
| Value x -> Task.FromResult(x)
| AsyncValue t -> t
| Continuation _ as step -> StateMachine<'a>(step).Run().Unwrap()
with
| exn ->
let src = new TaskCompletionSource<_>()
src.SetException(exn)
src.Task
module Task =
let inline bind f task : Task<'a> =
TaskStep.bind f task |> TaskStep.toTask
let inline map f task : Task<'b> =
bind (f >> Value) task
FsToolkit.ErrorHandling implements it here. I'll paste the current version below as it's quite short. It uses the TaskBuilder library Aaron mentioned.
module Task =
let singleton value = value |> Task.FromResult
let bind (f : 'a -> Task<'b>) (x : Task<'a>) = task {
let! x = x
return! f x
}
let map f x = x |> bind (f >> singleton)
Additionally, FSharpPlus has an independent implementation of Task.map here.
Throwing the exception again in the continuation would make for an incorrect stack trace.
It's a mapping from 'A -> 'B, so it's probably best to lay it out explicitly.
let rec map (continuation: 'A -> 'B) (t: Task<'A>) =
let rec map_resolved (task: Task<'A>) =
match task.Status with
| TaskStatus.RanToCompletion -> Task.FromResult(continuation task.Result)
| TaskStatus.Faulted -> Task.FromException<'B>(task.Exception)
| TaskStatus.Canceled -> Task.FromCanceled<'B>(CancellationToken.None)
| _ -> task.ContinueWith(map_resolved).Unwrap()
map_resolved t

Why does type inference only work with a pipe here?

I have a piece of code like this:
let! deliveries =
async {
match Option.map (fun x -> x.Address) maybeUser with
| Some "" -> return []
| Some address -> return! fetchDeliveries address
| None -> return []
}
The type inference on this fails at x.Address.
However, if I reorder the code with a pipe it works:
let! deliveries =
async {
match maybeUser |> Option.map (fun x -> x.Address) with
| Some "" -> return []
| Some address -> return! fetchDeliveries address
| None -> return []
}
Why is this?
This is because F#'s compiler is a single-pass, top-down, left-to-right compiler. So when typechecking a file, any type information from the left of an expression is able to be used to verify the right. The reason why the |> example works is because the left-hand side has a definite type (in this case some kind of User option type that you've defined, which tells Option.map that yes the item coming in is an User option, so the lambda function you pass in must be of type User -> something else, which of course it is.

Parameterizing/Extracting Discriminated Union cases

Currently i'm working in a game and use Event/Observables much, one thing i run into was to eliminate some redundant code, and i didn't found a way to do it. To explain it, let's assume we have following DU and an Observable for this DU.
type Health =
| Healed
| Damaged
| Died
| Revived
let health = Event<Health>()
let pub = health.Publish
I have a lot of this kind of structures. Having all "Health" Messages grouped together is helpfull and needed in some situations, but in some situations i only care for a special Message. Because that is still often needed i use Observable.choose to separate those message. I have then code like this.
let healed = pub |> Observable.choose (function
| Healed -> Some ()
| _ -> None
)
let damaged = pub |> Observable.choose (function
| Damaged -> Some ()
| _ -> None
)
Writing this kind of code is actually pretty repetitive and annoying. I have a lot of those types and messages. So one "rule" of functional programming is "Parametrize all the things". So i wrote a function only to just help me.
let only msg pub = pub |> Observable.choose (function
| x when x = msg -> Some ()
| _ -> None
)
With such a function in place, now the code becomes a lot shorter and less annoying to write.
let healed = pub |> only Healed
let damaged = pub |> only Damaged
let died = pub |> only Died
let revived = pub |> only Revived
EDIT:
The important thing to note. healed, damaged, died, revived are now of type IObservable<unit> not IObservable<Health>. The idea is not just to separate the messages. This can be easily achieved with Observable.filter. The idea is that the the data for each case additional get extracted. For DU case that don't carry any additional data this is easy, as i only have to write Some () in the Observable.choose function.
But this only works, as long the different cases in a DU don't expect additional values. Unlucky i also have a lot of cases that carry additional information. For example instead of Healed or Damaged i have HealedBy of int. So a message also contains additional how much something got healed. What i'm doing is something like this, in this case.
let healedBy = pub |> Observable.choose (function
| HealedBy x -> Some x
| _ -> None
)
But what i really want is to write it something like this
let healedBy = pub |> onlyWith HealeadBy
What i'm expecting is to get an Observable<int>. And i didn't found any way how to do it. I cannot write a function like only above. because when i try to evaluate msg inside a Pattern Matching then it is just seen as a variable to Pattern Match all cases. I cannot say something like: "Match on the case inside the variable."
I can check if a variable is of some specific case. I can do if x = HealedBy then but after that, i cannot extract any kind of data from x. What i'm really need is something like an "unsecure" extracting like option for example provide it with optional.Value. Does there exists any way to implement such a "onlyWith" function to remove the boilerplate?
EDIT:
The idea is not just separating the different messages. This can be achieved through Observable.filter. Here healedBy is of type IObservable<int> NOT IObservable<Health> anymore. The big idea is to separate the messages AND extract the data it carries along AND doing it without much boilerplate. I already can separate and extract it in one go with Observable.choose currently. As long as a case don't have any additional data i can use the only function to get rid of the boilerplate.
But as soon a case has additional data i'm back at writing the repetitive Observable.Choose function and do all the Pattern Matching again. The thing is currently i have code like this.
let observ = pub |> Observable.choose (function
| X (a) -> Some a
| _ -> None
)
And i have this kind of stuff for a lot of messages and different types. But the only thing that changes is the "X" in it. So i obviously want to Parameterize "X" so i don't have to write the whole construct again and again. At best it just should be
let observ = anyObservable |> onlyWith CaseIWantToSeparate
But the new Observable is of the type of the specific case i separated. Not the type of the DU itself.
The behaviour it appears you are looking for doesn't exist, it works fine in your first example because you can always consistently return a unit option.
let only msg pub =
pub |> Observable.choose (function
| x when x = msg -> Some ()
| _ -> None)
Notice that this has type: 'a -> IObservable<'a> -> IObservable<unit>
Now, let's imagine for the sake of creating a clear example that I define some new DU that can contain several types:
type Example =
|String of string
|Int of int
|Float of float
Imagine, as a thought exercise, I now try to define some general function that does the same as the above. What might its type signature be?
Example -> IObservable<Example> -> IObservable<???>
??? can't be any of the concrete types above because the types are all different, nor can it be a generic type for the same reason.
Since it's impossible to come up with a sensible type signature for this function, that's a pretty strong implication that this isn't the way to do it.
The core of the problem you are experiencing is that you can't decide on a return type at runtime, returning a data type that can be of several different possible but defined cases is precisely the problem discriminated unions help you solve.
As such, your only option is to explicitly handle each case, you already know or have seen several options for how to do this. Personally, I don't see anything too horrible about defining some helper functions to use:
let tryGetHealedValue = function
|HealedBy hp -> Some hp
|None -> None
let tryGetDamagedValue = function
|DamagedBy dmg -> Some dmg
|None -> None
The usual route in these situations is to define predicates for cases, and then use them for filtering:
type Health = | Healed | Damaged | Died | Revived
let isHealed = function | Healed -> true | _ -> false
let isDamaged = function | Damaged -> true | _ -> false
let isDied = function | Died -> true | _ -> false
let isRevived = function | Revived -> true | _ -> false
let onlyHealed = pub |> Observable.filter isHealed
UPDATE
Based on your comment: if you want to not only filter messages, but also unwrap their data, you can define similar option-typed functions and use them with Observable.choose:
type Health = | HealedBy of int | DamagedBy of int | Died | Revived
let getHealed = function | HealedBy x -> Some x | _ -> None
let getDamaged = function | DamagedBy x -> Some x | _ -> None
let getDied = function | Died -> Some() | _ -> None
let getRevived = function | Revived -> Some() | _ -> None
let onlyHealed = pub |> Observable.choose getHealed // : Observable<int>
let onlyDamaged = pub |> Observable.choose getDamaged // : Observable<int>
let onlyDied = pub |> Observable.choose getDied // : Observable<unit>
You can use reflection to do this I think. This might be pretty slow:
open Microsoft.FSharp.Reflection
type Health =
| Healed of int
| Damaged of int
| Died
| Revived
let GetUnionCaseInfo (x:'a) =
match FSharpValue.GetUnionFields(x, typeof<'a>) with
| case, [||] -> (case.Name, null )
| case, value -> (case.Name, value.[0] )
let health = Event<Health>()
let pub = health.Publish
let only msg pub = pub |> Observable.choose (function
| x when x = msg -> Some(snd (GetUnionCaseInfo(x)))
| x when fst (GetUnionCaseInfo(x)) = fst (GetUnionCaseInfo(msg))
-> Some(snd (GetUnionCaseInfo(x)))
| _ -> None
)
let healed = pub |> only (Healed 0)
let damaged = pub |> only (Damaged 0)
let died = pub |> only Died
let revived = pub |> only Revived
[<EntryPoint>]
let main argv =
let healing = Healed 50
let damage = Damaged 100
let die = Died
let revive = Revived
healed.Add (fun i ->
printfn "We healed for %A." i)
damaged.Add (fun i ->
printfn "We took %A damage." i)
died.Add (fun i ->
printfn "We died.")
revived.Add (fun i ->
printfn "We revived.")
health.Trigger(damage)
//We took 100 damage.
health.Trigger(die)
//We died.
health.Trigger(healing)
//We healed for 50.
health.Trigger(revive)
//We revived.
0 // return an integer exit code
It doesn't feel like you can get your onlyWith function without making some significant changes elsewhere. You can't really generalize the function you pass in for the HealedBy case while staying within the type system (I suppose you could cheat with reflection).
One thing that seems like a good idea would be to introduce a wrapper for the Healed type instead of having a HealedBy type:
type QuantifiedHealth<'a> = { health: Health; amount: 'a }
and then you can have an onlyWith function like this:
let onlyWith msg pub =
pub |> Observable.choose (function
| { health = health; amount = amount } when health = msg -> Some amount
| _ -> None)
I guess you can even go one step further while you're at it, and parameterize your type by both the label and the amount types to make it truly generic:
type Quantified<'label,'amount> = { label: 'label; amount: 'amount }
Edit: To reitarate, you keep this DU:
type Health =
| Healed
| Damaged
| Died
| Revived
Then you make your health event - still a single one - use the Quantified type:
let health = Event<Quantified<Health, int>>()
let pub = health.Publish
You can trigger the event with messages like { label = Healed; amount = 10 } or { label = Died; amount = 0 }. And you can use the only and onlyWith functions to filter and project the event stream to IObservable<unit> and IObservable<int> respectively, without introducing any boilerplate filtering functions.
let healed : IObservable<int> = pub |> onlyWith Healed
let damaged : IObservable<int> = pub |> onlyWith Damaged
let died : IObservable<unit> = pub |> only Died
let revived : IObservable<unit> = pub |> only Revived
The label alone is enough to differentiate between records representing "Healed" and "Died" cases, you no longer need to walk around the payload you would have in your old "HealedBy" case. Also, if you now add a Mana or Stamina DU, you can reuse the same generic functions with Quantified<Mana, float> type etc.
Does this make sense to you?
Arguably it's slightly contrived compared to a simple DU with "HealedBy" and "DamagedBy", but it does optimize the use case that you care for.

Resources