I have two lists listA and listB where I want to return true if listB contains any element also in listA.
let listA = ["A";"B";"C"]
let listB = ["D";"E";"A"]
Should return true in this case. I feel like this should be easy to solve and I'm missing something fundamental somewhere.
For example, why can't I do like this?
let testIntersect = for elem in listA do List.exists (fun x -> x = elem) listB
You can't write something like your example code because a plain for doesn't return a result, it just evaluates an expression for its side-effects. You could write the code in a for comprehension:
let testIntersect listA listB =
[for elem in listA do yield List.exists (fun x -> x = elem) listB]
Of course, this then returns a bool list rather than a single bool.
val testIntersect :
listA:seq<'a> -> listB:'a list -> bool list when 'a : equality
let listA = ["A";"B";"C"]
let listB = ["D";"E";"A"]
testIntersect listA listB
val it : bool list = [true; false; false]
So, we can use the List.exists function to ensure that a true occurs at least once:
let testIntersect listA listB =
[for elem in listA do yield List.exists (fun x -> x = elem) listB]
|> List.exists id
val testIntersect :
listA:seq<'a> -> listB:'a list -> bool list when 'a : equality
val listA : string list = ["A"; "B"; "C"]
val listB : string list = ["D"; "E"; "A"]
val it : bool = false
It's pretty inefficient to solve this problem using List though, it's better to use Set. With Set, you can calculate intersection in O(log N * log M) time rather than O(N*M).
let testSetIntersect listA listB =
Set.intersect (Set.ofList listA) (Set.ofList listB)
|> Set.isEmpty
|> not
One function that you could use is List.except, which is not yet documented (!) but can be seen in this pull request that was merged a couple of years ago. You'd probably use it like this:
let testIntersect a b =
let b' = b |> List.except a
// If b' is shorter than b, then b contained at least one element of a
List.length b' < List.length b
However, this runs through list B about three times, once to do the except algorithm and once each to do both the length calls. So another approach might be to do what you did, but turn list A into a set so that the exists call won't be O(N):
let testIntersect a b =
let setA = a |> Set.ofList
match b |> List.tryFind (fun x -> setA |> Set.contains x) with
| Some _ -> true
| None -> false
The reason I used tryFind is because List.find would throw an exception if the predicate didn't match any items of the list.
Edit: An even better approach is to use List.exists, which I temporarily forgot about (thanks to Honza Brestan for reminding me about it):
let testIntersect a b =
let setA = a |> Set.ofList
b |> List.exists (fun x -> setA |> Set.contains x)
Which, of course, is pretty much what you were originally wanting to do in your testIntersect code sample. The only difference is that you were using the for ... in syntax in your code sample, which wouldn't work. In F#, the for loop is exclusively for expressions that return unit (and thus, probably have side effects). If you want to return a value, the for loop won't do that. So using the functions that do return value, like List.exists, is the approach you want to take.
let testIntersect listA listB =
(Set.ofList listA) - (Set.ofList listB) |> Set.isEmpty |> not
I have the following code
let rec consume() : Async<unit> = async {
.....
listA
|> Seq.iter(fun i ->
.....
let listB : seq<...> option =
let c = getListB a b
match c with
| Some d -> Seq.filter(....) |> Some
| None -> None
match listB with .....
....
Now the function getListB is converted to return async<Seq<B>> instead of Seq<B>. So the code was converted to the following. However, the getListB blocked the execution. How to rewrite it nonblocking? Simply convert the line to let! c = getListB a b won't work because the code is in an inner lambda? The error message is "This construct may only be used within computation expressions".
let rec consume() : Async<unit> = async {
.....
listA
|> Seq.iter(fun i ->
.....
let listB : seq<...> option =
let c = getListB a b |> Async.RunSynchronously
match c with
| Some d -> Seq.filter(....) |> Some
| None -> None
I believe the problem you are describing boils down to how to convert an seq<Async> to an Async<seq>. This is described comprehensively in this post by Scott Wlaschin.
This is a poor man's implementation of the concepts described in his post which are far more powerful and generic. The general idea is that we want to delay the creation of the sequence until we have the values promised by the instance of Async<_>
let traverseSequence ( seqAsync : seq<Async<'a>>) =
let promiseOfAnEmptySequence = async { return Seq.empty }
let delayedCalculation (asyncHead : Async<'a>) (asyncTail : Async<seq<'a>>) =
async {
let! calculatedHead = asyncHead
return!
async {
let! calculatedTail = asyncTail
return calculatedHead |> Seq.singleton |> Seq.append(calculatedTail)
}
}
Seq.foldBack delayedCalculation seqAsync promiseOfAnEmptySequence
The answer depends on whether you want to run each element of the sequence sequentially or in parallel.
In both cases, start by using Seq.map instead of Seq.iter, then you can put another async block inside the lambda such that the result of the map is seq<Async<'a>>.
Sequential
For this, you need define some extra functions in an extra Async module.
module Async =
let map f x =
async{
let! x = x
return f x
}
let lift2 f x1 x2 =
async{
let! x1 = x1
let! x2 = x2
return f x1 x2
}
let return' x = async { return x }
let mapM mFunc sequ =
let consF x ys = lift2 (fun h t -> h::t) (mFunc x) ys
Seq.foldBack(consF) sequ (return' [])
|> map (Seq.ofList)
let sequence sequ = mapM id sequ
You might have seen mapM called traverse elsewhere, they are basically just different names for the same concept.
The sequence function is just a special case of mapM where the supplied binding function is just the identity (id) function. It has type seq<Async<'a>> -> Async<seq<'a>>, i.e. it flips the Async from being inside the Seq to being outside.
You then simply pipe the result of your Seq.map to the sequence function, which gives you an async value.
Your example code isn't complete so I made up some example code to use this:
let sleep = Async.Sleep 100
let sleeps = Seq.init 15 (fun _ -> sleep)
let sequencedSleeps = Async.sequence sleeps
Async.RunSynchronously sequencedSleeps
Real: 00:00:01.632, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0
val it : seq<unit> =
[null; null; null; null; null; null; null; null; null; null; null; null;
null; null; null]
Parallel
To execute each element of the sequence in parallel, instead of sequentially, you could do:
let pSequence sequ = Async.Parallel sequ |> Async.map (Seq.ofArray)
Example test code:
let pSleeps = pSequence sleeps;;
Async.RunSynchronously pSleeps;;
Real: 00:00:00.104, CPU: 00:00:00.000, GC gen0: 0, gen1: 0, gen2: 0
val it : seq<unit> = seq [null; null; null; null; ...]
Note how the execution time depends on the chosen approach.
For the cases where you're getting back a seq<unit> and so want to ignore the result it can be useful to define some extra helper functions, such as:
let sequenceIgnore sequ = sequ |> Async.sequence |> Async.map (ignore)
let pSequenceIgnore sequ = sequ |> pSequence |> Async.map (ignore)
That lets you return a single unit rather than a superfluous sequence of them.
nums is indeed seq of int when I mouse over. Any idea what's going on?
This function line is intended to be the equivalent of C#'s DefaultIfEmpty Linq function.
The general idea is take a space delimited line of strings and write out which ones occur count number of times.
code:
open System
[<EntryPoint>]
let main argv =
let tests = Console.ReadLine() |> int
for i in [0..tests] do
let (length, count) = Console.ReadLine()
|> (fun s -> s.Split [|' '|])
|> (fun split -> Int32.Parse(split.[0]), Int32.Parse(split.[1]))
Console.ReadLine()
|> (fun s -> s.Split [|' '|])
|> Seq.map int
|> Seq.take length
|> Seq.groupBy (fun x -> x)
|> Seq.map (fun (key, group) -> key, Seq.sum group)
|> Seq.where (fun (_, countx) -> countx = count)
|> Seq.map (fun (n, _) -> n)
|> (fun nums -> if Seq.isEmpty nums then "-1" else String.Join(" ", nums))
|> Console.WriteLine
0 // return an integer exit code
Sample input:
3
9 2
4 5 2 5 4 3 1 3 4
So, sequences in F# use lazy evaluation. That means that when you use functions such as map, where, take etc, the results are not evaluated immediately.
The results are only evaluated when the sequence is actually enumerated. When you call Seq.isEmpty you trigger a call to MoveNext() which results in the first element of the result sequence being evaluated - in your case this results in a large chain of functions being evaluated.
In this case, the InvalidOperationException is actually being triggered by Seq.take which throws if the sequence doesn't have sufficient elements. This might surprise you coming from C# where Enumerable.Take will take up to the requested number of elements but could take fewer if you reach the end of the sequence.
If you want this behaviour in F#, you need to replace Seq.take with Seq.truncate.
I wrote this function which merges two lists together but as I'm fairly new to functional programming I was wondering whether there is a better (simpler) way to do it?
let a = ["a"; "b"; "c"]
let b = ["d"; "b"; "a"]
let merge a b =
// take all a and add b
List.fold (fun acc elem ->
let alreadyContains = acc |> List.exists (fun item -> item = elem)
if alreadyContains = true then
acc
else
elem :: acc |> List.rev
) b a
let test = merge a b
Expected result is: ["a"; "b"; "c"; "d"], I'm reverting the list in order to keep the original order. I thought I would be able to achieve the same using List.foldBack (and dropping List.rev) but it results in an error:
Type mismatch. Expecting a
'a
but given a
'a list
The resulting type would be infinite when unifying ''a' and ''a list'
Why is there a difference when using foldBack?
You could use something like the following
let merge a b =
a # b
|> Seq.distinct
|> List.ofSeq
Note that this will preserve order and remove any duplicates.
In F# 4.0 this will be simplified to
let merge a b = a # b |> List.distinct
If I wanted to write this in a way that is similar to your original version (using fold), then the main change I would do is to move List.rev outside of the function (you are calling List.rev every time you add a new element, which is wrong if you're adding even number of elements!)
So, a solution very similar to yours would be:
let merge a b =
(b, a)
||> List.fold (fun acc elem ->
let alreadyContains = acc |> List.exists (fun item -> item = elem)
if alreadyContains = true then acc
else elem :: acc)
|> List.rev
This uses the double-pipe operator ||> to pass two parameters to the fold function (this is not necessary, but I find it a bit nicer) and then passes the result to List.rev.
Hi I am trying to merge a number of observables to an observable array. Here an example that works in fsi. (sorry that it is lengthy)
#r "./bin/Debug/System.Reactive.dll"
open System
open System.Reactive.Linq
/// Subscribes to the Observable with all 3 callbacks.
let subscribeComplete next error completed (observable: IObservable<'T>) =
observable.Subscribe(
(fun x -> next x),
(fun e -> error e),
(fun () -> completed()))
/// Subscribes to the Observable with a next and an error-function.
let subscribeWithError next error observable =
subscribeComplete next error (fun () -> ()) observable
/// Subscribes to the Observable with a next-function
let subscribe (next: 'T -> unit) (observable: IObservable<'T>) : IDisposable =
subscribeWithError next ignore observable
/// Static method to generate observable from input functions
let ObsGenerate (initState: 'TS) (termCond: 'TS -> bool) (iterStep: 'TS -> 'TS)
(resSelect: 'TS -> 'TR) (timeSelect : 'TS -> System.TimeSpan) =
Observable.Generate(initState, termCond, iterStep, resSelect, timeSelect)
//maps the given observable with the given function
let obsMap (f: 'T -> 'U) (observable : IObservable<'T>) : IObservable<'U> =
Observable.Select(observable, Func<_,_>(f))
/// Merges two observable sequences into one observable sequence whenever one of the observable sequences has a new value.
let combineLatest (obs1: IObservable<'T>) (obs2: IObservable<'U>) : IObservable<'T * 'U> =
Observable.CombineLatest(
obs1, obs2, Func<_,_,_>(fun a b -> a, b))
/// Merges three observable sequences into one observable sequence whenever one of the observable sequences has a new value.
let combineLatest3 (obs1: IObservable<'T>) (obs2: IObservable<'U>) (obs3: IObservable<'V>) : IObservable<'T * 'U * 'V> =
let obs12 =obs1.CombineLatest(obs2, Func<_,_,_>(fun a b -> a, b))
obs12.CombineLatest(obs3, Func<_,_,_>(fun (a,b) c -> a, b, c))
/// Merges four observable sequences into one observable sequence whenever one of the observable sequences has a new value.
let combineLatest4 (obs1: IObservable<'T>) (obs2: IObservable<'U>) (obs3: IObservable<'V>) (obs4: IObservable<'W>) : IObservable<'T * 'U * 'V * 'W> =
let obsNew = combineLatest3 obs1 obs2 obs3
obsNew.CombineLatest(obs4, Func<_,_,_>(fun (a,b,c) d -> a, b, c, d))
// second section generating arrays
let combineLatestArray (obs1: IObservable<'T>) (obs2: IObservable<'T>) =
combineLatest obs1 obs2
|> obsMap (fun (a, b) -> [a; b] |> List.toArray)
let combineLatest3Array (obs1: IObservable<'T>) (obs2: IObservable<'T>) (obs3: IObservable<'T>) =
combineLatest3 obs1 obs2 obs3
|> obsMap (fun (a, b, c) -> [a; b; c] |> List.toArray)
let combineLatest4Array (obs1: IObservable<'T>) (obs2: IObservable<'T>) (obs3: IObservable<'T>) (obs4: IObservable<'T>) =
combineLatest4 obs1 obs2 obs3 obs4
|> obsMap (fun (a, b, c, d) -> [a; b; c; d] |> List.toArray)
let combineLatestListToArray (list: IObservable<'T> List) =
match list.Length with
| 2 -> combineLatestArray list.[0] list.[1]
| 3 -> combineLatest3Array list.[0] list.[1] list.[2]
| 4 -> combineLatest4Array list.[0] list.[1] list.[2] list.[3]
| _ -> failwith "combine latest on unsupported list size"
type FooType =
{ NameVal : string
IdVal : int
RetVal : float }
member x.StringKey() =
x.NameVal.ToString() + ";" + x.IdVal.ToString()
// example code starts here
let rnd = System.Random()
let fooListeners = Collections.Generic.Dictionary()
let AddAFoo (foo : FooType) =
let fooId = foo.StringKey()
if fooListeners.ContainsKey(fooId)
then fooListeners.[fooId]
else
let myObs = ObsGenerate {NameVal = foo.NameVal; IdVal = foo.IdVal; RetVal = foo.RetVal} (fun x -> true) (fun x -> {NameVal = (x.NameVal); IdVal = (x.IdVal); RetVal = (x.RetVal + rnd.NextDouble() - 0.5)}) (fun x -> x) (fun x -> System.TimeSpan.FromMilliseconds(rnd.NextDouble() * 2000.0))
fooListeners.Add(fooId,myObs)
myObs
let fooInit = [6..9]
|> List.map (fun index -> {NameVal = (string index + "st"); IdVal = index; RetVal = (float index + 1.0)})
|> List.map (fun foo -> AddAFoo foo)
let fooValuesArray = fooInit
|> List.map(fun x -> (x |> obsMap (fun x -> x.RetVal)))
|> combineLatestListToArray
let mySub =
fooValuesArray
|> subscribe (fun fooVals -> printfn "fooArray: %A" fooVals)
//execute until here to start example
// execute this last line to unsubscribe
mySub.Dispose()
I have two questions about this code:
Is there a smarter way of merging the observables to arrays? (it gets very lengthy as I need to merge larger arrays)
I want to throttle the updates. What I mean by that is that I want all updates that occur within (say) the same half a second window to be handled as one update on the array. Ideally, I want this window to open only when a first update comes in, i.e if no updates arrive in 2 seconds, then one update arrives, then we wait and include further updates for 0.5 seconds and then trigger the observable. I don't want it to publish periodically every 0.5 seconds although no observables are triggered. I hope this description is clear enough.
update: I have decided to accept one of the F# answers, but I haven't done the C# answers justice yet. I hope to be able to check them out properly soon.
For 1, an application of List.fold and List.toArray and a few Observable operators should work nicely. Something like:
let combineLatest observables =
Observable.Select(
(observables
|> List.fold (fun ol o
-> Observable.CombineLatest(o, ol, (fun t tl -> t :: tl))
) (Observable.Return<_>([]))
),
List.toArray)
Due to the nesting, you may end up with performance issues if you have a large list of Observables, but it's worth at least trying before you resort to writing it by hand.
For 2, I would agree with the other answers to apply Throttling to the result.
I'm sorry this isn't F# - I wish I had time to learn it - but here's a possible answer in C#.
Here are a set of extension methods that will combine the latest from an IEnumerable<IObservable<T>> to an IObservable<IEnumerable<T>>:
public static IObservable<IEnumerable<T>> CombineLatest<T>(this IObservable<T> first, IObservable<T> second)
{
if (first == null) throw new ArgumentNullException("first");
if (second == null) throw new ArgumentNullException("second");
return first.CombineLatest(second, (t0, t1) => EnumerableEx.Return(t0).Concat(EnumerableEx.Return(t1)));
}
public static IObservable<IEnumerable<T>> CombineLatest<T>(this IObservable<IEnumerable<T>> firsts, IObservable<T> second)
{
if (firsts == null) throw new ArgumentNullException("firsts");
if (second == null) throw new ArgumentNullException("second");
return firsts.CombineLatest(second, (t0s, t1) => t0s.Concat(EnumerableEx.Return(t1)));
}
public static IObservable<IEnumerable<T>> CombineLatest<T>(this IEnumerable<IObservable<T>> sources)
{
if (sources == null) throw new ArgumentNullException("sources");
return sources.CombineLatest(() => sources.First().CombineLatest(sources.Skip(1)), () => Observable.Empty<IEnumerable<T>>());
}
public static IObservable<IEnumerable<T>> CombineLatest<T>(this IObservable<T> first, IEnumerable<IObservable<T>> seconds)
{
if (first == null) throw new ArgumentNullException("first");
if (seconds == null) throw new ArgumentNullException("seconds");
return seconds.CombineLatest(() => first.CombineLatest(seconds.First()).CombineLatest(seconds.Skip(1)), () => first.Select(t => EnumerableEx.Return(t)));
}
public static IObservable<IEnumerable<T>> CombineLatest<T>(this IObservable<IEnumerable<T>> firsts, IEnumerable<IObservable<T>> seconds)
{
if (firsts == null) throw new ArgumentNullException("firsts");
if (seconds == null) throw new ArgumentNullException("seconds");
return seconds.CombineLatest(() => firsts.CombineLatest(seconds.First()).CombineLatest(seconds.Skip(1)), () => firsts);
}
private static IObservable<IEnumerable<T>> CombineLatest<T>(this IEnumerable<IObservable<T>> sources, Func<IObservable<IEnumerable<T>>> any, Func<IObservable<IEnumerable<T>>> none)
{
if (sources == null) throw new ArgumentNullException("sources");
if (any == null) throw new ArgumentNullException("any");
if (none == null) throw new ArgumentNullException("none");
return Observable.Defer(() => sources.Any() ? any() : none());
}
They may not be very efficient, but they do handle any number of observables that need to be combined.
I'd be keen to see these methods converted to F#.
As for your second question, I'm not sure I can answer with what you've said so far because CombineLatest and Throttle both lose values so it is probably prudent to understand your use case in more detail before attempting an answer.
Although Gideon Engelberth has answered your question with one of the possible way to solve the problem. Other possible way could be something like below, it doesn't use nesting.
let combineLatestToArray (list : IObservable<'T> list) =
let s = new Subject<'T array>()
let arr = Array.init list.Length (fun _ -> Unchecked.defaultof<'T>)
let cb (i:int,v:'T) =
arr.[i] <- v
s.OnNext(arr |> Array.toList |> List.toArray)
let main = list |> List.mapi (fun i o -> o.Select(fun t -> (i,t)))
|> Observable.Merge
main.Subscribe(new Action<int * 'T>(cb)
,new Action<exn>(fun e -> s.OnError(e))
,new Action(fun () -> s.OnCompleted()) ) |> ignore
s :> IObservable<'T array>
Let me know if this solved your problem as I haven't testing it much :)
NOTE: This is for the first part, for second part everyone has already mentioned what you need to do
UPDATE:
Another implementation :
let combineLatestToArray (list : IObservable<'T> list) =
let s = new Subject<'T array>()
let arr = Array.init list.Length (fun _ -> Unchecked.defaultof<'T>)
let main = list |> List.mapi (fun i o -> o.Select(fun t -> (i,t)))
|> Observable.Merge
async {
try
let se = main.ToEnumerable() |> Seq.scan (fun ar (i,t) -> Array.set ar i t; ar) arr
for i in se do
s.OnNext(i |> Array.toList |> List.toArray)
s.OnCompleted()
with
| :? Exception as ex -> s.OnError(ex)
} |> Async.Start
s :> IObservable<'T array>
Seems that Observable.Merge() which has overloads for variable number of IObservables is closer to what you want.
Observable.Buffer() with the time overloads will do what you want here. In the "no events" situation, Buffer will still OnNext() an empty list, letting you react to that stiuation.
This is the best I could come up with. I've been wanting to solve this for a while.
public static class Extensions
{
public static IObservable<IEnumerable<T>> CombineLatest<T>(this Observable observable, IEnumerable<IObservable<T>> observableCollection)
{
return observableCollection.CombineLatest();
}
public static IObservable<IEnumerable<T>> CombineLatest<T>(this IEnumerable<IObservable<T>> observables)
{
return observables.Aggregate<IObservable<T>, IObservable<IEnumerable<T>>>
(
Observable.Return(Enumerable.Empty<T>()),
(o, n) => o.CombineLatest
(
n,
(list, t) => list.Concat(EnumerableEx.Return(t))
)
);
}
}
So an example usage would be:
var obs = new List<IObservable<bool>>
{
Observable.Return(true),
Observable.Return(false),
Observable.Return(true)
};
var result = obs.CombineLatest().Select(list => list.All(x => x));
result.Subscribe(Console.WriteLine);
Console.ReadKey();
You would have to operate on the knowledge, though, that the resulting IObservable<IEnumerable<T>> will not fire until all observables have yielded a value. This is what I needed in my scenarios, but might not be appropriate for your scenario.
My worry with this is the performance of all of the .Concats. Might be more performant to deal in a mutable collection in the extension method. Not sure.
Sorry, I don't know F#. I'll get around to it one of these days.
Throttling is just done with the .Throttle operator after you get your final observable.
Edit: This answer is the iterative Ying to Enigmativity's recursive Yang.