We know that in F#, seq is lazy evaluated. My question is, if I have a seq with limited number of values, how to convert it into some data type that contains all its value evaluated?
> seq { for i in 1 .. 10 do yield i * i };;
val it : seq<int> = seq [1; 4; 9; 16; ...]
Thanks a lot.
The answer from #Carsten is correct: you can use Seq.toArray or Seq.toList if you wish to convert lazily evaluated sequences to lists or arrays. Don't use these function to force evaluation, though.
The most common reason people tend to ask about this is because they have a projection that involves side effects, and they want to force evaluation. Take this example, where one wishes to print the values to the console:
let lazySeq = seq { for i in 1 .. 10 do yield i * i }
let nothingHappens = lazySeq |> Seq.map (printfn "%i")
The problem is that when you evaluate these two expressions, nothing happens:
>
val lazySeq : seq<int>
val nothingHappens : seq<unit>
Because nothingHappens is a lazily evaluated sequence, no side-effects occur from the map.
People often resort to Seq.toList or Seq.toArray in order to force evaluation:
> nothingHappens |> Seq.toList;;
1
4
9
16
25
36
49
64
81
100
val it : unit list =
[null; null; null; null; null; null; null; null; null; null]
While this works, it's not particularly idiomatic; it produces a weird return type: unit list.
A more idiomatic solution is to use Seq.iter:
> lazySeq |> Seq.iter (printfn "%i");;
1
4
9
16
25
36
49
64
81
100
val it : unit = ()
As you can see, this forces evaluation, but has the more sane return type unit.
use Seq.toArray (for an array) or Seq.toList (for an list) ;)
there are plenty more - just choose ;)
example:
> seq { for i in 1 .. 10 do yield i * i } |> Seq.toArray;;
val it : int [] = [|1; 4; 9; 16; 25; 36; 49; 64; 81; 100|]
Related
I have a solution to this, and several working-but-unsatisfactory solutions, but it took a lot of work and seems unnecessarily complex.
Am I missing something in F#?
The Problem
I have a sequence of numbers
let nums = seq { 9; 12; 4; 17; 9; 7; 13; }
I want to decorate each number with an "index", so the result is
seq [(9, 0); (12, 1); (4, 2); (17, 3); ...]
Looks simple!
In practice the input can be very large and of indeterminate size. In my application, it is coming from a REST service.
Further
the operation must support lazy evaluation (because of the REST backend)
must be purely functional, which eliminates the obvious seq { let mutable i = o; for num in nums do .. } solution, ditto for while ... do ...
Lets call the function decorate, of type (seq<'a> -> seq<'a * int>), so it would work as follows:
nums
|> decorate
|> Seq.iter (fun (n,index) -> printfn "%d: %d" index n)
Producing:
0: 9
1: 12
2: 4
...
6: 13
This is a trivial problem with Lists (apart from the lazy evaluation), but tricky with Sequences.
My solution is to use Seq.unfold, as follows:
let decorate numSeq =
(0,numSeq)
|> Seq.unfold
(fun (count,(nums : int seq)) ->
if Seq.isEmpty nums then
None
else
let result = ((Seq.head nums),count)
let remaining = Seq.tail nums
Some( result, (count+1,remaining)))
This meets all requirements, and is the best I've come up with.
Here's the whole solution, with diagnostics to show lazy evaluation:
let nums =
seq {
// With diagnostic
let getN n =
printfn "get: %d" n
n
getN 9;
getN 12;
getN 4;
getN 17;
getN 9;
getN 7;
getN 13
}
let decorate numSeq =
(0,numSeq)
|> Seq.unfold
(fun (count,(nums : int seq)) ->
if Seq.isEmpty nums then
None
else
let result = ((Seq.head nums),count)
let remaining = Seq.tail nums
printfn "unfold: %A" result
Some( result, (count+1,remaining)))
nums
|> Seq.cache
// To prevent re-computation of the sequence.
// Will be necessary for any solution. This solution required only one.
|> decorate
|> Seq.iter (fun (n,index) -> printfn "ITEM %d: %d" index n)
PROBLEM: This took a LOT of work to reach. It looks complex, compared to the (apparently) simple requirement.
QUESTION: Is there a simpler solution?
Discussion of some alternatives.
All work, but are unsatisfactory for the reasons given
// Most likely: Seq.mapFold
// Fails lazy evalation. The final state must be evaluated, even if not used
let decorate numSeq =
numSeq
|> Seq.mapFold
(fun count num ->
let result = (num,count)
printfn "yield: %A" result
(result,(count + 1)))
0
|> fun (nums,finalState) -> nums // And, no, using "_" doesn't fix it!
// 'for' loop, with MUTABLE
// Lazy evaluation works
// Not extensible, as the state 'count' is specific to this problem
let decorate numSeq =
let mutable count = 0
seq {
for num in numSeq do
let result = num,count
printfn "yield: %A" result
yield result;
count <- count+1
}
// 'for' loop, without mutable
// Fails lazy evaluation, and is ugly
let decorate numSeq =
seq {
for index in 0..((Seq.length numSeq) - 1) do
let result = ((Seq.item index numSeq), // Ugly!
index)
printfn "yield: %A" result
yield result
}
// "List" like recursive descent,
// Fails lazy evaluation. Ugly, because we are not meant to use recursion on Sequences
// https://stackoverflow.com/questions/11451727/recursive-functions-for-sequences-in-f
let rec decorate' count (nums : int seq) =
if Seq.isEmpty nums then
Seq.empty
else
let hd = Seq.head nums
let tl = Seq.tail nums
let result = (hd,count)
let tl' = decorate' (count+1) tl
printfn "yield: %A" result
seq { yield result; yield! tl'}
let decorate : (seq<'a> -> seq<'a * int>) = decorate' 0
You can use Seq.mapi to do what you need.
let nums = seq { 9; 12; 4; 17; 9; 7; 13; }
nums |> Seq.mapi (fun i num -> (num, i))
This gives (9, 0); (12, 1); etc...
Seq is "lazy" in the same sense as IEnumerable in C#.
You can read about Seq.mapi here:
https://fsharp.github.io/fsharp-core-docs/reference/fsharp-collections-seqmodule.html#mapi
Read more about the use of map here:
https://fsharpforfunandprofit.com/posts/elevated-world/#map
In addition to the Seq.mapi function mentioned in Sean's answer, F# also has a built-in Seq.indexed function, which decorates a sequence with index. This does not do exactly what you're asking, because the index becomes the first element of the tuple, but depending on your use case, it may do the trick:
> let nums = seq { 9; 12; 4; 17; 9; 7; 13; };;
val nums : seq<int>
> Seq.indexed nums;;
val it : seq<int * int> = seq [(0, 9); (1, 12); (2, 4); (3, 17); ...]
If I was trying to implement this on my own using a more primitive function, it could be done using Seq.scan, which is a bit like fold but produces a lazy sequence of states. The only tricky thing is that you have to construct the initial state and then process the rest of the sequence:
Seq.tail nums
|> Seq.scan (fun (prevIndex, _) v -> (prevIndex+1, v)) (0, Seq.head nums)
This will not work for empty lists, even though the function should logically be able to handle this.
Using for is not bad, or wrong. for and yield in a seq {} is how you write new seq functions, if none of the provided functions in Seq Module is a best-fit. It is neither wrong, or bad to use this special construct. It's the same as C# foreach and yield syntax.
Using a mutable in a limited scope, is also not wrong. Mutables are a bad idea, if they escape the scope. For example, you return a mutable value, from a function.
Its important to put the mutable inside the seq, and not outside. Your version is wrong.
Let's assume this
let xs = decorate [3;6;7;12;9]
for x in xs do
printfn "%A" x
for x in xs do
printfn "%A" x
Now you have two versions of decorate. The first version
let decorate numSeq =
let mutable count = 0
seq {
for num in numSeq do
yield (num,count)
count <- count + 1
}
will print:
(3, 0)
(6, 1)
(7, 2)
(12, 3)
(9, 4)
(3, 5)
(6, 6)
(7, 7)
(12, 8)
(9, 9)
Or in other words. The mutable is shared across all invocation whenever you iterate through the sequence. As a general tip. If you want to return a seq then put all your code into seq. And put the seq {} after the = sign. If you do this instead.
let decorate numSeq = seq {
let mutable count = 0
for num in numSeq do
yield (num,count)
count <- count + 1
}
you get the correct output:
(3, 0)
(6, 1)
(7, 2)
(12, 3)
(9, 4)
(3, 0)
(6, 1)
(7, 2)
(12, 3)
(9, 4)
Forther you explain, that this version is not "extensible". But the version with mapi you select as "correct". Has the same problem, it only provides an index, nothing more.
If you want a more generic version, you always can make a function that expects its values as a function argument. You could for example change the above function to this code.
let decorate2 f (state:'State) (xs:'T seq) = seq {
let mutable state = state
for x in xs do
yield state, x
let newState = f state x
state <- newState
}
Now decorate2 expects a state that you can freely pass, and a function to change the state. With this function you could then write:
decorate2 (fun state _ -> state+1) 0 [3;6;7;12;9]
The function signature is nearly the same as Seq.scan, but still a little bit different. But if you want to create a indexed function, you could use scan like this.
let indexed xs =
Seq.scan (fun (count,_) x -> (count+1,x)) (0,Seq.head xs) (Seq.skip 1 xs)
Just in my opinion. This version is harder rot read, understand, and just fugly compared to decorate or decorate2.
And just a note. There is already a Seq.indexed function in the standard library, that does what you wish.
for x in Seq.indexed [3;6;7;12;9] do
printfn "%A" x
will print
(0, 3)
(1, 6)
(2, 7)
(3, 12)
(4, 9)
Are the results from F#'s Async.Parallel operation guaranteed to arrive in the order jobs were submitted? My sample code returns the results in order, but I can't find any mention in the MSDN docs, or the F# spec, assuring this must be the case -- that it's not a coincidence.
Here is my sample code:
let r = System.Random()
Async.Parallel [
for i in 0..10 ->
async {
let rand_num = r.Next(10)
do! Async.Sleep(rand_num) (* Simulate jobs taking a variable amount of time *)
printfn "%i %i" i rand_num
return i
}
]
|> Async.RunSynchronously
|> printfn "%A"
And here's the output.
0 0
5 1
4 1
3 3
10 6
9 4
7 5
2 5
1 5
8 7
6 9
[|0; 1; 2; 3; 4; 5; 6; 7; 8; 9; 10|]
You can see that, in this run, the async functions complete in indeterminate order, yet the resulting array is sorted. Is this behavior guaranteed?
At the moment, the source of the function is written so that this guarantee is enforced. Looking at control.fs around line #1300 for the definition, we can see the function that puts the results into the output array is
let recordSuccess i res =
results.[i] <- res;
finishTask(Interlocked.Decrement count)
this function is called in this segment
tasks |> Array.iteri (fun i p ->
queueAsync
innerCTS.Token
// on success, record the result
(fun res -> recordSuccess i res)
where tasks has the original tasks in sorted order. This guarantees that the output list is in the same order as the input.
UPDATE
The spec at least seems to imply that the order is fixed - it contains this code:
let rec fib x = if x < 2 then 1 else fib(x-1) + fib(x-2)
let fibs =
Async.Parallel [ for i in 0..40 -> async { return fib(i) } ]
|> Async.RunSynchronously
printfn "The Fibonacci numbers are %A" fibs //I changed this line to be accurate
System.Console.ReadKey(true)
If the spec didn't guarantee the output order, this code would be incorrect.
I'm trying to learn F# by rewriting some C# algorithms I have into idiomatic F#.
One of the first functions I'm trying to rewrite is a batchesOf where:
[1..17] |> batchesOf 5
Which would split the sequence into batches with a max of five in each, i.e:
[[1; 2; 3; 4; 5]; [6; 7; 8; 9; 10]; [11; 12; 13; 14; 15]; [16; 17]]
My first attempt at doing this is kind of ugly where I've resorted to using a mutable ref object after running into errors trying to use mutable type inside the closure. Using ref is particularly unpleasant since to dereference it you have to use the ! operator which when inside a condition expression can be counter intuitive to some devs who will read it as logical not. Another problem I ran into is where Seq.skip and Seq.take are not like their Linq aliases in that they will throw an error if size exceeds the size of the sequence.
let batchesOf size (sequence: _ seq) : _ list seq =
seq {
let s = ref sequence
while not (!s |> Seq.isEmpty) do
yield !s |> Seq.truncate size |> List.ofSeq
s := System.Linq.Enumerable.Skip(!s, size)
}
Anyway what would be the most elegant/idiomatic way to rewrite this in F#? Keeping the original behaviour but preferably without the ref mutable variable.
Implementing this function using the seq<_> type idiomatically is difficult - the type is inherently mutable, so there is no simple nice functional way. Your version is quite inefficient, because it uses Skip repeatedly on the sequence. A better imperative option would be to use GetEnumerator and just iterate over elements using IEnumerator. You can find various imperative options in this snippet: http://fssnip.net/1o
If you're learning F#, then it is better to try writing the function using F# list type. This way, you can use idiomatic functional style. Then you can write batchesOf using pattern matching with recursion and accumulator argument like this:
let batchesOf size input =
// Inner function that does the actual work.
// 'input' is the remaining part of the list, 'num' is the number of elements
// in a current batch, which is stored in 'batch'. Finally, 'acc' is a list of
// batches (in a reverse order)
let rec loop input num batch acc =
match input with
| [] ->
// We've reached the end - add current batch to the list of all
// batches if it is not empty and return batch (in the right order)
if batch <> [] then (List.rev batch)::acc else acc
|> List.rev
| x::xs when num = size - 1 ->
// We've reached the end of the batch - add the last element
// and add batch to the list of batches.
loop xs 0 [] ((List.rev (x::batch))::acc)
| x::xs ->
// Take one element from the input and add it to the current batch
loop xs (num + 1) (x::batch) acc
loop input 0 [] []
As a footnote, the imperative version can be made a bit nicer using computation expression for working with IEnumerator, but that's not standard and it is quite advanced trick (for example, see http://fssnip.net/37).
A friend asked me this a while back. Here's a recycled answer. This works and is pure:
let batchesOf n =
Seq.mapi (fun i v -> i / n, v) >>
Seq.groupBy fst >>
Seq.map snd >>
Seq.map (Seq.map snd)
Or an impure version:
let batchesOf n =
let i = ref -1
Seq.groupBy (fun _ -> i := !i + 1; !i / n) >> Seq.map snd
These produce a seq<seq<'a>>. If you really must have an 'a list list as in your sample then just add ... |> Seq.map (List.ofSeq) |> List.ofSeq as in:
> [1..17] |> batchesOf 5 |> Seq.map (List.ofSeq) |> List.ofSeq;;
val it : int list list = [[1; 2; 3; 4; 5]; [6; 7; 8; 9; 10]; [11; 12; 13; 14; 15]; [16; 17]]
Hope that helps!
This can be done without recursion if you want
[0..20]
|> Seq.mapi (fun i elem -> (i/size),elem)
|> Seq.groupBy (fun (a,_) -> a)
|> Seq.map (fun (_,se) -> se |> Seq.map (snd));;
val it : seq<seq<int>> =
seq
[seq [0; 1; 2; 3; ...]; seq [5; 6; 7; 8; ...]; seq [10; 11; 12; 13; ...];
seq [15; 16; 17; 18; ...]; ...]
Depending on how you think this may be easier to understand. Tomas' solution is probably more idiomatic F# though
Hurray, we can use List.chunkBySize, Seq.chunkBySize and Array.chunkBySize in F# 4, as mentioned by Brad Collins and Scott Wlaschin.
This isn't perhaps idiomatic but it works:
let batchesOf n l =
let _, _, temp', res' = List.fold (fun (i, n, temp, res) hd ->
if i < n then
(i + 1, n, hd :: temp, res)
else
(1, i, [hd], (List.rev temp) :: res))
(0, n, [], []) l
(List.rev temp') :: res' |> List.rev
Here's a simple implementation for sequences:
let chunks size (items:seq<_>) =
use e = items.GetEnumerator()
let rec loop i acc =
seq {
if i = size then
yield (List.rev acc)
yield! loop 0 []
elif e.MoveNext() then
yield! loop (i+1) (e.Current::acc)
else
yield (List.rev acc)
}
if size = 0 then invalidArg "size" "must be greater than zero"
if Seq.isEmpty items then Seq.empty else loop 0 []
let s = Seq.init 10 id
chunks 3 s
//output: seq [[0; 1; 2]; [3; 4; 5]; [6; 7; 8]; [9]]
My method involves converting the list to an array and recursively chunking the array:
let batchesOf (sz:int) lt =
let arr = List.toArray lt
let rec bite curr =
if (curr + sz - 1 ) >= arr.Length then
[Array.toList arr.[ curr .. (arr.Length - 1)]]
else
let curr1 = curr + sz
(Array.toList (arr.[curr .. (curr + sz - 1)])) :: (bite curr1)
bite 0
batchesOf 5 [1 .. 17]
[[1; 2; 3; 4; 5]; [6; 7; 8; 9; 10]; [11; 12; 13; 14; 15]; [16; 17]]
I found this to be a quite terse solution:
let partition n (stream:seq<_>) = seq {
let enum = stream.GetEnumerator()
let rec collect n partition =
if n = 1 || not (enum.MoveNext()) then
partition
else
collect (n-1) (partition # [enum.Current])
while enum.MoveNext() do
yield collect n [enum.Current]
}
It works on a sequence and produces a sequence. The output sequence consists of lists of n elements from the input sequence.
You can solve your task with analog of Clojure partition library function below:
let partition n step coll =
let rec split ss =
seq {
yield(ss |> Seq.truncate n)
if Seq.length(ss |> Seq.truncate (step+1)) > step then
yield! split <| (ss |> Seq.skip step)
}
split coll
Being used as partition 5 5 it will provide you with sought batchesOf 5 functionality:
[1..17] |> partition 5 5;;
val it : seq<seq<int>> =
seq
[seq [1; 2; 3; 4; ...]; seq [6; 7; 8; 9; ...]; seq [11; 12; 13; 14; ...];
seq [16; 17]]
As a premium by playing with n and step you can use it for slicing overlapping batches aka sliding windows, and even apply to infinite sequences, like below:
Seq.initInfinite(fun x -> x) |> partition 4 1;;
val it : seq<seq<int>> =
seq
[seq [0; 1; 2; 3]; seq [1; 2; 3; 4]; seq [2; 3; 4; 5]; seq [3; 4; 5; 6];
...]
Consider it as a prototype only as it does many redundant evaluations on the source sequence and not likely fit for production purposes.
This version passes all my tests I could think of including ones for lazy evaluation and single sequence evaluation:
let batchIn batchLength sequence =
let padding = seq { for i in 1 .. batchLength -> None }
let wrapped = sequence |> Seq.map Some
Seq.concat [wrapped; padding]
|> Seq.windowed batchLength
|> Seq.mapi (fun i el -> (i, el))
|> Seq.filter (fun t -> fst t % batchLength = 0)
|> Seq.map snd
|> Seq.map (Seq.choose id)
|> Seq.filter (fun el -> not (Seq.isEmpty el))
I am still quite new to F# so if I'm missing anything - please do correct me, it will be greatly appreciated.
I need to create an infinite sequence containing a subsequence of elements which repeats infinitely.
[1; 2; 3; 4; 1; 2; 3; 4; 1; 2; 3; 4; ...]
So I wrote this:
let l = [1; 2; 3; 4]
let s = seq { while true do yield! l }
Is there a standard way (function) to do this?
I think that your approach is good in this scenario. There is no built-in function to implement repetition, but if you need to repeat sequences often, you can define one yourself and make it available in the Seq module:
module Seq =
let repeat items =
seq { while true do yield! items }
Then you can nicely write Seq.repeat [ 1 .. 4 ], as if repeat was a standard F# library function, because F# IntelliSense shows both functions from your Seq module and from the Seq module as if they were defined in a single module.
Aside from your implementation, you can also use recursive sequence expression, which is another quite common pattern when generating sequences. Using while is in some ways imperative (although you don't need any state for simple repetitions) compared to functional recursion:
let rec repeat items =
seq { yield! items
yield! repeat items }
This approach is better when you want to keep some state while generating. For example, generating all numbers 1 .. using while would not be so nice, because you'd need mutable state. Using recursion, you can write the same thing as:
let rec numbersFrom n =
seq { yield n
yield! numbersFrom (n + 1) }
I don't think there's an idiom for this, and what you have is fine, but here are some alternatives.
If you change your subsequence to an array, you can do
let a = [|1; 2; 3; 4|]
let s = Seq.initInfinite (fun i -> a.[i % a.Length])
Using what you have, you could also do
let l = [1; 2; 3; 4]
let s = Seq.initInfinite (fun _ -> l) |> Seq.concat
but it's no shorter.
Similar to Daniel's answer, but encapsulating it into a function, and pretending that function is in the Seq module:
module Seq =
let infiniteOf repeatedList =
Seq.initInfinite (fun _ -> repeatedList)
|> Seq.concat
// Tests
let intList = [1; 2; 3; 4]
let charList = ['a'; 'b'; 'c'; 'd']
let objList = [(new System.Object()); (new System.Object()); (new System.Object()); (new System.Object())]
do
Seq.infiniteOf intList |> Seq.take 20 |> Seq.iter (fun item -> printfn "%A" item)
Seq.infiniteOf charList |> Seq.take 20 |> Seq.iter (fun item -> printfn "%A" item)
Seq.infiniteOf objList |> Seq.take 20 |> Seq.iter (fun item -> printfn "%A" item)
This will do it as a (more-or-less) one-liner, without having to create any helper objects.
let s = seq { while true do
for i in 1 .. 4 -> i }
a simple question I cannot find an answer to: how to add an element to a sequence? Eg I have a seq and a newElem XElement I'd like to append to it.
Thanks
Seq.append:
> let x = { 1 .. 5 };;
val x : seq<int>
> let y = Seq.append x [9];; // [9] is a single-element list literal
val y : seq<int>
> y |> Seq.toList;;
val it : int list = [1; 2; 3; 4; 5; 9]
You can also use
let newSeq = Seq.append oldSeq (Seq.singleton newElem)
Which is a slight modification of the first answer but appends sequences instead of a list to a sequence.
given the following code
let startSeq = seq {1..100}
let AppendTest = Seq.append startSeq [101] |> List.ofSeq
let AppendTest2 = Seq.append startSeq (Seq.singleton 101) |> List.ofSeq
let AppendTest3 = seq { yield! startSeq; yield 101 } |> List.ofSeq
looped 10000 executions the run times are
Elapsed 00:00:00.0001399
Elapsed 00:00:00.0000942
Elapsed 00:00:00.0000821
Take from that what you will.
There's also an imperative solution...
> let x = seq {1..5}
> let y = seq { yield! x; yield 9 } // Flatten the list,then append your element
> Seq.to_list y;;
val it : int list = [1; 2; 3; 4; 5; 9]
This may be better if the underlying problem is an imperative one, and it is most natural to use a yield statement in a loop.
let mySeq = seq { for i in 1..10 do yield i };;