The Array.scan function returns an array of length n+1, where n was the length of its input array, and whose first item is the initial state passed to Array.scan. E.g.,
[|1;2;3;4|] |> Array.scan (+) 0 // Returns [|0;1;3;6;10|]
However, I usually find that that isn't what I wanted: I wanted an array of length n, without the initial state being preserved in my output array. I could easily get this result by simply doing:
input |> Array.scan f initialState |> Array.skip 1
But this will create an intermediate array which is immediately thrown away. Is there a good way to get this result without creating an intermediate array? I could easily implement that myself:
let scanWithoutInitial f initState input =
let result = Array.zeroCreate (Array.length input)
let mutable state = initState
for i = 0 to (Array.length input - 1) do
state <- f state input.[i]
result.[i] <- state
result
However, this seems like a lot of work to reimplement something that I would think would be in the standard F# core library. Is there a function that I'm overlooking? Or am I the only one with this use case, and most people want the initial value as the first item of their Array.scan results?
mapFold does exactly that:
Array.mapFold (fun acc x -> (acc + x, acc + x)) 0 [|1;2;3;4|] |> fst
For more of a brainteaser, ref this post.
[|1;2;3;4|] |> Seq.scan (+) 0 |> Seq.skip 1 |> Seq.toArray
Related
I am trying to create a function that takes two lists that removes values in one list that are also in the other. E.g if we have the lists [1;2;3] and [1;2;3;4] then the first list becomes empty []
and the second list is just [4]. At the end I just when to compare both lists.
I am trying to use List.fold for this since I want to understand it better. Also I created my own folder function that deletes elements from a list.
I am very new to F# so I only came up with a partial solution
let rec delete x list =
match list with
| [] -> []
| hd:: tl when hd = x -> tl
| hd:: tl-> hd:: delete x tl
let myFunc list1 list2 =
let x = list1 |> List.fold(delete) [] list2
let y = list2 |> List.fold(delete) [] list1
x = y
but this does not work and the compiler is telling me "The type '('a -> 'b)' does not support the 'equality' constraint because it is a function type" when I try to use the delete function with the list.fold method.
Although you say you are trying to use List.fold for this to understand it better, there is another List function that makes this simpler. This is to use List.except which is one of a number of methods that treats lists as sets.
let list1 = [1;2;3]
let list2 = [1;2;3;4]
let myFunc list1 list2=
list1 |> List.except list2, list2 |> List.except list1
printfn "%A" (myFunc list1 list2)
[],[4]
If you want to understand List.fold here you could try and create an explicit implementation of except using List.fold. However, again, this is simpler to implement using List.filter.
let list1 = [1;2;3]
let list2 = [1;2;3;4]
let except exclude src =
src |> List.filter (fun i -> exclude |> List.contains i |> not)
let myFuncCustom list1 list2 =
(list1 |> except list2), (list2 |> except list1)
printfn "%A" (myFuncCustom list1 list2)
[],[4]
So really you want to implement filter using List.fold. In this case you would actually need List.foldBack:
let filter f src =
List.foldBack (fun item filtered ->
if f item then item :: filtered else filtered) src []
You can use List.fold but then results are reversed and you need to pipe this into List.rev. And note that List.fold only takes three arguments: the first a folder function; second the accumulator which becomes the output - in this case a list too; and, the last, the source list to fold over. (Let us expand List.contains as well):
let list1 = [1;2;3]
let list2 = [1;2;3;4]
let rec contains item = function
| [] -> false
| hd::tl when hd = item -> true
| hd::tl -> contains item tl
let filter f src =
src
|> List.fold (fun filtered item ->
if f item then item :: filtered else filtered) []
|> List.rev
let except exclude src =
src |> filter (fun i -> exclude |> contains i |> not)
let myFuncCustom list1 list2 =
(list1 |> except list2), (list2 |> except list1)
printfn "%A" (myFuncCustom list1 list2)
[],[4]
This should be what you want:
let difference list blacklist =
let folder acc a =
if List.contains a blacklist
then acc
else a::acc
List.fold folder [] list
difference [1;2;3;4] [1;2;3] // [4]
difference [1;2;3] [1;2;3;4] // []
Looking at the code you posted, there seems to be some confusion on how fold works.
the arguments to fold are
A function that somehow combines a given state with an element of the list. This function can be as simple as summing the two arguments together resulting in a single scalar or it can be something really complicated that creates some weird data structure.
An initial state which must be of the type that you want fold to produce
And, of course, the list you want to fold over
Fold iterates the list, by calling your fold function for every element of the list.
The first time your fold function is called, it will get the initial state. Every other time it will get the state produced from the previous iteration.
Fold will return the last state that was produced by your fold function (or the initial state if the list is empty)
As your goal is to better understand fold I try to explain fold instead of explaining how you achive your goal.
fold is bacially a for loop for immutable data-types. It allows you to eliminate mutable variables. For example,
lets assume you want to sum all values of an integer list. In an "imperative" style you are probaly used to
write something like this.
(* This xs is used through all exampes *)
let xs = [1..10]
(* Example A1 *)
let mutable sum = 0
for x in xs do
sum <- sum + x
(* sum = 55 *)
Before you loop through a list, you define a mutable sum and then mutate the sum and updating it on everey iteration.
This is how you achive it with List.fold.
(* Example A2 *)
let sum =
List.fold (fun sum x ->
sum + x
) 0 xs
(* sum = 55 *)
You can think of List.fold as the following.
The function is the body of the loop that gets executed for every item in your list.
The second argument to List.fold (here 0) is the state you want to compute. This is the sum.
The last argument of List.fold is finally the list you want to traverse.
The function always gets two arguments. The state and the next item of your list. Your function must return
the next state.
With the for-loop you also have state. But the state is outside of the for-loop and you achieve your goal
by mutating the state.
You also can think of the List.fold by mentally mapping the values to the lambda function you provide. The second
argument 0 will be sum in your lambda and x in your lambda is one value of xs. The result of your lambda is
the sum for the next call.
Let's say you want to compute three things on the fly. A mutable version looks like this
(* Helper Function *)
let isEven x = x &&& 1 = 0
(* Example B1 *)
let mutable count = 0
let mutable evens = 0
let mutable sum = 0
for x in xs do
count <- count + 1
if isEven x then
evens <- evens + 1
sum <- sum + x
(* count=10; evens=5; sum=55 *)
Here we compute the amount of values in a list, how many even values exists, and the sum in one go.
List.fold only allows one state, but the state can be a complex value. For example a tuple with three values. The
same example with List.fold looks like this:
(* Example B2 *)
let count,evens,sum =
List.fold (fun (count,evens,sum) x ->
(count+1), (if isEven x then evens + 1 else evens), (sum + x)
) (0,0,0) xs
(* count=10; evens=5; sum=55 *)
To better understand fold it is crucial to understand recursion and immutable data-strucutres like how list works.
You could implement fold yourself like this:
(* Self-defined fold *)
let rec myFold f state xs =
match xs with
| [] -> state
| x::rest -> myFold f (f state x) rest
(* Example C *)
let sum = myFold (fun sum x -> sum + x) 0 xs
(* sum = 55 *)
fold just do two things, it checks if the list is empty and in that case returns the state. Or it removes one element from the top of your list and calls itself recursively by
Keeping the function.
Producing the next state with (f state x)
Use the remaining list rest
Maybe you wonder about performance. This is tail-recursive, and tail-recursive functions are basically turned into for-loops by the compiler. So it has no performance penalty compared to the code that mutate things.
This is at least the case in F#. Just a reminder, not every compiler or run-time for other languages support tail-recursion.
I am trying to filter out values from a sequence, that are not in another sequence. I was pretty sure my code worked, but it is taking a long time to run on my computer and because of this I am not sure, so I am here to see what the community thinks.
Code is below:
let statezip =
StateCsv.GetSample().Rows
|> Seq.map (fun row -> row.State)
|> Seq.distinct
type State = State of string
let unwrapstate (State s) = s
let neededstates (row:StateCsv) = Seq.contains (unwrapstate row.State) statezip
I am filtering by the neededstates function. Is there something wrong with the way I am doing this?
let datafilter =
StateCsv1.GetSample().Rows
|> Seq.map (fun row -> row.State,row.Income,row.Family)
|> Seq.filter neededstates
|> List.ofSeq
I believe that it should filter the sequence by the values that are true, since neededstates function is a bool. StateCsv and StateCsv1 have the same exact structure, although from different years.
Evaluation of contains on sequences and lists can be slow. For a case where you want to check for the existence of an element in a collection, the F# Set type is ideal. You can convert your sequences to sets using Set.ofSeq, and then run the logic over the sets instead. The following example uses the numbers from 1 to 10000 and then uses both sequences and sets to filter the result to only the odd numbers by checking that the values are not in a collection of even numbers.
Using Sequences:
let numberSeq = {0..10000}
let evenNumberSeq = seq { for n in numberSeq do if (n % 2 = 0) then yield n }
#time
numberSeq |> Seq.filter (fun n -> evenNumberSeq |> Seq.contains n |> not) |> Seq.toList
#time
This runs in about 1.9 seconds for me.
Using sets:
let numberSet = numberSeq |> Set.ofSeq
let evenNumberSet = evenNumberSeq |> Set.ofSeq
#time
numberSet |> Set.filter (fun n -> evenNumberSet |> Set.contains n |> not)
#time
This runs in only 0.005 seconds. Hopefully you can materialize your sequences to sets before performing your contains operation, thereby getting this level of speedup.
Consider I have an array of options like [|Some 1;Some0;None;None;Some0|]
and i am going to get the indexes of elements with None value, in this case the correct answer would be [|2;3|].
My current idea is to change the array to a list and then go throw it using recursive function but in this case i will need mutable value to compute index, and i do not want to use mutable?
Is there any other solution
Here's another solution:
[|Some 1;Some 0;None;None;Some 0|]
|> Array.indexed
|> Array.filter (fun (i, x) -> x.IsNone)
|> Array.map fst
As an optimization last 2 lines can be replaced by a single |> Array.choose (function (i, None) -> Some i | _ -> None).
And here's another way using sequence expressions:
let x = [|Some 1;Some 0;None;None;Some 0|]
[|for i = 0 to x.Length-1 do
if x.[i].IsNone then yield i|]
In APL one can use a bit vector to select out elements of another vector; this is called compression. For example 1 0 1/3 5 7 would yield 3 7.
Is there a accepted term for this in functional programming in general and F# in particular?
Here is my F# program:
let list1 = [|"Bob"; "Mary"; "Sue"|]
let list2 = [|1; 0; 1|]
[<EntryPoint>]
let main argv =
0 // return an integer exit code
What I would like to do is compute a new string[] which would be [|"Bob"; Sue"|]
How would one do this in F#?
Array.zip list1 list2 // [|("Bob",1); ("Mary",0); ("Sue",1)|]
|> Array.filter (fun (_,x) -> x = 1) // [|("Bob", 1); ("Sue", 1)|]
|> Array.map fst // [|"Bob"; "Sue"|]
The pipe operator |> does function application syntactically reversed, i.e., x |> f is equivalent to f x. As mentioned in another answer, replace Array with Seq to avoid the construction of intermediate arrays.
I expect you'll find many APL primitives missing from F#. For lists and sequences, many can be constructed by stringing together primitives from the Seq, Array, or List modules, like the above. For reference, here is an overview of the Seq module.
I think the easiest is to use an array sequence expression, something like this:
let compress bits values =
[|
for i = 0 to bits.Length - 1 do
if bits.[i] = 1 then
yield values.[i]
|]
If you only want to use combinators, this is what I would do:
Seq.zip bits values
|> Seq.choose (fun (bit, value) ->
if bit = 1 then Some value else None)
|> Array.ofSeq
I use Seq functions instead of Array in order to avoid building intermediary arrays, but it would be correct too.
One might say this is more idiomatic:
Seq.map2 (fun l1 l2 -> if l2 = 1 then Some(l1) else None) list1 list2
|> Seq.choose id
|> Seq.toArray
EDIT (for the pipe lovers)
(list1, list2)
||> Seq.map2 (fun l1 l2 -> if l2 = 1 then Some(l1) else None)
|> Seq.choose id
|> Seq.toArray
Søren Debois' solution is good but, as he pointed out, but we can do better. Let's define a function, based on Søren's code:
let compressArray vals idx =
Array.zip vals idx
|> Array.filter (fun (_, x) -> x = 1)
|> Array.map fst
compressArray ends up creating a new array in each of the 3 lines. This can take some time, if the input arrays are long (1.4 seconds for 10M values in my quick test).
We can save some time by working on sequences and creating an array at the end only:
let compressSeq vals idx =
Seq.zip vals idx
|> Seq.filter (fun (_, x) -> x = 1)
|> Seq.map fst
This function is generic and will work on arrays, lists, etc. To generate an array as output:
compressSeq sq idx |> Seq.toArray
The latter saves about 40% of computation time (0.8s in my test).
As ildjarn commented, the function argument to filter can be rewritten to snd >> (=) 1, although that causes a slight performance drop (< 10%), probably because of the extra function call that is generated.
I've trying to learn F#. I'm a complete beginner, so this might be a walkover for you guys :)
I have the following function:
let removeEven l =
let n = List.length l;
let list_ = [];
let seq_ = seq { for x in 1..n do if x % 2 <> 0 then yield List.nth l (x-1)}
for x in seq_ do
let list_ = list_ # [x];
list_;
It takes a list, and return a new list containing all the numbers, which is placed at an odd index in the original list, so removeEven [x1;x2;x3] = [x1;x3]
However, I get my already favourite error-message: Incomplete construct at or before this point in expression...
If I add a print to the end of the line, instead of list_:
...
print_any list_;
the problem is fixed. But I do not want to print the list, I want to return it!
What causes this? Why can't I return my list?
To answer your question first, the compiler complains because there is a problem inside the for loop. In F#, let serves to declare values (that are immutable and cannot be changed later in the program). It isn't a statement as in C# - let can be only used as part of another expression. For example:
let n = 10
n + n
Actually means that you want the n symbol to refer to the value 10 in the expression n + n. The problem with your code is that you're using let without any expression (probably because you want to use mutable variables):
for x in seq_ do
let list_ = list_ # [x] // This isn't assignment!
list_
The problematic line is an incomplete expression - using let in this way isn't allowed, because it doesn't contain any expression (the list_ value will not be accessed from any code). You can use mutable variable to correct your code:
let mutable list_ = [] // declared as 'mutable'
let seq_ = seq { for x in 1..n do if x % 2 <> 0 then yield List.nth l (x-1)}
for x in seq_ do
list_ <- list_ # [x] // assignment using '<-'
Now, this should work, but it isn't really functional, because you're using imperative mutation. Moreover, appending elements using # is really inefficient thing to do in functional languages. So, if you want to make your code functional, you'll probably need to use different approach. Both of the other answers show a great approach, although I prefer the example by Joel, because indexing into a list (in the solution by Chaos) also isn't very functional (there is no pointer arithmetic, so it will be also slower).
Probably the most classical functional solution would be to use the List.fold function, which aggregates all elements of the list into a single result, walking from the left to the right:
[1;2;3;4;5]
|> List.fold (fun (flag, res) el ->
if flag then (not flag, el::res) else (not flag, res)) (true, [])
|> snd |> List.rev
Here, the state used during the aggregation is a Boolean flag specifying whether to include the next element (during each step, we flip the flag by returning not flag). The second element is the list aggregated so far (we add element by el::res only when the flag is set. After fold returns, we use snd to get the second element of the tuple (the aggregated list) and reverse it using List.rev, because it was collected in the reversed order (this is more efficient than appending to the end using res#[el]).
Edit: If I understand your requirements correctly, here's a version of your function done functional rather than imperative style, that removes elements with odd indexes.
let removeEven list =
list
|> Seq.mapi (fun i x -> (i, x))
|> Seq.filter (fun (i, x) -> i % 2 = 0)
|> Seq.map snd
|> List.ofSeq
> removeEven ['a'; 'b'; 'c'; 'd'];;
val it : char list = ['a'; 'c']
I think this is what you are looking for.
let removeEven list =
let maxIndex = (List.length list) - 1;
seq { for i in 0..2..maxIndex -> list.[i] }
|> Seq.toList
Tests
val removeEven : 'a list -> 'a list
> removeEven [1;2;3;4;5;6];;
val it : int list = [1; 3; 5]
> removeEven [1;2;3;4;5];;
val it : int list = [1; 3; 5]
> removeEven [1;2;3;4];;
val it : int list = [1; 3]
> removeEven [1;2;3];;
val it : int list = [1; 3]
> removeEven [1;2];;
val it : int list = [1]
> removeEven [1];;
val it : int list = [1]
You can try a pattern-matching approach. I haven't used F# in a while and I can't test things right now, but it would be something like this:
let rec curse sofar ls =
match ls with
| even :: odd :: tl -> curse (even :: sofar) tl
| even :: [] -> curse (even :: sofar) []
| [] -> List.rev sofar
curse [] [ 1; 2; 3; 4; 5 ]
This recursively picks off the even elements. I think. I would probably use Joel Mueller's approach though. I don't remember if there is an index-based filter function, but that would probably be the ideal to use, or to make if it doesn't exist in the libraries.
But in general lists aren't really meant as index-type things. That's what arrays are for. If you consider what kind of algorithm would require a list having its even elements removed, maybe it's possible that in the steps prior to this requirement, the elements can be paired up in tuples, like this:
[ (1,2); (3,4) ]
That would make it trivial to get the even-"indexed" elements out:
thelist |> List.map fst // take first element from each tuple
There's a variety of options if the input list isn't guaranteed to have an even number of elements.
Yet another alternative, which (by my reckoning) is slightly slower than Joel's, but it's shorter :)
let removeEven list =
list
|> Seq.mapi (fun i x -> (i, x))
|> Seq.choose (fun (i,x) -> if i % 2 = 0 then Some(x) else None)
|> List.ofSeq