with an array like this:
let a = [| 1.; 2.; nan; 4.; 5. |]
I know that the last value is not a NaN (the list comes from an operation that generates leading NaNs) I want to scan the list from the end and if the value is a nan, replace it with the previous value.
In the case, the output would be:
[| 1.; 2.; 4.; 4.; 5. |]
first, I made this monstrocity:
Array.append a [|a.[^0]|] |> Array.pairwise |> Array.rev |> Array.map (fun (a, b) -> if Double.IsNaN(a) then b else a) |> Array.rev
then a cleaner loop:
seq {
for i = a.Length - 1 downto 0 do
yield if Double.IsNaN(a.[i]) then a.[i+1] else a.[i]
} |> Seq.rev |> Seq.toArray
I was thinking that maybe foldBack would work:
Array.foldBack (fun x acc -> Array.append acc (seq {yield if Double.IsNaN(x) then acc.[^0] else x} |> Seq.toArray)) a Array.empty
but this is ugly as well..
I can't help to think that there has to be a simple elegant way to do this... anyone has an idea?
First of all, you can make your foldBack solution a lot cleaner by realizing that seq { yield foo } |> Seq.toArray is the same as just [|foo|]:
Array.foldBack (fun x acc -> Array.append acc [|if Double.IsNaN(x) then acc.[^0] else x|]) a Array.empty
But a better approach is scan. It's a nice little function that's kind of like map, but each iteration also has access to the result of the previous one. Except, of course, since you're going back-to-front, you'll have to use scanBack:
Array.scanBack (fun x prev -> if Double.IsNaN x then prev else x) a nan
The only problem is that the "seed" of the scan (which is the last parameter - in my example nan) will remain in the resulting array, becoming its last element. So you'd have to filter it out afterwards, but I didn't think it would be a problem since you seemed to be ok reallocating multiple arrays in your foldBack approach:
Array.scanBack (fun x prev -> if Double.IsNaN x then prev else x) a nan
|> Array.takeWhile (not << Double.IsNaN)
Alternatively, you can remember that arrays are mutable, and simply mutate your original array:
for i in 0..(Array.length a - 1) do
if Double.IsNaN a.[i] then a.[i] <- a.[i+1]
You can do this with a single Array.mapi. This is like Array.map but it provides the index of each item.
let fillNans xs =
xs
|> Array.mapi (fun i x -> if Double.IsNaN x then xs.[i+1] else x)
Related
I have a map reduce code for which I group in each of the threads by some key and then in the reduce part merge the results. My current approach is to search for an specific key index in the accumulator and then mapi to retrieve the combined result only for this key, leaving the rest unmodified:
let rec groupFolder sequence acc =
match sequence with
| (by:string, what) :: rest ->
let index = acc |> Seq.tryFindIndex( fun (byInAcc, _) -> byInAcc.Equals(by) )
match index with
| Some (idx) ->
acc |> Seq.mapi( fun i (byInAcc, whatInAcc) -> if i = idx then (by, (what |> Array.append whatInAcc) ) else byInAcc, whatInAcc )
|> groupFolder rest
| None -> acc |> Seq.append( seq{ yield (by, what) } )
|> groupFolder rest
My question is, is it a more functional way to achieve just this?
As an example input to this reducer
let GroupsCommingFromMap = [| seq { yield! [|("key1", [|1;2;3|] ); ("key2", [|1;2;3|] ); ("key3", [|1;2;3|]) |] }, seq { yield! [|("key1", [|4;5;6|] ); ("key2", [|4;5;6|] ); ("key3", [|4;5;6|]) |] } |];;
GroupsCommingFromMap |> Seq.reduce( fun acc i ->
acc |> groupFolder (i |> Seq.toList))
the expected result should contain all key1..key3 each with the array 1..6
From the code you posted, it is not very clear what you're trying to do. Could you include some sample inputs (together with the output that you would like to get)? And does your code actually work on any of the inputs (it has incomplete pattern match, so I doubt that...)
Anyway, you can implement key-based map reduce using Seq.groupBy. For example:
let mapReduce mapper reducer input =
input
|> Seq.map mapper
|> Seq.groupBy fst
|> Seq.map (fun (k, vs) ->
k, vs |> Seq.map snd |> Seq.reduce reducer)
Here:
The mapper takes a value from the input sequence and turns it into key value pair. The mapReduce function then groups the values using the key
The reducer is then used to reduce all values associated with each key
This lets you create a word count function like this (using simple mapper that returns the word as the key with 1 as a value and reducer that just adds all the numbers):
"hello world hello people hello world".Split(' ')
|> mapReduce (fun w -> w, 1) (+)
EDIT: The example you mentioned does not really have "mapper" part, but instead it has array of arrays as an input - so perhaps it is easier to write this directly using Seq.groupBy like this:
let GroupsCommingFromMap =
[| [|("key1", [|1;2;3|] ); ("key2", [|1;2;3|] ); ("key3", [|1;2;3|]) |]
[|("key1", [|4;5;6|] ); ("key2", [|4;5;6|] ); ("key3", [|4;5;6|]) |] |]
GroupsCommingFromMap
|> Seq.concat
|> Seq.groupBy fst
|> Seq.map (fun (k, vs) -> k, vs |> Seq.map snd |> Array.concat)
In APL one can use a bit vector to select out elements of another vector; this is called compression. For example 1 0 1/3 5 7 would yield 3 7.
Is there a accepted term for this in functional programming in general and F# in particular?
Here is my F# program:
let list1 = [|"Bob"; "Mary"; "Sue"|]
let list2 = [|1; 0; 1|]
[<EntryPoint>]
let main argv =
0 // return an integer exit code
What I would like to do is compute a new string[] which would be [|"Bob"; Sue"|]
How would one do this in F#?
Array.zip list1 list2 // [|("Bob",1); ("Mary",0); ("Sue",1)|]
|> Array.filter (fun (_,x) -> x = 1) // [|("Bob", 1); ("Sue", 1)|]
|> Array.map fst // [|"Bob"; "Sue"|]
The pipe operator |> does function application syntactically reversed, i.e., x |> f is equivalent to f x. As mentioned in another answer, replace Array with Seq to avoid the construction of intermediate arrays.
I expect you'll find many APL primitives missing from F#. For lists and sequences, many can be constructed by stringing together primitives from the Seq, Array, or List modules, like the above. For reference, here is an overview of the Seq module.
I think the easiest is to use an array sequence expression, something like this:
let compress bits values =
[|
for i = 0 to bits.Length - 1 do
if bits.[i] = 1 then
yield values.[i]
|]
If you only want to use combinators, this is what I would do:
Seq.zip bits values
|> Seq.choose (fun (bit, value) ->
if bit = 1 then Some value else None)
|> Array.ofSeq
I use Seq functions instead of Array in order to avoid building intermediary arrays, but it would be correct too.
One might say this is more idiomatic:
Seq.map2 (fun l1 l2 -> if l2 = 1 then Some(l1) else None) list1 list2
|> Seq.choose id
|> Seq.toArray
EDIT (for the pipe lovers)
(list1, list2)
||> Seq.map2 (fun l1 l2 -> if l2 = 1 then Some(l1) else None)
|> Seq.choose id
|> Seq.toArray
Søren Debois' solution is good but, as he pointed out, but we can do better. Let's define a function, based on Søren's code:
let compressArray vals idx =
Array.zip vals idx
|> Array.filter (fun (_, x) -> x = 1)
|> Array.map fst
compressArray ends up creating a new array in each of the 3 lines. This can take some time, if the input arrays are long (1.4 seconds for 10M values in my quick test).
We can save some time by working on sequences and creating an array at the end only:
let compressSeq vals idx =
Seq.zip vals idx
|> Seq.filter (fun (_, x) -> x = 1)
|> Seq.map fst
This function is generic and will work on arrays, lists, etc. To generate an array as output:
compressSeq sq idx |> Seq.toArray
The latter saves about 40% of computation time (0.8s in my test).
As ildjarn commented, the function argument to filter can be rewritten to snd >> (=) 1, although that causes a slight performance drop (< 10%), probably because of the extra function call that is generated.
I have a list like [1..12] and I would like to get a piece like [4..9]. No clue how I can do that, I'm new with F#. I don't know if there is a built-in method for that, but I would like to know the manual way.
[1..12] |> List.filter (fun x -> x >= 4 && x <= 9)
or
[1..12] |> Seq.skip 3 |> Seq.take 6 |> Seq.toList
Lists don't support slicing, but if you use an array instead you can also do this:
[|1..12|].[3..8]
(note 3..8 instead of 4..9 because of 0-based indexing)
To give an immediate answer to your question: how do you take a piece of a list? Pattern matching.
You can use pattern matching to write a function that extracts a range from a list. The basic algorithm is skip each element of the list while E < Min, then take each element while E <= Max. Something like this:
let range min max xs =
let rec skipWhile f = function
| x::xs when f x -> skipWhile f xs
| xs -> xs
let rec takeWhile f acc = function
| x::xs when f x -> takeWhile f (x::acc) xs
| _ -> List.rev acc
xs
|> skipWhile ((>) min)
|> takeWhile ((>=) max) []
[1..12] |> range 4 9
> val it : int list = [4; 5; 6; 7; 8; 9]
Assuming you use the Ocaml-like subset of F# you probably want to use the List standard module, probably its filter function.
Otherwise, a tail recursive function with matching could do.
Hullo all.
I am a C# programmer, exploring F# in my free time. I have written the following little program for image convolution in 2D.
open System
let convolve y x =
y |> List.map (fun ye -> x |> List.map ((*) ye))
|> List.mapi (fun i l -> [for q in 1..i -> 0] # l # [for q in 1..(l.Length - i - 1) -> 0])
|> List.reduce (fun r c -> List.zip r c |> List.map (fun (a, b) -> a + b))
let y = [2; 3; 1; 4]
let x = [4; 1; 2; 3]
printfn "%A" (convolve y x)
My question is: Is the above code an idiomatic F#? Can it be made more concise? (e.g. Is there some shorter way to generate a filled list of 0's (I have used list comprehension in my code for this purpose)). Any changes that can improve its performance?
Any help would be greatly appreciated. Thanks.
EDIT:
Thanks Brian. I didn't get your first suggestion. Here's how my code looks after applying your second suggestion. (I also abstracted out the list-fill operation.)
open System
let listFill howMany withWhat = [for i in 1..howMany -> withWhat]
let convolve y x =
y |> List.map (fun ye -> x |> List.map ((*) ye))
|> List.mapi (fun i l -> (listFill i 0) # l # (listFill (l.Length - i - 1) 0))
|> List.reduce (List.map2 (+))
let y = [2; 3; 1; 4]
let x = [4; 1; 2; 3]
printfn "%A" (convolve y x)
Anything else can be improved? Awaiting more suggestions...
As Brian mentioned, the use of # is generally problematic, because the operator cannot be efficiently implemented for (simple) functional lists - it needs to copy the entire first list.
I think Brians suggestion was to write a sequence generator that would generate the list at once, but that's a bit more complicated. You'd have to convert the list to array and then write something like:
let convolve y x =
y |> List.map (fun ye -> x |> List.map ((*) ye) |> Array.ofList)
|> List.mapi (fun i l -> Array.init (2 * l.Length - 1) (fun n ->
if n < i || n - i >= l.Length then 0 else l.[n - i]))
|> List.reduce (Array.map2 (+))
In general, if performance is an important concern, then you'll probably need to use arrays anyway (because this kind of problem can be best solved by accessing elements by index). Using arrays is a bit more difficult (you need to get the indexing right), but perfectly fine approach in F#.
Anyway, if you want to write this using lists, then here ara some options. You could use sequence expressions everywhere, which would look like this:
let convolve y (x:_ list) =
[ for i, v1 in x |> List.zip [ 0 .. x.Length - 1] ->
[ yield! listFill i 0
for v2 in y do yield v1 * v2
yield! listFill (x.Length - i - 1) 0 ] ]
|> List.reduce (List.map2 (+))
... or you can also combine the two options and use a nested sequence expression (with yield! to generate zeros and lists) in the lambda function that you're passing to List.mapi:
let convolve y x =
y |> List.map (fun ye -> x |> List.map ((*) ye))
|> List.mapi (fun i l ->
[ for _ in 1 .. i do yield 0
yield! l
for _ in 1 .. (l.Length - i - 1) do yield 0 ])
|> List.reduce (List.map2 (+))
The idiomatic solution would be to use arrays and loops just as you would in C. However, you may be interested in the following alternative solution that uses pattern matching instead:
let dot xs ys =
Seq.map2 (*) xs ys
|> Seq.sum
let convolve xs ys =
let rec loop vs xs ys zs =
match xs, ys with
| x::xs, ys -> loop (dot ys (x::zs) :: vs) xs ys (x::zs)
| [], _::(_::_ as ys) -> loop (dot ys zs :: vs) [] ys zs
| _ -> List.rev vs
loop [] xs ys []
convolve [2; 3; 1; 4] [4; 1; 2; 3]
Regarding the zeroes, how about e.g.
[for q in 0..l.Length-1 -> if q=i then l else 0]
(I haven't tested to verify that is exactly right, but hopefully the idea is clear.) In general, any use of # is a code smell.
Regarding overall performance, for small lists this is probably fine; for larger ones, you might consider using Seq rather than List for some of the intermediate computations, to avoid allocating as many temporary lists along the way.
It looks like maybe the final zip-then-map could be replaced by just a call to map2, something like
... fun r c -> (r,c) ||> List.map2 (+)
or possibly even just
... List.map2 (+)
but I'm away from a compiler so haven't double-checked it.
(fun ye -> x |> List.map ((*) ye))
Really ?
I'll admit |> is pretty, but you could just wrote :
(fun ye -> List.map ((*) ye) x)
Another thing that you could do is fuse the first two maps. l |> List.map f |> List.mapi g = l |> List.mapi (fun i x -> g i (f x)), so incorporating Tomas and Brian's suggestions, you can get something like:
let convolve y x =
let N = List.length x
y
|> List.mapi (fun i ye ->
[for _ in 1..i -> 0
yield! List.map ((*) ye) x
for _ in 1..(N-i-1) -> 0])
|> List.reduce (List.map2 (+))
I am trying to think of an elegant way of getting a random subset from a set in F#
Any thoughts on this?
Perhaps this would work: say we have a set of 2x elements and we need to pick a subset of y elements. Then if we could generate an x sized bit random number that contains exactly y 2n powers we effectively have a random mask with y holes in it. We could keep generating new random numbers until we get the first one satisfying this constraint but is there a better way?
If you don't want to convert to an array you could do something like this. This is O(n*m) where m is the size of the set.
open System
let rnd = Random(0);
let set = Array.init 10 (fun i -> i) |> Set.of_array
let randomSubSet n set =
seq {
let i = set |> Set.to_seq |> Seq.nth (rnd.Next(set.Count))
yield i
yield! set |> Set.remove i
}
|> Seq.take n
|> Set.of_seq
let result = set |> randomSubSet 3
for x in result do
printfn "%A" x
Agree with #JohannesRossel. There's an F# shuffle-an-array algorithm here you can modify suitably. Convert the Set into an array, and then loop until you've selected enough random elements for the new subset.
Not having a really good grasp of F# and what might be considered elegant there, you could just do a shuffle on the list of elements and select the first y. A Fisher-Yates shuffle even helps you in this respect as you also only need to shuffle y elements.
rnd must be out of subset function.
let rnd = new Random()
let rec subset xs =
let removeAt n xs = ( Seq.nth (n-1) xs, Seq.append (Seq.take (n-1) xs) (Seq.skip n xs) )
match xs with
| [] -> []
| _ -> let (rem, left) = removeAt (rnd.Next( List.length xs ) + 1) xs
let next = subset (List.of_seq left)
if rnd.Next(2) = 0 then rem :: next else next
Do you mean a random subset of any size?
For the case of a random subset of a specific size, there's a very elegant answer here:
Select N random elements from a List<T> in C#
Here it is in pseudocode:
RandomKSubset(list, k):
n = len(list)
needed = k
result = {}
for i = 0 to n:
if rand() < needed / (n-i)
push(list[i], result)
needed--
return result
Using Seq.fold to construct using lazy evaluation random sub-set:
let rnd = new Random()
let subset2 xs = let insertAt n xs x = Seq.concat [Seq.take n xs; seq [x]; Seq.skip n xs]
let randomInsert xs = insertAt (rnd.Next( (Seq.length xs) + 1 )) xs
xs |> Seq.fold randomInsert Seq.empty |> Seq.take (rnd.Next( Seq.length xs ) + 1)