Is it possible to set 'MaxDegreeOfParallelism' (that is maximum number of threads to use) for Array.Parallel module since under the hood it uses Parallel.For?
According to this post, it seems that there is no way to limit the number of threads globally in the final version of Parallel Extensions. An alternative to what brian suggests would be to use PLINQ (which works with parallel sequences) instead of functions that work with arrays.
This can be done using the PSeq module from F# PowerPack. It provides functions such as PSeq.map, PSeq.filter and many other that work with parallel sequences (which can be also nicely composed using pipelining). For parallel sequences, you can use the WithDegreeOfParallelism extension method to specify the behavior.
You could implement a wrapper function for it:
[EDIT: It is already there!]
let withDegreeOfParallelism n (pq:ParallelQuery<_>) =
pq.WithDegreeOfParallelsm(n)
And then write:
let res =
data |> PSeq.map (fun n -> ...)
|> PSeq.withDegreeOfParallelism ParallelOptions.MaxDegreeOfParallelism
|> Array.ofSeq
This may have different perfromance, because it is implemented differently than functions in the Array.Parallel module, but this certainly depends on your scenario.
No, I don't think so.
You can always create your own versions of any of the methods in the Array.Parallel module, using the source code from array.fs (in the CTP release) as a starter.
Assuming I want say at most 10 threads I've been replacing:
myArray
|> Array.Parallel.iter (fun item -> doWork item)
with
let maxPara = 10
myArray
|> Array.splitInto maxPara
|> Array.Parallel.iter (fun items -> items |> List.iter (fun item -> doWork item))
Related
I have a recursive function in f# that iterates a string[] of commands that need to be run, each command runs a new command to generate a map to be passed to the next function.
The commands run correctly but are large and cumbersome to read, I believe that there is a better way to order / format these composite functions using pipe syntax however coming from c# as a lot of us do i for the life of me cannot seem to get it to work.
my command is :
let rec iterateCommands (map:Map<int,string array>) commandPosition =
if commandPosition < commands.Length then
match splitCommand(commands.[0]).[0] with
|"comOne" ->
iterateCommands (map.Add(commandPosition,create(splitCommand commands.[commandPosition])))(commandPosition+1)
The closest i have managed is by indenting the function but this is messy :
iterateCommands
(map.Add
(commandPosition,create
(splitCommand commands.[commandPosition])
)
)
(commandPosition+1)
Is it even possible to reformat this in f#? From what i have read i believe it possible, any help would be greatly appreciated
The command/variable types are:
commandPosition - int
commands - string[]
splitCommand string -> string[]
create string[] -> string[]
map : Map<int,string[]>
and of course the map.add map -> map + x
It's often hard to make out what is going on in a big statement with multiple inputs. I'd give names to the individual expressions, so that a reader can jump into any position and have a rough idea what's in the values used in a calculation, e.g.
let inCommands = splitCommand commands.[commandPosition]
let map' = map.Add (commandPosition, inCommands)
iterateCommands map' inCommands
Since I don't know what is being done here, the names aren't very meaningful. Ideally, they'd help to understand the individual steps of the calculation.
It'd be a bit easier to compose the call if you changed the arguments around:
let rec iterateCommands commandPosition (map:Map<int,string array>) =
// ...
That would enable you to write something like:
splitCommand commands.[commandPosition]
|> create
|> (fun x -> commandPosition, x)
|> map.Add
|> iterateCommands (commandPosition + 1)
The fact that commandPosition appears thrice in the composition is, in my opinion, a design smell, as is the fact that the type of this entire expression is unit. It doesn't look particularly functional, but since I don't understand exactly what this function attempts to do, I can't suggest a better design.
If you don't control iterateCommands, and hence can't change the order of arguments, you can always define a standard functional programming utility function:
let flip f x y = f y x
This enables you to write the following against the original version of iterateCommands:
splitCommand commands.[commandPosition]
|> create
|> (fun x -> commandPosition, x)
|> map.Add
|> (flip iterateCommands) (commandPosition + 1)
I need a function like Seq.head, but returning None instead of throwing an exception when the sequence is empty, i.e., seq<'T> -> 'T option.
There are a jillion ways to do this. Here are several:
let items = Seq.init 10 id
let a = Seq.tryFind (fun _ -> true) items
let b = Seq.tryPick Some items
let c = if Seq.isEmpty items then None else Some (Seq.head items)
let d =
use e = items.GetEnumerator()
if e.MoveNext() then Some e.Current
else None
b is the one I use. Two questions:
Is there a particularly idiomatic way to do this?
Since there's no built-in Seq.tryHead function, does that indicate this shouldn't be necessary, is uncommon, or is better implemented without a function?
UPDATE
tryHead has been added to the standard library in F# 4.0.
I think (b) is probably the most idiomatic, for the same reason #Ramon gave.
I think the lack of Seq.tryHead just means that it is not super common.
I'm not sure, but my guess is that functional languages with Hindley-Milner type inference in general are sparse about implementing such specific functions on collection types because overloading isn't available and composing higher-order functions can be done tersely.
For example, C# Linq extensions are much more exhaustive than functions in F#'s Seq module (which itself is more exhaustive than functions on concrete collection types), and even has IEnumerable.FirstOrDefault. Practically every overload has a variation which performs a map.
I think emphasis on pattern matching and concrete types like list is also a reason.
Now, most of the above is speculation, but I think I may have a notion closer to being objective. I think a lot of the time tryPick and tryFind can be used in the first place instead of filter |> tryHead. For example, I find myself writing code like the following fairly frequently:
open System.Reflection
let ty = typeof<System.String> //suppose this type is actually unknown at compile time
seq {
for name in ["a";"b";"c"] do
yield ty.GetMethod(name)
} |> Seq.tryFind((<>)null)
instead of like
...
seq {
for name in ["a";"b";"c"] do
match ty.GetMethod(name) with
| null -> ()
| mi -> yield mi
} |> tryHead
You could define:
let seqTryHead s = Seq.tryPick Some s
It is of type seq<'a> -> 'a option. Note that I don't beta-reduce because of the generic value limitation.
I can do
for event in linq.Deltas do
or I can do
linq.Deltas |> Seq.iter(fun event ->
So I'm not sure if that is the same. If that is not the same I want to know the difference. I don't know what to use: iter or for.
added - so if that is the matter of choice I prefer to use iter on a top level and for is for closures
added some later - looking like iter is map + ignore - it's the way to run from using imperative ignore word. So it's looking like functional way ...
As others mentioned, there are some differences (iter supports non-generic IEnumerator and you can mutate mutable values in for). These are sometimes important differences, but most of the times you can freely choose which one to use.
I generally prefer for (if there is a language construct, why not use it?). The cases where iter looks nicer are when you have a function that you need to call (e.g. using partial application):
// I would write this:
strings |> Seq.iter (printfn "%c")
// instead of:
for s in strings do printfn "%c" s
Similarly, using iter is nicer if you use it at the end of some processing pipeline:
// I would write this:
inputs |> Seq.filter (fun x -> x > 0)
|> Seq.iter (fun x -> foo x)
// instead of:
let filtered = inputs |> Seq.filter (fun x -> x > 0)
for x in filtered do foo x
You can modify mutable variables from the body of a for loop. You can't do that from a closure, which implies you can't do that using iter. (Note: I'm talking about mutable variables declared outside of the for / iter. Local mutable variables are accessible.)
Considering that the point of iter is to perform some side effect, the difference can be important.
I personally seldom use iter, as I find for to be clearer.
For most of the situations, they are the same. I would prefer the first use. It looks clear to me.
The difference is that for in loop support IEnumerable objects, while Seq.iter requires that your collection (linq.deltas) is IEnumerable<T>.
E.g. MatchCollection class in .net regular expression inherits IEnumerable not IEnumerable<T>, you cannot use Seq.map or Seq.iter directly on it. But you can use for in loop.
It is the style of programming. Imperative vs using functional programming. Keep in mind that F# is not a pure functional programming language.
Generally, use Seq.Iter if it is a part of some large pipeline processing, as that makes it much more clearer, but for ordinary case I think the imperative way is clearer. Sometime it is a personal preference, sometimes it is other issues like performance.
for in F# is a form of list comprehension - bread and butter of functional programming while Seq.iter is a 'for side-effects only' imperative construct - not a sign of a functional code. Here what you can do with for:
let pairsTo n = seq {
for i in [1..n] do
for j in [i..n] do
if j%i <> 0 then yield (i,j) }
printf "%A" (pairsTo 10 |> Seq.toList)
I have a list of type IList<Effort>. The model Effort contains a float called Amount. I would like to return the sum of Amount for the whole list, in F#. How would this be achieved?
efforts |> Seq.sumBy (fun e -> e.Amount)
Upvoted the answers of Seq.fold, pipelined Seq.fold, and pipelined Seq.sumBy (I like the third one best).
That said, no one has mentioned that seq<'T> is F#'s name for IEnumerable<T>, and so the Seq module functions work on any IEnumerable, including ILists.
Seq.fold (fun acc (effort: Effort) -> acc + effort.Amount) 0.0 efforts
One detail that may be interesting is that you can also avoid using type annotations. In the code by sepp2k, you need to specify that the effort value has a type Effort, because the compiler is processing code from the left to the right (and it would fail on the call effort.Amount if it didn't know the type). You can write this using pipelining operator:
efforts |> Seq.fold (fun acc effort -> acc + effort.Amount) 0.0
Now, the compiler knows the type of effort because it knows that it is processing a collection efforts of type IList<Effort>. It's a minor improvement, but I think it's quite nice.
Is there a way to "generate" functions like this one:
fun x -> x.ToString
I would like to be able to turn an instance method into a static method which takes "this" as a parameter, like so:
items |> Seq.filter my_predicate |> Seq.map Object.ToString
this has been discussed several times on the F# hub. See for example instance methods as functions. This is quite tricky problem, so there are no plans to have something like this in the first version of F# as far as I know, but it would be great to have something like that eventually :-).
Another workaround that you could do is to add static member as an extension method in F#:
type System.Object with
static member ObjToString(o:obj) = o.ToString()
open System
[ 1 .. 10 ] |> Seq.map Object.ObjToString;;
But that is a bit ugly. Also, it seems that this works only if you use different name for the method. I guess that F# doesn't allow you to overload existing method with an extension method and always prefer the intrinsic one.
I don't know if I exactly understood you, but for this specific example you could write:
items |> Seq.filter my_predicate |> Seq.map (fun x -> x.ToString)
or
let f = fun x -> x.ToString
items |> Seq.filter my_predicate |> Seq.map f