In F# we have List.partition and Array.partition which return a tuple of lists and a tuple of arrays respectively.
so, why is there no Seq.partition returning a tuple of sequences?
here is a very simple implementation:
F# Snippets
so... why isn't this part of the core?
In F# 4.0 (Visual Studio 2015), the core libraries are a lot more uniform than before, but they still do not come with an implementation of Seq.partition. You can find more about this in the F# language design discussion: Regular functional operators producing two or more output collections.
The summary is that the Seq.partition function is quite tricky and a having it could introduce potential performance issues. There a couple of ways it can work:
It can iterate over the input collection twice (like the FsSnip version), which can cause issues if you have complex delayed computation (you're doing everything twice)
It can iterate over the input once, but then it would have to do some complex mutable state sharing (which could secretly allocate memory).
So, Seq.partition cannot be implemented reasonably while keeping all the good properties that you would expect about the seq<'T> type.
Seq.partition is just a specialized version of Seq.groupBy, so the standard library could implement the former as a wrapper around the latter without introducing any new issues.
let partition predicate source =
let map =
source
|> Seq.groupBy predicate
|> Map.ofSeq
let get flag =
map
|> Map.tryFind flag
|> Option.defaultValue Seq.empty
get true, get false
Related
There is need for tracing. The decorator should print function name, parameters values and return value. Instead of writing each time a decorator for each function, it would be terrific if it could be possible to do this programmatically.
The current function name can be discovered using reflection via MethodBase.GetCurrentMethod. Functions could be easily decorated with an inline function for logging:
let inline log args f =
let mi = System.Reflection.MethodBase.GetCurrentMethod()
let result = f ()
printf "%s %A -> %A" mi.Name args result
let add a b = log (a,b) (fun () -> a + b)
add 1 1
Which prints: add (1, 1) -> 2
EDIT: Another option would be to create a wrap function, i.e.:
let inline wrap f =
fun ps ->
let result = f args
printfn "%A -> %A" args result
result
let add (a,b) = a + b
wrap add (1,1)
However in this case there is not an easy way to programmatically retrieve the function name.
Yet another option might be to develop a Type Provider that takes an assembly path as a parameter and provides wrapped versions of all members.
I had the same desire previously and found that there are no current automated solutions for F#.
See: Converting OCaml to F#: Is there a simple way to simulate OCaml top-level #trace in F#
While OCaml's trace facility with time travel is the most useful debugging feature of comparison to what is desired, it is not an exact fit; but when I use OCaml it is the first inspection tool I use.
See: Using PostSharp with F# - Need documentation with working example
Also the suggestion of using AOP, i.e. PostSharp, was another good suggestion, as the response from Gael Fraiteur, the Principal Engineer of PostSharp points out:
PostSharp does not officially support F#.
Other than using reflection as suggested by Phillip Trelford, which I have not tried, the best solution I have found is to manually modify each function to be traced as I noted in Converting OCaml to F#: Is there a simple way to simulate OCaml top-level #trace in F# and save the results to a separate log file using NLog.
See: Using NLog with F# Interactive in Visual Studio - Need documentation
Another route to pursue would be check out the work of F# on Mono as there is a lot of work being done there to add extra tooling for use with F#.
Basically what I have found is that as my F# skills increase my need
to use a debugger or tracing decrease.
At present when I do run into a problem needing this level of inspection, adding the inspection code as noted in Converting OCaml to F#: Is there a simple way to simulate OCaml top-level #trace in F# helps to resolve my misunderstanding.
Also of note is that people who come from the C# world to the F# world tend to expect a debugger to be just as useful. Remember that imperative languages tend to be about manipulating data held in variables and that the debugger is used to inspect the values in these variables, while with functional programming, at least for me, I try to avoid mutable values and so the only values one needs to inspect are the values being passed to the function and no other values, thus reducing or obviating the need for a debugger or inspection beyond that of the function of question.
Seq.windowed in F# returns a sequence where each window within is an array. Is there a reason why each window is returned as an array (a very concrete type) as opposed to say, another sequence or IList<'T>? An IList<'T>, for example, would be sufficient if the purpose was to communicate that the items of the window can be randomly accessed but an array says two things: elements are mutable and randomly accessible. If you can rationalise the choice of array, how is windowed different from Seq.groupBy? Why does that latter (or operators in the same vein) not also return the members of a group as an array?
I'm wondering if this is simply a design oversight or is there a deeper, contractual reason for an array?
I do not know what is the design principle behind this. I suppose it might just be an accidental aspect of the implementation - Seq.windowed can be quite easily implemented by storing items in arrays, while Seq.groupBy probably needs to use some more complicated structure.
In general, I think that F# APIs either use 'T[] if using array is the natural efficient implementation, or return seq<'T> when the data source may be infinite, lazy, or when the implementation would have to convert the data to an array explicitly (then this can be left to the caller).
For Seq.windowed, I think that array makes a good sense, because you know the length of the array and so you are likely to use indexing. For example, assuming that prices is a sequence of date-price tuples (seq<DateTime * float>) you can write:
prices
|> Seq.windowed 5
|> Seq.map (fun win -> fst (win.[2]), Seq.averageBy snd win)
The sample calculates floating average and uses indexing to get the date in the middle.
In summary, I do not really have a good explanation for the design rationale, but I'm quite happy with the choices made - they seem to work really well with the usual use cases for the functions.
A couple of thoughts.
First, know that in their current version, both Seq.windowed and Seq.groupBy use non-lazy collections in their implementation. windowed uses arrays and returns you arrays. groupBy builds up a Dictionary<'tkey, ResizeArray<'tvalue>>, but keeps that a secret and returns the group values back as a seq instead of ResizeArray.
Returning ResizeArrays from groupBy wouldn't fit with anything else, so that obviously needs to be hidden. The other alternative is to give back the ToArray() of the data. This would require another copy of the data to be created, which is a downside. And there isn't really much upside, since you don't know ahead of time how big your group will be anyways, so you aren't expecting to do random access or any of the other special stuff arrays enable. So simply wrapping in a seq seems like a good option.
For windowed, it's a different story. You want an array back in this case. Why? Because you already know how big that array is going to be, so you can safely do random access or, better yet, pattern matching. That's a big upside. The downside remains, though - the data needs to be re-copied into a newly-allocated array for every window.
seq{1 .. 100} |> Seq.windowed 3 |> Seq.map (fun [|x; _; y|] -> x + y)
There is still the open question - "but couldn't we avoid the array allocation/copy downside by internally only using true lazy seqs, and returning them as such? Isn't that more in the 'spirit of the seq'?" It would be kind of tricky (would need some fancy cloning of enumerators?), but sure, probably with some careful coding. There's a huge downside to this, though. You'd need to cache the entire unspooled seq in memory to make it work, which kind of negates the whole goal of doing things lazily. Unlike lists or arrays, enumerating a seq multiple times is not guaranteed to yield the same results (e.g. a seq that returns random numbers), so the backing data for these seq windows you are returning needs to be cached somewhere. When that window is eventually accessed, you can't just tap in and re-enumerate through the original source seq - you might get back different data, or the seq might end at a different place. This points to the other upside of using arrays in Seq.windowed - only windowSize elements need to be kept in memory at once.
This is of course pure guess. I think this is related to the way both functions are implemented.
As already mentioned, in Seq.groupBy the groups are of variable length and in Seq.windowed they are of a fixed size.
So in the implementation from Seq.windowed it makes more sense to use a fixed size array, as opposed to the Generic.List used in Seq.groupBy, which btw in F# is called ResizeArray.
Now to the outside world, an Array though mutable is widely used in F# code and libraries and F# provides syntactic support for creating, initializing and manipulating arrays, whereas the ResizeArray is not that widely used in F# code and the language provides no syntactic support apart from the type alias, so I think that's why they decided to expose it as a Seq.
I'm following a language called 'elm' which is an attempt to bring a Haskel-esque syntax and FRP to Javascript. There has been some discussion here about implementing the pipeline operator from F# but the language designer has concerns about the increased cost (I assume in increased compilation time or compiler implementation complexity) over the more standard (in other FP langs at least) reverse pipeline operator (which elm already implements). Can anyone speak to this? [Feel free to post directly to that thread as well or I will paste back the best answers if no one else does].
https://groups.google.com/forum/?fromgroups=#!topic/elm-discuss/Kt0MbDyRpO4
Thanks!
In the discussion you reference, I see Evan poses two challenges:
Show me some F# project that uses it
Find some credible F# programmer talking about why it is a good idea and what costs come with it (blog post or something).
I'd answer as follows:
The forward pipe-idiom is very common in F# programming, both for stylistic (we like it) and practical (it helps type inference) reasons. Just about any F# project you'll find will use it frequently. Certainly all of my open source projects use it (Unquote, FsEye, NL found here). No doubt you'll find the same with all of the Github located F# projects including the F# compiler source itself.
Brian, a developer on the F# compiler team at Microsoft, blogged about Pipelining in F# back in 2008, a still very interesting and relevant blog which relates F# pipes to POSIX pipes. In my own estimation, there is very little cost to implementing a pipe operator. In the F# compiler, this is certainly true in every sense (it's a one-line, inline function definition).
The pipeline operator is actually incredibly simple - here is the standard definition
let inline (|>) a b = b a
Also, the . operator discussed in the thread is the reverse pipe operator in F# (<|) which enables you to eliminate some brackets.
I don't think adding pipeline operators would have a significant impact on complexity
In addition to the excellent answers already given here, I'd like to add a couple more points.
Firstly, one of the reasons why the pipeline operator is common in F# is that it helps to circumvent a shortcoming the way type inference is currently done. Specifically, if you apply an aggregate operation with a lambda function that uses OOP to a collection type inference will typically fail. For example:
Seq.map (fun z -> z.Real) zs
This fails because F# does not yet know the type of z when it encounters the property Real so it refuses to compile this code. The idiomatic fix is to use the pipeline operator:
xs |> Seq.map (fun z -> z.Real)
This is strictly uglier (IMO) but it works.
Secondly, the F# pipe operator is nice to a point but you cannot currently get the inferred type of an intermediate result. For example:
x
|> h
|> g
|> f
If there is a type error at f then the programmer will want to know the type of the value being fed into f in case the problem was actually with h or g but this is not currently possible in Visual Studio. Ironically, this was easy in OCaml with the Tuareg mode for Emacs because you could get the inferred type of any subexpression, not just an identifier.
Following this from the Real-World Functional Programming blog about drawing bar and column charts, I was trying to draw a histogram for my data which is stored at a set of tuples (data_value, frequency) in a lazy sequence.
It does not work unless I convert the sequence into a List, the error message being that in case of sequence "the IEnumerable 'T does not support the Reset function". Is there any way to generate a histogram/chart etc. using the .NET library from a lazily-evaluated sequence?
Also (ok newbie query alert), is there any way to make the chart persist when the program is run from the console? The usual System.Console.ReadKey() |> ignore makes the chart window hang, and otherwise it disappears in an instant. I've been using "Send to Interactive" to see results til now.
The problem is that sequences (of type seq<T>, which is just an alias for IEnumerable<T>) generated using the F# sequence expression notation do not support the Reset method. The method is required by the charting library (because it needs to obtain the data each time it redraws the screen).
This means that, for example, the following will not work:
seq { for x in data -> x } |> FSharpChart.Line
Many of the standard library functions from the Seq module are implemented using sequence expressions, so the result will not support Reset. You can fix that by converting the data to a list (using List.ofSeq) or to an array (using Array.ofSeq) or by writing the code using lists:
[ for x in data -> x ] |> FSharpChart.Line
... and if you're using some function, you can take the one from List (not all of the Seq functions are available for List, so sometimes you will need to use conversion):
[ for x in data -> x ] |> List.choose op |> FSharpChart.Line
No, it does not accept sequences.
That said, there is a good reason for not supporting seq. it is about the structure itself : A seq is just that, a seq, and therefore does not and should not, support the kind of operations needed in drawing a graph. That said, I really wish this stack was more advanced and supported more use style.
So the answer is to
|> Seq.toArray
or
|> Seq.toList
before sending to the chart library
I am trying to define Bill Of Materials kind of queries using the recently introduced F# 3.0 query comprehension syntax. Though it is possible to define these kind of queries using yield! seq comprehensions for in-memory collections I had no lack in translating those into query comprehensions that target remotable IQueryable sources. I guess the hard part would be to "train" the provider into recognizing Common Table Expressions out of recursive patterns.
Any ideas?
Unfortunately, I don't think that the current query syntax support in F# 3.0 is capable of dealing with recursive queries. The main problem is that F# 3.0 relies on standard IQueryable implementations that were designed mainly for C# and so they do not expect recursive structures.
I think that supporting this would be quite difficult. You could either implement your own F# quotations to SQL translator (which is hard) or you could implement some sort of pre-processor that takes an F# quotation (query) that contains recursion and translates the recursion to something that the LINQ to SQL translator can deal with (but this is probably hard too).
In general, the approach would be to define your own query builder:
open System.IO
open Microsoft.FSharp.Quotations
type MyQueryBuilder() =
member x.For(a, body) = Seq.collect body a
member x.Quote(e) = e
member x.YieldFrom(s) = s
member x.Run(e:Expr<'T>) : 'T = failwithf "%A" e
// Example using the custom query builder
// (fails, printing the quoted query)
let mquery = MyQueryBuilder()
let n = [1 .. 10]
let rec nums a : seq<int> =
mquery { for b in n do
yield! nums b }
In the Run method, you get a quotation that represents the query. You could pre-process that and replace all calls to MyQueryBuilder with calls to standard query operations and replace recursion with something else. Then you could call query.Run (to run the standard IQueryable implementation).
Though as I said, this is probably going to be quite difficult to implement - but perhaps, if you have some specific kind of recursion that you can easily deal with, it might be an option. However, if LINQ to SQL does not generate Common Table Expressions for any standard patterns, then I don't think you can train it to generate them - as far as I know, the translator is not really extensible.