Following this from the Real-World Functional Programming blog about drawing bar and column charts, I was trying to draw a histogram for my data which is stored at a set of tuples (data_value, frequency) in a lazy sequence.
It does not work unless I convert the sequence into a List, the error message being that in case of sequence "the IEnumerable 'T does not support the Reset function". Is there any way to generate a histogram/chart etc. using the .NET library from a lazily-evaluated sequence?
Also (ok newbie query alert), is there any way to make the chart persist when the program is run from the console? The usual System.Console.ReadKey() |> ignore makes the chart window hang, and otherwise it disappears in an instant. I've been using "Send to Interactive" to see results til now.
The problem is that sequences (of type seq<T>, which is just an alias for IEnumerable<T>) generated using the F# sequence expression notation do not support the Reset method. The method is required by the charting library (because it needs to obtain the data each time it redraws the screen).
This means that, for example, the following will not work:
seq { for x in data -> x } |> FSharpChart.Line
Many of the standard library functions from the Seq module are implemented using sequence expressions, so the result will not support Reset. You can fix that by converting the data to a list (using List.ofSeq) or to an array (using Array.ofSeq) or by writing the code using lists:
[ for x in data -> x ] |> FSharpChart.Line
... and if you're using some function, you can take the one from List (not all of the Seq functions are available for List, so sometimes you will need to use conversion):
[ for x in data -> x ] |> List.choose op |> FSharpChart.Line
No, it does not accept sequences.
That said, there is a good reason for not supporting seq. it is about the structure itself : A seq is just that, a seq, and therefore does not and should not, support the kind of operations needed in drawing a graph. That said, I really wish this stack was more advanced and supported more use style.
So the answer is to
|> Seq.toArray
or
|> Seq.toList
before sending to the chart library
Related
In F# we have List.partition and Array.partition which return a tuple of lists and a tuple of arrays respectively.
so, why is there no Seq.partition returning a tuple of sequences?
here is a very simple implementation:
F# Snippets
so... why isn't this part of the core?
In F# 4.0 (Visual Studio 2015), the core libraries are a lot more uniform than before, but they still do not come with an implementation of Seq.partition. You can find more about this in the F# language design discussion: Regular functional operators producing two or more output collections.
The summary is that the Seq.partition function is quite tricky and a having it could introduce potential performance issues. There a couple of ways it can work:
It can iterate over the input collection twice (like the FsSnip version), which can cause issues if you have complex delayed computation (you're doing everything twice)
It can iterate over the input once, but then it would have to do some complex mutable state sharing (which could secretly allocate memory).
So, Seq.partition cannot be implemented reasonably while keeping all the good properties that you would expect about the seq<'T> type.
Seq.partition is just a specialized version of Seq.groupBy, so the standard library could implement the former as a wrapper around the latter without introducing any new issues.
let partition predicate source =
let map =
source
|> Seq.groupBy predicate
|> Map.ofSeq
let get flag =
map
|> Map.tryFind flag
|> Option.defaultValue Seq.empty
get true, get false
There is need for tracing. The decorator should print function name, parameters values and return value. Instead of writing each time a decorator for each function, it would be terrific if it could be possible to do this programmatically.
The current function name can be discovered using reflection via MethodBase.GetCurrentMethod. Functions could be easily decorated with an inline function for logging:
let inline log args f =
let mi = System.Reflection.MethodBase.GetCurrentMethod()
let result = f ()
printf "%s %A -> %A" mi.Name args result
let add a b = log (a,b) (fun () -> a + b)
add 1 1
Which prints: add (1, 1) -> 2
EDIT: Another option would be to create a wrap function, i.e.:
let inline wrap f =
fun ps ->
let result = f args
printfn "%A -> %A" args result
result
let add (a,b) = a + b
wrap add (1,1)
However in this case there is not an easy way to programmatically retrieve the function name.
Yet another option might be to develop a Type Provider that takes an assembly path as a parameter and provides wrapped versions of all members.
I had the same desire previously and found that there are no current automated solutions for F#.
See: Converting OCaml to F#: Is there a simple way to simulate OCaml top-level #trace in F#
While OCaml's trace facility with time travel is the most useful debugging feature of comparison to what is desired, it is not an exact fit; but when I use OCaml it is the first inspection tool I use.
See: Using PostSharp with F# - Need documentation with working example
Also the suggestion of using AOP, i.e. PostSharp, was another good suggestion, as the response from Gael Fraiteur, the Principal Engineer of PostSharp points out:
PostSharp does not officially support F#.
Other than using reflection as suggested by Phillip Trelford, which I have not tried, the best solution I have found is to manually modify each function to be traced as I noted in Converting OCaml to F#: Is there a simple way to simulate OCaml top-level #trace in F# and save the results to a separate log file using NLog.
See: Using NLog with F# Interactive in Visual Studio - Need documentation
Another route to pursue would be check out the work of F# on Mono as there is a lot of work being done there to add extra tooling for use with F#.
Basically what I have found is that as my F# skills increase my need
to use a debugger or tracing decrease.
At present when I do run into a problem needing this level of inspection, adding the inspection code as noted in Converting OCaml to F#: Is there a simple way to simulate OCaml top-level #trace in F# helps to resolve my misunderstanding.
Also of note is that people who come from the C# world to the F# world tend to expect a debugger to be just as useful. Remember that imperative languages tend to be about manipulating data held in variables and that the debugger is used to inspect the values in these variables, while with functional programming, at least for me, I try to avoid mutable values and so the only values one needs to inspect are the values being passed to the function and no other values, thus reducing or obviating the need for a debugger or inspection beyond that of the function of question.
I have a F# query that returns seq<float>.
I was wondering how I could stuff the data within the seq<float> in a ILNumerics dense array (http://ilnumerics.net/FunctionRules.html) using an F# idiomatic functional style.
The data in the seq may correspond to a row, a column or the entire dense array.
References to F# code are also welcome.
Looking at the C# example on this page, something like this should work
let toILNUM input = array(input |> Seq.toArray)
Seq.windowed in F# returns a sequence where each window within is an array. Is there a reason why each window is returned as an array (a very concrete type) as opposed to say, another sequence or IList<'T>? An IList<'T>, for example, would be sufficient if the purpose was to communicate that the items of the window can be randomly accessed but an array says two things: elements are mutable and randomly accessible. If you can rationalise the choice of array, how is windowed different from Seq.groupBy? Why does that latter (or operators in the same vein) not also return the members of a group as an array?
I'm wondering if this is simply a design oversight or is there a deeper, contractual reason for an array?
I do not know what is the design principle behind this. I suppose it might just be an accidental aspect of the implementation - Seq.windowed can be quite easily implemented by storing items in arrays, while Seq.groupBy probably needs to use some more complicated structure.
In general, I think that F# APIs either use 'T[] if using array is the natural efficient implementation, or return seq<'T> when the data source may be infinite, lazy, or when the implementation would have to convert the data to an array explicitly (then this can be left to the caller).
For Seq.windowed, I think that array makes a good sense, because you know the length of the array and so you are likely to use indexing. For example, assuming that prices is a sequence of date-price tuples (seq<DateTime * float>) you can write:
prices
|> Seq.windowed 5
|> Seq.map (fun win -> fst (win.[2]), Seq.averageBy snd win)
The sample calculates floating average and uses indexing to get the date in the middle.
In summary, I do not really have a good explanation for the design rationale, but I'm quite happy with the choices made - they seem to work really well with the usual use cases for the functions.
A couple of thoughts.
First, know that in their current version, both Seq.windowed and Seq.groupBy use non-lazy collections in their implementation. windowed uses arrays and returns you arrays. groupBy builds up a Dictionary<'tkey, ResizeArray<'tvalue>>, but keeps that a secret and returns the group values back as a seq instead of ResizeArray.
Returning ResizeArrays from groupBy wouldn't fit with anything else, so that obviously needs to be hidden. The other alternative is to give back the ToArray() of the data. This would require another copy of the data to be created, which is a downside. And there isn't really much upside, since you don't know ahead of time how big your group will be anyways, so you aren't expecting to do random access or any of the other special stuff arrays enable. So simply wrapping in a seq seems like a good option.
For windowed, it's a different story. You want an array back in this case. Why? Because you already know how big that array is going to be, so you can safely do random access or, better yet, pattern matching. That's a big upside. The downside remains, though - the data needs to be re-copied into a newly-allocated array for every window.
seq{1 .. 100} |> Seq.windowed 3 |> Seq.map (fun [|x; _; y|] -> x + y)
There is still the open question - "but couldn't we avoid the array allocation/copy downside by internally only using true lazy seqs, and returning them as such? Isn't that more in the 'spirit of the seq'?" It would be kind of tricky (would need some fancy cloning of enumerators?), but sure, probably with some careful coding. There's a huge downside to this, though. You'd need to cache the entire unspooled seq in memory to make it work, which kind of negates the whole goal of doing things lazily. Unlike lists or arrays, enumerating a seq multiple times is not guaranteed to yield the same results (e.g. a seq that returns random numbers), so the backing data for these seq windows you are returning needs to be cached somewhere. When that window is eventually accessed, you can't just tap in and re-enumerate through the original source seq - you might get back different data, or the seq might end at a different place. This points to the other upside of using arrays in Seq.windowed - only windowSize elements need to be kept in memory at once.
This is of course pure guess. I think this is related to the way both functions are implemented.
As already mentioned, in Seq.groupBy the groups are of variable length and in Seq.windowed they are of a fixed size.
So in the implementation from Seq.windowed it makes more sense to use a fixed size array, as opposed to the Generic.List used in Seq.groupBy, which btw in F# is called ResizeArray.
Now to the outside world, an Array though mutable is widely used in F# code and libraries and F# provides syntactic support for creating, initializing and manipulating arrays, whereas the ResizeArray is not that widely used in F# code and the language provides no syntactic support apart from the type alias, so I think that's why they decided to expose it as a Seq.
I am a beginner learning Erlang. After reading about list comprehensions and recursion in Erlang, I wanted to try to implement my own map function, which turned out like this:
% Map: Map all elements in a list by a function
map(List,Fun) -> map(List,Fun,[]).
map([],_,Acc) -> lists:reverse(Acc);
map([H|T],Fun,Acc) -> map(T,Fun,[Fun(H)|Acc]).
My question is: It feels wrong to build up a list through the recursive function, and then reverse it at the end. Is there any way to build up the list in the right order, so we don't need the reverse ?
In oder to understand why accumulating and reversing is quite fast you have to understand how lists are build in Erlang. Erlangs lists like those in Lisp are build out of cons cells (look at the picture in the link).
In a singly linked list like the Erlang lists it is very cheap to prepend a element (or a short list). This is was the List = [H|T] construct does.
Reversing a singly linked list made of cons cells is quite fast since you only need one pass along the list, just prepending the next element to you already reversed partial result. And as was already mentioned, it is also implemented in C in Erlang.
Also building a result in reverse order often can be accomplished by a tail recursive function which means that no stack is built up and (in old versions of Erlang only!) therefore some memory can be saved.
Having said all this: it is one of The Eight Myths of Erlang Performance that it is always better to build in reverse in a tail recursive function and call lists:reverse/1 at the end.
Here is a body recursive version without lists:reverse/1 this would use more memory on Erlang versions before 12 but not so on current versions:
map([H|T], Fun) ->
[ Fun(H) | map(T,Fun) ];
map([],_) -> [].
And here is a version of map using list comprehensions:
map(List, Fun) ->
[ Fun(X) || X <- List ].
As you can see this is so simple because map is just a built in part of the list comprehension, so you would use the list comprehension directly and have no need for map anymore.
As an extra a pure Erlang implementation that shows how reversing a list of cons cells wors efficiently (in Erlang its always faster to call lists:reverse/1 because it is in C, but does the same thing).
reverse(List) ->
reverse(List, []).
reverse([H|T], Acc) ->
reverse(T, [H|Acc]);
reverse([], Acc) ->
Acc.
As you can see there are only [A|B] operations on the list, taking cons cells apart (when pattern matching) and building new when doing the [H|Acc].
Such construction [ Element | Acc] and then lists:reverse(Acc) is quite common pattern in functional programming.
You can write code with an accumulator storing data in correct order, but this code will be slower than the one in your question (hint: use Acc ++ [Fun(H)]).
Actually within Erlang lists:reverse is implemented in C and it works quite fast.