The ValuesAll function of the Deedle series according to the doco is to
Returns a collection of values, including possibly missing values. Note that the length of this sequence matches the `Keys` sequence.
However the following code will raise an error OptionalValue.Value: is not available. Is this expected behaviour? I was expecting ValuesAll can return double.nan
#I "..\..\packages\Deedle.1.2.4"
#load "Deedle.fsx"
open System
open System.Globalization
open System.Collections.Generic
open Deedle
let ts = [DateTime.Now.Date => Double.NaN; DateTime.Now.Date.AddDays(1.0) => 1.0] |> series
ts.Print()
ts.ValuesAll
>
27/01/16 12:00:00 AM -> <missing>
28/01/16 12:00:00 AM -> 1
val ts : Series<DateTime,float> =
27/01/16 12:00:00 AM -> <missing>
28/01/16 12:00:00 AM -> 1
val it : seq<float>
> ts.ValuesAll
;;
val it : seq<float> = Error: OptionalValue.Value: Value is not available
>
There are different implementations for valuesAll here and C#-friendly ValuesAll here. The latter one just accesses the .Value property of optional value and its signature is seq<float>, not seq<float opt>. Either implementation or docs are not consistent here.
When I used Deedle I filtered series with |> Series.dropMissing as a quick workaround when I needed only present values.
Related
I have a Deedle DataFrame of type Frame<int,string> that contains some missing values. I would like to convert the missing values into empty strings "". I tried to use the valueOr function but that did not help. Is there a way to do this?
Here is my DataFrame:
let s1 = Series.ofOptionalObservations [ 1 => Some("A"); 2 => None ]
let s2 = Series.ofOptionalObservations [ 1 => Some("B"); 2 => Some("C") ]
let df = Frame.ofColumns ["A", s1; "BC", s2]
Typing df;; in FSI yields some information including
ColumnTypes = seq [System.String; System.String];. So the values of df are of type string and not string option.
This is the function valueOr:
let valueOr (someDefault: 'a) (xo: 'a option) : 'a =
match xo with
| Some v -> v
| None -> someDefault
I defined an auxiliary function emptyFoo as:
let emptyFoo = valueOr ""
The signature of emptyFoo is string option -> string. This means emptyFoo should not be acceptable to the compiler in the following command:
let df' = Frame.mapValues emptyFoo df
This is because the values of df are of type string and not string option.
Still, the compiler does not complain and the code runs. However, df' still has a missing value.
Is there a way to transform the missing value into the empty string?
The Deedle documentation for Frame.mapValues:
Builds a new data frame whose values are the results of applying the specified function on these values, but only for those columns which can be converted to the appropriate type for input to the mapping function
So the mapping does nothing because strings are found, rather than string options.
I noticed another function that seems to do exactly what you want.
let df' = Frame.fillMissingWith "" df
The key thing I noticed was that Deedle shows those missing values as <missing>, suggesting that it uses it's own representation (as opposed to option for example). With that knowledge I guessed that the library would provide some way of manipulating missing values, so I explored the API by doing Frame. in my IDE and browsing the list of available functions and their documentation.
I am getting a strange output from Deedle when I use FirstKey and LastKey on a Frame of Rows. cl is a DataFrame in the below examples.
cl.Rows.FirstKey
returns: val it : (unit -> DateTime) = <fun:it#156-43> in the FSI window.
cl.Rows.LastKey
returns: val it : (unit -> DateTime) = <fun:it#157-44> in the FSI window.
I am expecting a Key Value of DateTime. Does anyone recognize the output? Do I have to do some kind of further processing to get useful output?
Update. I edited the question so that the complete output from the FSI window is now visible.
The comment by John Palmer has resolved my question. The correct way to code the FirstKey and LastKey function in my example(s):
cl.Rows.FirstKey()
cl.Rows.LastKey()
I am trying to work with Quartz.NET in F# and have run into a few issues with the fact that, while Quartz.NET is usable in F#, there does not seem to be much documentation on it, and I've had some difficulty with differences between it and what can find in C#.
One issue I have currently run into is setting SystemTime such as shown in this question,
Quartz.net + testing with SystemTime.UtcNow.
I could be wrong, but I thought that the code in F# should be:
SystemTime.Now = fun () -> DateTime(someDate)
SystemTime.UtcNow = fun () -> DateTime(someDate)
But I get an error about either too many arguments or function used where not expected. If I just use the DateTime constructor, I get an error related to the fact it is expecting a function.
The single = is an equality comparison operation. If you want to do assignment, use the <- assignment operator.
Apart from that, F# functions aren't the same as Func<T>. Normally, when you use them as method arguments, the conversion happens automatically, but in this case, it seems you'll need to explicitly perform the conversion:
open System
open Quartz
SystemTime.Now <-
Func<DateTimeOffset>(
fun () -> DateTimeOffset(DateTime(2015, 4, 18), TimeSpan.FromHours 2.))
SystemTime.UtcNow <-
Func<DateTimeOffset>(
fun () -> DateTimeOffset(DateTime(2015, 4, 18), TimeSpan.FromHours 2.))
To invoke them from F# is also a bit more involved:
> SystemTime.Now.Invoke();;
val it : DateTimeOffset = 18.04.2015 00:00:00 +02:00
> SystemTime.UtcNow.Invoke();;
val it : DateTimeOffset = 18.04.2015 00:00:00 +02:00
In reading John Palmer's answer to What is the difference between mutable values and immutable value redefinition?, John notes that
This sort of redefinition will only work in fsi.
In working with F# Interactive (fsi) I guess I subconsciously knew it, but never paid attention to it.
Now that it is apparent, why the difference?
More specifically please explain how the internals are different between fsi and the compiler such that this occurs by design or result of differences?
If the answer can elaborate on the internal structures that hold the bindings that would be appreciated.
The semantics are consistent with the way FSI compiles interactive submissions to an FSI session: each interactive submission is compiled as module which is open to subsequent interactive submissions.
The following is close to what FSI actual does and illustrates how let binding shadowing works across interactive submissions:
FSI Submission #1: let x = 1;;
module FSI_1 =
let x = 1
open FSI_1 //FSI_1.x is now bound to 1 and available at the top level
FSI Submission #2: let x = 2;;
module FSI_2 =
let x = 2
open FSI_2 //FSI_1.x is now shadowed by FSI_2.x which is bound to 2 and available at the top level
You can see the actual details how how the dynamic FSI assembly is compiled by using reflection on the FSI_ASSEMBLY assembly within the FSI app domain. Each interactive submission is literally emitted as a module (.NET class) with the naming pattern FSI_####. FsEye uses these facts to discover the state of FSI top-level bindings: https://code.google.com/p/fseye/source/browse/tags/2.0.1/FsEye/Fsi/SessionQueries.fs#24
The key takeaway in regard to #JohnPalmer's answer is that top-level FSI definitions cannot be mutated, when they are "redefined" they are merely being shadowed. We can show this as follows:
> let x = 1;; //our original definition of x
val x : int = 1
> let f () = x;; //capture x
val f : unit -> int
> let x = 2;; //shadow our original definition of x
val x : int = 2
> f();; //returns the original x value, which is still 1 rather than 2
val it : int = 1
I'm just beginning F# and haven't really done functional programming since my programming languages class 15 years ago (exception being "modern" C#).
I'm looking at this F# snippet using LINQPad 4:
let add x y = x + y
let lazyPlusOne x = lazy add x 1
let e = lazyPlusOne 15
Dump e
let plusOne x = add x 1
let f = plusOne 15
Dump f
The output it produces is:
Lazy<Int32>
Value is not created.
IsValueCreated False
Value 16
16
I understand the lazy keyword to delay evaluation until needed, same as C# delayed execution.
What is the meaning of: "Value is not created" here?
If you use lazy keyword to construct a lazy value (as in your lazyPlusOne function), then the result is a value of type Lazy<int>. This represents a value of type int that is evaluated only when it is actually needed.
I assume that Dump function tries to print the value including all its properties - when it starts printing, the value is not evaluated, so ToString method prints Value is not created. Then it iterates over other properties and when it accesses Value, the lazy value is evaluated (because its value is now needed). After evaluation, the property returns 16, which is then printed.
You can replace Dump with an F#-friendly printing function (or just use F# Interactive, which is extremely convenient way to play with F# inside Visual Studio with the usual IntelliSense, background error checking etec.)
F#-friendly printing function like printfn "%A" doesn't access the Value property, so it doesn't accidentally evaluate the value. Here is a snippet from F# Interactive:
> let a = lazy (1 + 2);;
val a : Lazy<int> = Value is not created. // Creates lazy value that's not evaluated
> a;;
val it : Lazy<int> = Value is not created. // Still not evaluated!
> a.Value;; // Now, the lazy value needs to be evaluated (to get the Value)
val it : int = 3
> a;; // After evaluation, the value stays cached
val it : Lazy<int> = 3
As of 'Dump e', 'lazyPlusOne 15' has not been evaluated. The 'let e = lazyPlusOne 15' does not require the evaluation of 'lazyPlusOne 15'. We don't yet need to know what e evaluates to yet.
The dump is triggering the evaluation and that is semantically different that just dumping the value after the evaluation.