I use the CSV Type Provider quite a bit which uses tuples for all the underlying data. My issue is that I run into csv files with lots of columns and find that building the infrastructure around manipulating data of that size to be rather cumbersome. How do I build a function that will update only a specific subset of the columns without having to create huge pattern matching statements? See code example below.
// I want to generate lots of sample data for the first three columns
// And I would like to avoid creating giant record types in order to
// do this task as the csv's are always changing and the records would
// have to change as well
type CumbersomeCSVRow = // 17 fields now, but imagine 212
string * string * string * Option<int> * string *
Option<int> * string * string * string * string *
string * string * Option<float> * string *
Option<float> * Option<float> * string
// Here is a sample row of data
let sampleCumbersomeRow : CumbersomeCSVRow =
("First","Second","Third",Some(52),"MSCI",None,"B74A123","",
"Airlines","Transportation","","",Some(1.04293),"Updated",
Some(0.95),Some(56.7423),"Measured")
// Record type of the sample data that I want to 'insert' into the cumbersome data type
type FirstThreeStrings =
{ First : string; Second : string; Third : string}
// and some instances of the new data
let generatedFrontMatters =
// imagine lots of sample data and workflows to create it, hence the records
seq { for letter in ["A";"B";"C"] ->
{ First = letter;
Second = letter + letter
Third = letter + letter + letter } }
Take a look at the FSharp.Reflection module
// Your Code --------------------------------------------------
// I want to generate lots of sample data for the first three columns
// And I would like to avoid creating giant record types in order to
// do this task as the csv's are always changing and the records would
// have to change as well
type CumbersomeCSVRow = // 17 fields now, but imagine 212
string * string * string * Option<int> * string *
Option<int> * string * string * string * string *
string * string * Option<float> * string *
Option<float> * Option<float> * string
// Here is a sample row of data
let sampleCumbersomeRow : CumbersomeCSVRow =
("First","Second","Third",Some(52),"MSCI",None,"B74A123","",
"Airlines","Transportation","","",Some(1.04293),"Updated",
Some(0.95),Some(56.7423),"Measured")
// Record type of the sample data that I want to 'insert' into the cumbersome data type
type FirstThreeStrings =
{ First : string; Second : string; Third : string}
// and some instances of the new data
let generatedFrontMatters =
// imagine lots of sample data and workflows to create it, hence the records
seq { for letter in ["A";"B";"C"] ->
{ First = letter;
Second = letter + letter
Third = letter + letter + letter } }
// Response ---------------------------------------------------
open FSharp.Reflection
// create a static back matter to append to
// the slicing here is the part that will need to change as the data changes
// ++ maybe there's a better way to handle this part??
let staticBackMatter =
sampleCumbersomeRow
|> FSharpValue.GetTupleFields
|> (fun x -> x.[3 .. 16])
// cast the front matter using FSharp.Reflection
let castedFrontMatters = Seq.map FSharpValue.GetRecordFields generatedFrontMatters
// append the data arrays together, create a tuple using reflection, and downcast the result
// to your type
let generatedRows =
castedFrontMatters
|> Seq.map (fun frontArray -> Array.append frontArray staticBackMatter)
|> Seq.map (fun data -> FSharpValue.MakeTuple(data,typeof<CumbersomeCSVRow>) :?> CumbersomeCSVRow)
|> Seq.toList
Whether this is proper F#, I don't know.
Related
I'm having some issues finding out how to cast a string array to a specific type.
Here is my code
type plItem = {
sku: string
name: string
size: string
buy: decimal
sell: decimal
barcode: string
}
// .... ( get values from google sheets )
let values:IList<IList<Object>> = response.Values
let pl = values |> Seq.map ( fun item -> Seq.toArray )
At the end of the code - pl is now an array of strings. I want to make it an aray of the type ( above ) plItem I'm not sure of the easiest way to to this.
If I understand the question correctly, you have a list of list where the outer list represents rows and the nested list represents columns. You want to turn the rows into records and the columns correspond to individual record fields.
There is no automatic way of doing that. You'll just have to extract individual columns and convert them to an appropriate type (either by unboxing the type into a string or by getting the string value and then parsing it). Something like this:
open System
open System.Collections.Generic
type plItem = {
sku: string
name: string
size: string
buy: decimal
sell: decimal
barcode: string
}
let values:IList<IList<Object>> =
[| [| box "12660"; box "Probiotics 14 Strains"; box "60 VCaps";
box "31.46"; box "50.35"; box "9403067126606"; |] :> IList<_>
|] :> IList<_>
let pl = values |> Seq.map (fun row ->
{ sku = unbox row.[0]
name = unbox row.[1]
size = unbox row.[2]
buy = decimal (unbox<string> row.[3])
sell = decimal (unbox<string> row.[4])
barcode = unbox row.[5] })
Consider the following code:
let pair = System.Tuple.Create (10, "foo") // val pair : int * string = (10, "foo")
let tuple = System.Tuple.Create <| (10, "foo") // val tuple : System.Tuple<int * string> = ((10, "foo"))
Why doesn't the two lines yield values of the same type? Does the type of the argument (10, "foo") somehow change between the two lines?
What's the exact difference between int * string and System.Tuple<int * string>?
For 2, at least the latter has null as a value (this is how this question came up). Are there other differences?
There are two different overloads of Tuple.Create:
Tuple.Create<'T1>(item1: 'T1)
Tuple.Create<'T1, 'T2>(item1: 'T1, item2: 'T2)
In the first case you just calling a method with two arguments. So the second Tuple.Create overload is obviously picked. No surprise.
But with piping you first create a tuple instance. And then pass it to Tuple.Create method. This is what happens in the second example
let intermediate : Tuple<int, string> = (10, "foo")
let tuple = Tuple.Create(intermediate)
With a single argument the first Tuple.Create overload will be picked.
Note: star type is a way tuple type names are written in F#. So Tuple<int, string, bool> will be (int * string * bool). It's the same thing.
Your tuple is a tuple of one (1) element, namely the tuple 10,"foo". It is equivalent of
System.Tuple.Create(System.Tuple.Create(10, "foo"))
Your pair on the other hand is a tuple of the two elements 10 and "foo".
So pair has type System.Tuple<int,string> (which is the same as int * string), but tuple has type System.Tuple<System.Tuple<int,string>> (which is System.Tuple<int * string>)
I'm trying to create a communication library that interacts with hardware. The protocol is made up of byte arrays with a header (source/destination address, command number, length) and a command specific payload. I'm creating Record Types for each of the commands to make them more user friendly.
Is there a more idiomatic way of converting an array to a record than
let data = [0;1]
type Rec = {
A : int
B : int
}
let convert d =
{
A = d.[0]
B = d.[1]
}
This can become very tedious when the records are much larger.
A few comments:
You record type definition is bogus - there should be no = in there. I assume you want
type Rec = {
A : int
B : int
}
You mentioned byte arrays, but your data value is a List. Accessing List items by index is expensive (O(n)) and should be avoided. If you meant to declare it as an array, the syntax is let data = [|0;1|]
But I wonder if records are the right fit here. If your goal is to have a single function that accepts a byte array and returns back various strongly-typed interpretations of that data, then a discriminated union might be best.
Maybe something along these lines:
// various possible command types
type Commands =
| Command1 of byte * int // maybe payload of Command1 is known to be an int
| Command2 of byte * string // maybe payload of Command1 is known to be a string
// active pattern for initial data decomposition
let (|Command|) (bytes : byte[]) =
(bytes.[0], bytes.[1], Array.skip 2 bytes)
let convert (bytes : byte[]) =
match bytes with
| Command(addr, 1uy, [| intData |]) ->
Command1(addr, int intData)
| Command(addr, 2uy, strData) ->
Command2(addr, String(Text.Encoding.ASCII.GetChars(strData)))
| _ ->
failwith "unknown command type"
// returns Command1(0x10, 42)
convert [| 0x10uy; 0x01uy; 0x2Auy |]
// returns Command2(0x10, "foobar")
convert [| 0x10uy; 0x02uy; 0x66uy; 0x6Fuy; 0x6Fuy; 0x62uy; 0x61uy; 0x72uy |]
I am trying to read a csv file (without headers) using F#-data. So far I have got:
let filename = #"data.csv"
let file = File.OpenText(filename)
let data = CsvFile.Load(file)
for row in data.Rows do
// ..
I would like to convert each row to an array of integers. How can I accomplish this?
The row value is an object that exposes all values of the row via row.Columns. This is an array of strings, so you can use Array.map and turn each value into a float using the float function.
for row in data.Rows do
let asFloatArray = Array.map float row.Columns
printfn "%A" asFloatArray // TODO: Do something useful here :-)
Why is disabled types like
type t = A of int | B of string * mutable int
while such types are allowed:
type t = A of int | B of string * int ref
The question is, how would you modify the value of a mutable element of discriminated union case? For ref types, this is quite easy, because ref is a reference cell (a record actually) which contains the mutable value:
match tval with
| B(str, refNum) -> refNum := 4
We extract the reference cell and assign it to a new symbol (or a new variable) refNum. Then we modify the value inside the ref cell, which also modifies tval, because the two references to the cell (from discriminated union case and from refNum variable) are aliased.
On the other hand, when you write let mutable n = 0, you're creating a variable, which can be directly mutated, but there is no cell holding the mutable value - the variable n is directly mutable. This shows the difference:
let mutable a = 10
let mutable b = a
b <- 5 // a = 10, b = 5
let a = ref 10
let b = a
b := 5 // a = 5, b = 5 (because of aliasing!)
So, to answer your question - there is no way to directly refer to the value stored inside the discriminated union case. You can only extract it using pattern matching, but that copies the value to a new variable. This means that there isn't any way you could modify the mutable value.
EDIT
To demonstrate limitations of mutable values in F#, here is one more example - you cannot capture mutable values inside a closure:
let foo() =
let mutable n = 0
(fun () -> n <- n + 1; n) // error FS0407
I think the reason is same as with discriminated union cases (even though it's not as obvious in this case). The compiler needs to copy the variable - it is stored as a local variable and as a field in the generated closure. And when copying, you want to modify the same variable from multiple references, so aliasing semantics is the only reasonable thing to do...
Ref is a type (int ref = ref<int>). Mutable is not a type, it's a keyword that allows you to update a value.
Example:
let (bla:ref<int>) = ref 0 //yup
let (bla:mutable<int>) = 3 //no!