F# store data in ms access table - f#

how can i export arrays to ms access table using OleDbConnection. Below is the code i have so far.
let A=y|>Array.map (fun x->x.a)
let B=y|>Array.map (fun x->x.b)
let C=y|>Array.map (fun x->x.c)
let D=y|>Array.map (fun x->x.d)
let cnn = #"Provider=Microsoft.ACE.OLEDB.12.0;
Data Source=U:\test.accdb;
Persist Security Info=False;"
let conn = new OleDbConnection(cnn)
conn.Open()
use com = conn.CreateCommand()
com.CommandText <- "INSERT INTO Test values" {A,B,C,D}
com.ExecuteNonQuery() |> ignore
conn.Close()

In order to insert multiple rows, you need to issue multiple commands. Access doesn't allow inserting multiple values with one insert statement, see this question for more: Insert multiple rows using one insert statement in Access 2010
As long as you're fine with using multiple statements, you have to prepare the command correctly - that is, include the actual values after the VALUES part. And while you're at it, you should use parameters, too, if you don't want to be vulnerable to SQL injection. Like this:
use com = conn.CreateCommand()
com.CommandText <- "INSERT INTO Test values( ?, ?, ?, ? )"
let param name =
let p = com.CreateParameter()
p.ParameterName <- name
com.Parameters.Add p
p
let pA, pB, pC, pD = param "#a", param "#b", param "#c", param "#d"
y |> Array.iter (fun x ->
pA.Value <- x.a
pB.Value <- x.b
pC.Value <- x.c
pD.Value <- x.d
com.ExecuteNonQuery() )
conn.Close()
If you really want to speed it up, you'll have to find a way to bulk-insert records into Access, similar to SqlBulkInsert for SQL server. I don't know of such way, if it even exists.
Alternatively, you can create a commands consisting of N insert statements (where N = y.Length), but then that statement will have 4N parameters, and I believe Access had a limitation on the number of parameters.
Alternatively-alternatively, you can do the above, but without parameters, inserting values right into the text of the SQL command:
let commandText =
y
|> Seq.map (fun x ->
// assuming all four parameters are strings
sprintf "INSERT INTO Test values( '%s', '%s', '%s', '%s' )" x.a x.b x.c x.d
|> String.concat " "
use com = conn.CreateCommand()
com.CommandText <- commandText
com.ExecuteNonQuery()
conn.Close()
But in that case, you open yourself up to SQL injection attacks. You'll have to sanitize your array values very carefully to prevent it. I do not recommend doing this just to gain a few extra seconds in performance.
And speaking of performance: are you absolutely sure multiple insert statements are going to be too slow for your purposes? Why? Have you measured it? How?
Remember the golden rule of performance tuning: first measure, then optimize.

Related

CSV Type Provider & Accessing Data

Good evening! I am a very new programmer getting my feet wet with F#. I am attempting to do some simple data analysis and plotting but I cannot figure out how access the data properly. I get everything set up and use the CSVProvider and it works perfectly:
#load #"packages\FsLab\FsLab.fsx"
#load #"packages\FSharp.Charting\FSharp.Charting.fsx"
open Deedle
open FSharp.Data
type Pt = CsvProvider<"C:/Users/berkl/Test10/CGC.csv">
let data = Pt.Load("C:/Users/berkl/Test10/CGC.csv")
Then, I pull out the data for a specific entry:
let test = data.Rows |> Seq.filter (fun r -> r.``Patient number`` = 2104)
This works as expected and prints the following to FSI:
test;;
val it : seq<CsvProvider<...>.Row> =
seq
[(2104, "Cita 1", "Nuevo", "Femenino", nan, nan, nan);
(2104, "Cita 2", "Establecido", "", 18.85191818, 44.0, 103.0);
(2104, "Cita 3", "Establecido", "Femenino", 17.92617533, 46.0, 108.0);
(2104, "Cita 4", "Establecido", "Femenino", nan, nan, nan); ...]
Here is where I'm at a loss. I want to take out the fifth column and plot it against the sixth column. And I don't know how to access it.
What I can do so far is access a single value in one of the columns:
let Finally = Seq.item 1 test
let PtHt = Finally.Ht_cm
Any help is much appreciated!!
I would probably recommend using the XPlot library instead of F# Charting, because that is the one that's going to be available in FsLab in the long term (it is cross-platform).
To create a chart using XPlot, you need to give it a sequence of pairs with X and Y values:
#load "packages/FsLab/FsLab.fsx"
open XPlot.Plotly
Chart.Scatter [ for x in 0.0 .. 0.1 .. 10.0 -> x, sin x ]
In your example, you can get the required format using sequence comprehensions (as in the above example) or using Seq.map as in the existing answer - both options do the same thing:
// Using sequence comprehensions
Chart.Scatter [ for row in test -> row.Ht_cm, row.Wt_kg ]
// Using Seq.map and piping
test |> Seq.map (fun row -> row.Ht_cm, row.Wt_kg) |> Chart.Scatter
The key thing is that you need to produce one sequence (or a list) containing the X and Y values as a tuple (rather than producing two separate sequences).
What you want to do is transform your sequence of rows to a sequence of values from a column. You use Seq.map for any such transformation.
In your case, you could do (modulo the correct column names which I don't have)
let col5 =
test
|> Seq.map (fun row -> row.Ht_cm)
let col6 =
test
|> Seq.map (fun row -> row.Wt_kg)

How to sortby a variable ascending in F#?

How would I sort a list by a variable but ascending?
Something like:
Data |> List.sortBy(fun t -> t.Date,ascending(t.Value))
The above is an example, I know that this will not work if run.
Based on your example, it looks like you want to use multiple sorting keys and some of them should be in the ascending order while others in descending order. I think this is a scenario that has not been answered by any of the other questions.
In general, you can use multiple sorting keys in F# by using tuples. F# has functions List.sortBy and List.sortByDescending which give you the two possible orders:
data |> Seq.sortByDescending (fun x -> x.FirstKey, x.SecondKey)
However, this way the sort order for both of the keys will be the same. There is no easy way to use one key in one order and another key in another order. In many cases, you could just use numerical minus and do something like:
data |> Seq.sortByDescending (fun x -> x.FirstKey, -x.SecondKey)
This is not entirely bullet-proof because of MaxInt values, but it will probably often work. In F# query expressions (which are inspired by how LINQ works), you can use multiple sorting keys using sortBy and thenBy (or sortByDescending and thenByDescending):
query {
for x in data do
sortByDescending x.FirstKey
thenBy x.SecondKey }
Here, the first key will be used for descending sort and, when there are multiple items with the same FirstKey, the second key will be used for ascending sort within that group. I suspect this is probably what you need in the general case - but it's a bit unfortunate there is no nice way of writing this with the pipeline syntax.
You can easily sort by multiple keys, ascending, descending, or any other complex order, with List.sortWith and the power of function composition:
All you need is a couple of helper functions and an operator:
let asc f a b = compare (f a) (f b)
let desc f a b = compare (f b) (f a)
let (&>) c1 c2 a b = match c1 a b with 0 -> c2 a b | r -> r
Both asc and desc receive a key-retrieve-function of type 'T->'K and call the generic function compare to sort in ascending or descending order.
The operator &> lets you compose them to sort by as many keys as you like. And since you can also add your custom comparers, any kind of sorting is possible with this technique:
let ls = [ "dd"; "a"; "b"; "c"; "aa"; "bb"; "cc"]
ls |> List.sortWith(desc Seq.length &>
asc id)
// result = ["aa"; "bb"; "cc"; "dd"; "a"; "b"; "c"]
ls |> List.sortWith( asc Seq.length &>
desc id)
// result = ["c"; "b"; "a"; "dd"; "cc"; "bb"; "aa"]
Your example would look like this:
Data |> List.sortWith( desc (fun t -> t.Date) &>
asc (fun t -> t.Value))

Query Expressions and Lazy Evaluation

I am hoping to understand how query expressions are really evaluated. I have a situation where I'm using a query expression to access a large amount of data from a database. I then interact with this data via a GUI. For example the user might supply an additive factor that I want to apply to one column and then plot. What I'm not clear on is how to structure this so that the same data isn't being pulled from the database each time the GUI updates.
For example:
let a state= query{...}
let results = a "ALASKA"
let calcoutput y = results |> Seq.map (fun x -> x.Temperature + y)
or
let calcoutput state y = a state |> Seq.map (fun x -> x.Temperature + y)
I am not clear if these are actually the same code, and if so am I pulling data from the DB each time I execute calcoutput with a different y (it appears so). Should I be casting the "results" sequence as a List and then using that to avoid this?
You can use Seq.cache function.
http://msdn.microsoft.com/en-us/library/ee370430.aspx
Quote: "This result sequence will have the same elements as the input sequence. The result can be enumerated multiple times. The input sequence is enumerated at most once and only as far as is necessary. Caching a sequence is typically useful when repeatedly evaluating items in the original sequence is computationally expensive or if iterating the sequence causes side-effects that the user does not want to be repeated multiple times."

F# Sequences - how to return multiple rows

let x = [for p in db.ParamsActXes do
if p.NumGroupPar = grp then
yield p.Num, p.Name]
Here is my sequence but the problem is it returns the list of tuples, I can't access single tuple element like
let str = "hello" + x.[1]
and that is the trouble.
how can I realize this functionary ?
To access the second element of a 2-tuple you can either use snd or pattern matching. So if tup is a 2-tuple, where the second element of tup is a string, you can either do:
let str = "hello" + snd tup
or
let (a,b) = tup
let st = "hello" + b
Note that snd only works with 2-tuples, not tuples with more than two elements.
To give you one more alternative solution, you can just create a filtered sequence containing values of the original type by writing yield p:
let x = [ for p in db.ParamsActXes do
if p.NumGroupPar = grp then
yield p ]
And then just access the property you need:
let str = "hello" + x.[1].Name
This is probably a good idea if you're returning only several properties of the p value. The only reason for yielding tuples would be to hide something from the rest of the code or to create sequence that matches some function you use later.
(Also, I would avoid indexing into lists using x.[i], because this is inefficient - but maybe this is just for ilustration in the sample you posted. Use an array if you need index based access by wrapping sequence expression into [| ... |])
Just to throw one more possibility out there, if you are running the .NET 4.0 build of F# 2.0 you can perform a runtime cast from the F# tuple to the .NET 4.0 System.Tuple and then use the ItemX properties of the .NET 4.0 tuples to access the tuple element you need,
let x = (1, 1.2, "hello")
let y = ((box x) :?> System.Tuple<int, float, string>);;
y.Item3 //returns "hello"
However, I would never use that, instead opting for the pattern match extraction. (also, I've read places that the F# compiler may not always choose to represent its tuples as .NET 4.0 tuples, so there may be a possibility that the cast would fail).
Reading your comments in some of the other answers, I'm unsure why the pattern matching solution doesn't work for you. Perhaps you want to access the tuple item at a certain place within an expression? If so, the previous would certainly work:
let str = "hello" + ((box x.[1]) :?> System.Tuple<int,string>).Item2 //though might as well use snd and fst for length 2 F# tuples
but you can achieve the same ends using the pattern matching extraction technique too (again, assuming this is even what you're after):
let str = "hello" + (let (_,name) = x.[1] in name)
you can access individual tuple from the list of tuple using List.nth.
let first, second = List.nth x 0
first and second represents the individual element of the tuple.

Useful F# Scripts

I have been investigating the use of F# for development and have found (for my situations) building scripts to help me simplify some complex tasks is where I can get value from it (at the moment).
My most common complex task is concatenating files for many tasks (mostly SQL related).
I do this often and every time I try to improve on my F# script to do this.
This is my best effort so far:
open System.IO
let path = "C:\\FSharp\\"
let pattern = "*.txt"
let out_path = path + "concat.out"
File.Delete(out_path)
Directory.GetFiles(path, pattern)
|> Array.collect (fun file -> File.ReadAllLines(file))
|> (fun content -> File.WriteAllLines(out_path, content) )
I'm sure others have scripts which makes their sometimes complex/boring tasks easier.
What F# scripts have you used to do this or what other purposes for F# scripts have you found useful?
I found the best way for me to improve my F# was to browse other scripts to get ideas on how to tackle specific situations. Hopefully this question will help me and others in the future. :)
I have found an article on generating F# scripts that may be of interest:
http://blogs.msdn.com/chrsmith/archive/2008/09/12/scripting-in-f.aspx
I use F# in a similar way when I need to quickly pre-process some data or convert data between various formats. F# has a great advantage that you can create higher-order functions for doing all sorts of similar tasks.
For example, I needed to load some data from SQL database and generate Matlab script files that load the data. I needed to do this for a couple of different SQL queries, so I wrote these two functions:
// Runs the specified query 'str' and reads results using 'f'
let query str f = seq {
let conn = new SqlConnection("<conn.str>");
let cmd = new SqlCommand(str, conn)
conn.Open()
use rdr = cmd.ExecuteReader(CommandBehavior.CloseConnection)
while rdr.Read() do yield f(rdr) }
// Simple function to save all data to the specified file
let save file data =
File.WriteAllLines(#"C:\...\" + file, data |> Array.ofSeq)
Now I could easily write specific calls to read the data I need, convert them to F# data types, does some pre-processing (if needed) and print the outputs to a file. For example for processing companies I had something like:
let comps =
query "SELECT [ID], [Name] FROM [Companies] ORDER BY [ID]"
(fun rdr -> rdr.GetString(1) )
let cdata =
seq { yield "function res = companies()"
yield " res = {"
for name in comps do yield sprintf " %s" name
yield " };"
yield "end" }
save "companies.m" cdata
Generating output as a sequence of strings is also pretty neat, though you could probably write a more efficient computation builder using StringBuilder.
Another example of using F# in the interactive way is described in my functional programming book in Chapter 13 (you can get the source code here). It connects to the World Bank database (which contains a lots of information about various countries), extracts some data, explores the structure of the data, convert them to F# data types and calculates some results (and visualizes them). I think this is (one of many) kinds of tasks that can be very nicely done in F#.
Sometimes if I want a brief of an XML structure (or have a recursive list to use in other forms such as searches), I can print out a tabbed list of nodes in the XML using the following script:
open System
open System.Xml
let path = "C:\\XML\\"
let xml_file = path + "Test.xml"
let print_element level (node:XmlNode) = [ for tabs in 1..level -> " " ] # [node.Name]
|> (String.concat "" >> printfn "%s")
let rec print_tree level (element:XmlNode) =
element
|> print_element level
|> (fun _ -> [ for child in element.ChildNodes -> print_tree (level+1) child ])
|> ignore
new XmlDocument()
|> (fun doc -> doc.Load(xml_file); doc)
|> (fun doc -> print_tree 0 doc.DocumentElement)
I am sure it can be optimised/cut down and would encourage by others' improvements on this code. :)
(For an alternative snippet see the answer below.)
This snippet transforms an XML using an XSLT. I wasn't sure of hte best way to use the XslCompiledTransform and XmlDocument objects the best in F#, but it seemed to work. I am sure there are better ways and would be happy to hear about them.
(* Transforms an XML document given an XSLT. *)
open System.IO
open System.Text
open System.Xml
open System.Xml.Xsl
let path = "C:\\XSL\\"
let file_xml = path + "test.xml"
let file_xsl = path + "xml-to-xhtml.xsl"
(* Compile XSL file to allow transforms *)
let compile_xsl (xsl_file:string) = new XslCompiledTransform() |> (fun compiled -> compiled.Load(xsl_file); compiled)
let load_xml (xml_file:string) = new XmlDocument() |> (fun doc -> doc.Load(xml_file); doc)
(* Transform an Xml document given an XSL (compiled *)
let transform (xsl_file:string) (xml_file:string) =
new MemoryStream()
|> (fun mem -> (compile_xsl xsl_file).Transform((load_xml xml_file), new XmlTextWriter(mem, Encoding.UTF8)); mem)
|> (fun mem -> mem.Position <- (int64)0; mem.ToArray())
(* Return an Xml fo document that has been transformed *)
transform file_xsl file_xml
|> (fun bytes -> File.WriteAllBytes(path + "out.html", bytes))
After clarifying approaches to writing F# code with existing .net classes, the following useful code came up for transforming xml documents given xsl documents. The function also allows you to create a custom function to transform xml documents with a specific xsl document (see example):
let transform =
(fun xsl ->
let xsl_doc = new XslCompiledTransform()
xsl_doc.Load(string xsl)
(fun xml ->
let doc = new XmlDocument()
doc.Load(string xml)
let mem = new MemoryStream()
xsl_doc.Transform(doc.CreateNavigator(), null, mem)
mem
)
)
This allows you to transform docs this way:
let result = transform "report.xml" "report.xsl"
or you can create another function which can be used multiple times:
let transform_report "report.xsl"
let reports = [| "report1.xml"; "report2.xml" |]
let results = [ for report in reports do transform_report report ]

Resources