Useful F# Scripts - f#

Question

I have been investigating the use of F# for development and have found (for my situations) building scripts to help me simplify some complex tasks is where I can get value from it (at the moment).
My most common complex task is concatenating files for many tasks (mostly SQL related).
I do this often and every time I try to improve on my F# script to do this.
This is my best effort so far:
open System.IO
let path = "C:\\FSharp\\"
let pattern = "*.txt"
let out_path = path + "concat.out"
File.Delete(out_path)
Directory.GetFiles(path, pattern)
|> Array.collect (fun file -> File.ReadAllLines(file))
|> (fun content -> File.WriteAllLines(out_path, content) )
I'm sure others have scripts which makes their sometimes complex/boring tasks easier.
What F# scripts have you used to do this or what other purposes for F# scripts have you found useful?
I found the best way for me to improve my F# was to browse other scripts to get ideas on how to tackle specific situations. Hopefully this question will help me and others in the future. :)
I have found an article on generating F# scripts that may be of interest:
http://blogs.msdn.com/chrsmith/archive/2008/09/12/scripting-in-f.aspx

Answer 1

I use F# in a similar way when I need to quickly pre-process some data or convert data between various formats. F# has a great advantage that you can create higher-order functions for doing all sorts of similar tasks.
For example, I needed to load some data from SQL database and generate Matlab script files that load the data. I needed to do this for a couple of different SQL queries, so I wrote these two functions:
// Runs the specified query 'str' and reads results using 'f'
let query str f = seq {
let conn = new SqlConnection("<conn.str>");
let cmd = new SqlCommand(str, conn)
conn.Open()
use rdr = cmd.ExecuteReader(CommandBehavior.CloseConnection)
while rdr.Read() do yield f(rdr) }
// Simple function to save all data to the specified file
let save file data =
File.WriteAllLines(#"C:\...\" + file, data |> Array.ofSeq)
Now I could easily write specific calls to read the data I need, convert them to F# data types, does some pre-processing (if needed) and print the outputs to a file. For example for processing companies I had something like:
let comps =
query "SELECT [ID], [Name] FROM [Companies] ORDER BY [ID]"
(fun rdr -> rdr.GetString(1) )
let cdata =
seq { yield "function res = companies()"
yield " res = {"
for name in comps do yield sprintf " %s" name
yield " };"
yield "end" }
save "companies.m" cdata
Generating output as a sequence of strings is also pretty neat, though you could probably write a more efficient computation builder using StringBuilder.
Another example of using F# in the interactive way is described in my functional programming book in Chapter 13 (you can get the source code here). It connects to the World Bank database (which contains a lots of information about various countries), extracts some data, explores the structure of the data, convert them to F# data types and calculates some results (and visualizes them). I think this is (one of many) kinds of tasks that can be very nicely done in F#.

Answer 2

Sometimes if I want a brief of an XML structure (or have a recursive list to use in other forms such as searches), I can print out a tabbed list of nodes in the XML using the following script:
open System
open System.Xml
let path = "C:\\XML\\"
let xml_file = path + "Test.xml"
let print_element level (node:XmlNode) = [ for tabs in 1..level -> " " ] # [node.Name]
|> (String.concat "" >> printfn "%s")
let rec print_tree level (element:XmlNode) =
element
|> print_element level
|> (fun _ -> [ for child in element.ChildNodes -> print_tree (level+1) child ])
|> ignore
new XmlDocument()
|> (fun doc -> doc.Load(xml_file); doc)
|> (fun doc -> print_tree 0 doc.DocumentElement)
I am sure it can be optimised/cut down and would encourage by others' improvements on this code. :)

Answer 3

(For an alternative snippet see the answer below.)
This snippet transforms an XML using an XSLT. I wasn't sure of hte best way to use the XslCompiledTransform and XmlDocument objects the best in F#, but it seemed to work. I am sure there are better ways and would be happy to hear about them.
(* Transforms an XML document given an XSLT. *)
open System.IO
open System.Text
open System.Xml
open System.Xml.Xsl
let path = "C:\\XSL\\"
let file_xml = path + "test.xml"
let file_xsl = path + "xml-to-xhtml.xsl"
(* Compile XSL file to allow transforms *)
let compile_xsl (xsl_file:string) = new XslCompiledTransform() |> (fun compiled -> compiled.Load(xsl_file); compiled)
let load_xml (xml_file:string) = new XmlDocument() |> (fun doc -> doc.Load(xml_file); doc)
(* Transform an Xml document given an XSL (compiled *)
let transform (xsl_file:string) (xml_file:string) =
new MemoryStream()
|> (fun mem -> (compile_xsl xsl_file).Transform((load_xml xml_file), new XmlTextWriter(mem, Encoding.UTF8)); mem)
|> (fun mem -> mem.Position <- (int64)0; mem.ToArray())
(* Return an Xml fo document that has been transformed *)
transform file_xsl file_xml
|> (fun bytes -> File.WriteAllBytes(path + "out.html", bytes))

Answer 4

After clarifying approaches to writing F# code with existing .net classes, the following useful code came up for transforming xml documents given xsl documents. The function also allows you to create a custom function to transform xml documents with a specific xsl document (see example):
let transform =
(fun xsl ->
let xsl_doc = new XslCompiledTransform()
xsl_doc.Load(string xsl)
(fun xml ->
let doc = new XmlDocument()
doc.Load(string xml)
let mem = new MemoryStream()
xsl_doc.Transform(doc.CreateNavigator(), null, mem)
mem
)
)
This allows you to transform docs this way:
let result = transform "report.xml" "report.xsl"
or you can create another function which can be used multiple times:
let transform_report "report.xsl"
let reports = [| "report1.xml"; "report2.xml" |]
let results = [ for report in reports do transform_report report ]