I just finish writing my first F# program. Functionality wise the code works the way I wanted, but not sure if the code is efficient. I would much appreciate if someone could review the code for me and point out the areas where the code can be improved.
Thanks
Sudaly
open System
open System.IO
open System.IO.Pipes
open System.Text
open System.Collections.Generic
open System.Runtime.Serialization
[<DataContract>]
type Quote = {
[<field: DataMember(Name="securityIdentifier") >]
RicCode:string
[<field: DataMember(Name="madeOn") >]
MadeOn:DateTime
[<field: DataMember(Name="closePrice") >]
Price:float
}
let m_cache = new Dictionary<string, Quote>()
let ParseQuoteString (quoteString:string) =
let data = Encoding.Unicode.GetBytes(quoteString)
let stream = new MemoryStream()
stream.Write(data, 0, data.Length);
stream.Position <- 0L
let ser = Json.DataContractJsonSerializer(typeof<Quote array>)
let results:Quote array = ser.ReadObject(stream) :?> Quote array
results
let RefreshCache quoteList =
m_cache.Clear()
quoteList |> Array.iter(fun result->m_cache.Add(result.RicCode, result))
let EstablishConnection() =
let pipeServer = new NamedPipeServerStream("testpipe", PipeDirection.InOut, 4)
let mutable sr = null
printfn "[F#] NamedPipeServerStream thread created, Wait for a client to connect"
pipeServer.WaitForConnection()
printfn "[F#] Client connected."
try
// Stream for the request.
sr <- new StreamReader(pipeServer)
with
| _ as e -> printfn "[F#]ERROR: %s" e.Message
sr
while true do
let sr = EstablishConnection()
// Read request from the stream.
printfn "[F#] Ready to Receive data"
sr.ReadLine()
|> ParseQuoteString
|> RefreshCache
printfn "[F#]Quot Size, %d" m_cache.Count
let quot = m_cache.["MSFT.OQ"]
printfn "[F#]RIC: %s" quot.RicCode
printfn "[F#]MadeOn: %s" (String.Format("{0:T}",quot.MadeOn))
printfn "[F#]Price: %f" quot.Price
In general, you should try using immutable data types and avoid imperative constructs such as global variables and imperative loops - although using them in F# is fine in many cases, they should be used only when there is a good reason for doing so. Here are a couple of examples where you could use functional approach:
First of all, to make the code more functional, you should avoid using global mutable cache. Instead, your RefreshCache function should return the data as the result (preferably using some functional data structure, such as F# Map type):
let PopulateCache quoteList =
quoteList
// Generate a sequence of tuples containing key and value
|> Seq.map (fun result -> result.RicCode, result)
// Turn the sequence into an F# immutable map (replacement for hashtable)
|> Map.ofSeq
The code that uses it would be changed like this:
let cache =
sr.ReadLine()
|> ParseQuoteString
|> PopulateCache
printfn "[F#]Quot Size, %d" m_cache.Count
let quot = m_cache.["MSFT.OQ"]
// The rest of the sample stays the same
In the EstablishConnection function, you definitely don't need to declare a mutable variable sr, because in case of an exception, the function will return null. I would instead use option type to make sure that this case is handled:
let EstablishConnection() =
let pipeServer =
new NamedPipeServerStream("testpipe", PipeDirection.InOut, 4)
printfn "[F#] NamedPipeServerStream thread created..."
pipeServer.WaitForConnection()
printfn "[F#] Client connected."
try // Wrap the result in 'Some' to denote success
Some(new StreamReader(pipeServer))
with e ->
printfn "[F#]ERROR: %s" e.Message
// Return 'None' to denote a failure
None
The main loop can be written using a recursive function that stops when EstablishConnection fails:
let rec loop() =
match EstablishConnection() with
| Some(conn) ->
printfn "[F#] Ready to Receive data"
// rest of the code
loop() // continue looping
| _ -> () // Quit
Just a couple thoughts...
You probably want a 'use' rather than a 'let' in a few places, as I think some of the objects in the program are IDisposable.
You may consider wrapping the EstablishConnection method and the final while loop in async blocks (and make other minor changes), so that e.g. you can wait asynchronously for connections without blocking a thread.
At first glance it is written in imperative style rather than functional style, which does make sense given that most of the program involves side effects (i.e. I/O). Line for line, it almost looks like a C# program.
Given the amount of I/O that is taking place, I don't know that there is much you can do to this particular program to make it more of a functional style of coding.
Related
Anyone have a decent example, preferably practical/useful, they could post demonstrating the concept?
I came across this term somewhere that I’m unable to find, probably it has to do something with a function returning a function while enclosing on some mutable variable. So there’s no visible mutation.
Probably Haskell community has originated the idea where mutation happens in another area not visible to the scope. I maybe vague here so seeking help to understand more.
It's a good idea to hide mutation, so the consumers of the API won't inadvartently change something unexpectedly. This just means that you have to encapsulate your mutable data/state. This can be done via objects (yes, objects), but what you are referring to in your question can be done with a closure, the canonical example is a counter:
let countUp =
let mutable count = 0
(fun () -> count <- count + 1
count)
countUp() // 1
countUp() // 2
countUp() // 3
You cannot access the mutable count variable directly.
Another example would be using mutable state within a function so that you cannot observe it, and the function is, for all intents and purposes, referentially transparent. Take for example the following function that reverses a string not character-wise, but rather by taking individual text elements (which, depending on language, can be more than one character):
let reverseStringU s =
if Core.string.IsNullOrEmpty s then s else
let rec iter acc (ee : System.Globalization.TextElementEnumerator) =
if not <| ee.MoveNext () then acc else
let e = ee.GetTextElement ()
iter (e :: acc) ee
let inline append x s = (^s : (member Append : ^x -> ^s) (s, x))
let sb = System.Text.StringBuilder s.Length
System.Globalization.StringInfo.GetTextElementEnumerator s
|> iter []
|> List.fold (fun a e -> append e a) sb
|> string
It uses a StringBuilder internally but you cannot observe this externally.
I am a little curious about the code example below and what people think.
The idea was to read from a NetworkStream (~20 msg/s) and instead of working in the main, pass things to MainboxProcessor to handle and get things back for bindings when done.
The usual way is to use PostAndReply, but I want to bind to ListView or other control in C#. Must do magic with LastN items and filtering anyway.
Plus, Rx has some error handling.
The example below observes numbers from 2..10 and returns "hello X". On 8 it stops like it was EOF. Made it to ToEnumerable because other thread finishes before otherwise, but it works with Subscribe as well.
What bothers me:
passing Subject(obj) around in recursion. I don't see any problems having around 3-4 of those. Good idea?
Lifetime of Subject.
open System
open System.Threading
open System.Reactive.Subjects
open System.Reactive.Linq // NuGet, take System.Reactive.Core also.
open System.Reactive.Concurrency
type SerializedLogger() =
let _letters = new Subject<string>()
// create the mailbox processor
let agent = MailboxProcessor.Start(fun inbox ->
// the message processing function
let rec messageLoop (letters:Subject<string>) = async{
// read a message
let! msg = inbox.Receive()
printfn "mailbox: %d in Thread: %d" msg Thread.CurrentThread.ManagedThreadId
do! Async.Sleep 100
// write it to the log
match msg with
| 8 -> letters.OnCompleted() // like EOF.
| x -> letters.OnNext(sprintf "hello %d" x)
// loop to top
return! messageLoop letters
}
// start the loop
messageLoop _letters
)
// public interface
member this.Log msg = agent.Post msg
member this.Getletters() = _letters.AsObservable()
/// Print line with prefix 1.
let myPrint1 x = printfn "onNext - %s, Thread: %d" x Thread.CurrentThread.ManagedThreadId
// Actions
let onNext = new Action<string>(myPrint1)
let onCompleted = new Action(fun _ -> printfn "Complete")
[<EntryPoint>]
let main argv =
async{
printfn "Main is on: %d" Thread.CurrentThread.ManagedThreadId
// test
let logger = SerializedLogger()
logger.Log 1 // ignored?
let xObs = logger
.Getletters() //.Where( fun x -> x <> "hello 5")
.SubscribeOn(Scheduler.CurrentThread)
.ObserveOn(Scheduler.CurrentThread)
.ToEnumerable() // this
//.Subscribe(onNext, onCompleted) // or with Dispose()
[2..10] |> Seq.iter (logger.Log)
xObs |> Seq.iter myPrint1
while true
do
printfn "waiting"
System.Threading.Thread.Sleep(1000)
return 0
} |> Async.RunSynchronously // return an integer exit code
I have done similar things, but using the plain F# Event type rather than Subject. It basically lets you create IObservable and trigger its subscribes - much like your use of more complex Subject. The event-based version would be:
type SerializedLogger() =
let letterProduced = new Event<string>()
let lettersEnded = new Event<unit>()
let agent = MailboxProcessor.Start(fun inbox ->
let rec messageLoop (letters:Subject<string>) = async {
// Some code omitted
match msg with
| 8 -> lettersEnded.Trigger()
| x -> letterProduced.Trigger(sprintf "hello %d" x)
// ...
member this.Log msg = agent.Post msg
member this.LetterProduced = letterProduced.Publish
member this.LettersEnded = lettersEnded.Publish
The important differences are:
Event cannot trigger OnCompleted, so I instead exposed two separate events. This is quite unfortunate! Given that Subject is very similar to events in all other aspects, this might be a good reason for using subject instead of plain event.
The nice aspect of using Event is that it is a standard F# type, so you do not need any external dependencies in the agent.
I noticed your comment noting that the first call to Log was ignored. That's because you subscribe to the event handler only after this call happens. I think you could use ReplaySubject variation on the Subject idea here - it replays all events when you subscribe to it, so the one that happened earlier would not be lost (but there is a cost to caching).
In summary, I think using Subject is probably a good idea - it is essentially the same pattern as using Event (which I think is quite standard way of exposing notifications from agents), but it lets you trigger OnCompleted. I would probably not use ReplaySubject, because of the caching cost - you just have to make sure to subscribe before triggering any events.
Does anyone know of 'prior art' regarding the following subject :
I have data that take some decent time to load. they are historical level for various stocks.
I would like to preload them somehow, to avoid the latency when using my app
However, preloading them in one chunk at start makes my app unresponsive first which is not user friendly
So I would like to not load my data.... unless the user is not requesting any and playing with what he already has, in which case I would like to get little by little. So it is neither 'lazy' nor 'eager', more 'lazy when you need' and 'eager when you can', hence the acronym LWYNEWYC.
I have made the following which seems to work, but I just wonder if there is a recognized and blessed approach for such thing ?
let r = LoggingFakeRepo () :> IQuoteRepository
r.getHisto "1" |> ignore //prints Getting histo for 1 when called
let rc = RepoCached (r) :> IQuoteRepository
rc.getHisto "1" |> ignore //prints Getting histo for 1 the first time only
let rcc = RepoCachedEager (r) :> IQuoteRepository
rcc.getHisto "100" |> ignore //prints Getting histo 1..100 by itself BUT
//prints Getting histo 100 immediately when called
And the classes
type IQuoteRepository =
abstract getUnderlyings : string seq
abstract getHisto : string -> string
type LoggingFakeRepo () =
interface IQuoteRepository with
member x.getUnderlyings = printfn "getting underlyings"
[1 .. 100] |> List.map string :> _
member x.getHisto udl = printfn "getting histo for %A" udl
"I am a historical dataset in a disguised party"
type RepoCached (rep : IQuoteRepository) =
let memoize f =
let cache = new System.Collections.Generic.Dictionary<_, _>()
fun x ->
if cache.ContainsKey(x) then cache.[x]
else let res = f x
cache.[x] <- res
res
let udls = lazy (rep.getUnderlyings )
let gethistom = memoize rep.getHisto
interface IQuoteRepository with
member x.getUnderlyings = udls.Force()
member x.getHisto udl = gethistom udl
type Message = string * AsyncReplyChannel<UnderlyingWrap>
type RepoCachedEager (rep : IQuoteRepository) =
let udls = rep.getUnderlyings
let agent = MailboxProcessor<Message>.Start(fun inbox ->
let repocached = RepoCached (rep) :> IQuoteRepository
let rec loop l =
async { try
let timeout = if l|> List.isEmpty then -1 else 50
let! (udl, replyChannel) = inbox.Receive(timeout)
replyChannel.Reply(repocached.getHisto udl)
do! loop l
with
| :? System.TimeoutException ->
let udl::xs = l
repocached.getHisto udl |> ignore
do! loop xs
}
loop (udls |> Seq.toList))
interface IQuoteRepository with
member x.getUnderlyings = udls
member x.getHisto udl = agent.PostAndReply(fun reply -> udl, reply)
I like your solution. I think using agent to implement some background loading with a timeout is a great way to go - agents can nicely encapsulate mutable state, so it is clearly safe and you can encode the behaviour you want quite easily.
I think asynchronous sequences might be another useful abstraction (if I'm correct, they are available in FSharpX these days). An asynchronous sequence represents a computation that asynchronously produces more values, so they might be a good way to separate the data loader from the rest of the code.
I think you'll still need an agent to synchronize at some point, but you can nicely separate different concerns using async sequences.
The code to load the data might look something like this:
let loadStockPrices repo = asyncSeq {
// TODO: Not sure how you detect that the repository has no more data...
while true do
// Get next item from the repository, preferably asynchronously!
let! data = repo.AsyncGetNextHistoricalValue()
// Return the value to the caller...
yield data }
This code represents the data loader, and it separates it from the code that uses it. From the agent that consumes the data source, you can use AsyncSeq.iterAsync to consume the values and do something with them.
With iterAsync, the function that you specify as a consumer is asynchronous. It may block (i.e. using Sleep) and when it blocks, the source - that is.your loader - is also blocked. This is quite nice implicit way to control the loader from the code that consumes the data.
A feature that is not in the library yet (but would be useful) is an partially eager evaluator that takes AsyncSeq<'T> and returns a new AsyncSeq<'T> but obtains a certain number of elements from the source as soon as possible and caches them (so that the consumer does not have to wait when it asks for a value, as long as the source can produce values fast enough).
I wrote the following code to execute a SQLServer StoredProc in F#
module SqlUtility =
open System
open System.Data
open System.Data.SqlClient
SqlUtility.GetSqlConnection "MyDB"
|> Option.bind (fun con -> SqlUtility.GetSqlCommand "dbo.usp_MyStordProc" con)
|> Option.bind (fun cmd ->
let param1 = new SqlParameter("#User", SqlDbType.NVarChar, 50)
param1.Value <- user
cmd.Parameters.Add(param1) |> ignore
let param2 = new SqlParameter("#PolicyName", SqlDbType.NVarChar, 10)
param2.Value <- policyName
cmd.Parameters.Add(param2) |> ignore
Some(cmd)
)
|> Option.bind (fun cmd -> SqlUtility.ExecuteReader cmd)
|> Option.bind (fun rdr -> ExtractValue rdr)
let GetSqlConnection (conName : string) =
let conStr = ConfigHandler.GetConnectionString conName
try
let con = new SqlConnection(conStr)
con.Open()
Some(con)
with
| :? System.Exception as ex -> printfn "Failed to connect to DB %s with Error %s " conName ex.Message; None
| _ -> printfn "Failed to connect to DB %s" conName; None
let GetSqlCommand (spName : string) (con : SqlConnection) =
let cmd = new SqlCommand()
cmd.Connection <- con
cmd.CommandText <- spName
cmd.CommandType <- CommandType.StoredProcedure
Some(cmd)
let AddParameters (cmd : SqlCommand) (paramList : SqlParameter list) =
paramList |> List.iter (fun p -> cmd.Parameters.Add p |> ignore)
let ExecuteReader (cmd : SqlCommand ) =
try
Some(cmd.ExecuteReader())
with
| :? System.Exception as ex -> printfn "Failed to execute reader with error %s" ex.Message; None
I have multiple problems with this code
First and foremost the repeated use of Option.bind is very irritating... and is adding noise. I need a more clearer way to check if the output was None and if not then proceed.
At the end there should be a cleanupfunction where I should be able to close + dispose the reader, command and connection. But currently at the end of the pipeline all I have is the reader.
The function which is adding parameters... it looks like it is modifying the "state" of the command parameter because the return type is still the same command which was sent it... with some added state. I wonder how a more experienced functional programmer would have done this.
Visual Studio gives me a warning at each of the place where i do exception handling. what's wrong with that" it says
This type test or downcast will always hold
The way I want this code to look is this
let x : MyRecord seq = GetConnection "con" |> GetCommand "cmd" |> AddParameter "#name" SqlDbType.NVarchar 50 |> AddParameter "#policyname" SqlDbType.NVarchar 50 |> ExecuteReader |> FunctionToReadAndGenerateSeq |> CleanEverything
Can you recommend how can I take my code to the desired level and also any other improvement?
I think that using options to represent failed computations is more suitable to purely functional langauges. In F#, it is perfectly fine to use exceptions to denote that a computation has failed.
Your code simply turns exceptions into None values, but it does not really handle this situation - this is left to the caller of your code (who will need to decide what to do with None). You may as well just let them handle the exception. If you want to add more information to the exception, you can define your own exception type and throw that instead of leaving the standard exceptions.
The following defines a new exception type and a simple function to throw it:
exception SqlUtilException of string
// This supports the 'printf' formatting style
let raiseSql fmt =
Printf.kprintf (SqlUtilException >> raise) fmt
Using plain .NET style with a few simplifications using F# features, the code looks a lot simpler:
// Using 'use' the 'Dispose' method is called automatically
let connName = ConfigHandler.GetConnectionString "MyDB"
use conn = new SqlConnection(connName)
// Handle exceptions that happen when opening the connection
try conn.Open()
with ex -> raiseSql "Failed to connect to DB %s with Error %s " connName ex.Message
// Using object initializer, we can nicely set the properties
use cmd =
new SqlCommand( Connection = conn, CommandText = "dbo.usp_MyStordProc",
CommandType = CommandType.StoredProcedure )
// Add parameters
// (BTW: I do not think you need to set the type - this will be infered)
let param1 = new SqlParameter("#User", SqlDbType.NVarChar, 50, Value = user)
let param2 = new SqlParameter("#PolicyName", SqlDbType.NVarChar, 10, Value = policyName)
cmd.Parameters.AddRange [| param1; param2 |]
use reader =
try cmd.ExecuteReader()
with ex -> raiseSql "Failed to execute reader with error %s" ex.Message
// Do more with the reader
()
It looks more like .NET code, but that is perfectly fine. Dealing with databases in F# is going to use imperative style and trying to hide that will only make the code confusing. Now, there is a number of other neat F# features you could use - especially the support for dynamic operators ?, which would give you something like:
let connName = ConfigHandler.GetConnectionString "MyDB"
// A wrapper that provides dynamic access to database
use db = new DynamicDatabase(connName)
// You can call stored procedures using method call syntax
// and pass SQL parameters as standard arguments
let rows = db.Query?usp_MyStordProc(user, policy)
// You can access columns using the '?' syntax again
[ for row in rows -> row?Column1, row?Column2 ]
For more information about this, see the following MSDN series:
How to: Dynamically Invoke a Stored Procedure
Step 1: Create a Database and Show the Poll Options
Step 2: Implement Voting for an Option
Is there a way to creating an observable from a sequence in F#?
The required behaviour is that an observer subscribing to the resulting observable receives all the values of the sequence one at a time.
Edit: The question can be framed as: Is there an equivalent to Rx.Observable.FromArray([1,2,3]) as shown here in F#?
Edit 2: Thanks everyone. It took me some time to find the answer, only to find that desco has answered already. For the sake of completeness, here is how it works:
//Using the Reactive Framework.
//References to System.CoreEx.dll and System.Reactive.dll are needed.
open System.Linq
let observable = Observable.ToObservable [1..10]
let odds = observable |> Observable.filter (fun i -> i%2=1)
let disp = odds.Subscribe (fun i -> printfn "%d" i)
disp.Dispose ()
Just in case you wanted to avoid the additional dependency, it is actually quite easy to implement the behavior using F# events and Observable.guard function that I already used in another answer here at StackOverflow.
Here is the implementation together with a brief example:
module Observable =
let ofSeq s =
let evt = new Event<_>()
evt.Publish |> Observable.guard (fun o ->
for n in s do evt.Trigger(n))
[ 1 .. 10 ] |> Observable.ofSeq
|> Observable.filter (fun n -> n%2 = 0)
|> Observable.add (printfn "%d")
And for completeness, the implementation of Observable.guard looks like this:
/// Creates an observable that calls the specified function after someone
/// subscribes to it (useful for waiting using 'let!' when we need to start
/// operation after 'let!' attaches handler)
let guard f (e:IObservable<'Args>) =
{ new IObservable<'Args> with
member x.Subscribe(observer) =
let rm = e.Subscribe(observer) in f(); rm }
Is System.Reactive.Linq.Observable.ToObservable(this IEnumerable source) located in System.Reactive.dll the thing your wanted?
Yoou can try Reactive Extensions. Please refer to the following blog entry which might help you a lot.
http://www.navision-blog.de/2009/10/20/iobservableiobserver-using-the-reactive-framework-with-f/