I am doing a direct combination of 2 functions using a lambda function. Because indentation is a problem I am doing direct copy/paste from Visual Code:
version 1 of the function is:
module myrun =
...
let mywrite(price: string, volume: string, time:string) =
use cmd99 = new SqlCommandProvider<"INSERT INTO swirepacific(ric, price, volume, inst, ttime, views, strike, etime) VALUES (#st1, #st2, #st3, #st4, #st5, #st6, #st7, #st8)", connStr>(connStr)
cmd99.Execute(st1=myric,st2=float price,st3=int volume,st4=inst,st5=System.DateTime.Parse time, st6 = myview, st7 = strike, st8=System.DateTime.Parse etime)
|> ignore
let toDB (data:seq<IData>) =
data
|> Seq.map (fun (r:IData) -> (r.AllFields.["VALUE"].ToString(),r.AllFields.["VOLUME"].ToString()
,r.Timestamp.Value.ToString("yyyy-MM-dd hh:mm:ss.fff")))
|> Seq.iter mywrite
Version 1 is a write to database code. It uses SqlCommandProvider to parse a SQL statement directly.
The problem with version 1 is that while it accepts price, volume and time from previous functions, the myric, inst, strike and etime are Global Variables.
With version 2, I want to skip global variables completely and take all inputs as parameters. It should be a better practice and pretty much required if I want to run the functions in parallel:
Version 2 of the function is
module myrun =
...
let toDB2 (data:seq<IData>, aric: Aric, ainst: Ainst, astrike: Astrike, aetime: Aetime) =
data
|> Seq.map (fun (r:IData) -> (r.AllFields.["VALUE"].ToString(),r.AllFields.["VOLUME"].ToString(),r.Timestamp.Value.ToString("yyyy-MM-dd hh:mm:ss.fff")))
|> Seq.iter (fun (price, volume, time) ->
(use cmd99 = new SqlCommandProvider<"INSERT INTO swirepacific(ric, price, volume, inst, ttime, views, strike, etime) VALUES (#st1, #st2, #st3, #st4, #st5, #st6, #st7, #st8)", connStr>(connStr)
cmd99.Execute(st1=aric,st2=float price,st3=int volume,st4=ainst,st5=System.DateTime.Parse time, st6 = myview, st7 = astrike, st8=System.DateTime.Parse aetime)
|> ignore)
There is a red line under the let in "let toDB2". The error is
Incomplete value or function definition. If this is an expression, the body of the expression must be indented to the same column as the 'let' keyword
I suspect it has something to do with the indentation of the lambda function.
Can you please help me? Thanks!
I think you're just missing a single closing parenthesis. I just added one at the end to close your 2nd lambda expression and the compiler is happy:
module myrun =
let toDB2 (data:seq<IData>, aric: Aric, ainst: Ainst, astrike: Astrike, aetime: Aetime) =
data
|> Seq.map (fun (r:IData) -> (r.AllFields.["VALUE"].ToString(),r.AllFields.["VOLUME"].ToString(),r.Timestamp.Value.ToString("yyyy-MM-dd hh:mm:ss.fff")))
|> Seq.iter (fun (price, volume, time) ->
(use cmd99 = new SqlCommandProvider<"INSERT INTO swirepacific(ric, price, volume, inst, ttime, views, strike, etime) VALUES (#st1, #st2, #st3, #st4, #st5, #st6, #st7, #st8)", connStr>(connStr)
cmd99.Execute(st1=aric,st2=float price,st3=int volume,st4=ainst,st5=System.DateTime.Parse time, st6 = myview, st7 = astrike, st8=System.DateTime.Parse aetime)
|> ignore))
The answer from #Aron shows how to correct your code without changing its structure. However, I think that there is not much benefit in using functions like map in your particular case.
You are essentially writing imperative bit of code that iterates over some data and writes it to a database. This performs IO and so it is inherently imperative - in such cases, I often find it easier to write a more imperative style of F# - the nice side effect of doing this is that anyone reading the code will immediately see that it is, in fact, imperative and will not confuse it with some functional data processing.
So, my version would actually remove all lambdas from this snippet:
module myrun =
[<Literal>]
let InsertQuery =
"INSERT INTO swirepacific(ric, price, volume, inst, ttime, views, strike, etime)
VALUES (#st1, #st2, #st3, #st4, #st5, #st6, #st7, #st8)"
let toDB2 (data:seq<IData>, aric: Aric, ainst: Ainst, astrike: Astrike, aetime: Aetime) =
for r in data do
let price = r.AllFields.["VALUE"].ToString()
let volume = r.AllFields.["VOLUME"].ToString()
let time = r.Timestamp.Value.ToString("yyyy-MM-dd hh:mm:ss.fff")
use cmd99 = new SqlCommandProvider<InsertQuery, connStr>(connStr)
cmd99.Execute
( st1=aric,st2=float price,st3=int volume,st4=ainst,st5=System.DateTime.Parse time,
st6 = myview, st7 = astrike, st8=System.DateTime.Parse aetime) |> ignore
Related
No problem here:
module Seq =
let private rnd = Random Environment.TickCount
let random =
fun (items : 'T seq) ->
let count = Seq.length items
items |> Seq.nth (rnd.Next count)
The signature of Seq.random is items:seq<'T> -> 'T. All good.
Yes, I know that I could just let random items = [...], that is not the point.
The point is that items is suddenly constrained to be type seq<obj> when I do this:
module Seq =
let random =
let rnd = Random Environment.TickCount
fun (items : 'T seq) ->
let count = Seq.length items
items |> Seq.nth (rnd.Next count)
... i.e. I add the Random object as a closure. If I hover over random, Intellisense shows me that the signature has become items:seq<obj> -> obj.
Interestingly, if I select the code and hit [Alt]+[Enter] to execute it in F# Interactive, the signature shows as seq<'a> -> 'a. WTH??
So, what's going on, here? Why the confusion and inconsistency in type inference?
This is due to the so-called Value Restriction. Cutting a long story short, syntactical values cannot be generic, because it might break things when mutations occur, and the compiler cannot always reliably prove immutability. (note that, even though random is a function semantically, it is still a value syntactically, and that's what matters)
But sometimes the compiler can prove immutability. This is why your first example works: when the right side of a let is a straight up lambda expression, the compiler can tell with certainty that it is immutable, and so it lets this pass.
Another example would be let x = [] - here the compiler can see that the nil list [] is immutable. On the other hand, let x = List.append [] [] won't work, because the compiler can't prove immutability in that case.
This "relaxation" of value restriction is done in F# on a case-by-case basis. F# compiler only goes as far as to handle a few special cases: literals, lambda expressions, etc., but it doesn't have a full-fledged mechanism for proving immutability in general. This is why, once you step outside of those special cases, you're not allowed to have generic values.
You can technically defeat this by adding explicit type arguments. Logically, this tells the compiler "Yes, I know it's a generic value, and that's what I meant for it to be".
let random<'t> : seq<'t> -> 't =
let rnd = Random Environment.TickCount
fun items ->
let count = Seq.length items
items |> Seq.nth (rnd.Next count)
let x = random [1;2;3]
But this will still not do what you want, because behind the scenes, such definition will be compiled to a parameterless generic method, and every time you reference such "value", the method will be called and return you a new function - with a brand new rnd baked in for every call. In other words, the above code will be equivalent to this:
let random() =
let rnd = Random Environment.TickCount
fun items ->
let count = Seq.length items
items |> Seq.nth (rnd.Next count)
let x = random() [1;2;3]
Why are parentheses needed on read_rest_of_csv below?
let read_rest_of_csv() =
csv_data.Add(csv_fileH.ReadFields()) |> ignore
not csv_fileH.EndOfData
while read_rest_of_csv() do ignore None
Without the parentheses, the loop will not terminate.
open System
open System.Threading
open System.Collections.Generic
open System.Linq
open System.Text
open System.Threading.Tasks
open System.IO
open Microsoft.VisualBasic.FileIO
[<EntryPoint>]
let main argv =
let csv_fileH = new TextFieldParser("test1.csv")
csv_fileH.TextFieldType = FieldType.Delimited |> ignore
let x = csv_fileH.SetDelimiters(",")
let csv_data = new List<string[]>()
let eod = csv_fileH.EndOfData
if not eod then
let column_headings = csv_fileH.ReadFields()
csv_data.Add(column_headings) |> ignore
let read_rest_of_csv =
csv_data.Add(csv_fileH.ReadFields()) |> ignore
not csv_fileH.EndOfData
while read_rest_of_csv do ignore None
0
I apologize that I cannot remember where I saw this. I think it was in SO. It's a nice example.
Could this be that without parens I'm dealing with a function object of sorts?
I am indeed coming from not only a C, C++, and C# background, but also an intermediate Clojure background as well. In my case with F# syntax, reading my Haskell manual in a little more detail might have helped, because the syntaxes seem similar.
It seems that people coming from C-family languages (C#, Java, C, C++, JavaScript) are having problems understanding the use of brackets in F#. I certainly had, and it took me some years learning how things work.
In a nutshell, the most basic building block in F# is a value. Values can be let-bound:
let foo = bar
This means that foo is a value, which happens to be equal to bar.
Functions are also values:
// 'a -> 'a * 'a
let f = fun x -> x, x
Here, f is a function that takes some value (x) and returns a tuple with x as both the first and the second element.
That's a bit cumbersome to write, so there's a shorthand for that:
// 'a -> 'a * 'a
let f x = x, x
Notice that there are no brackets in these expressions.
Sometimes you need to adjust the precedence of operators. Just like in maths, 1 + 2 * 3 (which is equivalent to 1 + (2 * 3)) isn't the same as (1 + 2) * 3. In F#, you also use brackets to override precedence. Thus
// 'a -> string * 'a
let f x = someOtherFunction x, x
isn't the same as
// x:'a -> string
let f x = someOtherFunction (x, x)
(in this case, someOtherFunction is a function that returns a string.)
Notice that the brackets don't denote a function call; they're only there to control order of evaluation.
Sometimes, you want to define a function that doesn't take any input. You can't, however, define it like this:
let f = whatever
because that would make it a value that's immediately let-bound to whatever. Instead, you can let the function take a value of the built-in type unit. This type only has a single value, which is written ():
let f () = whatever
This means that f is a function that pattern matches its input against the only known value of unit.
Whenever you invoke f with (), the expression whatever is evaluated and returned.
Without the parentheses, the content executes once and never again. read_rest_of_csv has a type of bool: You are basically saying while true do ignore None.
The parentheses indicate that read_rest_of_csv has type unit -> bool, so every time you invoke it, it reads a row and moves the cursor. Otherwise, it will only do this once.
The answer to your question is that:
let read_rest_of_csv =
csv_data.Add(csv_fileH.ReadFields()) |> ignore
not csv_fileH.EndOfData
is not a function at all. This is no different from:
> let i = 1;;
val i : int = 1
This declares a binding with an integer value. If you want to declare a binding with a function value which takes no parameters, that looks like this:
> let i () = 1;;
val i : unit -> int
The exact same reasoning applies to read_rest_of_csv. Without the parenthesis, you are declaring a binding with type bool. With the parenthesis, you are declaring a binding with type unit->bool i.e. a binding with a function value where the function takes no inputs and returns a bool value.
I am trying to use the Math.NET numerics implementation of the FFT algorithm, but I must be doing something wrong because the output is always unit
The following is the the setup:
open MathNet.Numerics
open MathNet.Numerics.Statistics
open MathNet.Numerics.IntegralTransforms
let rnd = new Random()
let rnddata = Array.init 100 (fun u -> rnd.NextDouble())
let x = rnddata |> Array.Parallel.map (fun d -> MathNet.Numerics.complex.Create(d, 0.0) )
then when I run this:
let tt = MathNet.Numerics.IntegralTransforms.Fourier.BluesteinForward(x, FourierOptions.Default)
I receive an empty output below?
val tt : unit = ()
Any ideas why?
I think the Fourier.BluesteinForward method stores the results in the input array (by overwriting whatever was there originally).
If you do not need the input after running the transform, you can just use x and read the results (this saves some memory copying, which is why Math.NET does that by default). Otherwise, you can clone the array and wrap it in a more functional style code like this:
let bluesteinForward input =
let output = Array.copy input
MathNet.Numerics.IntegralTransforms.Fourier.BluesteinForward
(output, FourierOptions.Default)
output
Does anyone know of 'prior art' regarding the following subject :
I have data that take some decent time to load. they are historical level for various stocks.
I would like to preload them somehow, to avoid the latency when using my app
However, preloading them in one chunk at start makes my app unresponsive first which is not user friendly
So I would like to not load my data.... unless the user is not requesting any and playing with what he already has, in which case I would like to get little by little. So it is neither 'lazy' nor 'eager', more 'lazy when you need' and 'eager when you can', hence the acronym LWYNEWYC.
I have made the following which seems to work, but I just wonder if there is a recognized and blessed approach for such thing ?
let r = LoggingFakeRepo () :> IQuoteRepository
r.getHisto "1" |> ignore //prints Getting histo for 1 when called
let rc = RepoCached (r) :> IQuoteRepository
rc.getHisto "1" |> ignore //prints Getting histo for 1 the first time only
let rcc = RepoCachedEager (r) :> IQuoteRepository
rcc.getHisto "100" |> ignore //prints Getting histo 1..100 by itself BUT
//prints Getting histo 100 immediately when called
And the classes
type IQuoteRepository =
abstract getUnderlyings : string seq
abstract getHisto : string -> string
type LoggingFakeRepo () =
interface IQuoteRepository with
member x.getUnderlyings = printfn "getting underlyings"
[1 .. 100] |> List.map string :> _
member x.getHisto udl = printfn "getting histo for %A" udl
"I am a historical dataset in a disguised party"
type RepoCached (rep : IQuoteRepository) =
let memoize f =
let cache = new System.Collections.Generic.Dictionary<_, _>()
fun x ->
if cache.ContainsKey(x) then cache.[x]
else let res = f x
cache.[x] <- res
res
let udls = lazy (rep.getUnderlyings )
let gethistom = memoize rep.getHisto
interface IQuoteRepository with
member x.getUnderlyings = udls.Force()
member x.getHisto udl = gethistom udl
type Message = string * AsyncReplyChannel<UnderlyingWrap>
type RepoCachedEager (rep : IQuoteRepository) =
let udls = rep.getUnderlyings
let agent = MailboxProcessor<Message>.Start(fun inbox ->
let repocached = RepoCached (rep) :> IQuoteRepository
let rec loop l =
async { try
let timeout = if l|> List.isEmpty then -1 else 50
let! (udl, replyChannel) = inbox.Receive(timeout)
replyChannel.Reply(repocached.getHisto udl)
do! loop l
with
| :? System.TimeoutException ->
let udl::xs = l
repocached.getHisto udl |> ignore
do! loop xs
}
loop (udls |> Seq.toList))
interface IQuoteRepository with
member x.getUnderlyings = udls
member x.getHisto udl = agent.PostAndReply(fun reply -> udl, reply)
I like your solution. I think using agent to implement some background loading with a timeout is a great way to go - agents can nicely encapsulate mutable state, so it is clearly safe and you can encode the behaviour you want quite easily.
I think asynchronous sequences might be another useful abstraction (if I'm correct, they are available in FSharpX these days). An asynchronous sequence represents a computation that asynchronously produces more values, so they might be a good way to separate the data loader from the rest of the code.
I think you'll still need an agent to synchronize at some point, but you can nicely separate different concerns using async sequences.
The code to load the data might look something like this:
let loadStockPrices repo = asyncSeq {
// TODO: Not sure how you detect that the repository has no more data...
while true do
// Get next item from the repository, preferably asynchronously!
let! data = repo.AsyncGetNextHistoricalValue()
// Return the value to the caller...
yield data }
This code represents the data loader, and it separates it from the code that uses it. From the agent that consumes the data source, you can use AsyncSeq.iterAsync to consume the values and do something with them.
With iterAsync, the function that you specify as a consumer is asynchronous. It may block (i.e. using Sleep) and when it blocks, the source - that is.your loader - is also blocked. This is quite nice implicit way to control the loader from the code that consumes the data.
A feature that is not in the library yet (but would be useful) is an partially eager evaluator that takes AsyncSeq<'T> and returns a new AsyncSeq<'T> but obtains a certain number of elements from the source as soon as possible and caches them (so that the consumer does not have to wait when it asks for a value, as long as the source can produce values fast enough).
I'm writing a quick DB perf test, and chose F# so I can get more practice.
I've created a method, measureSelectTimes, which has the signature Guid list * Guid list -> IDbCommand -> TimeSpan * TimeSpan.
Then, I call it:
let runTests () =
let sqlCeConn : IDbConnection = initSqlCe() :> IDbConnection
let sqlServerConn : IDbConnection = initSqlServer() :> IDbConnection
let dbsToTest = [ sqlCeConn; sqlServerConn ]
let cmds : seq<IDbCommand> = dbsToTest |> Seq.map initSchema
let ids : seq<Guid list * Guid list> = cmds |> Seq.map loadData
let input = Seq.zip ids cmds
let results = input |> Seq.map (fun i -> measureSelectTimes (fst i) (snd i))
// ...
I've annotated explicitly with types to clarify.
What I can't figure out is how to call measureSelectTimes without the lambda. I'd like to partially apply the ids to it like this: ids |> Seq.map measureSelectTimes but then I don't know what to do with the resulting partially applied functions to then map onto the cmds. What's the syntax for this?
You can use Seq.map2:
Seq.map2 measureSelectTimes ids cmds
Or
(ids, cmds) ||> Seq.map2 measureSelectTimes
Your measureSelectTimes function takes two arguments as separate arguments, but you instead need a function that takes them as a tuple. One option is to just change the function to take a tuple (if it is logical for the arguments to be tupled).
Alternative, you can write a cobinator that turns a function taking two arguments into a function taking tuple. This is usually called uncurry and it exists in some functional language:
let uncurry f (a, b) = f a b
Then you can write:
input |> Seq.map (uncurry measureSelectTimes)
This looks okay for a simple use like this, but I think that using combinators too much in F# is not a good idea as it makes code difficult to read for less experienced functional programmers. I would probably write something like this (because I find that more readable):
[ for (time1, time2) in input -> measureSelectTimes time1 time2 ]
One approach is to change the signature of measureSelectTimes to
(Guid list * Guid list) * IDbCommand -> TimeSpan * TimeSpan
Then you can change the map call to
let results = input |> Seq.map measureSelectTimes
// or
let results = Seq.map measureSelectTimes input