Hi I'm using the following code:
let timeser = ser |> Series.sampleTimeInto(TimeSpan(0,5,0)) Direction.Backward Series.lastValue
However often get the following error; System.IndexOutOfRangeException with the Additional information: Index was outside the bounds of the array.
Does anyone know how to solve this error.
It is a bit difficult to guess what is the problem, because we cannot run your code. But the most obvious reason for the exception is that Series.lastValue fails because one of your chunks has no data in it.
Let's say that you have a series with values for just 2 days:
let ser = series [ DateTime(2000, 1, 1) => 1.0; DateTime(2000, 1, 2) => 2.0 ]
If you try to sample it into 12 hour chunks using Series.lastValue, then this fails (because only two of the chunks you get actually contain some values):
// This shows you that some of the chunks are empty
ser |> Series.sampleTime (TimeSpan(12,0,0)) Direction.Backward
// This will fail
ser |> Series.sampleTimeInto(TimeSpan(12,0,0)) Direction.Backward Series.lastValue
You can handle this in various ways, but the easiest one would be to return a missing value for chunks with no data:
ser |> Series.sampleTimeInto(TimeSpan(12,0,0)) Direction.Backward (fun s ->
if s.IsEmpty then nan else Series.lastValue s)
Related
//Return a tuple from a text file:
let ExtractFromLine (line:string) =
let strings = line.Split('\t') //data members are spaced by tab
let strlist = Array.toList(strings) //each data member is now a list of str
let year = System.Int32.Parse(strlist.Head) //year is first in file, so Head
let values = List.map System.Double.Parse strlist.Tail //tail are all values
let average = List.average values //not part of text file
let min = List.min values //not part of text file
let max = List.max values //not part of text file
(year, values, average, min, max) //return tuple with all info
//----------
let rec createList fileline =
if fileline = [] then
[]
else
let (year, values, average, min, max) = ExtractFromLine fileline.Head
let l = (year, values, average, min, max) :: createList fileline.Tail
l
//------------
let main argv =
let file = ReadFile "data.txt"
let biglist = createList file //recursive function to make a list of tuples
printfn"%A" biglist //This prints the year, all values, average, min, and max for each tuple created
I now have a giant list of tuples with all of the information that I need.
Have I retained the possibility of accessing all elements inside and performing calculations on them? I program in C++, and the solution is doable in that language, but F# is so much more powerful in my opinion. I'm sure its possible, I'm just missing the basics.
For example, how do I print the average of all the values for all years?
I'm thinking of a for loop, but I'm not sure how to iterate.
for(all tuples in biglist)
printfn"%A:%A" tuple.year tuple.average
It's wrong obviously, but I think you guys understand what I'm trying to do.
The above question involves pulling data from one tuple at a time across the list. What if I wanted to print the largest average?This would involve accessing each tuple's average data member and comparing them to return the largest one. Do I have to create another list containing these averages?
I learned about fst and snd but I had a hard time applying it to this example.
You don't have to answer all questions if it is too much, but any help is greatly appreciated as I start out in this language, thank you
You can loop in F# but it's a construct from imperative programming world. More idiomatic approach is to access items of the list recursively.
Below some sample code that creates tuples, constructs a list, and then access items and checks which one is bigger. Looking at your code the average was third item in the tuple. That's why I've added a trd function. It takes a 5-item tuple and returns a third item.
The prcsLst function takes 2 arguments: a list and a starting max value. The idea is that when processing the list we take the head (first item on the list), compare it's average with current max. Whichever is bigger is passed to the next recursive round together with list's tail (the list without the first item).In this case as the initial max I passed in the average of the first item.
You can run the example in F# Interactive to see the results.
// create sample tuples
let t1 = (2014, 35, 18, 5, 45)
let t2 = (2014, 32, 28, 8, 75)
let t3 = (2014, 25, 11, 9, 55)
let t4 = (2015, 16, 13, 2, 15)
let t5 = (2015, 29, 15, 1, 35)
// create sample list
let lst = [t1;t2;t3;t4;t5]
// a function to return third item in a tuple
let trd (_,_,t,_,_) = t
// process list recursively
let rec prcsLst l max =
match l with
| [] -> max
| hd::tl ->
if (trd hd) > max then
prcsLst tl (trd hd)
else
prcsLst tl max
// invoke the method on the sample list
// as a starting point use the first item in the list
prcsLst lst (trd t1);;
On a mobile so forgive me for not doing any code examples :)
I suspect that the missing piece of your puzzle is called pattern matching. In F# you address elements of a tuple like so:
let (y, v, Av, mn, mx) = mytuple
Note that you can also use this in function declarations, and when doing a 'match'.
(there is an exception for 'twoples' where you can use the functions 'fst' and 'snd')
Another thing you should play with is the |> operator.
I have a function which returns a sequence of records. In that function I start the list building with a blank dummy record (there is probably a better way to do it) because I need to accumulate records that are similar, so I "prime the pump" with a blank record. Here's my code:
let consolidate(somethings:seq<Something>) =
let mutable results = ResizeArray()
let mutable accumulatedSomething = {Foo = ""; Bar = ""; Count = 0;}
for s in somethings do
if s.Foo = accumulatedSomething.Foo && s.Bar = accumulatedSomething.Bar then
accumulatedSomething <- {Foo = s.Foo; Bar = s.Bar;
Count = s.Count + accumulatedSomething.Count}
else
results.Add(accumulatedSomething)
accumulatedSomething <- e
results |> Seq.cast |> Seq.skip 1
If you have a way to make this better I'm all ears (I'm still thinking procedurally) but I'm still interested in an answer to this specific question. Later on in my code, I try to print out the list:
somethings |> Seq.iter( fun s -> printfn "%A" s)
This works fine when there is stuff in the list. But if the list is empty and the only record that was in the list was the skipped blank starter record, then this line fails with an InvalidOperationException with the message The input sequence has an insufficient number of elements?
Why does this happen and how can I fix it?
The problem occurs when somethings is an empty list.
In this case, results is empty and calling Seq.skip 1 on the empty list fails with an error.
I think an elegant solution would be to change the last line to
match results.Length with
| 0 -> results |> Seq.cast
| _ -> results |> Seq.cast |> Seq.skip 1
I am just getting started with F# and am trying Problem Euler problem #3. To find primes I came up with the following code to compute all primes up to a maximum number:
let rec allPrimes foundPrimes, current, max =
// make sure the current number isn't too high
// or the current number isn't divisible by any known primes
if current >= max then
foundPrimes
else
let nextValue = current + 1
if not List.exists (fun x -> current % x = 0L) foundPrimes then
allPrimes foundPrimes nextValue max
else
allPrimes (foundPrimes :: current) nextValue max
Unfortunately, this gives the error:
Only simple variable patterns can be bound in 'let rec' constructs
Why am I getting this error?
You don't want to put the commas in the declaration - change
let rec allPrimes foundPrimes, current, max =
to
let rec allPrimes foundPrimes current max =
The correct version of your original would be
let rec allPrimes (foundPrimes, current, max) =
note the brackets around the tuple. However, this would require modifying the recursive calls to also use tuple form. In the original version the compiler thinks you are trying to do something like
let a,b,c=1,2,3
which won't work for recursive functions.
There is any way to do it like C/C#?
For example (C# style)
for (int i = 0; i < 100; i++)
{
if (i == 66)
break;
}
The short answer is no. You would generally use some higher-order function to express the same functionality. There is a number of functions that let you do this, corresponding to different patterns (so if you describe what exactly you need, someone might give you a better answer).
For example, tryFind function returns the first value from a sequence for which a given predicate returns true, which lets you write something like this:
seq { 0 .. 100 } |> Seq.tryFind (fun i ->
printfn "%d" i
i=66)
In practice, this is the best way to go if you are expressing some high-level logic and there is a corresponding function. If you really need to express something like break, you can use a recursive function:
let rec loop n =
if n < 66 then
printfn "%d" n
loop (n + 1)
loop 0
A more exotic option (that is not as efficient, but may be nice for DSLs) is that you can define a computation expression that lets you write break and continue. Here is an example, but as I said, this is not as efficient.
This is really ugly, but in my case it worked.
let mutable Break = false
while not Break do
//doStuff
if breakCondition then
Break <- true
done
This is useful for do-while loops, because it guarantees that the loop is executed at least once.
I hope there's a more elegant solution. I don't like the recursive one, because I'm afraid of stack overflows. :-(
You have to change it to a while loop.
let (i, ans) = (ref 0, ref -1)
while(!i < 100 and !ans < 0) do
if !i = 66 then
ans := !i
ans
(This breaks when i gets to 66--but yes the syntax is quite different, another variable is introduced, etc.)
seq {
for i = 0 to 99 do
if i = 66 then yield ()
}
|> Seq.tryItem 0
|> ignore
Try this:
exception BreakException
try
for i = 0 to 99 do
if i = 66 then
raise BreakException
with BreakException -> ()
I know that some folks don't like to use exceptions. But it has merits.
You don't have to think about complicated recursive function. Of
cause you can do that, but sometimes it is unnecessarily bothersome
and using exception is simpler.
This method allows you to break at halfway of the loop body. (Break "flag" method is simple too but it only allows to break at the end of the loop body.)
You can easily escape from nested loop.
For these kind of problems you could use a recursive function.
let rec IfEqualsNumber start finish num =
if start = finish then false
elif
start = num then true
else
let start2 = start + 1
IfEqualsNumber start2 finish num
Recently I tried to solve a similar situation:
A list of, say, 10 pieces of data. Each of them must be queried against a Restful server, then get a result for each.
let lst = [4;6;1;8]
The problem:
If there is a failed API call (e.g. network issue), there is no point making further calls as we need all the 10 results available. The entire process should stop ASAP when an API call fails.
The naive approach: use List.map()
lst |> List.map (fun x ->
try
use sqlComd = ...
sqlComd.Parameters.Add("#Id", SqlDbType.BigInt).Value <- x
sqlComd.ExecuteScala() |> Some
with
| :? System.Data.SqlClient.SqlException as ex -> None
)
But as said, it's not optimal. When a failed API occurs, the remaining items keep being processed. They do something that is ignored at the end anyway.
The hacky approach: use List.tryFindIndex()
Unlike map(), we must store the results somewhere in the lamda function. A reasonable choice is to use mutable list. So when tryFindIndex() returns None, we know that everything was ok and can start making use of the mutable list.
val myList: List<string>
let res = lst |> List.tryFindIndex (fun x ->
try
use sqlComd = ...
sqlComd.Parameters.Add("#Id", SqlDbType.BigInt).Value <- x
myList.Add(sqlComd.ExecuteScala())
false
with
|:? System.Data.SqlClient.SqlException as ex -> true
)
match res with
| Some _ -> printfn "Something went wrong"
| None -> printfn "Here is the 10 results..."
The idiomatic approach: use recursion
Not very idiomatic as it uses Exception to stop the operation.
exception MyException of string
let makeCall lstLocal =
match lstLocal with
| [] -> []
| head::tail ->
try
use sqlComd = ...
sqlComd.Parameters.Add("#Id", SqlDbType.BigInt).Value <- x
let temp = sqlComd.ExecuteScala()
temp :: makeCall (tail)
with
|:? System.Data.SqlClient.SqlException as ex -> raise MyException ex.Message
try
let res = makeCall lst
printfn "Here is the 10 results..."
with
| :? MyException -> printfn "Something went wrong"
The old-fashion imperative approach: while... do
This still involves mutable list.
Does anyone know of 'prior art' regarding the following subject :
I have data that take some decent time to load. they are historical level for various stocks.
I would like to preload them somehow, to avoid the latency when using my app
However, preloading them in one chunk at start makes my app unresponsive first which is not user friendly
So I would like to not load my data.... unless the user is not requesting any and playing with what he already has, in which case I would like to get little by little. So it is neither 'lazy' nor 'eager', more 'lazy when you need' and 'eager when you can', hence the acronym LWYNEWYC.
I have made the following which seems to work, but I just wonder if there is a recognized and blessed approach for such thing ?
let r = LoggingFakeRepo () :> IQuoteRepository
r.getHisto "1" |> ignore //prints Getting histo for 1 when called
let rc = RepoCached (r) :> IQuoteRepository
rc.getHisto "1" |> ignore //prints Getting histo for 1 the first time only
let rcc = RepoCachedEager (r) :> IQuoteRepository
rcc.getHisto "100" |> ignore //prints Getting histo 1..100 by itself BUT
//prints Getting histo 100 immediately when called
And the classes
type IQuoteRepository =
abstract getUnderlyings : string seq
abstract getHisto : string -> string
type LoggingFakeRepo () =
interface IQuoteRepository with
member x.getUnderlyings = printfn "getting underlyings"
[1 .. 100] |> List.map string :> _
member x.getHisto udl = printfn "getting histo for %A" udl
"I am a historical dataset in a disguised party"
type RepoCached (rep : IQuoteRepository) =
let memoize f =
let cache = new System.Collections.Generic.Dictionary<_, _>()
fun x ->
if cache.ContainsKey(x) then cache.[x]
else let res = f x
cache.[x] <- res
res
let udls = lazy (rep.getUnderlyings )
let gethistom = memoize rep.getHisto
interface IQuoteRepository with
member x.getUnderlyings = udls.Force()
member x.getHisto udl = gethistom udl
type Message = string * AsyncReplyChannel<UnderlyingWrap>
type RepoCachedEager (rep : IQuoteRepository) =
let udls = rep.getUnderlyings
let agent = MailboxProcessor<Message>.Start(fun inbox ->
let repocached = RepoCached (rep) :> IQuoteRepository
let rec loop l =
async { try
let timeout = if l|> List.isEmpty then -1 else 50
let! (udl, replyChannel) = inbox.Receive(timeout)
replyChannel.Reply(repocached.getHisto udl)
do! loop l
with
| :? System.TimeoutException ->
let udl::xs = l
repocached.getHisto udl |> ignore
do! loop xs
}
loop (udls |> Seq.toList))
interface IQuoteRepository with
member x.getUnderlyings = udls
member x.getHisto udl = agent.PostAndReply(fun reply -> udl, reply)
I like your solution. I think using agent to implement some background loading with a timeout is a great way to go - agents can nicely encapsulate mutable state, so it is clearly safe and you can encode the behaviour you want quite easily.
I think asynchronous sequences might be another useful abstraction (if I'm correct, they are available in FSharpX these days). An asynchronous sequence represents a computation that asynchronously produces more values, so they might be a good way to separate the data loader from the rest of the code.
I think you'll still need an agent to synchronize at some point, but you can nicely separate different concerns using async sequences.
The code to load the data might look something like this:
let loadStockPrices repo = asyncSeq {
// TODO: Not sure how you detect that the repository has no more data...
while true do
// Get next item from the repository, preferably asynchronously!
let! data = repo.AsyncGetNextHistoricalValue()
// Return the value to the caller...
yield data }
This code represents the data loader, and it separates it from the code that uses it. From the agent that consumes the data source, you can use AsyncSeq.iterAsync to consume the values and do something with them.
With iterAsync, the function that you specify as a consumer is asynchronous. It may block (i.e. using Sleep) and when it blocks, the source - that is.your loader - is also blocked. This is quite nice implicit way to control the loader from the code that consumes the data.
A feature that is not in the library yet (but would be useful) is an partially eager evaluator that takes AsyncSeq<'T> and returns a new AsyncSeq<'T> but obtains a certain number of elements from the source as soon as possible and caches them (so that the consumer does not have to wait when it asks for a value, as long as the source can produce values fast enough).