I'm processing a large number of object housed in an array. This processing takes a long time, and I'd like to be able to monitor were the fx is in the processing step.
My goal is to be able to print to the console some sort of Processing thing number *x* while continuing to operate. For example, with this
let x = [|1..10..100000|]
x
|> Array.mapi (fun i n -> (i, n))
|> Array.map (fun (i, n) -> printfn "Processing n %i" i, (n * 2)))
|> Array.map snd
I get an output for every line. What'd I'd like to have is every 10th or 100th or 1000 print a statement, rather than every line. So I've tried
x
|> Array.mapi (fun i n -> (i, n))
|> Array.map (fun (i, n) -> (if (i % 100 = 0) then printfn "Processing n %i" i, (n * 2)))
|> Array.map snd
but this provides an error over the printfn... bit with
The 'if' expression is missing an else branch. The 'then' branch has type
''a * 'b'. Because 'if' is an expression, and not a statement, add an 'else'
branch which returns a value of the same type.
I essentially want the else... branch to do nothing, print nothing to the console, just be ignored.
Interestingly, in writing this question and trying things in FSI, I tried this:
x
|> Array.mapi (fun i n -> (i, n))
|> Array.map (fun (i, n) -> match (i % 100 = 0) with
| true -> printfn "Processing n %i" i, (n * 2)
| false -> (), n * 2)
|> Array.map snd
which seems to work. Is this the best way to provide the console text?
It looks like you want:
let x' = x |> Array.mapi (fun i n ->
if i % 100 = 0 then
printfn "Processing n %i" i
n)
Both branches of an if expression must have the same type and
if (i % 100 = 0) then printfn "Processing n %i" i, (n * 2)
returns a value of type (unit, int) for the true case. The missing else case implicitly has type () so the types do not match. You can just print the value, ignore the result and then return the current value.
Related
I am trying to create a recursive function that is conditionally calls itself and so far is is defined as follows:
let rec crawlPage (page : String, nestingLevel : int) =
HtmlDocument.Load(page)
|> fun m -> m.CssSelect("a")
|> List.map(fun a -> a.AttributeValue("href"))
|> Seq.distinctBy id
|> Seq.map (fun x -> baseUrl + x)
|> Seq.map (fun x ->
match nestingLevel with
// Compiler says it is expecting a but given seq<a> in reference to the recursive call
| _ when (nestingLevel > 0) -> crawlPage(x, (nestingLevel - 1))
| _ when (nestingLevel <= 0) -> ignore
| _ -> (* To silence warnigs.*) ignore
)
It is that the Seq.map (fun x -> ...) cannot handle the return sequence or can the match condition not handle the returned sequence? Given that the crawlPage is underlined by the compiler it seems that the match statement cannot handle the seq returned so how can this be done?
The rule is that all the matching branches must return the same type, so you have to:
Replace ignore with Seq.singleton x to indicate that this branch yields nothing more except the x itself.
At the end, concat (flat map) the seq<seq<string>> to transform it to a seq<string>.
The code would be:
|> Seq.map (fun x ->
match nestingLevel with
| _ when (nestingLevel > 0) -> crawlPage(x, (nestingLevel - 1))
| _ -> Seq.singleton x)
|> Seq.concat
The existing post answers your specific question, but I think it is worth noting that there are a few other changes that could be done to your code snippet. Some of those are a matter of personal preference, but I believe they make your code simpler:
You can use sequence comprehension, which lets you handle recursive calls nicely using yield! (and non-recursive using yield)
You do not actually need match, because you have just two branches that are more easily tested using ordinary if
I would also avoid the |> fun m -> m.Xyz pattern, because it's not necessary here.
With all those tweaks, my preferred version of the code snippet would be:
let rec crawlPage (page : String, nestingLevel : int) = seq {
let urls =
HtmlDocument.Load(page).CssSelect("a")
|> List.map(fun a -> a.AttributeValue("href"))
|> Seq.distinctBy id
|> Seq.map (fun x -> baseUrl + x)
for x in urls do
if nestingLevel > 0 then
yield! crawlPage(x, (nestingLevel - 1))
else
yield x }
Given a simple function, where we do pattern matching on the result of a recursive call, such as:
let rec sumProd = function
| [] -> (0,1)
| x::rest -> let (rSum,rProd) = sumProd rest
(x + rSum,x * rProd)
sumProd [2;5] //Expected (7, 10)
How would I go about changing it into something using higher order functions, e.g. foldBack?
let sumProdHigherOrder lst =
List.foldBack (fun x acc -> (acc + x, acc * x)) lst (0,0)
The above seemed almost like the way to do it, but calling it gives the error: The type 'int' does not match the type 'int * int'
sumProdHigherOrder [2;5] //Expected (7, 10)
What am I missing?
Your missing the tuple functions fst and snd:
List.foldBack (fun x acc -> (fst acc + x, snd acc * x)) [2;5] (0,1)
// val it : int * int = (7, 10)
Or even better, decomposing the tuple at the lambda. I see you just found it:
List.foldBack (fun x (s, m) -> (s + x, m * x)) [2;5] (0,1)
Also note that since the operations are commutative you can do a straight fold:
List.fold (fun (s, m) x -> (s + x, m * x)) (0,1) [2;5]
It will be more efficient.
Right! Of course it shouldn't be the same accumulator that gets passed through the list. After staring intensely at the code for some minutes, I figured it out:
let sumProdHigherOrder lst =
List.foldBack (fun x (acc,acc') -> (acc + x, acc' * x)) lst (0,1)
I would do something like
let last n xs = xs |> List.rev |> Seq.take n |> List.ofSeq |> List.rev
I am not sure about turning a list to a sequence and back though. Is this how you do it F#?
Seq + Skip
Taking the last N items is equivalent to skipping the first (length - N) items, so for a Sequence as input (and output), you could do something like:
let last n xs = Seq.skip ((Seq.length xs) - n) xs
(or, with piping, let last n xs = xs |> Seq.skip (Seq.length xs - n)
and for a List as input (and output) you could do:
let last n xs = List.toSeq xs |> Seq.skip (xs.Length - n) |> Seq.toList
or by defining both, just pipe it to the sequence one:
let lastList n xs = List.toSeq xs |> last n |> Seq.toList
Tail + Recursion
Alternatively, this can be achieved by (tail) recursively applying Tail as so:
let rec last n xs =
if List.length xs <= n then xs
else last n xs.Tail
You could use List.foldBack to traverse the list from the end:
let takeLast n list =
let (_, r) = List.foldBack (fun e (i, acc) -> (i - 1, if i <= 0 then acc else e :: acc)) list (n, [])
r
To avoid rebuilding the list, you may use a simple recursive algorithm.
Note, we are not using neither List.Cons nor Seq.toList which does the same internally.
let lastN n xs =
let rec skip n xs =
match n, xs with
| _, [] -> [] // empty list, returning unchanged
| 0, _ -> xs // found an element at which the remainder
// of the list is to be returned
| n', h::t -> skip (n-1) t // proceed to next iteration
let toSkip = (List.length xs) - n // how many elements to skip
if toSkip < 0 then xs // or an exception, depending on expected behavior
elif toSkip = 0 then xs // requested exactly as many elements
// as the list contains
else skip toSkip xs
// usage
let data = [1 .. 10000000]
let stopWatch = new System.Diagnostics.Stopwatch()
stopWatch.Start()
data
|> lastN 3
|> List.iter (printf "%d ")
stopWatch.Stop()
printfn "\nelapsed: %f ms" stopWatch.Elapsed.TotalMilliseconds
Output:
9999998 9999999 10000000
elapsed: 194.846700 ms
Variation on chamila_c's function:-
/// Returns the last abs(n) items in the specified sequence.
let lastN n xs =
// The number to skip will be negative if n is too large; this will result in 0 items being skipped.
// By taking abs(n), the number to skip can't get too large, and we avoid an exception being thrown.
xs |> Seq.skip (Seq.length xs - abs n)
I write some code to learning F#.
Here is a example:
let nextPrime list=
let rec loop n=
match n with
| _ when (list |> List.filter (fun x -> x <= ( n |> double |> sqrt |> int)) |> List.forall (fun x -> n % x <> 0)) -> n
| _ -> loop (n+1)
loop (List.max list + 1)
let rec findPrimes num=
match num with
| 1 -> [2]
| n ->
let temp = findPrimes <| n-1
(nextPrime temp ) :: temp
//find 10 primes
findPrimes 10 |> printfn "%A"
I'm very happy that it just works!
I'm totally beginner to recursion
Recursion is a wonderful thing.
I think findPrimes is not efficient.
Someone help me to refactor findPrimes to tail recursion if possible?
BTW, is there some more efficient way to find first n primes?
Regarding the first part of your question, if you want to write a recursive list building function tail-recursively you should pass the list of intermediate results as an extra parameter to the function. In your case this would be something like
let findPrimesTailRecursive num =
let rec aux acc num =
match num with
| 1 -> acc
| n -> aux ((nextPrime acc)::acc) (n-1)
aux [2] num
The recursive function aux gathers its results in an extra parameter conveniently called acc (as in acc-umulator). When you reach your ending condition, just spit out the accumulated result. I've wrapped the tail-recursive helper function in another function, so the function signature remains the same.
As you can see, the call to aux is the only, and therefore last, call to happen in the n <> 1 case. It's now tail-recursive and will compile into a while loop.
I've timed your version and mine, generating 2000 primes. My version is 16% faster, but still rather slow. For generating primes, I like to use an imperative array sieve. Not very functional, but very (very) fast.
An alternative is to use an extra continuation argument to make findPrimes tail recursive. This technique always works. It will avoid stack overflows, but probably won't make your code faster.
Also, I put your nextPrime function a little closer to the style I'd use.
let nextPrime list=
let rec loop n = if list |> List.filter (fun x -> x*x <= n)
|> List.forall (fun x -> n % x <> 0)
then n
else loop (n+1)
loop (1 + List.head list)
let rec findPrimesC num cont =
match num with
| 1 -> cont [2]
| n -> findPrimesC (n-1) (fun temp -> nextPrime temp :: temp |> cont)
let findPrimes num = findPrimesC num (fun res -> res)
findPrimes 10
As others have said, there's faster ways to generate primes.
Why not simply write:
let isPrime n =
if n<=1 then false
else
let m = int(sqrt (float(n)))
{2..m} |> Seq.forall (fun i->n%i<>0)
let findPrimes n =
{2..n} |> Seq.filter isPrime |> Seq.toList
or sieve (very fast):
let generatePrimes max=
let p = Array.create (max+1) true
let rec filter i step =
if i <= max then
p.[i] <- false
filter (i+step) step
{2..int (sqrt (float max))} |> Seq.iter (fun i->filter (i+i) i)
{2..max} |> Seq.filter (fun i->p.[i]) |> Seq.toArray
BTW, is there some more efficient way to find first n primes?
I described a fast arbitrary-size Sieve of Eratosthenes in F# here that accumulated its results into an ever-growing ResizeArray:
> let primes =
let a = ResizeArray[2]
let grow() =
let p0 = a.[a.Count-1]+1
let b = Array.create p0 true
for di in a do
let rec loop i =
if i<b.Length then
b.[i] <- false
loop(i+di)
let i0 = p0/di*di
loop(if i0<p0 then i0+di-p0 else i0-p0)
for i=0 to b.Length-1 do
if b.[i] then a.Add(p0+i)
fun n ->
while n >= a.Count do
grow()
a.[n];;
val primes : (int -> int)
I know that this is a bit late, and an answer was already accepted. However, I believe that a good step by step guide to making something tail recursive may be of interest to the OP or anyone else for that matter. Here are some tips that have certainly helped me out. I'm going to use a strait-forward example other than prime generation because, as others have stated, there are better ways to generate primes.
Consider a naive implementation of a count function that will create a list of integers counting down from some n. This version is not tail recursive so for long lists you will encounter a stack overflow exception:
let rec countDown = function
| 0 -> []
| n -> n :: countDown (n - 1)
(* ^
|... the cons operator is in the tail position
as such it is evaluated last. this drags
stack frames through subsequent recursive
calls *)
One way to fix this is to apply continuation passing style with a parameterized function:
let countDown' n =
let rec countDown n k =
match n with
| 0 -> k [] (* v--- this is continuation passing style *)
| n -> countDown (n - 1) (fun ns -> n :: k ns)
(* ^
|... the recursive call is now in tail position *)
countDown n (fun ns -> ns)
(* ^
|... and we initialize k with the identity function *)
Then, refactor this parameterized function into a specialized representation. Notice that the function countDown' is not actually counting down. This is an artifact of the way the continuation is built up when n > 0 and then evaluated when n = 0. If you have something like the first example and you can't figure out how to make it tail recursive, what I'm suggesting is that you write the second one and then try to optimize it to eliminate the function parameter k. That will certainly improve the readability. This is an optimization of the second example:
let countDown'' n =
let rec countDown n ns =
match n with
| 0 -> List.rev ns (* reverse so we are actually counting down again *)
| n -> countDown (n - 1) (n :: ns)
countDown n []
mySequence
|> Seq.iteri (fun i x -> ...)
...
How do I bind i at the end of the sequence? In other words how do I bind the value representing the number of iterations iterated by iteri?
Of course I could create a ref and assign i for all iterations, but I wonder if there is a more elegant way?
You could use fold, so that
Seq.iteri (fun i x -> ...)
becomes
Seq.fold (fun i x -> ... ; i+1) 0
along these lines:
let aSeq =
seq {
for i in 1..10 do
printfn "eval %d" i
yield i
}
let r =
aSeq
|> Seq.fold (fun i x ->
printfn "iter %d" x // or whatever is "..."
i+1) 0
printfn "result: %d" r
As I understand you could just use function that will directly return length of the sequence that is passed to Seq.iteri (since Seq.iteri will iterate over the whole sequence). This will be more functional programming way instead of thinking about mutable variables:
Seq.length mySequence
In your case:
mySequence |> Seq.iteri (fun i x -> ...)
let i = Seq.length mySequence