I'm trying to make a little function to interpolate between two values with a given increment.
[ 1.0 .. 0.5 .. 20.0 ]
The compiler tells me that this is deprecated, and suggests using ints then casting to float. But this seems a bit long-winded if I have a fractional increment - do I have to divide my start and end values by my increment, then multiple again afterwards? (yeuch!).
I saw something somewhere once about using sequence comprehensions to do this, but I can't remember how.
Help, please.
TL;DR: F# PowerPack's BigRational type is the way to go.
What's Wrong with Floating-point Loops
As many have pointed out, float values are not suitable for looping:
They do have Round Off Error, just like with 1/3 in decimal, we inevitably lose all digits starting at a certain exponent;
They do experience Catastrophic Cancellation (when subtracting two almost equal numbers, the result is rounded to zero);
They always have non-zero Machine epsilon, so the error is increased with every math operation (unless we are adding different numbers many times so that errors mutually cancel out -- but this is not the case for the loops);
They do have different accuracy across the range: the number of unique values in a range [0.0000001 .. 0.0000002] is equivalent to the number of unique values in [1000000 .. 2000000];
Solution
What can instantly solve the above problems, is switching back to integer logic.
With F# PowerPack, you may use BigRational type:
open Microsoft.FSharp.Math
// [1 .. 1/3 .. 20]
[1N .. 1N/3N .. 20N]
|> List.map float
|> List.iter (printf "%f; ")
Note, I took my liberty to set the step to 1/3 because 0.5 from your question actually has an exact binary representation 0.1b and is represented as +1.00000000000000000000000 * 2-1; hence it does not produce any cumulative summation error.
Outputs:
1.000000; 1.333333; 1.666667; 2.000000; 2.333333; 2.666667; 3.000000; (skipped) 18.000000; 18.333333; 18.666667; 19.000000; 19.333333; 19.666667; 20.000000;
// [0.2 .. 0.1 .. 3]
[1N/5N .. 1N/10N .. 3N]
|> List.map float
|> List.iter (printf "%f; ")
Outputs:
0.200000; 0.300000; 0.400000; 0.500000; (skipped) 2.800000; 2.900000; 3.000000;
Conclusion
BigRational uses integer computations, which are not slower than for floating-points;
The round-off occurs only once for each value (upon conversion to a float, but not within the loop);
BigRational acts as if the machine epsilon were zero;
There is an obvious limitation: you can't use irrational numbers like pi or sqrt(2) as they have no exact representation as a fraction. It does not seem to be a very big problem because usually, we are not looping over both rational and irrational numbers, e.g. [1 .. pi/2 .. 42]. If we do (like for geometry computations), there's usually a way to reduce the irrational part, e.g. switching from radians to degrees.
Further reading:
What Every Computer Scientist Should Know About Floating-Point Arithmetic
Numeric types in PowerPack
Interestingly, float ranges don't appear to be deprecated anymore. And I remember seeing a question recently (sorry, couldn't track it down) talking about the inherent issues which manifest with float ranges, e.g.
> let xl = [0.2 .. 0.1 .. 3.0];;
val xl : float list =
[0.2; 0.3; 0.4; 0.5; 0.6; 0.7; 0.8; 0.9; 1.0; 1.1; 1.2; 1.3; 1.4; 1.5; 1.6;
1.7; 1.8; 1.9; 2.0; 2.1; 2.2; 2.3; 2.4; 2.5; 2.6; 2.7; 2.8; 2.9]
I just wanted to point out that you can use ranges on decimal types with a lot less of these kind of rounding issues, e.g.
> [0.2m .. 0.1m .. 3.0m];;
val it : decimal list =
[0.2M; 0.3M; 0.4M; 0.5M; 0.6M; 0.7M; 0.8M; 0.9M; 1.0M; 1.1M; 1.2M; 1.3M;
1.4M; 1.5M; 1.6M; 1.7M; 1.8M; 1.9M; 2.0M; 2.1M; 2.2M; 2.3M; 2.4M; 2.5M;
2.6M; 2.7M; 2.8M; 2.9M; 3.0M]
And if you really do need floats in the end, then you can do something like
> {0.2m .. 0.1m .. 3.0m} |> Seq.map float |> Seq.toList;;
val it : float list =
[0.2; 0.3; 0.4; 0.5; 0.6; 0.7; 0.8; 0.9; 1.0; 1.1; 1.2; 1.3; 1.4; 1.5; 1.6;
1.7; 1.8; 1.9; 2.0; 2.1; 2.2; 2.3; 2.4; 2.5; 2.6; 2.7; 2.8; 2.9; 3.0]
As Jon and others pointed out, floating point range expressions are not numerically robust. For example [0.0 .. 0.1 .. 0.3] equals [0.0 .. 0.1 .. 0.2]. Using Decimal or Int Types in the range expression is probably better.
For floats I use this function, it first increases the total range 3 times by the smallest float step. I am not sure if this algorithm is very robust now. But it is good enough for me to insure that the stop value is included in the Seq:
let floatrange start step stop =
if step = 0.0 then failwith "stepsize cannot be zero"
let range = stop - start
|> BitConverter.DoubleToInt64Bits
|> (+) 3L
|> BitConverter.Int64BitsToDouble
let steps = range/step
if steps < 0.0 then failwith "stop value cannot be reached"
let rec frange (start, i, steps) =
seq { if i <= steps then
yield start + i*step
yield! frange (start, (i + 1.0), steps) }
frange (start, 0.0, steps)
Try the following sequence expression
seq { 2 .. 40 } |> Seq.map (fun x -> (float x) / 2.0)
You can also write a relatively simple function to generate the range:
let rec frange(from:float, by:float, tof:float) =
seq { if (from < tof) then
yield from
yield! frange(from + by, tof) }
Using this you can just write:
frange(1.0, 0.5, 20.0)
Updated version of Tomas Petricek's answer, which compiles, and works for decreasing ranges (and works with units of measure):
(but it doesn't look as pretty)
let rec frange(from:float<'a>, by:float<'a>, tof:float<'a>) =
// (extra ' here for formatting)
seq {
yield from
if (float by > 0.) then
if (from + by <= tof) then yield! frange(from + by, by, tof)
else
if (from + by >= tof) then yield! frange(from + by, by, tof)
}
#r "FSharp.Powerpack"
open Math.SI
frange(1.0<m>, -0.5<m>, -2.1<m>)
UPDATE I don't know if this is new, or if it was always possible, but I just discovered (here), that this - simpler - syntax is also possible:
let dl = 9.5 / 11.
let min = 21.5 + dl
let max = 40.5 - dl
let a = [ for z in min .. dl .. max -> z ]
let b = a.Length
(Watch out, there's a gotcha in this particular example :)
Related
I'm learning F# and have an assignment where I have to treat a float as a coordinate. For example float 2.3 would be treated as a coordinate (2.3) where x is 2 and y is 3.
How can I split the float to calculate with it?
I am trying to make a function to calculate the length of a vector:
let lenOfVec (1.2, 2.3) and using pythagoras' method to get the length of hypotenuse.
But I am already stuck at splitting up the float.
Hope some can help!
Having at your disposal libraries as rich as F#/.NET offer the task of splitting a float into two can be done with one short line of code:
let splitFloat n = n.ToString().Split('.') |> Array.map float
library function ToString() converts the argument n (supposedly float) to a string
library functionSplit('.') applied to this string converts it into an array of two strings representing the first number before decimal dot and the second number after the dot
finally this array of 2 strings is converted by applying library function float to the each array element with the help of just another library function Array.map, producing the array of two sought floats
Being applied to a random float number the outlined chain of conversions looks like
123.456 --> "123.456" --> [|123;456|] --> [|123.0;456.0|]
Stealing from a few other answers on here, something like this seems to work for a few examples:
open System
///Takes in a float and returns a tuple of the the two parts.
let split (n: float) =
let x = Math.Truncate(n)
let bits = Decimal.GetBits(decimal n)
let count = BitConverter.GetBytes(bits.[3]).[2]
let dec = n - x
let y = dec * Math.Pow(10., float count)
x, y
Examples:
2.3 -> (2.0, 3.0)
200.123 -> (200.0, 123.0)
5.23 -> (5.0, 23.0)
Getting the X is easy, as you can just truncate the decimal part.
Getting the Y took input from this answer and this one.
I'm quite new to F# and have a problem.
I want to solve a nonlinear, constrained optimization problem.
The goal is to minimize a function minFunc with six parameters a, b, c, d, gamma and rho_infty, (the function is quite long so I don´t post it here) and the additional conditions:
a + d > 0,
d > 0,
c > 0,
gamma > 0,
0 <= gamma <= -ln(rho_infty),
0 < roh_infty <= 1.
I´ve tried it with with the Nelder Mead Solver from the Microsoft Solver Foundation, but I don´t know how to add the nonlinear conditions a + d > 0 and 0 <= gamma <= -ln(rho_infty).
My Code so far:
open Microsoft.SolverFoundation.Common
open Microsoft.SolverFoundation.Solvers
let funcFindParameters (startValues:float list) minimizationFunc =
let xInitial = startValues |> List.toArray
let lowerBound = [|-infinity; -infinity; 0.0; 0.0; 0.0; 0.0|]
let upperBound = [|infinity; infinity; infinity; infinity; infinity; 1.0|]
let solution = NelderMeadSolver.Solve(Func<float [], _>(fun parameters -> (minimizationFunc
parameters.[0] parameters.[1] parameters.[2] parameters.[3] parameters.[4] parameters.[5])),
xInitial, lowerBound, upperBound)
where parameters.[0] = a, and so one...
Is there perhaps some possibility to solve it with the Nelder Mead Solver or some other solver?
One comment, is that I would stay away from the Microsoft.SolverFoundation, I have wasted hours of my life on bad algorithms coded there. The R type provider is much better.
With that said, a common hack is simply to simply reparameterize the model to handle the constraints. For example, set:
e=a+d
as the parameter, and inside the optimzation calculate d as:
d=e-a
And now you just have to satisfy the constraint e>0, which is fixed. You can do something similar for the gamma parameter.
Consider the following code:
let dl = 9.5 / 11.
let min = 21.5 + dl
let max = 40.5 - dl
let a = [ for z in min .. dl .. max -> z ] // should have 21 elements
let b = a.Length
"a" should have 21 elements but has got only 20 elements. The "max - dl" value is missing. I understand that float numbers are not precise, but I hoped that F# could work with that. If not then why F# supports List comprehensions with float iterator? To me, it is a source of bugs.
Online trial: http://tryfs.net/snippets/snippet-3H
Converting to decimals and looking at the numbers, it seems the 21st item would 'overshoot' max:
let dl = 9.5m / 11.m
let min = 21.5m + dl
let max = 40.5m - dl
let a = [ for z in min .. dl .. max -> z ] // should have 21 elements
let b = a.Length
let lastelement = List.nth a 19
let onemore = lastelement + dl
let overshoot = onemore - max
That is probably due to lack of precision in let dl = 9.5m / 11.m?
To get rid of this compounding error, you'll have to use another number system, i.e. Rational. F# Powerpack comes with a BigRational class that can be used like so:
let dl = 95N / 110N
let min = 215N / 10N + dl
let max = 405N / 10N - dl
let a = [ for z in min .. dl .. max -> z ] // Has 21 elements
let b = a.Length
Properly handling float precision issues can be tricky. You should not rely on float equality (that's what list comprehension implicitely does for the last element). List comprehensions on float are useful when you generate an infinite stream. In other cases, you should pay attention to the last comparison.
If you want a fixed number of elements, and include both lower and upper endpoints, I suggest you write this kind of function:
let range from to_ count =
assert (count > 1)
let count = count - 1
[ for i = 0 to count do yield from + float i * (to_ - from) / float count]
range 21.5 40.5 21
When I know the last element should be included, I sometimes do:
let a = [ for z in min .. dl .. max + dl*0.5 -> z ]
I suspect the problem is with the precision of floating point values. F# adds dl to the current value each time and checks if current <= max. Because of precision problems, it might jump over max and then check if max+ε <= max (which will yield false). And so the result will have only 20 items, and not 21.
After running your code, if you do:
> compare a.[19] max;;
val it : int = -1
It means max is greater than a.[19]
If we do calculations the same way the range operator does but grouping in two different ways and then compare them:
> compare (21.5+dl+dl+dl+dl+dl+dl+dl+dl) ((21.5+dl)+(dl+dl+dl+dl+dl+dl+dl));;
val it : int = 0
> compare (21.5+dl+dl+dl+dl+dl+dl+dl+dl+dl) ((21.5+dl)+(dl+dl+dl+dl+dl+dl+dl+dl));;
val it : int = -1
In this sample you can see how adding 7 times the same value in different order results in exactly the same value but if we try it 8 times the result changes depending on the grouping.
You're doing it 20 times.
So if you use the range operator with floats you should be aware of the precision problem.
But the same applies to any other calculation with floats.
Hello everyone
I have converted a project in C# to F# that paints the Mandelbrot set.
Unfortunately does it take around one minute to render a full screen so I am try to find some ways to speed it up.
It is one call that take almost all of the time:
Array.map (fun x -> this.colorArray.[CalcZ x]) xyArray
xyArray (double * double) [] => (array of tuple of double)
colorArray is an array of int32 length = 255
CalcZ is defined as:
let CalcZ (coord:double * double) =
let maxIterations = 255
let rec CalcZHelper (xCoord:double) (yCoord:double) // line break inserted
(x:double) (y:double) iters =
let newx = x * x + xCoord - y * y
let newy = 2.0 * x * y + yCoord
match newx, newy, iters with
| _ when Math.Abs newx > 2.0 -> iters
| _ when Math.Abs newy > 2.0 -> iters
| _ when iters = maxIterations -> iters
| _ -> CalcZHelper xCoord yCoord newx newy (iters + 1)
CalcZHelper (fst coord) (snd coord) (fst coord) (snd coord) 0
As I only use around half of the processor capacity is an idea to use more threads and specifically Array.Parallel.map, translates to system.threading.tasks.parallel
Now my question
A naive solution, would be:
Array.Parallel.map (fun x -> this.colorArray.[CalcZ x]) xyArray
but that took twice the time, how can I rewrite this to take less time, or can I take some other way to utilize the processor better?
Thanks in advance
Gorgen
---edit---
the function that is calling CalcZ looks like this:
let GetMatrix =
let halfX = double bitmap.PixelWidth * scale / 2.0
let halfY = double bitmap.PixelHeight * scale / 2.0
let rect:Mandelbrot.Rectangle =
{xMax = centerX + halfX; xMin = centerX - halfX;
yMax = centerY + halfY; yMin = centerY - halfY;}
let size:Mandelbrot.Size =
{x = bitmap.PixelWidth; y = bitmap.PixelHeight}
let xyList = GenerateXYTuple rect size
let xyArray = Array.ofList xyList
Array.map (fun x -> this.colorArray.[CalcZ x]) xyArray
let region:Int32Rect = new Int32Rect(0,0,bitmap.PixelWidth,bitmap.PixelHeight)
bitmap.WritePixels(region, GetMatrix, bitmap.PixelWidth * 4, region.X, region.Y);
GenerateXYTuple:
let GenerateXYTuple (rect:Rectangle) (pixels:Size) =
let xStep = (rect.xMax - rect.xMin)/double pixels.x
let yStep = (rect.yMax - rect.yMin)/double pixels.y
[for column in 0..pixels.y - 1 do
for row in 0..pixels.x - 1 do
yield (rect.xMin + xStep * double row,
rect.yMax - yStep * double column)]
---edit---
Following a suggestion from kvb (thanks a lot!) in a comment to my question, I built the program in Release mode. Building in the Relase mode generally speeded up things.
Just building in Release took me from 50s to around 30s, moving in all transforms on the array so it all happens in one pass made it around 10 seconds faster. At last using the Array.Parallel.init brought me to just over 11 seconds.
What I learnt from this is.... Use the release mode when timing things and using parallel constructs...
One more time, thanks for the help I have recieved.
--edit--
by using SSE assember from a native dll I have been able to slash the time from around 12 seconds to 1.2 seconds for a full screen of the most computational intensive points. Unfortunately I don't have a graphics processor...
Gorgen
Per the comment on the original post, here is the code I wrote to test the function. The fast version only takes a few seconds on my average workstation. It is fully sequential, and has no parallel code.
It's moderately long, so I posted it on another site: http://pastebin.com/Rjj8EzCA
I'm suspecting that the slowdown you are seeing is in the rendering code.
I don't think that the Array.Parallel.map function (which uses Parallel.For from .NET 4.0 under the cover) should have trouble parallelizing the operation if it runs a simple function ~1 million times. However, I encountered some weird performance behavior in a similar case when F# didn't optimize the call to the lambda function (in some way).
I'd try taking a copy of the Parallel.map function from the F# sources and adding inline. Try adding the following map function to your code and use it instead of the one from F# libraries:
let inline map (f: 'T -> 'U) (array : 'T[]) : 'U[]=
let inputLength = array.Length
let result = Array.zeroCreate inputLength
Parallel.For(0, inputLength, fun i ->
result.[i] <- f array.[i]) |> ignore
result
As an aside, it looks like you're generating an array of coordinates and then mapping it to an array of results. You don't need to create the coordinate array if you use the init function instead of map: Array.Parallel.init 1000 (fun y -> Array.init 1000 (fun x -> this.colorArray.[CalcZ (x, y)]))
EDIT: The following may be inaccurate:
Your problem could be that you call a tiny function a million times, causing the scheduling overhead to overwhelm that actual work you're doing. You should partition the array into much larger chunks so that each individual task takes a millisecond or so. You can use an array of arrays so that you would call Array.Parallel.map on the outer arrays and Array.map on the inner arrays. That way each parallel operation will operate on a whole row of pixels instead of just a single pixel.
I'm trying to experiment with software defined radio concepts. From this article I've tried to implement a GPU-parallelism Discrete Fourier Transform.
I'm pretty sure I could pre-calculate 90 degrees of the sin(i) cos(i) and then just flip and repeat rather than what I'm doing in this code and that that would speed it up. But so far, I don't even think I'm getting correct answers. An all-zeros input gives a 0 result as I'd expect, but all 0.5 as inputs gives 78.9985886f (I'd expect a 0 result in this case too). Basically, I'm just generally confused. I don't have any good input data and I don't know what to do with the result or how to verify it.
This question is related to my other post here
open Microsoft.ParallelArrays
open System
// X64MulticoreTarget is faster on my machine, unexpectedly
let target = new DX9Target() // new X64MulticoreTarget()
ignore(target.ToArray1D(new FloatParallelArray([| 0.0f |]))) // Dummy operation to warm up the GPU
let stopwatch = new System.Diagnostics.Stopwatch() // For benchmarking
let Hz = 50.0f
let fStep = (2.0f * float32(Math.PI)) / Hz
let shift = 0.0f // offset, once we have to adjust for the last batch of samples of a stream
// If I knew that the periodic function is periodic
// at whole-number intervals, I think I could keep
// shift within a smaller range to support streams
// without overflowing shift - but I haven't
// figured that out
//let elements = 8192 // maximum for a 1D array - makes sense as 2^13
//let elements = 7240 // maximum on my machine for a 2D array, but why?
let elements = 7240
// need good data!!
let buffer : float32[,] = Array2D.init<float32> elements elements (fun i j -> 0.5f) //(float32(i * elements) + float32(j)))
let input = new FloatParallelArray(buffer)
let seqN : float32[,] = Array2D.init<float32> elements elements (fun i j -> (float32(i * elements) + float32(j)))
let steps = new FloatParallelArray(seqN)
let shiftedSteps = ParallelArrays.Add(shift, steps)
let increments = ParallelArrays.Multiply(fStep, steps)
let cos_i = ParallelArrays.Cos(increments) // Real component series
let sin_i = ParallelArrays.Sin(increments) // Imaginary component series
stopwatch.Start()
// From the documentation, I think ParallelArrays.Multiply does standard element by
// element multiplication, not matrix multiplication
// Then we sum each element for each complex component (I don't understand the relationship
// of this, or the importance of the generalization to complex numbers)
let real = target.ToArray1D(ParallelArrays.Sum(ParallelArrays.Multiply(input, cos_i))).[0]
let imag = target.ToArray1D(ParallelArrays.Sum(ParallelArrays.Multiply(input, sin_i))).[0]
printf "%A in " ((real * real) + (imag * imag)) // sum the squares for the presence of the frequency
stopwatch.Stop()
printfn "%A" stopwatch.ElapsedMilliseconds
ignore (System.Console.ReadKey())
I share your surprise that your answer is not closer to zero. I'd suggest writing naive code to perform your DFT in F# and seeing if you can track down the source of the discrepancy.
Here's what I think you're trying to do:
let N = 7240
let F = 1.0f/50.0f
let pi = single System.Math.PI
let signal = [| for i in 1 .. N*N -> 0.5f |]
let real =
seq { for i in 0 .. N*N-1 -> signal.[i] * (cos (2.0f * pi * F * (single i))) }
|> Seq.sum
let img =
seq { for i in 0 .. N*N-1 -> signal.[i] * (sin (2.0f * pi * F * (single i))) }
|> Seq.sum
let power = real*real + img*img
Hopefully you can use this naive code to get a better intuition for how the accelerator code ought to behave, which could guide you in your testing of the accelerator code. Keep in mind that part of the reason for the discrepancy may simply be the precision of the calculations - there are ~52 million elements in your arrays, so accumulating a total error of 79 may not actually be too bad. FWIW, I get a power of ~0.05 when running the above single precision code, but a power of ~4e-18 when using equivalent code with double precision numbers.
Two suggestions:
ensure you're not somehow confusing degrees with radians
try doing it sans-parallelism, or just with F#'s asyncs for parallelism
(In F#, if you have an array of floats
let a : float[] = ...
then you can 'add a step to all of them in parallel' to produce a new array with
let aShift = a |> (fun x -> async { return x + shift })
|> Async.Parallel |> Async.RunSynchronously
(though I expect this might be slower that just doing a synchronous loop).)