F#: Derivative Function - f#

I'm trying to figure out how to get this function to work. I'm very inept with F#, so explanations are appreciated,
let deriv (f:(float->float), dx: float) =
fun f:(float -> float) * dx:float -> x:float -> float
let (f, dx, x) = ((f(x + dx) - f(x))/dx)
Am I using f:(float->float) wrong?

In stead of trying to fix your problems I will explain it from the ground up.
The symbolic derivative is a function that takes a function and returns a new function. However you are trying to calculate the numeric derivative that returns a value given a function, a value, and a delta.
First we will give the function a name: deriv
and it needs three parameters:
1. A function that takes in a float and returns a float: (f : float -> float)
2. A value of where the derivative is to be evaluated: (x0 : float)
3. A delta: (dx : float)
You had two of the three parameters in your example, but were missing x0.
It should also return a float.
So the signature is
deriv (f : (float -> float)) (x0 : float) (dx : float) : float
Now to calculate the numeric derivative.
I won't explain this, but just reference derivative
For this example we'll use a simple function that has derivative, x^2.
Here is the code in F#
// val deriv : f:(float -> float) -> x0:float -> dx:float -> float
let deriv (f : (float -> float)) (x0 : float) (dx : float) : float =
let (x1 : float) = x0 - dx
let (x2 : float) = x0 + dx
let (y1 : float) = f x1
let (y2 : float) = f x2
let (result : float) = (y2 - y1) / (x2 - x1)
result
// val f : x:float -> float
let f x = x**2.0
and a quick test show it works correctly.
// val it : float = 2.0
deriv f 1.0 0.000005
For a more thorough test of a range of values.
Using Visual Studio and NuGet install FSharp.Charting
In F# Interactive
#I "..\packages"
#load "FSharp.Charting.0.90.13\FSharp.Charting.fsx"
open FSharp.Charting
let xs1 = [ for x in (double)(-3.10) .. 0.05 .. 3.10 do yield x]
let ys1 = xs1 |> List.map f
let values1 = List.zip xs1 ys1
Chart.Line(values1)
.WithXAxis(Min=(-4.0), Max=4.0, MajorTickMark = ChartTypes.TickMark(Interval=2.0, IntervalOffset = 1.0, LineWidth = 2))
.WithYAxis(Min=(0.0), Max=10.0, MajorTickMark = ChartTypes.TickMark(Interval=2.0, IntervalOffset = 1.0, LineWidth = 2))
which can also be confirmed using Wolfram Alpha: x^2
A simpler example:
// val d : x:float -> float
let d x = deriv f x 0.00000005
let xs2 = [ for x in (double)(-3.10) .. 0.05 .. 3.10 do yield x]
let ys2 = xs2 |> List.map d
let values2 = List.zip xs2 ys2
Chart.Line(values2)
.WithXAxis(Min=(-4.0), Max=4.0, MajorTickMark = ChartTypes.TickMark(Interval=2.0, IntervalOffset = 1.0, LineWidth = 2))
.WithYAxis(Min=(-6.0), Max=6.0, MajorTickMark = ChartTypes.TickMark(Interval=2.0, IntervalOffset = 1.0, LineWidth = 2))
which can also be confirmed using Wolfram Alpha: d/dx x^2

Related

Represent finite coordinates as an F# type

I'm learning F# by implementing a board game engine. In accordance with the principle that invalid states should not be representable, I want to create a type which represents (x, y) coordinates (with x and y integers) where x and y must satisfy 0 <= x <= n and 0 <= y <= m. Creating an instance that does not satisfy those inequalities shouldn't compile. Is this possible?
I don't think compile time assurances on values like this are possible.
I would handle this with a private constructor and the Result or Option type depending whether I wanted to propagate error information.
type Position =
private {
X: int
Y: int
}
static member TryNew(x, y) =
if x >= 0 && x <= 10 then
if y >= 0 && y <=10 then
Ok {
X = x
Y = y
}
else
Error $"Supplied y ({y}) must be between 0 and 10"
else
Error $"Supplied x ({x}) must be between 0 and 10"
module Position =
let moveX1 pos =
Position.TryNew(pos.X + 1, pos.Y)
This does create the issue where Results are now everywhere in your code but they don't have to be visible or cause you a lot of typing. This problem is precisely what computation expression libraries e.g. FsToolkit.ErrorHandling are designed to eliminate.
let example =
result {
let! start = Position.TryNew(0,0)
let! next = start |> moveX1
}
There's no way to do this that takes integers as input, but it can be done with a type that only provides integers as output:
type XCoord = X0 | X1 | X2 with
member this.Int =
match this with
| X0 -> 0
| X1 -> 1
| X2 -> 2
static member (-)(xa : XCoord, xb: XCoord) =
xa.Int - xb.Int
type YCoord = Y0 | Y1 | Y2 with
member this.Int =
match this with
| Y0 -> 0
| Y1 -> 1
| Y2 -> 2
static member (-)(ya : YCoord, yb: YCoord) =
ya.Int - yb.Int
type Point = MkPoint of (XCoord * YCoord) with
member this.Ints =
let (MkPoint (x, y)) = this
x.Int, y.Int
static member (-)((MkPoint (ax, ay)), (MkPoint (bx, by))) =
ax - bx, ay - by
let ptA = MkPoint (X0, Y1)
let ptB = MkPoint (X2, Y0)
printfn "%A" ptA.Ints
printfn "%A" ptB.Ints
printfn "%A" (ptA - ptB)
I think this would be pretty clunky to use in practice, but it might work, depending on exactly what you want to do with it. I guess it's a case of "careful what you ask for". 😀

Function works without passing the parameters

/!\ F# brand newbie /!\
I have this code
#r #"..\packages\Accord.3.4.0\lib\net40\Accord.dll"
#r #"..\packages\Accord.Statistics.3.4.0\lib\net40\Accord.Statistics.dll"
#r #"..\packages\Accord.Math.3.4.0\lib\net40\Accord.Math.dll"
open Accord
open Accord.Statistics.Models.Regression.Linear
let input = [|1,1.;2,2.;3,2.25;4,4.75;5,5.|]
let x = input |> Array.map(fun (x,y) -> float x)
let y = input |> Array.map(fun (x,y) -> y)
let regression = SimpleLinearRegression()
let sse = regression.Regress(x,y)
let intercept = regression.Intercept
let slope = regression.Slope
let mse = sse/float x.Length
let rmse = sqrt mse
let r2 = regression.CoefficientOfDetermination(x,y)
Which gives me the result
val input : (int * float) [] = [|(1, 1.0); (2, 2.0); (3, 2.25); (4, 4.75); (5, 5.0)|]
val x : float [] = [|1.0; 2.0; 3.0; 4.0; 5.0|]
val y : float [] = [|1.0; 2.0; 2.25; 4.75; 5.0|]
val regression : SimpleLinearRegression = y(x) = 1,075x + -0,224999999999998
val sse : float = 1.06875
val intercept : float = -0.225
val slope : float = 1.075
val mse : float = 0.21375
val rmse : float = 0.4623310502
val r2 : float = 0.9153465347
How is that possible that SimpleLinearRegression function works but we don't even pass it x and y?
Can you point me to a reference to understand what is this F# magic behind that?
I am guessing you are using the F-Sharp Interactive, and sending all of the code to FSI in one swoop.
The magic is that the code is executed first, then the results are written. Even if the sequence of 'val' outputs get a bit counter intuitive that way.
I can illustrate with this example (for future readers):
let mutable a = 1
let f = a <- 2; fun () -> 3
do a <- f ()
which gives this output in FSI:
val mutable a : int = 3
val f : (unit -> int)
val it : unit = ()
notice how a is updated to its final value before being printed (we never even see 1 and 2, even though that are the values of a after line one and two respectively).

MathNumerics.LinearAlgebra Matrix.mapRows dimensionality issues

So I have verified that the starting version of what I'm trying to do works, but for some reason when putting it into the Matrix.map high order function it breaks down.
Here is the failing function:
let SumSquares (theta:Vector<float>) (y:Vector<float>) (trainingData:Matrix<float>) =
let m = trainingData.RowCount
let theta' = theta.ToRowMatrix()
trainingData
|> Matrix.mapRows(fun a r -> (theta' * r) - y.[a] )
Here are some sample tests
Set up:
let tData = matrix [[1.0; 2.0]
[1.0; 3.0]
[1.0; 3.0]
[1.0; 4.0]]
let yVals = vector [5.0; 6.0; 7.0; 11.0]
let theta = vector [1.0; 0.2]
Test raw functionality of basic operation (theta transpose * vector - actual)
let theta' = theta.ToRowMatrix()
(theta.ToRowMatrix() * tData.[0, 0 .. 1]) - yVals.[0]
Testing in actual function:
tData |> SumSquares theta yVals
Here is a copy/paste of actual error. It reads as though its having issues of me mapping a larger vector to a smaller vector.
Parameter name: target
at MathNet.Numerics.LinearAlgebra.Storage.VectorStorage1.CopyToRow(MatrixStorage1 target, Int32 rowIndex, ExistingData existingData)
at FSI_0061.SumSquares(Vector1 theta, Vector1 y, Matrix`1 trainingData) in C:\projects\deleteme\ASPNet5Test\ConsoleApplication1\ConsoleApplication1\MachineLearning.fsx:line 23
at .$FSI_0084.main#() in C:\projects\deleteme\ASPNet5Test\ConsoleApplication1\ConsoleApplication1\MachineLearning.fsx:line 39
Stopped due to error
I found an even better easier way to do this. I have to credit s952163 for starting me down a good path, but this approach is even more optimized:
let square (x:Vector<float>) = x * x
let subtract (x:Vector<float>) (y:Vector<float>) = y - x
let divideBy (x:float) (y:float) = y / x
let SumSquares (theta:Vector<float>) (y:Vector<float>) (trainingData:Matrix<float>) =
let m = trainingData.RowCount |> float
(trainingData * theta)
|> subtract y
|> square
|> divideBy m
Since you know the number of rows you can just map to that. Arguably this is not pretty:
let SumSquares (theta:Vector<float>) (y:Vector<float>) (trainingData:Matrix<float>) =
let m = trainingData.RowCount
let theta' = theta.ToRowMatrix()
[0..m-1] |> List.map (fun i -> (((theta' * trainingData.[i,0..1]) |> Seq.exactlyOne) - yVals.[i] ))
Edit:
My guess is that mapRows wants everything to be in the same shape, and your output vector is different. So if you want to stick to the Vector type, this will just enumerate the indexed rows:
tData.EnumerateRowsIndexed() |> Seq.map (fun (i,r) -> (theta' * r) - yVals.[i])
and you can also use Matrix.toRowSeqi if you prefer to pipe it through, and get back a Matrix:
tData
|> Matrix.toRowSeqi
|> Seq.map (fun (i,x) -> (theta' * x) - yVals.[i])
|> DenseMatrix.ofRowSeq

F#: why using outer function version faster than pass function as argument

Version (calc1) using direct outer function take about 1s.
But version (calc2) with pass function as parameter of function take about 2s, that is 2x slower. Why?
open System.Diagnostics
open System.Numerics
let width = 1920
let height = 1200
let xMin = -2.0
let xMax = 1.0
let yMin = -1.0
let yMax = 1.0
let scaleX x = float x * (xMax - xMin) / float width + xMin
let scaleY y = float y * (yMax - yMin) / float height - yMax
let fn (z:Complex) (c:Complex) = z * z + c
let calc1 width height =
let iterFn z c =
let rec iterFn' (z:Complex) c n =
if z.Magnitude > 2.0 || n >= 255 then n
else iterFn' (fn z c) c (n + 1)
iterFn' z c 0
Array.Parallel.init (width * height) (fun i ->
let x, y = i % width, i / width
let z, c = Complex.Zero, Complex(scaleX x, scaleY y)
(x, y, iterFn z c)
)
let calc2 width height fn =
let iterFn z c =
let rec iterFn' (z:Complex) c n =
if z.Magnitude > 2.0 || n >= 255 then n
else iterFn' (fn z c) c (n + 1)
iterFn' z c 0
Array.Parallel.init (width * height) (fun i ->
let x, y = i % width, i / width
let z, c = Complex.Zero, Complex(scaleX x, scaleY y)
(x, y, iterFn z c)
)
Execute in F# interactive get the following results:
> calc1 width height |> ignore
Real: 00:00:00.943, CPU: 00:00:03.046, GC gen0: 10, gen1: 8, gen2: 2
val it : unit = ()
> calc2 width height fn |> ignore
Real: 00:00:02.033, CPU: 00:00:07.484, GC gen0: 9, gen1: 8, gen2: 1
val it : unit = ()
F# 4.0.1, .NET 4.6.1
I suspect that in the first case, the fn is inlined.
Passing it as a paramter prevents this optimisation from occuring, so it is slower

Is there generic MultiplyByInt?

There's a generic function LanguagePrimitives.DivideByInt to divide by int without losing generic behavior, we can use it like this:
let inline Divideby2 n = LanguagePrimitives.DivideByInt n 2
val inline Divideby2 :
^a -> ^a when ^a : (static member DivideByInt : ^a * int -> ^a)
But there's no function called MultiplyByInt to perform generic multiplication by int. Is there anything to perform generic multiplication? Like this:
let inline MultiplyBy2 n = SomeGenericFunctionsModule.MultiplybyInt n 2;
P.S. we can always use some non-standard approach like:
let inline MultiplyByInt n m = seq { for i in 1..m -> n} |> Seq.sum
but I'm interested if it is possible to do in the right way.
I'm afraid there's no built-in function, but I can propose two alternative solutions:
type MulExtension = MulExtension of int with
static member (=>) (x:float , MulExtension y) = x * (float y)
static member (=>) (x:decimal, MulExtension y) = x * (decimal y)
static member (=>) (x:int64 , MulExtension y) = x * (int64 y)
// More overloads
let inline MultiplyByInt x y = x => MulExtension y
But you'll have to specify each type.
I would rather use this function:
let inline MultiplyByInt x y =
let fromInt (n:int) : ^a when ^a : (static member (*) : ^a * ^a -> ^a) =
System.Convert.ChangeType(n, typeof<'a>) :?> 'a
x * (fromInt y)
I can't see any difference in performance between both methods.
I managed to get a solution in O(log(N)) which beats yours, but it still feels very ugly
let inline MulInt (m:^t) (n:int) =
let r : ^t ref = ref LanguagePrimitives.GenericZero
let addv : ^t ref= ref LanguagePrimitives.GenericOne
while ((int) !r) < n do
if int(!r + !addv + !addv) < n then
addv := !addv + !addv
else
r := !r + !addv
addv := LanguagePrimitives.GenericOne
!r * m
Using some of the library only features could make this a little better, but would result in warnings.
Note: This solution assumes that n is representable in ^t - i.e.
MulInt 2uy 5000
will loop forever
let inline MultiplyByInt n (x: ^a) =
let zero : ^a = LanguagePrimitives.GenericZero
let one : ^a = LanguagePrimitives.GenericOne
if n = 0 then zero
else
let mutable q, r = System.Math.DivRem(abs n, 2)
let mutable y = x
while q > 0 do
y <- y + y
q <- q / 2
let y = if r = 0 then y else y + x
if n > 0 then y
else y * (zero - one)
I've received an answer from Don Syme (via fsbugs email) when I've asked about missing MutliplyByInt and limited support of DivideByInt:
Don's answer:
This operator exists to support “Seq.averageBy” etc. This represents pseudo-precise division of the total by the count. We didn’t extend the mechanism beyond what was needed for that.
So it looks like I've misunderstood the purpose of this mechanism.

Resources