Dividing a DoubleTensor matrix by a DoubleTensor number in torch - lua

I'm trying to update a matrix of real valued numbers in a for loop using torch.Tensor.
Here is what I'd like to do:
-- W and P are of size NxN, r is of size N
delta_W = P * r:view(N, 1) * r:view(1, N) * P -- this is an NxN
denominator = 1 + r:view(1, N) * P * r:view(N, 1) -- this is a number
delta_W = delta_w / denominator -- ## THIS ONE RAISES ERROR ##
W = W + delta_W
Just to be clear:
denom -> [torch.DoubleTensor of size 1x1]
P, delta_W, W -> [torch.DoubleTensor of size 200x200]
The error when I do the division is:
bad argument #2 to '?' (number expected at /usr/local/torch/pkg/torch/generic/TensorOperator.c:145)
I'm a heavy numpy users so I thought "broadcasting" was the issue, therefore I tried simulating it through torch.repeatTensor(denom, N, N) with no luck. If denom is just a number (not a DoubleTensor) everything works fine anyway. Using the element doesn't work either, delta_P / denom[1] gives the same error.
What am I doing wrong?
Edit:
I tried using
denominator = (1 + r:view(1, N) * P * r:view(N, 1)):apply(function(x) return x^(-1) end)
delta_w = delta_w * torch.repeatTensor(denominator, N, N)
which doesn't throw an error but the results are wrong. To see this, try the following:
torch.linspace(0, 3, 4):view(2, 2) * torch.Tensor(2, 2):fill(0.5)

I managed by using apply, repeatTensor and finally cmul for element-wise multiplication
rPr = r:view(1, N) * P * r:view(N, 1)
denominator = (1 + rPr):apply(function(x) return x^(-1) end)
delta_w:cmul(torch.repeatTensor(denominator, N, N))
though I wonder if this can be transferred to GPU with cutorch.

If you do denominator[1][1], you will get a 'number' instead of a 'torch.Tensor'. Then you can just write the division statement normally.
-- W and P are of size NxN, r is of size N
delta_W = P * r:view(N, 1) * r:view(1, N) * P -- this is an NxN
denominator = 1 + r:view(1, N) * P * r:view(N, 1) -- this is a 1x1
delta_W = delta_w / denominator[1][1]
W = W + delta_W
By the way, in the 1st statement did you want one of the P matrices transposed (P:t())?

Related

Performing Laplace transform on non-continuous function in Maxima

I am calculating the following function's Laplace transform, but I get some strange output with limit and realpart etc on (%t5) and (%t6). What I expect is something like a_n = 0 and b_n = -1/(n*%pi).
Why is this happening? Am I defining the function at (%i1) incorrectly? Or is this a limitation of Maxima?
(%i1) f(t) := mod(t, 1);
(%o1) f(t) := mod(t, 1)
(%i2) plot2d(f(t), [t, -2, 2]);
(%o2) [/tmp/maxout1266174.gnuplot_pipes]
(%i3) load(fourie);
(%o3) /usr/share/maxima/5.43.0/share/calculus/fourie.mac
(%i4) fourier(f(t), t, 1);
1
(%t4) a = -
0 2
/
[
(%t5) a = limit I cos(%pi n t) realpart(floor(t)) dt
n t -> - 1 ]
/
/
[
- limit I cos(%pi n t) realpart(floor(t)) dt
t -> 1 ]
/
/
[
(%t6) b = (- limit I sin(%pi n t) realpart(floor(t)) dt)
n t -> 1 ]
/
/
[ 2 sin(%pi n) 2 cos(%pi n)
+ limit I sin(%pi n t) realpart(floor(t)) dt + ------------ - ------------
t -> - 1 ] 2 2 %pi n
/ %pi n
(%o6) [%t4, %t5, %t6]
(%i7)

Is there an equivalent bsxfun in TensorFlow as there in Matlab?

I was trying to translate the following MATLAB code to tensorflow:
WW = sum(W.^2, 1); % ( 1 x D^(l)= sum( (D^(l-1) x D^(l)), 1 )
XX = sum(A.^2, 2); % (M x 1) = sum( (M x D^(l-1)), 2 )
bsxfun(#plus, WW, XX) ; % (M x D^(l)) - (M x D^(l)) = (M x D^(l-1)) * (D^(l-1) x D^(l)) - (M x D^(l))
which is very simple MATLAB code and was wondering if there was an equivalent code in TensorFlow. Ideally, W and/or X should be tf.Variable(init) variables because I'd like to compute the derivatives with respect to each variable.
Tensorflow, like NumPy, does broadcasting.
You can do
WW + XX
and it'll figure out the sizes itself
See the documentation here

How to pass same parameters to multiple functions?

I have a bunch of functions that I want to compute with the same inputs. Is there a better way to see the outputs than the way I chose below?
open MathNet.Numerics.Distributions
// The functions
let EuVanillaPut S0 K T r sigma =
let d1 = (log(S0/K) + (r + sigma ** 2.0 / 2.0) * T)/(sqrt(T)*sigma)
let d2 = d1 - sqrt(T)*sigma
K*exp(-r*T)*Normal.CDF(0.0,1.0,-d2) - S0*Normal.CDF(0.0,1.0,-d1)
let BSMdelta S0 K T r sigma =
let d1 = (log(S0/K) + (r + sigma ** 2.0 / 2.0) * T)/(sqrt(T)*sigma)
Normal.CDF(0.0,1.0,d1)
let BSMgamma S0 K T r sigma =
let d1 = (log(S0/K) + (r + sigma ** 2.0 / 2.0) * T)/(sqrt(T)*sigma)
Normal.PDF(0.0,1.0,d1) / (S0 * sigma * sqrt(T))
let BSMvega S0 K T r sigma =
let d1 = (log(S0/K) + (r + sigma ** 2.0 / 2.0) * T)/(sqrt(T)*sigma)
Normal.PDF(0.0,1.0,d1) * S0 * sqrt(T)
let BSMthetacall S0 K T r sigma =
let d1 = (log(S0/K) + (r + sigma ** 2.0 / 2.0) * T)/(sqrt(T)*sigma)
let d2 = d1 - sqrt(T)*sigma
-S0 * Normal.PDF(0.0,1.0,d1) * sigma / (2.0*sqrt(T)) - r*K*exp(-r*T)*Normal.CDF(0.0,1.0,d2)
let BSMthetaput S0 K T r sigma =
let d1 = (log(S0/K) + (r + sigma ** 2.0 / 2.0) * T)/(sqrt(T)*sigma)
let d2 = d1 - sqrt(T)*sigma
-S0 * Normal.PDF(0.0,1.0,d1) * sigma / (2.0*sqrt(T)) + r*K*exp(-r*T)*Normal.CDF(0.0,1.0,-d2)
// Calling them all at once on the same inputs
// So ugly! Is there a better way?
(30.0, 25.0, 5.0, 0.02, 0.05)
|> fun (S0, K, T, r, sigma) -> [EuVanillaPut S0 K T r sigma;
BSMdelta S0 K T r sigma;
BSMgamma S0 K T r sigma;
BSMvega S0 K T r sigma;
BSMthetacall S0 K T r sigma;
BSMthetaput S0 K T r sigma]
I'm pretty new to F#, should I make a type for this? Should I be using a different data structure as an input for the functions? Any and all pointers are much appreciated.
As suggested in the comments, one option is to create a list of functions and then use List.map to iterate over all the functions and call them:
let results =
[ EuVanillaPut; BSMdelta; BSMgamma ]
|> List.map (fun f -> f 30.0 25.0 5.0 0.02 0.05)
I suppose you'd then also want to extract the individual results - to do that, you can use pattern matching (but you will get a warning, because the compiler cannot know that the number of elements in the list is correct):
let [euVanillaPut; bsmdelta; bsmgamma] = results
To avoid the warning, you'd have to write:
match results with
| [euVanillaPut; bsmdelta; bsmgamma] -> // all good
| _ -> failwith "This should not happen..."
Alternatively, you could change the function definition to use tuple (or a record):
let EuVanillaPut (S0, K, T, r, sigma) =
let d1 = (log(S0/K) + (r + sigma ** 2.0 / 2.0) * T)/(sqrt(T)*sigma)
let d2 = d1 - sqrt(T)*sigma
K*exp(-r*T)*Normal.CDF(0.0,1.0,-d2) - S0*Normal.CDF(0.0,1.0,-d1)
Then you can define a single tuple to hold the parameters and use it as an argument to multiple functions:
let ps = (30.0, 25.0, 5.0, 0.02, 0.05)
let euVanillaPut = EuVanillaPut ps
let bsmdelta = BSMdelta ps
let bsmgamma = BSMgamma ps
The first approach is a clever trick, but if you are doing this often, then extracting the individual results from the list will be a bit ugly. The second approach is simpler and makes more sense if you have a lot of functions with the same group of parameters.

Torch - Randomly choose dimension of tensor to slice

I have a torch tensor of size (1 x n x n x n) and I would like to randomly choose one of the last 3 dimensions to randomly slice at s and then do. For example it could output the below tensors with equal probability;
(1 x s x n x n)
(1 x n x s x n)
(1 x n x n x s)
I realise I could just do a few if else statements but I am curious if there is a "neater" option using a function like torch.random(1,4) to select the dimension.
assuming that you want to narrow a block of s elements randomly, out of n elements.
Let's use :narrow.
n = 100
s = 20
x = torch.randn(1, n, n, n)
y = x:narrow(torch.random(2, 4), torch.random(1, n - s + 1), s)

Divide a Point in Elliptic Curve Cryptography

I'm using Elliptic Curve to design a security system. P is a point on elliptic curve. The receiver must obtain P using formula k^-1(kP). The receiver does not know P but knows k. I need to compute k^-1(R) where R=kP. How can I do this using Point Multiplication or Point Addition.
I suggest first learning a bit more about ECC (for example, read some of Paar's book and listen to his course at http://www.crypto-textbook.com/) before tackling something this complex. For this particular question, ask yourself: "What does the inverse of k mean?"
Very interesting question you have! I was happy to implement from scratch Python solution for your task, see code at the bottom of my answer.
Each elliptic curve has an integer order q. If we have any point P on curve then it is well known that q * P = Zero, in other words multiplying any point by order q gives zero-point (infinity point).
Multiplying zero (infinity) point by any number gives zero again, i.e. j * Zero = Zero for any integer j. Adding any point P to zero-point gives P, i.e. Zero + P = P.
In our task we have some k such that R = k * P. We can very easily (very fast) compute Modular Inverse of k modulo order q, using for example Extended Euclidean Algorithm.
Inverse of k modulo q by definition is such that k * k^-1 = 1 (mod q), which by definition of modulus is equal k * k^-1 = j * q + 1 for some integer j.
Then k^-1 * R = k^-1 * k * P = (j * q + 1) * P = j * (q * P) + P = j * Zero + P = Zero + P = P. Thus multiplying R by k^-1 gives P, if k^-1 is inverse of k modulo q.
You can read about point addition and multiplication formulas on this Wiki.
Lets now check our formulas in Python programming language. I decided to implement from scratch simple class ECPoint, which implements all curve operations (addition and multiplication), see code below.
We take any ready-made curve, for example most popular 256-bit curve secp256k1, which is used in Bitcoin. Its parameters can be found here (this doc contains many other popular standard curves), also you can read about this specific curve on Bitcoin Wiki Page.
Following code is fully self-contained Python script, doesn't need any external dependencies and modules. You can run it straight away on any computer. ECPoint class implements all curve arithmetics. Function test() does following operations: we take standard secp256k1 params with some base point G, we compute any random point P = random * G, then we generate random k, compute R = k * P, compute modular inverse k^-1 (mod q) by using function modular_inverse() (which uses extended Euclidean algorithm egcd()), compute found_P = k^-1 * R and check that it is equal to P, i.e. check that k^-1 * R == P, print resulting k^-1 * R. All random values are 256-bit.
Try it online!
def egcd(a, b):
# https://en.wikipedia.org/wiki/Extended_Euclidean_algorithm
ro, r, so, s, to, t = a, b, 1, 0, 0, 1
while r != 0:
ro, (q, r) = r, divmod(ro, r)
so, s = s, so - q * s
to, t = t, to - q * t
return ro, so, to
def modular_inverse(a, mod):
# https://en.wikipedia.org/wiki/Modular_multiplicative_inverse
g, s, t = egcd(a, mod)
assert g == 1, 'Value not invertible by modulus!'
return s % mod
class ECPoint:
#classmethod
def Int(cls, x):
return int(x)
#classmethod
def std_point(cls, name):
if name == 'secp256k1':
# https://en.bitcoin.it/wiki/Secp256k1
# https://www.secg.org/sec2-v2.pdf
p = 0xFFFFFFFF_FFFFFFFF_FFFFFFFF_FFFFFFFF_FFFFFFFF_FFFFFFFF_FFFFFFFE_FFFFFC2F
a = 0
b = 7
x = 0x79BE667E_F9DCBBAC_55A06295_CE870B07_029BFCDB_2DCE28D9_59F2815B_16F81798
y = 0x483ADA77_26A3C465_5DA4FBFC_0E1108A8_FD17B448_A6855419_9C47D08F_FB10D4B8
q = 0xFFFFFFFF_FFFFFFFF_FFFFFFFF_FFFFFFFE_BAAEDCE6_AF48A03B_BFD25E8C_D0364141
else:
assert False
return ECPoint(x, y, a, b, p, q)
def __init__(self, x, y, A, B, N, q, *, prepare = True):
if prepare:
N = self.Int(N)
A, B, x, y, q = [self.Int(e) % N for e in [A, B, x, y, q]]
assert (4 * A ** 3 + 27 * B ** 2) % N != 0
assert (y ** 2 - x ** 3 - A * x - B) % N == 0, (
x, y, A, B, N, (y ** 2 - x ** 3 - A * x) % N)
assert N % 4 == 3
assert y == pow(x ** 3 + A * x + B, (N + 1) // 4, N)
self.A, self.B, self.N, self.x, self.y, self.q = A, B, N, x, y, q
def __add__(self, other):
A, N = self.A, self.N
Px, Py, Qx, Qy = self.x, self.y, other.x, other.y
if Px == Qx and Py == Qy:
s = ((Px * Px * 3 + A) * self.inv(Py * 2, N)) % N
else:
s = ((Py - Qy) * self.inv(Px - Qx, N)) % N
x = (s * s - Px - Qx) % N
y = (s * (Px - x) - Py) % N
return ECPoint(x, y, A, self.B, N, self.q, prepare = False)
def __rmul__(self, other):
other = self.Int(other - 1)
r = self
while True:
if other & 1:
r = r + self
if other == 1:
return r
other >>= 1
self = self + self
#classmethod
def inv(cls, a, n):
return modular_inverse(a, n)
def __repr__(self):
return str(dict(x = self.x, y = self.y, A = self.A,
B = self.B, N = self.N, q = self.q))
def __eq__(self, other):
for i, (a, b) in enumerate([
(self.x, other.x), (self.y, other.y), (self.A, other.A),
(self.B, other.B), (self.N, other.N), (self.q, other.q)]):
if a != b:
return False
return True
def test():
import random
bits = 256
P = random.randrange(1 << bits) * ECPoint.std_point('secp256k1')
k = random.randrange(1 << bits)
R = k * P
found_P = modular_inverse(k, R.q) * R
assert found_P == P
print(found_P)
if __name__ == '__main__':
test()
Output:
{
'x': 108051465657467150531748691374311160382608428790397210924352716318223953013557,
'y': 4462548165448905789984443302412298811224817997977472205419179335194291964455,
'A': 0,
'B': 7,
'N': 115792089237316195423570985008687907853269984665640564039457584007908834671663,
'q': 115792089237316195423570985008687907852837564279074904382605163141518161494337
}

Resources