Is there an equivalent bsxfun in TensorFlow as there in Matlab? - machine-learning

I was trying to translate the following MATLAB code to tensorflow:
WW = sum(W.^2, 1); % ( 1 x D^(l)= sum( (D^(l-1) x D^(l)), 1 )
XX = sum(A.^2, 2); % (M x 1) = sum( (M x D^(l-1)), 2 )
bsxfun(#plus, WW, XX) ; % (M x D^(l)) - (M x D^(l)) = (M x D^(l-1)) * (D^(l-1) x D^(l)) - (M x D^(l))
which is very simple MATLAB code and was wondering if there was an equivalent code in TensorFlow. Ideally, W and/or X should be tf.Variable(init) variables because I'd like to compute the derivatives with respect to each variable.

Tensorflow, like NumPy, does broadcasting.
You can do
WW + XX
and it'll figure out the sizes itself
See the documentation here

Related

Incorrect model of max value in Z3Py

I want to find a maximal interval in which an expression e is true for all x. A way to write such a formula should be: Exists d : ForAll x in (-d,d) . e and ForAll x not in (-d,d) . !e.
To get such a d, the formula f in Z3 (looking at the one above) could be the following:
from __future__ import division
from z3 import *
x = Real('x')
delta = Real('d')
s = Solver()
e = And(1/10000*x**2 > 0, 1/5000*x**3 + -1/5000*x**2 < 0)
f = ForAll(x,
And(Implies(And(delta > 0,
-delta < x, x < delta,
x != 0),
e),
Implies(And(delta > 0,
Or(x > delta, x < -delta),
x != 0),
Not(e))
)
)
s.add(Not(f))
s.check()
print s.model()
Which outputs [d = 1/4].
To check it, I set delta = RealVal('1/4'), drop the ForAll quantifier from f and I get x = 1/2. I replace delta with 1/2 and get 3/4, then 7/8 and so on. The bound should be 1. Can I get Z3 to output that immediately?
If you do the math yourself, you can see that the solution is x != 0, x < 1. Or you can simply ask Wolfram Alpha to do it for you. So, there's no such delta.
The issue you're having is that you're asserting:
s.add(Not(f))
This turns the universal quantification on x into an existential; asking z3 to find a delta such that there is some x that fits the bill. (That is, you're negating your whole formula.) Instead, you should do:
s.add(delta > 0, f)
which also makes sure that delta is positive. With that change, z3 will correctly respond:
unsat
(And then you'll get an error for the call to s.model(), you should only call s.model() if the previous call to s.check() returns sat.)

Dividing a DoubleTensor matrix by a DoubleTensor number in torch

I'm trying to update a matrix of real valued numbers in a for loop using torch.Tensor.
Here is what I'd like to do:
-- W and P are of size NxN, r is of size N
delta_W = P * r:view(N, 1) * r:view(1, N) * P -- this is an NxN
denominator = 1 + r:view(1, N) * P * r:view(N, 1) -- this is a number
delta_W = delta_w / denominator -- ## THIS ONE RAISES ERROR ##
W = W + delta_W
Just to be clear:
denom -> [torch.DoubleTensor of size 1x1]
P, delta_W, W -> [torch.DoubleTensor of size 200x200]
The error when I do the division is:
bad argument #2 to '?' (number expected at /usr/local/torch/pkg/torch/generic/TensorOperator.c:145)
I'm a heavy numpy users so I thought "broadcasting" was the issue, therefore I tried simulating it through torch.repeatTensor(denom, N, N) with no luck. If denom is just a number (not a DoubleTensor) everything works fine anyway. Using the element doesn't work either, delta_P / denom[1] gives the same error.
What am I doing wrong?
Edit:
I tried using
denominator = (1 + r:view(1, N) * P * r:view(N, 1)):apply(function(x) return x^(-1) end)
delta_w = delta_w * torch.repeatTensor(denominator, N, N)
which doesn't throw an error but the results are wrong. To see this, try the following:
torch.linspace(0, 3, 4):view(2, 2) * torch.Tensor(2, 2):fill(0.5)
I managed by using apply, repeatTensor and finally cmul for element-wise multiplication
rPr = r:view(1, N) * P * r:view(N, 1)
denominator = (1 + rPr):apply(function(x) return x^(-1) end)
delta_w:cmul(torch.repeatTensor(denominator, N, N))
though I wonder if this can be transferred to GPU with cutorch.
If you do denominator[1][1], you will get a 'number' instead of a 'torch.Tensor'. Then you can just write the division statement normally.
-- W and P are of size NxN, r is of size N
delta_W = P * r:view(N, 1) * r:view(1, N) * P -- this is an NxN
denominator = 1 + r:view(1, N) * P * r:view(N, 1) -- this is a 1x1
delta_W = delta_w / denominator[1][1]
W = W + delta_W
By the way, in the 1st statement did you want one of the P matrices transposed (P:t())?

Torch - Randomly choose dimension of tensor to slice

I have a torch tensor of size (1 x n x n x n) and I would like to randomly choose one of the last 3 dimensions to randomly slice at s and then do. For example it could output the below tensors with equal probability;
(1 x s x n x n)
(1 x n x s x n)
(1 x n x n x s)
I realise I could just do a few if else statements but I am curious if there is a "neater" option using a function like torch.random(1,4) to select the dimension.
assuming that you want to narrow a block of s elements randomly, out of n elements.
Let's use :narrow.
n = 100
s = 20
x = torch.randn(1, n, n, n)
y = x:narrow(torch.random(2, 4), torch.random(1, n - s + 1), s)

How to define piece-wise functions in Z3py

I would like to define a piece-wise (linear) function in Z3py, for example, the function f(x) has the form
f(x) = a*x + b when 0 <= x <= 1
f(x) = exp(c*x) when 1 < x <= 2
f(x) = 1/(1+10^x) when 2 < x <= 3
etc.
where a, b and c are constants.
I guess the z3.If() function will be relevant, but as the number of pieces grows, the expression gets convoluted.
My questions is, does Z3pyprovides the if-else statement, or is there an elegant way to define piece-wise function in Z3py?
Yes, Z3 supports if-then-elses and in Python they can be constructed using the If function. An example from the documentation of If:
>>> x = Int('x')
>>> y = Int('y')
>>> max = If(x > y, x, y)
max = If(x > y, x, y)

Divide a Point in Elliptic Curve Cryptography

I'm using Elliptic Curve to design a security system. P is a point on elliptic curve. The receiver must obtain P using formula k^-1(kP). The receiver does not know P but knows k. I need to compute k^-1(R) where R=kP. How can I do this using Point Multiplication or Point Addition.
I suggest first learning a bit more about ECC (for example, read some of Paar's book and listen to his course at http://www.crypto-textbook.com/) before tackling something this complex. For this particular question, ask yourself: "What does the inverse of k mean?"
Very interesting question you have! I was happy to implement from scratch Python solution for your task, see code at the bottom of my answer.
Each elliptic curve has an integer order q. If we have any point P on curve then it is well known that q * P = Zero, in other words multiplying any point by order q gives zero-point (infinity point).
Multiplying zero (infinity) point by any number gives zero again, i.e. j * Zero = Zero for any integer j. Adding any point P to zero-point gives P, i.e. Zero + P = P.
In our task we have some k such that R = k * P. We can very easily (very fast) compute Modular Inverse of k modulo order q, using for example Extended Euclidean Algorithm.
Inverse of k modulo q by definition is such that k * k^-1 = 1 (mod q), which by definition of modulus is equal k * k^-1 = j * q + 1 for some integer j.
Then k^-1 * R = k^-1 * k * P = (j * q + 1) * P = j * (q * P) + P = j * Zero + P = Zero + P = P. Thus multiplying R by k^-1 gives P, if k^-1 is inverse of k modulo q.
You can read about point addition and multiplication formulas on this Wiki.
Lets now check our formulas in Python programming language. I decided to implement from scratch simple class ECPoint, which implements all curve operations (addition and multiplication), see code below.
We take any ready-made curve, for example most popular 256-bit curve secp256k1, which is used in Bitcoin. Its parameters can be found here (this doc contains many other popular standard curves), also you can read about this specific curve on Bitcoin Wiki Page.
Following code is fully self-contained Python script, doesn't need any external dependencies and modules. You can run it straight away on any computer. ECPoint class implements all curve arithmetics. Function test() does following operations: we take standard secp256k1 params with some base point G, we compute any random point P = random * G, then we generate random k, compute R = k * P, compute modular inverse k^-1 (mod q) by using function modular_inverse() (which uses extended Euclidean algorithm egcd()), compute found_P = k^-1 * R and check that it is equal to P, i.e. check that k^-1 * R == P, print resulting k^-1 * R. All random values are 256-bit.
Try it online!
def egcd(a, b):
# https://en.wikipedia.org/wiki/Extended_Euclidean_algorithm
ro, r, so, s, to, t = a, b, 1, 0, 0, 1
while r != 0:
ro, (q, r) = r, divmod(ro, r)
so, s = s, so - q * s
to, t = t, to - q * t
return ro, so, to
def modular_inverse(a, mod):
# https://en.wikipedia.org/wiki/Modular_multiplicative_inverse
g, s, t = egcd(a, mod)
assert g == 1, 'Value not invertible by modulus!'
return s % mod
class ECPoint:
#classmethod
def Int(cls, x):
return int(x)
#classmethod
def std_point(cls, name):
if name == 'secp256k1':
# https://en.bitcoin.it/wiki/Secp256k1
# https://www.secg.org/sec2-v2.pdf
p = 0xFFFFFFFF_FFFFFFFF_FFFFFFFF_FFFFFFFF_FFFFFFFF_FFFFFFFF_FFFFFFFE_FFFFFC2F
a = 0
b = 7
x = 0x79BE667E_F9DCBBAC_55A06295_CE870B07_029BFCDB_2DCE28D9_59F2815B_16F81798
y = 0x483ADA77_26A3C465_5DA4FBFC_0E1108A8_FD17B448_A6855419_9C47D08F_FB10D4B8
q = 0xFFFFFFFF_FFFFFFFF_FFFFFFFF_FFFFFFFE_BAAEDCE6_AF48A03B_BFD25E8C_D0364141
else:
assert False
return ECPoint(x, y, a, b, p, q)
def __init__(self, x, y, A, B, N, q, *, prepare = True):
if prepare:
N = self.Int(N)
A, B, x, y, q = [self.Int(e) % N for e in [A, B, x, y, q]]
assert (4 * A ** 3 + 27 * B ** 2) % N != 0
assert (y ** 2 - x ** 3 - A * x - B) % N == 0, (
x, y, A, B, N, (y ** 2 - x ** 3 - A * x) % N)
assert N % 4 == 3
assert y == pow(x ** 3 + A * x + B, (N + 1) // 4, N)
self.A, self.B, self.N, self.x, self.y, self.q = A, B, N, x, y, q
def __add__(self, other):
A, N = self.A, self.N
Px, Py, Qx, Qy = self.x, self.y, other.x, other.y
if Px == Qx and Py == Qy:
s = ((Px * Px * 3 + A) * self.inv(Py * 2, N)) % N
else:
s = ((Py - Qy) * self.inv(Px - Qx, N)) % N
x = (s * s - Px - Qx) % N
y = (s * (Px - x) - Py) % N
return ECPoint(x, y, A, self.B, N, self.q, prepare = False)
def __rmul__(self, other):
other = self.Int(other - 1)
r = self
while True:
if other & 1:
r = r + self
if other == 1:
return r
other >>= 1
self = self + self
#classmethod
def inv(cls, a, n):
return modular_inverse(a, n)
def __repr__(self):
return str(dict(x = self.x, y = self.y, A = self.A,
B = self.B, N = self.N, q = self.q))
def __eq__(self, other):
for i, (a, b) in enumerate([
(self.x, other.x), (self.y, other.y), (self.A, other.A),
(self.B, other.B), (self.N, other.N), (self.q, other.q)]):
if a != b:
return False
return True
def test():
import random
bits = 256
P = random.randrange(1 << bits) * ECPoint.std_point('secp256k1')
k = random.randrange(1 << bits)
R = k * P
found_P = modular_inverse(k, R.q) * R
assert found_P == P
print(found_P)
if __name__ == '__main__':
test()
Output:
{
'x': 108051465657467150531748691374311160382608428790397210924352716318223953013557,
'y': 4462548165448905789984443302412298811224817997977472205419179335194291964455,
'A': 0,
'B': 7,
'N': 115792089237316195423570985008687907853269984665640564039457584007908834671663,
'q': 115792089237316195423570985008687907852837564279074904382605163141518161494337
}

Resources