Function closures with mapslices - closures

In the code snipped below, functions f and g are returning different values. From reading the code, you would expect them to behave the same. I am guessing it is to do with closure of v -> innerprodfn(m, v). How do I do it to get the desired behaviour where f and g return the same values.
type Mat{T<:Number}
data::Matrix{T}
end
innerprodfn{T}(m::Mat{T}, v::Array{T}) = i -> (m.data*v)[i]
innerprodfn{T}(m::Mat{T}, vv::Matrix{T}) = mapslices(v->innerprodfn(m, v), vv, 1)
m = Mat(collect(reshape(0:5, 2, 3)))
v = collect(reshape(0:11, 3, 4))
f = innerprodfn(m, v[:,1])
g = innerprodfn(m, v)[1]
m.data * v
# 10 28 46 64
# 13 40 67 94
[f(1) g(1); f(2) g(2)]
# 10 64
# 13 94

I don't have an explanation for the observed behavior, but on a recent nightly version of Julia one gets the expected result.
On 0.5, a workaround is to use a comprehension:
innerprodfn{T}(m::Mat{T}, vv::Matrix{T}) = [innerprodfn(m, vv[:,i]) for i in indices(vv, 2)]
Of course, this works on 0.6 as well.

Related

Incorrect behaviour of .check() in z3py

Consider a set of constraints F = [a + b > 10, a*a + b + 10 < 50].
When I run it using:
s = Solver()
s.add(F)
s.check()
I get sat solution.
If I run it with:
s = Solver()
s.check(F)
I get an unknown solution. Can someone explain why this is happening?
Let's see:
from z3 import *
a = Int('a')
b = Int('b')
F = [a + b > 10, a*a + b + 10 < 50]
s = Solver()
s.add(F)
print (s.check())
print (s.model())
This prints:
sat
[b = 15, a = -4]
That looks good to me.
Let's try your second variant:
from z3 import *
a = Int('a')
b = Int('b')
F = [a + b > 10, a*a + b + 10 < 50]
s = Solver()
print (s.check(F))
print (s.model())
This prints:
sat
[b = 7, a = 4]
That looks good to me too.
So, I don't know how you're getting the unknown answer. Maybe you have an old version of z3; or you've some other things in your program you're not telling us about.
The important thing to note, however, is that s.add(F); s.check() AND s.check(F) are different operations:
s.add(F); s.check() means: Assert the constraints in F; check that they are satisfiable.
s.check(F) means: Check that all the other constraints are satisfiable, assuming F is. In particular, it does not assert F. (This is important if you do further asserts/checks later on.)
So, in general these two different ways of using check are used for different purposes; and can yield different answers. But in the presence of no other assertions around, you'll get a solution for both, though of course the models might be different.
Aside One reason you can get unknown is in the presence of non-linear constraints. And your a*a+b+10 < 50 is non-linear, since it does have a multiplication of a variable by itself. You can deal with that either by using a bit-vector instead of an Int (if applicable), or using the nonlinear-solver; which can still give you unknown, but might perform better. But just looking at your question as you asked it, z3 is just fine handling it.
To find out what is going on within s.check(F), you can do the following:
from z3 import *
import inspect
a = Int('a')
b = Int('b')
F = [a + b > 10, a*a + b + 10 < 50]
s = Solver()
print (s.check(F))
print (s.model())
source_check = inspect.getsource(s.check)
print(source_check)
The resulting output:
sat
[b = 10, a = 1]
def check(self, *assumptions):
"""Check whether the assertions in the given solver plus the optional assumptions are consistent or not.
>>> x = Int('x')
>>> s = Solver()
>>> s.check()
sat
>>> s.add(x > 0, x < 2)
>>> s.check()
sat
>>> s.model().eval(x)
1
>>> s.add(x < 1)
>>> s.check()
unsat
>>> s.reset()
>>> s.add(2**x == 4)
>>> s.check()
unknown
"""
s = BoolSort(self.ctx)
assumptions = _get_args(assumptions)
num = len(assumptions)
_assumptions = (Ast * num)()
for i in range(num):
_assumptions[i] = s.cast(assumptions[i]).as_ast()
r = Z3_solver_check_assumptions(self.ctx.ref(), self.solver, num, _assumptions)
return CheckSatResult(r)
The semantics of assumptions vs. assertions are discussed here and here. But if have to admit that they are not really clear to me yet.

Adaptation of SHA2 512 gives incorrect results

I am trying to adapt the pure Lua implementation of the SecureHashAlgorithm found here for SHA2 512 instead of SHA2 256. When I try to use the adaptation, it does not give the correct answer.
Here is the adaptation:
--
-- UTILITY FUNCTIONS
--
-- transform a string of bytes in a string of hexadecimal digits
local function str2hexa (s)
local h = string.gsub(s, ".", function(c)
return string.format("%02x", string.byte(c))
end)
return h
end
-- transforms number 'l' into a big-endian sequence of 'n' bytes
--(coded as a string)
local function num2string(l, n)
local s = ""
for i = 1, n do
--most significant byte of l
local remainder = l % 256
s = string.char(remainder) .. s
--remove from l the bits we have already transformed
l = (l-remainder) / 256
end
return s
end
-- transform the big-endian sequence of eight bytes starting at
-- index 'i' in 's' into a number
local function s264num (s, i)
local n = 0
for i = i, i + 7 do
n = n*256 + string.byte(s, i)
end
return n
end
--
-- MAIN SECTION
--
-- FIRST STEP: INITIALIZE HASH VALUES
--(second 32 bits of the fractional parts of the square roots of the first 9th through 16th primes 23..53)
local HH = {}
local function initH512(H)
H = {0x6a09e667f3bcc908, 0xbb67ae8584caa73b, 0x3c6ef372fe94f82b, 0xa54ff53a5f1d36f1, 0x510e527fade682d1, 0x9b05688c2b3e6c1f, 0x1f83d9abfb41bd6b, 0x5be0cd19137e2179}
return H
end
-- SECOND STEP: INITIALIZE ROUND CONSTANTS
--(first 80 bits of the fractional parts of the cube roots of the first 80 primes 2..409)
local k = {
0x428a2f98d728ae22, 0x7137449123ef65cd, 0xb5c0fbcfec4d3b2f, 0xe9b5dba58189dbbc, 0x3956c25bf348b538,
0x59f111f1b605d019, 0x923f82a4af194f9b, 0xab1c5ed5da6d8118, 0xd807aa98a3030242, 0x12835b0145706fbe,
0x243185be4ee4b28c, 0x550c7dc3d5ffb4e2, 0x72be5d74f27b896f, 0x80deb1fe3b1696b1, 0x9bdc06a725c71235,
0xc19bf174cf692694, 0xe49b69c19ef14ad2, 0xefbe4786384f25e3, 0x0fc19dc68b8cd5b5, 0x240ca1cc77ac9c65,
0x2de92c6f592b0275, 0x4a7484aa6ea6e483, 0x5cb0a9dcbd41fbd4, 0x76f988da831153b5, 0x983e5152ee66dfab,
0xa831c66d2db43210, 0xb00327c898fb213f, 0xbf597fc7beef0ee4, 0xc6e00bf33da88fc2, 0xd5a79147930aa725,
0x06ca6351e003826f, 0x142929670a0e6e70, 0x27b70a8546d22ffc, 0x2e1b21385c26c926, 0x4d2c6dfc5ac42aed,
0x53380d139d95b3df, 0x650a73548baf63de, 0x766a0abb3c77b2a8, 0x81c2c92e47edaee6, 0x92722c851482353b,
0xa2bfe8a14cf10364, 0xa81a664bbc423001, 0xc24b8b70d0f89791, 0xc76c51a30654be30, 0xd192e819d6ef5218,
0xd69906245565a910, 0xf40e35855771202a, 0x106aa07032bbd1b8, 0x19a4c116b8d2d0c8, 0x1e376c085141ab53,
0x2748774cdf8eeb99, 0x34b0bcb5e19b48a8, 0x391c0cb3c5c95a63, 0x4ed8aa4ae3418acb, 0x5b9cca4f7763e373,
0x682e6ff3d6b2b8a3, 0x748f82ee5defb2fc, 0x78a5636f43172f60, 0x84c87814a1f0ab72, 0x8cc702081a6439ec,
0x90befffa23631e28, 0xa4506cebde82bde9, 0xbef9a3f7b2c67915, 0xc67178f2e372532b, 0xca273eceea26619c,
0xd186b8c721c0c207, 0xeada7dd6cde0eb1e, 0xf57d4f7fee6ed178, 0x06f067aa72176fba, 0x0a637dc5a2c898a6,
0x113f9804bef90dae, 0x1b710b35131c471b, 0x28db77f523047d84, 0x32caab7b40c72493, 0x3c9ebe0a15c9bebc,
0x431d67c49c100d4c, 0x4cc5d4becb3e42b6, 0x597f299cfc657e2a, 0x5fcb6fab3ad6faec, 0x6c44198c4a475817
}
-- THIRD STEP: PRE-PROCESSING (padding)
local function preprocess(toProcess, len)
--append a single '1' bit
--append K '0' bits, where K is the minimum number >= 0 such that L + 1 + K = 896mod1024
local extra = 128 - (len + 9) % 128
len = num2string(8 * len, 8)
toProcess = toProcess .. "\128" .. string.rep("\0", extra) .. len
assert(#toProcess % 128 == 0)
return toProcess
end
local function rrotate(rot, n)
return (rot >> n) | ((rot << 64 - n))
end
local function digestblock(msg, i, H)
local w = {}
for j = 1, 16 do w[j] = s264num(msg, i + (j - 1)*4) end
for j = 17, 80 do
local v = w[j - 15]
local s0 = rrotate(v, 1) ~ rrotate(v, 8) ~ (v >> 7)
v = w[j - 2]
w[j] = w[j - 16] + s0 + w[j - 7] + ((rrotate(v, 19) ~ rrotate(v, 61)) ~ (v >> 6))
end
local a, b, c, d, e, f, g, h = H[1], H[2], H[3], H[4], H[5], H[6], H[7], H[8]
for i = 1, 80 do
a, b, c, d, e, f, g, h = a , b , c , d , e , f , g , h
local s0 = rrotate(a, 28) ~ (rrotate(a, 34) ~ rrotate(a, 39))
local maj = ((a & b) ~ (a & c)) ~ (b & c)
local t2 = s0 + maj
local s1 = rrotate(e, 14) ~ (rrotate(e, 18) ~ rrotate(e, 41))
local ch = (e & f) ~ (~e & g)
local t1 = h + s1 + ch + k[i] + w[i]
h, g, f, e, d, c, b, a = g, f, e, d + t1, c, b, a, t1 + t2
end
H[1] = (H[1] + a)
H[2] = (H[2] + b)
H[3] = (H[3] + c)
H[4] = (H[4] + d)
H[5] = (H[5] + e)
H[6] = (H[6] + f)
H[7] = (H[7] + g)
H[8] = (H[8] + h)
end
local function finalresult512 (H)
-- Produce the final hash value:
return
str2hexa(num2string(H[1], 8)..num2string(H[2], 8)..num2string(H[3], 8)..num2string(H[4], 8)..
num2string(H[5], 8)..num2string(H[6], 8)..num2string(H[7], 8)..num2string(H[8], 8))
end
-- Returns the hash512 for the given string.
local function hash512 (msg)
msg = preprocess(msg, #msg)
local H = initH512(HH)
-- Process the message in successive 1024-bit (128 bytes) chunks:
for i = 1, #msg, 128 do
digestblock(msg, i, H)
end
return finalresult512(H)
end
Given hash512("a"):
Expect: 1f40fc92da241694750979ee6cf582f2d5d7d28e18335de05abc54d0560e0f5302860c652bf08d560252aa5e74210546f369fbbbce8c12cfc7957b2652fe9a75
Actual: e0b9623f2194cb81f2a62616a183edbe390be0d0b20430cadc3371efc237fa6bf7f8b48311f2fa249131c347fee3e8cde6acfdab286d648054541f92102cfc9c
I know that I am creating a message of the correct bit size (1024 bits) and also working in 1024-bit chunks, or at least I believe I am.
I am not sure if it has to do with the handling of the integers (the standard requires unsigned integers) or whether I made a mistake in one of the utility functions, or both. If it is indeed an issue with the handling of the integers, how would I go about taking care of the problem. I was able to resolve this when working on the 256-bit version of the adaptation by using mod 2^32 when working with numbers in the digestblock method. I attempted to do mod 2^64 and 2^63 with the 512-bit version and it does not correct the problem. I am stumped.
I should mention that I cannot use one of the many library implementations as I am using a sandboxed Lua that does not provide this access, which is why I need a pure lua implementation. Thanks in advance.
Unfortunately, after introducing integers in Lua 5.3 writing scripts for Lua becomes a more complicated task.
You must always think about transformations between integers and floating point numbers.
ALWAYS. Yes, that's boring.
One of your mistakes is an excellent example of this "dark corner of Lua".
local remainder = l % 256
s = string.char(remainder) .. s
--remove from l the bits we have already transformed
l = (l-remainder) / 256
Your value l is initially a 64-bit integer.
After cutting off its first byte l contains (64-8) = 56 bits, but now it's a floating point-number (with 53-bit precision, of course).
Possible solution: use l = l >> 8 or l = l // 256 instead of l = (l-remainder) / 256
Another mistake is using s264num(msg, i + (j - 1) * 4) instead of s264num(msg, i + (j - 1) * 8)
One more mistake is in the following line:
local extra = 128 - (len + 9) % 128
The correct code is
local extra = - (len + 17) % 128 + 8
(Please note that -a%m+b is not the same as b-a%m due to operator precedence)
After fixing these 3 mistakes your code works correctly.

isinf(mu) error in Scipy stats when calling std for exponweib?

I have been getting this error when I call std on a frozen exponweib distribution?
Here is the code:
d = st.exponweib
params = d.fit(y)
arg = params[:-2]
loc = params[-2]
scale = params[-1]
rv1 = d(arg,loc,scale)
print rv1.std()
The parameters after fitting are:
arg: (3.445136651705262, 0.10885378466279112)
loc: 11770.05
scale: 3.87424773976
Here is the error:
ValueError Traceback (most recent call last)
<ipython-input-637-4394814bbb8c> in <module>()
11 rv1 = d(arg,loc,scale)
12
---> 13 print rv1.std()
.../anaconda/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.pyc in std(self)
487
488 def std(self):
--> 489 return self.dist.std(*self.args, **self.kwds)
490
491 def moment(self, n):
.../anaconda/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.pyc in std(self, *args, **kwds)
1259 """
1260 kwds['moments'] = 'v'
-> 1261 res = sqrt(self.stats(*args, **kwds))
1262 return res
1263
.../anaconda/lib/python2.7/site-packages/scipy/stats/_distn_infrastructure.pyc in stats(self, *args, **kwds)
1032 mu = self._munp(1, *goodargs)
1033 mu2 = mu2p - mu * mu
-> 1034 if np.isinf(mu):
1035 # if mean is inf then var is also inf
1036 mu2 = np.inf
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Please let me what is wrong with what I'm doing or how to avoid this.
The exponweib distribution has two required parameters a, c and two optional, loc and scale. When you call d(arg, loc, scale) the result is that arg is interpreted as a, loc is interpreted as c, and scale is interpreted as loc. And since your arg is a tuple of two elements, you end up with a tuple of random variables, neither of which is what you want.
Solution: unpack the tuple: d(*arg, loc, scale). Or even simpler, use
rv1 = d(*params)
which unpacks all the parameters for you, without you having to extract and name them.
By the way, when you want to provide your own loc and scale of a random variable, it's better to pass them as named arguments, like d(3, 5, loc=90, scale=0.3). This avoids the situation you encountered, when some of these parameters get interpreted as something else because you didn't get some argument right. In your example, d(arg, loc=loc, scale=scale) would immediately throw an error, "missing 1 required positional argument: 'c'" instead of taking loc instead of c.

"Bitwise AND" in Lua

I'm trying to translate a code from C to Lua and I'm facing a problem.
How can I translate a Bitwise AND in Lua?
The source C code contains:
if ((command&0x80)==0)
...
How can this be done in Lua?
I am using Lua 5.1.4-8
Implementation of bitwise operations in Lua 5.1 for non-negative 32-bit integers
OR, XOR, AND = 1, 3, 4
function bitoper(a, b, oper)
local r, m, s = 0, 2^31
repeat
s,a,b = a+b+m, a%m, b%m
r,m = r + m*oper%(s-a-b), m/2
until m < 1
return r
end
print(bitoper(6,3,OR)) --> 7
print(bitoper(6,3,XOR)) --> 5
print(bitoper(6,3,AND)) --> 2
Here is a basic, isolated bitwise-and implementation in pure Lua 5.1:
function bitand(a, b)
local result = 0
local bitval = 1
while a > 0 and b > 0 do
if a % 2 == 1 and b % 2 == 1 then -- test the rightmost bits
result = result + bitval -- set the current bit
end
bitval = bitval * 2 -- shift left
a = math.floor(a/2) -- shift right
b = math.floor(b/2)
end
return result
end
usage:
print(bitand(tonumber("1101", 2), tonumber("1001", 2))) -- prints 9 (1001)
Here's an example of how i bitwise-and a value with a constant 0x8000:
result = (value % 65536) - (value % 32768) -- bitwise and 0x8000
In case you use Adobe Lightroom Lua, Lightroom SDK contains LrMath.bitAnd() method for "bitwise AND" operation:
-- x = a AND b
local a = 11
local b = 6
local x = import 'LrMath'.bitAnd(a, b)
-- x is 2
And there are also LrMath.bitOr(a, b) and LrMath.bitXor(a, b) methods for "bitwise OR" and "biwise XOR" operations.
This answer is specifically for Lua 5.1.X
you can use
if( (bit.band(command,0x80)) == 0) then
...
in Lua 5.3.X and onwards it's very straight forward...
print(5 & 6)
hope that helped 😉

Recurrence relation - equal roots of characteristic equation

I have the following problem:
Solve the following recurrence relation, simplifying your final answer
using 'O' notation.
f(0)=3
f(1)=12
f(n)=6f(n-1)-9f(n-2)
We know this is a homogeneous 2nd order relation so we write the characteristic equation: a^2-6a+9=0 and the solutions are a1,2=3.
The problem is when I replace these values I get:
f(n)=c1*3^n+c2*3^n
and using the 2 initial relations I have:
f(0)=c1+c2=3
f(1)=3(c1+c2)=12
which gives me that there no values such that c1 and c2 such that these 2 relation are true.
Am I doing something wrong? Is the way it should be solved different when it comes to identical roots for the characteristic equation?
You can't solve it this way, because your matrix A is not diagonalizable.
However, here is what you get if you use Jordan's normal form instead:
f(n) = 3^{n-1}(3n + 9)
The Jordan matrix and the basis (with notation from wikipedia + Octave) is:
J := [3,1;0,3]
P := [3,4;1,1]
such that PJP^{-1} = A, where
A := [6,-9;1,0]
is your recurrence matrix. Furthermore, the Jordan matrix is almost as good as a diagonal matrix for computing powers:
J^n = 3^(n-1) * [3,n;0,3].
The recurrence is then:
[f(n+1); f(n)] = A^n [12,3] = PJ^nP^-1[12,3] = (<whatever>, 3^(n-1)*(3n+9)).
Here a quick numerical check (Scala, but you can take whatever you want, Octave or I whatever you like):
scala> def f(n: Int): Int = { if (n == 0) 3 else if (n == 1) 12 else (6 * f(n-1) - 9 * f(n-2)) }
f: (n: Int)Int
scala> for (i <- 0 until 20) println(f(i))
3
12
45
162
567
1944
6561
21870
72171
236196
767637
2480058
7971615
25509168
81310473
258280326
817887699
^
scala> def explicit(n: Int): Int = (Math.pow(3, n -1) * (3 * n + 9)).toInt
explicit: (n: Int)Int
scala> for (i <- 0 until 20) println(explicit(i))
3
12
45
162
567
1944
6561
21870
72171
236196
767637
2480058
7971615
25509168
81310473
258280326
817887699

Resources