I have a brute force optimization algorithm with the objective function of the form:
np.clip(x # M, a_min=0, a_max=1) # P
where x is a Boolean decision vector, M is a Boolean matrix/tensor and P is a probability vector. As you can guess, x # M as an inner product can have values higher than 1 where is not allowed as the obj value should be a probability scalar or vector (if M is a tensor) between 0 to 1. So, I have used numpy.clip to fix the x # M to 0 and 1 values. How can I set up a mechanism like clip in cvxpy to achieve the same result? I have spent ours on internet with no lock so I appreciate any hint. I have been trying to use this to replicate clip but it raises Exception: Cannot evaluate the truth value of a constraint or chain constraints, e.g., 1 >= x >= 0. As a side note, since cvxpy cannot handle tensors, I loop through tensor slices with M[s].
n = M.shape[0]
m = M.shape[1]
w = M.shape[2]
max_budget_of_decision_variable = 7
x = cp.Variable(n, boolean=True)
obj = 0
for s in range(m):
for w in range(w):
if (x # M[s])[w] >= 1:
(x # M[s])[w] = 1
obj += x # M[s] # P
objective = cp.Maximize(obj)
cst = []
cst += [cp.sum(y) <= max_budget_of_decision_variable ]
prob = cp.Problem(objective, constraints = cst)
As an example, consider M = np.array([ [1, 0, 0, 1, 1, 0], [0, 0, 1, 0, 1, 0], [1, 1, 1, 0, 1, 0]]) and P = np.array([0.05, 0.15, 0.1, 0.15, 0.5, 0.05]).
Related
I have a problem where I want to limit the range of a real variable between the maximum and minimum value of another set of real variables.
s = Solver()
y = Real('y')
Z = RealVector('z', 10)
s.add(And(y >= min(Z), y <= max(Z)))
Is there a way to do this in z3py?
You can use Axel's solution; though that one requires you to create an extra variable and also asserts more constraints than needed. Moreover, it doesn't let you use min and max as simple functions. It might be easier to just program this in a functional way, like this:
# Return minimum of a vector; error if empty
def min(vs):
m = vs[0]
for v in vs[1:]:
m = If(v < m, v, m)
return m
# Return maximum of a vector; error if empty
def max(vs):
m = vs[0]
for v in vs[1:]:
m = If(v > m, v, m)
return m
Another difference is that in the functional style we throw an error if the vector is empty. In the other style, the result will essentially be unconstrained. (i.e., min/max can take any value.) You should consider which semantics is right for your application, in case the vector you're passing might be empty. (At the least, you should change it so it prints out a nicer error message. Currently it'll throw an IndexError: list index out of range error if given an empty vector.)
Now you can say:
s = Solver()
y = Real('y')
Z = RealVector('z', 10)
s.add(And(y >= min(Z), y <= max(Z)))
print (s.check())
print (s.model())
This prints:
sat
[z__7 = -1,
z__0 = -7/2,
z__4 = -5/2,
z__5 = -2,
z__3 = -9/2,
z__2 = -4,
z__8 = -1/2,
y = 0,
z__9 = 0,
z__6 = -3/2,
z__1 = -3]
You could benefit from Hakan Kjellerstrand's collection of useful z3py definitions:
from z3 import *
# Functions written by Hakan Kjellerstrand
# http://hakank.org/z3/
# The following can be used by importing http://www.hakank.org/z3/z3_utils_hakank.py
# v is the maximum value of x
def maximum(sol, v, x):
sol.add(Or([v == x[i] for i in range(len(x))])) # v is an element in x)
for i in range(len(x)):
sol.add(v >= x[i]) # and it's the greatest
# v is the minimum value of x
def minimum(sol, v, x):
sol.add(Or([v == x[i] for i in range(len(x))])) # v is an element in x)
for i in range(len(x)):
sol.add(v <= x[i]) # and it's the smallest
s = Solver()
y = Real('y')
zMin = Real('zMin')
zMax = Real('zMax')
Z = RealVector('z', 10)
maximum(s, zMin, Z)
minimum(s, zMax, Z)
s.add(And(y >= zMin, y <= zMax))
print(s.check())
print(s.model())
I am trying to write my own code in Octave for convolution of two discreet signals. But when I compared the output with the in-built conv() function, it is coming different. What am I doing wrong? Here is my code:
clc; clear; close all;
[h, fs] = audioread('sound_h.wav');
h = h(1:10000,1);
[x, fs] = audioread('sound_x.wav');
x = x(1:50000,1);
subplot(4, 1, 1)
plot(x);
title("x[n]");
subplot(4, 1, 2)
plot(h);
title("h[n]");
flip_h = fliplr(h);
len_h = length(h);
len_x = length(x);
padded_x = [zeros(len_h-1,1);x;zeros(len_h-1,1)];
y = zeros(len_x+len_h-1,1);
for i = 1:length(y)
y(i) = sum(padded_x(i:i+len_h-1).*flip_h);
endfor
subplot(4, 1, 3)
plot(y);
title("y[n]");
subplot(4, 1, 4)
plot(conv(h, x));
title("y[n] using conv()");
Here are the plots:
The line
flip_h = fliplr(h);
does nothing, because
h = h(1:10000,1);
is a column vector. You need to use flipud in this case.
I'm trying to implement a function that computes the Relu derivative for each element in a matrix, and then return the result in a matrix. I'm using Python and Numpy.
Based on other Cross Validation posts, the Relu derivative for x is
1 when x > 0, 0 when x < 0, undefined or 0 when x == 0
Currently, I have the following code so far:
def reluDerivative(self, x):
return np.array([self.reluDerivativeSingleElement(xi) for xi in x])
def reluDerivativeSingleElement(self, xi):
if xi > 0:
return 1
elif xi <= 0:
return 0
Unfortunately, xi is an array because x is an matrix. reluDerivativeSingleElement function doesn't work on array. So I'm wondering is there a way to map values in a matrix to another matrix using numpy, like the exp function in numpy?
Thanks a lot in advance.
That's an exercise in vectorization.
This code
if x > 0:
y = 1
elif xi <= 0:
y = 0
Can be reformulated into
y = (x > 0) * 1
This is something that will work for numpy arrays, since boolean expressions involving them are turned into arrays of values of these expressions for elements in said array.
I guess this is what you are looking for:
>>> def reluDerivative(x):
... x[x<=0] = 0
... x[x>0] = 1
... return x
>>> z = np.random.uniform(-1, 1, (3,3))
>>> z
array([[ 0.41287266, -0.73082379, 0.78215209],
[ 0.76983443, 0.46052273, 0.4283139 ],
[-0.18905708, 0.57197116, 0.53226954]])
>>> reluDerivative(z)
array([[ 1., 0., 1.],
[ 1., 1., 1.],
[ 0., 1., 1.]])
Basic function to return derivative of relu could be summarized as follows:
f'(x) = x > 0
So, with numpy that would be:
def relu_derivative(z):
return np.greater(z, 0).astype(int)
def dRelu(z):
return np.where(z <= 0, 0, 1)
Here z is a ndarray in my case.
def reluDerivative(self, x):
return 1 * (x > 0)
You are on a good track: thinking on vectorized operation. Where we define a function, and we apply this function to a matrix, instead of writing a for loop.
This threads answers your question, where it replace all the elements satisfy the condition. You can modify it into ReLU derivative.
https://stackoverflow.com/questions/19766757/replacing-numpy-elements-if-condition-is-met
In addition, python supports functional programming very well, try to use lambda function.
https://www.python-course.eu/lambda.php
This works:
def dReLU(x):
return 1. * (x > 0)
As mentioned by Neil in the comments, you can use heaviside function of numpy.
def reluDerivative(self, x):
return np.heaviside(x, 0)
If you want to use pure Python:
def relu_derivative(x):
return max(sign(x), 0)
If you want it with the derivative you can use:
def relu(neta):
relu = neta * (neta > 0)
d_relu = (neta > 0)
return relu, d_relu
When x is larger than 0, the slope is 1.
When x is smaller than or equal to 0, the slope is 0.
if (x > 0):
return 1
if (x <= 0):
return 0
This can be written more compact:
return 1 * (x > 0)
I understand how and why to use an ImageDataGenerator, but I am interested in casting an eyeball on how the ImageDataGenerator affects my images so I can decide whether I have chosen a good amount of latitude in augmenting my data. I see that I can iterate over the images coming from the generator. I am looking for a way to see whether it's an original image or a modified image, and if the latter what parameters were modified in that particular instance I'm looking at. How/can I see this?
Most of the transformations (except flipping) will always modify the input image. For example, if you've specified rotation_range, from the source code:
theta = np.pi / 180 * np.random.uniform(-self.rotation_range, self.rotation_range)
it's unlikely that the random number will be exactly 0.
There's no convenient way to print out the amount of transformations applied to each image. You have to modify the source code and add some printing functions inside ImageDataGenerator.random_transform().
If you don't want to touch the source code (for example, on a shared machine), you can extend ImageDataGenerator and override random_transform().
import numpy as np
from keras.preprocessing.image import *
class MyImageDataGenerator(ImageDataGenerator):
def random_transform(self, x, seed=None):
# these lines are just copied-and-pasted from the original random_transform()
img_row_axis = self.row_axis - 1
img_col_axis = self.col_axis - 1
img_channel_axis = self.channel_axis - 1
if seed is not None:
np.random.seed(seed)
if self.rotation_range:
theta = np.pi / 180 * np.random.uniform(-self.rotation_range, self.rotation_range)
else:
theta = 0
if self.height_shift_range:
tx = np.random.uniform(-self.height_shift_range, self.height_shift_range) * x.shape[img_row_axis]
else:
tx = 0
if self.width_shift_range:
ty = np.random.uniform(-self.width_shift_range, self.width_shift_range) * x.shape[img_col_axis]
else:
ty = 0
if self.shear_range:
shear = np.random.uniform(-self.shear_range, self.shear_range)
else:
shear = 0
if self.zoom_range[0] == 1 and self.zoom_range[1] == 1:
zx, zy = 1, 1
else:
zx, zy = np.random.uniform(self.zoom_range[0], self.zoom_range[1], 2)
transform_matrix = None
if theta != 0:
rotation_matrix = np.array([[np.cos(theta), -np.sin(theta), 0],
[np.sin(theta), np.cos(theta), 0],
[0, 0, 1]])
transform_matrix = rotation_matrix
if tx != 0 or ty != 0:
shift_matrix = np.array([[1, 0, tx],
[0, 1, ty],
[0, 0, 1]])
transform_matrix = shift_matrix if transform_matrix is None else np.dot(transform_matrix, shift_matrix)
if shear != 0:
shear_matrix = np.array([[1, -np.sin(shear), 0],
[0, np.cos(shear), 0],
[0, 0, 1]])
transform_matrix = shear_matrix if transform_matrix is None else np.dot(transform_matrix, shear_matrix)
if zx != 1 or zy != 1:
zoom_matrix = np.array([[zx, 0, 0],
[0, zy, 0],
[0, 0, 1]])
transform_matrix = zoom_matrix if transform_matrix is None else np.dot(transform_matrix, zoom_matrix)
if transform_matrix is not None:
h, w = x.shape[img_row_axis], x.shape[img_col_axis]
transform_matrix = transform_matrix_offset_center(transform_matrix, h, w)
x = apply_transform(x, transform_matrix, img_channel_axis,
fill_mode=self.fill_mode, cval=self.cval)
if self.channel_shift_range != 0:
x = random_channel_shift(x,
self.channel_shift_range,
img_channel_axis)
if self.horizontal_flip:
if np.random.random() < 0.5:
x = flip_axis(x, img_col_axis)
if self.vertical_flip:
if np.random.random() < 0.5:
x = flip_axis(x, img_row_axis)
# print out the trasformations applied to the image
print('Rotation:', theta / np.pi * 180)
print('Height shift:', tx / x.shape[img_row_axis])
print('Width shift:', ty / x.shape[img_col_axis])
print('Shear:', shear)
print('Zooming:', zx, zy)
return x
I just add 5 prints at the end of the function. Other lines are copied and pasted from the original source code.
Now you can use it with, e.g.,
gen = MyImageDataGenerator(rotation_range=15,
width_shift_range=0.1,
height_shift_range=0.1,
zoom_range=0.5)
flow = gen.flow_from_directory('data', batch_size=1)
img = next(flow)
and see information like this printed on your terminal:
Rotation: -9.185074669096467
Height shift: 0.03791625365979884
Width shift: -0.08398553078553198
Shear: 0
Zooming: 1.40950509832 1.12895574928
I'm trying to use Pybrain to predict sequences of characters belonging to the Reber grammar.
Concretely what I'm doing is generating strings using the Reber grammar graph (you can check it here : http://www.felixgers.de/papers/phd.pdf page 22). An example of such string could be BPVVE. I want my neural network to learn the underlying rules of the grammar. For each of these string I create a sequence that would typically look like this :
[B, T, S, X, P, V, E,] , [B, T, S, X, P, V, E,]
B -> value = [1, 0, 0, 0, 0, 0, 0,] , target = [0, 0, 0, 0, 1, 0, 0,]
P -> value = [0, 0, 0, 0, 1, 0, 0,] , target = [0, 0, 0, 0, 0, 1, 0,]
V -> value = [0, 0, 0, 0, 0, 1, 0,] , target = [0, 0, 0, 0, 0, 1, 0,]
V -> value = [0, 0, 0, 0, 0, 1, 0,] , target = [0, 0, 0, 0, 0, 0, 1,]
E -> E is ignored for now because it marks the end
as you can see the value is just a 7-d vector representing the current letter and the target is the next letter in the Reber word.
Here is the code I'm trying to run :
#!/usr/bin/python
import reberGrammar as reber
import random as rnd
from pylab import *
from pybrain.supervised import RPropMinusTrainer
from pybrain.supervised import BackpropTrainer
from pybrain.datasets import SequenceClassificationDataSet
from pybrain.structure.modules import LSTMLayer, SoftmaxLayer
from pybrain.tools.validation import testOnSequenceData
from pybrain.tools.shortcuts import buildNetwork
def reberToListInt(word): #e.g. "BPVVE" -> [0,4,3,3,5]
out = [None]*len(word)
for i,l in enumerate(word):
if l == 'B':
out[i] = 0
elif l == 'T':
out[i] = 1
elif l == 'S':
out[i] = 2
elif l == 'V':
out[i] = 3
elif l == 'P':
out[i] = 4
elif l == 'E':
out[i] = 5
else :
out[i] = 6
return out
def buildReberDataSet(numSample):
"""Generate a 7 class dataset"""
reberLexicon = reber.ReberGrammarLexicon(numSample)
DS = SequenceClassificationDataSet(7, 7, nb_classes=7)
for rw in reberLexicon.lexicon:
DS.newSequence()
rw2 = reberToListInt(rw)
for i in range(len(rw2)-1): #inserting one letter at a time
inpt = outpt = [0.0]*7
inpt[rw2[i]]=1.0
outpt[rw2[i+1]]=1.0
DS.addSample(inpt,outpt)
return DS
def printDataSet(DS, numLines): #just to print some stat
print "\t############"
print "Number of sequences: ",DS.getNumSequences()
print "Input and output dimensions: ", DS.indim,"\t", DS.outdim
print "\n"
for i in range(numLines):
for inp, target in DS.getSequenceIterator(i):
print inp,
print "\n"
print "\t#############"
'''Dataset creation / split into training and test sets'''
fullDS = buildReberDataSet(700)
tstdata, trndata = fullDS.splitWithProportion( 0.25 )
trndata._convertToOneOfMany( bounds=[0.,1.])
tstdata._convertToOneOfMany( bounds=[0.,1.])
#printDataSet(trndata,2)
'''Network setup / training'''
rnn = buildNetwork( trndata.indim, 7, trndata.outdim, hiddenclass=LSTMLayer, outclass=SoftmaxLayer, outputbias=False, recurrent=True)
trainer = RPropMinusTrainer( rnn, dataset=trndata, verbose=True )
#trainer = BackpropTrainer( rnn, dataset=trndata, verbose=True, momentum=0.9, learningrate=0.5 )
trainError=[]
testError =[]
#errors = trainer.trainUntilConvergence()
for i in range(9):
trainer.trainEpochs( 2 )
trainError.append(100. * (1.0-testOnSequenceData(rnn, trndata)))
testError.append(100. * (1.0-testOnSequenceData(rnn, tstdata)))
print "train error: %5.2f%%" % trainError[i], ", test error: %5.2f%%" % testError[i]
plot(trainError)
hold(True)
plot(testError)
show()
I fail to train this net. The errors are fluctuating a lot and there is no real convergence. I would really appreciate some advises on this.
Here is the code I'm using to generate Reber strings :
#!/usr/bin/python
import random as rnd
class ReberGrammarLexicon(object):
lexicon = set() #contain Reber words
graph = [ [(1,'T'), (5,'P')], \
[(1, 'S'), (2, 'X')], \
[(3,'S') ,(5, 'X')], \
[(6, 'E')], \
[(3, 'V'),(2, 'P')], \
[(4, 'V'), (5, 'T')] ] #store the graph
def __init__(self, num, maxSize = 1000): #fill Lexicon with num words
self.maxSize = maxSize
if maxSize < 5:
raise NameError('maxSize too small, require maxSize > 4')
while len(self.lexicon) < num:
word = self.generateWord()
if word != None:
self.lexicon.add(word)
def generateWord(self): #generate one word
c = 2
currentEdge = 0
word = 'B'
while c <= self.maxSize:
inc = rnd.randint(0,len(self.graph[currentEdge])-1)
nextEdge = self.graph[currentEdge][inc][0]
word += self.graph[currentEdge][inc][1]
currentEdge = nextEdge
if currentEdge == 6 :
break
c+=1
if c > self.maxSize :
return None
return word
Thanks,
Best