How to use Z3Py online to determine the DC operating point of a MOSFET amplifier? - z3

Z3Py online is a very powerful tool to determine the DC operating point of a given MOSFET amplifier. With the aim to illustrate that we will consider the following circuit
The parameters of the MOSFET amplifier are:
k = 1.25 * 10^(-4) , V_T = 1.5 , R_D = 10^4 , V = 15
To determine the DC operating point we use the following Z3Py code:
I_D, k, V_GS, V_T, V_DS, V, R_D = Reals('I_D k V_GS V_T V_DS V R_D')
equations = [
I_D == k * (V_GS - V_T)**2, V_GS == V_DS,
V_DS == V - R_D * I_D,
]
print "MOSFET equations:"
print equations
problem = [
k == 1.25*10**(-4),
V_T == 1.5,
R_D == 10**4,
V == 15, V_GS > V_T
]
print "Problem:"
print problem
print "Solution:"
solve(equations + problem)
Using this code online, we obtain the following output
MOSFET equations:
[I_D = k·(V_GS - V_T)2, V_GS = V_DS, V_DS = V - R_D·I_D]
Problem:
[k = 1/8000, V_T = 3/2, R_D = 10000, V = 15, V_GS > V_T]
Solution:
[I_D = 0.0010589410?,
V = 15,
R_D = 10000,
V_T = 3/2,
k = 1/8000,
V_GS = 4.4105890714?,
V_DS = 4.4105890714?]
It can also try it online here.
As we can see Z3PY is able to compute the correct solution I_D , V_GS and V_DS in a very compact and efficient form.
Other example
This problem is solved using the following code
I_D, k, V_GS = Reals('I_D k V_GS')
V_T, V_I, R_S = Reals('V_T V_I R_S')
V_O, V_DD, R_L = Reals('V_O V_DD R_L')
equations = [
I_D == (k/2) * (V_GS - V_T)**2, V_I == V_GS + R_S * I_D,
V_O == V_DD - R_L * I_D,
]
print "MOSFET equations:"
print equations
problem = [
k == 2,
V_T == 2,
V_DD == 145,
V_I == 6.5, R_S == 22, R_L ==82, V_GS > V_T
]
print "Problem:"
print problem
print "Solution:"
solve(equations + problem)
problem1 = [
k == 2,
V_T == 2,
V_DD == 145,
V_I == 6.5 + 1, R_S == 22, R_L ==82, V_GS > V_T
]
print "Problem 1"
print problem1
print "Solution 1:"
solve(equations + problem1)
problem2 = [
k == 2,
V_T == 2,
V_DD == 145,
V_I == 6.5 + 10**(-3), R_S == 22, R_L ==82, V_GS > V_T
]
print "Problem 2"
print problem2
print "Solution 2:"
solve(equations + problem2)
Using this code online we obtain the following output
MOSFET equations:
[I_D = (k/2)·(V_GS - V_T)2, V_I = V_GS + R_S·I_D, V_O = V_DD - R_L·I_D]
Problem:
[k = 2, V_T = 2, V_DD = 145, V_I = 13/2, R_S = 22, R_L = 82, V_GS > V_T]
Solution:
[I_D = 0.1849949805?,
R_L = 82,
R_S = 22,
V_I = 13/2,
V_DD = 145,
V_T = 2,
k = 2,
V_O = 129.8304115963?,
V_GS = 2.4301104282?]
Problem 1
[k = 2, V_T = 2, V_DD = 145, V_I = 15/2, R_S = 22, R_L = 82, V_GS > V_T]
Solution 1:
[I_D = 0.2282823186?,
R_L = 82,
R_S = 22,
V_I = 15/2,
V_DD = 145,
V_T = 2,
k = 2,
V_O = 126.2808498705?,
V_GS = 2.4777889896?]
Problem 2
[k = 2, V_T = 2, V_DD = 145, V_I = 6501/1000, R_S = 22, R_L = 82, V_GS > V_T]
Solution 2:
[I_D = 0.1850381539?,
R_L = 82,
R_S = 22,
V_I = 6501/1000,
V_DD = 145,
V_T = 2,
k = 2,
V_O = 129.8268713797?,
V_GS = 2.4301606140?]
It can also try it online here.
Other example
This problem is solved using the following code
I_D, k, V_GS = Reals('I_D k V_GS')
V_T, V_EQ, R_S = Reals('V_T V_EQ R_S')
V_DS, V_DD, R_D = Reals('V_DS V_DD R_D')
equations = [
I_D == (k/2) * (V_GS - V_T)**2, V_EQ == V_GS + R_S * I_D,
V_DS == V_DD - (R_D + R_S )* I_D,
]
print "MOSFET equations:"
print equations
problem = [
k == 25*10**(-6),
V_T == 1,
V_DD == 10,
V_EQ == 4, R_S == 39*10**3, R_D == 75*10**3, V_GS > V_T
]
print "Problem:"
print problem
print "Solution:"
solve(equations + problem)
Using this code online we obtain the following output
MOSFET equations:
[I_D = (k/2)·(V_GS - V_T)2, V_EQ = V_GS + R_S·I_D, V_DS = V_DD - (R_D + R_S)·I_D]
Problem:
[k = 1/40000, V_T = 1, V_DD = 10, V_EQ = 4, R_S = 39000, R_D = 75000, V_GS > V_T]
Solution:
[I_D = 0.0000343918?,
R_D = 75000,
R_S = 39000,
V_EQ = 4,
V_DD = 10,
V_T = 1,
k = 1/40000,
V_DS = 6.0793307846?,
V_GS = 2.6587184263?]
It can also try it online here .
Other example:
This problem is solved using the following code
I_D, k, V_GS = Reals('I_D k V_GS')
V_T = Real('V_T')
R_D = Real('R_D')
V_DS, V_DD = Reals('V_DS V_DD')
equations = [
I_D == (k) * (V_GS - V_T - (V_DS) /2)*V_DS, V_DD == V_GS,
V_DS == V_DD - (R_D)* I_D,
]
print "MOSFET equations:"
print equations
problem = [
k == 250*10**(-6),
V_T == 1,
V_DD == 4,
R_D == 1600, V_DS < V_GS - V_T
]
print "Problem:"
print problem
print "Solution:"
solve(equations + problem)
Using this code online we obtain the following output
MOSFET equations:
[I_D = k·(V_GS - V_T - V_DS/2)·V_DS, V_DD = V_GS, V_DS = V_DD - R_D·I_D]
Problem:
[k = 1/4000, V_T = 1, V_DD = 4, R_D = 1600, V_DS < V_GS - V_T]
Solution:
[I_D = 0.0010634763?,
R_D = 1600,
V_DD = 4,
V_T = 1,
k = 1/4000,
V_GS = 4,
V_DS = 2.2984378812?]
It can also try it online here .
Please let me know what do you think about this and if you know a more efficient code for such class of problems. Many thnaks.

Related

Got loss NaN when train YOLOv2 object detection model

Currently, I'm doing research to create object detection model using YOLOv2 from scratch. I have target data with shape (13, 13, 32), where 32 is become from 4 anchor box, 5 prediction value (confidence_score, delta_x, delta_y, delta_w, delta_h) and 3 class probability (my dataset has 3 class).
I'm stuck in a loss function and have built and tried several versions, and still got loss with nan value.
This is the code :
class YOLOLoss(tf.keras.losses.Loss):
def __init__(self, anchor_box, lambda_coord, lambda_noobj, name='yolo_loss'):
super(YOLOLoss, self).__init__(name=name)
self.anchor_box = anchor_box
self.lambda_coord = lambda_coord
self.lambda_noobj = lambda_noobj
cy, cx = tf.meshgrid(tf.range(13), tf.range(13))
self.cell = tf.stack([cx, cy], axis = -1)
self.cell = tf.expand_dims(self.cell, axis = -2)
self.cell = tf.tile(self.cell, [1, 1, 4, 1])
self.cell = tf.cast(self.cell, tf.float32)
def call(self, y_true, y_pred):
y_true = tf.reshape(y_true, (-1, 13, 13, 4, 8))
y_pred = tf.reshape(y_pred, (-1, 13, 13, 4, 8))
true_conf = y_true[..., 0]
pred_conf = y_pred[..., 0]
true_coord = y_true[..., 1:5]
pred_coord = y_pred[..., 1:5]
true_prob = y_true[..., 5:]
pred_prob = y_pred[..., 5:]
objectness = tf.where(true_conf == 1, 1., 0.)
ious = self.iou(true_coord, pred_coord)
coord_loss = tf.reduce_sum(tf.square(pred_coord - true_coord), axis = -1)
coord_loss = self.lambda_coord * tf.reduce_sum(objectness * coord_loss)
object_loss = tf.reduce_sum(objectness * tf.square(tf.math.sigmoid(pred_conf) - ious))
no_object_loss = self.lambda_noobj * tf.reduce_sum((1 - objectness) * tf.square((tf.math.sigmoid(pred_conf) - 0)))
class_loss = tf.reduce_sum(tf.square(tf.nn.softmax(pred_prob) - true_prob), axis = -1)
class_loss = tf.reduce_sum(objectness * class_loss)
print('')
print(tf.math.sigmoid(pred_conf))
print(f'COORDINATE LOSS\t: {coord_loss.numpy()}')
print(f'OBJECT LOSS\t: {object_loss.numpy()}')
print(f'NO OBJECT LOSS\t: {no_object_loss.numpy()}')
print(f'CLASS LOSS\t: {class_loss.numpy()}')
print('')
total_loss = coord_loss + object_loss + no_object_loss + class_loss
return total_loss
def convXY(self, delta_xy):
xy_grid = delta_xy * self.anchor_box + 0.5
xy = 32 * xy_grid + 32 * self.cell
return tf.round(xy)
def convWH(self, delta_wh):
wh_grid = self.anchor_box * tf.math.exp(delta_wh)
wh = wh_grid * 32
return tf.round(wh)
def iou(self, true_coord, pred_coord):
true_delta_xy = true_coord[..., :2]
pred_delta_xy = pred_coord[..., :2]
true_delta_wh = true_coord[..., 2:]
pred_delta_wh = pred_coord[..., 2:]
true_xy = self.convXY(true_delta_xy)
true_wh = self.convWH(true_delta_wh)
pred_xy = self.convXY(pred_delta_xy)
pred_wh = self.convWH(pred_delta_wh)
x1, y1 = true_xy[..., 0], true_xy[..., 1]
w1, h1 = true_wh[..., 0], true_wh[..., 1]
x2, y2 = pred_xy[..., 0], pred_xy[..., 1]
w2, h2 = pred_wh[..., 0], pred_wh[..., 1]
intersection = tf.math.minimum(x1 + w1, x2 + w2) - tf.math.minimum(x1, x2)
intersection *= tf.math.minimum(y1 + h1, y2 + h2) - tf.math.minimum(y1, y2)
area1 = w1 * h1
area2 = w2 * h2
union = area1 + area2 - intersection
iou = intersection / union
return iou
and this is the Darknet-19 architecture that I created, I don't know if this is correct or not :
input_layer = tf.keras.layers.Input((416, 416, 3))
# Convolution 1
x = tf.keras.layers.Conv2D(32, (3, 3), padding = 'same', name = 'conv_1')(input_layer)
x = tf.keras.layers.BatchNormalization(name = 'norm_1')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
#Maxpool 1
x = tf.keras.layers.MaxPooling2D((2, 2), strides = 2)(x)
# Convolution 2
x = tf.keras.layers.Conv2D(64, (3, 3), padding = 'same', name = 'conv_2')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_2')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
#Maxpool 2
x = tf.keras.layers.MaxPooling2D((2, 2), strides = 2)(x)
# Convolution 3
x = tf.keras.layers.Conv2D(128, (3, 3), padding = 'same', name = 'conv_3')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_3')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 4
x = tf.keras.layers.Conv2D(64, (1, 1), padding = 'same', name = 'conv_4')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_4')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 5
x = tf.keras.layers.Conv2D(128, (3, 3), padding = 'same', name = 'conv_5')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_5')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
#Maxpool 3
x = tf.keras.layers.MaxPooling2D((2, 2), strides = 2)(x)
# Convolution 6
x = tf.keras.layers.Conv2D(256, (3, 3), padding = 'same', name = 'conv_6')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_6')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 7
x = tf.keras.layers.Conv2D(128, (1, 1), padding = 'same', name = 'conv_7')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_7')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 8
x = tf.keras.layers.Conv2D(256, (3, 3), padding = 'same', name = 'conv_8')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_8')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
#Maxpool 4
x = tf.keras.layers.MaxPooling2D((2, 2), strides = 2)(x)
# Convolution 9
x = tf.keras.layers.Conv2D(512, (3, 3), padding = 'same', name = 'conv_9')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_9')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 10
x = tf.keras.layers.Conv2D(256, (1, 1), padding = 'same', name = 'conv_10')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_10')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 11
x = tf.keras.layers.Conv2D(512, (3, 3), padding = 'same', name = 'conv_11')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_11')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 12
x = tf.keras.layers.Conv2D(256, (1, 1), padding = 'same', name = 'conv_12')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_12')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 13
x = tf.keras.layers.Conv2D(512, (3, 3), padding = 'same', name = 'conv_13')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_13')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
#Maxpool 5
x = tf.keras.layers.MaxPooling2D((2, 2), strides = 2)(x)
# Convolution 14
x = tf.keras.layers.Conv2D(1024, (3, 3), padding = 'same', name = 'conv_14')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_14')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 15
x = tf.keras.layers.Conv2D(512, (3, 3), padding = 'same', name = 'conv_15')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_15')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 16
x = tf.keras.layers.Conv2D(1024, (3, 3), padding = 'same', name = 'conv_16')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_16')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 17
x = tf.keras.layers.Conv2D(512, (3, 3), padding = 'same', name = 'conv_17')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_17')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 18
x = tf.keras.layers.Conv2D(1024, (3, 3), padding = 'same', name = 'conv_18')(x)
x = tf.keras.layers.BatchNormalization(name = 'norm_18')(x)
x = tf.keras.layers.LeakyReLU(alpha = 0.1)(x)
# Convolution 19
output = tf.keras.layers.Conv2D(32, (1, 1), padding = 'same', activation = 'linear', name = 'conv_19')(x)
for the optimizer I use SGD like this :
optimizer = tf.keras.optimizers.SGD(
learning_rate = 0.001,
momentum = 0.9,
decay = 0.0005
)
model.compile(
optimizer = optimizer,
loss = YOLOLoss(anchor_box = anchor_grid, lambda_coord = 5.0, lambda_noobj = 0.5),
metrics = ['accuracy'],
run_eagerly = True
)
when I try to train a model, I get nan for loss
enter image description here
Please someone tell me what's wrong with my code. I Appreciate it! :)
You can see the full jupyter notebook right here : https://colab.research.google.com/drive/19T3geZakP2Wc6oaEqCJX_3tHwBMt60qG?usp=sharing
I try to train YOLOv2 object detection model, I hope the model have a good accuracy, but when I try to train the model I got the loss with nan value.

Is it possible to solve for function operations given inputs and outputs?

I have a function that takes 2 integers as input, and outputs an integer.
I have a set of inputs and their known outputs.
Is it possible to figure out what operations the function is applying to the inputs to arrive at the output?
I'm not sure how to even begin modeling such a problem with z3. Any help would be greatly appreciated.
Example data:
f(1, 1) = 1
f(3, 7) = 28
f(3, 2) = 3
f(4, 6) = 56
f(10, 3) = 55
f(x, y) = f(y, x)
Yes you can. But the results are not going to be interesting by any means, unless you at least prescribe some structure on what f looks like.
The obvious way to do this is to declare f as un interpreted-function, and then look at the model constructed for it. But this will be a rudimentary definition of f; that is, it'll satisfy your axioms in the most uninteresting way. Here's how you'd write it:
from z3 import *
f = Function('f', IntSort(), IntSort(), IntSort())
s = Solver()
s.add(f( 1, 1) == 1)
s.add(f( 3, 7) == 28)
s.add(f( 3, 2) == 3)
s.add(f( 4, 6) == 56)
s.add(f(10, 3) == 55)
x, y = Ints('x y')
s.add(ForAll([x, y], f(x, y) == f(y, x)))
print(s.check())
print(s.model())
And here's what z3 prints:
sat
[f = [else ->
If(And(Not(Var(0) == 6),
Not(Var(0) == 4),
Not(Var(0) == 10),
Var(0) == 3,
Not(Var(1) == 6),
Not(Var(1) == 4),
Var(1) == 10),
55,
If(And(Var(0) == 6, Not(Var(1) == 6), Var(1) == 4),
56,
If(And(Not(Var(0) == 6),
Not(Var(0) == 4),
Not(Var(0) == 10),
Not(Var(0) == 3),
Not(Var(0) == 1),
Var(0) == 2,
Not(Var(1) == 6),
Not(Var(1) == 4),
Not(Var(1) == 10),
Var(1) == 3),
3,
If(And(Not(Var(0) == 6),
Not(Var(0) == 4),
Not(Var(0) == 10),
Not(Var(0) == 3),
Not(Var(0) == 1),
Not(Var(0) == 2),
Not(Var(1) == 6),
Not(Var(1) == 4),
Not(Var(1) == 10),
Var(1) == 3),
28,
If(And(Not(Var(0) == 6),
Not(Var(0) == 4),
Var(0) == 10,
Not(Var(1) == 6),
Not(Var(1) == 4),
Not(Var(1) == 10),
Var(1) == 3),
55,
If(And(Not(Var(0) == 6),
Var(0) == 4,
Var(1) == 6),
56,
If(And(Not(Var(0) == 6),
Not(Var(0) == 4),
Not(Var(0) == 10),
Var(0) == 3,
Not(Var(1) == 6),
Not(Var(1) == 4),
Not(Var(1) == 10),
Not(Var(1) == 3),
Not(Var(1) == 1),
Var(1) == 2),
3,
If(And(Not(Var(0) == 6),
Not(Var(0) == 4),
Not(Var(0) == 10),
Var(0) == 3,
Not(Var(1) == 6),
Not(Var(1) == 4),
Not(Var(1) == 10),
Not(Var(1) == 3),
Not(Var(1) == 1),
Not(Var(1) == 2)),
28,
If(And(Not(Var(0) == 6),
Not(Var(0) == 4),
Not(Var(0) == 10),
Not(Var(0) == 3),
Var(0) == 1,
Not(Var(1) == 6),
Not(Var(1) == 4),
Not(Var(1) == 10),
Not(Var(1) == 3),
Var(1) == 1),
1,
12)))))))))]]
The way to read this output is to substitute x for Var(0) and y for Var(1), and take the nested-if-then-elses as your defining clauses of the definition.
While I haven't checked the output line-by-line, I'm sure it is correct; in the sense that it satisfies your requirements perfectly. But I can hear you say "that's not what I really wanted!" And indeed, this is not what you wanted to see as a general/minimal function that satisfies the requirements. The way SMT solvers work, you'll never get a minimal answer unless you describe some skeleton for z3 to fill in.
Note that this is an active research area: How to use (semi-)automated theorem provers, SMT solvers, etc. to "write" code for us. The general area is known as SyGus (Syntax-Guided Synthesis). If you want to learn more, start with https://sygus.org, which contains a general description of the problem and read this paper: https://sygus.org/assets/pdf/FMCAD'13_SyGuS.pdf
A somewhat simplified model with assumed limited range for input variables:
from z3 import *
s = Solver()
# upper limit for an input integer
dimIn = 10+1
vars = [[0 for _ in range(dimIn)] for _ in range(dimIn)]
# define decision variables and constraints
for a in range(dimIn):
for b in range(dimIn):
vars[a][b] = Int('v' + str(a).zfill(2) + str(b).zfill(2))
for a in range(dimIn):
for b in range(dimIn):
x = min(a, b)
y = max(a, b)
v = vars[x][y]
xy = str(x) + ',' + str(y);
# we don't have Python 3.10 with match/case ...
if xy == '1,1':
s.add(v == 1)
elif xy == '3,7':
s.add(v == 28)
elif xy == '2,3':
s.add(v == 3)
elif xy == '4,6':
s.add(v == 56)
elif xy == '3,10':
s.add(v == 55)
s.add(v == vars[y][x])
print(s.check())
m = s.model()
for a in range(dimIn):
for b in range(dimIn):
if a <= b:
r = str(m.eval(vars[a][b], model_completion=True))
print('f(' + str(a) + ',' + str(b) + ') = ' + r)
Resulting output:
sat
f(0,0) = 0
f(0,1) = 0
f(0,2) = 0
f(0,3) = 0
f(0,4) = 0
f(0,5) = 0
f(0,6) = 0
f(0,7) = 0
f(0,8) = 0
f(0,9) = 0
f(0,10) = 0
f(1,1) = 1
f(1,2) = 0
f(1,3) = 0
f(1,4) = 0
f(1,5) = 0
f(1,6) = 0
f(1,7) = 0
f(1,8) = 0
f(1,9) = 0
f(1,10) = 0
f(2,2) = 0
f(2,3) = 3
f(2,4) = 0
f(2,5) = 0
f(2,6) = 0
f(2,7) = 0
f(2,8) = 0
f(2,9) = 0
f(2,10) = 0
f(3,3) = 0
f(3,4) = 0
f(3,5) = 0
f(3,6) = 0
f(3,7) = 28
f(3,8) = 0
f(3,9) = 0
f(3,10) = 55
f(4,4) = 0
f(4,5) = 0
f(4,6) = 56
f(4,7) = 0
f(4,8) = 0
f(4,9) = 0
f(4,10) = 0
f(5,5) = 0
f(5,6) = 0
f(5,7) = 0
f(5,8) = 0
f(5,9) = 0
f(5,10) = 0
f(6,6) = 0
f(6,7) = 0
f(6,8) = 0
f(6,9) = 0
f(6,10) = 0
f(7,7) = 0
f(7,8) = 0
f(7,9) = 0
f(7,10) = 0
f(8,8) = 0
f(8,9) = 0
f(8,10) = 0
f(9,9) = 0
f(9,10) = 0
f(10,10) = 0

CNN for cifar10 dataset in Tensorflow

I am trying to replicate results obtained by a convolutional neural network for CIFAR10 using Tensorflow, however after some epochs (~60 epochs) my performance (accuracy) is around 10%, so I do not if the CNN is well trained?
This code is based on Deep mnist for experts https://www.tensorflow.org/get_started/mnist/pros , however in Cifar10 it does not work
import numpy as np
import tensorflow as tf
def unpickle(file):
import cPickle
fo = open(file, 'rb')
dict = cPickle.load(fo)
fo.close()
return dict
#unpacking training and test data
b1 = unpickle("~/cifar-10-batches-py/data_batch_1")
b2 = unpickle("~/cifar-10-batches-py/data_batch_2")
b3 = unpickle("~/cifar-10-batches-py/data_batch_3")
b4 = unpickle("~/cifar-10-batches-py/data_batch_4")
b5 = unpickle("~/cifar-10-batches-py/data_batch_5")
test = unpickle("~/cifar-10-batches-py/test_batch")
#Preparing test data
test_data = test['data']
test_label = test['labels']
#Preparing training data
train_data = np.concatenate([b1['data'],b2['data'],b3['data'],b4['data'],b5['data']],axis=0)
train_label = np.concatenate([b1['labels'],b2['labels'],b3['labels'],b4['labels'],b5['labels']],axis=0)
#Reshaping data
train_data = np.reshape(train_data,[50000,32,32,3])
test_data = np.reshape(test_data,[10000,32,32,3])
batch_size = 100
image_width = 32
image_height = 32
channels = 3
#Constructing Graph
x = tf.placeholder(tf.float32, [None, image_width, image_height, channels])#Training Data
y = tf.placeholder(tf.int32, [None])
one_hot = tf.one_hot(y,depth=10)#Converting in one hot vectors
#Constructing CNN Layers
def weight_variable(shape):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial)
def bias_variable(shape):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial)
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
def max_pool_2x2(x):
return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
#Given an input tensor of shape [batch, in_height, in_width, in_channels] and a filter / kernel tensor of shape [filter_height, filter_width, in_channels, out_channels], taken from: http://textminingonline.com/dive-into-tensorflow-part-v-deep-mnist
W_conv1 = weight_variable([7, 7, 3, 32])
b_conv1 = bias_variable([32])
h_conv1 = tf.nn.relu(conv2d(x, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
W_conv2 = weight_variable([5, 5, 32, 32])
b_conv2 = bias_variable([32])
h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)
W_conv3 = weight_variable([5, 5, 32, 64])
b_conv3 = bias_variable([64])
h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3) + b_conv3)
#Constructing MLP layers
W_fc1 = weight_variable([8 * 8 * 64, 64])
b_fc1 = bias_variable([64])
h_pool3_flat = tf.reshape(h_conv3, [-1, 8*8*64])
h_fc1 = tf.nn.relu(tf.matmul(h_pool3_flat, W_fc1) + b_fc1)
W_fc2 = weight_variable([64, 10])
b_fc2 = bias_variable([10])
y_conv = tf.nn.softmax(tf.matmul(h_fc1, W_fc2) + b_fc2)
#Computing Cost function
cross_entropy = -tf.reduce_sum(one_hot*tf.log(tf.clip_by_value(y_conv,1e-10,1e20)))
train_step = tf.train.MomentumOptimizer(learning_rate = 0.0001, momentum = 0.9).minimize(cross_entropy)
correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(one_hot,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
init = tf.initialize_all_variables()
sess = tf.Session(config=tf.ConfigProto(intra_op_parallelism_threads=16))
sess.run(init)
epochs = 100
b_per = 0
row = []
for e in range(epochs):
print( "epoch", e)
avg_cost = 0
#foreach batch
for j in range(int(train_data.shape[0]/batch_size)):
subset=range((j*batch_size),((j+1)*batch_size))
data = train_data[subset,:,:,:]
label = train_label[subset]
_,c = sess.run([train_step,cross_entropy], feed_dict={x: data, y: label})
avg_cost += c / data.shape[0]
#print(avg_cost)
b_per = b_per + 1
if b_per%10==0 :
row.append(sess.run(accuracy, feed_dict={x: test_data, y: test_label }))
print(row[-1])
It is wrong in data reshape part! It should be,
# Reshaping data
train_data = train_data.reshape(50000, 3, 32, 32).transpose(
0, 2, 3, 1).astype("uint8")
test_data = test_data.reshape(10000, 3, 32, 32).transpose(
0, 2, 3, 1).astype("uint8")

How to write complicated Vertex Buffers

So I have a particle game, the goal is to have as many as possible. I am working in OpenGL-es 2.0. Each particle is represented with 2 dots and a rectangle, this results in 6 vertexs' per particle.
I have been running into performance issues using traditional separate buffer methods and so I am now trying some new things such as drawing with arrays.
Here is how I generate my buffers: They are basically
Square (XyzhsvaXyzhsvaXyzhsvaXyzhsva) (I am hoping I can get indices working)
Circle (Xyzhsvas) (s is size so I can make it a nice dot)
//Array size is preallocated at beginning
func buildArrays()
{
var circlePos:Int = 0
var rectPos:Int = 0
var indcPos:Int = 0
for obj in particles
{
let pos:Point = obj.position
let lpos:Point = obj.lastPosition
let color:ColorHSV = obj.color
let width:GLfloat = obj.width
let rect = Math.makeRectangle(pos, p2: lpos, w: width)
circles[circlePos] = pos.x
circles[circlePos + 1] = pos.y
circles[circlePos + 2] = 0.0
circles[circlePos + 3] = color.h
circles[circlePos + 4] = color.s
circles[circlePos + 5] = color.v
circles[circlePos + 6] = color.a
circles[circlePos + 7] = width
circles[circlePos + 8] = lpos.x
circles[circlePos + 9] = lpos.y
circles[circlePos + 10] = 0.0
circles[circlePos + 11] = color.h
circles[circlePos + 12] = color.s
circles[circlePos + 13] = color.v
circles[circlePos + 14] = color.a
circles[circlePos + 15] = width
circlePos += 16
squares[rectPos] = rect.0.x
squares[rectPos + 1] = rect.0.y
squares[rectPos + 2] = 0.0
squares[rectPos + 3] = color.h
squares[rectPos + 4] = color.s
squares[rectPos + 5] = color.v
squares[rectPos + 6] = color.a
squares[rectPos + 7] = rect.1.x
squares[rectPos + 8] = rect.1.y
squares[rectPos + 9] = 0.0
squares[rectPos + 10] = color.h
squares[rectPos + 11] = color.s
squares[rectPos + 12] = color.v
squares[rectPos + 13] = color.a
squares[rectPos + 14] = rect.2.x
squares[rectPos + 15] = rect.2.y
squares[rectPos + 16] = 0.0
squares[rectPos + 17] = color.h
squares[rectPos + 18] = color.s
squares[rectPos + 19] = color.v
squares[rectPos + 20] = color.a
squares[rectPos + 21] = rect.3.x
squares[rectPos + 22] = rect.3.y
squares[rectPos + 23] = 0.0
squares[rectPos + 24] = color.h
squares[rectPos + 25] = color.s
squares[rectPos + 26] = color.v
squares[rectPos + 27] = color.a
rectPos += 28
indices[indcPos] = 3
indices[indcPos + 1] = 4
indices[indcPos + 2] = 2
indices[indcPos + 3] = 3
indices[indcPos + 4] = 2
indices[indcPos + 5] = 4
indcPos += 6
}
}
I have been researching this a lot with things about having elements be in powers of 4, and how to use Indices with triangle strips but still draw separate triangles.
Last time I did buffers I had a separate one for each part however this time I am trying to do better. I know how to put this data into a buffer object
let sDraw = GLenum(GL_STATIC_DRAW)
let sType = GLenum(GL_ARRAY_BUFFER)
glGenBuffers(1, &buffer_point)
glBindBuffer(sType, buffer_point)
glBufferData(sType, circles.count * sizeof(GLfloat), circles, sDraw)
However I do not know what to do with this object, how to tell OpenGL what all the separate components work. Nor do I know how to get indices working when the vertex data has color intermingled
I also don't understand the claim someone made that you could get triangle strips to draw separate rectangles somehow with indices.
And I don't understand how OpenGL is supposed to separate each of the pieces of data to get them to my shaders.
Also I don't understand the whole 4 component data thing, or why apple claims I should put random padding before each new vertice (I don't even know how to add two bytes of space)
If anyone could help me with understanding, and solving this that would be appreciated! Any optimization advice is appreciated.

Why is my convolution autoencoder not getting trained properly?

Why is my convolutional autoencoder not converging properly? I have a very simple layer stack.
Encoder: Conv/ReLU(Kernel size: 7x7, stride = 1, padding = 0) => maxPool(kernel size=2x2, stride = 2) => Conv/ReLU(Kernel size: 5x5, stride = 1, padding = 0) => MaxPool(kernel size=2x2, stride = 2)
Decoder: Nearest Neighbour Upsampling => Deconv/ReLU => Nearest Neighbour Upsampling => Deconv/ReLU
Training Images are of size 30x30x1.
I tried to train it with 1000 images over 1000 epoch, but the error (MSE) is still 120.
BATCH_SIZE = 100
IMAGE_SIZE = 30
NUM_CHANNELS = 1
num_images = 1000
def init_weights(shape):
return tf.Variable(tf.random_normal(shape, stddev=0.01))
def encoder(X, w, w2, wd, wd2):
l1a = tf.nn.relu(tf.nn.conv2d(X, w,strides=[1, 1, 1, 1], padding='VALID'))
l1 = tf.nn.max_pool(l1a, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
l2a = tf.nn.relu(tf.nn.conv2d(l1, w2,strides=[1, 1, 1, 1], padding='VALID'))
l2 = tf.nn.max_pool(l2a, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
l1da = tf.image.resize_images(l2, 8, 8, 1, align_corners=False)
output_shapel1d = tf.convert_to_tensor([BATCH_SIZE, 12, 12, 32], dtype=tf.int32);
l1d = tf.nn.relu(tf.nn.conv2d_transpose(l1da, wd, output_shapel1d, strides=[1, 1, 1, 1], padding='VALID'))
l2da = tf.image.resize_images(l1d, 24, 24, 1, align_corners=False)
output_shapel2d = tf.convert_to_tensor([BATCH_SIZE, IMAGE_SIZE, IMAGE_SIZE, NUM_CHANNELS], dtype=tf.int32);
l2d = tf.nn.relu(tf.nn.conv2d_transpose(l2da, wd2, output_shapel2d, strides=[1, 1, 1, 1], padding='VALID'))
return l2d
complete_image = extract_data(0, 1000)
trX = complete_image[0:900]
trY = trX
teX = complete_image[900:1000]
teY = teX
X = tf.placeholder("float", [BATCH_SIZE, IMAGE_SIZE, IMAGE_SIZE, NUM_CHANNELS])
Y = tf.placeholder("float", [BATCH_SIZE, IMAGE_SIZE, IMAGE_SIZE, NUM_CHANNELS])
w = init_weights([7, 7, 1, 32])
w2 = init_weights([5, 5, 32, 64])
wd = init_weights([5, 5, 32, 64])
wd2 = init_weights([7, 7, 1, 32])
py_x = encoder(X, w, w2, wd, wd2)
cost = tf.reduce_mean(tf.squared_difference(py_x, Y, name = None))
train_op = tf.train.RMSPropOptimizer(0.001, 0.9).minimize(cost)
predict_op = py_x;
global_step = tf.Variable(0, name='global_step', trainable=False)
saver = tf.train.Saver()
with tf.Session() as sess:
tf.initialize_all_variables().run()
start = global_step.eval() # get last global_step
print "Start from:", start
if FLAGS.output == "train":
for i in range(start, 500):
training_batch = zip(range(0, num_images - BATCH_SIZE, batch_size),
range(batch_size, num_images - BATCH_SIZE, batch_size))
for start, end in training_batch:
sess.run(train_op, feed_dict={X: trX[start:end], Y: trY[start:end]})
total_epoch_cost += sess.run(cost, feed_dict={X: trX[start:end], Y: trY[start:end]})
avg_epoch_cost = total_epoch_cost/BATCH_SIZE
print "cost during epoch " + `i` + "is ", avg_epoch_cost
I have added the complete code in this gist with slight modifications. I am training this with around 10,000 images, and the error after 488 epochs is 74.8.

Resources