I am arguing with my TA aboutt this question. I have researched the answer for XNOR and XOR independently for ANN to set up. So I found the answer for XOR and XNOR respectively are
Y1 = (X1 AND ~X2) OR (~X1 AND X2) for XOR
Y2 = (X1 AND X2) OR (~X1 AND ~X2) for XNOR
Yet can we just do a negation of XOR to get XNOR such as
Y3 = ~ ((X1 AND ~X2) OR (~X1 AND X2))
I haven't found anything that using the implementation above so I could be wrong. If so let me know what you all think or any resources would be helpful. Thank you so much.
Related
Where does the graph of the loss function in machine learning come from?
I am studying about machine learning. I sometimes don't understand models that have been optimized using regularization terms.
In the explanation of regularization, the following figure may appear.
Here is an example of the L1 regularization term. I have assumed that the model has two weight parameters w1, w2. That is, the equation of model y is expressed by the following equation.
y = w1x1 + w2x2
For simplicity, I ignored the bias term.
The red squares represent regularization terms. And the blue ellipses are represents the loss function without the regularization term.
The regularization term is given by
| w1 | ^ q + | w2 | ^ q = r ^ q (r is const.)
Therefore, the equation of the graph at w1> 0 and w2> 0 is expressed as follows.
w2 = (r ^ q-| w1 | ^ q) ^ (1 / q)
By substituting w1 for this equation (q = 0 at Lasso), you can draw a graph of the regularized term.
On the other hand, I could not draw a graph of the loss function. Perhaps you need more than one piece of data to draw this graph. For simplicity, I have assumed that I have only two pieces of data. I define them as (x11, x12, t1), (x21, x22, t2). When the loss function is MSE, it is expressed by the following equation.
Ed = 1/2 * {(t1-w1x11-w2x12) + (t1-w1x21-w2x22)}
If I simplify this, it is expressed as
Ed = a*w1^2 + b*w1 + c*w2^2 + d*w2 + e*w1*w2 + f
Here, a, b, c, d, e, and f are functions represented by all or part of x11, x12, x21, and x22. After finding a, b, c, d, e, and f, I thought that if we substitute w1 for this equation, we could draw a graph of the loss function. However, I cannot draw well.
Is the above understanding correct? Thank you.
To visualize the loss function, Ed which is a function of w1 and w2, we should visualize it as a 3 dimensional plot. For example, you can use Geogebra to visualize a 3 dimensional surface plot.
Here is an example, where a=3, b=-1, c=1, d =-1 , e=2.
The 2D plot that you see is called a countor plot. This link enables you to draw it online.
To draw a contour plot manually, you fix the value of Ed, then you obtained a quadratic equation, after which, as you varies w1, you can solve for your w2, for each w1, you can obtain up to 2 w2 as it is quadratic.
Remark: If you are looking for closed form expression in terms of arbitrary q, that could be more challenging.
Say, I write a simple code with quantifiers as below:
from z3 import *
s = SolverFor("LIA")
x1, y1 = Ints('x1 y1')
s.add(ForAll(x1, Implies(x1>=0, Exists(y1, (y1>x1)))))
print(s.check())
print(s.model())
The result is :
sat
[ ]
Shouldn't this output a value of y1 for which it is satisfiable?
And what would that value be? Note that the value of y1 depends on x1 in your formula, so it wouldn't make sense to print y1 only.
In general, once you go under a universal quantifier z3 can no longer print the value of variables. To do so meaningfully, it would have to print all the prefix universals, and unless all your domains are finite this wouldn't make sense. (Nor it would be useful.)
I a nutshell, I want z3 to verify implications of the form
forall x1, x2, ..., xn. exists y1, ..., yn. P => Q
My difficulty is that I need to obtain the definitions of y1, ..., yn in terms of the variables x1, ..., xn that cause the implication to hold.
For example. Suppose I want to check:
forall x1, x2. exists y. (= x1 (f x2)) => (= x1 (f y))
I would like to understand that by setting y to x2, this implication holds.
For this concrete example, I'm trying to do this by asking z3 to give me a model for:
(declare-fun f (Int) Int)
(forall ((_x Int) (_y Int))(exists ((_b Int)) (=> (= _x (f _y)) (= _x (f _b)))))
However, it is giving me models which instantiate f to
functions that do not make sense in my case. For instance, functions that
map all integers to 0. When I tried to constrain the choice of f by asserting that it is surjective, the solver just timed out.
I'm grateful for any help I can get :)
I was reading about non-parametric kernel density estimation.
http://en.wikipedia.org/wiki/Kernel_density_estimation
For uni-variate where D = 1, we can write like
For Multivariate Kernel density estimation (KDE), more preciously for d=3 and X = (x,y,z) can we write:
Is this technically correct? Can any one help with this?
This is very difficult to do on your own, and you really should do this through some package. Nevertheless, the definition is:
fH(x)= 1 / n \sum{i=1}n KH (x - xi), where
x = (x1, x2, …, xd)T, xi = (xi1, xi2, …, xid)T, i = 1, 2, …, n are d-vectors;
H is the bandwidth (or smoothing) d×d matrix which is symmetric and positive definite;
K is the kernel function which is a symmetric multivariate density;
KH(x) = |H|−1/n K(H−1/2x).
I am trying to find an appropriate neural network structure to learn a function of the following form: F(x1,x2,x3,x4,x5)= a*x1+b*(x2-x4)/(x3-x4) + c*x5.
I am using the matlab's neural network toolbox to create a feedforwardnet, but without any luck.
Is it even possible to learn this kind of function using a neural network?
If yes, what can be an appropriate structure?
If no, are there any other models that can learn this kind of function?
Thanks.
I suggest that you start by preparing a training dataset in which you have the following:
1- Dataset
x1, x6, x5; x6 = (x2 - x4) / (x3 - x4)
2- Target label Y
Y = f(x1, x6, x5); you may assume some values of a,b,c
So, you have 3 input variables or features with one target variable Y.
Then, you define the ANN to have only one single layer (single layer Perceptron) and make sure that the output function is linear.
Finally, train the ANN and give it new values in terms of x1, x5 & x6 and compare
with the actual function.
If I understand correctly, you are trying to estimate the values of a, b, and c. Although the function is not linear with respect to its input, it is linear with respect to a, b, and c. So you should be able to solve your problem with linear regression.
More precisely, if you define x6 = (x2 - x4) / (x3 - x4), then you get F(x1, x5, x6) = a * x1 + b * x6 + c * x5, which is linear.