I have converted the non-linear expression [1]: https://i.stack.imgur.com/MzzSO.png into linear equations 11, 12 and 13. But when I run my code, I got these errors: "constraint labeling not supported for dimensions with variable size, use named constraints instead", "CPLEX cannot extract expression", "Element "cons12" not defined" and "Invalid initialization expression for element "cons12" ". Can you help me, what should I do? Thanks in advance.
using CPLEX;
//Total nodes number.
range Nodes = 1..9;
//{int} Nodes = {1,2,3,4,5,6,7,8,9};
//................................................................................
//Total links number
//two_directed
tuple edge{
int node_out;
int node_in;
};
{edge} L with node_out, node_in in Nodes = {<1,3>, <3,1>, <2,3>, <3,2>, <3,4>, <4,3>, <3,5>,
<5,3>, <3,6>, <6,3>, <4,5>, <5,4>, <4,6>, <6,4>,
<4,8>, <8,4>, <5,6>, <6,5>, <6,7>, <7,6>, <6,9>,
<9,6>};
{edge} Lout[Nodes] = [{<1,3>},//node1
{<2,3>},//node2
{<3,1>, <3,2>, <3,4>, <3,5>, <3,6>},//node3
{<4,3>, <4,5>, <4,6>, <4,8>},//node4
{<5,3>, <5,4>, <5,6>},//node5
{<6,3>, <6,4>, <6,5>, <6,7>, <6,9>},//node6
{<7,6>},//node7
{<8,4>},//node8
{<9,6>}];//node9
//Flows
tuple cflow{
int origin;
int destination;
}
{cflow} F with origin,destination in Nodes = {<1,2>, <1,3>, <1,4>, <1,5>, <1,6>, <1,7>,
<1,8>, <1,9>, <2,1>, <2,3>, <2,4>, <2,5>, <2,6>, <2,7>, <2,8>, <2,9>,
<3,1>, <3,2>, <3,4>, <3,5>, <3,6>, <3,7>, <3,8>, <3,9>,
<4,1>, <4,2>, <4,3>, <4,5>, <4,6>, <4,7>, <4,8>, <4,9>,
<5,1>, <5,2>, <5,3>, <5,4>, <5,6>, <5,7>, <5,8>, <5,9>,
<6,1>, <6,2>, <6,3>, <6,4>, <6,5>, <6,7>, <6,8>, <6,9>, <7,1>, <7,2>};
float landa_f[f in F]=[0.86, 0.3, 0.75, 0.23, 0.32, 0.4, 0.5, 0.6, 0.22, 0.14,
0.23, 0.42, 0.33, 0.5, 0.62, 0.36, 0.42, 0.35, 0.2, 0.16,
0.33, 0.9, 0.41, 0.51, 0.61, 0.33, 0.42, 0.51, 0.87, 0.96,
0.31, 0.55, 0.91, 0.36, 0.32, 0.72, 0.76, 0.32, 0.45, 0.64,
0.38, 0.71, 0.43, 0.55, 0.53, 0.9, 0.58, 0.97, 0.5, 0.33 ];
{string} V = {"IDS", "DPI", "NAT", "Proxy", "Firewall"};
//MAIN DECISION VARIABLES
dvar int I[v in V][n in Nodes][f in F][j in 1..2] in 0..1; //denotes that an NF instance v
hosted at node n is used by the j-th service on the service chain of flow f.
dvar int IL[l in L][f in F][j in 1..2][n in Nodes] in 0..1;//denotes that link l is used by
flow f to route from the j-th to (j + 1)-th NF service, hosted at node nj and nj+1.
dvar int Y[v in V][n in Nodes];
//Decision variables related with non linear equations
dvar int z[l in L][f in F][j in 1..2][n in Nodes][v in V] in 0..1;
subject to{
//convert non_linear_equations to new linear constraints
forall (f in F, j in 1..2, v in V)
cons11: sum( l in Lout[item(Routes[f],j-1)] ) z[l][f][j][item(Routes[f],j-1)][v] == 1;
forall (f in F, j in 1..2, l in Lout[item(Routes[f],j-1)], v in V) {
cons12: 3 * z[l][f][j][item(Routes[f],j-1)][v] <= ( IL[l][f][j][item(Routes[f],j-1)] +
I[v][item(Routes[f],j-1)][f][j] );
cons13: z[l][f][j][item(Routes[f],j-1)][v] >= ( IL[l][f][j][item(Routes[f],j-1)] +
I[v][item(Routes[f],j-1)][f][j] ) - 2; }
}
Indeed
dvar int x[1..2][1..2];
{int} s[1..2]=[{1,2},{1}];
subject to
{
forall(i in 1..2) forall(j in s[i]) ct:x[i][j]<=0;
}
execute
{
writeln(ct[1][1].UB);
}
does not work but if you write
dvar float x[1..2][1..2];
{int} s[1..2]=[{1,2},{1}];
subject to
{
forall(i in 1..2) forall(j in 1..2:j in s[i]) ct:x[i][j]<=0;
}
execute
{
writeln(ct[1][1].UB);
}
then it works fine
range r=1..2;
dvar float x[1..2][1..2];
constraint ct[r][r];
{int} s[1..2]=[{1,2},{1}];
subject to
{
forall(i in 1..2,j in 1..2)
ct[i][j]= if (j in s[i]) x[1][i]<=0;
}
execute
{
writeln(ct[1][1].UB);
}
works fine too
I have a brute force optimization algorithm with the objective function of the form:
np.clip(x # M, a_min=0, a_max=1) # P
where x is a Boolean decision vector, M is a Boolean matrix/tensor and P is a probability vector. As you can guess, x # M as an inner product can have values higher than 1 where is not allowed as the obj value should be a probability scalar or vector (if M is a tensor) between 0 to 1. So, I have used numpy.clip to fix the x # M to 0 and 1 values. How can I set up a mechanism like clip in cvxpy to achieve the same result? I have spent ours on internet with no lock so I appreciate any hint. I have been trying to use this to replicate clip but it raises Exception: Cannot evaluate the truth value of a constraint or chain constraints, e.g., 1 >= x >= 0. As a side note, since cvxpy cannot handle tensors, I loop through tensor slices with M[s].
n = M.shape[0]
m = M.shape[1]
w = M.shape[2]
max_budget_of_decision_variable = 7
x = cp.Variable(n, boolean=True)
obj = 0
for s in range(m):
for w in range(w):
if (x # M[s])[w] >= 1:
(x # M[s])[w] = 1
obj += x # M[s] # P
objective = cp.Maximize(obj)
cst = []
cst += [cp.sum(y) <= max_budget_of_decision_variable ]
prob = cp.Problem(objective, constraints = cst)
As an example, consider M = np.array([ [1, 0, 0, 1, 1, 0], [0, 0, 1, 0, 1, 0], [1, 1, 1, 0, 1, 0]]) and P = np.array([0.05, 0.15, 0.1, 0.15, 0.5, 0.05]).
I would like to find optimal solution for the problem discribed in the attached figure https://i.stack.imgur.com/W6fhf.jpg
My code is
from scipy.optimize import minimize
def create_constraints(t, tau, ub):
constraints = [{
'type': 'ineq',
'fun': lambda x: x[0] * (-1),
}]
con1 = {
'type': 'ineq',
'fun': lambda x: x[0] - x[1] + tau[0],
}
constraints.append(con1)
con2 = {
'type': 'ineq',
'fun': lambda x: x[1] - ub + tau[1],
}
constraints.append(con2)
return constraints
def f(x):
return (x[0] + x[1]) * (-1)
t0 = np.array([1, 10])
tau0 = [5, 5]
cons = create_constraints(t0, tau0, 30)
res = minimize(f, t0, constraints=cons, method='trust-constr')
with the above parameters for constraints I expect that optimal solution would be close to [20, 25]. However the result of optimization is x: array([ 6.66666669, 18.33333334]). What am I doing wrong? why minimizer does not work as expected?
Any help appreciated.
I found that if I create constraints in this way
def create_constraints_2(t, tau, ub):
constr_f = lambda x: np.array([
x[0] * (-1),
x[0] - x[1] + tau[0],
x[1] - ub + tau[1],
])
return NonlinearConstraint(constr_f, -np.inf, 0)
And then call the functions
cons_2 = create_constraints_2(t0, tau0, 30)
minimize(f, t0, constraints=cons_2, method='trust-constr')
Then it works fine. The output is x: array([19.99974476, 24.99993157])
I am trying to import a Geotiff with multiple bands using Dask and xarray and the following code:
import xarray as xr
chunks = {'x': 15886, 'y': 2400, 'band': 1}
df= xr.open_rasterio('multiband.tif',chunks = chunks)
df
which df looks like:
<xarray.DataArray (band: 6, y: 2400, x: 15886)>
dask.array<open_rasterio-b9dd4de67eb722145cdc7b5a3510e05e<this-array>, shape=(6, 2400, 15886), dtype=uint8, chunksize=(1, 2400, 15886), chunktype=numpy.ndarray>
Coordinates:
* band (band) int32 1 2 3 4 5 6
* y (y) float64 70.0 69.99 69.99 69.99 69.98 ... 60.01 60.01 60.01 60.0
* x (x) float64 -146.2 -146.2 -146.2 -146.2 ... -80.01 -80.0 -80.0
Attributes:
transform: (0.0041666666662862895, 0.0, -146.190219951, 0.0, -0.0...
crs: +init=epsg:4326
res: (0.0041666666662862895, 0.0041666666662862895)
is_tiled: 0
nodatavals: (nan, nan, nan, nan, nan, nan)
scales: (1.0, 1.0, 1.0, 1.0, 1.0, 1.0)
offsets: (0.0, 0.0, 0.0, 0.0, 0.0, 0.0)
AREA_OR_POINT: Area
TIFFTAG_SOFTWARE: HEG-Modis Reprojection Tool Nov 4, 2004
And the bands are stored in a Dask.array. I wonder how can I give names to each band (similar to "data variables" in xarray). Then for example I can get access to each band by:
df['band1name']
Currently what I'm doing to get access to the bands is something like:
df.isel(band=1)
which is not that intuitive.
Thanks
I found a simple way to this for anyone else stumbling upon this issue in the Digital Earth Australia documentation (https://docs.dea.ga.gov.au/notebooks/Frequently_used_code/Opening_GeoTIFFs_NetCDFs.html).
We simply convert our dataarray to a dataset
ds = df.to_dataset('bands')
to get all bands as separate variables.
Afterwards, we rename the variables to the correct names:
ds = ds.rename({i + 1: name for i, name in enumerate(bandnames)}
The i + 1 stems from the beginning of the indexing at 1 in bands and bandnames needs to be given.
If you download your GeoTiffs from GEE, you can get all bandnames in ds.attrs['long_name'].
I am training a logistic regression classification model and trying to compare the results using confusion matrix, and calculating precision, recall, accuracy
code is given below
# logistic regression classification model
clf_lr = sklearn.linear_model.LogisticRegression(penalty='l2', class_weight='balanced')
logistic_fit=clf_lr.fit(TrainX, np.where(TrainY >= delay_threshold,1,0))
pred = clf_lr.predict(TestX)
# print results
cm_lr = confusion_matrix(np.where(TestY >= delay_threshold,1,0), pred)
print("Confusion matrix")
print(pd.DataFrame(cm_lr))
report_lr = precision_recall_fscore_support(list(np.where(TestY >= delay_threshold,1,0)), list(pred), average='micro')
print ("\nprecision = %0.2f, recall = %0.2f, F1 = %0.2f, accuracy = %0.2f\n" % \
(report_lr[0], report_lr[1], report_lr[2], accuracy_score(list(np.where(TestY >= delay_threshold,1,0)), list(pred))))
print(pd.DataFrame(cm_lr.astype(np.float64) / cm_lr.sum(axis=1)))
show_confusion_matrix(cm_lr)
#linear_score = cross_validation.cross_val_score(linear_clf, ArrX, ArrY,cv=10)
#print linear_score
expected results are
Confusion matrix
0 1
0 4303 2906
1 1060 1731
precision = 0.37, recall = 0.62, F1 = 0.47, accuracy = 0.60
0 1
0 0.596893 1.041204
1 0.147038 0.620208
however my outputs are
Confusion matrix
0 1
0 4234 2891
1 1097 1778
precision = 0.60, recall = 0.60, F1 = 0.60, accuracy = 0.60
0 1
0 0.594246 1.005565
1 0.153965 0.618435
how do I get correct results ?
In a 'binary' case like yours (2 classes) you need to use average='binary' instead of average='micro'.
For example:
TestY = [0, 1, 1, 0, 1, 1, 1, 0, 0, 0]
pred = [0, 1, 1, 0, 0, 1, 0, 1, 0, 0]
# print results
cm_lr = metrics.confusion_matrix(TestY, pred)
print("Confusion matrix")
print(pd.DataFrame(cm_lr))
report_lr = metrics.precision_recall_fscore_support(TestY, pred, average='binary')
print ("\nprecision = %0.2f, recall = %0.2f, F1 = %0.2f, accuracy = %0.2f\n" % \
(report_lr[0], report_lr[1], report_lr[2], metrics.accuracy_score(TestY, pred)))
and the output:
Confusion matrix
0 1
0 4 1
1 2 3
precision = 0.75, recall = 0.60, F1 = 0.67, accuracy = 0.70
Binary has a default definition of which class is the positive one (the class with the 1 label).
You can read the differences between all the average option in this link.