We are solving our large scale MI optimization problems with Cvxpy and Mosek.
Often times it happens that Mosek consumes higher runtime then our stipulated timeout of two hours.
Is there a way to systematically capture those timeout exception?
Minimum reproducible example:
import cvxpy as cp
import numpy as np
import mosek
m = 15
n = 10
np.random.seed(1)
s0 = np.random.randn(m)
lamb0 = np.maximum(-s0, 0)
s0 = np.maximum(s0, 0)
x0 = np.random.randn(n)
A = np.random.randn(m, n)
b = A # x0 + s0
c = -A.T # lamb0
# Define and solve the CVXPY problem.
x = cp.Variable(n)
prob = cp.Problem(cp.Minimize(c.T#x),
[A # x <= b])
# try:
prob.solve(cp.MOSEK, mosek_params={mosek.dparam.optimizer_max_time: 0.01}) # set verbose=True (to see actual error in solver logs)
# except Timeout exception
# print('Timeout occured')
print(prob.value)
def execute_other_important_stuff():
print("Hello world")
execute_other_important_stuff() # Not executed currently
When MOSEK terminates because of a timeout there will never be any exception - you set a timeout, so terminating at that point is a normal, not an abnormal, situation. I am not sure what "Error" you are referring to.
If you mean something like
Cannot unpack invalid solution: Solution(status=UNKNOWN
then it is not related to the timeout itself, but to the fact that there is no solution available (yet, so the only available "solution" has UNKNOWN status), and CVXPY handles this by throwing an exception. So the question is, has any solution at all to your problem been found after those 2 hours? If yes, it should be returned without problem. If not, you might see the above and I would guess the only way is to catch the ValueError CVXPY happens to throw.
If you were using the native Mosek interface then you could find out why it terminated from various response codes, but I CVXPY does not propagate them.
I am not sure if this is a Cvxpy issue.
However, a general comment is that Mosek cannot check the time limit continuously so it will most likely go over time.
For instance for a SDP it has to compute eigenvalues of a potentially large matrix and the time limit cannot be checked before it is completed.
I have no other insights (other than what ErlingMOSEK suggested) on why it happened, but you can use signals library to force the .solve method to stop.
This is an independent way to make sure you can stop the function after whenever time you wise.
See example in Timeout a function call
Related
Let's assume that we have the following files:
func.smt
(declare-datatypes (T) ((AVL leafA (nodeA (val T) (alt Int) (izq AVL) (der AVL)))))
espec.smt
(declare-const t (AVL Int))
And the following code:
from z3 import *
s = Solver()
s.from_file("func.smt")
s.from_file("espec.smt")
When instruction "s.from_file("espec.smt")" is executed, z3 throws the next exception:
z3.z3types.Z3Exception: b'(error "line 1 column 18: unknown sort \'AVL\'")\n'
It seems that the Solver "s" doesn't save the information of datatypes (and functions too). That is why he throws the exception (I guess).
The following code is a way to solve it:
s = Solver()
file = open("func.smt", "r")
func = file.read()
file.close()
file = open("espec.smt", "r")
espec = file.read()
file.close()
s.from_string(func + espec)
But this way it's not efficient.
Is there another more efficient way to solve this and make z3 save datatypes and functions for future asserts and declarations?
Edit:
In my real case for example, the file "func.smt" has a total of 54 declarations between functions and datatypes (some quite complex). The "spec.smt" file has few declarations and asserts. And finally I have a file in which there are many asserts which I have to read little by little and generate a model.
That is, imagine that I have 600 lines of asserts, and I read from 10 to 10, so every 10 lines I generate a model. That is, if we assume that those asserts that I have read are stored in a variable called "aux", every time I want to get a new model, I will have to do "s.from_string (func + spec + aux)" 60 times in total, and I don't know if that could be made more efficient.
Separately reading the files and loading to solver is your best bet, as you found out.
Unless efficiency is of paramount importance, I would not worry about trying to optimize this any further. For any reasonable problem, your solver time will dominate any time spent in loading the assertions from a few (or a few thousand!) files. Do you have a use case that really requires this part to be significantly faster?
I am trying to set "mosek_param" settings, but, am getting errors. For instance, for the following case
MSK_IPAR_INTPNT_SOLVE_FORM
Controls whether the primal or the dual problem is solved.
Default:
"FREE"
Accepted:
"FREE", "PRIMAL", "DUAL"
Example:
param.MSK_IPAR_INTPNT_SOLVE_FORM = 'MSK_SOLVE_FREE'
Groups:
Interior-point method
from https://docs.mosek.com/9.0/toolbox/parameters.html --> I tried
prob.solve(solver=MOSEK,
mosek_params={mosek.iparam.intpnt_solve_form: mosek.solve.primal}, # mosek.iparam.presolve_use:mosek.presolvemode.off
verbose=True)
but, run into errors .... the commented part works.
When I was working in Matlab --> using
cvx_solver_settings('MSK_IPAR_INTPNT_SOLVE_FORM','MSK_SOLVE_PRIMAL')
worked well for me. But, doesn't work in the present case. Also, I was able to set precision as follows
cvx_precision low
but, cannot do so now. Is there another way to do both of these in cvxpy? thank you.
PS: this question has also been posted in the CVXPY forum --> https://groups.google.com/forum/#!topic/cvxpy/MEAewGMlqjI
Below is an example
# Solves a bounded least-squares problem.
import mosek
from cvxpy import *
import numpy
# Problem data.
m = 10
n = 5
numpy.random.seed(1)
A = numpy.random.randn(m, n)
b = numpy.random.randn(m)
# Construct the problem
x = Variable(n)
objective = Minimize(sum_squares(A*x - b))
constraints = [0 <= x, x <= 1]
prob = Problem(objective, constraints)
prob.solve(solver=MOSEK,
mosek_params={mosek.iparam.intpnt_solve_form: mosek.solve.primal}, # mosek.iparam.presolve_use:mosek.presolvemode.off
verbose=True)
which gives me the error. I also tried using 'MSK_IPAR_INTPNT_SOLVE_FORM' = 'MSK_SOLVE_PRIMAL' but to no avail.
Look at
https://docs.mosek.com/9.1/pythonapi/parameters.html#mosek.iparam.intpnt_solve_form
The correct form is
mosek.solveform.primal
I am trying to improve my loop computation speed by using foreach, but there is a simple Rcpp function I defined inside of this loop. I saved the Rcpp function as mproduct.cpp, and I call out the function simply using
sourceCpp("mproduct.cpp")
and the Rcpp function is a simple one, which is to perform matrix product in C++:
// [[Rcpp::depends(RcppArmadillo, RcppEigen)]]
#include <RcppArmadillo.h>
#include <RcppEigen.h>
// [[Rcpp::export]]
SEXP MP(const Eigen::Map<Eigen::MatrixXd> A, Eigen::Map<Eigen::MatrixXd> B){
Eigen::MatrixXd C = A * B;
return Rcpp::wrap(C);
}
So, the function in the Rcpp file is MP, referring to matrix product. I need to perform the following foreach loop (I have simplified the code for illustration):
foreach(j=1:n, .package='Rcpp',.noexport= c("mproduct.cpp"),.combine=rbind)%dopar%{
n=1000000
A<-matrix(rnorm(n,1000,1000))
B<-matrix(rnorm(n,1000,1000))
S<-MP(A,B)
return(S)
}
Since the size of matrix A and B are large, it is why I want to use foreach to alleviate the computational cost.
However, the above code does not work, since it provides me error message:
task 1 failed - "NULL value passed as symbol address"
The reason I added .noexport= c("mproduct.cpp") is to follow some suggestions from people who solved similar issues (Can't run Rcpp function in foreach - "NULL value passed as symbol address"). But somehow this does not solve my issue.
So I tried to install my Rcpp function as a library. I used the following code:
Rcpp.package.skeleton('mp',cpp_files = "<my working directory>")
but it returns me a warning message:
The following packages are referenced using Rcpp::depends attributes however are not listed in the Depends, Imports or LinkingTo fields of the package DESCRIPTION file: RcppArmadillo, RcppEigen
so when I tried to install my package using
install.packages("<my working directory>",repos = NULL,type='source')
I got the warning message:
Error in untar2(tarfile, files, list, exdir, restore_times) :
incomplete block on file
In R CMD INSTALL
Warning in install.packages :
installation of package ‘C:/Users/Lenovo/Documents/mproduct.cpp’ had non-zero exit status
So can someone help me out how to solve 1) using foreach with Rcpp function MP, or 2) install the Rcpp file as a package?
Thank you all very much.
The first step would be making sure that you are optimizing the right thing. For me, this would not be the case as this simple benchmark shows:
set.seed(42)
n <- 1000
A<-matrix(rnorm(n*n), n, n)
B<-matrix(rnorm(n*n), n, n)
MP <- Rcpp::cppFunction("SEXP MP(const Eigen::Map<Eigen::MatrixXd> A, Eigen::Map<Eigen::MatrixXd> B){
Eigen::MatrixXd C = A * B;
return Rcpp::wrap(C);
}", depends = "RcppEigen")
bench::mark(MP(A, B), A %*% B)[1:5]
#> # A tibble: 2 x 5
#> expression min median `itr/sec` mem_alloc
#> <bch:expr> <bch:tm> <bch:tm> <dbl> <bch:byt>
#> 1 MP(A, B) 277.8ms 278ms 3.60 7.63MB
#> 2 A %*% B 37.4ms 39ms 22.8 7.63MB
So for me the matrix product via %*% is several times faster than the one via RcppEigen. However, I am using Linux with OpenBLAS for matrix operations while you are on Windows, which often means reference BLAS for matrix operations. It might be that RcppEigen is faster on your system. I am not sure how difficult it is for Windows user to get a faster BLAS implementation (https://csgillespie.github.io/efficientR/set-up.html#blas-and-alternative-r-interpreters might contain some pointers), but I would suggest spending some time on investigating this.
Now if you come to the conclusion that you do need RcppEigen or RcppArmadillo in your code and want to put that code into a package, you can do the following. Instead of Rcpp::Rcpp.package.skeleton() use RcppEigen::RcppEigen.package.skeleton() or RcppArmadillo::RcppArmadillo.package.skeleton() to create a starting point for a package based on RcppEigen or RcppArmadillo, respectively.
I am trying to use the dask client to parallelize my compute. When I run df.compute() I get the correct output (though it is very slow), but when I run the same thing after setting up a client, I get the following error:
distributed.protocol.pickle - INFO - Failed to serialize <function part at 0x7fd5186ed730>. Exception: can't pickle _thread.RLock objects
Here is my code, in the first df.compute(), I get the expected result, in the second I do not.
#dask.delayed
def part(x):
lower, upper = x
q = "SELECT id,tfidf_vec,emb_vec FROM document_table"
lines=man.session.execute(q)
counter = lower
df = []
for line in lines:
df.append(line)
counter += 1
if counter == upper:
break
return pd.DataFrame(df)
parts = [part(x) for x in [[0,100000],[100000,200000]]]
df = dd.from_delayed(parts)
df.compute()
from dask.distributed import Client
client = Client('127.0.0.1:8786')
df.compute()
Your function contains a reference to man.session, which is part of the function closure. When you use the default scheduler, threads, the object can be shared between the threads that execute your code. When you use the distributed scheduler, the function must be serialised and sent to workers in difference process(es).
You should make a function which creates the session object on each invocation, as was suggested as an answer to your very similar question.
When I run this code (in a programming assignment for Coursera):
J = 1/m * [-y.*log(sigmoid((theta)'*X))-(1-y).*log(1-sigmoid((theta)'*X))]
where m = length(y), y is an m-dimensional vector, X is an m*2 matrix, and theta = 0.1, Octave outputs:
g =
[long (#rows)*2 matrix, each entry <1 but extremely close to 1]
g =
[another long (#rows)*2 matrix as before]
J =
[(#rows)*2 matrix with entries such as 3.4932e-002 and 7.8914e-005]
What is g? I never defined it, and it does not appear in my code, yet is outputted with some seemingly unrelated numbers? (I know that the function itself may have problems, but that is a separate issue from what I'm interested in here. I figured that if I know what g is, I might be able to troubleshoot better. If you have any comments on the function, please don't hesitate to point out what's wrong.)
Whenever you have a statement (inside a function or otherwise) which is not terminated with a semicolon, the output of that statement will display on the terminal.
Assuming that this is the only code you're running, then my guess is that inside your sigmoid function there is a statement of this kind:
g = dosomething() % note: not semicolon terminated!
resulting in terminal output during its execution.
The fact that g is reported twice in the terminal also makes sense, since you are calling the sigmoid function twice in that expression you just wrote.
Also, for the sake of clarity, please do not refer to your one-liner as a function, since that means something entirely different in the context of programming.