For this problem: http://rise4fun.com/Z3/YNBG
Z3 produces the model:
sat
((s0 FP!val!0))
I was expecting to see a true number as the model. It's almost as if it's treating FP as an uninterpreted sort for this case. Is there a way to get Z3 to produce a real number here?
Thanks for reporting this. Indeed, there was a bug in the model completion for FPA. The fix is already available in the unstable branch at Codeplex.
Note that == (floating point equal) with NaN is always false, i.e., in this example, any s0 satisfies the formula. Such values are now correctly omitted at all (in get-model) or the model is completed with a NaN (for get-value or generally when model completion is enabled).
Related
Is it possible to extract the upper and (or) lower bound of some numerical variables in Z3? Suppose there are some constraints on numerical variable x, and the cause of the constraints is that x must be in the interval [x_min, x_max]. Is there a way in Z3 to extract these bounds (x_min and x_max) (in case the solver calculates these values internally), without doing optimization (minimization and maximization).
You could try to increase Z3's verbosity, maybe you can find bounds in the output.
I doubt it, though: since Z3 is ultimately a SAT solver, any numerical solver that (tries to) decide satisfiability could be applied, but deciding satisfiability doesn't necessary require computing (reasonable) numerical bounds.
Out of curiosity: why would you like to avoid optimisation queries?
In general, no.
The minimum/maximum optimal values of a variable x provide the tightest over-approximation of the satisfiable domain interval of x. This requires enumerating all possible Boolean assignments, not just one.
The (Dual) Simplex Algorithm inside the T-Solver for linear arithmetic keeps track of bounds for all arithmetic variables. However, these bounds are only valid for the (possibly partial) Boolean assignment that is currently being constructed by the SAT engine. In early pruning calls, there is no guarantee whatsoever about the significance of these bounds: the corresponding domain for a given variable x may be an under-approximation, an over-approximation or neither (compared to the domain of x wrt. the input formula).
The Theory Combination approach implemented by a SMT solver can also affect the significance of the bounds available inside the LA-Solver. In this regard, I can vouch that Model-Based Theory Combination can be particularly nasty to deal with. With this approach, the SMT Solver may not generate some interface equalities/inequalities when the T-Solvers agree on the Model Value of an interface variable. However, this is counterproductive when one wants to know from the LA-Solver the valid domain of a variable x because it can provide an over-approximated interval even after finding a model of the input formula for a given total Boolean assignment.
Unless the original problem --after preprocessing-- contains terms of the form (x [<|<=|=|=>|>] K), for all possibly interesting values of K, it is hardly likely that the SMT solver generates any valid T-lemma of this form during the search. The main exception is when x is an Int and the LIA-Solver uses splitting on demand. As a consequence, the Boolean stack is not that much helpful to discover bounds either and, even if they were generated, they would only provide an under-approximation of the feasible interval of x (when they are contained in a satisfiable total Boolean assignment).
I've come up with an SMT formula in Z3 which outputs one solution to a constraint solving problem using only BitVectors and IntVectors of fixed length. The logic I use for the IntVectors is only simple Presburger arithmetic (of the form (x[i] - x[i + 1] <=/>= z) for some x and z). I also take the sum of all of the bits in the bitvector (NOT the binary value), and set that value to be within a range of [a, b].
This works perfectly. The only problem is that, as z3 works by always taking the easiest path towards determining satisfiability, I always get the same answer back, whereas in my domain I'd like to find a variety of substantially different solutions (I know for a fact that multiple, very different solutions exist). I'd like to use this nifty tool I found https://bitbucket.org/kuldeepmeel/weightgen, which lets you uniformly sample a constrained space of possibilities using SAT. To use this though, I need to convert my SMT formula into a SAT formula.
Do you know of any resources that would help me learn how to perform Presburger arithmetic and adding the bits of a bitvector as a SAT instance? Alternatively, do you know of any SMT solver which as an intermediate step outputs a readable description of the problem as a SAT instance?
Many thanks!
[Edited to reflect the fact that I really do need the uniform sampling feature.]
I wonder what is the correct way to calculate average performance over several folds in cross-validation.
For example, I have 5 folds of F1 with values
[0.5 0.3 0.25 null 0.7]
What's is the average F1 of this system?
I could take null as 0 or just output null as an average result.
Alternatively, I can take only defined four values and divide by 4, but this is not correct either, because if there is some system that did 0.1 on this fold, it's performance will be poorer that the one with null, however, 0.1 is much better that null.
It really depends on the context. (In the following I'm including references to numpy just for future references for those using it.)
If the null occurred because the cv-fold was somehow undefined for the problem, then you could ignore it (e.g., by calling np.nanmean. Presumably, for "real life", you just wouldn't have a dataset equivalent to such a fold.
If the null occurred because the predictor utterly failed for this fold, then the result could either be (it's a matter of your interpretation):
nan, because the overall predictor behavior is undefined (in this case, you might just use np.mean).
The average with the worst case possible (indeed 0 for the f1 score), if you'd spot for a given set that it's malfunctioning, and output just some arbitrary result (in this case, you might use np.nan_to_num).
By far, the best thing you could do is figure out the reason for this value, and then eliminate it. This should ideally just never happen, and probably should be considered a bug; before solving the bug, just consider your estimator unsuitable for performance estimation.
I am using the Z3 solver with Python API to tackle a Circuit SAT problem.
It consists of many Xor expressions with up to 21 inputs and three-input And expressions. Z3 is able to solve my smaller examples but does not cope with the bigger ones.
Rather than creating the Solver object with
s = Solver()
I tried to optimize the solver tactics like in
t = Then('simplify', 'symmetry-reduce', 'aig', 'tseitin-cnf', 'sat' )
s = t.solver()
I got the tactics list via describe_tactics()
Unfortunately, my attempts have not been fruitful. The default sequence of tactics seems to do a pretty good job. The tactics tutorial previously available in rise4fun is no longer accessible.
Another attempt - without visible effect - was to set the phase parameter, as I am expecting the majority of my variables to have false values. (cf related post)
set_option("sat.phase", "always-false")
What sequence of tactics is recommended for Circuit SAT problems?
If I compare two floating-point numbers, are there cases where a>=b is not equivalent to b<=a and !(a<b), or where a==b is not equivalent to b==a and !(a!=b)?
In other words: are comparisons always "symmetrical", such that I can get the same result on a comparison by swapping the operands and mirroring the operator? And are they always "negatable", such that negating an operator (e.g. > to <=) is equivalent to to applying a logical NOT (!) to the result?
Assuming IEEE-754 floating-point:
a >= b is always equivalent to b <= a.*
a >= b is equivalent to !(a < b), unless one or both of a or b is NaN.
a == b is always equivalent to b == a.*
a == b is equivalent to !(a != b), unless one or both of a or b is NaN.
More generally: trichotomy does not hold for floating-point numbers. Instead, a related property holds [IEEE-754 (1985) ยง5.7]:
Four mutually exclusive relations are possible: less than, equal, greater than, and unordered. The last case arises when at least one operand is NaN. Every NaN shall compare unordered with everything, including itself.
Note that this is not really an "anomaly" so much as a consequence of extending the arithmetic to be closed in a way that attempts to maintain consistency with real arithmetic when possible.
[*] true in abstract IEEE-754 arithmetic. In real usage, some compilers might cause this to be violated in rare cases as a result of doing computations with extended precision (MSVC, I'm looking at you). Now that most floating-point computation on the Intel architecture is done on SSE instead of x87, this is less of a concern (and it was always a bug from the standpoint of IEEE-754, anyway).
In Python at least a>=b is not equivalent to !(a<b) when there is a NaN involved:
>>> a = float('nan')
>>> b = 0
>>> a >= b
False
>>> not (a < b)
True
I would imagine that this is also the case in most other languages.
Another thing that might surprise you is that NaN doesn't even compare equal to itself:
>>> a == a
False
The set of IEEE-754 floating-point numbers are not ordered so some relational and boolean algebra you are familiar with no longer holds. This anomaly is caused by NaN which has no ordering with respect to any other value in the set including itself so all relational operators return false. This is exactly what Mark Byers has shown.
If you exclude NaN then you now have an ordered set and the expressions you provided will always be equivalent. This includes the infinities and negative zero.
Aside from the NaN issue, which is somewhat analogous to NULL in SQL and missing values in SAS and other statistical packages, there is always the problem of floating point arithmetic accuracy. Repeating values in the fractional part (1/3, for example) and irrational numbers cannot be represented accurately. Floating point arithmatic often truncates results because of the finite limit in precision. The more arithematic you do with a floating point value, the larger the error that creeps in.
Probably the most useful way to compare floating point values would be with an algorithm:
If either value is NaN, all comparisons are false, unless you are explicitly checking for NaN.
If the difference between two numbers is within a certain "fuzz factor", consider them equal. The fuzz factor is your tolerance for accumulated mathematical imprecision.
After the fuzzy equality comparison, then compare for less than or greater than.
Note that comparing for "<=" or ">=" has the same risk as comparison for precise equality.
Nope, not for any sane floating point implementation: basic symmetry and boolean logic applies. However, equality in floating point numbers is tricky in other ways. There are very few cases where testing a==b for floats is the reasonable thing to do.