I obtained several statistics from runs of Z3. I need to understand what these mean.
I am rather rusty and non up to date for the recent developments of sat and SMT solving, for this reason I tried to find explanations myself and I might be dead wrong.
So my questions are mainly:
1) What do the measures' names mean?
2) If wrong, can you give me pointers to understand better to what they refer to?
Other observations are made below and conceptually belong to the two questions above.
Thanks in advance!
My interpretation follows.
DPLL. All the metrics below refer to the jargon of the DPLL algorithm which is the foundation of most solvers.
:decisions
Number of decisions
:propagations
Number of propagations (I guess unit propagations)
:binary-propagations, :ternary-propagations
Propagations of two and three literals at once
:conflicts
Number of conflicts
RESOLUTION. Operations made interpreting clauses as sets, roughly speaking; techniques taken from resolution which is another paradigm for solving SAT.
:subsumed
:subsumption-resolution
What is the difference between the above two?
:dyn-subsumption-resolution
Should be described here: Learning for Dynamic Subsumption, by Hamadi et al.
OTHER TECHNIQUES
:minimized-lits
No clear idea. Is it probably related with clause learning?
:probing-assigned
I guess it counts the number of assignment made when "probing", which I guess is some kind of lookahead technique.
:del-clause
Number of deleted clauses (for what reason? Redundant?)
:elim-literals :elim-clauses :elim-bool-vars :elim-blocked-clauses
Number of entities after the elim- eliminated.
These metrics refer to particular SAT solving techniques
(see for reference Blocked Clause Elimination, by M.Järvisalo et al.)
:restarts
Number of restarts.
OTHER ASPECTS
:mk-bool-var :mk-binary-clause :mk-ternary-clause :mk-clause
Number of boolean variables and binary,ternary and generic clauses created.
:memory
Maximum amount of memory used.
:gc-clause
Garbage-collected clauses ...?
This interpretation is plausible according to my experiments since it's always the case that
:gc-clause <= :del-clause ; in my case the disequality is strict.
It is not always the case that
:gc-clause<=:elim-clauses; it can also be :gc-clause > :elim-clauses
I am afraid this is an open-ended question.
Z3 exposes many counters that are collected in many different ways.
While many capture abstract concepts, their meanings are ultimately
based on implementation behaviors of the code.
Fortunately the source code is available and provides the full context
for understanding the behavior of each counter. So there is no single
document that tracks the meaning of the counters, but the source code
is made available to give the full context.
Related
So, I'm trying to implement the NEAT(Neuroevolution of augmenting topologies) algorithm and have stumbled into a problem. How are networks in species with only one member crossed over?
One solution I came up with is to perform inter-species crossover. But I don't know if it would be effective.
In NEAT, there are four ways in which you can create candidate individuals for the next generation:
Pass an exact copy of an individual
Pass a mutated copy of an individual
Do crossover using two individuals from a given species
Do crossover with two individuals of different species (iter-species)
Of course, you can always do (1). This is often applied to "elites", which may be the best of all, or the best of each species.
You can also always do (2), again to a subset of all individuals or to a subset (random or sorted) within each species.
As you correctly anticipate, (4) is also always a possibility, as long as you do have at least two species (it seems things would be a bit broken otherwise).
Regarding (3) in case you have a species with only one individual? You can't really do it, right?
There are two things that can help in this situation. First, use a mix of 1 to 4 options. The frequency for each option is normally determined using hyperparameters (as well as the frequency for each type of mutation and so on).
But here I would actually reconsider your speciation algorithm. Speciation means separating your population into groups, where hopefully more similar individuals are grouped together. There are different ways in which you can do this, and you can re-examine your species with different frequencies as well (you can reset your species every generation!). It does not seem very efficient if your clustering algorithm (because speciation is a type of clustering) is returning species with one or even zero individuals. So this is where I would actually work!
As a final note, I remember a full NEAT implementation is no basic project. I would recommend not trying to implement this on your own. I think it is a better use of your time to work with a well-established implementation, so you can focus on understanding how things work and how to adapt them for your needs, and not so much on bugs and other implementation details.
I have been searching on whether z3 supports complex numbers and have found the following: https://leodemoura.github.io/blog/2013/01/26/complex.html
The author states that (1) Complex numbers are not yet implemented in Z3 as a built-in (this was written in 2013), and (2) that Complex numbers can be encoded on top of the Real numbers provided by Z3.
The basic idea is to represent a Complex number as a pair of Real numbers. He defines the basic imaginary number with I=(0,1), that is: I means the real part equals 0 and the imaginary part equals 1.
He offers the encoding (I mean, we can test it on our machines), where we can solve the equation x^2+2=0. I received the following result:
sat
x = (-1.4142135623?)*I
The sat result does make sense, since this equation is solvable in the simulation of the theory of complex numbers (as a result of the theory of algebraically closed fields) we have just made. However, the root result does not make sense to me. I mean: what about (1.4142135623?)*I?
I would understand to receive the two roots, but, if only one received, I do not understand why I get the negated solution.
Maybe I misread something or I missed something.
Also, I would like to say if complex numbers have already been implemented built in Z3. I mean, with a standard:
x = Complex("x")
And with tactics of kind of a NCA (from nonlinear complex arithmetic).
I have not seen any reference to this theory in SMT-LIB either.
AFAIK there is no plan to add complex numbers to SMT-LIB. There's a Google group for SMT-LIB and it might make sense to send a post there to see if there is any interest there.
Note, that particular blog post says "find a root"; this is just satisfiability, i.e. it finds one solution, not all of them. (But you can ask for another one by adding an assertion that says x should be different from the first result.)
I am trying to find an optimal solution using the Z3 API for python. I have used set_option("verbose", 1) to print statements that Z3 generates while checking for sat. One of the statements it prints is pb.conflict statements. The statements look something like this -
pb.conflict statements.
I want to know what exactly is pb.conflict. What do these statements signify? Also, what are the two numbers that get printed along with it?
pb stands for Pseudo-boolean. A pseudo-boolean function is a function from booleans to some other domain, usually Real. A conflict happens when the choice of a variable leads to an unsatisfiable clause set, at which point the solver has to backtrack. Keeping the backtracking to a minimum is essential for efficiency, and many of the SAT engines carefully track that number. While the details are entirely solver specific (i.e., those two numbers you're asking about), in general the higher the numbers, the more conflict cases the solver met, and hence might decide to reset the state completely or take some other action. Often, there are parameters that users can set to specify when such actions are taken and exactly what those are. But again, this is entirely solver and implementation specific.
A google search on pseudo-boolean optimization will result in a bunch of scholarly articles that you might want to peruse.
If you really want to find Z3's treatment of pseudo-booleans, then your best bet is probably to look at the implementation itself: https://github.com/Z3Prover/z3/blob/master/src/smt/theory_pb.cpp
I am trying to count the number of satisfying assignments by Z3. I am wondering if Z3 provides such information. If so, how can I count models in Z3 and particularly in Z3Py?
While Taylor's answer will give you the number of satisfying assignments, it will iterate over all of them. In principle, it is possible to do it without such an expensive iteration, but Z3 does not offer it.
There are efficient model counters for propositional logic, the same language used in SAT (search for sharpSAT to find such a system), but as far as I know there is no available model counter modulo theories.
No, such information is not available by default. However, you could easily implement this (assuming finite number of models) in any of the APIs by combining the model generation capability with adding assertions to prevent future assignments from being assigned the same values as past models. See the following answer for a Z3py script accomplishing this:
Z3: finding all satisfying models
To count the models, simply add a counter to the loop until it becomes unsat, and this will give you the number of models.
I am testing the generality of some simplifications (mainly: a directed partial quantifier instantiation). Therefore I ran a collection of the benchmarks in the "AUFLIA-p" section of the smtComp with and without simplification. In order to have as less as possible side effects, I am interested in running Z3 without (user provided) patterns.
I examined some benchmarks in the "AUFLIA-p" section, and I am wondering why the benchmarks of this section contain patterns. Maybe you have run Z3 for this section with an option for disabling patterns. Recently, I just dropped the patterns form some benchmarks an observe the dramatic performance decrease.
Questions:
Is there any difference between the "AUFLIA-p" and "AUFLIA+p" sections?
How can I tell Z3 to ignore the (user provided) patterns?
Regards,
Aboubakr Achraf El Ghazi
The difference between AUFLIA-p and AUFLIA+p is that the first contains no patterns, but the latter does. There is however only one category of benchmarks in SMTLIB, called AUFLIA. (Some of) These benchmarks contain patterns which during SMTCOMP are removed before running the tools. For example, compare unscrambled benchmark and the scrambled benchmark, where the first does contain patterns, but the latter does not.
I believe the removal of patterns is done through the benchmark scrambler.