I used fixedpoint of z3, and I found that the running time of the fixedpoint is always different. Have you meet the same problem? why does this happen?
Sounds unexpected if you get large variety for the same code starting from the same state.
If you start from different states (that is, if you have made miscellaneous calls to Z3
whether over the text API or programmatic API, between the rounds). Z3 should otherwise not
exhibit hugely non-deterministic behavior. Non-deterministic behavior may arise from bugs, so
it will be appreciated if you can further and more precisely describe the scenario that is exercising it.
Related
Is there a way to see what Z3 is doing under the hood? I would like to be able to see the steps it is taking, how long they take, how many steps, etc. I'm checking the equality of floating point addition/multiplication hardware designs with Z3's builtin floating point addition/multiplication. It is taking quite longer than expected, and it would be helpful to see what exactly it's doing in the process.
You can run it with higher verbosity:
z3 -v:10
This will print a lot of diagnostic info. But the output is unlikely to be readable/understandable unless you're really familiar with the source code. Of course, being an open source project, you can always study the code itself: https://github.com/Z3Prover/z3
If you're especially interested in e-matching and quantifiers, you can use the Axiom Profiler (GitHub source, research paper) to analyse a Z3 trace. The profiler shows instantiation graphs, tries to explain instantiations (which term triggered, which equalities were involved), and can even detect and explain matching loops.
For this, run z3 trace=true, and open the generated trace file in the profiler. Note that the tool is a research prototype: I've noticed some bugs, it seems to work more reliable on Windows than on *nix, and it might not always support the latest Z3 versions.
I'm trying to debug a program that is using the Z3 API, and I'm wondering if there's a way, either from within the API or by giving Z3 a command, to print the current logical context, hopefully as if it had been read in an SMT-LIB file.
This question from 7 years ago seemed to indicate that there would be a way to do this, but I couldn't find it in the API docs.
Part of my motivation is that I'm trying to debug whether my program is slow because it's creating an SMT problem that's hard to solve, or whether the slowdown is elsewhere. Being able to view the current context as an SMT-LIB file, and run it in Z3 on the command line, would make this easier.
It's not quite clear what you mean by "logical context." If you mean all the assertions the user has given to the solver, then the command:
(get-assertions)
will return it as an S-expression like list; see Section 4.2.4 of http://smtlib.cs.uiowa.edu/papers/smt-lib-reference-v2.6-r2017-07-18.pdf
But this doesn't sound useful for your purposes; after all it is going to return precisely everything you yourself have asserted.
If you're looking for a dump of all the learned-lemmas, internal assertions the solver created etc; I'm afraid there's no way to do that from SMTLib. You probably can't even do that using the programmatic API either. (Though this needs to be checked.) That would only be possible by actually modifying the source code of z3 itself (which is open-source), and putting in relevant debug traces. But that would require a lot of study of the internals of z3 and would unlikely to help unless you're intimately knowledgeable about z3 code base itself.
I find that running z3 -v:10 can sometimes provide diagnostic info; if you see it repeatedly printing something, it's a good indication that something has gone wrong in that area. But again, what it prints and what it exactly means is guess work unless you study the source code itself.
Answers to other questions have explained that all Agda programs are terminating.
My understanding is that the termination of any valid Adga program is a requirement governed by Agda's advanced dependent type system. This strict requirement seems that it would eliminate many errors. However, it also seems the prohibition of non-terminating programs would prevent the language from being able to express some useful programs. A server, for example, is a program in which the possibility of non-termination is a critical aspect of it's function.
Is it possible to write a server in Agda? I figure it's possible to practically get around the non-termination restriction in this case by setting the server to eventually terminate in a billion years or something. But I'm wondering if there is some trick of the type system that can permit Agda to express some such non-terminating programs, perhaps only those which reach a static closed cycle of some sort. If not, then theoretically could such a trick ever be invented?
Without that possibility it seems the Agda concept is fundamentally limited in the set of useful programs it can express.
All Agda programs need to be total. Which means that:
recursive programs must be terminating
corecursive programs must be productive
Productivity means that any finite observation of the process needs to return an answer in a finite amount of time. A server will be a corecursive program offering the user a set of commands they can issue, returning a response in finite time and (if applicable) offering the next set of commands.
for a study on genetic programming, I would like to implement an evolutionary system on basis of llvm and apply code-mutations (possibly on IR level).
I found llvm-mutate which is quite useful executing point mutations.
As far as I have understood, the instructions get count/numbered, one can then e.g. delete a numbered instruction.
However, introduction of new instructions seems to be possible as one of the availeable statements in the code.
Real mutation however would allow to insert any of the allowed IR instructions, irrespective of it beeing used in the code to be mutated.
In addition, it should be possible to insert library function calls of linked libraries (not used in the current code, but possibly available, because the lib has been linked in clang).
Did I overlook this in the llvm-mutate or is it really not possible so far?
Are there any projects trying to /already have implement(ed) such mutations for llvm?
llvm has lots of code analysis tools which should allow the implementation of the afore mentioned approach. llvm is huge, so I'm a bit disoriented. Any hints which tools could be helpful (e.g. getting a list of available library functions etc.)?
Thanks
Alex
Very interesting question. I have been intrigued by the possibility of doing binary-level genetic programming for a while. With respect to what you ask:
It is apparent from their documentation that LLVM-mutate can't do what you are asking. However, I think it is wise for it not to. My reasoning is that any machine-language genetic program would inevitably face the "Halting Problem", e.g. it would be impossible to know if a randomly generated instruction would completely crash the whole computer (for example, by assigning a value to a OS-reserved pointer), or it might run forever and take all of your CPU cycles. Turing's theorem tells us that it is impossible to know in advance if a given program would do that. Mind you, LLVM-mutate can cause for a perfectly harmless program to still crash or run forever, but I think their approach makes it less likely by only taking existing instructions.
However, such a thing as "impossibility" only deters scientists, not engineers :-)...
What I have been thinking is this: In nature, real mutations work a lot more like LLVM-mutate that like what we do in normal Genetic Programming. In other words, they simply swap letters out of a very limited set (A,T,C,G) and every possible variation comes out of this. We could have a program or set of programs with an initial set of instructions, plus a set of "possible functions" either linked or defined in the program. Most of these functions would not be actually used, but they will be there to provide "raw DNA" for mutations, just like in our DNA. This set of functions would have the complete (or semi-complete) set of possible functions for a problem space. Then, we simply use basic operations like the ones in LLVM-mutate.
Some possible problems though:
Given the amount of possible variability, the only way to have
acceptable execution times would be to have massive amounts of
computing power. Possibly achievable in the Cloud or with GPUs.
You would still have to contend with Mr. Turing's Halting Problem.
However I think this could be resolved by running the solutions in a
"Sandbox" that doesn't take you down if the solution blows up:
Something like a single-use virtual machine or a Docker-like
container, with a time limitation (to get out of infinite loops). A
solution that crashes or times out would get the worst possible
fitness, so that the programs would tend to diverge away from those
paths.
As to why do this at all, I can see a number of interesting applications: Self-healing programs, programs that self-optimize for an specific environment, program "vaccination" against vulnerabilities, mutating viruses, quality assurance, etc.
I think there's a potential open source project here. It would be insane, dangerous and a time-sucking vortex: Just my kind of project. Count me in if someone doing it.
I am sure everybody knows the 5Ws, a formula for getting the "full" story on something which is used in journalism.
Is there a formula like this -I mean, like that questions should be answered for example- so that a "spec" can be considered as totally complete.
Because sometimes I face some features with a spec that sounds like a "pray" more than a specification. -Well, maybe that is even the reason some prays do not come real; specs are not clear enough for the God.
So, what makes a "spec" perfect? Any consensus on this exist?
Thanks,
burak ozdogan
The only perfect specification is working, running code. Anything else is just an approximation.
Clear, shared concepts and terminology play a crucial role.
Regarding terminology, everybody needs to understand what words mean in the spec. If you are inconsistent with words or use words that the spec's audience will not recognise, then there is a risk of failure.
But that's not all. Even with perfect terminology, you need to have a clear and shared agreement on the concepts underlying the words. If differen stakeholders "cut up" reality in different ways, or, in other words, don't see the same things when they look around, and your spec fails to address this, then you are at risk too.
So, what makes a "spec" perfect?
A specification will never be perfect. A specification will be good if it answers the who, what, and when questions to everyone's satisfaction.
Who interfaces with the system?
What does the system need to do?
When does the system need to do what?
Peer review also goes a long way. Once you get all stakeholders - including management - to agree on the spec, you can be confident that it is headed in the right direction.
Both functional and non-functional (aka ilities) requirements have to be considered in order to be complete.
One system that could help is FURPS (or FURPS+).
Functionality: functional requirements
Usability: aesthetics and consistency in the user interface
Reliability: availability ("up time"), accuracy of calculations, and ability to recover from failures
Performance: throughput, response time, recovery time, start-up time
Supportability: testability, adaptability, maintainability, compatibility, - configurability, installability, scalability, and localizability
The "+" in FURPS+ to remember concerns such as:
Design requirements
Implementation requirements
Interface requirements
Physical requirements