I am writing a parser for a query engine. My parser DCG query is not deterministic.
I will be using the parser in a relational manner, to both check and synthesize queries.
Is it appropriate for a parser DCG to not be deterministic?
In code:
If I want to be able to use query/2 both ways, does it require that
?- phrase(query, [q,u,e,r,y]).
true;
false.
or should I be able to obtain
?- phrase(query, [q,u,e,r,y]).
true.
nevertheless, given that the first snippet would require me to use it as such
?- bagof(X, phrase(query, [q,u,e,r,y]), [true]).
true.
when using it to check a formula?
The first question to ask yourself, is your grammar deterministic, or in the terminology of grammars, unambiguous. This is not asking if your DCG is deterministic, but if the grammar is unambiguous. That can be answered with basic parsing concepts, no use of DCG is needed to answer that question. In other words, is there only one way to parse a valid input. The standard book for this is "Compilers : principles, techniques, & tools" (WorldCat)
Now you are actually asking about three different uses for parsing.
A recognizer.
A parser.
A generator.
If your grammar is unambiguous then
For a recognizer the answer should only be true for valid input that can be parsed and false for invalid input.
For the parser it should be deterministic as there is only one way to parse the input. The difference between a parser and an recognizer is that a recognizer only returns true or false and a parser will return something more, typically an abstract syntax tree.
For the generator, it should be semi-deterministic so that it can generate multiple results.
Can all of this be done with one, DCG, yes. The three different ways are dependent upon how you use the input and output of the DCG.
Here is an example with a very simple grammar.
The grammar is just an infix binary expression with one operator and two possible operands. The operator is (+) and the operands are either (1) or (2).
expr(expr(Operand_1,Operator,Operand_2)) -->
operand(Operand_1),
operator(Operator),
operand(Operand_2).
operand(operand(1)) --> "1".
operand(operand(2)) --> "2".
operator(operator(+)) --> "+".
recognizer(Input) :-
string_codes(Input,Codes),
DCG = expr(_),
phrase(DCG,Codes,[]).
parser(Input,Ast) :-
string_codes(Input,Codes),
DCG = expr(Ast),
phrase(DCG,Codes,[]).
generator(Generated) :-
DCG = expr(_),
phrase(DCG,Codes,[]),
string_codes(Generated,Codes).
:- begin_tests(expr).
recognizer_test_case_success("1+1").
recognizer_test_case_success("1+2").
recognizer_test_case_success("2+1").
recognizer_test_case_success("2+2").
test(recognizer,[ forall(recognizer_test_case_success(Input)) ] ) :-
recognizer(Input).
recognizer_test_case_fail("2+3").
test(recognizer,[ forall(recognizer_test_case_fail(Input)), fail ] ) :-
recognizer(Input).
parser_test_case_success("1+1",expr(operand(1),operator(+),operand(1))).
parser_test_case_success("1+2",expr(operand(1),operator(+),operand(2))).
parser_test_case_success("2+1",expr(operand(2),operator(+),operand(1))).
parser_test_case_success("2+2",expr(operand(2),operator(+),operand(2))).
test(parser,[ forall(parser_test_case_success(Input,Expected_ast)) ] ) :-
parser(Input,Ast),
assertion( Ast == Expected_ast).
parser_test_case_fail("2+3").
test(parser,[ forall(parser_test_case_fail(Input)), fail ] ) :-
parser(Input,_).
test(generator,all(Generated == ["1+1","1+2","2+1","2+2"]) ) :-
generator(Generated).
:- end_tests(expr).
The grammar is unambiguous and has only 4 valid strings which are all unique.
The recognizer is deterministic and only returns true or false.
The parser is deterministic and returns a unique AST.
The generator is semi-deterministic and returns all 4 valid unique strings.
Example run of the test cases.
?- run_tests.
% PL-Unit: expr ........... done
% All 11 tests passed
true.
To expand a little on the comment by Daniel
As Daniel notes
1 + 2 + 3
can be parsed as
(1 + 2) + 3
or
1 + (2 + 3)
So 1+2+3 is an example as you said is specified by a recursive DCG and as I noted a common way out of the problem is to use parenthesizes to start a new context. What is meant by starting a new context is that it is like getting a new clean slate to start over again. If you are creating an AST, you just put the new context, items in between the parenthesizes, as a new subtree at the current node.
With regards to write_canonical/1, this is also helpful but be aware of left and right associativity of operators. See Associative property
e.g.
+ is left associative
?- write_canonical(1+2+3).
+(+(1,2),3)
true.
^ is right associative
?- write_canonical(2^3^4).
^(2,^(3,4))
true.
i.e.
2^3^4 = 2^(3^4) = 2^81 = 2417851639229258349412352
2^3^4 != (2^3)^4 = 8^4 = 4096
The point of this added info is to warn you that grammar design is full of hidden pitfalls and if you have not had a rigorous class in it and done some of it you could easily create a grammar that looks great and works great and then years latter is found to have a serious problem. While Python was not ambiguous AFAIK, it did have grammar issues, it had enough issues that when Python 3 was created, many of the issues were fixed. So Python 3 is not backward compatible with Python 2 (differences). Yes they have made changes and libraries to make it easier to use Python 2 code with Python 3, but the point is that the grammar could have used a bit more analysis when designed.
The only reason why code should be non-deterministic is that your question has multiple answers. In that case, you'd of course want your query to have multiple solutions. Even then, however, you'd like it to not leave a choice point after the last solution, if at all possible.
Here is what I mean:
"What is the smaller of two numbers?"
min_a(A, B, B) :- B < A.
min_a(A, B, A) :- A =< B.
So now you ask, "what is the smaller of 1 and 2" and the answer you expect is "1":
?- min_a(1, 2, Min).
Min = 1.
?- min_a(2, 1, Min).
Min = 1 ; % crap...
false.
?- min_a(2, 1, 2).
false.
?- min_a(2, 1, 1).
true ; % crap...
false.
So that's not bad code but I think it's still crap. This is why, for the smaller of two numbers, you'd use something like the min() function in SWI-Prolog.
Similarly, say you want to ask, "What are the even numbers between 1 and 10"; you write the query:
?- between(1, 10, X), X rem 2 =:= 0.
X = 2 ;
X = 4 ;
X = 6 ;
X = 8 ;
X = 10.
... and that's fine, but if you then ask for the numbers that are multiple of 3, you get:
?- between(1, 10, X), X rem 3 =:= 0.
X = 3 ;
X = 6 ;
X = 9 ;
false. % crap...
The "low-hanging fruit" are the cases where you as a programmer would see that there cannot be non-determinism, but for some reason your Prolog is not able to deduce that from the code you wrote. In most cases, you can do something about it.
On to your actual question. If you can, write your code so that there is non-determinism only if there are multiple answers to the question you'll be asking. When you use a DCG for both parsing and generating, this sometimes means you end up with two code paths. It feels clumsy but it is easier to write, to read, to understand, and probably to make efficient. As a word of caution, take a look at this question. I can't know that for sure, but the problems that OP is running into are almost certainly caused by unnecessary non-determinism. What probably happens with larger inputs is that a lot of choice points are left behind, there is a lot of memory that cannot be reclaimed, a lot of processing time going into book keeping, huge solution trees being traversed only to get (as expected) no solutions.... you get the point.
For examples of what I mean, you can take a look at the implementation of library(dcg/basics) in SWI-Prolog. Pay attention to several things:
The documentation is very explicit about what is deterministic, what isn't, and how non-determinism is supposed to be useful to the client code;
The use of cuts, where necessary, to get rid of choice points that are useless;
The implementation of number//1 (towards the bottom) that can "generate extract a number".
(Hint: use the primitives in this library when you write your own parser!)
I hope you find this unnecessarily long answer useful.
If I want to compute a^b mod c then there is an efficient way of doing this, without computing a^b in full.
However, when programming, if I write f g x then g(x) is computed regardless of f.
J provides for composing f and g in special cases, and the modulo power function is one of them. For instance, the following runs extremely quickly.
1000&| # (2&^) 10000000x
This is because the 'atop' conjunction # tells the language to compose the functions if possible. If I remove it, it goes unbearably slowly.
If however I want to work with x^x , then ^~ no longer works and I get limit errors for large values. Binding those large values however does work.
So
999&| # (100333454&^) 100333454x
executes nice and fast but
999&| # ^~ 100333454x
gives me a limit error - the RHS is too big.
Am I right in thinking that in this case, J is not using an efficient power-modulo algorithm?
Per the special code page:
m&|#^ dyad avoids exponentiation for integer arguments
m&|#(n&^) monad avoids exponentiation for integer arguments
The case for ^~ is not supported by special code.
https://github.com/Pascal-J/BN-openssl-bindings-for-J
includes bindings for openssl (distributed with J on windows, and pre-installed on all other compatible systems)'s BN library including modexp. The x ^ x case will be especially efficient due to fewer parameters converted from J to "C" (BN)
I am learning Erlang and one of the problems as per Joe's book states
The function even(X) should return true if X is an even integer and
otherwise false. odd(X) should return true if X is an odd integer.
The way I solve this is
-module(math_functions).
%% API
-export([even/1, odd/1]).
even(Integer) -> (Integer >= 0) and (Integer rem 2 =:= 0).
odd(Integer) -> (Integer >= 1) and (Integer rem 2 =/= 0).
and run this as
Eshell V6.2 (abort with ^G)
1> math_functions:odd(13).
true
2> math_functions:odd(-13).
false
3> math_functions:odd(1).
true
4> math_functions:even(1).
false
5> math_functions:even(2).
true
6> math_functions:even(-2).
false
7>
The question I have is if there are better ways to do this
Thanks
You could use guards to limit yourself to integers greater than or equal to zero, and then simply check the least-significant bit as suggested in the comment to your question. You can also define odd/1 in terms of even/1:
even(X) when X >= 0 -> (X band 1) == 0.
odd(X) when X > 0 -> not even(X).
The guards are part of the function-head, so if you call even(-1) it will fail to match in exactly the same way as if you called even(1, 2) (i.e. with the wrong number of arguments).
Answer to Daydreamer comment about Steve answer.
When you write a function, a frequent usage in erlang is to code only the "success" cases and let crash the unsuccessful cases (I'll come back later to explain why it is important).
Another criteria, valid for any language, is to avoid surprise when someone use or read your code.
In one of your comment you say that the odd and even functions you want to write are limited to positive or null integers (I won't discuss this choice, and at least the odd and even functions are limited to integers). This means that you have to ask yourself a first question: what is the behavior of my function if it is called with a bad parameter.
First choice: let it crash this the Steve proposition: the function works only with the correct arguments. I always prefer this solution. The only exception is if I do not master the input parameters, for example if they come directly from a file, a user interface ... Then I prefer the third choice.
Second choice: return a result this is your choice: you return false. From a logic point of view, for odd and even function, returning false is valid: is something is not true, it is false :o). I don't like this solution for 2 reasons. The first one is that it is not something you can generalize easily to something else than boolean answer. The second and more important to me, is that it may surprise the user. When the function odd(N) return false, it is reasonable to think that N is even, while in this case odd(-2) and even(-2) will both return false.
third choice: return a tagged result this is something you see very often in erlang: a function return either {ok,Value} or {Error,Term}. doing this you let the choice to the calling function to manage or not a bad arguments errors. the Error variable allows you to have explicit error messages, useful for debug and also user interface. In your example the code becomes:
even(X) when is_integer(X), X >= 0 -> {ok,(X band 1) == 0};
even(X) -> {illegal_param,X}.
odd(X) when is_integer(X), X >= 0 -> {ok,(X band 1) == 1};
odd(X) -> {illegal_param,X}.
When programming, it is important to detect error as soon as possible, in erlang it is even more important. If one process does not detect (and the simplest detection is crash) and error and propagate some invalid information through messages, it may be very difficult to find the root cause of a problem, ignoring which process (maybe died) issued this message. Coding only the success cases is an easy way to detect problems as soon as possible.
Find the no if even
%functions that manipulate functions are called higher-order %functions, and the data type that represents a function in Erlang is %called a fun. here in below example you see FUNCTIONS THAT HAVE %FUNCTIONS AS THEIR ARGUMENTS
% K is an example list
1> K=[4,5,6,8,10].
[4,5,6,8,10]
% Use lisst:filter to perform no/2 and filter if rem=0
2> lists:filter(fun(J)->(J rem 2)=:=0 end, K).
[4,6,8,10]
Another way:
% Function to check even
22> Checkeven=fun(U)->(U rem 2)=:=0 end.
#Fun<erl_eval.7.126501267>
23> Checkeven(5).
false
% Define a test list
25> K=[1,2,3,4,5].
[1,2,3,4,5]
% Use lists filter to apply Checkeven func to all elements of k
26> lists:filter(Checkeven,K).
[2,4]
%Using List comprehension
42> K.
[1,2,3,4,5]
% For all elements of K check remainder of elem/2 is zero
43> [(S rem 2=:=0) || S<-K].
[false,true,false,true,false]
I have to partition a multiset into two sets who sums are equal. For example, given the multiset:
1 3 5 1 3 -1 2 0
I would output the two sets:
1) 1 3 3
2) 5 -1 2 1 0
both of which sum to 7.
I need to do this using Z3 (smt2 input format) and "Linear Arithmetic Logic", which is defined as:
formula : formula /\ formula | (formula) | atom
atom : sum op sum
op : = | <= | <
sum : term | sum + term
term : identifier | constant | constant identifier
I honestly don't know where to begin with this and any advice at all would be appreciated.
Regards.
Here is an idea:
1- Create a 0-1 integer variable c_i for each element. The idea is c_i is zero if element is in the first set, and 1 if it is in the second set. You can accomplish that by saying that 0 <= c_i and c_i <= 1.
2- The sum of the elements in the first set can be written as 1*(1 - c_1) + 3*(1 - c_2) + ... +
3- The sum of the elements in the second set can be written as 1*c1 + 3*c2 + ...
While SMT-Lib2 is quite expressive, it's not the easiest language to program in. Unless you have a hard requirement that you have to code directly in SMTLib2, I'd recommend looking into other languages that have higher-level bindings to SMT solvers. For instance, both Haskell and Scala have libraries that allow you to script SMT solvers at a much higher level. Here's how to solve your problem using the Haskell, for instance: https://gist.github.com/1701881.
The idea is that these libraries allow you to code at a much higher level, and then perform the necessary translation and querying of the SMT solver for you behind the scenes. (If you really need to get your hands onto the SMTLib encoding of your problem, you can use these libraries as well, as they typically come with the necessary API to dump the SMTLib they generate before querying the solver.)
While these libraries may not offer everything that Z3 gives you access to via SMTLib, they are much easier to use for most practical problems of interest.
I was using the Pow function of the BigInteger class in F# when my compiler told me :
This construct is deprecated. This member has been removed to ensure that this
type is binary compatible with the .NET 4.0 type System.Numerics.BigInteger
Fair enough I guess, but I didn't found a replacement immediately.
Is there one? Should we only use our own Pow functions? And (how) will it be replaced in NET4.0?
You can use the pown function
let result = pown 42I 42
pown works on any type that 'understands' multiplication and 'one'.
If you look at F# from the perspective of being based on OCaml, then the OCaml Num module has power_num. Since OCaml type num are arbitrary-precision rational numbers they can handle any size number, e.g. they are not limited by the CPU register because they can do the math symbolically. Also since num is defined as
type num =
| Int of int
| Big_int of Big_int.big_int
| Ratio of Ratio.ratio
they can handle very small numbers with out loss of precision because of the Ratio type.
Since F# does not have the num type, Jack created the FSharp.Compatibility.OCaml module which has num.fs and is available via NuGet.
So you can get all the precision you want using this, and the num functions can handle negative exponents.