The following grammar is failing to make the distinction between negation and subtraction operations. It ignores the negation operation completely.
I assume this is happening because there is an abiguity... negAtom is very similar to OPERATOR multOp. How would i rewrite my grammar to accomodate both negation and subtraction while maintaining operator precedence with multiplication and division?
grammar Expr.g4
op:
multOp (OPERATOR multOp)*;
multOp:
negAtom (MULT_OPERATOR multOp)*;
negAtom:
NEG? atom;
atom:
group | INT;
group:
L op R;
L : '(';
R : ')';
NEG : '-';
ADD : '+';
MLT : '*';
DIV : '/';
OPERATOR : (NEG|ADD);
MULT_OPERATOR : (MLT|DIV);
INT : '0'..'9'+;
Example Parse Tree with input "-1":
OPERATOR will never match, since either NEG or ADD match first.
In short: the first lexer rule that can match will match.
You should make lexer rules like OPERATOR a parser rule like operator.
This problem has been solved many times. Have a look at existing grammars.
For Antlr4 grammar have a look here: list of antlr4 grammars at github
Related
As I said in the title, I am trying to do an exercise where I need to write a DCG capable of reading propositional logic, which are represented by lowercase letters, operators (not, and , and or), with the tokens separated by whitespace. So the expression:
a or not b and c
is parsed as
a or ( not b and c )
producing a parse tree that looks like:
or(
a,
and(
not(b),
c
)
)
To be completely honest I have been having a hard time understanding how to effectively use DCGs, but this is what I've got so far:
bexpr([T]) --> [T].
bexpr(not,R1) --> oper(not), bexpr(R1).
bexpr(R1,or,R2) --> bexpr(R1),oper(or), bexpr(R2).
bexpr(R1, and ,R2) --> bexpr(R1),oper(and), bexpr(R2).
oper(X) --> X.
I would appreciate any suggestions, either on the exercise itself, or on how to better understand DCGs.
The key to understanding DCGs is that they are syntactic sugar over writing a recursive descent parser. You need to think about operator precedence (how tightly do your operators bind?). Here, the operator precedence, from tightest to loosest is
not
and
or
so a or not b and c is evaluated as a or ( (not b) and c ) ).
And we can say this (I've included parenthetical expressions as well, because they're pretty trivial to do):
% the infix OR operator is the lowest priority, so we start with that.
expr --> infix_OR.
% and an infix OR expression is either the next highest priority operator (AND),
% or... it's an actual OR expression.
infix_OR --> infix_AND(T).
infix_OR --> infix_AND(X), [or], infix_OR(Y).
% and an infix AND expression is either next highest priority operator (NOT)
% or... it's an actual AND expression.
infix_AND --> unary_NOT(T).
infix_AND --> unary_NOT(X), [and], infix_AND(Y).
% and the unary NOT expression is either a primary expression
% or... an actual unary NOT expression
unary_NOT --> primary(T).
unary_NOT --> [not], primary(X).
% and a primary expression is either an identifer
% or... it's a parenthetical expression.
%
% NOTE that the body of the parenthetical expression starts parsing at the root level.
primary --> identifier(ID).
primary --> ['(', expr(T), ')' ].
identifier --> [X], {id(X)}. % the stuff in '{...}' is evaluated as normal prolog code.
id(a).
id(b).
id(c).
id(d).
id(e).
id(f).
id(g).
id(h).
id(i).
id(j).
id(k).
id(l).
id(m).
id(n).
id(o).
id(p).
id(q).
id(r).
id(s).
id(t).
id(u).
id(v).
id(w).
id(x).
id(y).
id(z).
But note that all this does is to recognize sentences of the grammar (pro tip: if you write your grammar correctly, it should also be able to generate all possible valid sentences of the grammar). Note that this might take a while to do, depending on your grammar.
So, to actually DO something with the parse, you need to add a little extra. We do this by adding extra arguments to the DCG, viz:
expr( T ) --> infix_OR(T).
infix_OR( T ) --> infix_AND(T).
infix_OR( or(X,Y) ) --> infix_AND(X), [or], infix_OR(Y).
infix_AND( T ) --> unary_NOT(T).
infix_AND( and(X,Y) ) --> unary_NOT(X), [and], infix_AND(Y).
unary_NOT( T ) --> primary(T).
unary_NOT( not(X) ) --> [not], primary(X).
primary( ID ) --> identifier(ID).
primary( T ) --> ['(', expr(T), ')' ].
identifier( ID ) --> [X], { id(X), ID = X }.
id(a).
id(b).
id(c).
id(d).
id(e).
id(f).
id(g).
id(h).
id(i).
id(j).
id(k).
id(l).
id(m).
id(n).
id(o).
id(p).
id(q).
id(r).
id(s).
id(t).
id(u).
id(v).
id(w).
id(x).
id(y).
id(z).
And that is where the parse tree is constructed. One might note that one could just as easily evaluate the expression instead of building the parse tree... and then you're on you way to writing an interpreted language.
You can fiddle with it at this fiddle: https://swish.swi-prolog.org/p/gyFsAeAz.pl
where you'll notice that executing the goal phrase(expr(T),[a, or, not, b, and, c]). yields the desired parse T = or(a, and(not(b), c)).
I want to create an expression tree given expression in infix form. Is it necessary to convert the expression to postfix first and then create the tree? I understand that it somehow depends on the problem itself. But assume that it is simple expression of mathematical function with unknowns and operators like: / * ^ + -.
No. If you're going to build an expression tree, then it's not necessary to convert the expression to postfix first. It will be simpler just to build the expression tree as you parse.
I usually write recursive descent parsers for expressions. In that case each recursive call just returns the tree for the subexpression that it parses. If you want to use an iterative shunting-yard-like algorithm, then you can do that too.
Here's a simple recursive descent parser in python that makes a tree with tuples for nodes:
import re
def toTree(infixStr):
# divide string into tokens, and reverse so I can get them in order with pop()
tokens = re.split(r' *([\+\-\*\^/]) *', infixStr)
tokens = [t for t in reversed(tokens) if t!='']
precs = {'+':0 , '-':0, '/':1, '*':1, '^':2}
#convert infix expression tokens to a tree, processing only
#operators above a given precedence
def toTree2(tokens, minprec):
node = tokens.pop()
while len(tokens)>0:
prec = precs[tokens[-1]]
if prec<minprec:
break
op=tokens.pop()
# get the argument on the operator's right
# this will go to the end, or stop at an operator
# with precedence <= prec
arg2 = toTree2(tokens,prec+1)
node = (op, node, arg2)
return node
return toTree2(tokens,0)
print toTree("5+3*4^2+1")
This prints:
('+', ('+', '5', ('*', '3', ('^', '4', '2'))), '1')
Try it here:
https://ideone.com/RyusvI
Note that the above recursive descent style is the result of having written many parsers. Now I pretty much always parse expressions in this way (the recursive part, not the tokenization). It is just about as simple as an expression parser can be, and it makes it easy to handle parentheses, and operators that associate right-to-left like the assignment operator.
I would like to know, how expressions are parsed when are mixed with control flow.
Let's assume such syntax:
case
when a == Method() + 1
then Something(1)
when a == Other() - 2
then 1
else 0
end
We've got here two conditional expressions, Method() + 1, Something(1) and 0. Each can be translated to postfix by Shunting-yard algorithm and then easily translated into AST. But is it possible to extend this algorithm to handle control - flow also? Or are there other approaches to solve such mixing of expressions and control flows?
another example:
a == b ? 1 : 2
also how can I classify such expression: a between b and c, can I say that between is three arguments function? Or is there any special name for such expressions?
You can certainly parse the ternary operator with an operator-precedence grammar. In
expr ? expr : expr
the binary "operator" here is ? expr :, which conveniently starts and ends with an operator token (albeit different ones). To adapt shunting yard to that, assign the right-precedence of ? and the left-precedence of : to the precedence of the ?: operator. The left-precedence of ? and the right-precedence of : are ±∞, just like parentheses (which, in effect, they are).
Since the case statement is basically repeated application of the ternary operator, using slightly different spellings for the tokens, and yields to a similar solution. (Here case when and end are purely parenthetic, while then and the remaining whens correspond to ? and :.)
Having said that, it really is simpler to use an LALR(1) parser generator, and there is almost certainly one available for whatever language you are writing in.
It's clear that both the ternary operator and OP's case statement are operator grammars:
Ternary operator:
ternary-expr: non-ternary-expr
| non-ternary-expr '?' expr ':' ternary-expr
Normally, the ternary operator will be lower precedence from any other operator and associate to the right, which is how the above is written. In C and other languages ternary expressions have the same precedence as assignment expressions, which is straightforward to add. That results in the relationships
X ·> ?
? <· X
? ·=· :
X ·> :
: <· X
Case statement (one of many possible formulations):
case_statement: 'case' case_body 'else' expr 'end'
case_body: 'when' expr 'then' expr
| case_body 'when' expr 'then' expr
Here are the precedence relationships for the above grammar:
case <· when
case ·=· else
when <· X (see below)
when ·=· then
then ·> when
then ·> else
else <· X
else ·=· end
X ·> then
X ·> when
X ·> end
X in the above relations refers to any binary or unary operator, any value terminal, ( and ).
It's straightforward to find left- and right-precedence functions for all of those terminals; the pattern will be similar to that of parentheses in a standard algebraic grammar.
The Shunting-yard algorithm is for expressions with unary and binary operators. You need something more powerful such as LL(1) or LALR(1) to parse control flow statements, and once you have that it will also handle expressions as well. No need for the Shunting-yard algorithm at all.
I've been trying to use this code for a while now and it says there is a syntax error but I'm not sure what it is.
studies(ahmed,history(77,63)).
studies(john,chemistry(0,21)).
passed(Person,Subj):-
studies(Person, Subj(Work, Exam)),
Final is Work + Exam,
Final >=60.
You can't directly "parameterize" the functor, but you can use the =../2 operator, which unifies a functor and arguments with a list:
passed(Person, Subj):-
studies(Person, SubjWorkExam),
SubjWorkExam =.. [Subj, Work, Exam],
Work + Exam >= 60.
This avoids hard-coding the various subjects in your predicate. Also, the comparison operator >=/2 will evaluate expressions, so the separate is/2 is not required.
You can't use a variable for the name of a clause, you could write instead:
passed(Person,Subj):-
(Subj=history-> studies(Person, history(Work, Exam))
;Subj=chemistry-> studies(Person, chemistry(Work, Exam)),
Final is Work + Exam,
Final >=60.
I'm writing a program in Prolog where I'm given a set of grammar rules and the user inputs a sentence, I must make sure the sentence follows the given rules.
I'm only stuck on one rule:
expr -> ( expr ) also written as expr -> ( id op expr )
Here is my code for this part:
expr(X) :- list(X), length(X, Length), =(Length, 5),
=(X, [Left, Id, Op, Expr | Right]),
=(Left, ‘(‘),
id(Id), op(Op), expr([Expr]),
=(Right, ‘)’).
I believe the issue is with checking the parentheses since the other parts of this code are used elsewhere with no errors. When using =(Left, '(') or =(Right, ')') I get a syntax error: expression expected why do I get this error and what would be a better way to check for left and right parentheses?
I think you should use single quotes here =(Left, ‘(‘), and here =(Right, ‘)’). I.e. =(Left, '('), and =(Right, ')').
That said, your Expr will only match a single token, and this is not what I expect. Consider to match the entire 'right' sequence with
X = [Left, Id, Op | Expr],
and further split Expr to get the right parenthesi. Anyway, as I advised in another answer, your parsing (also after correction) will fail on [a,=,'(',b,')',+,c].