In Python, what is the difference between expressions and statements?
Expressions only contain identifiers, literals and operators, where operators include arithmetic and boolean operators, the function call operator () the subscription operator [] and similar, and can be reduced to some kind of "value", which can be any Python object. Examples:
3 + 5
map(lambda x: x*x, range(10))
[a.x for a in some_iterable]
yield 7
Statements (see 1, 2), on the other hand, are everything that can make up a line (or several lines) of Python code. Note that expressions are statements as well. Examples:
# all the above expressions
print 42
if x: do_y()
return
a = 7
Expression -- from the New Oxford American Dictionary:
expression: Mathematics a collection
of symbols that jointly express a
quantity : the expression for the
circumference of a circle is 2πr.
In gross general terms: Expressions produce at least one value.
In Python, expressions are covered extensively in the Python Language Reference In general, expressions in Python are composed of a syntactically legal combination of Atoms, Primaries and Operators.
Python expressions from Wikipedia
Examples of expressions:
Literals and syntactically correct combinations with Operators and built-in functions or the call of a user-written functions:
>>> 23
23
>>> 23l
23L
>>> range(4)
[0, 1, 2, 3]
>>> 2L*bin(2)
'0b100b10'
>>> def func(a): # Statement, just part of the example...
... return a*a # Statement...
...
>>> func(3)*4
36
>>> func(5) is func(a=5)
True
Statement from Wikipedia:
In computer programming a statement
can be thought of as the smallest
standalone element of an imperative
programming language. A program is
formed by a sequence of one or more
statements. A statement will have
internal components (e.g.,
expressions).
Python statements from Wikipedia
In gross general terms: Statements Do Something and are often composed of expressions (or other statements)
The Python Language Reference covers Simple Statements and Compound Statements extensively.
The distinction of "Statements do something" and "expressions produce a value" distinction can become blurry however:
List Comprehensions are considered "Expressions" but they have looping constructs and therfore also Do Something.
The if is usually a statement, such as if x<0: x=0 but you can also have a conditional expression like x=0 if x<0 else 1 that are expressions. In other languages, like C, this form is called an operator like this x=x<0?0:1;
You can write you own Expressions by writing a function. def func(a): return a*a is an expression when used but made up of statements when defined.
An expression that returns None is a procedure in Python: def proc(): pass Syntactically, you can use proc() as an expression, but that is probably a bug...
Python is a bit more strict than say C is on the differences between an Expression and Statement. In C, any expression is a legal statement. You can have func(x=2); Is that an Expression or Statement? (Answer: Expression used as a Statement with a side-effect.) The assignment statement of x=2 inside of the function call of func(x=2) in Python sets the named argument a to 2 only in the call to func and is more limited than the C example.
Though this isn't related to Python:
An expression evaluates to a value.
A statement does something.
>>> x + 2 # an expression
>>> x = 1 # a statement
>>> y = x + 1 # a statement
>>> print y # a statement (in 2.x)
2
An expression is something that can be reduced to a value, for example "1+3" is an expression, but "foo = 1+3" is not.
It's easy to check:
print(foo = 1+3)
If it doesn't work, it's a statement, if it does, it's an expression.
Another statement could be:
class Foo(Bar): pass
as it cannot be reduced to a value.
Statements represent an action or command e.g print statements, assignment statements.
print 'hello', x = 1
Expression is a combination of variables, operations and values that yields a result value.
5 * 5 # yields 25
Lastly, expression statements
print 5*5
An expression is something, while a statement does something.
An expression is a statement as well, but it must have a return.
>>> 2 * 2 #expression
>>> print(2 * 2) #statement
PS:The interpreter always prints out the values of all expressions.
An expression is a statement that returns a value. So if it can appear on the right side of an assignment, or as a parameter to a method call, it is an expression.
Some code can be both an expression or a statement, depending on the context. The language may have a means to differentiate between the two when they are ambiguous.
STATEMENT:
A Statement is a action or a command that does something. Ex: If-Else,Loops..etc
val a: Int = 5
If(a>5) print("Hey!") else print("Hi!")
EXPRESSION:
A Expression is a combination of values, operators and literals which yields something.
val a: Int = 5 + 5 #yields 10
Expressions always evaluate to a value, statements don't.
e.g.
variable declaration and assignment are statements because they do not return a value
const list = [1,2,3];
Here we have two operands - a variable 'sum' on the left and an expression on the right.
The whole thing is a statement, but the bit on the right is an expression as that piece of code returns a value.
const sum = list.reduce((a, b)=> a+ b, 0);
Function calls, arithmetic and boolean operations are good examples of expressions.
Expressions are often part of a statement.
The distinction between the two is often required to indicate whether we require a pice of code to return a value.
References
Expressions and statements
2.3 Expressions and statements - thinkpython2 by Allen B. Downey
2.10. Statements and Expressions - How to Think like a Computer Scientist by Paul Resnick and Brad Miller
An expression is a combination of values, variables, and operators. A value all by itself is
considered an expression, and so is a variable, so the following are all legal expressions:
>>> 42
42
>>> n
17
>>> n + 25
42
When you type an expression at the prompt, the interpreter evaluates it, which means that
it finds the value of the expression. In this example, n has the value 17 and n + 25 has the
value 42.
A statement is a unit of code that has an effect, like creating a variable or displaying a
value.
>>> n = 17
>>> print(n)
The first line is an assignment statement that gives a value to n. The second line is a print
statement that displays the value of n.
When you type a statement, the interpreter executes it, which means that it does whatever
the statement says. In general, statements don’t have values.
An expression translates to a value.
A statement consumes a value* to produce a result**.
*That includes an empty value, like: print() or pop().
**This result can be any action that changes something; e.g. changes the memory ( x = 1) or changes something on the screen ( print("x") ).
A few notes:
Since a statement can return a result, it can be part of an expression.
An expression can be part of another expression.
Statements before could change the state of our Python program: create or update variables, define function, etc.
And expressions just return some value can't change the global state or local state in a function.
But now we got :=, it's an alien!
Expressions:
Expressions are formed by combining objects and operators.
An expression has a value, which has a type.
Syntax for a simple expression:<object><operator><object>
2.0 + 3 is an expression which evaluates to 5.0 and has a type float associated with it.
Statements
Statements are composed of expression(s). It can span multiple lines.
A statement contains a keyword.
An expression does not contain a keyword.
print "hello" is statement, because print is a keyword.
"hello" is an expression, but list compression is against this.
The following is an expression statement, and it is true without list comprehension:
(x*2 for x in range(10))
Python calls expressions "expression statements", so the question is perhaps not fully formed.
A statement consists of pretty much anything you can do in Python: calculating a value, assigning a value, deleting a variable, printing a value, returning from a function, raising an exception, etc. The full list is here: http://docs.python.org/reference/simple_stmts.html#
An expression statement is limited to calling functions (e.g.,
math.cos(theta)"), operators ( e.g., "2+3"), etc. to produce a value.
The single pipe "or" | exists as a method on TrueClass and FalseClass, but the short circuit || operator does not. Neither does it exist as a method on Object.
This seems to be an exception to ruby's "everything is an object" metaphor.
Main Question: Syntactically speaking, what are || and &&? Are they just baked bits of global syntax?
Secondary Question: I'm flagging this as not part of the main question, because it is potentially subjective, though I suspect it probably isn't.
Is there a language design or performance reason for this asymmetry? It seems to me both operators could have been implemented as methods on Object. Something like:
class Object
def short_circuit_or(other)
!nil? ? true :
!other.nil? ? true : false
end
end
I assume there is a reason they were not. What is it?
Both | and || are operators. || is part of the language while | is implemented as a method by some classes (Array, FalseClass, Integer, NilClass and TrueClass) .
In programming languages, | is used in general as the bitwise OR operator. It combines the bits of its integer operands and produces a new integer value. When used with non-integer operands, some languages convert them to integer, others prohibit such usage.
|| is the logical OR operator. It combines two boolean values (true or false) and produces another boolean value. When its operands are not boolean values, they are converted to boolean by some languages. Ruby (and JavaScript and other languages) evaluate its first operand as boolean and the value of the expression is the value of its first operand if its boolean value is true or the value of its second operand if the logical value of its first one is false. The type of the resulting value is its original type, it is not converted to boolean.
Each language uses its own rules to decide what non-boolean values are converted to false (usually the number 0, the empty string '' and null or undefined); all the other values are converted to true. The only "false" values in Ruby are false (boolean) and nil (non-boolean); all the other values (including 0) are "true".
Because true || anything is true and false && anything is false, many programming languages including Ruby implement short-circuit evaluation for logical expressions.
Using short-circuit evaluation, a logical expression is evaluated from left to right, one operand at a time until the value of the expression can be computed without the need to compute the other operands. In the examples above, the value of anything doesn't change the value of the entire expression. Using short-circuit evaluation, the value of anything is not computed at all because it does not influence the value of the entire expression. Being anything a method call that takes considerable time to execute, the short-circuit evaluation avoids calling it and saves execution time.
As others already mentioned in comments to the question, implementing || as a method of some class is not possible. The value of its second operand must be evaluated in order to be passed as argument to the method and this breaks the short-circuiting behaviour.
The usual representation of the logical values in programming languages uses only one bit (and I guess Ruby does the same.) Results of | and || are the same for operands stored on one bit.
Ruby uses the | symbol to implement different flavors of the OR operation as follows:
bitwise OR for integers;
non-short-circuit logical OR for booleans and nil;
union for arrays.
An expression like:
x = false | a | b | c
ensures that all a, b and c expressions are evaluated (no short-circuit) and the value of x is the logical OR of the logical values of a, b and c.
If a, b and c are method calls, to achieve the same result using the logical OR operator (||) the code needs to look like this:
aa = a
bb = b
cc = c
x = aa || bb || cc
This way each method is called no matter what values are returned by the methods called before it.
For TrueClass, FalseClass and NilClass, the | operator is useful when short-circuit evaluation is not desired.
Also, for Array (an array is just an ordered set), the | operator implements union, an operation that is the semantically equivalent of logical OR for sets.
I would like to know, how expressions are parsed when are mixed with control flow.
Let's assume such syntax:
case
when a == Method() + 1
then Something(1)
when a == Other() - 2
then 1
else 0
end
We've got here two conditional expressions, Method() + 1, Something(1) and 0. Each can be translated to postfix by Shunting-yard algorithm and then easily translated into AST. But is it possible to extend this algorithm to handle control - flow also? Or are there other approaches to solve such mixing of expressions and control flows?
another example:
a == b ? 1 : 2
also how can I classify such expression: a between b and c, can I say that between is three arguments function? Or is there any special name for such expressions?
You can certainly parse the ternary operator with an operator-precedence grammar. In
expr ? expr : expr
the binary "operator" here is ? expr :, which conveniently starts and ends with an operator token (albeit different ones). To adapt shunting yard to that, assign the right-precedence of ? and the left-precedence of : to the precedence of the ?: operator. The left-precedence of ? and the right-precedence of : are ±∞, just like parentheses (which, in effect, they are).
Since the case statement is basically repeated application of the ternary operator, using slightly different spellings for the tokens, and yields to a similar solution. (Here case when and end are purely parenthetic, while then and the remaining whens correspond to ? and :.)
Having said that, it really is simpler to use an LALR(1) parser generator, and there is almost certainly one available for whatever language you are writing in.
It's clear that both the ternary operator and OP's case statement are operator grammars:
Ternary operator:
ternary-expr: non-ternary-expr
| non-ternary-expr '?' expr ':' ternary-expr
Normally, the ternary operator will be lower precedence from any other operator and associate to the right, which is how the above is written. In C and other languages ternary expressions have the same precedence as assignment expressions, which is straightforward to add. That results in the relationships
X ·> ?
? <· X
? ·=· :
X ·> :
: <· X
Case statement (one of many possible formulations):
case_statement: 'case' case_body 'else' expr 'end'
case_body: 'when' expr 'then' expr
| case_body 'when' expr 'then' expr
Here are the precedence relationships for the above grammar:
case <· when
case ·=· else
when <· X (see below)
when ·=· then
then ·> when
then ·> else
else <· X
else ·=· end
X ·> then
X ·> when
X ·> end
X in the above relations refers to any binary or unary operator, any value terminal, ( and ).
It's straightforward to find left- and right-precedence functions for all of those terminals; the pattern will be similar to that of parentheses in a standard algebraic grammar.
The Shunting-yard algorithm is for expressions with unary and binary operators. You need something more powerful such as LL(1) or LALR(1) to parse control flow statements, and once you have that it will also handle expressions as well. No need for the Shunting-yard algorithm at all.
Take this example Ruby expression:
case
when 3 then "foo"
when 4 then "bar"
end
I was surprised to learn that this is not a syntax error. Instead, it evaluates to "foo"!
Why? What are the syntax and evaluation rules being applied here?
In this form of the case expression, the then clause associated with the lexically first when clause that evaluates to a truthy value is evaluated.
See clause b) 2) of §11.5.2.2.4 Semantics of the ISO Ruby Language Specification (bold emphasis mine):
Semantics
A case-expression is evaluated as follows:
a) […]
b. The meaning of the phrase “O is matching” in Step c) is defined as follows:
[…]
If the case-expression is a case-expression-without-expression, O is matching if and only if O is a trueish object.
c) Take the following steps:
Search the when-clauses in the order they appear in the program text for a matching when-clause as follows:
i) If the operator-expression-list of the when-argument is present:
I) For each of its operator-expressions, evaluate it and test if the resulting value is matching.
II) If a matching value is found, other operator-expressions, if any, are not evaluated.
ii) If no matching value is found, and the splatting-argument (see 11.3.2) is present:
I) Construct a list of values from it as described in 11.3.2. For each element of the resulting list, in the same order in the list, test if it is matching.
II) If a matching value is found, other values, if any, are not evaluated.
iii) A when-clause is considered to be matching if and only if a matching value is found in its when-argument. Later when-clauses, if any, are not tested in this case.
If one of the when-clauses is matching, evaluate the compound-statement of the then-clause of this when-clause. The value of the case-expression is the resulting value.
If none of the when-clauses is matching, and if there is an else-clause, then evaluate the compound-statement of the else-clause. The value of the case-expression is the resulting value.
Otherwise, the value of the case-expression is nil.
The RDoc documentation, while much less precise, also states that truthiness is the selection criteria, when the condition is omitted; and lexical ordering determines the order in which when clauses are checked (bold emphasis mine):
case
The case statement operator. Case statements consist of an optional condition, which is in the position of an argument to case, and zero or more when clauses. The first when clause to match the condition (or to evaluate to Boolean truth, if the condition is null) "wins", and its code stanza is executed. The value of the case statement is the value of the successful when clause, or nil if there is no such clause.
It is by design that case statement without a value to match against behaves as an if statement.
It is actually the same as writing:
if 3
'foo'
elsif 4
'bar'
end
I'm trying to evaluate the expression (a=10) || (rr=20) while the rr variable is not defined
so typing rr in the ruby console before evaluating the previous expression returns
rr
NameError: undefined local variable or method `rr' for main:Object
from (irb):1
from :0
When I write the expression (a=10) || (rr=20) it returns 10, and when I write rr afterwards it says nil
(a=10) || (rr=20)
rr # => nil
so, why is this happening? Shouldn't rr be defined only if the second argument of the || operator is evaluated, which should be never based on the documentation?
This happens because the ruby interpreter defines a variable when it sees an assignment to it (but before it executes the actual line of code). You can read more about it in this answer.
Boolean OR (||) expression will evaluate to the value of left hand expression if it is not nil and not false, else || will evaluate to the value of right hand expression.
In your example the ruby interpreter sees an assignment to a and rr (but it doesn't execute this line yet), and initializes (defines, creates) a and rr with nil. Then it executes the || expression. In this || expression, a is assigned to 10 and 10 is returned. r=20 is not evaluated, and rr is not changed (it is still nil). This is why in the next line rr is nil.
As #DOC said, && and || are known as short circuited conditional operators.
In case of ||, if the left part of || expression returns true, the right part won't be executed. That means the right part will be executed only if the left part of || expression returns false.
In case of &&, right part of the&&expression will be executed only if left part of && returns true.
In the given scenario (a=10) || (rr=20), rr=20 won't be executed since the ruby expression a=10 returns true. Note that in ruby assignment expression returns true except nil and false.
I think variable definement happens at the parsing stage, not the execution moment. So when it evaluates the line, it parses the whole thing and the variable is defined, but unassigned.
When the parser discovers a variable it is automatically valid within the context it's defined in. Evaluating rr on its own is not valid. Evaluating rr=20 is sufficient to cause a definition even if the value assignment never occurs.
This is a quirk of how Ruby tries to discern between variables and method calls. It's imperfect but usually works out for the best.