I've read in multiple articles that semicolon(;) in UNIX-like shells is equal to a new line.
However, the following is confusing me and I'm having no luck googling it either.
I'm guessing it's an issue with do in shell, but "bash semicolon do" is not exactly the most google-friendly search term combination.
Below is a simple for statement.
for i in {1..10}
do
echo "hi"
echo "bye"
done
As many Stack Overflow gurus have posted, every newline can be substituted with semicolons.
So.. we have this following "same" statement.
for i in {1..10}; do; echo "hi"; echo "bye"; done
and we get:
-bash: syntax error near unexpected token `;'
What exactly is the semicolon? Is this just an unique issue with do?
Looking at the syntax of the for/do loop,
for name [ [in [words …] ] ; ] do commands; done
we can see that do is followed by commands immediately, so using a newline after do doesn't replace a semicolon, but a space.
The description for compound commands also says
In most cases a list of commands in a compound command’s description may be separated from the rest of the command by one or more newlines, and may be followed by a newline in place of a semicolon.
but nowhere does it say that you can insert random semicolons. "Every newline can be substituted with semicolons" is simply too general a statement and not correct.
More manual evidence: in the section about lists of commands, it says (emphasis mine):
A list is a sequence of one or more pipelines separated by one of the operators ;, &, &&, or ||, and optionally terminated by one of ;, &, or a newline.
Of these list operators, && and || have equal precedence, followed by ; and &, which have equal precedence.
A sequence of one or more newlines may appear in a list to delimit commands, equivalent to a semicolon.
So a newline is equivalent to a semicolon within a list of commands.
It's not just with do; there are a number of contexts where the shell allows a newline where it would not allow a semicolon.
Here are most of them (leaving out the uses of newlines or semicolons inside quotes, which are always distinct from each other):
After && or ||
some_command &&
some_other_command
After a pipe (|):
some_producer |
some_consumer
Before the in in a for statement:
for x
in $(produce_values); do
Although empty commands are illegal, the shell grammar does allow empty value lists in a for command, so for x in; do do_something; done is legal, and so would be the same thing written with newlines instead of semicolons.
Before or after the in in a case statement; also after the closing ) of each pattern and the ;; which closes each case:
case $x
in
pattern)
do_something ;;
esac
After the keywords if, then, elif, else, while or until:
if
some_condition
then
do_something
else
do_something_else
fi
After the { or ( which opens a compound command or subshell:
{
a_command
another_command
}
(
a_command
another_command
)
Similarly, after the $( which starts a command substitution:
a=$(
echo hello
)
In bash, this also applies to the ( in process substitution: <(...). See below for the slightly different handling in the bash extensions dealing with conditionals and arithmetic substitution.
After the ) in a function definition:
my_function()
{
do_something
}
After the ; or & which terminates a command:
do_something;
do_something_in_background &
(Note that empty commands are illegal, so do_something; ; would be rejected.)
I took that list from the shell grammar in the Posix standard. Bash allows newlines in other places; I couldn't think of any when I wrote this answer, but #BenjaminW reminded me:
Inside a parenthesized array literal, either in an array assignment or an array declaration, newlines are considered whitespace:
a+=(
first_thing
second_thing
)
local -A b=(
[x]="value of x"
[y]="value of y"
)
And then I remembered these other pseudo-quoted environments:
Bash also accepts newlines as whitespace inside arithmetic expressions:
i=$((
i +
3 * j
))
if ((
i + 3 * j
>
9
))
and after the [[ or before the ]] in [[-style conditional statements:
if
[[
$a -eq "done"
]]
then break
fi
What #Benjamin W. explains in a quite exhaustive while lengthy answer can be summed up as a concise and simple general rule which also in my opinion sheds light on the reason behind the misunderstanding of this phenomena. What needs to be remembered is that:
In Bash
Every semicolon CAN be substituted with a newline.
but
Every newline CANNOT be substituted with a semicolon.
Here, do <command> is the original syntax. A newline can replace the space, but as a semicolon cannot replace a newline, we can't put a semicolon here.
However for the syntax command1 ; command2, a newline can replace a semicolon and give us
command1
command2
With time, we got much more familiar with a newline as the command-line separator instead of the semicolon and tend to see semicolons as a replacement for newlines in order to make one-liners, but it seems that bash was in fact designed the other way around: semicolons are the original separator, newlines are just syntactic sugar (in the context of the command-line separator).
NB: according to the explanations of Benjamin W.
They work in zsh:
for i in {1..10}; do; echo "hi"; echo "bye"; done
; for i in {1..10}; do; echo "hi"; echo "bye"; done
; for i in {1..10}; do; echo "hi"; echo "bye"; done;
(Pay attention to the special case, and if you meet similar case, come back to this topic for details)
Related
I have written a simple calculator in C using recursive-descent parser. It all works great but having a problem when I start with a '(' sign.
I take the user input from argv[1] and put the content in a global variable for temporary simplicity. Than I simply go through each character in the string and looking for what pattern it may be, ex number, add sign or multiplication (just like a regular recursive-descent parser work, no real fancy work).
But, if I do this:
./calculator (1+2)*0.5
I receive an error message,
bash: syntax error near unexpected token `1+2'
This is because I have to escape the '(' and ')' so
./calculator \(1+2\)*0.5
works fine.
So my questions is:
How do I solve this without having to think about eater putting single or double quotations around the equation or escape them?
Why is 0.5*(1+2) still working? Should I not have to escape the parenthesis there also?
What happens here is that your line looks like a function definition:
$ ./calculator () {
> echo "function called with arguments '$#'"
> }
defines a shell function called ./calculator. It can then be called like a command would be executed:
$ ./calculator arguments go here
function called with arguments 'arguments go here'
Your error stems from the fact that Bash expects that ( be followed by ) for it to be a proper function definition, but your parents weren't empty!
Bash shell reserves many meta characters on command line, not only parentheses. * is used for pathname generation. Different shells work differently; in Z shell (zsh), even this wouldn't work:
% ./calculator 1*2
zsh: no match
You should must escape all these metacharacters when given on command line. Do not learn a "safe subset" because soon you will try another shell and it fails. Or this might happen:
$ echo 1*2
1*2
$ touch 1-31337-2
$ echo 1*2
1-31337-2
There are two simple solutions to avoid backslashitis:
use single quotes around everything:
$ ./calculator '(1+2)*0.5'
works nicely if your string doesn't contain '.
Double quotes would also work but there are more meta characters that are reserved by bash within double quotes, for example $.
read the calculation from standard input instead, with a prompt
$ ./calculator
calculator> 1 + 2 * 0.5
you can use for example readline library for easy interactive editing, too!
Summary: Either quote your expressions (preferably with single quotes) or use something other than ( and ) for grouping.
For an answer to "why does 0.5*(1+2) work?", go to the end. (Hint: it's because you don't have a file named 0.5.)
Parentheses are what the bash manual refers to as metacharacters. (Posix no longer uses this term; instead, it refers to such characters as "operators". But the basic effect is the same.) Unless quoted, metacharacters are always tokens by themselves (or, in cases like << and &&, along with the rest of the operator they start), and they have syntactic significance.
This is different from braces ({ and }) which are reserved words, not metacharacters, and so do not delimit tokens. As reserved words, they only have special significance when they are tokens by themselves and are the first token in a command:
{echo x # The command to be executed is `{echo`, which probably doesn't exist
echo {x # No problem. Prints the string '{x'
echo { } # Also no problem. Prints '{ }'
{ echo x; } # A compound command. The ; is necessary.
(echo x) # Also a compound command but ; and whitespace are optional
[ and ] are somewhat similar. [ is a command (not even a reserved word), while [[ is a reserved word which starts a conditional compound command but only if it is the first word in a command.
So you could use brackets or braces as grouping operators without worrying about quoting, because the arguments to your function are never going to be the first words in the command.
As a side-note, the difference between [ (a command) and [[ (a reserved word) is shown by the fact that only the first one can be preceded by a variable assignment (in this case, the assignment has no useful effect):
$ foo=3 [ -z "$foo" ] && echo yes
yes
$ foo=3 [[ -z "$foo" ]] && echo yes
[[: command not found
$ [[ -z "$foo" ]] && echo yes
yes
The precise syntactic significance of parentheses depends, as usual, on the syntax in which they appear. In the case of (, this might be:
A function definition:
func () { echo "$#"; }
A compound command executed in a subshell
(sleep 1; echo "Hello..."; sleep 5; echo "World!")&
Surrounding the pattern in a case clause:
case "$word"; in
(Hello) echo "Hi" ;;
Bye) echo "Seeya" ;; # The open parenthesis is optional in this syntax
esac
In bash, it may also be used as part of array assignment:
local numbers=(one two three)
and it can form part of the (( operator, used in arithmetic conditional compound commands and arithmetic for statements.
Parentheses might also appear as part of a longer construct not starting with a parenthesis, such as command substitution: $(. But if a parenthesis is recognised as a token and it doesn't fit any of the syntactic constructs which include parentheses, a syntax error will be signalled:
$ echo a b(c)
bash: syntax error near unexpected token `('
That leaves us with a small mystery: how do we explain the following:
$ echo a+(b+4)
a+(b+4)
$ echo a-(b+4)
bash: syntax error near unexpected token `('
$ echo a*(b+4)
a*(b+4)
$ echo a/(b+4)
bash: syntax error near unexpected token `('
The answer is that I have shopt -s extglob in my bash start-up files. And you probably do, too, because many distributions do that for you by default. If "extended glob" patterns are available, then the following are patterns:
?(pattern-list)
Matches zero or one occurrence of the given patterns
*(pattern-list)
Matches zero or more occurrences of the given patterns
+(pattern-list)
Matches one or more occurrences of the given patterns
#(pattern-list)
Matches one of the given patterns
!(pattern-list)
Matches anything except one of the given patterns
A pattern-list can contain only a single pattern, so b+4 is a valid pattern, and a+(b+4) will therefore match a file whose name starts with an a and is followed by one or more instances of the characters b+4:
$ touch ab+4b+4b+4
$ echo a+(b+4)
ab+4b+4b+4
Like any other filename pattern, if no filename is matched, the pattern is not substituted:
$ rm ab+4b+4b+4
$ echo a+(b+4)
a+(b+4)
Unless you have other shell options set:
$ shopt -s failglob
$ echo a+(b+4)
bash: no match: a+(b+4)
This question already has answers here:
Bash semicolon being equal to newline is not exactly true?
(4 answers)
Closed 2 years ago.
I have a shell script which contains the following:
case $1 in
0 )
echo $1 = 0;
OUTPUT=3;;
1 )
echo $1 = 1;
OUTPUT=4;;
2 )
echo $1 = 2;
OUTPUT=4;;
esac
HID=$2;
BUNCH=16;
LR=.008;
Are semicolons completely superfluous in the snippet above? And is there any reason for some people using double semicolons?
It appears semicolons are only a separator, something you would use instead of a new line.
Single semicolons at the end of a line are superfluous, since the newline is also a command separator. case specifically needs double semicolons at the end of the last command in each pattern block; see help case for details.
According to man bash:
metacharacter
A character that, when unquoted, separates words. One of the following:
| & ; ( ) < > space tab
control operator
A token that performs a control function. It is one of the following symbols:
|| & && ; ;; ( ) | |& <newline>
So, the ; can be metacharacter or control operator, while the ;; is always a control operator (in case command).
In your particular code, all ; at the end of line are not needed. The ;; is needed however.
In the special case of find, ; is used to terminate commands invoked by -exec. See the answer of #kenorb to this question.
#Opensourcebook-Amit
newlines equivalent to single semicolon ; on terminal or in shell script.
See the below examples:
On terminal:
[root#server test]# ls;pwd;
On shell script:
[root#server test]# cat test4.sh
echo "Current UserName:"
whoami
echo -e "\nCurrent Date:";date;
[root#server test]#
But I am not agree with the comment that & is equivalent to newline or single semicolon
& is run commands in background also a command separator but not worked as semicolon or newline.
I'm trying to evaluate multiple lines of shell commands using eval, but when I try to resolve variables with eval separated by a newline \n the variables are not resolved.
x='echo a'
y='echo b'
z="$x\n$y"
eval $x
eval $y
eval $z
Which outputs:
a
b
anecho b
The last command gives anecho b, and apparently \n was treated as n there. So is there a way to evaluate multiple lines of commands (say, separated by \n)?
\n is not a newline; it's an escape sequence that in some situations will be translated into a newline, but you haven't used it in one of those situations. The variable $z doesn't wind up containing a newline, just backslash followed by "n". As a result, this is what's actually being executed:
$ echo a\necho b
anecho b
You can either use a semicolon instead (which requires no translation), or use \n in a context where it will be translated into a newline:
$ newline=$'\n'
$ x='echo a'
$ y='echo b'
$ z="$x$newline$y"
$ eval "$z"
a
b
Note the double-quotes around "$z" -- they're actually critical here. Without them, bash will word-split the value of $z, turning all whitespace (spaces, tabs, newlines) into word breaks. If that happens, eval will receive the words "echo" "a" "echo" b", effectively turning the newline into a space:
$ eval $z
a echo b
This is yet another in the long list of cases where it's important to double-quote variable references.
You are passing the newline into eval. So it's like you are on the console typing this:
el#voyager$ echo a\necho b
anecho b
So the first echo is understood correctly, and it thinks you want quotes around the rest. The backslash seems to be ignored. Perhaps you meant something like this:
el#voyager$ echo -e 'a\n'; echo b
a
b
Option 1:
delimit statements passed into eval with a semicolon like this:
x='echo a'
y='echo b'
z="$x;$y"
eval $x
eval $y
eval $z
prints:
a
b
a
b
Option 2:
Put the newline in the place where it will be interpreted by the echo, like this:
x='echo -e "a\n"'
y='echo b'
z="$x;$y"
eval $x
eval $y
eval $z
prints:
a
b
a
b
Now the newline is preserved and interpreted by the echo, not the eval.
Not necessarily the optimal way as it will fail if the x and y variables contain sequences processed by printf like %s and similar but anyway, here is a method to do it while keeping \n as a separator:
x='echo a'
y='echo b'
z="$x\n$y"
eval $x
eval $y
export IFS=" "
eval $(printf "$z")
prints:
a
b
a
b
A slightly different approach:
read -r -d '' script <<'EOF'
echo a
echo b
EOF
eval "$script"
outputs
a
b
Explanation
read -r -d '' script
-r - do not allow backslashes to escape any characters
-d '' - continue until the first character of DELIM is read, rather than newline (makes it read until EOF)
script - the name of the variable to save the result in
<<'EOF' - use a heredoc WITHOUT variable expansion (the single quotes around the EOF stops the variable expansion)
Alternative
This could also be done using $(cat <<'EOF'...EOF), but this way does not needlessly use cat and does not use a subshell.
Example with useless cat:
script=$(cat <<'EOF'
echo a
echo b
EOF
)
eval "$script"
On my FreeBSD box, I was trying do something in a Bourne script which at first seemed fairly trivial - but clouded my mind for a few moments. Since this page is what I referred to trying to fix my problem, I will explain what I needed to do and how I got it done :
a=A
b=B
eval ${a}_${b}="something"
No problems so far. I get a new variable A_B that stores "something"
But if I spread the assignment over 2 lines as under :
eval ${a}_${b}="some
thing"
The shell barks back at me that it could find no command called 'thing'. It is important to understand that eval tries to evaluate RHS as a command. To get eval to evaluate RHS as a string, you have to double-double quote RHS :
eval ${a}_${b}="\"some
thing\""
Hope this helps someone.
Manish Jain
Are you thinking of using the more powerful zsh instead of bash?
If so, you may want to take advantage of process substitution and heredoc. The following code works similarly to the title.
source <(cat << EOF
command1
command2
EOF
)
I'm studyng Bash, and I see that the form
C=example
echo "$C"
give the same result of the form
C="example"
echo $C
I'd like to know if is better put the " " in the assignment of the variable or after the $. Or if it is indifferent. or if one is consider "more beautiful" than the other.
If you're certain that a variable's value is a single word (no white space) then it's OK to use $varname or ${varname}. If you can't guarantee this, then you should use "$varname" or "${varname}". Note that bash does word-splitting before interpreting your command, so you may actually get a syntax error if you don't quote the expression, for example
C="white space"
if [ -z $C ]
then
...
fi
will result in syntax error:
-bash: [: white: binary operator expected
while this works fine:
C="white space"
if [ -z "$C" ]
then
...
fi
This is due to the fact after variable expansion in the first, unquoted case bash sees this:
if [ -z white space ]
then
...
fi
and the -z operator expects just one, not two arguments. In the second, quoted case bash sees this:
if [ -z "white space" ]
then
...
fi
i.e. just a single argument as required. Note also that quotes were used in assignment
C="white space"
as it would also produce an error if you wrote
C=white space
since this would mean: execute command space with environment containing an added variable C=white.
So, in general you should quote these expressions to ensure your code is more robust against unforeseen variable values. This is especially true if the variable value comes from input, file etc. It is usually safe to drop the quotes for integer variables or when you just want to display the value of a variable as in echo $C.
It matters when the string contains whitespace characters. Without the quotes, whitespace characters are treated as token delimiters and bash tries to interpret the substituted string as an expression.
Always put quotes to be safe, when you don't intend to evaluate the variable as a part of the expression.
Imagine you change the input from "example" to "two words", then you could encounter strange behaviour or even syntax errors when executing the script, in case you have overlooked the above.
In other words,
C="abc def"
# the echo command receives one argument: "abc def"
echo "$C"
# echo receives two arguments: "abc" and "def"
echo $C
# bash tries to execute the program "abc" with a first argument "def"
$C
# bash tries to execute the program "abc def"
"$C"
A good documentation about quotes and word-spliting :
"USE MORE QUOTES!" They are vital. Also, learn the difference between ' and " and `. See http://mywiki.wooledge.org/Quotes and http://wiki.bash-hackers.org/syntax/words
greybot sample from IRC freenode #bash is talking to the world =)
If it's a one-word constant, it's irrelevant.
However, you should read about the two kinds of quoting. Try this article and this documentation. There is also a SO question.
Try with a real example with whitespace. For the string example you do not need any quoting at all. So create a file called This is an example.txt and then retry. Substitute echo with ls...
This question already has answers here:
Bash semicolon being equal to newline is not exactly true?
(4 answers)
Closed 2 years ago.
I have a shell script which contains the following:
case $1 in
0 )
echo $1 = 0;
OUTPUT=3;;
1 )
echo $1 = 1;
OUTPUT=4;;
2 )
echo $1 = 2;
OUTPUT=4;;
esac
HID=$2;
BUNCH=16;
LR=.008;
Are semicolons completely superfluous in the snippet above? And is there any reason for some people using double semicolons?
It appears semicolons are only a separator, something you would use instead of a new line.
Single semicolons at the end of a line are superfluous, since the newline is also a command separator. case specifically needs double semicolons at the end of the last command in each pattern block; see help case for details.
According to man bash:
metacharacter
A character that, when unquoted, separates words. One of the following:
| & ; ( ) < > space tab
control operator
A token that performs a control function. It is one of the following symbols:
|| & && ; ;; ( ) | |& <newline>
So, the ; can be metacharacter or control operator, while the ;; is always a control operator (in case command).
In your particular code, all ; at the end of line are not needed. The ;; is needed however.
In the special case of find, ; is used to terminate commands invoked by -exec. See the answer of #kenorb to this question.
#Opensourcebook-Amit
newlines equivalent to single semicolon ; on terminal or in shell script.
See the below examples:
On terminal:
[root#server test]# ls;pwd;
On shell script:
[root#server test]# cat test4.sh
echo "Current UserName:"
whoami
echo -e "\nCurrent Date:";date;
[root#server test]#
But I am not agree with the comment that & is equivalent to newline or single semicolon
& is run commands in background also a command separator but not worked as semicolon or newline.