This question already has answers here:
What does the line '!/bin/sh -e' do? [closed]
(2 answers)
How to echo shell commands as they are executed
(14 answers)
Closed 4 years ago.
In the repository on GitHub of Tribler, I see a file with the interpreter: #!/bin/sh -xe.
On Mac and Linux man sh redirects to BASH(1) (same as man bash).
In this man file, under the section:
OPTIONS
,there is no mention of an option -x , nor an option -e.
What is the meaning of the interpreter #!/bin/sh -xe ?
I found this:
man sh:
-x xtrace Write each command to standard error (preceded by
a ‘+ ’) before it is executed. Useful for debug‐
ging.
-e errexit If not interactive, exit immediately if any
untested command fails. The exit status of a com‐
mand is considered to be explicitly tested if the
command is used to control an if, elif, while, or
until; or if the command is the left hand operand
of an “&&” or “||” operator.
It seems these are just error control options that make sure nothing worse happens if a command fails.
Per man page: read more
The -a, -b, -C, -e, -f, -m, -n, -o option, -u, -v, and -x options are described as part of the set utility in Special Built-In Utilities. The option letters derived from the set special built-in shall also be accepted with a leading plus sign ( '+' ) instead of a leading hyphen (meaning the reverse case of the option as described in this volume of IEEE Std 1003.1-2001).
when you read set man pages you get:
-x The shell shall write to standard error a trace for each command after it expands the command and before it executes it. It is unspecified whether the command that turns tracing off is traced.
and
-e When this option is on, if a simple command fails for any of the reasons listed in Consequences of Shell Errors or returns an exit status value >0, and is not part of the compound list following a while, until, or if keyword, and is not a part of an AND or OR list, and is not a pipeline preceded by the ! reserved word, then the shell shall immediately exit.
Related
This question already has an answer here:
'which' vs 'command -v' in Bash [duplicate]
(1 answer)
Closed 11 months ago.
I dont understand what does this programme does :
if command -v apt-get > /dev/null; then
dpkg-query -W -f='...'
i dont understand how the command : "command" works
There are different ways to do things in the shell.
The default is to execute shell builtins.
Next commands (binaries, scripts, etc; basically something external to the shell) are looked up in the directories listed in the PATH variable.
This is followed by aliases. These are reusable abbreviations to commands, you can think of them as a way to save typing. They are usually defined in the shell's rc files. For example I have this defined in my ~/.bashrc: alias lt='ls --color=tty -lhtr'; it lets me type only lt to get a reverse sorted list of files (newest at the bottom).
More complex commands can be grouped into shell functions.
The command command is a shell builtin that allows you to bypass any functions that might also match a binary in your PATH, or any aliases. Say I use rsync frequently between specific hosts, so I wrote a function called rsync that includes the destination with all the options, allowing me to type less. However if I want to run rsync with a different destination, I could use command rsync <other rsync options> to bypass the function definition. Note that aliases are not bypassed; this is relevant if you are using command interactively. You can find the documentation in the builtin page linked above, or by typing help command
See man bash or help command in bash for help:
Runs COMMAND with ARGS suppressing shell function lookup, ...
The if checks whether there is a command apt-get; if there is a function apt-get, it's ignored.
I have a bash script in which I define the below function,
function start-if-exists()
{
if [ "`docker container ls -a|grep $1`" ]; then
echo "Container $1 exists. Starting $1..."
return `docker start $1`
else
echo "Container $1 doesn't exists."
return ""
fi
}
While executing the above function in terminal(zsh) directly I am not getting any error. But when I execute it using sh command(sh my_script.sh), I am getting the below error.
my_script.sh: line 10: `start-if-exists': not a valid identifier
where my_script.sh is the name of file.
What am I missing that my script works with zsh but fails in sh?
/bin/sh is bash, but, when started as /bin/sh, it starts in POSIX mode. According to the bash man page, in POSIX mode:
Function names must be valid shell `name's. That is, they may not
contain characters other than letters, digits, and underscores, and
may not start with a digit. Declaring a function with an invalid
name causes a fatal syntax error in non-interactive shells.
A note about how to figure things like this out:
At the Terminal command line, I executed /bin/sh --version to see information about it. It printed “GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)”.
Then I referred to the bash man page, using the command man bash. Since that is long, you might prefer to save a copy to a file and view it in your preferred text editor. The raw man output includes archaic underscores and backspaces. You can get a copy without these by executing man bash | col -b > file.txt.
In the man page, I searched for ”sh” (as a complete word, not a raw search for those letters, since they appear as parts of many unrelated words). This quickly revealed discussion that bash behaves differently when started as “sh”; it starts in POSIX mode.
Searching further for “POSIX” revealed a list of things that are different in POSIX mode.
This question already has answers here:
What does `set -x` do?
(3 answers)
Closed 4 years ago.
So I have a file deploy.sh, and it has the shell script. Since I know about it, I have a little confusion, that is what does that set -x actually means.
After running the file I have observed that the command written after it in the file gets mentioned in the terminal with a + sign.
Like if I have this,
#!/bin/bash
set -x
ng build
So the output mentions +ng build, and when I comment the set -x from the file, everything executes, but the later commands does not show up in the terminal.
I have researched about it, but specifically couldn't find the real meaning and work of this particular command.
You can read the bash online manual for set:
-x
Print a trace of simple commands, for commands, case commands, select
commands, and arithmetic for commands and their arguments or
associated word lists after they are expanded and before they are
executed. The value of the PS4 variable is expanded and the resultant
value is printed before the command and its expanded arguments.
So it does exactly what you described.
It's a bit hard to find but this is from the bash man page:
set [+abefhkmnptuvxBCEHPT] [+o option-name] [arg ...]
Without options, the name and value of each shell variable are
displayed in a format that can be reused as input for setting or
resetting the currently-set variables. Read-only variables can‐
not be reset. In posix mode, only shell variables are listed.
The output is sorted according to the current locale. When
options are specified, they set or unset shell attributes. Any
arguments remaining after option processing are treated as val‐
ues for the positional parameters and are assigned, in order, to
$1, $2, ... $n. Options, if specified, have the following
meanings:
[...]
-x After expanding each simple command, for command, case
command, select command, or arithmetic for command, dis‐
play the expanded value of PS4, followed by the command
and its expanded arguments or associated word list.
So basically it's a debug option. It does not only execute all the commands but also prints them before it executes them (to stderr).
This question already has answers here:
How do I syntax check a Bash script without running it?
(10 answers)
Closed 6 years ago.
The following shell script executes well when provided /bin/true for the first argument, but may otherwise fail with a syntax error during execution!
#!/bin/sh
if $1 ; then exit; fi
/tmp/asdf <<< ASDF # Something with syntax error in POSIX
Surely some syntax errors (if not all?) can be avoided by static checking? How do I statically check whether a given Shell Command Language script is syntactically valid?
EDIT: Checking for syntax errors in Bash scripts has been answered in this question.
EDIT #2: Note that Bash fails to properly check whether the syntax adheres to POSIX even when executed with the +B and --posix flags in addition to -n.
All POSIX-compatible Shell Command Language shells support the set -n built-in which can be used to check the syntax of the script. Therefore it is possible to prepend
set -n
to your code to syntax check it. Note also that standard sh utility is also required to support a command-line -n flag, which has equivalent semantics to using set -n. Bash and possibly other shells also support this command-line flag. Therefore you can simply run the following to syntax check your script:
sh -n yourScriptFilename.sh
WARNING: This does not give you a guarantee that the script has fully POSIX compatible syntax. For example, Bash allows bashisms (e.g. arrays and c{a,u}t) to go unnoticed even when using the --posix (and/or +B) command line option in addition to -n when invoked as sh. Other shells might have similar issues.
With bash you can use -n:
bash -n file.sh
Output:
a.sh: line 3: syntax error near unexpected token `then'
a.sh: line 3: `if then fi # Something with syntax error'
Since bash supports the --posix options you may run
bash --posix -n file.sh
to perform a posix compatible check. I don't know how posixly correct that mode is in detail.
Consider this little shell script.
# Save the first command line argument
cmd="$1"
# Execute the command specified in the first command line argument
out=$($cmd)
# Do something with the output of the specified command
# Here we do a silly thing, like make the output all uppercase
echo "$out" | tr -s "a-z" "A-Z"
The script executes the command specified as the first argument, transforms the output obtained from that command and prints it to standard output. This script may be executed in this manner.
sh foo.sh "echo select * from table"
This does not do what I want. It may print something like the following,
$ sh foo.sh "echo select * from table"
SELECT FILEA FILEB FILEC FROM TABLE
if fileA, fileB and fileC is present in the current directory.
From a user perspective, this command is reasonable. The user has quoted the * in the command line argument, so the user doesn't expect the * to be globbed. But my script astonishes the user by using this argument in a command substitution which causes globbing of * as seen in the above output.
I want the output to be the following instead.
SELECT * FROM TABLE
The entire text in cmd actually comes from command line arguments to the script so I would like to preserve any * symbol present in the argument without globbing them.
I am looking for a solution that works for any POSIX shell.
One solution I have come up with is to disable globbing with set -o noglob just before the command substitution. Here is the complete code.
# Save the first command line argument
cmd="$1"
# Execute the command specified in the first command line argument
set -o noglob
out=$($cmd)
# Do something with the output of the specified command
# Here we do a silly thing, like make the output all uppercase
echo "$out" | tr -s "a-z" "A-Z"
This does what I expect.
$ sh foo.sh "echo select * from table"
SELECT * FROM TABLE
Apart from this, is there any other concept or trick (such as a quoting mechanism) I need to be aware of to disable globbing only within a command substitution without having to use set -o noglob.
I am not against set -o noglob. I just want to know if there is another way. You know, globbing can be disabled for normal command line arguments just by quoting them, so I was wondering if there is anything similar for command substiution.
If I understand correctly, you want the user to provide a shell command as a command-line argument, which will be executed by the script, and is expected to produce an SQL string, which will be processed (upper-cased) and echoed to stdout.
The first thing to say is that there is no point in having the user provide a shell command that the script just blindly executes. If the script applied some kind of modification/preprocessing of the command before it executed it then perhaps it could make sense, but if not, then the user might as well execute the command himself and pass the output to the script as a command-line argument, or via stdin.
But that being said, if you really want to do it this way, then there are two things that need to be said. Firstly, this is the proper form to use:
out=$(eval "$cmd");
A fairly advanced understanding of the shell grammer and expansion rules would be required to fully understand the rationale for using the above syntax, but basically executing $cmd and executing eval "$cmd" have subtle differences that render the $cmd form inappropriate for executing a given shell command string.
Just to give some detail that will hopefully clarify the above point, there are seven kinds of expansion that are performed by the shell in the following order when processing input: (1) brace expansion, (2) tilde expansion, (3) parameter and variable expansion, (4) arithmetic expansion, (5) command substitution, (6) word splitting, and (7) pathname expansion. Notice that variable expansion happens somewhat in the middle of that sequence, and thus the variable-expanded shell command (which was provided by the user) will not receive the benefit of the prior expansion types. Other issues are that leading variable assignments, pipelines, and command list tokens will not be executed correctly under the $cmd form, because they are parsed and processed prior to variable expansion (actually prior to all expansions) as well.
By running the command through eval, properly double-quoted, you ensure that the full shell parsing/processing/execution algorithm will be applied to the shell command string that was given by the user of your script.
The second thing to say is this: If you try the above proper form in your script, you will find that it has not solved your problem. You will still get SELECT FILEA FILEB FILEC FROM TABLE as output.
The reason is this: Since you've decided you want to accept an arbitrary shell command from the user of your script, it is now the user's responsibility to properly quote all metacharacters that may be embedded in that piece of code. It does not make sense for you to accept a shell command as a command-line argument, but somehow change the processing rules for shell commands so that certain metacharacters will no longer be metacharacters when the given shell command is executed. Actually, you could do something like that, perhaps using set -o noglob as you discovered, but then that must become a contract between the script and the user of the script; the user must be made aware of exactly what the precise processing rules will be when the command is executed so that he can properly use the script.
Under this design, the user could call the script as follows (notice the extra layer of quoting for the shell command string evaluation; could alternatively backslash-escape just the asterisk):
$ sh foo.sh "echo 'select * from table'";
I'd like to return to my earlier comment about the overall design; it doesn't really make sense to do it this way. It makes more sense to take the text-to-process itself, not a shell command that is expected to produce the text-to-process.
Here is how that could be done:
## take the text-to-process via a command-line argument
sql="$1";
## process and echo it
echo "$sql"| tr a-z A-Z;
(I also removed the -s option of tr, which really doesn't make sense here.)
Notice that the script is simpler now, and usage is also simpler:
$ sh foo.sh 'select * from table';