Command substitution and $PATH variable - bash

Background
This [ article ] says :
The command substitution expands to the output of commands. These
commands are executed in a subshell ..
But the bash manual says nothing about a subshell in its command substitution section.
My test below
$ ps
PID TTY TIME CMD
26483 pts/25 00:00:00 bash
26866 pts/25 00:00:00 ps
$ hpid="$(ps | grep bash)"
$ echo "$hpid"
26483 pts/25 00:00:00 bash
26899 pts/25 00:00:00 bash
shows that a new shell with pid 26899 was spawned during the command substitution. At this point I changed the PATH environment variable.
$ PATH="/some/rogue/path"
did the below stuff :
VAR="$(echo "Do|Die" | cut -d"|" -f 2)"
and got the below error :
Command 'cut' is available in '/usr/bin/cut'
The command could not be located because '/usr/bin' is not included in the PATH environment variable.
cut: command not found
I understand that the error is due to the modification of PATH environment variable which helps the shell locate the binaries. However I am confused when reading this together with command substitution.
If by $(..) a subshell is spawned, then PATH environment variable should be intact and should point to the binary (cut in this case) and so bash should not complain that it cannot locate the cut binary.
Question
How did the modification of the PATH affect the command substitution here?

Consider below example:
$ export PS1='\$\$=$$ \$ '
$$=30862 $ a=123 # Note: No export a here.
$$=30862 $ echo $a
123
$$=30862 $ bash
$$=31133 $ echo $a # Subshell explicitly created does not have it.
$$=31133 $ exit
$$=30862 $ echo $(eval 'echo $a') # This subshell however does inherit it. The single quote ensures that this is not evaluated by parent shell.
123 # echo $(echo $a) would probably cause $a to be evaluated by parent shell.
$$=30862 $
In short, subshells spawned by $(...) inherit same environment as parent shell, even if the variable is not exported. (Even $$ is same as parent shell.)

Related

How can I tell if I'm in a child shell

If I'm using bash and type bash I'm in the child shell and need to type exit to go back to the original parent shell. If I forget which one I'm in how do I check?
Use the SHLVL environment variable.
man bash:
SHLVL : Incremented by one each time an instance of bash is started.
$ echo "$SHLV"
1
$ bash
$ echo "$SHLV"
2
$ exit
$ echo "$SHLV"
1
This is an inferior answer but you can also use pstree:
$ pstree -s $BASHPID
systemd───systemd───gnome-terminal-───bash───pstree
$ bash
$ pstree -s $BASHPID
systemd───systemd───gnome-terminal-───bash───bash───pstree

How to save PID to variable in bash -c?

I'm having trouble with saving the PID into a variable when I'm using bash -c. For example:
bash -c "PID=$$; echo $PID"
In this case the output is empty. How to save the child PID (the PID of the command inside the double quotes) now in the variable PID
just use simple quotes or you expression is evaluated inside your current command line (too soon) and not in the child bash command, and PID isn't defined yet at this moment and you're actually passing
bash -c "PID=4353; echo"
(where 4353 is the pid of the current bash process)
Someone noted that it's not clear if you want to pass parent pid or child pid
to pass parent pid, fix it like this (only the part within double quotes is evaluated before calling bash child process):
bash -c "PID=$$; "'echo $PID'
to pass child pid, fix it like this (nothing is evaluated in the current shell, same trick used for awk scripts):
bash -c 'PID=$$; echo $PID'
set -x is useful for debugging and will show the actual command you end up running prefixed with +:
$ set -x
$ bash -c "PID=$$; echo $PID"
+ bash -c 'PID=1900; echo '
And indeed, in that command you would expect empty output. This happens because $$ and $PID are substituted before bash is called.
To avoid this, you can single quote the string or escape the "$"s:
$ set -x
$ bash -c 'PID=$$; echo $PID'
+ bash -c 'PID=$$; echo $PID'
1925
$ bash -c "PID=\$\$; echo \$PID"
+ bash -c 'PID=$$; echo $PID'
1929

cygwin 1.7.15 handling of "set -e" in shell scripts (error in sub-shell causes parent to exit)

I am using this:
$ uname -a
CYGWIN_NT-6.1 bassoon 1.7.15(0.260/5/3) 2012-05-09 10:25 i686 Cygwin
$ bash --version
GNU bash, version 4.1.10(4)-release (i686-pc-cygwin)
$ cat myexpr.sh
#!/bin/sh
echo "In myexpr, Before expr"
ac_optarg=`expr x--with-gnu-as : 'x[^=]*=\(.*\)'`
echo "ac_optarg=$ac_optarg"
echo "In myexpr, After expr"
$ cat myexpr2.sh
#!/bin/sh
set -e
echo "In myexpr, Before expr"
ac_optarg=`expr x--with-gnu-as : 'x[^=]*=\(.*\)'`
echo "ac_optarg=$ac_optarg"
echo "In myexpr, After expr"
The only difference between the two scripts is that myexpr2.sh uses "set -e"
$ echo $$
2880
$ ./myexpr.sh
In myexpr, Before expr
ac_optarg=
In myexpr, After expr
$ ./myexpr2.sh
In myexpr, Before expr
Expected behavior, so far.
If I do this in the parent shell (PID 2880, above):
$ set -e
$ ./myexpr.sh
The parent shell exits! That is pID 2880 above where I did the "set -e"
This is not the behavior on Linux or cygwin 1.5.12. Is this a bug in cygwin or BASH on cygwin?
This is not a bug, it's a feature of the Bash environment. This happens when you don't have the Bash shell environment variable execfail set, and/or the Shell environment variable errexit.
execfail - (is a BASHOPTS)
If set, a non-interactive shell will not exit if it cannot execute
the file specified as an argument to the exec builtin command.
An interactive shell does not exit if exec fails.
errexit - (is a SHELLOPTS)
Exit immediately if a pipeline (see Pipelines), which may consist of a
single simple command (see Simple Commands), a subshell command enclosed
in parentheses (see Command Grouping), or one of the commands executed as
part of a command list enclosed by braces (see Command Grouping) returns a
non-zero status. The shell does not exit if the command that fails is part
of the command list immediately following a while or until keyword, part
of the test in an if statement, part of any command executed in a && or ||
list except the command following the final && or ||, any command in a
pipeline but the last, or if the command’s return status is being inverted
with !. A trap on ERR, if set, is executed before the shell exits.
This option applies to the shell environment and each subshell environment
separately (see Command Execution Environment), and may cause subshells to
exit before executing all the commands in the subshell.
Different Linux versions have different defaults for these.
You can check which are enabled with:
echo "SHELLOPTS=$SHELLOPTS"
echo "BASHOPTS=$BASHOPTS"
and you can see all of them using:
set -o && echo -e "\n" && shopt -p
So, you need to enable yours with:
shopt -s execfail
If that doesn't work, you may also have to unset (off) the errexit of $SHELLOPTS with:
set -o errexit
For further info, see: The GNU Bash Manual!
PS. "set" is using reverse logic so if you wanna use the 'e' flag you have to use a "+": set +e

How can a ksh script determine the full path to itself, when sourced from another?

How can a script determine it's path when it is sourced by ksh? i.e.
$ ksh ". foo.sh"
I've seen very nice ways of doing this in BASH posted on stackoverflow and elsewhere but haven't yet found a ksh method.
Using "$0" doesn't work. This simply refers to "ksh".
Update: I've tried using the "history" command but that isn't aware of the history outside the current script.
$ cat k.ksh
#!/bin/ksh
. j.ksh
$ cat j.ksh
#!/bin/ksh
a=$(history | tail -1)
echo $a
$ ./k.ksh
270 ./k.ksh
I would want it echo "* ./j.ksh".
If it's the AT&T ksh93, this information is stored in the .sh namespace, in the variable .sh.file.
Example
sourced.sh:
(
echo "Sourced: ${.sh.file}"
)
Invocation:
$ ksh -c '. ./sourced.sh'
Result:
Sourced: /var/tmp/sourced.sh
The .sh.file variable is distinct from $0. While $0 can be ksh or /usr/bin/ksh, or the name of the currently running script, .sh.file will always refer to the file for the current scope.
In an interactive shell, this variable won't even exist:
$ echo ${.sh.file:?}
-ksh: .sh.file: parameter not set
I believe the only portable solution is to override the source command:
source() {
sourced=$1
. "$1"
}
And then use source instead of . (the script name will be in $sourced).
The difference of course between sourcing and forking is that sourcing results in the invoked script being executed within the calling process. Henk showed an elegant solution in ksh93, but if, like me, you're stuck with ksh88 then you need an alternative. I'd rather not change the default ksh method of sourcing by using C-shell syntax, and at work it would be against our coding standards, so creating and using a source() function would be unworkable for me. ps, $0 and $_ are unreliable, so here's an alternative:
$ cat b.sh ; cat c.sh ; ./b.sh
#!/bin/ksh
export SCRIPT=c.sh
. $SCRIPT
echo "PPID: $$"
echo "FORKING c.sh"
./c.sh
If we set the invoked script in a variable, and source it using the variable, that variable will be available to the invoked script, since they are in the same process space.
#!/bin/ksh
arguments=$_
pid=$$
echo "PID:$pid"
command=`ps -o args -p $pid | tail -1`
echo "COMMAND (from ps -o args of the PID): $command"
echo "COMMAND (from c.sh's \$_ ) : $arguments"
echo "\$SCRIPT variable: $SCRIPT"
echo dirname: `dirname $0`
echo ; echo
Output is as follows:
PID:21665
COMMAND (from ps -o args of the PID): /bin/ksh ./b.sh
COMMAND (from c.sh's $_ ) : SCRIPT=c.sh
$SCRIPT variable: c.sh
dirname: .
PPID: 21665
FORKING c.sh
PID:21669
COMMAND (from ps -o args of the PID): /bin/ksh ./c.sh
COMMAND (from c.sh's $_ ) : ./c.sh
$SCRIPT variable: c.sh
dirname: .
So when we set the SCRIPT variable in the caller script, the variable is either accessible from the sourced script's operands, or, in the case of a forked process, the variable along with all other environment variables of the parent process are copied for the child process. In either case, the SCRIPT variable can contain your command and arguments, and will be accessible in the case of both sourcing and forking.
You should find it as last command in the history.

How to determine the current interactive shell that I'm in (command-line)

How can I determine the current shell I am working on?
Would the output of the ps command alone be sufficient?
How can this be done in different flavors of Unix?
There are three approaches to finding the name of the current shell's executable:
Please note that all three approaches can be fooled if the executable of the shell is /bin/sh, but it's really a renamed bash, for example (which frequently happens).
Thus your second question of whether ps output will do is answered with "not always".
echo $0 - will print the program name... which in the case of the shell is the actual shell.
ps -ef | grep $$ | grep -v grep - this will look for the current process ID in the list of running processes. Since the current process is the shell, it will be included.
This is not 100% reliable, as you might have other processes whose ps listing includes the same number as shell's process ID, especially if that ID is a small number (for example, if the shell's PID is "5", you may find processes called "java5" or "perl5" in the same grep output!). This is the second problem with the "ps" approach, on top of not being able to rely on the shell name.
echo $SHELL - The path to the current shell is stored as the SHELL variable for any shell. The caveat for this one is that if you launch a shell explicitly as a subprocess (for example, it's not your login shell), you will get your login shell's value instead. If that's a possibility, use the ps or $0 approach.
If, however, the executable doesn't match your actual shell (e.g. /bin/sh is actually bash or ksh), you need heuristics. Here are some environmental variables specific to various shells:
$version is set on tcsh
$BASH is set on bash
$shell (lowercase) is set to actual shell name in csh or tcsh
$ZSH_NAME is set on zsh
ksh has $PS3 and $PS4 set, whereas the normal Bourne shell (sh) only has $PS1 and $PS2 set. This generally seems like the hardest to distinguish - the only difference in the entire set of environment variables between sh and ksh we have installed on Solaris boxen is $ERRNO, $FCEDIT, $LINENO, $PPID, $PS3, $PS4, $RANDOM, $SECONDS, and $TMOUT.
ps -p $$
should work anywhere that the solutions involving ps -ef and grep do (on any Unix variant which supports POSIX options for ps) and will not suffer from the false positives introduced by grepping for a sequence of digits which may appear elsewhere.
Try
ps -p $$ -oargs=
or
ps -p $$ -ocomm=
If you just want to ensure the user is invoking a script with Bash:
if [ -z "$BASH" ]; then echo "Please run this script $0 with bash"; exit; fi
or ref
if [ -z "$BASH" ]; then exec bash $0 ; exit; fi
You can try:
ps | grep `echo $$` | awk '{ print $4 }'
Or:
echo $SHELL
$SHELL need not always show the current shell. It only reflects the default shell to be invoked.
To test the above, say bash is the default shell, try echo $SHELL, and then in the same terminal, get into some other shell (KornShell (ksh) for example) and try $SHELL. You will see the result as bash in both cases.
To get the name of the current shell, Use cat /proc/$$/cmdline. And the path to the shell executable by readlink /proc/$$/exe.
There are many ways to find out the shell and its corresponding version. Here are few which worked for me.
Straightforward
$> echo $0 (Gives you the program name. In my case the output was -bash.)
$> $SHELL (This takes you into the shell and in the prompt you get the shell name and version. In my case bash3.2$.)
$> echo $SHELL (This will give you executable path. In my case /bin/bash.)
$> $SHELL --version (This will give complete info about the shell software with license type)
Hackish approach
$> ******* (Type a set of random characters and in the output you will get the shell name. In my case -bash: chapter2-a-sample-isomorphic-app: command not found)
ps is the most reliable method. The SHELL environment variable is not guaranteed to be set and even if it is, it can be easily spoofed.
I have a simple trick to find the current shell. Just type a random string (which is not a command). It will fail and return a "not found" error, but at start of the line it will say which shell it is:
ksh: aaaaa: not found [No such file or directory]
bash: aaaaa: command not found
I have tried many different approaches and the best one for me is:
ps -p $$
It also works under Cygwin and cannot produce false positives as PID grepping. With some cleaning, it outputs just an executable name (under Cygwin with path):
ps -p $$ | tail -1 | awk '{print $NF}'
You can create a function so you don't have to memorize it:
# Print currently active shell
shell () {
ps -p $$ | tail -1 | awk '{print $NF}'
}
...and then just execute shell.
It was tested under Debian and Cygwin.
The following will always give the actual shell used - it gets the name of the actual executable and not the shell name (i.e. ksh93 instead of ksh, etc.). For /bin/sh, it will show the actual shell used, i.e. dash.
ls -l /proc/$$/exe | sed 's%.*/%%'
I know that there are many who say the ls output should never be processed, but what is the probability you'll have a shell you are using that is named with special characters or placed in a directory named with special characters? If this is still the case, there are plenty of other examples of doing it differently.
As pointed out by Toby Speight, this would be a more proper and cleaner way of achieving the same:
basename $(readlink /proc/$$/exe)
My variant on printing the parent process:
ps -p $$ | awk '$1 == PP {print $4}' PP=$$
Don't run unnecessary applications when AWK can do it for you.
Provided that your /bin/sh supports the POSIX standard and your system has the lsof command installed - a possible alternative to lsof could in this case be pid2path - you can also use (or adapt) the following script that prints full paths:
#!/bin/sh
# cat /usr/local/bin/cursh
set -eu
pid="$$"
set -- sh bash zsh ksh ash dash csh tcsh pdksh mksh fish psh rc scsh bournesh wish Wish login
unset echo env sed ps lsof awk getconf
# getconf _POSIX_VERSION # reliable test for availability of POSIX system?
PATH="`PATH=/usr/bin:/bin:/usr/sbin:/sbin getconf PATH`"
[ $? -ne 0 ] && { echo "'getconf PATH' failed"; exit 1; }
export PATH
cmd="lsof"
env -i PATH="${PATH}" type "$cmd" 1>/dev/null 2>&1 || { echo "$cmd not found"; exit 1; }
awkstr="`echo "$#" | sed 's/\([^ ]\{1,\}\)/|\/\1/g; s/ /$/g' | sed 's/^|//; s/$/$/'`"
ppid="`env -i PATH="${PATH}" ps -p $pid -o ppid=`"
[ "${ppid}"X = ""X ] && { echo "no ppid found"; exit 1; }
lsofstr="`lsof -p $ppid`" ||
{ printf "%s\n" "lsof failed" "try: sudo lsof -p \`ps -p \$\$ -o ppid=\`"; exit 1; }
printf "%s\n" "${lsofstr}" |
LC_ALL=C awk -v var="${awkstr}" '$NF ~ var {print $NF}'
My solution:
ps -o command | grep -v -e "\<ps\>" -e grep -e tail | tail -1
This should be portable across different platforms and shells. It uses ps like other solutions, but it doesn't rely on sed or awk and filters out junk from piping and ps itself so that the shell should always be the last entry. This way we don't need to rely on non-portable PID variables or picking out the right lines and columns.
I've tested on Debian and macOS with Bash, Z shell (zsh), and fish (which doesn't work with most of these solutions without changing the expression specifically for fish, because it uses a different PID variable).
If you just want to check that you are running (a particular version of) Bash, the best way to do so is to use the $BASH_VERSINFO array variable. As a (read-only) array variable it cannot be set in the environment,
so you can be sure it is coming (if at all) from the current shell.
However, since Bash has a different behavior when invoked as sh, you do also need to check the $BASH environment variable ends with /bash.
In a script I wrote that uses function names with - (not underscore), and depends on associative arrays (added in Bash 4), I have the following sanity check (with helpful user error message):
case `eval 'echo $BASH#${BASH_VERSINFO[0]}' 2>/dev/null` in
*/bash#[456789])
# Claims bash version 4+, check for func-names and associative arrays
if ! eval "declare -A _ARRAY && func-name() { :; }" 2>/dev/null; then
echo >&2 "bash $BASH_VERSION is not supported (not really bash?)"
exit 1
fi
;;
*/bash#[123])
echo >&2 "bash $BASH_VERSION is not supported (version 4+ required)"
exit 1
;;
*)
echo >&2 "This script requires BASH (version 4+) - not regular sh"
echo >&2 "Re-run as \"bash $CMD\" for proper operation"
exit 1
;;
esac
You could omit the somewhat paranoid functional check for features in the first case, and just assume that future Bash versions would be compatible.
None of the answers worked with fish shell (it doesn't have the variables $$ or $0).
This works for me (tested on sh, bash, fish, ksh, csh, true, tcsh, and zsh; openSUSE 13.2):
ps | tail -n 4 | sed -E '2,$d;s/.* (.*)/\1/'
This command outputs a string like bash. Here I'm only using ps, tail, and sed (without GNU extesions; try to add --posix to check it). They are all standard POSIX commands. I'm sure tail can be removed, but my sed fu is not strong enough to do this.
It seems to me, that this solution is not very portable as it doesn't work on OS X. :(
echo $$ # Gives the Parent Process ID
ps -ef | grep $$ | awk '{print $8}' # Use the PID to see what the process is.
From How do you know what your current shell is?.
This is not a very clean solution, but it does what you want.
# MUST BE SOURCED..
getshell() {
local shell="`ps -p $$ | tail -1 | awk '{print $4}'`"
shells_array=(
# It is important that the shells are listed in descending order of their name length.
pdksh
bash dash mksh
zsh ksh
sh
)
local suited=false
for i in ${shells_array[*]}; do
if ! [ -z `printf $shell | grep $i` ] && ! $suited; then
shell=$i
suited=true
fi
done
echo $shell
}
getshell
Now you can use $(getshell) --version.
This works, though, only on KornShell-like shells (ksh).
Do the following to know whether your shell is using Dash/Bash.
ls –la /bin/sh:
if the result is /bin/sh -> /bin/bash ==> Then your shell is using Bash.
if the result is /bin/sh ->/bin/dash ==> Then your shell is using Dash.
If you want to change from Bash to Dash or vice-versa, use the below code:
ln -s /bin/bash /bin/sh (change shell to Bash)
Note: If the above command results in a error saying, /bin/sh already exists, remove the /bin/sh and try again.
I like Nahuel Fouilleul's solution particularly, but I had to run the following variant of it on Ubuntu 18.04 (Bionic Beaver) with the built-in Bash shell:
bash -c 'shellPID=$$; ps -ocomm= -q $shellPID'
Without the temporary variable shellPID, e.g. the following:
bash -c 'ps -ocomm= -q $$'
Would just output ps for me. Maybe you aren't all using non-interactive mode, and that makes a difference.
Get it with the $SHELL environment variable. A simple sed could remove the path:
echo $SHELL | sed -E 's/^.*\/([aA-zZ]+$)/\1/g'
Output:
bash
It was tested on macOS, Ubuntu, and CentOS.
On Mac OS X (and FreeBSD):
ps -p $$ -axco command | sed -n '$p'
Grepping PID from the output of "ps" is not needed, because you can read the respective command line for any PID from the /proc directory structure:
echo $(cat /proc/$$/cmdline)
However, that might not be any better than just simply:
echo $0
About running an actually different shell than the name indicates, one idea is to request the version from the shell using the name you got previously:
<some_shell> --version
sh seems to fail with exit code 2 while others give something useful (but I am not able to verify all since I don't have them):
$ sh --version
sh: 0: Illegal option --
echo $?
2
One way is:
ps -p $$ -o exe=
which is IMO better than using -o args or -o comm as suggested in another answer (these may use, e.g., some symbolic link like when /bin/sh points to some specific shell as Dash or Bash).
The above returns the path of the executable, but beware that due to /usr-merge, one might need to check for multiple paths (e.g., /bin/bash and /usr/bin/bash).
Also note that the above is not fully POSIX-compatible (POSIX ps doesn't have exe).
Kindly use the below command:
ps -p $$ | tail -1 | awk '{print $4}'
This one works well on Red Hat Linux (RHEL), macOS, BSD and some AIXes:
ps -T $$ | awk 'NR==2{print $NF}'
alternatively, the following one should also work if pstree is available,
pstree | egrep $$ | awk 'NR==2{print $NF}'
You can use echo $SHELL|sed "s/\/bin\///g"
And I came up with this:
sed 's/.*SHELL=//; s/[[:upper:]].*//' /proc/$$/environ

Resources