My question is a twofold.
First:
Is it possible to achieve this without using a subshell?
FOO=$((6-5))
or this?
BAR=`echo "$FOO/100" | bc -l`
If I understand the second one correctly, I'm creating 2 subshells by using ´ and |
Second
Does creating/using subshells for this kind of stuff impact the overall performance of a script?
--
Thanks
You can check count of subprocess allocated with that simple line:
bash -c 'echo $$'
It create new shell and outputs current process id.
Since processes in linux are sequential numbers, we can use this "hack" to detect how many processes was started between commands. (some processes can be started in background by cron or at, so need to check results multiple times).
Also this command will start process, so if you start it multiple time you will see increasing number. To get real count of processes that was started between this command you must substract 1.
So starting checking.
$ bash -c 'echo $$'
4240
$ FOO=$((6-5))
$ bash -c 'echo $$'
4241
4241 - 4240 - 1 = 0. No subprocess was started.
$ FOO=1111
$ bash -c 'echo $$'
4244
$ BAR=`echo "$FOO/100" | bc -l`
$ bash -c 'echo $$'
4248
4248 - 4244 - 1 = 3. There is 3 process was started.
If we start with "here is a string" removing useless echo:
$ bash -c 'echo $$'
4256
$ BAR=`bc -l <<< "$FOO/100"`
$ bash -c 'echo $$'
4259
4259 - 4256 - 1 = 2. Now 2 subprocess was started.
seems like echo implicitly allocates new shell??
backticks allocates new shell - new process to read output
bc allocates new process
This variant will create two subprocess too:
read BAR < <(bc -l <<< "$FOO / 100")
read is a bash command - it does not fork subprocess and executed in the same shell
() will create shell - subprocess
bc will create subprocess
One way to see that $(( ... )) does not invoke a subshell is to modify the value of a variable inside the construct, and see that the change is visible in the current shell.
$ count=5
$ : $(( count++ ))
$ echo $count
6
If a subshell was created by $(( ... )), the output of the following echo would still be 5.
Related
let's say echo command, we can run that command by two ways:
# by 1
echo 'hello'
# or by 2
sh -c "echo 'hello'"
Is there any difference between the two ways? By the way, I can see the way 2 is very popular in yaml config files.
- name: init-mydb
image: busybox:1.28
command: ['sh', '-c', "sleep 2; done"]
The first way calls an inherited command interpreter, eg from a terminal running /bin/bash ; the second way exec sh (aka Bourne Shell) as the interpreter and instruct him ( -c ) to do something.
sh, ksh, csh, bash are all shell interpreters. They provide some features that are not always compatible between them. So, if you don't know the environment where your program will run, the best is to specify the interpreter you want, which is less error prone.
This is a single command:
foo 1 2 3
So is this
sh -c 'foo 1 2 3'
This is not a single command (but rather a pair of commands separated by a ;)
foo; bar
but this is
sh -c "foo; bar"
This does not specify a command using the name of a executable file
for x in 1 2 3; do echo "$x"; done
but this does
sh -c 'for x in 1 2 3; do echo "$x"; done'
sh -c is basically way to specify an arbitrary shell script as a single argument to a command that can be executed from a file.
If I'm using bash and type bash I'm in the child shell and need to type exit to go back to the original parent shell. If I forget which one I'm in how do I check?
Use the SHLVL environment variable.
man bash:
SHLVL : Incremented by one each time an instance of bash is started.
$ echo "$SHLV"
1
$ bash
$ echo "$SHLV"
2
$ exit
$ echo "$SHLV"
1
This is an inferior answer but you can also use pstree:
$ pstree -s $BASHPID
systemd───systemd───gnome-terminal-───bash───pstree
$ bash
$ pstree -s $BASHPID
systemd───systemd───gnome-terminal-───bash───bash───pstree
I have a bash script (this_script.sh) that invokes multiple instances of another TCL script.
set -m
for vars in $( cat vars.txt );
do
exec tclsh8.5 the_script.tcl "$vars" &
done
while [ 1 ]; do fg 2> /dev/null; [ $? == 1 ] && break; done
The multi threading portion was taken from Aleksandr's answer on: Forking / Multi-Threaded Processes | Bash.
The script works perfectly (still trying to figure out the last line). However, this line is always displaed: exec tclsh8.5 the_script.tcl "$vars"
How do I hide that line? I tried running the script as :
bash this_script.sh > /dev/null
But this hides the output of the invoked tcl scripts too (I need the output of the TCL scripts).
I tried adding the /dev/null to the end of the statement within the for statement, but that too did not work either. Basically, I am trying to hide the command but not the output.
You should use $! to get the PID of the background process just started, accumulate those in a variable, and then wait for each of those in turn in a second for loop.
set -m
pids=""
for vars in $( cat vars.txt ); do
tclsh8.5 the_script.tcl "$vars" &
pids="$pids $!"
done
for pid in $pids; do
wait $pid
# Ought to look at $? for failures, but there's no point in not reaping them all
done
I realize the fact but I don't know why:
cat abc | echo $?
if abc does not exist, but above command still return 0. Anyone knows the theory about why?
The reason why it must be this way is that a pipeline is made of processes running simultaneously. cat's exit code can't possibly be passed to echo as an argument because arguments are set when the command begins running, and echo begins running before cat has finished.
echo doesn't take input from stdin, so echo on the right side of a pipe character is always a mistake.
UPDATE:
Since it is now clear that you are asking about a real problem, not just misunderstanding what you saw, I tried it myself. I get what I think is the correct result (1) from a majority of shells I tried (dash, zsh, pdksh, posh, and bash 4.2.37) but 0 from bash 4.1.10 and ksh (Version JM 93u+ 2012-02-29).
I assume the change in bash's behavior between versions is intentional, and the 4.1.x behavior is considered a bug. You'd probably find it in the changelog if you looked hard enough. Don't know about ksh.
csh and tcsh (with $status in place of $?) also say 0, but I bet nobody cares about that.
People with bigger shell collections are invited to test:
for sh in /bin/sh /bin/ksh /bin/bash /bin/zsh ...insert more shells here...; do
echo -n "$sh "
$sh -c 'false;true|echo $?'
done
It does not have anything to do with cat abc, but with the previous command you executed. So the code you get when doing cat abc | echo $? is telling if the previous command in your history was successful or not.
From man bash:
Special Parameters
? - Expands to the exit status of the most recently executed foreground pipeline.
So when you do:
cat abc | echo $?
The echo $? refers to the previous command you used, not cat abc.
Example
$ cat a
hello
$ echo $?
0
$ cat aldsjfaslkdfj
cat: aldsjfaslkdfj: No such file or directory
$ echo $?
1
So
$ cat a
$ cat a | echo $?
0
$ cat aldsjfaslkdfj
cat: aldsjfaslkdfj: No such file or directory
$ cat a | echo $?
1
echo $? will give output of previous command which you have executed before not output of piped command. So, you will always get echo $? as 0 even if command failed before pipe.
You pipe the output from 'cat abc' to 'echo $?' which is not what you want.
You want to echo the exit code of 'cat'
cat abc; echo $?
is what you want. Or simply write it in two lines if you can.
Using echo "blah" | myscript, as an example pipeline am looking for a way to from within "myscript":
echo "$SOME_BUILTIN"
whose output would be the original pipeline:
echo "blah" | myscript
Any ideas?
Here you go:
#!/usr/bin/env bash
siblings=($(pgrep -P "$(ps -p $$ o ppid=)" | grep -Fvx $$))
pids=$$
sibling=$$
while true
do
pid=$sibling
for sibling in ${siblings[#]}
do
if [ "$(readlink "/proc/${sibling}/fd/1")" = "$(readlink "/proc/${pid}/fd/0")" ]
then
# Found previous pipe process
pids="${sibling},${pids}"
continue 2
fi
done
break # No previous pipe process
done
while IFS= read -r line
do
pipeline="${pipeline+${pipeline} | }$line"
done < <(ps --sort=pid -p "$pids" o args=)
printf '%s\n' "$pipeline"
Example:
$ ./test.sh
bash ./test.sh
$ cat /dev/zero | sleep 1 | ./test.sh
cat /dev/zero | sleep 1 | bash ./test.sh
Working on a small project to make this a proper tool.
It's not easy to do this directly, but you could use following information to recreate the pipeline:
Your script's PID is in $$ env variable
Using ps -fe and looking at columns 2 and 3 you can find your script's parent and the parent's parent and so on. The pipeline consists of a series of direct ancestors, and you can follow the path down to process with PID 1 (init process or equivalent).
Knowing each process's PID, you can get its command line from /proc/PID/cmdline (replace '\0''s with spaces using tr for human-readable format
If you know that the script was called from a pipeline, you're essentially done. If you need to find out whether it was called from a pipeline or interactively, you would probably need to check what ls -lh /proc/self/fd/0 prints. That's the 0-th file descriptor, meaning stdin. If it points to /dev/someting (like /dev/pts/XX) you're probably interactive. If the target of the link starts with pipe:[, it's likely you are receiving data from a pipeline (it could also be a network socket but hopefully you can rule this case out as unlikely in your setup).