I have a shell script that outputs information about successes to stdout, and also does a grep looking for errors in logs
inner.sh:
# do some things
echo success
# do other things
echo success
grep 'error' logs/*
I have another shell script that calls this one, counts up the successes and compares them to an expected number of successes:
outer.sh:
bash ./inner.sh | grep success | wc -l # I compare this number to the expected number
What I can't figure out how to do is have the output of grep go to stderr, so its not counted by the wc -l in outer.sh, but rather makes it around the wc to the terminal for the operator to see.
So I want a command like stdin_to_stderr that I can pipe the grep to, that would make any results it finds leave inner.sh on its stderr.
Is there already such a thing? Or do I just need to write the tiny script that would do this? Or am I thinking about this wrong?
bash ./inner.sh >&2 output_log_file
grep -c success output_log_file -- Count of success
grep -v -c success output_log_file -- Count of not "success"
Example :
echo -e "success.\nerror.\nsuccess.\nError.\nsuccess.\nerror" | grep -c success
echo -e "success.\nerror.\nsuccess.\nError.\nsuccess." | grep -v -c success
output :
3
2
Related
I have the below code to find out the number of instances of current script running that is running with same arg1. But looks like the script creates a subshell and executes this command which also shows up in output. What would be the better approach to find the number of instances of running script ?
$cat test.sh
#!/bin/bash
num_inst=`ps -ef | grep $0 | grep $1 | wc -l`
echo $num_inst
$ps aux | grep test.sh | grep arg1 | grep -v grep | wc -l
0
$./test.sh arg1 arg2
3
$
I am looking for a solution that matches all running instance of ./test.sh arg1 arg2 not the one with ./test.sh arg10 arg20
The reason this creates a subshell is that there's a pipeline inside the command substitution. If you run ps -ef alone in a command substitution, and then separately process the output from that, you can avoid this problem:
#!/bin/bash
all_processes=$(ps -ef)
num_inst=$(echo "$all_processes" | grep "$0" | grep -c "$1")
echo "$num_inst"
I also did a bit of cleanup on the script: double-quote all variable references to avoid weird parsing, used $() instead of backticks, and replaced grep ... | wc -l with grep -c.
You might also replace the echo "$all_processes" | ... with ... <<<"$all_processes" and maybe the two greps with a single grep -c "$0 $1":
...
num_inst=$(grep -c "$0 $1" <<<"$all_processes")
...
Modify your script like this:
#!/bin/bash
ps -ef | grep $0 | wc -l
No need to store the value in a variable, the result is printed to standard out anyway.
Now why do you get 3?
When you run a command within back ticks (fyi you should use syntax num_inst=$( COMMAND ) and not back ticks), it creates a new sub-shell to run COMMAND, then assigns the stdout text to the variable. So if you remove the use of $(), you will get your expected value of 2.
To convince yourself of that, remove the | wc -l, you will see that num_inst has 3 processes, not 2. The third one exists only for the execution of COMMAND.
On executing the following adb command with grep I get the following response in (darwin os) terminal. i count number of error after grep finds a "Error" string in response
adb -s derefedv645xxxxxx shell "am start -n com.samsung.networkui/.usa.EnhancedLteServices" | grep Error | wc -l
Output:
Error type 3
Error: Activity class {com.samsung.networkui/com.samsung.networkui.usa.EnhancedLteServices} does not exist.
0
Meaning there is no "Error" text in the response. why grep is not able to capture the text "Error" ? let me know if more information is required.
You need to redirect stderr to stdout before using next comment. I would also suggest using awk to avoid 2 commands grep + wc -l:
adb -s derefedv645xxxxxx shell "am start -n com.samsung.networkui/.usa.EnhancedLteServices" 2>&1 |
awk '/Error/ {++n} END {print n}'
I have a shell script, named test.sh :
#!/bin/bash
echo "start"
ps xc | grep test.sh | grep -v grep | wc -l
vartest=`ps xc | grep test.sh | grep -v grep | wc -l `
echo $vartest
echo "end"
The output result is :
start
1
2
end
So my question is, why are there two test.sh processes running when I call ps using `` (the same happens with $()) and not when I call ps directly?
How can I get the desired result (1)?
When you start a subshell, as with the backticks, bash forks itself, then executes the command you wanted to run. You then also run a pipeline which causes all of those to be run in their own subshells, so you end up with the "extra" copy of the script that's waiting for the pipeline to finish so it can gather up the output and return that to the original script.
We'll do a little expermiment using (...) to run processes explicitly in subshells and using the pgrep command which does ps | grep "name" | grep -v grep for us, just showing us the processes that match our string:
echo "Start"
(pgrep test.sh)
(pgrep test.sh) | wc -l
(pgrep test.sh | wc -l)
echo "end"
which on a run for me produces the output:
Start
30885
1
2
end
So we can see that running pgrep test.sh in a subshell only finds the single instance of test.sh, even when that subshell is part of a pipeline itself. However, if the subshell contains a pipeline then we get the forked copy of the script waiting for the pipeline to finish
I'm writing a unix shell script and need to check if there are currently running processes with "xyz" in their directory. If yes than continue to next command and show text like "Found It".
If not than don't continue and display text like "Process Not Found".
I tried something like this:
if ps -ef | grep xyz
then
echo "XYZ Process Found!"
else
echo "XYZ Process Not Found!"
fi
But it just showing me the processes and display "process found" even if there's no xyz process.
I believe you want to check the output of the command against a value using Command substition, from the linked bash-hackers wiki The command substitution expands to the output of commands. These commands are executed in a subshell, and their stdout data is what the substitution syntax expands to. Also, count the lines and remove grep. Something like,
if [[ $(ps -ef | grep xyz | grep -v grep | wc -l) != 0 ]]; then
echo "XYZ Process Found!"
else
echo "XYZ Process Not Found!"
fi
Edit
Based on the comments below, you should probably use
if [[ $(ps -ef | grep -c xyz) -ne 1 ]]; then
which is a lot easier to read.
When you run grep xyz, that process - grep xyz - is also running & thus shown in the output of ps -ef.
This running process command line contains xyz. Thus grep passes that line to output.
Hence you always get zero exit status - i.e. success.
2 Solutions:
use if ps -ef | grep '[x]yz'; then. (You may want to suppress grep output with -q)
The grep command being run is grep [x]yz. This gets printed in ps -ef output.
Obviously, grep filters out this line. [x]yz could be matched with \[x\]yz, not with [x]yz.
use if pgrep -f xyz >/dev/null; then
Check man pgrep for more details..
You can also use pgrep. From pgrep(1):
pgrep looks through the currently running processes and lists the
process IDs which match the selection criteria to stdout.
[...]
EXIT STATUS
0 One or more processes matched the criteria.
1 No processes matched.
2 Syntax error in the command line.
3 Fatal error: out of memory etc.
Example output:
[~]% pgrep xterm
18231
19070
31727
You can use it in an if statement like so:
if pgrep xterm > /dev/null; then
echo Found xterm
else
echo xterm not found
fi
Note: pgrep is not a standard utility (ie. it's not in POSIX), but widely available on at least Linux and I believe most BSD systems.
is_xyz_running() {
[ "$(pgrep xyz)" ] && echo true || echo false
}
This question already has answers here:
Pipe output and capture exit status in Bash
(16 answers)
Closed 5 years ago.
How do I get the correct return code from a unix command line application after I've piped it through another command that succeeded?
In detail, here's the situation :
$ tar -cEvhf - -I ${sh_tar_inputlist} | gzip -5 -c > ${sh_tar_file} -- when only the tar command fails $?=0
$ echo $?
0
And, what I'd like to see is:
$ tar -cEvhf - -I ${sh_tar_inputlist} 2>${sh_tar_error_file} | gzip -5 -c > ${sh_tar_file}
$ echo $?
1
Does anyone know how to accomplish this?
Use ${PIPESTATUS[0]} to get the exit status of the first command in the pipe.
For details, see http://tldp.org/LDP/abs/html/internalvariables.html#PIPESTATUSREF
See also http://cfajohnson.com/shell/cus-faq-2.html for other approaches if your shell does not support $PIPESTATUS.
Look at $PIPESTATUS which is an array variable holding exit statuses. So ${PIPESTATUS[0]} holds the exit status of the first command in the pipe, ${PIPESTATUS[1]} the exit status of the second command, and so on.
For example:
$ tar -cEvhf - -I ${sh_tar_inputlist} | gzip -5 -c > ${sh_tar_file}
$ echo ${PIPESTATUS[0]}
To print out all statuses use:
$ echo ${PIPESTATUS[#]}
Here is a general solution using only POSIX shell and no temporary files:
Starting from the pipeline:
foo | bar | baz
exec 4>&1
error_statuses=`((foo || echo "0:$?" >&3) |
(bar || echo "1:$?" >&3) |
(baz || echo "2:$?" >&3)) 3>&1 >&4`
exec 4>&-
$error_statuses contains the status codes of any failed processes, in random order, with indexes to tell which command emitted each status.
# if "bar" failed, output its status:
echo $error_statuses | grep '1:' | cut -d: -f2
# test if all commands succeeded:
test -z "$error_statuses"
# test if the last command succeeded:
echo $error_statuses | grep '2:' >/dev/null
As others have pointed out, some modern shells provide PIPESTATUS to get this info. In classic sh, it's a bit more difficult, and you need to use a fifo:
#!/bin/sh
trap 'rm -rf $TMPDIR' 0
TMPDIR=$( mktemp -d )
mkfifo ${FIFO=$TMPDIR/fifo}
cmd1 > $FIFO &
cmd2 < $FIFO
wait $!
echo The return value of cmd1 is $?
(Well, you don't need to use a fifo. You can have the commands early in the pipe echo a status variable and eval that in the main shell, redirecting file descriptors all over the place and basically bending over backwards to check things, but using a fifo is much, much easier.)