I am trying to check whether a docker container exists using grep.
The following script signals an error exit during running when IMAGE is empty and is fine when IMAGE is set. I can't find out the cause.
#!/usr/bin/env bash
set -e
IMAGE=$(docker ps | grep my-dev-image)
if [[ -z "$IMAGE" ]]; then
echo "It is not there"
else
echo "It is there"
fi
When you use set -e in a script the shell will exit whenever a command fails. It interacts badly with your grep call because grep exits with an error code if it doesn't find a match. If grep fails then the entire IMAGE=$(...) assignment fails and the script exits instead of setting IMAGE to an empty string.
You can fix this by ensuring the assignment always succeeds. A common idiom for this is to append || :. Adding || cmd will run cmd whenever the command on the left fails. And : is a command that always succeeds. (Yes, it's a command name consisting of a single colon. Strange, but it's a legal identifier.)
IMAGE=$(docker ps | grep my-dev-image) || :
Alternatively, you could check grep's exit code directly by using it in the if statement. This is what I would do if I didn't care about grep's output:
if docker ps | grep -q my-dev-image; then
echo "It is there"
else
echo "It is not there"
fi
Related
I have the following bash script which runs on my CI and intends to run my code on a physical MacOS and on several docker Linux images:
if [[ "$OS_NAME" == "mac_os" ]]; then
make all;
run_test1;
run_test2;
make install;
else
docker exec -i my_docker_image bash -c "make all";
docker exec -i my_docker_image bash -c "run_test1";
docker exec -i my_docker_image bash -c "run_test2";
docker exec -i my_docker_image bash -c "make install";
fi
If the tests fail (run_test1 or run_test2) they return error code 1. If they pass they return error code 0.
The whole script runs with set -e so whenever it sees exit code other than 0 it stops and fails the entire build.
The problem is that currently, when run_test1 and run_test2 are inside the conditional clause - when they fail and return error code 1 the conditional clause doesn't break and the build succeeds although tests didn't pass.
So I have 2 questions:
How to break the conditional clause if one of the commands return error code other than 0?
How to break the conditional clause in such a way that the entire clause will return an error code (so the whole build will fail)?
Your code should work as expected, let's demonstrate this:
#!/usr/bin/env bash
set -e
var="test"
if [[ $var = test ]]; then
echo "Hello"
cat non_existing_file &> /dev/null
echo "World"
else
echo "Else hello"
cat non_existing file &> /dev/null
fi
echo I am done
This will output only "Hello", as expected. If it works differently for you, it means you didn't provide enough code. Let's try to change the code above and show some examples when set -e is ignored, just like in your case:
Quoting Bash Reference manual:
If a compound command or shell function executes in a context where -e
is being ignored, none of the commands executed within the compound
command or function body will be affected by the -e setting, even if
-e is set and a command returns a failure status.
Now let's quote the same manual and see some cases where -e is ignored:
The shell does not exit if the command that fails is part of the
command list immediately following a while or until keyword, part of
the test in an if statement, part of any command executed in a && or
|| list except the command following the final && or ||, any command
in a pipeline but the last, or if the command’s return status is being
inverted with !.
From this we can see that, for example, if you had the code above in a function and tested that function with if, set -e would be ignored:
#!/usr/bin/env bash
set -e
f() {
var="test"
if [[ $var = test ]]; then
echo "Hello"
cat non_existing_file &> /dev/null
echo "World"
else
echo "Else hello"
cat non_existing file &> /dev/null
fi
}
if f; then echo "Function OK!"; fi
echo I am done
Function f is executed in a context where set -e is being ignored (if statement), meaning that set -e doesn't affect any of the commands executed within this function. This code outputs:
Hello
World
Function OK!
I am done
The same rules apply when you execute the function in a && or || list. If you change the line if f; then echo "Function OK!"; fi to f && echo "Function OK", you will get the same output. I believe the latter could be your case.
Even so, your second question can be solved easily by adding || exit:
run_test1 || exit 1;
run_test2 || exit 1;
Your first question is trivial if you are inside a function for example. Then you can simply return. Or inside a loop, then you can break. If you are not, breaking out of the conditional clause is not that easy. Take a look at this answer.
set -e can be a surprising thing as it is being ignored in many cases. Use it with care and be aware of these cases.
I want to build a small script (called check_process.sh) that checks if a certain process $PROC_NAME is running. If it does, it returns its PID or -1 otherwise.
My idea is to use pgrep -f <STRING> in a command substitution.
If I run this code directly in the command line:
export ARG1=foo_name
export RES=$(pgrep -f ${ARG1})
if [[ $RES == "" ]]; then echo "-1" ; else echo "$RES"; fi
everything goes fine: PID or -1 depending on the process status.
My script check_process.sh contains the same lines plus an extra variable to pass the process' name :
#!/bin/bash
export ARG1=$1
export RES=$(pgrep -f ${ARG1})
if [[ $RES == "" ]]; then echo "-1" ; else echo "$RES"; fi
But this code does not work!
If the process is currently running I get two PIDs (the process' PID and something else...), whereas when I check a process that is not running I get the something else !
I am puzzled. Any idea?
Thanks in advance!
If you add the -a flag to pgrep inside your script, you can see something like that (I ran ./check_process.sh vlc):
17295 /usr/bin/vlc --started-from-file ~/test.mkv
18252 /bin/bash ./check_process.sh vlc
So the "something else" is the pid of the running script itself.
The pgrep manual explains the -f flag:
The pattern is normally only matched against the process name. When -f is set, the full command line is used.
Obviously, the script command line contain the lookup process name ('vlc') as an argument, hence it appears at the pgrep -f result.
If you're looking just for the process name matches you can remove the -f flag and get your desired result.
If you wish to stay with the -f flag, you can filter out the current pid:
#!/bin/bash
ARG1=$1
TMP=$(pgrep -f ${ARG1})
RES=$(echo "${TMP}" | grep -v $$)
if [[ $RES == "" ]]; then echo "-1" ; else echo "${RES}"; fi
I have a scenario to copy file from one server to another, for that i need to check any existing scp is in progress, have wrote a sample shell script but the condition is not being met even though syntax is correct, the main problem here is the output of ps command will gets stored in variable scpstat and the same compared for matching string in if statement, here I'm getting the output of the variable is different from executing outside of the script. can see it is formatted different in script execution when executing sh -x scpsamp.sh, why there is "sh" appended to the output, but while comparing without ps and assigning as scpstat='scp' i can able to get the condition correct, am i doing anything wrong while getting output in to the variable. please help
#!/bin/sh
scpstat=`ps -ef | grep scp | egrep -v 'grep|ssh' | awk '{print $8}')`
if [ "$scpstat" = "scp" ];
then
echo "SCP is in progress"
else
echo "No SCP in progress"
fi
sh -x output
It's notoriously difficult to extract information from the output of ps. If your system has pgrep, it's much easier:
if pgrep scp >/dev/null
then
echo "SCP is in progress"
else
echo "No SCP in progress"
fi
I'm writing a unix shell script and need to check if there are currently running processes with "xyz" in their directory. If yes than continue to next command and show text like "Found It".
If not than don't continue and display text like "Process Not Found".
I tried something like this:
if ps -ef | grep xyz
then
echo "XYZ Process Found!"
else
echo "XYZ Process Not Found!"
fi
But it just showing me the processes and display "process found" even if there's no xyz process.
I believe you want to check the output of the command against a value using Command substition, from the linked bash-hackers wiki The command substitution expands to the output of commands. These commands are executed in a subshell, and their stdout data is what the substitution syntax expands to. Also, count the lines and remove grep. Something like,
if [[ $(ps -ef | grep xyz | grep -v grep | wc -l) != 0 ]]; then
echo "XYZ Process Found!"
else
echo "XYZ Process Not Found!"
fi
Edit
Based on the comments below, you should probably use
if [[ $(ps -ef | grep -c xyz) -ne 1 ]]; then
which is a lot easier to read.
When you run grep xyz, that process - grep xyz - is also running & thus shown in the output of ps -ef.
This running process command line contains xyz. Thus grep passes that line to output.
Hence you always get zero exit status - i.e. success.
2 Solutions:
use if ps -ef | grep '[x]yz'; then. (You may want to suppress grep output with -q)
The grep command being run is grep [x]yz. This gets printed in ps -ef output.
Obviously, grep filters out this line. [x]yz could be matched with \[x\]yz, not with [x]yz.
use if pgrep -f xyz >/dev/null; then
Check man pgrep for more details..
You can also use pgrep. From pgrep(1):
pgrep looks through the currently running processes and lists the
process IDs which match the selection criteria to stdout.
[...]
EXIT STATUS
0 One or more processes matched the criteria.
1 No processes matched.
2 Syntax error in the command line.
3 Fatal error: out of memory etc.
Example output:
[~]% pgrep xterm
18231
19070
31727
You can use it in an if statement like so:
if pgrep xterm > /dev/null; then
echo Found xterm
else
echo xterm not found
fi
Note: pgrep is not a standard utility (ie. it's not in POSIX), but widely available on at least Linux and I believe most BSD systems.
is_xyz_running() {
[ "$(pgrep xyz)" ] && echo true || echo false
}
I have a grep command that works in a bash script:
if grep 'stackoverflow' outFile.txt; then
exit 1
fi
This works fine when run on my host. When I call this from a Jenkins build step however, it exits 0 everytime, not seeing 'stackoverflow'. What is going wrong?
Add the following line as the first line in your "Execute Shell" command
#!/bin/sh
grep command exits with a non zero code when it does not find match and that causes jenkins to mark the job as failed. See Below.
In the help section of "Execute Shell"
Runs a shell script (defaults to sh, but this is configurable) for building the project. The script will be run with the workspace as the current directory. Type in the contents of your shell script. If your shell script has no header line like #!/bin/sh —, then the shell configured system-wide will be used, but you can also use the header line to write script in another language (like #!/bin/perl) or control the options that shell uses.
By default, the shell will be invoked with the "-ex" option. So all of the commands are printed before being executed, and the build is considered a failure if any of the commands exits with a non-zero exit code. Again, add the #!/bin/... line to change this behavior.
As a best practice, try not to put a long shell script in here. Instead, consider adding the shell script in SCM and simply call that shell script from Jenkins (via bash -ex myscript.sh or something like that), so that you can track changes in your shell script.
I am a bit confused by the answers on this question! i.e. Sorry, but the answers here are incorrect for this question. The question is good/interesting as plain grep in scripts does cause scripts to exit with failure if the grep is not successful (which can be unexpected), whereas a grep inside an if will not cause exit with failure.
For the example shown in the question exit 1 will be done IF the grep command runs successfully(file exists) AND if the string is found in file. (grep command returns 0 exit code to if).
#Gonen's comment to add 'ls -l outFile.txt' should have been followed up on to see what the real reason for failure was.
TLDR; if catches the exit code of commands inside the if clause:
A grep command that 'fails'(no match or error) inside an if statement in jenkins will not cause jenkins script to stop. Whereas a grep command that fails not inside an if will cause jenkins to stop and exit with fail.
The exit/return code handling is different for commands inside an if statement in shell. if catches the return code and no matter if command was successful or failed the if will return success to $0(after if) (and do actions in if or else).
From man bash:
if list; then list; [ elif list; then list; ] ... [ else list; ] fi
The if list is executed. If its exit status is zero, the then list is executed. Otherwise, each elif list is executed in turn, and
if its exit status is zero, the corresponding then list is executed
and the command completes. Otherwise, the else list is executed, if
present. The exit status is the exit status of the last command
executed, or zero if no condition tested true.
To illustrate, try this (same result in bash or sh):
$ if grep foo bar ; then echo got it; fi; echo $?
grep: bar: No such file or directory
0
$ touch bar
$ if grep foo bar ; then echo got it; fi; echo $?
0
$ echo foo >bar
$ if grep foo bar ; then echo got it; fi; echo $?
foo
got it
0
$ if grep foo bar ; then echo gotit; grep gah mah; fi; echo $?
foo
gotit
grep: mah: No such file or directory
2
I think you have error in your script. You must add 'fi' at the end of 'if' block:
if grep 'stackoverflow' outFile.txt; then
exit 1
fi
If the two were exactly the same it should work. Is your current directory or user different in the two environments? You might not be able to read the file.