This question already has answers here:
grep for expression containing variable
(2 answers)
Closed 9 years ago.
In my bash script, I am attempting to parse through a status file and detect errors based on some keywords. I store these prefixes in a list, and then loop through them.
Bash script:
status_page="/path/to/file.txt"
list="aaa bbb ccc ddd"
for pre in $list
do
echo "grep '\w\w\w${pre}-.*\.lin failed' ${status_page}" # debug
if grep '\w\w\w${pre}-.*\.lin failed' ${status_page}; then
echo "Found error!"
exit 8;
fi
done
/path/to/file.txt:
xyzfff-tool.lin failed
xyzggg-exec.lin failed
rstccc-tool.lin failed
The bash script should catch the line rstccc-tool.lin failed, but it skips right over it.
For debugging, I print the grep commands verbatim, and when I copy that line and issue the command in my shell (tcsh), it returns that line...
Shell:
$ grep '\w\w\wccc-.*\.lin failed' /path/to/file.txt
rstccc-tool.lin failed
$ echo $?
0
If grep can find the line when I issue the command normally, how come it won't find it when the bash script is calling grep?
The variable won't be expanded in single quotes. Try with double quotes:
if grep "\w\w\w${pre}-.*\.lin failed" "${status_page}"; then
The ${pre} portion of that script is not parsing it correctly. I believe you need that line to say:
if grep '\w\w\w'${pre}'-.*\.lin failed' ${status_page}; then
... where the ${pre} is outside the quotation, such that bash will do the correct string replacement before sending it to grep.
Related
This question already has answers here:
Why does shell ignore quoting characters in arguments passed to it through variables? [duplicate]
(3 answers)
Closed 4 years ago.
I am trying to use a variable to store grep's options:
#!/bin/sh
set -xe
GREP_OPTS="-e \"test this\""
echo "I want to test this." | grep $GREP_OPTS
Output:
+ GREP_OPTS=-e "test this"
+ echo I want to test this.
+ grep -e "test this"
grep: this": No such file or directory
how can I make this work?
Word-Splitting is occurring on your GREP_OPTS="-e \"test this\"" resulting in the command being
grep -e '"test' 'this"'
Resulting in the exact error:
grep: this": No such file or directory
(of course there is no file named "this\"")
See BashFAQ-50 - I'm trying to put a command in a variable, but the complex cases always fail.
In order to prevent word splitting use an array for options instead of trying to use a single variable, e.g.
#!/bin/sh
set -xe
GREP_OPTS=(-e "test this")
echo "I want to test this." | grep "${GREP_OPTS[#]}"
Example Use/Output
$ bash grepopts.sh
+ GREP_OPTS=(-e "test this")
+ echo 'I want to test this.'
+ grep -e 'test this'
I want to test this.
Let me know if you have further questions.
This question already has answers here:
Exit when one process in pipe fails
(2 answers)
Closed 4 years ago.
I am trying to understand how error catching works when piped to awk in a bash script. I have a bash script with :
#!/bin/bash
set -e
echo "hello"
OUTPUT=`ls /root/* | awk '{print $0}'`
echo "world"
When I run this as a non-privileged user, the output is
hello
ls: cannot access /root/*: Permission denied
world
despite the fact that set -e should cause my script to end if any error occurs. The line echo "world" should not be executed because the script does generate an error.
Now if I replace the awk statement with grep poo, the script exits as expected.
QUESTION: What is awk doing that manages to hide the errors from bash?
This is completely by definition; the output status of a pipeline is the exit status of the last command in the pipeline.
You can trap this specific error in Awk explicitly:
ls /root/* | awk 'BEGIN {x=1} {x=0; print $NF } END { exit x }'
This causes Awk to set its exit status to reflect whether it received any input at all. This is similar to the grep behavior you discovered (report success if any matches were found).
This question already has answers here:
How to run script commands from variables?
(3 answers)
Execute command in a variable don't execute the latter part of a pipe
(1 answer)
Running a command that is stored in a variable (including pipes and redirects)
(3 answers)
Closed 5 years ago.
#!/bin/bash
# 1st part
ret=$(ps aux | grep -v grep) # thats OK
echo $ret
# 2nd part
cmd="ps aux | grep -v grep" # a problem with the pipe |
ret=$($cmd)
echo $ret
How can I use a command-string as I have in the 2nd part? Think the pipe is the problem. Tried to escape it but it did not help. Get some snytax error of ps.
Thanks!
You need eval:
ret=$(eval "$cmd")
Using eval is not recommended here. It can lead to unexpected results, especially when variables can be read from untrusted sources (See BashFAQ/048 - Eval command and security issues.
You can solve this in a simple way by defining and calling a function as below
ps_cmd() {
ps aux | grep -v grep
}
and use it in the script as
output="$(ps_cmd)"
echo "$output"
Also a good read would be to see why storing commands in a variable is not a good idea and has a lot of potential pitfalls - BashFAQ/050 - I'm trying to put a command in a variable, but the complex cases always fail!
This question already has answers here:
How to run script commands from variables?
(3 answers)
Execute command in a variable don't execute the latter part of a pipe
(1 answer)
Running a command that is stored in a variable (including pipes and redirects)
(3 answers)
Closed 5 years ago.
#!/bin/bash
# 1st part
ret=$(ps aux | grep -v grep) # thats OK
echo $ret
# 2nd part
cmd="ps aux | grep -v grep" # a problem with the pipe |
ret=$($cmd)
echo $ret
How can I use a command-string as I have in the 2nd part? Think the pipe is the problem. Tried to escape it but it did not help. Get some snytax error of ps.
Thanks!
You need eval:
ret=$(eval "$cmd")
Using eval is not recommended here. It can lead to unexpected results, especially when variables can be read from untrusted sources (See BashFAQ/048 - Eval command and security issues.
You can solve this in a simple way by defining and calling a function as below
ps_cmd() {
ps aux | grep -v grep
}
and use it in the script as
output="$(ps_cmd)"
echo "$output"
Also a good read would be to see why storing commands in a variable is not a good idea and has a lot of potential pitfalls - BashFAQ/050 - I'm trying to put a command in a variable, but the complex cases always fail!
This question already has answers here:
What does set -e mean in a bash script?
(10 answers)
Closed 6 years ago.
In a bash script what is the use of
set -e
?
I expect it has something to do with environment variables but I have not come across it before
quoting from help set
-e Exit immediately if a command exits with a non-zero status.
i.e the script or shell would exit as soon as it encounters any command that exited with a non-0(failure) exit code.
Any command that fails would result in the shell exiting immediately.
As an example:
Open up a terminal and type the following:
$ set -e
$ grep abcd <<< "abc"
As soon you hit enter after grep command, the shell exits because grep exited with a non-0 status i.e it couldn't find regex abcd in text abc
Note: to unset this behavior use set +e.
man bash says
Exit immediately if a simple command (see SHELL GRAMMAR above) exits with a non-zero
status. The shell does not exit if the command that fails is part of the command list
immediately following a while or until keyword, part of the test in an if statement,
part of a && or ││ list, or if the command’s return value is being inverted via !. A
trap on ERR, if set, is executed before the shell exits.
It is super convenient way to get "fail-fast" behaviour if you want to avoid testing the return code of every command in a bash script.
Suppose there is no file named trumpet in the current directory below script :
#!/bin/bash
# demonstrates set -e
# set -e means exit immediately if a command exited with a non zero status
set -e
ls trumpet #no such file so $? is non-zero, hence the script aborts here
# you still get ls: cannot access trumpet: No such file or directory
echo "some other stuff" # will never be executed.
You may also combine the e with the x option like set -ex where :
-x Print commands and their arguments as they are executed.
This may help you debugging bash scripts.
Reference:Set Manpage