Match Substring from Command Output - bash

There are some close candidates for this question having already been answered, and I've tried several methods of trying to solve the issue. Specifically, my scenario is this:
I have an array of utility names that may or may NOT be installed on a linux machine (e.g.: ssh, sudo, etc.), so I am trying to check if the utility exists or not based on the result of trying to invoke the utilities in turn. I'm trying to do this in bash. Bash version is 4.1.5(1) running on Ubuntu 10.10 but planned to deploy on BusyBox.
If the utility doesn't exist, then usually you get a message saying "not found" or it includes that exact string. Otherwise you get a usage message. I have tried some regex expressions for the grep I use, but it hasn't made any difference, which leads me to believe there is something more fundamentally flawed with my code.
I am fully aware there are utilities that do this, but with the environment I am working in I do not have access to things like dpkg to check utilities/packages. In short, the environment I plan to deploy this on has NO PACKAGE MANAGEMENT.
What I have roughly goes like this:
#!/bin/bash
TOOLS=( 'ssh' 'soodo' 'dhclient' 'iperf')
#list of tools is abridged for convenience and added 'soodo' as a sure miss
#add a ridiculous option flag so don't accidentally trip any real flags
if `echo ${TOOLS[0]} -222222 | grep -q "not found"`; then
echo "${TOOLS[0]} is not installed."
else echo `${TOOLS[0]} --version`
#I am aware that --version is not applicable for all utilities, but this is just
#for sake of example.
My problem is that the if never seems to be accurately picked up. If I tweark ` marks around it either creates false positives or false negatives on the if (e.g.: a program like soodo will be claimed to exist when it doesn't, and something like ssh will be reported as not installed even though it is).
If you guys need any further clarification on what I'm trying to do or the like, please ask. It's the least I can provide back in exchange for some insight by others.

#!/bin/bash
TOOLS=( 'ssh' 'soodo' 'dhclient' 'iperf')
#list of tools is abridged for convenience and added 'soodo' as a sure miss
for TOOL in ${TOOLS[#]}
do
which $TOOL > /dev/null
RESULT=$?
if [ $RESULT -eq 0 ]
then
echo $TOOL is available
else
echo $TOOL is not available
fi
done

For bash, type is the way to determine if a command is a program in your PATH, or a function or an alias.
TOOLS=( 'ssh' 'soodo' 'dhclient' 'iperf')
for tool in "${TOOLS[#]}"; do
if type -p "$tool" > /dev/null; then
echo "$tool is installed"
else
echo "$tool is not installed"
fi
done
The errors in what you're doing:
if `echo ${TOOLS[0]} -222222 | grep -q "not found"`; then
What's happening there:
first, echo ${TOOLS[#]} -222222 prints "ssh -222222" to stdout
that pipes into grep -q "not found" which prints nothing to stdout
the backticks substitute the output from the pipeline (a blank line, which is always the output from grep -q) into the if command, so you get if <a newline> ; then
You'll get the same result as if $(printf "\n"); then echo Y; else echo N; fi which is always true.
To do what you're attempting, you'd have to write:
if "${TOOLS[0]}" -222222 2>&1 | grep -q "not found"; then ...
That will execute the pipeline and then if will consider the exit status. Exit status zero is considered true, any other exit status is considered false.
However, don't do this to find out if a program exists.

Related

Constructing an If statement based on the return of the environment $PATH piped through grep (with bash)

I've answered my own question in writing this, but it might be helpful for others as I couldn't find a straightforward answer anywhere else. Please delete if inappropriate.
I'm trying to construct an if statement depending whether some <STRING> is found inside the environment $PATH.
When I pipe $PATH through grep I get a successful hit:
echo $PATH | grep -i "<STRING>"
But I was really struggling to find the syntax required to construct an if statement around this. It appears that the line below works. I know that the $(...) essentially passes the internal commands to the if statement, but I'm not sure why the [[...]] double brackets are needed:
if [[ $(echo $PATH | grep -i "<STRING>") ]]; then echo "HEY"; fi
Maybe someone could explain that for me to have a better understanding.
Thanks.
You could make better use of shell syntax. Something like this:
$ STRING="bin"
$ grep -i $STRING <<< $PATH && echo "HEY"
That is: first, save the search string in a variable (which I called STRING so it's easy to remember), and use that as the search pattern. Then, use the <<< redirection to input a "here string" - namely, the PATH variable.
Or, if you don't want a variable for the string:
$ grep -i "bin" <<< $PATH && echo "HEY"
Then, the construct && <some other command> means: IF the exit status of grep is 0 (meaning at least one successful match), THEN execute the "other command" (otherwise do nothing - just exit as soon as grep completes). This is the more common, more natural form of an "if... then..." statement, exactly of the kind you were trying to write.
Question for you though. Why the -i flag? That means "case independent matching". But in Unix and Linux file names, command names, etc. are case sensitive. Why do you care if the PATH matches the string BIN? It will, because bin is somewhere on the path, but if you then search for the BIN directory you won't find one. (The more interesting question is - how to match complete words only, so that for example to match bin, a directory name bin should be present; sbin shouldn't match bin. But that's about writing regular expressions - not quite what you were asking about.)
The following version - which doesn't even use grep - is based on the same idea, but it won't do case insensitive matching:
$ [[ $PATH == *$STRING* ]] && echo "HEY"
[[ ... ]] evaluates a Boolean expression (here, an equality using the * wildcard on the right-hand side); if true, && causes the execution of the echo command.
you don't need to use [[ ]], just:
if echo $PATH | grep -qi "<STRING>"; then echo "HEY"; fi

How do you compare the output of a command with a string variable?

[root#ip-10-10-61-178 ~]# cat if.sh
#!/bin/bash
VPCCIDR=`curl -s http://169.254.169.254/latest/meta-data/local-ipv4 | cut -c 1-5`
I am trying to achieve the following with bash but am having trouble:
curl the AWS instance data URL and return the beginning of the local IP. e.g 10.10 for test or 10.20 for prod
If the output of the curl equals to the variable called $TESTENV then print "This is Test" else if it is equal to the variable called $PRODENV then print "This is Prod. If it matches none then echo "no match".
The problem I have is that my script just echoes "This is Test" regardless of what values I set for TESTENV and PRODENV.
#!/bin/bash
#Find out what VPC we are in. e.g 10.10 or 10.50
VPCCIDR=`curl -s http://169.254.169.254/latest/meta-data/local-ipv4 | cut -c 1-5`
TESTENV="10.10"
PRODENV="10.50"
if [ "$VPCCIDDR" != "$TESTENV" ]; then
echo "This is Test"
elif [ "$VPCCIDDR" != "$PRODENV" ]; then
echo "This is Prod"
else
echo "no match"
fi
Try this.
VPCCIDR=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4 | cut -c 1-5)
But as noted in the comments, that won't change your script.
as far as I know in bash command substitution does not technically
differ. Well, the $(...) syntax actually allows for more convenient
nesting and treats slashes differently as well as improving
readability. But no actual functional differencies.
-Zlog (in the ocomments)
A few things to note.
1) I assume that URL returns your IP address. I tested it with this API: $(curl -s https://api.ipify.org | cut -c 1-5) and learned that you're returning the FIRST digits of the IP address. Assuming you have a typical prod/dev set up, I would think that you'd want to check the LAST digits of the ip, not the first, but I may be completely wrong about that.
2) Here's where your problem is...
if [ "$VPCCIDDR" != "$TESTENV" ]; then
echo "This is Test"
Your conditionals are all freaked up. You're saying.. "if my ip does NOT match the test envireonment, tell the user we're in the test environment." I'm sure you meant to do the opposite. You probably want to do == instead of !=.
3) Also, you've got the -s flag which prevents errors from displaying. Test the url in the browser to make sure it's not throwing errors.
4) You're using different variables throughout your script. Make sure you're typing your variables the same way everywhere you use them.

Bash conditional on command exit code

In bash, I want to say "if a file doesn't contain XYZ, then" do a bunch of things. The most natural way to transpose this into code is something like:
if [ ! grep --quiet XYZ "$MyFile" ] ; then
... do things ...
fi
But of course, that's not valid Bash syntax. I could use backticks, but then I'll be testing the output of the file. The two alternatives I can think of are:
grep --quiet XYZ "$MyFile"
if [ $? -ne 0 ]; then
... do things ...
fi
And
grep --quiet XYZ "$MyFile" ||
( ... do things ...
)
I kind of prefer the second one, it's more Lispy and the || for control flow isn't that uncommon in scripting languages. I can see arguments for the first one too, although when the person reads the first line, they don't know why you're executing grep, it looks like you're executing it for it's main effect, rather than just to control a branch in script.
Is there a third, more direct way which uses an if statement and has the grep in the condition?
Yes there is:
if grep --quiet .....
then
# If grep finds something
fi
or if the grep fails
if ! grep --quiet .....
then
# If grep doesn't find something
fi
You don't need the [ ] (test) to check the return value of a command. Just try:
if ! grep --quiet XYZ "$MyFile" ; then
This is a matter of taste since there obviously are multiple working solutions. When I deal with a problem like this, I usually apply wc -l after grep in order to count the lines that match. Then you have a single integer number that you can evaluate within a test condition. If the question only is whether there is a match at all (the number of matching lines does not matter), then applying wc probably is OTT and evaluation of grep's return code seems to be the best solution:
Normally, the exit status is 0 if selected lines are found and 1
otherwise. But the exit status is 2 if an error occurred, unless the
-q or --quiet or --silent option is used and a selected line is found. Note, however, that POSIX only mandates, for programs such as grep,
cmp, and diff, that the exit status in case of error be greater than
1; it is therefore advisable, for the sake of portability, to use
logic that tests for this general condition instead of strict equality
with 2.

Detect whether script is being run by Bash

I have put together the following to detect if a script is being run by Bash or not:
################################################################################
# Checks whether execution is going through Bash, aborting if it isn't. TinoSino
current_shell="$(
ps `# Report a snapshot of the current processes` \
-p $$ `# select by PID` \
-o comm `# output column: Executable namename` \
|\
paste `# Merge lines of files ` \
-s `# paste one file at a time instead of in parallel` \
- `# into standard output` \
|\
awk `# Pick from list of tokens` \
'{ print $NF }' `# print only last field of the command output`
)"
current_shell="${current_shell#-}" # Remove starting '-' character if present
if [ ! "${current_shell}" = 'bash' ]; then
echo "This script is meant to be executed by the Bash shell but it isn't."
echo 'Continuing from another shell may lead to unpredictable results.'
echo 'Execution will be aborted... now.'
return 0
fi
unset current_shell
################################################################################
I am not asking you specifically to code review it because you would be sending me to CodeReview; my question is:
how would you go about testing whether this "execution guard" put at the top of my script does indeed do its job reliably?
I am thinking about installing Virtual Machines and on each machine to install things like zsh, csh, etc. But it looks way too time-consuming to me. Better ways to do this?
Should you spot an immediate mistake do point it out to me though please. Just glaring bugs waving their legs waiting to be squashed should be squashed, I think.
This is better written as
if [ -z "$BASH_VERSION" ]
then
echo "Please run me in bash"
exit 1
fi
As for testing, get a list of non-bash shells from /etc/shells, and just run the script with each of them verifying that you get your error message.
getshver (recommended)
whatshell
which_interpreter (silly)
I would only recommend rolling your own if it isn't critical to "guarantee" correct results. I don't think such a guarantee is even possible, but most of the time you're at most targeting a few shells and only care about modern versions. Very few people should even care about version detection. Writing portable code while going outside of POSIX requires knowing what you're doing.
Don't bother detecting the shell just to abort. If people want to shoot themselves in the foot by ignoring the shebang that's their problem.

How to prevent code/option injection in a bash script

I have written a small bash script called "isinFile.sh" for checking if the first term given to the script can be found in the file "file.txt":
#!/bin/bash
FILE="file.txt"
if [ `grep -w "$1" $FILE` ]; then
echo "true"
else
echo "false"
fi
However, running the script like
> ./isinFile.sh -x
breaks the script, since -x is interpreted by grep as an option.
So I improved my script
#!/bin/bash
FILE="file.txt"
if [ `grep -w -- "$1" $FILE` ]; then
echo "true"
else
echo "false"
fi
using -- as an argument to grep. Now running
> ./isinFile.sh -x
false
works. But is using -- the correct and only way to prevent code/option injection in bash scripts? I have not seen it in the wild, only found it mentioned in ABASH: Finding Bugs in Bash Scripts.
grep -w -- ...
prevents that interpretation in what follows --
EDIT
(I did not read the last part sorry). Yes, it is the only way. The other way is to avoid it as first part of the search; e.g. ".{0}-x" works too but it is odd., so e.g.
grep -w ".{0}$1" ...
should work too.
There's actually another code injection (or whatever you want to call it) bug in this script: it simply hands the output of grep to the [ (aka test) command, and assumes that'll return true if it's not empty. But if the output is more than one "word" long, [ will treat it as an expression and try to evaluate it. For example, suppose the file contains the line 0 -eq 2 and you search for "0" -- [ will decide that 0 is not equal to 2, and the script will print false despite the fact that it found a match.
The best way to fix this is to use Ignacio Vazquez-Abrams' suggestion (as clarified by Dennis Williamson) -- this completely avoids the parsing problem, and is also faster (since -q makes grep stop searching at the first match). If that option weren't available, another method would be to protect the output with double-quotes: if [ "$(grep -w -- "$1" "$FILE")" ]; then (note that I also used $() instead of backquotes 'cause I find them much easier to read, and quotes around $FILE just in case it contains anything funny, like whitespace).
Though not applicable in this particular case, another technique can be used to prevent filenames that start with hyphens from being interpreted as options:
rm ./-x
or
rm /path/to/-x

Resources