I'm writing a script where I need to use the output of a file test in several places, including inside a shell function. I would like to assign the file existence to a shell variable, like this: file_exists=[ -f $myfile ].
Just to make sure that I've got my bases covered, I start by touching a file, and testing its existance:
file='a'
touch $file
if [ -f $file ]
then
echo "1 -- '$file' exists"
fi
Output:
1 -- 'a' exists
The file was created successfully -- no surprises, but at least I know that I'm not dealing with any permissions issues or anything.
Next I test to make sure that I can store a boolean expression in a variable:
mytest=/bin/true
if $mytest
then
echo "2 -- \$mytest is true"
fi
Output:
2 -- $mytest is true
So I've got the basics covered -- conditional expressions should emit the same output as /bin/true or /bin/false... but that's not what I'm seeing:
mytest=[ -f $file ]
if $mytest
then
echo "3 -- \$mytest is true [expect true]"
else
echo "3 -- \$mytest is false [expect true]"
fi
This fails with the following error:
-f: command not found
I get the same error message if i use test -f $file rather than [ -f $file ].
If I put a space in front of the [, the error goes away...
mytest= [ -f $file ]
if $mytest
then
echo "4 -- \$mytest is true [expect true]"
else
echo "4 -- \$mytest is false [expect true]"
fi
The output appears to be correct:
4 -- $mytest is true [expect true]
... but if I remove the file, I should get the opposite result:
rm $file
mytest= [ -f $file ]
if $mytest
then
echo "5 -- \$mytest is true [expect false]"
else
echo "5 -- \$mytest is false [expect false]"
fi
... and I don't:
5 -- $mytest is true [expect false]
To be fair, I expected the space to mess with the truth value:
mytest= /bin/false
if $mytest
then
echo "6 -- \$mytest is true [expect false]"
else
echo "6 -- \$mytest is false [expect false]"
fi
Outputs:
6 -- $mytest is true [expect false]
So, how do I store the output from the test builtin in a shell variable?
As others have documented here, using the string "true" is a red herring; this is not an appropriate way to store boolean values in shell scripts, as evaluating it means dynamically invoking a command rather than simply inspecting the stored value using shell builtins hardcoded in your script.
Instead, if you really must store an exit status, do so as a numeric value:
[ -f "$file" ] # run the test
result=$? # store the result
if (( result == 0 )); then # 0 is success
echo "success"
else # nonzero is failure
echo "failure"
fi
If compatibility with set -e is desired, replace the first two lines of the above with:
result=0
[ -f "$file" ] || result=$?
...as putting the test on the left-hand side of || marks it as "checked", suppressing errexit behavior. (That said, see BashFAQ #105 describing the extent to which set -e harms predictable, portable behavior; I strongly advise against its use).
You need to quote whitespace:
mytest='[ -f $file ]'
if $mytest; then echo yes; fi
However, this is extremely brittle and potentially insecure. See http://mywiki.wooledge.org/BashFAQ/050 for a detailed discussion and some better ways to accomplish something similar.
If you want to encapsulate a complex piece of code, a function is usually the way to go:
mytest () { [ -f "$file" ]; }
if mytest; then echo yes; fi
If you want to run the code once and store its result so you can examine it later, I would rephrase it like this:
if [ -f "$file" ]; then
mytest=true
else
mytest=false
fi
if $mytest; then echo yes; fi
A old one but left this here for reference for people that might need it. Not the most beautiful solution but it works in bash:
mytest=$( [ -f $file ] ; echo $? )
More portable, using the test command, and the backticks:
set mytest=`test -f $file ; echo $?`
In a subprocess (<!> system load), the condition is evaluated, and then the result echoed to the output that is captured by the variable $mytest.
mytest=/bin/true is storing the string /bin/true in the $mytest variable.
mytest=[ -f $file ] is setting the $mytest variable to the value [ for the duration of the command -f $file ] (which as your output indicates fails as there is no -f command available).
mytest= [ -f $file ] (like the above) sets the value of the $mytest variable to blank for the duration of the [ -f $file ] command (and returns whatever [ returns).
mytest= /bin/false this is the same as the above case only the command being run is /bin/false.
If you want to store the return code from a command in a variable you can do
/bin/true
ret=$?
if you want to store the output from a command in a variable you can do
out=$(/bin/true)
(though with /bin/true that variable will be empty as it outputs no text.
For your case you want the former $? model.
Also, using set -x (and/or set -v) in your scripts might have helped you diagnose this.
Different version with test command.
fileExists=$(test -f /path/to/file && echo true || echo false)
if [[ ${fileExists} == "true" ]]; then
# Your code here
fi
Or even more simplier version.
fileExists=$(test -f /path/to/file && echo 1)
if [[ -z ${fileExists} ]]; then
# Your code here
fi
Related
I am trying to check for duplicate records in my database using shell scripting.
For this, I have created a function named "check()" which echo's True or False and is stored in variable "result". But while evaluating using if statement it is always returning "True".
#redundancy check function
check() {
temp=$(grep -w -c "$1" database.dat)
echo $temp
if [ "$temp" != 0 ]
then
echo True
else
echo False
fi
}
insert() {
option="y"
while [ "$option" == "y" ]
do
echo "Rollno: \c"
read roll
result="$(check $roll)"
echo $result
if [ "$result" == "False" ]
then
echo Do something
else
echo "ERROR: Duplicate record found...\nEXITING...\n"
option="n"
fi
done
}
If you're using a shell that doesn't support the == extension to test, then your tests will always, unconditionally fail simply on account of invalid syntax. Use = for string comparisons to be portable to all POSIX-compliant implementations.
Moreover, there's no point to storing and then comparing the output from grep at all: Use the exit status of grep -q when your only goal is to check whether the number of matches is zero or more-than-zero; this allows grep to exit immediately when a match is seen, rather than needing to read the rest of the file.
# with -q, this emits no stdout, but exits w/ status 0 (matches exist) or 1 (otherwise)
check() { grep -q -w -e "$1" database.dat; }
insert() {
option=y
while [ "$option" = y ]; do
printf '%b\n' "Rollno: \c"
read -r roll
if check "$roll"; then
printf "ERROR: Duplicate record found...\nEXITING...\n"
option=n
else
echo "Check failed; do something"
fi
done
}
So I'm currently just trying to traverse through my current directory where I'm calling the following bash script that prints 'We found a .c file' every time one is found. I have an if statement to check for args because I will be extending the script where if no args are found it will run anyway, and one arg will tell the script the directory to look in.
The issue is, this code does not work:
if [ -z "$#" ]
then
for i in *.c; do
echo "We found a .c file"
done
fi
But then if I add the echo "Test" in, it works?
if [ -z "$#" ]
echo "Test"
then
for i in *.c; do
echo "We found a .c file"
done
fi
I'm new to bash and no clue why this is happening. Can anyone help me out?
$#, which reports the count of arguments, is NEVER an empty string - if you don't specify arguments, $# evaluates to 0, which is still a nonempty string (-z tests for empty strings).
Therefore, [ -z "$#" ] is always (logically) false.
What you're looking for - using idiomatic Bash - is:
if [[ $# -eq 0 ]]; then ... # -eq compares *numerically*
As anishsane points out in a comment, the POSIX-compliant [ $# -eq 0 ] would work here as well; generally, though - unless your express intent is to write POSIX-compliant shell code - you're better off sticking with the more predictable, more feature-rich (and marginally faster) Bash-specific constructs.
or, using arithmetic evaluation:
if (( $# == 0 )); then ...
As for why your 2nd snippet caused the if branch to be entered:
Your misplaced echo "Test" - due to being placed before the then keyword, caused the echo command to be interpreted as part of the conditional.
In other words: the conditional that was evaluated was effectively
[ -z "$#" ]; echo "Test", a list of (two) commands only whose last command's exit code determined the outcome of the conditional.
Since echo always succeeds (exit code 0)[1]
, the conditional as a whole evaluated to (logical) true, and the if branch was entered.
[1] gniourf_gniourf points out in a comment that you can make a simple echo command fail (with exit code 1), if you use input/output redirection with an invalid source/target; e.g., echo 'fail' > /dev/full.
(Note that if the redirection source/target is fundamentally invalid - an nonexistent input file or an output file that can't be created / opened (as opposed to, say, an output target that can be opened with write permission but ultimately can't be written to, such as /dev/full on Linux) - Bash never even invokes the command at hand, as it "gives up" when it encounters the invalid redirection:
{ echo here >&2; echo hi; } >/dev/full # Linux: 'here' still prints (to stderr)
{ echo here >&2; echo hi; } >'' # invalid target: commands are never invoked)
Problem
The following loop will never run:
if [ -z "$#" ]
then
for i in *.c; do
echo "We found a .c file"
done
fi
The reason is that $# is a number, 0, 1 or more. It will never be an empty string. Thus [ -z "$#" ] will always fail
This loop will always run:
if [ -z "$#" ]
echo "Test"
then
for i in *.c; do
echo "We found a .c file"
done
fi
While [ -z "$#" ] always fails the second statement echo "Test" normally returns a success exit code.
Solution
If no arguments were specified on the command line, this sets the arguments to all .c files in the current directory:
[ "$1" ] || set -- *.c
for i in "$#"; do
echo "We found a .c file: $i"
done
Thus, this allows you to specify the file names on the command line and the script runs on those. If you don't specify any, it runs on all the .c files.
I have written a script that backs up and restores files. I have a problem in that when the user enters '2' for a restore the program says that this is an invalid input, all other options work fine. I feel it is something small that I have missed but I cant fix it
Update and Restore Script
#!/bin/bash
ROOT="/Users/Rory/Documents"
ROOT_EXCLUDE="--exclude=/dev --exclude=/proc --exclude=/sys --exclude=/temp --exclude=/run --exlucde=/mnt --exlcude=/media --exlude=/backup2.tgz"
DESTIN="/Users/Rory/test/"
BACKUP="backup2.tgz"
CREATE="/dev /proc /sys /temp /run /mnt /media "
if [ "$USER" != "root" ]; then
echo "You are not the root user"
echo "To use backup please use: sudo backup"
exit
fi
clear
echo "************************************************"
echo "********* Backup Menu **************************"
echo "************************************************"
OPTIONS="BACKUP RESTORE DESTINATION EXIT"
LIST="1)BACKUP 2)RESTORE 3)DESTINATION 4)EXIT"
select opt in $OPTIONS; do
if [ "$opt" = "EXIT" ]; then
echo "GOODBYE!"
sleep 3
clear
exit
elif [ "$opt" = "BACKUP" ]; then
echo "BACKING UP FILES..."
sleep 2
tar cvpfz $DESTIN/backup.`date +%d%m%y_%k:%M`.tgz $ROOT $ROOT_EXCLUDE_DIRS
echo "BACKUP COMPLETE"
sleep 2
exit
elif [ "$opt" = "RESTORE" ]; then
echo "RESTOTING FILES..."
sleep 2
tar xvpfz $BACKUP_FILE -C /
sleep2
echo "RESTORE COMPLETE..."
if [[ -e "/proc" ]]; then
echo "$CREATE_DIRS allready exists! "
else
mkdir $CREATE_DIRS
echo "$CREATE_DIRS are created! "
fi
exit
elif [ "$opt" = "DESTINATION" ]; then
echo "CURRENT DESTINATION: $DEST_DIR/backup.`date +%d/%m/%y_%k:%M`.tgz "
echo "TO CHANGE ENTER THE NEW DESTINATION..."
echo "TO LEAVE IT AS IS JUST PRESS ENTER..."
read NEW_DESTIN
#IF GREATER THEN 0 ASSIGN NEW DESTINATION
if [ ${#NEW_DESTIN} -gt 0 ]; then
DESTIN = "$NEW_DESTIN"
fi
clear
echo $BANNER1
echo $BANNER2
echo $BANNER3
echo $LIST
else
clear
echo "BAD INPUT!"
echo "ENTER 1 , 2, 3 or 4.."
echo $LIST
fi
done
Except where you missed the ending quote where you set ROOT_EXCLUDE (line #4), it looks okay to me. I take it the missing quote is a transcription error or your program wouldn't really work at all.
I've tried out the program and it seems to work.
A debugging trick is to put set -xv to turn on debugging in your script and set +xv to turn it off. The -x means to print out the line before executing, and the -v means to print out the line once the shell interpolates the line.
I'm sure that you'll immediately see the issue once you have set -xv in your program.
As part of this, you can set PS4 to the line prompt to print when the debugging information is printed. I like setting PS4 like this:
export PS4="[\$LINENO]> "
This way, the line prompt prints out the line it's executing which is nice.
In your case, I would put set -xv right before you set OPTIONS and then at the very end of the program. This way, you can see the if comparisons and maybe spot your issue.
export PS4="[\$LINENO]> "
set -xv
OPTIONS="BACKUP RESTORE DESTINATION EXIT"
LIST="1)BACKUP 2)RESTORE 3)DESTINATION 4)EXIT"
select opt in $OPTIONS; do
if [ "$opt" = "EXIT" ]; then
echo "GOODBYE!"
set +xv
By the way, it's better to use double square brackets like [[ ... ]] for testing rather than the single square brackets like [ ... ]. This has to do with the way the shell interpolates the values in the test.
The [ ... ] is an alias to the built in test command. The shell interpolates the line as is and the entire line is executed.
The [[ ... ]] are a compound statement where the shell will interpolate variables, but not the entire line. The line is kept as whole:
foo="one thing"
bar="another thing"
This will work:
if [ "$foo" = "$bar" ]
then
echo "Foo and bar are the same"
fi
This won't:
if [ $foo = $bar ]
then
echo "Foo and bar are the same"
fi
The shell interpolates the line as is:
if [ one thing = another thing ]
And this is the same as:
if test one thing = another thing
The test command looks at the first item to see if it's a standard test, or assumes three items and the second item is a comparison. In this case, neither is true.
However, this will work:
if [[ $foo = $bar ]] # Quotes aren't needed
then
echo "Foo and bar are the same"
fi
With the [[ ... ]] being a compound command, the $foo and $bar are replaced with their values, but their positions are kept. Thus, the = is recognized as a comparison operator.
Using [[ ... ]] instead of [ ... ] has solved a lot of hard to find shell scripting bugs I have.
I have a shell script where I pass (2) parameters, one to pass a dbname, the other to call one of (2) filenames. I want to check if either filename exists, then proceed with calling that script, else exit because the user can enter the wrong string and construct my_foo.sql which I don't want. I don't think I have the condition for setting "or" correctly since putting the correct param still produces error. Is there a better way to write this?
Here is what I have so far.
#/usr/bin/ksh
if [ $# != 2 ]; then
echo "Usage: test.sh <dbname> <test|live>" 2>&1
exit 1
fi
# Check actual file name
CHKSCRIPT1=/tmp/my_test.sql;
CHKSCRIPT2=/tmp/my_live.sql;
if [ -f "CHKSCRIPT1" ] || [ -f "CHKSCRIPT2" ]
then
/bin/sqlplus -s foo/bar #/my_$2.sql
else
echo "Correct sql script does not exist. Enter test or live"
exit 1
fi
Your issue is that you're not referencing your variables correctly:
if [ -f "$CHKSCRIPT1" ] || [ -f "$CHKSCRIPT2" ]
...
fi
edit: Per #chepner, you shouldn't use -o
In addition to the problem you had with expanding the parameters, you should separate what the user types from what files need to exist. If the user enters "live", the only thing that matters is whether or not /tmp/my_live.sql exists. If the user enters "injection_attack", your script should not execute /tmp/my_injection_attack.sql (which presumably was created without your knowledge). The right thing to do is to first verify that a valid command was entered, then check if the appropriate file exists.
if [ $# != 2 ]; then
echo "Usage: test.sh <dbname> <test|live>" 2>&1
exit 1
fi
case $2 in
test|live)
filename="/tmp/my_{$2}.sql"
;;
*) echo "Must enter test or live"
exit 1
;;
esac
if [ -f "$filename" ]; then
/bin/sqlplus -s foo/bar #/my_$2.sql
else
echo "SQL script $filename does not exist."
exit 1
fi
I've gone around and around on the quoting stuff on http://tldp.org for bash and googled until I am blue in the face. I've also tried every obvious quoting scheme for this issue, and yet nothing works.
The problem seems to be that a space inside of a quoted argument in the command run at the end of the script is being interpreted as a separator instead of as a quoted space.
Behold, here's my script (I know full well I'm a noob so comments on my style and/or uneccessary syntax is cool with me, I'll learn):
#!/bin/bash
date=`date`
args="$#"
MSEND_HOME=/home/patrol/Impact #Path to the Impact Directory
integrationName=Introscope #Name of the integration
persistEnabled=1 #1 for Yes, 0 for No
persist=""
bufDir=$MSEND_HOME/tmp/$integrationName #DO NOT CHANGE
cellName=linuxtest #Cell name to forward events to
loggingEnabled=1 #1 for Yes, 0 for No
logFile=$MSEND_HOME/log/$integrationName.$cellName.log
die () {
if [ $loggingEnabled -eq 1 ]
then
echo >>$logFile "$#"
fi
exit 1
}
[ "$#" -ge 1 ] || die "$date - At least 1 argument required, $# provided" "$#"
# This is where you would parse out your arguments and form the following
# slots as a minimum for sending an event.
class=$2
msg=\"$3\"
# Parse the first argument and assign the correct syntax
if [[ $1 == "INFORMATIONAL" ]]
then
severity=INFO
elif [[ $1 == "WARN" ]]
then
severity=WARNING
elif [[ $1 == "CRIT" ]]
then
severity=CRITICAL
else
severity=INFO
fi
#Additional slots can be set, parse them all in this variable;
#e.g., additionalSlots="slot1=value1;slot2=value2;slot3=\"value 3\""
additionalSlots=""
cmd="$MSEND_HOME/bin/msend"
cmd="$cmd -q"
cmd="$cmd -l $MSEND_HOME"
if [ $persistEnabled -eq 1 ]
then
cmd="$cmd -j $bufDir"
fi
cmd="$cmd -n $cellName"
cmd="$cmd -a $class"
cmd="$cmd -m $msg"
cmd="$cmd -r $severity"
if [ $additionalSlots ]
then
cmd="$cmd -b $additionalSlots"
fi
$cmd || die "$date - msend exited with error $? | Original arguments: $args | Command: $cmd"
#echo "msend exited with error $? | Original arguments: $args | Command: $cmd"
The script is executed like this:
./sendEvent.sh "CRIT" "EVENT" "Test Event"
The error I get from the msend executable is that the arguments are wrong, but I'm logging the command line in it's entirety to a file and when I run that logged command in the shell interactively, it works.
Here's the log output:
Tue Oct 4 20:31:29 CDT 2011 - msend exited with error 27 | Original arguments: CRIT EVENT Test Event | Command: /home/patrol/Impact/bin/msend -q -l /home/patrol/Impact -j /home/patrol/Impact/tmp/Introscope -n linuxtest -a EVENT -m "Test Event" -r CRITICAL
So if I paste /home/patrol/Impact/bin/msend -q -l /home/patrol/Impact -j /home/patrol/Impact/tmp/Introscope -n linuxtest -a EVENT -m "Test Event" -r CRITICAL and run it, it works.
If I run the script like ./sendEvent.sh "CRIT" "EVENT" "TestEvent" it works. But I need that argument to allow spaces.
I'm on the track that it's an $IFS issue or something... maybe a difference between the interactive shell and the script environment.
I'd appreciate any insight from smarter people than me!
tl;dr - My command doesn't work when run from within a script, but does when the logged command syntax is used in an interactive shell.
Short answer: see BashFAQ #50.
Long answer: When bash parses a line, it parses quote marks before doing variable substitution; as a result, when you put quotes inside a variable, they don't do what you'd expect. You're actually passing an argument list including '-m' '"Test' 'Event"' '-r' -- those double-quotes aren't around the arguments, they're in the arguments.
In this case, the best solution is to build the command in an array rather than a string. Also, get in the habbit of putting double-quotes around variables (e.g. filenames) when you use them, to prevent confusion if they contain spaces. With those changes (and a few other tweaks), here's my version of your script:
#!/bin/bash
date="$(date)" # Backquotes are confusing, use $() instead
args=("$#") # Save the args in an array rather than mushing them together in a string
MSEND_HOME=/home/patrol/Impact #Path to the Impact Directory
MSEND_HOME="$HOME/tmp" #Path to the Impact Directory
integrationName=Introscope #Name of the integration
persistEnabled=1 #1 for Yes, 0 for No
persist=""
bufDir="$MSEND_HOME/tmp/$integrationName" #DO NOT CHANGE
cellName=linuxtest #Cell name to forward events to
loggingEnabled=1 #1 for Yes, 0 for No
logFile="$MSEND_HOME/log/$integrationName.$cellName.log"
die () {
if [ $loggingEnabled -eq 1 ]
then
echo >>"$logFile" "$#"
fi
exit 1
}
[ "$#" -ge 1 ] || die "$date - At least 1 argument required, $# provided" "$#"
# This is where you would parse out your arguments and form the following
# slots as a minimum for sending an event.
class="$2" # Quotes not strictly needed here, but a good habbit
msg="$3"
# Parse the first argument and assign the correct syntax
if [[ "$1" == "INFORMATIONAL" ]]
then
severity=INFO
elif [[ "$1" == "WARN" ]]
then
severity=WARNING
elif [[ "$1" == "CRIT" ]]
then
severity=CRITICAL
else
severity=INFO
fi
#Additional slots can be set, parse them all in this array;
#e.g., additionalSlots="slot1=value1;slot2=value2;slot3=value 3" # Don't embed quotes
additionalSlots=""
cmd=("$MSEND_HOME/bin/msend") # Build the command as an array, not a string
cmd+=(-q) # Could equivalently use cmd=("${cmd[#]}" -q), but this is simpler
cmd+=(-l "$MSEND_HOME")
if [ $persistEnabled -eq 1 ]
then
cmd+=(-j "$bufDir")
fi
cmd+=(-n "$cellName")
cmd+=(-a "$class") # Possible bug: $2 and #3 aren't required, but they're getting added unconditionally
cmd+=(-m "$msg") # These should probably be conditional, like additionalSlots
cmd+=(-r "$severity")
if [ -n "$additionalSlots" ]
then
cmd+=(-b "$additionalSlots")
fi
"${cmd[#]}" || die "$date - msend exited with error $? | Original arguments:$(printf " %q" "${args[#]}") | Command:$(printf " %q" "${cmd[#]}")"
#echo "msend exited with error $? | Original arguments:$(printf " %q" "${args[#]}") | Command:$(printf " %q" "${cmd[#]}")"
I think the arg goes wrong with this assignment: cmd="$cmd -m $msg".
Change it to cmd="$cmd -m \"$msg\"".
Okay, I don't see the exact problem immediately, but I can tell you what it is; this hint should help.
Remember that the shell quoting mechanism only interprets a string once. As a result, if you're not careful, what you thought was "foo" "a" "b" is in fact "foo a b" -- that is, all one token, not three.
Run the script with bash -x which will show you at each step what the shell is actually seeing.