I'm creating a bash script and somewhere inside I have this code:
if [ $# -eq 2 -a (! -r "$2" -o ! -f "$2") ]; then
echo "rvf: bestand \""$2"\" bestaat niet of is onleesbaar" 1>&2
exit 2
fi
When i try to run this inside the script I get this error:
Syntax Error (bash -n):
rvf: line 14: syntax error in conditional expression
rvf: line 14: syntax error near `-a'
rvf: line 14: `if [[ $# -eq 2 -a (! -r "$2" -o ! -f "$2") ]]; then'
How does '()' work inside Bash scripts?
[[ doens't support -a, and it is considered obsolete and non portable for [. The correct solution using [ would be
if [ "$#" -eq 2 ] && { [ ! -r "$2" ] || [ ! -f "$2" ]; }; then
Grouping is done with { ... } rather than ( ... ) to avoid creating an unnecessary subshell.
Using [[ is simplifies to
if [[ "$#" -eq 2 && ( ! -r "$2" || ! -f "$2" ) ]]; then
Parentheses can be used for grouping inside [[; as a compound command, it uses separate parsing and evaluation rules, compared to an ordinary command like [ (which is just an alias for test, not syntax of any kind).
In either case, De Morgan's laws lets you refactor this to something a little simpler:
if [ "$#" -eq 2 ] && ! { [ -r "$2" ] && [ -f "$2" ] }; then
if [[ "$#" -eq 2 && ! ( -r "$2" && -f "$2" ) ]]; then
There are multiple points of confusion here.
[ can (as an optional XSI extension to the standard) support ( as a separate word (meaning there needs to be spaces around it), but the POSIX sh specification marks it (like -a and -o) as "obsolescent" and advises against its use.
[[ does support (, but again, it needs to always be a separate word.
Don't do that at all, though. You're using only well-supported and portable functionality if you keep each test its own simple command and combine them only with the shell's boolean logic support.
That is:
if [ "$#" -eq 2 ] && { [ ! -r "$2" ] || [ ! -f "$2" ]; }; then
echo "rvf: bestand \"$2\" bestaat niet of is onleesbaar" >&2
exit 2
fi
Restructure your logic.
"Not A or Not B" is just a more complicated way to say "not (A and B)".
In bash, try
if [[ "$#" == 2 ]] && ! [[ -r "$2" && -f "$2" ]]; then
Better,
if [[ "$#" == 2 && -r "$2" && -f "$2" ]]
then : all good code
else : nope code
fi
Even better,
if [[ "$#" == 2 ]] # correct args
then if [[ -r "$2" ]] # is readable
then if [[ if -f "$2" ]] # is a file
then echo "all good"
: do all good stuff
else echo "'$2' not a file"
: do not a file stuff
fi
else echo "'$2' not readable"
: do not readable stuff
fi
else echo "Invalid number of args"
: do wrong args stuff
fi
Clear error logging is worth breaking the pieces out.
Even better, imho -
if [[ "$#" != 2 ]]
then : wrong args stuff
fi
if [[ ! -r "$2" ]]
then : unreadable stuff
fi
if [[ ! -f "$2" ]]
then : do not a file stuff
fi
: do all good stuff
Related
Bash version 4.4.20
Ubuntu 16.04
I need to compare time and extensions for a particular project I have. Below is something similar to what I am trying to do but the error is the same. I am not sure where exactly the error is as shellcheck is not producing one.
#!/bin/bash
#
while read -r filename; do
extension="${filename##*.}"
if [ "$extension" == "zip" ] && [ "$filename" == "one.zip" ]; then
echo "Filename is $filename"
elif [ "$extension" == "zip" ] && [ "$filename" == "file_1.zip" ] -o [ "$filename" == "file_2.zip" ] -o [ "$filename" == "file_3.zip" ]; then
echo "Filename is $filename"
elif [ "$extension" == "csv" ] && [ "$filename" == "two.csv" ]; then
echo "Filename is $filename"
else
echo "Filename is $filename"
fi
done<fileList.txt
Error:
Filename is one.zip
check.sh: line 8: [: too many arguments
Filename is file_1.zip
check.sh: line 8: [: too many arguments
Filename is file_2.zip
check.sh: line 8: [: too many arguments
Filename is file_3.zip
Filename is two.csv
Filename is three.sql
Use pattern matching to your advantage:
while IFS= read -r filename; do
if [[ "$filename" = one.zip ]; then
echo "Filename is $filename"
elif [[ "$filename" = file_[123].zip ]; then
echo "Filename is $filename"
elif [[ "$filename" = two.csv ]; then
echo "Filename is $filename"
else
echo "Filename is $filename"
fi
done < fileList.txt
A case statement will work in any POSIX shell, not just a shell that supports a bash-like [[ ... ]] command.
while IFS= read -r filename; do
case $filename in
one.zip) echo "Filename is $filename" ;;
file_[123].zip) echo "..." ;;
two.csv) echo "..." ;;
*) echo "..." ;;
esac
done
To match a range of years(?), you can use
case $value in
200[0-9]|201[0-9]|202[0-1]) echo "Year between 2000 and 2021" ;;
esac
You can't do that as simply with [[ value = ... ]], since the | is part of the case statement's syntax, not an alternation operator in the pattern. Instead, you would need multiple match operators:
if [[ $value = 200[0-9] || $value = 201[0-19] || $value = 202[0-1] ]]; then
Here an explanation for why you get this error, as you get already good answers on how to do it correctly:
Note that the command [ is equivalent to test, hence your line 8 in effect (also removing unnecessary quotes) contains:
elif test "$extension" == zip ] && test "$filename" == file_1.zip ] -o [ "$filename" == file_2.zip ] -o [ "$filename" == file_3.zip ]
From the test man-page, we can see that the closing ] is optional, but if it is present, it terminates the expression to be tested.
The first test, which is just
test "$extension" == "zip" ]
is fine in this respect, but the second one starts with
test "$filename" == file_1.zip ] -o [ ...
Hence, when parsing the arguments, test encounters a ] and knows that this is the terminator for the arguments, but then finds another argument (-o), and doesn't know what to do with it. Hence it complains that it has got too many arguments
I have updated the syntax to what I think is the modern bash idioms. The script now works. Please correct me if I missed something else :
#!/bin/bash
#
while read -r filename
do
extension="${filename##*.}"
if [[ "$extension" = "zip" && "$filename" = "one.zip" ]]
then
echo "Filename is $filename"
elif [[ "$extension" = "zip" && "$filename" = "file_1.zip" ]] \
|| [[ "$filename" = "file_2.zip" ]] \
|| [[ "$filename" = "file_3.zip" ]]
then
echo "Filename is $filename"
elif [[ "$extension" = "csv" && "$filename" = "two.csv" ]]
then
echo "Filename is $filename"
else
echo "Filename is $filename"
fi
done < fileList.txt
Regards!
I've the below code:
#!/bin/bash
. ~/.bash_profile
dt=$(date +"%Y.%m.%d")
if [[ "$#" -eq '0' ]] || [[ "$#" -eq '1' ]]
then
echo -e "Not correct usage"
exit 1
fi
tar zvxf $SHARE/*.gz -C $SHARE/landing/
sleep 3
ops () {
//
some sets of command
//
}
while read -r line
do
if [[ "$1" == "A" ]] || [[ "$1" == "B" ]] || [[ "$1" == "C" ]] && [[ "$2" =~ "$line" ]] || [[ "$2" == "ALL" ]]
then
ops
fi
done < $SHARE/landing/dir/ApprovedList."$1"
When i run the script, i can see that it's able to untar the .gz file and place it into the folder :$SHARE/landing/
But i guess it's not able to read the file and perform the operations inside the ops function, below is the last line of output i'm getting after running the script in interactive mode:
+ read -r line
Any help is most welcome!!!
Please show the input format, the argument format, and some explanation of those. In the meantime, tweaked for readability and error handling - edit to taste. YMMV.
#!/bin/bash
. ~/.bash_profile
ops () { # do some stuff
: some sets of commands ...
}
die() {
printf "%s\n\n Use: $0 x y z ...\n\n" "$1" >&2
kill -term $$ # exit in function behaves like return
}
# dt=$(date +"%Y.%m.%d") # unused
case "$#" in
0|1) die "Insufficient arguments" ;;
esac
tar zvxf "$SHARE"/*.gz -C "$SHARE/landing/" || die "tar failed"
sleep 3
infile="$SHARE/landing/dir/ApprovedList.$1"
[[ -e "$infile" ]] || die "$infile does not exist"
[[ -r "$infile" ]] || die "$infile is nor readable by $0 as $LOGNAME"
while read -r line
do case "$1" in
A|B|C) case "$2" in
*"$line"*|ALL) ops ;;
*) : data does not match, skipping ;;
esac ;;
*) die "Invalid arg1 '$1'";;
esac
done < "$infile"
Notes:
If you require exactly 2 arguments, then let's check exactly that:
die() {
printf "%s\n\n Use: $0 x y\n\n" "$1" >&2
kill -term $$ # exit in function behaves like return
}
and
case "$#" in
2) ;;
*) die "Incorrect # of arguments" ;;
esac
Also, better than the kill - add a trap at the top:
trap 'echo >&2 "ERROR in $0($BASH_SOURCE) at line $LINENO: [$BASH_COMMAND]"; exit $LINENO;' ERR
and use a literal error return in the function.
die() {
printf "%s\n\n Use: $0 x y\n\n" "$1" >&2
return 1
}
The check (except for [[ "$2" =~ "$line" ]])
if [[ "$1" == "A" ]] || [[ "$1" == "B" ]] || [[ "$1" == "C" ]] && [[ "$2" =~ "$line" ]] || [[ "$2" == "ALL" ]]
doesn't change inside the while-loop. So better check once before entering the while-loop.
So now you want to perform ops for each line of $SHARE/landing/dir/ApprovedList."$1".
You can use
xargs -a "$SHARE/landing/dir/ApprovedList.$1" -L 1 ops
EDIT:
When you have a simple check to perform for each line, you can move this check into the ops function.
First save $2 before entering the function:
to_be_checked="$2"
...
ops() {
[[ "$0" =~ "${to_be_checked}" ]] && return
I am writing a bash script that sometimes will use environment variables GIT_DIR and GIT_WORK_TREE. The bash script can only operate correctly if either both variables exist or neither exist. In case there's a technical difference, it makes no difference
This works, but there has to be a better way:
if [[ -z "${GIT_DIR}" ]]; then
_GIT_DIR_EXISTS=0
else
_GIT_DIR_EXISTS=1
fi
if [[ -z "${GIT_WORK_TREE}" ]]; then
_GIT_WORK_TREE_EXISTS=0
else
_GIT_WORK_TREE_EXISTS=1
fi
if [[ "${_GIT_DIR_EXISTS}" -ne "${_GIT_WORK_TREE_EXISTS}" ]]; then
echo "GIT_DIR is ${GIT_DIR}"
echo "GIT_WORK_TREE is ${GIT_WORK_TREE}"
echo "Both or none must exist"
exit 1
fi
I tried:
if [[ (-z "${GIT_DIR}") -ne (-z "${GIT_WORK_TREE}") ]]; then
But that gives this error:
bash: syntax error in conditional expression
bash: syntax error near '-ne'
I then resorted to trying semi-random things, with varying errors:
if [[ -z "${GIT_DIR}" -ne -z "${GIT_WORK_TREE}" ]]; then
if [[ [-z "${GIT_DIR}"] -ne [-z "${GIT_WORK_TREE}"] ]]; then
if [[ [[-z "${GIT_DIR}"]] -ne [[-z "${GIT_WORK_TREE}"]] ]]; then
if [[ -z "${GIT_DIR}" ]] ^ [[ -z "${GIT_WORK_TREE}" ]]; then
if { [[ -z "${GIT_DIR}" ]] } -ne { [[ -z "${GIT_WORK_TREE}" ]] }; then
if [[ (( -z "${GIT_DIR}" )) -ne (( -z "${GIT_WORK_TREE}" )) ]]; then
I tried:
if [[ $(test -z "${GIT_DIR}") -ne $(test -z "${GIT_WORK_TREE}") ]]; then
But realized that doesn't work because it's a sub-process, and they'd need to be exported. as Socowl comments, this compares the outputs of the test commands which output nothing, not their exit statuses.
I apologize if this is a duplicate. I've searched here and google for a while, and must not be using the right terminology.
How about this:
if [[ "${GIT_DIR:+set}" != "${GIT_WORK_TREE:+set}" ]]; then
echo "GIT_DIR is '${GIT_DIR}'"
echo "GIT_WORK_TREE is '${GIT_WORK_TREE}'"
echo "Both or none must exist"
exit 1
fi
Explanation: ${var:+value} is a variant of parameter expansion that gives "value" if var is set to a nonempty string, or the empty string if var is unset or empty. So if both vars are unset/empty, it becomes if [[ "" != "" ]]; then, and if they're both set it becomes if [[ "set" != "set" ]]; then etc.
BTW, if you want to test whether the variables are set at all (even if to the empty string), use ${var+value} (note the lack of colon). The bash manual lists the :+ version, but not the + version.
#!/bin/bash
# Code to generate script usage
if [[ "$#" -ne 1 ]] && [[ "$#" -ne 2 ]]; then
flag=1;
elif ! [[ "$1" == "abcd" || "$1" == "dcba" ]]; then
echo "Invalid"
flag=1;
fi
while [ $# -gt 1 ]
do
case $2 in
'streams')
;;
*)
echo "unrecognised optional arg $2"; flag=1;
;;
esac
shift
done
if [ "$flag" == "1" ]; then
echo "Usage:"
exit
fi
function main {
arg1=$1
streams=$2
if [ "${streams}" == "streams" ]; then
echo entering here
else
echo entering there
fi
}
parent_dir=`pwd`
find $parent_dir -name "*" -type d | while read d; do
cd $denter code here
main $1 $2
done
Why the code does not enter "entering here" when script run with arguments "abcd" and "streams" ?
I feel that function having two arguments is causing the problem, code was working fine with one argument
Several things you might want to fix in your code, before attempts are made to find the specific problem. It is possible that it will disappear after modifying your script accordingly. If the problem is still alive, I'll edit my answer with a solution. If you decide to apply the following changes, please update your code in the question.
Consistent usage of either [[ or [. [[ is a Bash keyword similar to (but more powerful than) the [ command.
See
Bash FAQ 31
Tests And Conditionals
Unless you're writing for POSIX sh, I recommend [[.
Use (( for arithmetic expressions. ((...)) is an arithmetic command, which returns an exit status of 0 if the expression is nonzero, or 1 if the expression is zero. Also used as a synonym for let, if assignments are needed. See Arithmetic Expression.
Use the variable PWD instead of pwd. PWD is a builtin variable in all POSIX shells that contains the current working directory. pwd(1) is a POSIX utility that prints the name of the current working directory to stdout. Unless you're writing for some non-POSIX system, there is no reason to waste time executing pwd(1) rather than just using PWD.
The function keyword is not portable. I suggest you to avoid using it and simply write function_name() { your code here; } # Usage
$parent_dir is not double-quoted. "Double quote" every literal that contains spaces/metacharacters and every expansion: "$var", "$(command "$var")", "${array[#]}", "a & b". See
Quotes
Arguments
ShellCheck your code before uploading.
Replace the while condition logic with an if condition, so that shift is no longer required. Shift was the devil I was facing I found.
#!/bin/bash
# Code to generate script usage
if [[ "$#" -ne 1 ]] && [[ "$#" -ne 2 ]]; then
flag=1;
elif ! [[ "$1" == "abcd" || "$1" == "dcba" ]]; then
echo "Invalid"
flag=1;
fi
#while [[ $# -gt 1 ]]
#do
# case $2 in
# 'streams')
# ;;
# *)
# echo "unrecognised optional arg $2"; flag=1;
# ;;
# esac
# shift
#done
if [[ $2 == "streams" ]]; then
:
elif [[ (-z $2) ]]; then
:
else
echo "unrecognised optional arg $2"; flag=1;
fi
if [[ "$flag" == "1" ]]; then
echo "Usage:"
exit
fi
function main {
streams=$2
if [[ "${streams}" == "streams" ]]; then
echo entering here
else
echo entering there
fi
}
parent_dir=`pwd`
find $parent_dir -name "*" -type d | while read d; do
cd $d
main $1 $2
done
Often when writing for the bash shell, one needs to test if a file (or Directory) exists (or doesn't exist) and take appropriate action. Most common amongst these test are...
-e - file exists, -f - file is a regular file (not a directory or device file), -s - file is not zero size, -d - file is a directory, -r - file has read permission, -w - file has write, or -x execute permission (for the user running the test)
This is easily confirmed as demonstrated on this user-writable directory....
#/bin/bash
if [ -f "/Library/Application Support" ]; then
echo 'YES SIR -f is fine'
else echo 'no -f for you'
fi
if [ -w "/Library/Application Support" ]; then
echo 'YES SIR -w is fine'
else echo 'no -w for you'
fi
if [ -d "/Library/Application Support" ]; then
echo 'YES SIR -d is fine'
else echo 'no -d for you'
fi
➝ no -f for you ✓
➝ YES SIR -w is fine ✓
➝ YES SIR -d is fine ✓
My question, although seemingly obvious, and unlikely to be impossible - is how to simply combine these tests, without having to perform them separately for each condition... Unfortunately...
if [ -wd "/Library/Application Support" ]
▶ -wd: unary operator expected
if [ -w | -d "/Library/Application Support" ]
▶ [: missing `]'
▶ -d: command not found
if [ -w [ -d "/Library.... ]] & if [ -w && -d "/Library.... ]
▶ [: missing `]'
➝ no -wd for you ✖
➝ no -w | -d for you ✖
➝ no [ -w [ -d .. ]] for you ✖
➝ no -w && -d for you ✖
What am I missing here?
You can use logical operators to multiple conditions, e.g. -a for AND:
MYFILE=/tmp/data.bin
if [ -f "$MYFILE" -a -r "$MYFILE" -a -w "$MYFILE" ]; then
#do stuff
fi
unset MYFILE
Of course, you need to use AND somehow as Kerrek(+1) and Ben(+1) pointed it out. You can do in in few different ways. Here is an ala-microbenchmark results for few methods:
Most portable and readable way:
$ time for i in $(seq 100000); do [ 1 = 1 ] && [ 2 = 2 ] && [ 3 = 3 ]; done
real 0m2.583s
still portable, less readable, faster:
$ time for i in $(seq 100000); do [ 1 = 1 -a 2 = 2 -a 3 = 3 ]; done
real 0m1.681s
bashism, but readable and faster
$ time for i in $(seq 100000); do [[ 1 = 1 ]] && [[ 2 = 2 ]] && [[ 3 = 3 ]]; done
real 0m1.285s
bashism, but quite readable, and fastest.
$ time for i in $(seq 100000); do [[ 1 = 1 && 2 = 2 && 3 = 3 ]]; done
real 0m0.934s
Note, that in bash, "[" is a builtin, so bash is using internal command not a symlink to /usr/bin/test exacutable. The "[[" is a bash keyword. So the slowest possible way will be:
time for i in $(seq 100000); do /usr/bin/\[ 1 = 1 ] && /usr/bin/\[ 2 = 2 ] && /usr/bin/\[ 3 = 3 ]; done
real 14m8.678s
You want -a as in -f foo -a -d foo (actually that test would be false, but you get the idea).
You were close with & you just needed && as in [ -f foo ] && [ -d foo ] although that runs multiple commands rather than one.
Here is a manual page for test which is the command that [ is a link to. Modern implementations of test have a lot more features (along with the shell-builtin version [[ which is documented in your shell's manpage).
check-file(){
while [[ ${#} -gt 0 ]]; do
case $1 in
fxrsw) [[ -f "$2" && -x "$2" && -r "$2" && -s "$2" && -w "$2" ]] || return 1 ;;
fxrs) [[ -f "$2" && -x "$2" && -r "$2" && -s "$2" ]] || return 1 ;;
fxr) [[ -f "$2" && -x "$2" && -r "$2" ]] || return 1 ;;
fr) [[ -f "$2" && -r "$2" ]] || return 1 ;;
fx) [[ -f "$2" && -x "$2" ]] || return 1 ;;
fe) [[ -f "$2" && -e "$2" ]] || return 1 ;;
hf) [[ -h "$2" && -f "$2" ]] || return 1 ;;
*) [[ -e "$1" ]] || return 1 ;;
esac
shift
done
}
check-file fxr "/path/file" && echo "is valid"
check-file hf "/path/folder/symlink" || { echo "Fatal error cant validate symlink"; exit 1; }
check-file fe "file.txt" || touch "file.txt" && ln -s "${HOME}/file.txt" "/docs/file.txt" && check-file hf "/docs/file.txt" || exit 1
if check-file fxrsw "${HOME}"; then
echo "Your home is your home from the looks of it."
else
echo "You infected your own home."
fi
Why not write a function to do it?
check_file () {
local FLAGS=$1
local PATH=$2
if [ -z "$PATH" ] ; then
if [ -z "$FLAGS" ] ; then
echo "check_file: must specify at least a path" >&2
exit 1
fi
PATH=$FLAGS
FLAGS=-e
fi
FLAGS=${FLAGS#-}
while [ -n "$FLAGS" ] ; do
local FLAG=`printf "%c" "$FLAGS"`
if [ ! -$FLAG $PATH ] ; then false; return; fi
FLAGS=${FLAGS#?}
done
true
}
Then just use it like:
for path in / /etc /etc/passwd /bin/bash
{
if check_file -dx $path ; then
echo "$path is a directory and executable"
else
echo "$path is not a directory or not executable"
fi
}
And you should get:
/ is a directory and executable
/etc is a directory and executable
/etc/passwd is not a directory or not executable
/bin/bash is not a directory or not executable
This seems to work (notice the double brackets):
#!/bin/bash
if [[ -fwd "/Library/Application Support" ]]
then
echo 'YES SIR -f -w -d are fine'
else
echo 'no -f or -w or -d for you'
fi