Shell Script Syntax Error: Unexpected End of File [duplicate] - shell

This question already has answers here:
Bash syntax error: unexpected end of file
(21 answers)
Closed 3 years ago.
In the following script I get an error:
syntax error: unexpected end of file
What is this error how can I resove it? It is pointing at the line whee the function is called.
#!/bin/sh
expected_diskusage="264"
expected_dbconn="25"
expected_httpdconn="20"
expected_cpuusage="95"
#expected_fd="100"
httpdconn=`ps -ef|grep -i httpd|grep -v grep|wc -l` #httpd connections
cpu_usage=`ps aux|awk 'NR > 0 { s +=$3 }; END {print s}'`
disk_usage=`df -h|awk {'print $2'}|head -n3|awk 'NF{s=$0}END{print s}'`
#db_connections=`mysql -uroot -pexxxxxx -s -N -e "show processlist"|wc -l`
db_connections=6
cld_alert()
{
nwconn=$1
cpu_usage=$2
disk_usage=$3
db_connections=$4
message=$5
`touch /tmp/alert.txt && > /tmp/alert.txt`
date=`date`
echo -e "$date\n" > /tmp/alert.txt
echo -e "$message" >> /tmp/alert.txt
path="/proc/$httpd/fd/";
cd $path
tfd=`ls -l|wc -l`;
sfd=`ls -ltr|grep sock|wc -l`;
echo "Total fds: $tfd" >> /tmp/alert.txt
echo "Socket fds: $sfd" >> /tmp/alert.txt
echo "Other fds: $[$tfd - $sfd]" >> /tmp/alert.txt
freememory=`vmstat | awk '{if (NR == 3) print "Free Memory:"\$4}'`;
echo "Free memory :$freememory" >> /tmp/alert.txt
Bufferedmemory=`vmstat | awk '{if (NR == 3) print "Buffered Memory:"\$5}'`;
echo "Buffered memory $Bufferedmemory" >> /tmp/alert.txt
CacheMemory=`vmstat | awk '{if (NR == 3) print "Cache Memory:"\$6}'`;
echo "Cache memory : $CacheMemory" >> /tmp/alert.txt
sshconn=`netstat -an|grep 22|wc -l` #ssh connections
httpsconn=`netstat -an|grep 443|wc -l` #https connections
wwwconn=`netstat -an|grep 80|wc -l` #www connections
echo "Disk usage is $disk_usage" >> /tmp/alert.txt
echo "DB connections $db_connections" >> /tmp/alert.txt
echo "Network connections $nwconn" >> /tmp/alert.txt
echo "CPU Usage: $cpu_usage" >> /tmp/alert.txt
topsnapshot=`top -n 1 -b`
echo "===========================TOP COMMAND SNAPSHOT====================================================";
echo "$topsnapshot" >> /tmp/alert.txt
echo"==================PS COMMAND SNAPSHOT=============================================================="
entireprocesslist=`ps -ef`
echo "$entireprocesslist" >> /tmp/alert.txt
echo Hello hi"";
}
message=""
if [ ${disk_usage%?} -le $expected_diskusage ] ##{x%?} Removes last character
then
echo "disk usage exceeded";
message="Disk usage limit exceeded \nCurrent disk usage is $disk_usage\nConfigured disk usage is $expected_diskusage\n\n\n\n\n";
#Checking for CPU usage
if [ $cpu_usage -ge $expected_cpuusage] ##{x%?}
then
echo "CPU usage exceeded";
if [ $message -ne "" ]
then
message="$message\n\nCPU usage exceeded configured usage limit \nCurrent CPU usage is $cpu_usage\nConfigured CPU usage is $expected_cpuusage\n\n\n\n\n";
else
message="CPU usage exceeded configured usage limit \nCurrent CPU usage is $cpu_usage\nConfigured CPU usage is $expected_cpuusage\n\n\n\n\n";
fi ;
fi
#Checking for httpd connections
if [ $httpdconn -ge $expected_httpdconn] ##{x%?}
then
echo "HTTPD connections exceeded";
if [ $message -ne "" ]
then
message="$message\n\nHTTPD connections exceeded configured usage limit \nCurrent HTTPD connections is $httpdconn\nConfigured HTTPD connection is $expected_httpdconn";
else
message="HTTPD connections exceeded configured usage limit \nCurrent HTTPD connections is $httpdconn\nConfigured HTTPD connection is $expected_httpdconn";
fi ;
fi ;
message="$message\n\n\n\n\n";
value=$(cld_alert $message $httpdconn $cpu_usage $disk_usage $db_connections)

Edit: Note that the original post has been edited since this answer was written and has been reformatted. You should look at the history to see the original formatting to understand the context for this answer.
This error occurs often when you have mismatched structure - that is, you do not have matching double quotes, matching single quotes, have not closed a control structure such as a missing fi with an if, or a missing done with a for.
The best way to spot these is to use correct indentation, which will show you where you have a broken control structure, and syntax highlighting, which will show you where quotes are not matched.
In this particular case, I can see you are missing a fi. In the latter part of your code, you have 5 ifs and 4 fis. However you also have a number of other problems - your backquoted touch /tmp/alert.txt... command is syntactically invalid, and you need a space before the closing bracket of an if test.
Clean up your code, and errors start to stand out.

in my case the issue was in the EOL Conversion. (End Of Line).
i created the file on windows and only after i converted the EOL from windows(CR LF) to unix(LF), everything went well.
I did the conversion with Notepad++ very easily from: Edit -> EOL Conversion -> Unix(LF)

Unrelated to the OP's problem, but my issue was that I'm a noob shell scripter. All the other languages I've used require parentheses to invoke methods, whereas shell doesn't seem to like that.
function do_something() {
# do stuff here
}
# bad
do_something()
# works
do_something

In my case, I found that placing a here document (like sqplus ... << EOF) statements indented also raise the same error as shown below:
./dbuser_case.ksh: line 25: syntax error: unexpected end of file
So after removing the indentation for this, then it went fine.
Hope it helps...

Indentation when using a block can cause this error and is very hard to find.
if [ ! -d /var/lib/mysql/mysql ]; then
/usr/bin/mysql --protocol=socket --user root << EOSQL
SET ##SESSION.SQL_LOG_BIN=0;
CREATE USER 'root'#'%';
EOSQL
fi
=> Example above will cause an error because EOSQL is indented. Remove indentation as shown below. Posting this because it took me over an hour to figure out the error.
if [ ! -d /var/lib/mysql/mysql ]; then
/usr/bin/mysql --protocol=socket --user root << EOSQL
SET ##SESSION.SQL_LOG_BIN=0;
CREATE USER 'root'#'%';
EOSQL
fi

I have found that this is sometimes caused by running a MS Dos version of a file. If that's the case dos2ux should fix that.
dos2ux file1 > file2

I had this problem when running some script in cygwin. Fixed by running dos2unix on the script, with proper description of problem and solution given in that answer

You've got an unclosed quote, brace, bracket, if, loop, or something.
If you can't see it just by looking (I'd recommend a syntax colouring editor and a neat indentation style), take a copy of the script, and delete half of it, cutting it of somewhere that ought to be valid. If the script runs, as far as it can, then the problem is in the other half. Repeat until you've narrowed down the problem.

echo"==================PS COMMAND SNAPSHOT=============================================================="
needs to be
echo "==================PS COMMAND SNAPSHOT=============================================================="
Else, a program or command named echo"===... is searched.
more problems:
If you do a grep (-A1: + 1 line context)
grep -A1 "if " cldtest.sh
you find some embedded ifs, and 4 if/then blocks.
grep "fi " cldtest.sh
only reveals 3 matching fi statements. So you forgot one fi too.
I agree with camh, that correct indentation from the beginning helps to avoid such errors. Finding the desired way later means double work in such spaghetti code.

Helpful post, I found that my error was using else if instead of elif like so:
if [ -z "$VARIABLE1" ]; then
# do stuff
else if [ -z "$VARIABLE2" ]; then
# do other stuff
fi
Fixed it by changing to this:
if [ -z "$VARIABLE1" ]; then
# do stuff
elif [ -z "$VARIABLE2" ]; then
# do other stuff
fi

It can also be caused by piping out of a pair of curly braces on a line.
This fails:
{ /usr/local/bin/mycommand ; outputstatus=$? } >> /var/log/mycommand.log 2>&1h
do_something
#Get NOW that saved output status for the following $? invocation
sh -c "exit $outputstatus"
do_something_more
while this is allowed:
{
/usr/local/bin/mycommand
outputstatus=$?
} >> /var/log/mycommand.log 2>&1h
do_something
#Get NOW that saved output status for the following $? invocation
sh -c "exit $outputstatus"
do_something_more

I have encountered the same error while trying to execute a script file created in windows OS using textpad. so that one can select proper file format like unix/mac etc.. or recreate the script in linux iteself.

Related

How to detect a non-rolling log file and pattern match in a shell script which is using tail, while, read, and?

I am monitoring a log file and if PATTERN didn't appear in it within THRESHOLD seconds, the script should print "error", otherwise, it should print "clear". The script is working fine, but only if the log is rolling.
I've tried reading 'timeout' but didn't work.
log_file=/tmp/app.log
threshold=120
tail -Fn0 ${log_file} | \
while read line ; do
echo "${line}" | awk '/PATTERN/ { system("touch pattern.tmp") }'
code to calculate how long ago pattern.tmp touched and same is assigned to DIFF
if [ ${diff} -gt ${threshold} ]; then
echo "Error"
else
echo "Clear"
done
It is working as expected only when there is 'any' line printed in the app.log.
If the application got hung for any reason and the log stopped rolling, there won't be any output by the script.
Is there a way to detect the 'no output' of tail and do some command at that time?
It looks like the problem you're having is that the timing calculations inside your while loop never get a chance to run when read is blocking on input. In that case, you can pipe the tail output into a while true loop, inside of which you can do if read -t $timeout:
log_file=/tmp/app.log
threshold=120
timeout=10
tail -Fn0 "$log_file" | while true; do
if read -t $timeout line; then
echo "${line}" | awk '/PATTERN/ { system("touch pattern.tmp") }'
fi
# code to calculate how long ago pattern.tmp touched and same is assigned to diff
if [ ${diff} -gt ${threshold} ]; then
echo "Error"
else
echo "Clear"
fi
done
As Ed Morton pointed out, all caps variable names are not a good idea in bash scripts, so I used lowercase variable names.
How about something simple like:
sleep "$threshold"
grep -q 'PATTERN' "$log_file" && { echo "Clear"; exit; }
echo "Error"
If that's not all you need then edit your question to clarify your requirements. Don't use all upper case for non exported shell variable names btw - google it.
To build further on your idea, it might be beneficial to run the awk part in the background and a continuous loop to do the checking.
#!/usr/bin/env bash
log_file="log.txt"
# threshold in seconds
threshold=10
# run the following process in the background
stdbuf -oL tail -f0n "$log_file" \
| awk '/PATTERN/{system("touch "pattern.tmp") }' &
while true; do
match=$(find . -type f -iname "pattern.tmp" -newermt "-${threshold} seconds")
if [[ -z "${match}" ]]; then
echo "Error"
else
echo "Clear"
fi
done
This looks to me like a watchdog timer. I've implemented something like this by forcing a background process to update my log, so I don't have to worry about read -t. Here's a working example:
#!/usr/bin/env bash
threshold=10
grain=2
errorstate=0
while sleep "$grain"; do
date '+[%F %T] watchdog timer' >> log
done &
trap "kill -HUP $!" 0 HUP INT QUIT TRAP ABRT TERM
printf -v lastseen '%(%s)T'
tail -F log | while read line; do
printf -v now '%(%s)T'
if (( now - lastseen > threshold )); then
echo "ERROR"
errorstate=1
else
if (( errorstate )); then
echo "Recovered, yay"
errorstate=0
fi
fi
if [[ $line =~ .*PATTERN.* ]]; then
lastseen=$now
fi
done
Run this in one window, wait $threshold seconds for it to trigger, then in another window echo PATTERN >> log to see the recovery.
While this can be made as granular as you like (I've set it to 2 seconds in the example), it does pollute your log file.
Oh, and note that printf '%(%s)T' format requires bash version 4 or above.

"Argument list too long" for every command [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 8 years ago.
Improve this question
Occasionally, when I have a program that generates large arrays I get this bug where every command throws the error
"Argument list too long"
even if I just type:
$ cp
-bash: /bin/cp: Argument list too long
$
I can't use ls, or even open a new file with vim:
$ vim test.txt
-bash: /usr/bin/vim: Argument list too long
$
I tried using "wait" for all bg processes to finish, but no change. It seems to happen inconsistently, but when it does, the only fix is to restart the shell.
Any ideas what might be going on?
Update: I did some further testing and i got the error to be repeatable. It happens when a recursively defined array reaches 85 elements in length. The first command which throws the error is a bc that doesnt even depend on the array! and then from there on out, almost every other command throws the same error.
Update: The program I'm using has many bash scripts working together, but I've determined the problem always arises in this one:
function MPMDrun_prop()
{
PARDIR=$1
COMPDIR=$2
runSTR=$3
NUMNODES=$4
ForceRun=$5
if [ $# -le 3 ] ; then
echo "USAGE: MPMDrun_prop \$PARDIR \$COMPDIR \$runSTR \$NUMNODES \$ForceRun"
fi
echo "in MPMDrun_Prop"
. $PARDIR/ParameterScan.inp
. $MCTDHBDIR/Scripts/get_NumberOfJobs.sh
if [ "$MPMD" != "T" ]; then
MPMDnodes=1
fi
## If no runscripts in the $PARDIR, copy one and strip of the line which runs the program
if [ -z "$(ls $PARDIR/run*.sh 2> /dev/null)" ] ; then
if [ "$forhost" == "maia" ]; then
cp $MCTDHBDIR/../PBS_Scripts/run-example-maia.sh $PARDIR/run.tmp
sed 's|mpirun.*||' < $PARDIR/run.tmp > $PARDIR/run.sh
jobtime=86400
elif [ "$forhost" == "hermit" ]; then
cp $MCTDHBDIR/../PBS_Scripts/run-example-hermit.sh $PARDIR/run.tmp
sed 's|aprun.*||' < $PARDIR/run.tmp > $PARDIR/run.sh
jobtime=86400
elif [ "$forhost" == "hornet" ]; then
cp $MCTDHBDIR/../PBS_Scripts/run-example-hornet.sh $PARDIR/run.tmp
sed 's|aprun.*||' < $PARDIR/run.tmp > $PARDIR/run.sh
jobtime=86400
elif [ "$forhost" == "bwgrid" ]; then
cp $MCTDHBDIR/../PBS_Scripts/run-example-BWGRID.sh $PARDIR/run.tmp
sed 's|mpirun.*||' < $PARDIR/run.tmp > $PARDIR/run.sh
jobtime=86400
fi
sed 's|nodes=[0-9]*|nodes=0|' < $PARDIR/run.sh > $PARDIR/run.tmp
sed 's|#PBS -N.*|#PBS -N MONSTER_'$MonsterName'|' < $PARDIR/run.tmp > $PARDIR/run.sh_
rm $PARDIR/run.sh
rm $PARDIR/run.tmp
chmod 755 $PARDIR/run.sh_
echo ". $MCTDHBDIR/Scripts/RunFlagSleeper.sh" >> $PARDIR/run.sh_
## Include check_convergence.sh for mixed relax/prop compatibility
echo ". $MCTDHBDIR/Scripts/check_convergence.sh" >> $PARDIR/run.sh_
echo "RunFlagSleeper $jobtime " >> $PARDIR/run.sh_
echo "(" >> $PARDIR/run.sh_
cp $PARDIR/run.sh_ $PARDIR/run1.sh
fi
### Add $runSTR to the most recent runscript
### find runscript$N.sh (run1.sh, run 2.sh, etc) that has numnodes less than $MPMDnodes
for qq in $(ls $PARDIR/run[0-9]*.sh | sort -g ); do
NodesInRun=$(cat $qq | grep -o "nodes *= *[0-9]*" | grep -o "[0-9]*")
if [ "$NodesInRun" -lt "$MPMDnodes" ]; then
## The number of nodes already specified in the runscript doesnt exceed the maximum, so add on another job
NewNodes=$(echo "$NodesInRun+$NUMNODES" | bc)
## Start each aprun command in its own subshell
## wait for 24 hrs after aprun, to guarantee that no subshell finishes before the job is done
sed 's|nodes=[0-9]*|nodes='$NewNodes'|' < $qq > $qq-1
sed 's|\(RunFlagSleeper .*\)|\1 '$COMPDIR'|' <$qq-1 >$qq
rm $qq-1
echo " (" >> $qq
## Sleeps for $jobtime - 5 mins, then removes runflag. in case aprun doesnt finish in $jobtime
echo " cd $COMPDIR" >> $qq
echo " $runSTR" >> $qq
## remove runflag after aprun command has finished
echo " rm $COMPDIR/RunFlag" >> $qq
# echo "sleep $jobtime" >> $qq-1
echo " ) &" >> $qq
# mv $qq-1 $qq
## put a flag in the computation directory so it isnt computed multiple times
touch $COMPDIR/RunFlag
if [[ "$NewNodes" -ge "$MPMDnodes" || "$ForceRun" == "T" ]]; then
## This last process made the nodecount exceed the maximum, or there is a ForceRun flag passed
## So now, exceute the runscript and start another
echo " wait" >> $qq
echo ") &" >> $qq
echo "PID=\$!" >> $qq
echo "wait \$PID" >> $qq
## Ensure the queue has room for the next job, if not, wait for it
Njobs=$(get_NumberOfJobs $runhost)
while [ "$Njobs" -ge "$maxjobs" ]; do
echo "Njobs=$Njobs and maxjobs=$maxjobs"
echo "Waiting 30 minutes for que to clear"
sleep 1800
done
echo "qsub $qq"
# qsub $qq
RunCount=$(echo $qq | grep -o 'run[0-9]*.sh' | grep -o '[0-9]*')
let "RunCount++"
cp $PARDIR/run.sh_ $PARDIR/run$RunCount.sh
fi
fi
done
}
The error typically starts at the 80-90'th call of this function at the first cp or bc. I've commented ALL array manipulations, so there is zero chance this is caused by the array being too large. The environment stays at ~100-200 Kb so that isn't the problem either.
That error message is a bit misleading. It should say something like "Argument list and environment use too much space".
The environment consists of all the environment variables you have exported, plus the environment your shell was started with. Normally, the environment should only be a few kilobytes, but there is nothing stopping you from exporting a million-byte string, and if you do that, you'll use up all the space allowed.
It's not totally obvious how much space the system allows for arguments + environment. You should be able to query the limit with getconf ARG_MAX, and with Gnu xargs you can get more information from xargs --show-limits </dev/null (in both cases, assuming you haven't exceeded the limit :) ), but sometimes the actual space available will turn out to be less than what is indicated.
In any event, it's not a good idea to try to stuff megabytes into the environment. If you're tempted to do that, put the data in a temporary file instead, and just export the name of the file.
Since you stated that when you have a program that generates large arrays you get this bug where every command throws the error "Argument list too long". So, I presume that last command you executed is causing problem for next command. My suggestion is that don't use large argument list for any command. This could cause an overflow in the environment causing problems even for next command. Instead of large arg list, use a file having list of data and use the file redirected for input as in:
command < inputfile

Check if file exists [BASH]

How do I check if file exists in bash?
When I try to do it like this:
FILE1="${#:$OPTIND:1}"
if [ ! -e "$FILE1" ]
then
echo "requested file doesn't exist" >&2
exit 1
elif
<more code follows>
I always get following output:
requested file doesn't exist
The program is used like this:
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
Any ideas please?
I will be glad for any help.
P.S. I wish I could show the entire file without the risk of being fired from school for having a duplicate. If there is a private method of communication I will happily oblige.
My mistake. Fas forcing a binary file into a wrong place. Thanks for everyone's help.
Little trick to debugging problems like this. Add these lines to the top of your script:
export PS4="\$LINENO: "
set -xv
The set -xv will print out each line before it is executed, and then the line once the shell interpolates variables, etc. The $PS4 is the prompt used by set -xv. This will print the line number of the shell script as it executes. You'll be able to follow what is going on and where you may have problems.
Here's an example of a test script:
#! /bin/bash
export PS4="\$LINENO: "
set -xv
FILE1="${#:$OPTIND:1}" # Line 6
if [ ! -e "$FILE1" ] # Line 7
then
echo "requested file doesn't exist" >&2
exit 1
else
echo "Found File $FILE1" # Line 12
fi
And here's what I get when I run it:
$ ./test.sh .profile
FILE1="${#:$OPTIND:1}"
6: FILE1=.profile
if [ ! -e "$FILE1" ]
then
echo "requested file doesn't exist" >&2
exit 1
else
echo "Found File $FILE1"
fi
7: [ ! -e .profile ]
12: echo 'Found File .profile'
Found File .profile
Here, I can see that I set $FILE1 to .profile, and that my script understood that ${#:$OPTIND:1}. The best thing about this is that it works on all shells down to the original Bourne shell. That means if you aren't running Bash as you think you might be, you'll see where your script is failing, and maybe fix the issue.
I suspect you might not be running your script in Bash. Did you put #! /bin/bash on the top?
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
You may want to use getopts to parse your parameters:
#! /bin/bash
USAGE=" Usage:
script.sh [-g] [-p] [-r FUNCTION_ID|-d FUNCTION_ID] FILE
"
while getopts gpr:d: option
do
case $option in
g) g_opt=1;;
p) p_opt=1;;
r) rfunction_id="$OPTARG";;
d) dfunction_id="$OPTARG";;
[?])
echo "Invalid Usage" 1>&2
echo "$USAGE" 1>&2
exit 2
;;
esac
done
if [[ -n $rfunction_id && -n $dfunction_id ]]
then
echo "Invalid Usage: You can't specify both -r and -d" 1>&2
echo "$USAGE" >2&
exit 2
fi
shift $(($OPTIND - 1))
[[ -n $g_opt ]] && echo "-g was set"
[[ -n $p_opt ]] && echo "-p was set"
[[ -n $rfunction_id ]] && echo "-r was set to $rfunction_id"
[[ -n $dfunction_id ]] && echo "-d was set to $dfunction_id"
[[ -n $1 ]] && echo "File is $1"
To (recap) and add to #DavidW.'s excellent answer:
Check the shebang line (first line) of your script to ensure that it's executed by bash: is it #!/bin/bash or #!/usr/bin/env bash?
Inspect your script file for hidden control characters (such as \r) that can result in unexpected behavior; run cat -v scriptFile | fgrep ^ - it should produce NO output; if the file does contain \r chars., they would show as ^M.
To remove the \r instances (more accurately, to convert Windows-style \r\n newline sequences to Unix \n-only sequences), you can use dos2unix file to convert in place; if you don't have this utility, you can use sed 's/'$'\r''$//' file > outfile (CAVEAT: use a DIFFERENT output file, otherwise you'll destroy your input file); to remove all \r instances (even if not followed by \n), use tr -d '\r' < file > outfile (CAVEAT: use a DIFFERENT output file, otherwise you'll destroy your input file).
In addition to #DavidW.'s great debugging technique, you can add the following to visually inspect all arguments passed to your script:
i=0; for a; do echo "\$$((i+=1))=[$a]"; done
(The purpose of enclosing the value in [...] (for example), is to see the exact boundaries of the values.)
This will yield something like:
$1=[-g]
$2=[input.txt]
...
Note, though, that nothing at all is printed if no arguments were passed.
Try to print FILE1 to see if it has the value you want, if it is not the problem, here is a simple script (site below):
#!/bin/bash
file="${#:$OPTIND:1}"
if [ -f "$file" ]
then
echo "$file found."
else
echo "$file not found."
fi
http://www.cyberciti.biz/faq/unix-linux-test-existence-of-file-in-bash/
Instead of plucking an item out of "$#" in a tricky way, why don't you shift off the args you've processed with getopts:
while getopts ...
done
shift $(( OPTIND - 1 ))
FILE1=$1

How to make "if not true condition"?

I would like to have the echo command executed when cat /etc/passwd | grep "sysa" is not true.
What am I doing wrong?
if ! [ $(cat /etc/passwd | grep "sysa") ]; then
echo "ERROR - The user sysa could not be looked up"
exit 2
fi
try
if ! grep -q sysa /etc/passwd ; then
grep returns true if it finds the search target, and false if it doesn't.
So NOT false (! false) == true.
if evaluation in shells are designed to be very flexible, and many times doesn't require chains of commands (as you have written).
Also, looking at your code as is, your use of the $( ... ) form of cmd-substitution is to be commended, but think about what is coming out of the process. Try echo $(cat /etc/passwd | grep "sysa") to see what I mean. You can take that further by using the -c (count) option to grep and then do if ! [ $(grep -c "sysa" /etc/passwd) -eq 0 ] ; then which works but is rather old school.
BUT, you could use the newest shell features (arithmetic evaluation) like
if ! (( $(grep -c "sysa" /etc/passwd) == 0 )) ; then ...`
which also gives you the benefit of using the c-lang based comparison operators, ==,<,>,>=,<=,% and maybe a few others.
In this case, per a comment by Orwellophile, the arithmetic evaluation can be pared down even further, like
if ! (( $(grep -c "sysa" /etc/passwd) )) ; then ....
OR
if (( ! $(grep -c "sysa" /etc/passwd) )) ; then ....
Finally, there is an award called the Useless Use of Cat (UUOC). :-) Some people will jump up and down and cry gothca! I'll just say that grep can take a file name on its cmd-line, so why invoke extra processes and pipe constructions when you don't have to? ;-)
I hope this helps.
I think it can be simplified into:
grep sysa /etc/passwd || {
echo "ERROR - The user sysa could not be looked up"
exit 2
}
or in a single command line
$ grep sysa /etc/passwd || { echo "ERROR - The user sysa could not be looked up"; exit 2; }
This one
if [[ ! $(cat /etc/passwd | grep "sysa") ]]; then
echo " something"
exit 2
fi
What am I doing wrong?
$(...) holds the value, not the exit status, that is why this approach is wrong. However, in this specific case, it does indeed work because sysa will be printed which makes the test statement come true. However, if ! [ $(true) ]; then echo false; fi would always print false because the true command does not write anything to stdout (even though the exit code is 0). That is why it needs to be rephrased to if ! grep ...; then.
An alternative would be cat /etc/passwd | grep "sysa" || echo error. Edit: As Alex pointed out, cat is useless here: grep "sysa" /etc/passwd || echo error.
Found the other answers rather confusing, hope this helps someone.
Here is an answer by way of example:
In order to make sure data loggers are online a cron script runs every 15 minutes that looks like this:
#!/bin/bash
#
if ! ping -c 1 SOLAR &>/dev/null
then
echo "SUBJECT: SOLAR is not responding to ping" | ssmtp abc#def.com
echo "SOLAR is not responding to ping" | ssmtp 4151112222#txt.att.com
else
echo "SOLAR is up"
fi
#
if ! ping -c 1 OUTSIDE &>/dev/null
then
echo "SUBJECT: OUTSIDE is not responding to ping" | ssmtp abc#def.com
echo "OUTSIDE is not responding to ping" | ssmtp 4151112222#txt.att.com
else
echo "OUTSIDE is up"
fi
#
...and so on for each data logger that you can see in the montage
at http://www.SDsolarBlog.com/montage
FYI, using &>/dev/null redirects all output from the command, including errors, to /dev/null
(The conditional only requires the exit status of the ping command)
Also FYI, note that since cron jobs run as root there is no need to use sudo ping in a cron script.
simply:
if ! examplecommand arg1 arg2 ...; then
#code block
fi
without any brackets.
On Unix systems that supports it (not macOS it seems):
if getent passwd "$username" >/dev/null; then
printf 'User %s exists\n' "$username"
else
printf 'User %s does not exist\n' "$username"
fi
This has the advantage that it will query any directory service that may be in use (YP/NIS or LDAP etc.) and the local password database file.
The issue with grep -q "$username" /etc/passwd is that it will give a false positive when there is no such user, but something else matches the pattern. This could happen if there is a partial or exact match somewhere else in the file.
For example, in my passwd file, there is a line saying
build:*:21:21:base and xenocara build:/var/empty:/bin/ksh
This would provoke a valid match on things like cara and enoc etc., even though there are no such users on my system.
For a grep solution to be correct, you will need to properly parse the /etc/passwd file:
if cut -d ':' -f 1 /etc/passwd | grep -qxF "$username"; then
# found
else
# not found
fi
... or any other similar test against the first of the :-delimited fields.
I'd expect to see in the answers the direct use of grep with -q option (as we don't care the result but need only the return code.)
if ! grep -qs "sysa" /etc/passwd; then
echo "ERROR - The user sysa could not be looked up" >&2
exit 2
fi
or (to use chained execution on fail)
grep -qs "sysa" /etc/passwd || {
echo "ERROR - The user sysa could not be looked up" >&2
exit 2
}
Better, because the opposite is wanted, there's option -v for that
if grep -qsv "sysa" /etc/passwd; then
echo "ERROR - The user sysa could not be looked up" >&2
exit 2
fi
or (to use chained execution on success)
grep -qsv "sysa" /etc/passwd && {
echo "ERROR - The user sysa could not be looked up" >&2
exit 2
}
Notes
I like redirecting error messages to stderr, but echo output to stdout, hence >&2
Taylor the search pattern, e.g something like '^sysa:' if it's full login.

Possible spacing issue in a bash script. Command will not run in script but will when copied from output

I've gone around and around on the quoting stuff on http://tldp.org for bash and googled until I am blue in the face. I've also tried every obvious quoting scheme for this issue, and yet nothing works.
The problem seems to be that a space inside of a quoted argument in the command run at the end of the script is being interpreted as a separator instead of as a quoted space.
Behold, here's my script (I know full well I'm a noob so comments on my style and/or uneccessary syntax is cool with me, I'll learn):
#!/bin/bash
date=`date`
args="$#"
MSEND_HOME=/home/patrol/Impact #Path to the Impact Directory
integrationName=Introscope #Name of the integration
persistEnabled=1 #1 for Yes, 0 for No
persist=""
bufDir=$MSEND_HOME/tmp/$integrationName #DO NOT CHANGE
cellName=linuxtest #Cell name to forward events to
loggingEnabled=1 #1 for Yes, 0 for No
logFile=$MSEND_HOME/log/$integrationName.$cellName.log
die () {
if [ $loggingEnabled -eq 1 ]
then
echo >>$logFile "$#"
fi
exit 1
}
[ "$#" -ge 1 ] || die "$date - At least 1 argument required, $# provided" "$#"
# This is where you would parse out your arguments and form the following
# slots as a minimum for sending an event.
class=$2
msg=\"$3\"
# Parse the first argument and assign the correct syntax
if [[ $1 == "INFORMATIONAL" ]]
then
severity=INFO
elif [[ $1 == "WARN" ]]
then
severity=WARNING
elif [[ $1 == "CRIT" ]]
then
severity=CRITICAL
else
severity=INFO
fi
#Additional slots can be set, parse them all in this variable;
#e.g., additionalSlots="slot1=value1;slot2=value2;slot3=\"value 3\""
additionalSlots=""
cmd="$MSEND_HOME/bin/msend"
cmd="$cmd -q"
cmd="$cmd -l $MSEND_HOME"
if [ $persistEnabled -eq 1 ]
then
cmd="$cmd -j $bufDir"
fi
cmd="$cmd -n $cellName"
cmd="$cmd -a $class"
cmd="$cmd -m $msg"
cmd="$cmd -r $severity"
if [ $additionalSlots ]
then
cmd="$cmd -b $additionalSlots"
fi
$cmd || die "$date - msend exited with error $? | Original arguments: $args | Command: $cmd"
#echo "msend exited with error $? | Original arguments: $args | Command: $cmd"
The script is executed like this:
./sendEvent.sh "CRIT" "EVENT" "Test Event"
The error I get from the msend executable is that the arguments are wrong, but I'm logging the command line in it's entirety to a file and when I run that logged command in the shell interactively, it works.
Here's the log output:
Tue Oct 4 20:31:29 CDT 2011 - msend exited with error 27 | Original arguments: CRIT EVENT Test Event | Command: /home/patrol/Impact/bin/msend -q -l /home/patrol/Impact -j /home/patrol/Impact/tmp/Introscope -n linuxtest -a EVENT -m "Test Event" -r CRITICAL
So if I paste /home/patrol/Impact/bin/msend -q -l /home/patrol/Impact -j /home/patrol/Impact/tmp/Introscope -n linuxtest -a EVENT -m "Test Event" -r CRITICAL and run it, it works.
If I run the script like ./sendEvent.sh "CRIT" "EVENT" "TestEvent" it works. But I need that argument to allow spaces.
I'm on the track that it's an $IFS issue or something... maybe a difference between the interactive shell and the script environment.
I'd appreciate any insight from smarter people than me!
tl;dr - My command doesn't work when run from within a script, but does when the logged command syntax is used in an interactive shell.
Short answer: see BashFAQ #50.
Long answer: When bash parses a line, it parses quote marks before doing variable substitution; as a result, when you put quotes inside a variable, they don't do what you'd expect. You're actually passing an argument list including '-m' '"Test' 'Event"' '-r' -- those double-quotes aren't around the arguments, they're in the arguments.
In this case, the best solution is to build the command in an array rather than a string. Also, get in the habbit of putting double-quotes around variables (e.g. filenames) when you use them, to prevent confusion if they contain spaces. With those changes (and a few other tweaks), here's my version of your script:
#!/bin/bash
date="$(date)" # Backquotes are confusing, use $() instead
args=("$#") # Save the args in an array rather than mushing them together in a string
MSEND_HOME=/home/patrol/Impact #Path to the Impact Directory
MSEND_HOME="$HOME/tmp" #Path to the Impact Directory
integrationName=Introscope #Name of the integration
persistEnabled=1 #1 for Yes, 0 for No
persist=""
bufDir="$MSEND_HOME/tmp/$integrationName" #DO NOT CHANGE
cellName=linuxtest #Cell name to forward events to
loggingEnabled=1 #1 for Yes, 0 for No
logFile="$MSEND_HOME/log/$integrationName.$cellName.log"
die () {
if [ $loggingEnabled -eq 1 ]
then
echo >>"$logFile" "$#"
fi
exit 1
}
[ "$#" -ge 1 ] || die "$date - At least 1 argument required, $# provided" "$#"
# This is where you would parse out your arguments and form the following
# slots as a minimum for sending an event.
class="$2" # Quotes not strictly needed here, but a good habbit
msg="$3"
# Parse the first argument and assign the correct syntax
if [[ "$1" == "INFORMATIONAL" ]]
then
severity=INFO
elif [[ "$1" == "WARN" ]]
then
severity=WARNING
elif [[ "$1" == "CRIT" ]]
then
severity=CRITICAL
else
severity=INFO
fi
#Additional slots can be set, parse them all in this array;
#e.g., additionalSlots="slot1=value1;slot2=value2;slot3=value 3" # Don't embed quotes
additionalSlots=""
cmd=("$MSEND_HOME/bin/msend") # Build the command as an array, not a string
cmd+=(-q) # Could equivalently use cmd=("${cmd[#]}" -q), but this is simpler
cmd+=(-l "$MSEND_HOME")
if [ $persistEnabled -eq 1 ]
then
cmd+=(-j "$bufDir")
fi
cmd+=(-n "$cellName")
cmd+=(-a "$class") # Possible bug: $2 and #3 aren't required, but they're getting added unconditionally
cmd+=(-m "$msg") # These should probably be conditional, like additionalSlots
cmd+=(-r "$severity")
if [ -n "$additionalSlots" ]
then
cmd+=(-b "$additionalSlots")
fi
"${cmd[#]}" || die "$date - msend exited with error $? | Original arguments:$(printf " %q" "${args[#]}") | Command:$(printf " %q" "${cmd[#]}")"
#echo "msend exited with error $? | Original arguments:$(printf " %q" "${args[#]}") | Command:$(printf " %q" "${cmd[#]}")"
I think the arg goes wrong with this assignment: cmd="$cmd -m $msg".
Change it to cmd="$cmd -m \"$msg\"".
Okay, I don't see the exact problem immediately, but I can tell you what it is; this hint should help.
Remember that the shell quoting mechanism only interprets a string once. As a result, if you're not careful, what you thought was "foo" "a" "b" is in fact "foo a b" -- that is, all one token, not three.
Run the script with bash -x which will show you at each step what the shell is actually seeing.

Resources