I have shell script i've written that deletes the oldest logfile in a directory when the mount point reaches 90% capacity. When I run the script manually it works fine but when I attempt to use crontab to run it cannot seem to execute the actual rm command but it executes everything else in the script. See my crontab and script below.
0 * * * * /acsmgmt/iselogs/iselogcleanup.sh
#!/bin/bash
df -H | grep /acsmgmt | awk '{ print $4 " " $5 }' | while read output;
do
#!echo $output
usep=$(echo $output | awk '{ print $1 }' | cut -d '%' -f1)
#!echo $usep
if [ $usep -ge 90 ]; then
echo $(date) "Logs cleaned up" >> /tmp/isecleanup.log
rm -v `ls /acsmgmt/iselogs -rt | grep "iselog-" | head -1` >> /tmp/isecleanup.log
else
echo $(date) "No logs to clean up" >> /tmp/isecleanup.log
fi
done
So, the answer is indeed, as I suspected, to always make sure you specify a correct and complete PATH variable in any script called by cron.
(I keep making this same mistake myself, even after years of writing cron scripts -- some versions of cron allow you to specify a default PATH (and other environment variables) for all your scripts, and this can help, but it also needs careful maintenance.)
Related
I'm currently monitoring a log file and my ultimate goal is to write a script that uses tail -n0 -f and execute a certain command once grep finds a correspondence. My current code:
tail -n 0 -f $logfile | grep -q $pattern && echo $warning > $anotherlogfile
This works but only once, since grep -q stops when it finds a match. The script must keep searching and running the command, so I can update a status log and run another script to automatically fix the problem. Can you give me a hint?
Thanks
use a while loop
tail -n 0 -f "$logfile" | while read LINE; do
echo "$LINE" | grep -q "$pattern" && echo "$warning" > "$anotherlogfile"
done
awk will let us continue to process lines and take actions when a pattern is found. Something like:
tail -n0 -f "$logfile" | awk -v pattern="$pattern" '$0 ~ pattern {print "WARN" >> "anotherLogFile"}'
If you need to pass in the warning message and path to anotherLogFile you can use more -v flags to awk. Also, you could have awk take the action you want instead. It can run commands via the system() function where you pass the shell command to run
Trying to output a list of cronjobs, not a list of users. The raw output of crontab -l is way too dirty and I can't seem to clean it up. I run this with sudo script.sh or su and then run it. I've tried invoking it sudo script.sh | grep -v no also. I'm mystified why this doesn't work:
#!/bin/bash
#Trying to show all cronjobs but no extraneous info
#
# This shows "no crontab for USER" for every USER without
# a crontab - I only want to see actual cronjobs, not a long
# list of users without crontabs
echo "Here is the basic output that needs manipulation:
"
for USER in `cat /etc/passwd | cut -d":" -f1`; do
crontab -l -u $USER
done
#
# grep -v fails me
# (grep'ing the output of the script as a whole fails also)
echo "
trying with grep -v no on each line:
"
for USER in `cat /etc/passwd | cut -d":" -f1`; do
crontab -l -u $USER | grep -v no
done
echo "
maybe with quotes around the no:
"
for USER in `cat /etc/passwd | cut -d":" -f1`; do
crontab -l -u $USER | grep -v "no"
done
# string manipulation - I can't even get started
echo "
And here I try to put the commmand output into a string so I can manipulate it further, and use an if/then/fi on the product:
"
for USER in `cat /etc/passwd | cut -d":" -f1`; do
STRING="$(crontab -l -u $USER | grep -v no)"
echo "STRING: $STRING"
done
BTW, is there an easier way to get code to format correctly here than pasting in 4 spaces at the beginning of each line? I must have experimented for 40 minutes. Not complaining, just asking.
crontab is wiring the "no crontab for user" to standard error. To get rid of those messages, you can run
crontab -l -u $USER 2>/dev/null
within your loop.
I would also suggest renaming USER into something else. The USER variable name is reserved, and should be set to your user (login) name. Generally, you should use lower case variable names, to avoid this sort of name clash.
I am trying to use inotifywait within a bash script to monitor a directory for a file with a certain tag in it (*SDS.csv).
I also only want to execute once (once when the file is written to the directory data ).
example:
#! /bin/bash
inotifywait -m -e /home/adam/data | while read LINE
do
if [[ $LINE == *SDS.csv ]]; then
./another_script.sh
fi
done
While this may not be the ideal solution, it may do the trick:
#! /bin/bash
while true
do
FNAME="$(inotifywait -e close_write /home/adam/data | awk '{ print $NF }')"
if [ -f "/home/adam/data/$FNAME" ]
then
if grep -q 'SDS.csv' "/home/adam/data/$FNAME"
then
./another_script.sh
fi
done
done
Hi i wrote the following bash script:
cat /home/xyz/wlandiscovery.sh
#!/bin/bash
DATE=`date +%d-%m-%Y__%H:%M:%S`
#Get the current standard interface e.g. eth0
INTERFACE=`route | grep '*' | awk '{print $8}'`
#Check if mac is available
if /usr/bin/arp-scan --interface $INTERFACE -l -r 5 | grep "xx:xx:xx:xx:xx:xx"
then
echo -e "$DATE AVAILABLE!" >> /home/xyz/wlandiscovery.log
else
echo -e "$DATE NOT AVAILABLE" >> /home/xyz/wlandiscovery.log
fi
exit 0
If i run this and the mac is available i get "AVAILABLE", if i disconnect the device it give "NOT AVAILABLE"...so run as expected.
But if i run it as Cronjob every 5 Minutes I get always "NOT AVAILABLE": (on a Debian system)
crontab -e
#......
*/5 * * * * /bin/bash /home/xyz/wlandiscovery.sh
Whats the problem here?
INTERFACE=`route | grep '*' | awk '{print $8}'`
On my system, route is /usr/sbin/route. /usr/sbin is most likely not in cron's PATH. Specify the full path:
INTERFACE=`/usr/sbin/route | awk '$2 == "*" {print $8}'`
Compare the command line output of the following on your Mac and Debian boxes:
INTERFACE=route | grep '*' | awk '{print $8}'
Is it the same? It should be in order to work.
Then, compare the command line output of:
/usr/bin/arp-scan --interface $INTERFACE -l -r 5 | grep "xx:xx:xx:xx:xx:xx"
Alright, now its working. seems that $PATH with crontab is not equal to $PATH in my Terminal prompt... if i do /sbin/route and /usr/bin/awk and /bin/grep it works.
i need to run hadoop command in bash script, which go through bunch of folders on amazon S3, then write those folder names into a txt file, then do further process. but the problem is when i ran the script, seems no folder names were written to txt file. i wonder if it's the hadoop command took too long to run and the bash script didn't wait until it finished and go ahead to do further process, if so how i can make bash wait until the hadoop command finished then go do other process?
here is my code, i tried both way, neither works:
1.
listCmd="hadoop fs -ls s3n://$AWS_ACCESS_KEY:$AWS_SECRET_KEY#$S3_BUCKET/*/*/$mydate | grep s3n | awk -F' ' '{print $6}' | cut -f 4- -d / > $FILE_NAME"
echo -e "listing... $listCmd\n"
eval $listCmd
...other process ...
2.
echo -e "list the folders we want to copy into a file"
hadoop fs -ls s3n://$AWS_ACCESS_KEY:$AWS_SECRET_KEY#$S3_BUCKET/*/*/$mydate | grep s3n | awk -F' ' '{print $6}' | cut -f 4- -d / > $FILE_NAME
... other process ....
any one knows what might be wrong? and is it better to use the eval function or just use the second way to run hadoop command directly
thanks.
I would prefer to eval in this case, prettier to append the next command to this one. and I would rather break down listCmd into parts, so that you know there is nothing wrong at the grep, awk or cut level.
listCmd="hadoop fs -ls s3n://$AWS_ACCESS_KEY:$AWS_SECRET_KEY#$S3_BUCKET/*/*/$mydate > $raw_File"
gcmd="cat $raw_File | grep s3n | awk -F' ' '{print $6}' | cut -f 4- -d / > $FILE_NAME"
echo "Running $listCmd and other commands after that"
otherCmd="cat $FILE_NAME"
eval "$listCmd";
echo $? # This will print the exit status of the $listCmd
eval "$gcmd" && echo "Finished Listing" && eval "$otherCmd"
otherCmd will only be executed if $gcmd succeeds. If you have too many commands that you need to execute, then this becomes a bit ugly. If you roughly know how long it will take, you can insert a sleep command.
eval "$listCmd"
sleep 1800 # This will sleep 1800 seconds
eval "$otherCmd"