When I wrote this script it works just fine. What it does is it diffs two file
and if the diff fails that indicates that their is no change between the todays file
and the previous days file. That menas the the download is stale.
However when I to email it it is not formatted. I treid putting in a newline
so that the email is legible. But the mail function does not work. The file is
to garbled to read.
#!/bin/bash
dayofweek=$(/bin/date +%w)
today=$(/bin/date +%Y.%m.%d_)
yesterday=$(/bin/date -d '1 day ago' +%Y.%m.%d_)
destination="/sbclocal/stmevt3/dailymetrics/EQ_PERFORMANCE/"
file1=OPTS_TRIP_TRIP_csv_Oct2014.csv
file2=OPTS_TRIPnon-penny1-20_TRIPnon-penny1-20_Oct2014.csv
file3=OPTS_TRIPnon-penny21-50_TRIPnon-penny21-50_Oct2014.csv
file4=OPTS_TRIPnon-penny51-100_TRIPnon-penny51-100_Oct2014.csv
file5=OPTS_TRIPpenny1-20_TRIPpenny1-20_Oct2014.csv
file6=OPTS_TRIPpenny21-50_TRIPpenny21-50_Oct2014.csv
file7=OPTS_TRIPpenny51-100_TRIPpenny51-100_Oct2014.csv
for i in $file1 $file2 $file3 $file4 $file5 $file6 $file7
do
if diff $destination$today$i $destination$yesterday$i > /dev/null ; then
printf "$today$i may be stale - please notify production\n" >> /tmp/eq_diffs.$today
sleep 2
else
echo " " > /dev/null
fi
done
#think we might have to do a if file exists
cat /tmp/eq_diffs.$today | mail -s "EQ performance diffs" casper#casper.com
Related
i have simple script :
lftp user#server -e "cd dir && ls -ltr ;exit" > list.txt
if files=$(cat list.txt | grep "`date | awk '{print $2" "$3}'`") ; then
echo "$files" | mailx -s "File exists" name#name.com
else
echo "$files" | mailx -s "File not exists" name#name.com
end if
problem is because this grep date from file not working corectly, sometimes is working.
Can someone tell me what is better way to check on ftp server if actual date file exists and send me email?
I think there are three possible problems here. The first is that date | awk '{print $2" "$3}' includes a comma on the end of the day number. The second is that the day number is space padded by ls and date, but awk will be stripping the padding. The third is that the terminator for if in bash is fi rather then end if.
Try
if files=$(grep "`date '+%b %e'`" list.txt); then
echo "$files" | mailx -s "File exists" name#name.com
else
echo "$files" | mailx -s "File not exists" name#name.com
fi
The %e in date assumes ls returns a space-padded day number. If your system uses a zero-padded day number, try %d.
So I have a script here that is taking a TCPDUMP output. We are trying to send (2) variables to a PHP script over the web ($SERVER). The filename header is created and contains both $FILETOSEND which is the filename and filedata. The actual data for the filedata variable is coming from a file called 1 (the data is formatted as you can tell). I am having issues with the section that calls out #send common 10 sec dump.
I am trying to CURL the file 1 and I am doing so by using curl --data "$(cat 1)" $SERVER
The script isn't sending the file 1 at all, mostly just sends the filename and no file data. Is there a problem with the way I am sending the file? Is there a better way to format it?
while true; do
sleep $DATASENDFREQ;
killall -9 tcpdump &> /dev/null
if [ -e $DUMP ]; then
mv $DUMP $DUMP_READY
fi
create_dump
DATE=`date +"%Y-%m-%d_%H-%M-%S"`
FILETOSEND=$MAC-$DATE-$VERSION
# we write fileheader to the file. 2 vars : filename, filedata.
FILEHEADER="filename=$FILETOSEND&filedata="
echo $FILEHEADER > 2
# change all colons to underscores for avoiding Windows filenames issues
sed -i 's/:/_/g' 2
# delete all newlines \n in the file
tr -d '\n' < 2 > 1
# parsing $DUMP_READY to awk.txt (no header in awk.txt)
awk '{ if (NF > 18 && $10 == "signal") {print "{\"mac\": \""$16"\",\"sig\": \""$9"\",\"ver\": \""$8"\",\"ts\": \""$1"\",\"ssid\": \""$19"\"}" }}' $DUMP_READY > awk.txt
sed -i 's/SA://g' awk.txt
sed -i 's/&/%26/g' awk.txt
cat awk.txt >> 1
sync
# send $OFFLINE
if [ -e $OFFLINE ]; then
curl -d $OFFLINE $SERVER
if [ $? -eq "0" ]; then
echo "status:dump sent;msg:offline dump sent"
rm $OFFLINE
else
echo "status:dump not sent;msg:offline dump not sent"
fi
fi
# send common 10 secs dump
curl --data "$(cat 1)" $SERVER
if [ $? -eq "0" ]; then
echo "status:dump sent"
else
cat 1 >> $OFFLINE
echo "status:dump not sent"
fi
if [ -e $DUMP_READY ]; then
rm -f $DUMP_READY 1 2 upload_file*
fi
In the process of writing a bash script to parse a tab-separated file and unfortunately I need the ask the user for a date outside of the contents/creation of the file. I've gotten everything working, except looping through badly entered dates from the user until they enter one matching the desired format.
My debug code-block is as follows...
Code:
#!/bin/bash
USER_INPUT="01-01-2011" # ARBITRARILY ASSIGNING A BAD DATE BECAUSE I'M TOO LAZY TO TYPE ONE IN EACH TIME
EXPERIMENTDATE="$USER_INPUT"
if [[ $OSTYPE == *"linux"* ]]
then
date -d \"$EXPERIMENTDATE\" +%Y-%m-%d > /dev/null 2>&1
else
date -j -f \"%Y-%m-%d\" \"$EXPERIMENTDATE\" +%Y-%m-%d > /dev/null 2>&1
fi
is_valid="$?"
echo -e "$is_valid"
# FYI - $? RETURNS A BINARY FLAG ON THE LAST COMMAND'S EXECUTION. 1 IF ERROR, 0 IF NORMAL EXIT
while [ $is_valid -ne 0 ]; do
echo -e "Invalid date entered. Please enter the day the experiement was conducted on, in exactly the following format. YYYY-MM-DD (e.g. 2011-04-22)"
read USER_INPUT
EXPERIMENTDATE=$USER_INPUT
echo -e "You entered $EXPERIMENTDATE"
if [[ $OSTYPE == *"linux"* ]]
then
echo -e "DEBUG: date -d \"$EXPERIMENTDATE\" +%Y-%m-%d > /dev/null 2>&1"
date -d \"$EXPERIMENTDATE\" +%Y-%m-%d > /dev/null 2>&1
else
echo -e "DEBUG: date -j -f \"%Y-%m-%d\" \"$EXPERIMENTDATE\" +%Y-%m-%d > /dev/null 2>&1"
date -j -f \"%Y-%m-%d\" \"$EXPERIMENTDATE\" +%Y-%m-%d > /dev/null 2>&1
fi
is_valid="$?"
echo -e "DEBUG: $is_valid"
done
echo -e "You entered $EXPERIMENTDATE"
From the above, none of the date commands seem to evaluate correctly within the if's, but executing the debug commands directly on the CLI work.
I'm sure this is going to be a quoting/back-tic deal, but I can't seem to figure it out.
You don't need to escape the quotes that aren't inside similar quotes. For example, not
date -j -f \"%Y-%m-%d\" \"$EXPERIMENTDATE\" +%Y-%m-%d > /dev/null 2>&1
but
date -j -f "%Y-%m-%d" "$EXPERIMENTDATE" +%Y-%m-%d > /dev/null 2>&1
Hi I'm a newby in Bash scripting.
I need to log a data stream from a specific IP address and generate a logfile for each day as "file-$date.log" (i.e at 00:00:00 UT close the previous day file and create the correspondig to the new one)
I need to show data stream on screen while it is logged in a file
I try this solution but not works well because never closesthe initial file
apparently the condition check never executes while the first command of the pipe it is something different to an constant string like echo "something".
#!/bin/bash
log_data(){
while IFS= read -r line ; do printf '%s %s\n' "$(date -u '+%j %Y-%m-%d %H:%M:%S')" "$line"; done
}
register_data() {
while : ;
do
> stream.txt
DATE=$(date -u "+%j %Y-%m-%d %H:%M")
HOUR=$(date -u "+%H:%M:%S")
file="file-$DATE.log"
while [[ "${HOUR}" != 00:00:00 ]];
do
tail -f stream.txt | tee "${file}"
sleep 1
HOUR=$(date -u "+%H:%M:%S")
done
> stream.txt
done
}
nc -vn $IP $IP_port | log_data >> stream.txt &
register_data
I'll will be glad if someone can give me some clues to solve this problem.
I have log file in which I'm trying to delete all entries older than a specified date. Though I haven't succeeded with this yet. What I've tested so far is having an input for what the entries must be older than to be deleted and then loop like this:
#!/bin/bash
COUNTER=7
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d -#${DATE} "+%Y-%m-%d")
while [ -n "$(grep $DATE test.txt)" ]; do
sed -i "/$DATE/d" test.txt
COUNTER=$((${COUNTER}+1))
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d #${DATE} +"%Y-%m-%d")
done
This kind of works except when a log entry doesn't exist for date. When it doesn't find a match it aborts the loop and the even older entries are kept.
Update
This was how I solved it:
#!/bin/bash
COUNTER=$((7+1))
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d -#${DATE} "+%Y-%m-%d")
if [ -z "$(grep $DATE test.txt)" ]; then
exit 1
fi
sed -i "1,/$DATE/d" test.txt
Sorry for answering my own question but I went with Martin Frost's suggestion in the comments. It was much easier than the other suggestions.
This was my implementation:
#!/bin/bash
# requirements for script script
COUNTER=$((7+1))
DATE=$(date -d "-${COUNTER} days" +%s)
DATE=$(date -d -#${DATE} "+%Y-%m-%d")
sed -i "1,/$DATE/d" test.txt
Thanks for all the help!
Depending on your logfile format, assuming that the timestamp is the first column in the file you can do it like this with (g)awk.
awk 'BEGIN { OneWeekEarlier=strftime("%Y-%m-%d",systime()-7*24*60*60) }
$1 <= OneWeekEarlier { next }
1' INPTUTLOG > OUTPUTLOG
This computes the date - surprise, surprise - one week earlier, then checks if the first column (white space separated columns by default) is less than or equal, and if true, skips the line, otherwise prints.
The hard part is doing the "in place" editing with awk. But it can be done:
{ rm LOGFILE && awk 'BEGIN { OneWeekEarlier=strftime("%Y-%m-%d",systime()-7*24*60*60) }
$1 <= OneWeekEarlier { next }
1' > LOGFILE ; } < LOGFILE
HTH
I deleted log records in syslog-ng files before 60 days ago with following code.
#!/bin/bash
LOGFILE=/var/log/syslog
DATE=`date +"%b %e" --date="-60days"`
sed -i "/$DATE/d" $LOGFILE