There is a bash while loop with a counter, the first iteration is correct, but on the second iteration the counter increments by a random number, and from then on continues to increment as expected value of 1. Running Bash v4.2
Any ideas as to why this is happening?
COUNTER=1
while true
do
echo -e "$(date)\nCount $COUNTER "
aws cloudwatch put-metric-data --region eu-west-2 --namespace example/test --metric-data file:///tmp/metrics.dat
COUNTER=$(($COUNTER+1))
sleep 60
done > /tmp/metrics.log 2>&1 &
Tail output for log
tail -f /tmp/metrics.log
Fri 18 Jun 08:42:15 BST 2021
Count 1
Fri 18 Jun 08:42:40 BST 2021
Count 28
Fri 18 Jun 08:43:42 BST 2021
Count 29
Fri 18 Jun 08:44:43 BST 2021
Count 30
Fri 18 Jun 08:45:45 BST 2021
If the process is not redirected to the log file, then it works as expected. So the issue appears to be around this:
> /tmp/metrics.log 2>&1 &
Related
I need to add the timestamp of all remote servers as part of output and check & compare whether the timestamp is the same or not,
I am able to print the machine IP and date.
#!/bin/bash
all_ip=(192.168.1.121 192.168.1.122 192.168.1.123)
for ip_addr in "${all_ip[#]}"; do
aws_ip=$"ip route get 1 | sed -n 's/^.*src \([0-9.]*\) .*$/\1/p'"
date=date
sshpass -p "password" ssh root#$ip_addr "$aws_ip & $date"
echo "==================================================="
done
Getting Output as :
Wed 27 Jul 2022 05:48:15 AM PDT
192.168.1.121
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.122
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.123
===================================================
How to check whether the timestamp ( ignoring seconds ) of all machines is the same or not ,
eg: (Wed 27 Jul 2022 05:48:15 || Wed 27 Jul 2022 05:48:15 || Wed 27 Jul 2022 05:48:15)
Expected Output:
|| Time are in sync on all machines || # if in sync
|| Time are not in sync on all machines || # if not sync
Wed 27 Jul 2022 05:48:15 AM PDT
192.168.1.121
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.122
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.123
===================================================
How to check whether the time ( ignoring seconds )
tmpdir=$(mktemp -d)
trap 'rm -r "$tmpdir"' EXIT
for ip in "${allips[#]}"; do
# Do N connections, in paralllel, each one writes to a separate file.
sshpass -p "password" ssh root#"$ip" "date +%Y-%m-%d_%H:%M" > "$tmpdir/$ip.txt" &
done
wait
times=$(
for i in "$tmpdir"/*.txt; do
# print filename with file contents.
echo "$i $(<$i)"
done |
# Sort them on second column
sort -k2 |
# Uniq on second field
uniq -f 2
)
echo "$times"
timeslines=$(wc -l <<<"$times")
if ((timeslines == 1)); then
echo "YAY! minutes on all servers the same"
fi
First, you may adjust your "date" command as folow in order to exclude the seconds:
date +%Y-%m-%d_%H:%M
Then, simply grep your output and validate that all the timestamps are identical. You may dump in a temporary file or any other way.
Ex:
grep [aPatternSpecificToTheLinewithTheDate] [yourTemporaryFile] | sort | uniq | wc -l
If the result is 1, it means that all the timestamps are identical.
However you will have to deal with the corner case where the minute shift while you are fetching the time form all your servers.
This question already has answers here:
How do I pipe a file line by line into multiple read variables?
(3 answers)
Closed 1 year ago.
I am currently struggling, with splitting a string with a varying amount of spaces, coming from a log file.
An excerpt of the log file:
ProcessA Mon Nov 9 09:59 - 10:48 (00:48)
ProcessB Sun Nov 8 11:16 - 11:17 (00:00)
ProcessC Sat Nov 7 12:52 - 12:53 (00:00)
ProcessD Fri Nov 6 09:31 - 11:25 (01:54)
ProcessE Thu Nov 5 16:41 - 16:41 (00:00)
ProcessF Thu Nov 5 11:39 - 11:40 (00:00)
As you can see the amount of spaces between the process name and the date of execution varies between 2 to 5 spaces.
I would like to split it up into three parts; - process, date of execution, and execution time.
However I don’t see a solution to that, because of the unequal amount of spaces. Am I wrong or is splitting such a string incredibly hard?
Hopefully somebody out there is way smarter than me and can provide me with a solution for that 😊
Thanks to everybody in advance, who is willing trying to help me with that!
You can also assign fields directly in a read.
while read -r prc wd mon md start _ end dur _; do
echo "prc='$prc' wd='$wd' mon='$mon' md='$md' start='$start' end='$end' dur='${dur//[)(]/}'"
done < file
Output:
prc='ProcessA' wd='Mon' mon='Nov' md='9' start='09:59' end='10:48' dur='00:48'
prc='ProcessB' wd='Sun' mon='Nov' md='8' start='11:16' end='11:17' dur='00:00'
prc='ProcessC' wd='Sat' mon='Nov' md='7' start='12:52' end='12:53' dur='00:00'
prc='ProcessD' wd='Fri' mon='Nov' md='6' start='09:31' end='11:25' dur='01:54'
prc='ProcessE' wd='Thu' mon='Nov' md='5' start='16:41' end='16:41' dur='00:00'
prc='ProcessF' wd='Thu' mon='Nov' md='5' start='11:39' end='11:40' dur='00:00'
read generally doesn't care how much whitespace is between.
In bash, you can use a regex to parse each line:
#! /bin/bash
while IFS=' ' read -r line ; do
if [[ "$line" =~ ([^\ ]+)\ +(.+[^\ ])\ +'('([^\)]+)')' ]] ; then
process=${BASH_REMATCH[1]}
date=${BASH_REMATCH[2]}
time=${BASH_REMATCH[3]}
echo "$process $date $time."
fi
done
Or, use parameter expansions:
#! /bin/bash
while IFS=' ' read -r process datetime ; do
shopt -s extglob
date=${datetime%%+( )\(*}
time=${datetime#*\(}
time=${time%\)}
echo "$process $date $time."
done
Using awk:
awk '{printf $1; for (i=2; i<NF; i++) printf " %s",$i; print "",$NF}' < file.txt
produces:
ProcessA Mon Nov 9 09:59 - 10:48 (00:48)
ProcessB Sun Nov 8 11:16 - 11:17 (00:00)
ProcessC Sat Nov 7 12:52 - 12:53 (00:00)
ProcessD Fri Nov 6 09:31 - 11:25 (01:54)
ProcessE Thu Nov 5 16:41 - 16:41 (00:00)
ProcessF Thu Nov 5 11:39 - 11:40 (00:00)
I wrote a shell script and added it to my cron. It's supposed to run every minute and check for the average server load, past 1 minute, and if it's over 40 it should log the load, date and then restart Apache httpd. Here is my script:
#!/bin/bash
LOGFILE=/home/user/public_html/domain.com/cron/restart.log
function float_to_int() {
echo $1 | cut -d. -f1
}
check=$(uptime | awk -F' *,? *' '{print $12}')
now=$(date)
checkk=$(float_to_int $check)
if [[ $checkk > 40 ]]; then
echo $now $checkk >> $LOGFILE 2>&1
/usr/bin/systemctl restart httpd.service
fi
If I look at the log file I see the following:
Wed Jul 3 20:02:01 EDT 2019 70
Wed Jul 3 23:03:01 EDT 2019 43
Wed Jul 3 23:12:01 EDT 2019 9
Wed Jul 3 23:13:01 EDT 2019 7
Wed Jul 3 23:14:01 EDT 2019 6
Wed Jul 3 23:15:02 EDT 2019 5
Wed Jul 3 23:16:01 EDT 2019 5
Something is clearly wrong as it should only log and restart Apache if the load is over 40 but as you can see from the logs the load was 9, 7, 6, 5 and 5. Could someone point me in the right direction?
From man bash, section CONDITIONAL EXPRESSIONS (emphasis mine) :
string1 > string2
True if string1 sorts after string2 lexicographically.
You will either want to use [['s -gt operator, or use arithmetic evaluation instead of [[ :
if (( chekk > 40 )); then
Here's one in GNU awk (GNU awk due to strftime()):
awk '
$1 > 0.4 { # interval above 0.4
logfile="./log.txt" # my logpath, change it
print strftime("%c"), $1 >> logfile # date and load to log
cmd="/usr/bin/systemctl restart httpd.service" # command to use for restarting
if((ret=(cmd|getline res)) !=0 ) # store return value and result
print "failed: " ret # if failed
else
print "success"
}' /proc/loadavg # getting load avg from /proc
I want to create a simple command which logs the date/time, error and stdout to a file. So if I move a file to a folder and it says "Permission denied", I get a line in my file which shows the current date/time and also the error.
I know how to write the stdout and the error of a command to a file but how do I add a time? Thanks in advance!
Here's some code that can help:
function add_date() {
while IFS= read -r line; do
echo "$(date): $line"
done
}
{
# Your code here
} 2>&1 | add_date >> $LOGFILE
This will add the date to the beginning of every line output by your code ( between the braces anyway).
There may be some issues with output buffering. This will appear as the same timestamp on all the lines in your logfile.
Here's an example of the code above applied:
: ${LOGFILE:=logfile}
function add_date() {
while IFS= read -r line; do
echo "$(date): $line"
done
}
{
for a in {1..10}
do
echo $a
sleep 2
done
} 2>&1 | add_date >> $LOGFILE
And the results:
$ cat logfile
Thu Mar 28 14:50:46 EDT 2019: 1
Thu Mar 28 14:50:48 EDT 2019: 2
Thu Mar 28 14:50:50 EDT 2019: 3
Thu Mar 28 14:50:52 EDT 2019: 4
Thu Mar 28 14:50:54 EDT 2019: 5
Thu Mar 28 14:50:56 EDT 2019: 6
Thu Mar 28 14:50:58 EDT 2019: 7
Thu Mar 28 14:51:00 EDT 2019: 8
Thu Mar 28 14:51:02 EDT 2019: 9
Thu Mar 28 14:51:04 EDT 2019: 10
I use a function to do something like what you're asking for
status_msg () {
echo "`hostname --short`:`date '+%m_%d_%Y_%H_%M_%S'`: $*"
}
The above function can be called via
status_msg "This is a test line"
Which would result in
hostname:03_28_2019_13_50_31: This is a test line
Or if you're running a command which produces output, you can use it like so ..
<command> 2>&1 | while read -r line
do
status_msg $line
done
You can redirect stderr to stdout like so:
2>&1
then add the time before each line of either like so:
sed "s/\(.\)/`date` \1/"
so we wind up with something like this
2>&1 | sed "s/\(.\)/`date` \1/"
okay so i run an openssl command to get the date of an expired script. Doing so gives me this:
enddate=Jun 26 23:59:59 2012 GMT
Then i cut everything out and just leave the month which is "Jun"
Now the next part of my script is to tell the user if the the certificate is expired or not and to do that i use an if statement in which it looks like this:
if [ $exp_year -lt $cur_year && $exp_month -lt $cur_month ]; then
echo ""
echo "Certificate is still valid until $exp_date"
echo ""
else
echo ""
echo "Certificate has expired on $exp_date, please renew."
echo ""
fi
I can't figure out how to convert the month into an integer to even do the comparison.
I thought of doing the brute force way which is this:
Jan=01
Feb=02
Mar=03
...
Clearly that's a terrible way to do it. Does anyone know what i can do?
well, you can use:
now=$(date +%s)
cert=$(date --date="$enddate" +%s)
if [ $cert -lt $now ]; then
echo "Old!"
fi
i.e. convert the date into the seconds past the epoch and compare those
I would recommend using Petesh's answer, but here's a way to set up an associative array if you have Bash 4:
months=(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec)
declare -A mlookup
for monthnum in ${!months[#]}
do
mlookup[${months[monthnum]]=$((monthnum + 1))
done
echo "${mlookup["Jun"]}" # outputs 6
If you have Bash before version 4, you can use AWK to help you out:
month=Feb
awk -v "month=$month" 'BEGIN {months = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"; print (index(months, month) + 3) / 4}'
Another way in pure Bash (any version):
months="Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
month=Aug
string="${months%$month*}"
echo "$((${#string}/4 + 1))"