Im trying to write a script in which a while loop reads a file line by line, performs a command on two values on that line (to get two new values), then replaces the two old values with the two new values, then moves on to the next line.
For example the txt file example.txt contains the following data:
1432771200 != 1432800000 OPTION VALUE
1432771200 != 1432800210 OPTION VALUE
1432771200 != 1432800033 OPTION VALUE
And I run the following script:
#!/bin/bash -x
#
while read line
do
arr=($line)
CURRENTDATE=`h2e ${arr[0]}`
PROPOSEDDATE=`h2e ${arr[2]}`
echo $CURRENTDATE
echo $PROPOSEDDATE
# echo $line |
sed -i "s/${arr[0]}/$CURRENTDATE/"
# echo $line |
sed -i "s/${arr[2]}/$PROPOSEDDATE/"
done < /srg/pro/data/example.txt
What I expect to see now in example.txt file is the replacement of the first and third value on each line.. so it should look like this:
Thu May 28 01:00:00.000 2015 BST != Thu May 28 09:00:00.000 2015 BST OPTION VALUE
Thu May 28 01:00:00.000 2015 BST != Wed May 27 01:00:00.000 2015 BST OPTION VALUE
Fri May 28 01:00:00.000 2015 BST != Fri May 29 06:00:00.000 2015 BST OPTION VALUE
And so on and so forth..
When I run the shell script with bash -x Interface.sh Im getting this:
read line
+ arr=($line)
++ h2e 1432771200
+ CURRENTDATE='1432771200.000 2015148 Thu May 28 01:00:00.000 2015 BST (1)'
++ h2e 1432800000
+ PROPOSEDDATE='1432800000.000 2015148 Thu May 28 09:00:00.000 2015 BST (1)'
+ echo 1432771200.000 2015148 Thu May 28 01:00:00.000 2015 BST '(1)'
1432771200.000 2015148 Thu May 28 01:00:00.000 2015 BST (1)
+ echo 1432800000.000 2015148 Thu May 28 09:00:00.000 2015 BST '(1)'
1432800000.000 2015148 Thu May 28 09:00:00.000 2015 BST (1)
+ sed -i 's/1432771200/1432771200.000 2015148 Thu May 28 01:00:00.000 2015 BST (1)/'
sed: no input files
+ sed -i 's/1432800000/1432800000.000 2015148 Thu May 28 09:00:00.000 2015 BST (1)/'
sed: no input files
+ read line
+ arr=($line)
++ h2e 1432771200
+ CURRENTDATE='1432771200.000 2015148 Thu May 28 01:00:00.000 2015 BST (1)'
++ h2e 1432800000
+ PROPOSEDDATE='1432800000.000 2015148 Thu May 28 09:00:00.000 2015 BST (1)'
+ echo 1432771200.000 2015148 Thu May 28 01:00:00.000 2015 BST '(1)'
1432771200.000 2015148 Thu May 28 01:00:00.000 2015 BST (1)
+ echo 1432800000.000 2015148 Thu May 28 09:00:00.000 2015 BST '(1)'
1432800000.000 2015148 Thu May 28 09:00:00.000 2015 BST (1)
+ sed -i 's/1432771200/1432771200.000 2015148 Thu May 28 01:00:00.000 2015 BST (1)/'
sed: no input files
+ sed -i 's/1432800000/1432800000.000 2015148 Thu May 28 09:00:00.000 2015 BST (1)/'
sed: no input files
Please help! Dont know how to fix this!
The read command can read split the line into fields:
outfile=/tmp/some_tmp_file_you_will_move
while read date1 marker date2 otherfields; do
CURRENTDATE=`h2e ${date1}`
PROPOSEDDATE=`h2e ${date2}`
echo $CURRENTDATE
echo $PROPOSEDDATE
echo "${CURRENTDATE} ${marker} ${PROPOSEDDATE} ${otherfields}" >> ${outfile}
done < /srg/pro/data/example.txt
When you don't need to see the CURRENTDATE/PROPOSEDDATE inside your loop, you can redirect the output outside the loop for a better performance:
outfile=/tmp/some_tmp_file_you_will_move
while read date1 marker date2 otherfields; do
CURRENTDATE=`h2e ${date1}`
PROPOSEDDATE=`h2e ${date2}`
echo "${CURRENTDATE} ${marker} ${PROPOSEDDATE} ${otherfields}"
done < /srg/pro/data/example.txt > ${outfile}
Note: Consider using $(h2e ...) and not ``
Using your batch (there are other way)
#!/bin/bash -x
#
cp /srg/pro/data/example.txt /tmp/temp.file
cat /srg/pro/data/example.txt \
| while read line
do
arr=($line)
CURRENTDATE=`h2e ${arr[0]}`
PROPOSEDDATE=`h2e ${arr[2]}`
echo $CURRENTDATE
echo $PROPOSEDDATE
# echo $line |
sed -i "s/${arr[0]}\(.*\)${arr[2]}/${CURRENTDATE}\1${PROPOSEDDATE}/" /tmp/temp.file
done
mv /tmp/temp.file /srg/pro/data/example.txt
sed -i does not work on stream, need a file
I use a temporary file to avoid problem reading the same file that is modified inside the loop that often create unexpected result (like empty file)
this work for your sample but it depend of line structure for your array and sed substitution in real structure of data file
Related
I need to add the timestamp of all remote servers as part of output and check & compare whether the timestamp is the same or not,
I am able to print the machine IP and date.
#!/bin/bash
all_ip=(192.168.1.121 192.168.1.122 192.168.1.123)
for ip_addr in "${all_ip[#]}"; do
aws_ip=$"ip route get 1 | sed -n 's/^.*src \([0-9.]*\) .*$/\1/p'"
date=date
sshpass -p "password" ssh root#$ip_addr "$aws_ip & $date"
echo "==================================================="
done
Getting Output as :
Wed 27 Jul 2022 05:48:15 AM PDT
192.168.1.121
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.122
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.123
===================================================
How to check whether the timestamp ( ignoring seconds ) of all machines is the same or not ,
eg: (Wed 27 Jul 2022 05:48:15 || Wed 27 Jul 2022 05:48:15 || Wed 27 Jul 2022 05:48:15)
Expected Output:
|| Time are in sync on all machines || # if in sync
|| Time are not in sync on all machines || # if not sync
Wed 27 Jul 2022 05:48:15 AM PDT
192.168.1.121
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.122
===================================================
Wed Jul 27 05:48:15 PDT 2022
192.168.1.123
===================================================
How to check whether the time ( ignoring seconds )
tmpdir=$(mktemp -d)
trap 'rm -r "$tmpdir"' EXIT
for ip in "${allips[#]}"; do
# Do N connections, in paralllel, each one writes to a separate file.
sshpass -p "password" ssh root#"$ip" "date +%Y-%m-%d_%H:%M" > "$tmpdir/$ip.txt" &
done
wait
times=$(
for i in "$tmpdir"/*.txt; do
# print filename with file contents.
echo "$i $(<$i)"
done |
# Sort them on second column
sort -k2 |
# Uniq on second field
uniq -f 2
)
echo "$times"
timeslines=$(wc -l <<<"$times")
if ((timeslines == 1)); then
echo "YAY! minutes on all servers the same"
fi
First, you may adjust your "date" command as folow in order to exclude the seconds:
date +%Y-%m-%d_%H:%M
Then, simply grep your output and validate that all the timestamps are identical. You may dump in a temporary file or any other way.
Ex:
grep [aPatternSpecificToTheLinewithTheDate] [yourTemporaryFile] | sort | uniq | wc -l
If the result is 1, it means that all the timestamps are identical.
However you will have to deal with the corner case where the minute shift while you are fetching the time form all your servers.
There is a bash while loop with a counter, the first iteration is correct, but on the second iteration the counter increments by a random number, and from then on continues to increment as expected value of 1. Running Bash v4.2
Any ideas as to why this is happening?
COUNTER=1
while true
do
echo -e "$(date)\nCount $COUNTER "
aws cloudwatch put-metric-data --region eu-west-2 --namespace example/test --metric-data file:///tmp/metrics.dat
COUNTER=$(($COUNTER+1))
sleep 60
done > /tmp/metrics.log 2>&1 &
Tail output for log
tail -f /tmp/metrics.log
Fri 18 Jun 08:42:15 BST 2021
Count 1
Fri 18 Jun 08:42:40 BST 2021
Count 28
Fri 18 Jun 08:43:42 BST 2021
Count 29
Fri 18 Jun 08:44:43 BST 2021
Count 30
Fri 18 Jun 08:45:45 BST 2021
If the process is not redirected to the log file, then it works as expected. So the issue appears to be around this:
> /tmp/metrics.log 2>&1 &
I wrote a shell script and added it to my cron. It's supposed to run every minute and check for the average server load, past 1 minute, and if it's over 40 it should log the load, date and then restart Apache httpd. Here is my script:
#!/bin/bash
LOGFILE=/home/user/public_html/domain.com/cron/restart.log
function float_to_int() {
echo $1 | cut -d. -f1
}
check=$(uptime | awk -F' *,? *' '{print $12}')
now=$(date)
checkk=$(float_to_int $check)
if [[ $checkk > 40 ]]; then
echo $now $checkk >> $LOGFILE 2>&1
/usr/bin/systemctl restart httpd.service
fi
If I look at the log file I see the following:
Wed Jul 3 20:02:01 EDT 2019 70
Wed Jul 3 23:03:01 EDT 2019 43
Wed Jul 3 23:12:01 EDT 2019 9
Wed Jul 3 23:13:01 EDT 2019 7
Wed Jul 3 23:14:01 EDT 2019 6
Wed Jul 3 23:15:02 EDT 2019 5
Wed Jul 3 23:16:01 EDT 2019 5
Something is clearly wrong as it should only log and restart Apache if the load is over 40 but as you can see from the logs the load was 9, 7, 6, 5 and 5. Could someone point me in the right direction?
From man bash, section CONDITIONAL EXPRESSIONS (emphasis mine) :
string1 > string2
True if string1 sorts after string2 lexicographically.
You will either want to use [['s -gt operator, or use arithmetic evaluation instead of [[ :
if (( chekk > 40 )); then
Here's one in GNU awk (GNU awk due to strftime()):
awk '
$1 > 0.4 { # interval above 0.4
logfile="./log.txt" # my logpath, change it
print strftime("%c"), $1 >> logfile # date and load to log
cmd="/usr/bin/systemctl restart httpd.service" # command to use for restarting
if((ret=(cmd|getline res)) !=0 ) # store return value and result
print "failed: " ret # if failed
else
print "success"
}' /proc/loadavg # getting load avg from /proc
I want to create a simple command which logs the date/time, error and stdout to a file. So if I move a file to a folder and it says "Permission denied", I get a line in my file which shows the current date/time and also the error.
I know how to write the stdout and the error of a command to a file but how do I add a time? Thanks in advance!
Here's some code that can help:
function add_date() {
while IFS= read -r line; do
echo "$(date): $line"
done
}
{
# Your code here
} 2>&1 | add_date >> $LOGFILE
This will add the date to the beginning of every line output by your code ( between the braces anyway).
There may be some issues with output buffering. This will appear as the same timestamp on all the lines in your logfile.
Here's an example of the code above applied:
: ${LOGFILE:=logfile}
function add_date() {
while IFS= read -r line; do
echo "$(date): $line"
done
}
{
for a in {1..10}
do
echo $a
sleep 2
done
} 2>&1 | add_date >> $LOGFILE
And the results:
$ cat logfile
Thu Mar 28 14:50:46 EDT 2019: 1
Thu Mar 28 14:50:48 EDT 2019: 2
Thu Mar 28 14:50:50 EDT 2019: 3
Thu Mar 28 14:50:52 EDT 2019: 4
Thu Mar 28 14:50:54 EDT 2019: 5
Thu Mar 28 14:50:56 EDT 2019: 6
Thu Mar 28 14:50:58 EDT 2019: 7
Thu Mar 28 14:51:00 EDT 2019: 8
Thu Mar 28 14:51:02 EDT 2019: 9
Thu Mar 28 14:51:04 EDT 2019: 10
I use a function to do something like what you're asking for
status_msg () {
echo "`hostname --short`:`date '+%m_%d_%Y_%H_%M_%S'`: $*"
}
The above function can be called via
status_msg "This is a test line"
Which would result in
hostname:03_28_2019_13_50_31: This is a test line
Or if you're running a command which produces output, you can use it like so ..
<command> 2>&1 | while read -r line
do
status_msg $line
done
You can redirect stderr to stdout like so:
2>&1
then add the time before each line of either like so:
sed "s/\(.\)/`date` \1/"
so we wind up with something like this
2>&1 | sed "s/\(.\)/`date` \1/"
okay so i run an openssl command to get the date of an expired script. Doing so gives me this:
enddate=Jun 26 23:59:59 2012 GMT
Then i cut everything out and just leave the month which is "Jun"
Now the next part of my script is to tell the user if the the certificate is expired or not and to do that i use an if statement in which it looks like this:
if [ $exp_year -lt $cur_year && $exp_month -lt $cur_month ]; then
echo ""
echo "Certificate is still valid until $exp_date"
echo ""
else
echo ""
echo "Certificate has expired on $exp_date, please renew."
echo ""
fi
I can't figure out how to convert the month into an integer to even do the comparison.
I thought of doing the brute force way which is this:
Jan=01
Feb=02
Mar=03
...
Clearly that's a terrible way to do it. Does anyone know what i can do?
well, you can use:
now=$(date +%s)
cert=$(date --date="$enddate" +%s)
if [ $cert -lt $now ]; then
echo "Old!"
fi
i.e. convert the date into the seconds past the epoch and compare those
I would recommend using Petesh's answer, but here's a way to set up an associative array if you have Bash 4:
months=(Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec)
declare -A mlookup
for monthnum in ${!months[#]}
do
mlookup[${months[monthnum]]=$((monthnum + 1))
done
echo "${mlookup["Jun"]}" # outputs 6
If you have Bash before version 4, you can use AWK to help you out:
month=Feb
awk -v "month=$month" 'BEGIN {months = "Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"; print (index(months, month) + 3) / 4}'
Another way in pure Bash (any version):
months="Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec"
month=Aug
string="${months%$month*}"
echo "$((${#string}/4 + 1))"