Best timing method? - bash

I'm trying to write an SSH version of ICMP Ping, as follows:
TARGET_IP=""
count=0
time_start=0
time_end=0
time_taken=0
TARGET_IP=$1
while [ $count -lt 5 ]
do
((count=count+1))
time_start=$(date +%s%N)
temp_target_key=$(ssh-keyscan $TARGET_IP) > /dev/null 2>&1
time_end=$(date +%s%N)
time_taken=$((( $time_end - $time_start ) / 1000000))
echo "Time taken=$time_taken ms."
target_key=$(echo $temp_target_key | awk '{print $3}') > /dev/null 2>&1
echo $temp_target_key > target_key.txt
ssh-keygen -l -f target_key.txt > /dev/null 2>&1
test=$?
if [ $test -ne 0 ]
then
echo "Device returned invalid RSA Public Key"
echo -e "\n"
echo -e "\n"
else
echo "Device responding correctly."
echo -e "\n"
fi
done
exit
ICMP Ping reports ping times of 5ms compared to this script reporting 300ms for the same device. I realise that I'm timing the script AND device programming/firmware response times too, but am I doing this the best way please?
Thanks

I can't see a better method to simulate a ssh ping. Nevertheless I've tried to improve your script in a more pure bash style:
#! /bin/bash
if (($# != 1)); then
echo "Usage: ${0##*/} host" >&2
exit 1
fi
TARGET_IP="$1"
target_key_file="target_key.txt"
for ((count=0; count < 5; count++))
do
time_start=$(date +%s%N)
temp_target_key=$(ssh-keyscan "$TARGET_IP") > /dev/null 2>&1
time_end=$(date +%s%N)
time_taken=$(((time_end - time_start ) / 1000000))
echo "Time taken=$time_taken ms."
read _ _ target_key _ <<< "$temp_target_key"
echo "$target_key"
echo "$temp_target_key" >| "$target_key_file"
if ! ssh-keygen -l -f "$target_key_file" > /dev/null 2>&1
then echo -e "Device returned invalid RSA Public Key.\n\n"
else echo -e "Device responding correctly.\n\n"
fi
done

Related

I'm trying to find a alternative for timeout in bash 3

I have tried a method using timeout to identify and exit server comes to RUNNING state or else exit watching for RUNNING state at the timeout value, unfortunately one of the server has bash version 3 and doesn't support time out.
timeout 300s grep -q 'Server state changed to RUNNING' <(tail -f AdminServer.out)
if [ $? != 0];then
printf "\n===> unable to bring up the server.please check\n"
else
printf "\n===> server came to RUNNING state\n"
fi
I have tried the following but wasn't able to exit when the log is appeared
( cmdpid=$BASHPID; (sleep 60; kill $cmdpid) & exec `grep -q 'Server state changed to RUNNING' <(tail -f AdminServer.out)`)
but it first give command line substitution error for grep -q 'Server state changed to RUNNING' <(tail -f AdminServer.out) and then gives error for kill usage like below(seems the kill is not working) ,
kill: usage: kill [-s sigspec | etc....
I'm not in a position to upgrade the bash version of the server unfortunately, any guidance for this highly appreciated .Thank you!!
*server: suse distribution 2.6.32.12-0.7-default
Consider to try something like this:
data=$(
tail -f AdminServer.out & pid=$!
till=$[SECONDS+300]
while ((SECONDS<till)); do :; done
kill -9 $pid
)
grep -q 'Server state changed to RUNNING' <<< "$data" \
&& printf "\n===> server came to RUNNING state\n" \
|| printf "\n===> unable to bring up the server.please check\n"
When I was young... on OSF1, DEC Alpha, SPARC...
without timeout command, without <(...) or <<< operators...
I'm actively waiting...
File: timeout.sh
#! /bin/bash
PID="$1"
SEC="$2"
FILE="$3"
DATE_REF=$(date +%s)
DATE_MAX=$(( DATE_REF + ${SEC} ))
while ps -p "${PID}" >/dev/null 2>&1 && [[ $(date +%s) -lt ${DATE_MAX} ]] && ! grep OK "${FILE}" >/dev/null 2>&1; do
sleep 1
done
if grep OK "${FILE}" >/dev/null 2>&1; then
RET=0
else
RET=1
fi
if ps -p "${PID}" >/dev/null 2>&1; then
kill -9 "${PID}"
fi
rm -f "${FILE}"
exit $RET
Used like that:
nb_seconds_to_wait=5
tail -f my_log_file | grep -m 1 my_token_to_search && echo "OK" > my_flag_file &
timeout.sh $! nb_seconds_to_wait my_flag_file
if [[ $? -eq 0 ]]; then
echo "Found"
else
echo "Not found"
fi
I have tried again and came up with the following, am I doing anything harmful here?
( cmdpid=$BASHPID; ( sleep 10; if ps -p $cmdpid > /dev/null; then kill $cmdpid > /dev/null;fi ) & while `tail -f -n0 AdminServer.out | grep -q RUNNING`; do exit 0;done )
echo "proceeding with the rest of commands the previous command status is $?"

BASH - Timed Input - Show countdown

I have a bash script that asks the user for their details.
I'm setting a limit to how long we wait for the input. I've found this and it appears to what I want.
timelimit=5
echo -e " You have $timelimit seconds\n Enter your name quickly: \c"
name=""
read -t $timelimit name
#read -t $timelimit name <&1
# for bash versions bellow 3.x
if [ ! -z "$name" ]
then
echo -e "\n Your name is $name"
else
echo -e "\n TIME OUT\n You failed to enter your name"
fi
It shows "You have 5 seconds..." any way to update the output so it shows 4,3,2,1 etc as it counts down ?
Thanks
I have tried most of these answers and none of them worked perfectly for me.
Been playing with this for a local developer deployment script.
This solves a few of the issues noted, like including printed output, etc.
Also wrapped as a function for portability. I'm keen to see any improvements.
Here is my solution:
#!/bin/bash
# set -euo pipefail
READTIMEOUT=5
function read_yn {
MESSAGE=$1
TIMEOUTREPLY=$2
NORMALREPLY="Y"
if [ -z "${TIMEOUTREPLY}" ]; then
TIMEOUTREPLY="Y"
fi
TIMEOUTREPLY_UC=$( echo $TIMEOUTREPLY | awk '{print toupper($0)}' )
TIMEOUTREPLY_LC=$( echo $TIMEOUTREPLY | awk '{print tolower($0)}' )
if [ "${TIMEOUTREPLY_UC}" == "Y" ]; then
NORMALREPLY="N"
fi
NORMALREPLY_UC=$( echo $NORMALREPLY | awk '{print toupper($0)}' )
NORMALREPLY_LC=$( echo $NORMALREPLY | awk '{print tolower($0)}' )
for (( i=$READTIMEOUT; i>=0; i--)); do
printf "\r${MESSAGE} [${NORMALREPLY_UC}${NORMALREPLY_LC}/${TIMEOUTREPLY_UC}${TIMEOUTREPLY_LC}] ('${TIMEOUTREPLY_UC}' in ${i}s) "
read -s -n 1 -t 1 waitreadyn
if [ $? -eq 0 ]
then
break
fi
done
yn=""
if [ -z $waitreadyn ]; then
echo -e "\nNo input entered: Defaulting to '${TIMEOUTREPLY_UC}'"
yn="${TIMEOUTREPLY_UC}"
else
echo -e "\n${waitreadyn}"
yn="${waitreadyn}"
fi
}
read_yn "TESTING" "y"
GIST: https://gist.github.com/djravine/7a66478c37974940e8c39764d59d35fa
LIVE DEMO: https://repl.it/#DJRavine/read-input-with-visible-countdownsh
This should work and shouldn't overwrite input, bit more long winded than the other solutions.
#!/bin/bash
abend()
{
stty sane
exit
#Resets stty and then exits script
}
DoAction(){
stty -echo
#Turn off echo
tput sc
#Save cursor position
echo -ne "\033[0K\r"
# Remove previous line
tput cuu1
#Go to previous line
tput el
#clear to end of line
echo "You have $(($time-$count)) seconds"
#Echo timer
echo -n "$Keys"
#Echo currently typed text
stty echo
#turn echo on
tput rc
#return cursor
}
main()
{
trap abend SIGINT # Trap ctrl-c to return terminal to normal
stty -icanon time 0 min 0 -echo
#turn of echo and set read time to nothing
keypress=''
time=5
echo "You have $time seconds"
while Keys=$Keys$keypress; do
sleep 0.05
read keypress && break
((clock = clock + 1 ))
if [[ clock -eq 20 ]];then
((count++))
clock=0
DoAction $Keys
fi
[[ $count -eq $time ]] && echo "you have run out of time" && abend
done
stty sane
echo Your username was $Keys
echo "Thanks for using this script."
exit 0
}
main
This seems to work:
$ cat test.sh
total=5 # total wait time in seconds
count=0 # counter
while [ ${count} -lt ${total} ] ; do
tlimit=$(( $total - $count ))
echo -e "\rYou have ${tlimit} seconds to enter your name: \c"
read -t 1 name
test ! -z "$name" && { break ; }
count=$((count+1))
done
if [ ! -z "$name" ] ; then
echo -e "\nyour name is $name"
else
echo -e "\ntime out"
fi
#!/bin/bash
timelimit=6
name=""
for (( i = 1 ; i <= $timelimit; i++ )); do
echo -ne "\rYou have $(expr $timelimit - $i) seconds. Enter your name quickly: \c"
[ ! -z "$name" ] && { break ; }
read -t 1 name
done
if [ -z "$name" ]; then
echo -e "\n TIME OUT\n You failed to enter your name"
else
echo -e "\n Your name is $name"
fi
this should work
This works fine and fast for me:
#!/bin/bash
#Sets starttimestamp
starttime=$(date +%s)
#Sets timeout
timeout=5
#sets successflag default to false
success=false
#Save Cursorposition
echo -n -e "\033[s"
#While time not up
while [ $(($starttime+$timeout)) -gt $(date +%s) ] ; do
#Return to saved Cursorpositon
echo -n -e "\033[u"
#Display time left
echo "$(((starttime+timeout)-$(date +%s))) seconds left"
#Ask for 1 char then go on in loop make it look like an ongoing input by adding the user variable to the prompt
if read -p foo="Username: $user" -n 1 -t 1 c ; then
#If user hits return in time c will be empty then break out of loop and set success true
if [[ $c == "" ]] ; then
success=true
break
fi
# Append latest character to user variable
user=${user}${c}
unset c
fi
done
if $success ; then
echo "Yiha!"
else
echo "Too late!"
fi

BASH Keep counter through reboots

I'm trying to keep a external counter in my bash script.
As most of the script is specific to what it's for, I'm only going to insert the pertinent parts that I can't seem to get working. Everything before does get defined. Also before someone comments about the cat, I tried (< $TMPFILE) as well. The issue appears to be not writing to the file.
#!/bin/bash
TMPFILE="/tmp/nettestnum"
if [ -e "$TMPFILE" ]; then I=$(cat $TMPFILE); else I="1"; fi
nettest_start() {
grep -q "/root/bin/nettest" /etc/rc.local
if [ $? -eq 1 ]; then
sed -i 's|touch /var/lock/subsys/local|touch /var/lock/subsys/local \&\& /root/bin/nettest|' /etc/rc.local 2>&-
echo "Script started at $DATEVAL" >> "$LOGFILE" 2>&1
echo "Script will run every $TIMELb minutes, $LIMIT times"
touch "$TMPFILE"
nettest_run
else
echo "Nettest is already started and will relaunch upon next reboot"
fi
}
nettest_run() {
echo "$DATE : Starting scan" >> "$LOGFILE" 2>&1
#
while [[ "$I" -lt "$LIMIT" ]]
do
echo "$DATE : Starting scan" >> "$LOGFILE" 2>&1
#
HOSTS="$(...)"
for myHost in $HOSTS
do
PING=$(ping -f -c $COUNT "$myHost" |grep 'rtt' | awk '{print $4}')
echo "$myHost / $PING" >> "$LOGFILE" 2>&1
done
echo "$DATEVAL : Finished" >> "$LOGFILE" 2>&1
echo "$DATEVAL : Netstat Start" >> "$STATLOG" 2>&1
netstat -s |egrep -i "dropped|loss|reject|timeout|invalid" >> "$STATLOG" 2>&1
echo "$DATEVAL : Netstat Finished" >> "$STATLOG"G 2>&1
echo "." >> "$LOGFILE" 2>&1
sleep "$TIMEL"
let I++ || true
set +C
echo "$I" > "$TMPFILE"
done
}
Edit 4: re-added } and showing both functions.
I also need to figure out how to write an exit for the loop, but I'm pretty sure that's just a [ $? = 0 ]
Edit:
An example of some of the output I'm seeing when I run a status command (built into the script)..
Nettest is enabled on startup
PID number: 6059
Nettest is actively running in memory
On iteration of 100
iteration shows 1 on the first run.
Edit 2: redid the sample with full example, minus customized commands.
In the end, I got it working by adding 2>&1 to the line needed.
echo "$I" > "$TMPFILE" 2>&1
Try this:
if [ -e "$TMPFILE" ]; then I=`cat $TMPFILE`; else I=1; fi
while [[ $I -lt $LIMIT ]]
A while loop creates a subshell, with a copy of all your current variables.
#!/bin/bash
I=1
CHECK=1 # boolean for demonstration
while CHECK; do
let I++
CHECK=0
done
echo $I # prints 1, not two
This happens because the while subshell gets a copy of the I variable, which it increments. Once the subshell exits, the original variable comes back.
Your I variable is being incremented, and acting as your loop counter. I expect this is causing you all kinds of problems for properly ending your loop, and having other bits of this program see the "right" variable. I see no reason you could not replace the while loop with an equivalent for loop, and avoid all of these problems entirely.

bash: running cURLs in parallel slower than one after another

We have to cache quite a big database of data after each upload, so we created a bash script that should handle it for us. The script should start 4 paralel curls to the site and once they're done, start the next one from the URL list we store in the file.
In theory everything works ok, and the concept works if we run the run 4 processes from our local machines to the target site.
If i set the MAX_NPROC=1 the curl takes as long as it would if the browser hits the URL
i.e. 20s
If I set the MAX_NPROC=2 the time request took, triples.
Am I missing something? Is that an apache setting that is slowing us down? or is this a secret cURL setting that I'm missing?
Any help will be appreciated. Please find the bash script below
#!/bin/bash
if [[ -z $2 ]]; then
MAX_NPROC=4 # default
else
MAX_NPROC=$2
fi
if [[ -z $1 ]]; then
echo "File with URLs is missing"
exit
fi;
NUM=0
QUEUE=""
DATA=""
URL=""
declare -a URL_ARRAY
declare -a TIME_ARRAY
ERROR_LOG=""
function queue {
QUEUE="$QUEUE $1"
NUM=$(($NUM+1))
}
function regeneratequeue {
OLDREQUEUE=$QUEUE
echo "OLDREQUEUE:$OLDREQUEUE"
QUEUE=""
NUM=0
for PID in $OLDREQUEUE
do
process_count=`ps ax | awk '{print $1 }' | grep -c "^${PID}$"`
if [ $process_count -eq 1 ] ; then
QUEUE="$QUEUE $PID"
NUM=$(($NUM+1))
fi
done
}
function checkqueue {
OLDCHQUEUE=$QUEUE
for PID in $OLDCHQUEUE
do
process_count=`ps ax | awk '{print $1 }' | grep -c "^${PID}$"`
if [ $process_count -eq 0 ] ; then
wait $PID
my_status=$?
if [[ $my_status -ne 0 ]]
then
echo "`date` $my_status ${URL_ARRAY[$PID]}" >> $ERROR_LOG
fi
current_time=`date +%s`
old_time=${TIME_ARRAY[$PID]}
time_difference=$(expr $current_time - $old_time)
echo "`date` ${URL_ARRAY[$PID]} END ($time_difference seconds)" >> $REVERSE_LOG
#unset TIME_ARRAY[$PID]
#unset URL_ARRAY[$PID]
regeneratequeue # at least one PID has finished
break
fi
done
}
REVERSE_LOG="$1.rvrs"
ERROR_LOG="$1.error"
echo "Cache STARTED at `date`" > $REVERSE_LOG
echo "" > ERROR_LOG
while read line; do
# create the command to be run
DATA="username=user#server.com&password=password"
URL=$line
CMD=$(curl --data "${DATA}" -s -o /dev/null --url "${URL}")
echo "Command: ${CMD}"
# Run the command
$CMD &
# Get PID for process
PID=$!
queue $PID;
URL_ARRAY[$PID]=$URL;
TIME_ARRAY[$PID]=`date +%s`
while [ $NUM -ge $MAX_NPROC ]; do
checkqueue
sleep 0.4
done
done < $1
echo "Cache FINISHED at `date`" >> $REVERSE_LOG
exit
The network is almost always the bottleneck. Spawning more connections usually makes it slower.
You can try to see if parallel'izing it will do you any good by spawning several
time curl ...... &

Validate Emails with Bash

I try to validate an email with a shell script. Is there a easy example to validate a mail? Asked google but just found crap and PHP (also crap..).
Thanks and regards.
If you explicitly state bash on the #! line, you can uses regexes:
#!/bin/bash
if [[ $email =~ '(.+)#(.+)' ]] ; then
user=${BASH_REMATCH[1]}
host=${BASH_REMATCH[2]}
fi
If you are serious about programing with bash, read the man page all the way through. Twice. Also read the Bash FAQ.
Validate script.
The meaning of this script, is to be sure from all emails that are valid or not before sending them email.
this script check a list of emails.
#!/bin/bash
#:
#: e-mail.verify.sh
#:
#: Date: 2011/14/12 13:14 PM IST
#: Author: Louay _at_ louie.msh#gmail.com
#: Discription: Verify (or) Validate the Hotmail Adresses.
#:
#:
#: First we create a Expect script to be sourced for us.
if [ ! $# == 1 ]
then
echo "Invalid Args $0 Filename"
exit 0
fi
#: Verifying the Hotmail adressess.
#: First verify the network Connections
C_R="\e[01;31m" ## Colors
C_B="\e[01;30m"
C_G="\e[01;32m"
C_END="\e[00m"
SMTPSERV=`host -t mx hotmail.com |grep 5 | grep mx2.hotmail.com |cut -d " " -f 7| sed 's/\.$//'`
ping -c2 $SMTPSERV >/dev/null
if [ "$?" -eq 0 ]
then
echo -e "Internet Connection" "\t\t\t\t\t\t$C_G[ OK ]$C_END"
echo -e "$SMTPSERV is AVAILABLE."
echo -n "Verifing"
for (( i=0; i<5; i++ ))
do
echo -n ".."
sleep 1
done
echo
else
echo -e "Internet Connection:" "\t\t\t\t\t\t$C_R[ FAIL ]$C_END" ""
echo -e "$SMTPSERV is Unavialable."
echo -e "Check your Network settings."
exit 0
fi
COUNT=0
RM_FILE="validemails.txt"
rm -f $RM_FILE
cat $1 | while read LINE; do
{
MAFR="MAIL FROM: <louie.msh#gmail.COM>"
MATO="RCPT TO: <$LINE>"
#: ^variablies declared for not get escaped in the next cat command, where
#: we set the $MAFR in the expect script.
cat << __EOF > e-veri
#!/bin/expect
#:
#: Date: 2011/14/12 01:14 PM
#: Author: Louay Mshelim_at_ louie.msh#gmail.com
#: Discription: Expect Script to Verify/Validate the Hotmail Adresses.
#:
set VMAFR "$MAFR"
set VMATO "$MATO"
spawn nc -C mx4.hotmail.com 25
expect "Sending"
send "HELO mx4.hotmail.com\r"
expect "OK"
send "\$VMAFR\r"
expect "OK"
send "\$VMATO\r"
expect "250"
send "quit\r"
expect eof
__EOF
#: Running the expect script and extracting the Results.txt
expect e-veri > Results.txt
grep 550 Results.txt >/dev/null
if [ "$?" -eq 0 ]
then
echo -e $LINE >> invalid.txt #invalid E-mails
else
echo -e "$LINE" >> validemails.txt
fi
}
done
echo -e "Valid E-mail have been saved to $C_R[ validemails.txt ]$C_END"
#: END
Here is an improved and working version of the script by codevour:
#!/bin/bash
# check for valid usage
if [ x$1 = 'x' ]
then
echo "Usage: $0 <email address>"
exit 1
fi
mailcmd=`mktemp`
# grabbing fields
user=`echo $1 | perl -p -e 's/^([^#]+)#([^\#]+)$/$1/g'`
host=`echo $1 | perl -p -e 's/^([^#]+)#([^\#]+)$/$2/g'`
mxhost=`host -t mx $host|perl -p -e 's/.* ([^ ]+)\.$/$1/g'|sort -R|tail -1`
# compose email commands
echo -ne "helo example.com\r\n" > $mailcmd
echo -ne "mail from: <tester#example.com>\r\n" >> $mailcmd
echo -ne "rcpt to: <$1>\r\n" >> $mailcmd
echo -ne "quit\r\n" >> $mailcmd
# check for mail results
mailresult=`cat $mailcmd | nc $mxhost 25| grep ^550 | wc -c`
if [ $mailresult -eq 0 ]
then
echo $1 "is valid"
exit 0
else
echo $1 "is not valid"
exit 1
fi
# clean up
rm $mailcmd
You mean something like this?
#!/bin/bash
# check for valid usage
if [ x$1 = 'x' ]
then
echo "Usage: $0 <email address>"
exit 1
fi
# grabbing fields
user=`echo $1 | cut -f1 -d\#`
host=`echo $1 | cut -f2 -d\#`
mxhost=`host -t mx $host | cut -f7 -d\ `
len=`echo $mxhost | wc -c`
len=`expr $len - 2`
mxhost=`echo $mxhost | cut -b1 -$len`
# compose email commands
echo -ne "helo test.com\r\n" > mailcmd
echo -ne "mail from: test\#test.com\r\n" >> mailcmd
echo -ne "rcpt to: $1\r\n" >> mailcmd
echo -ne "quit\r\n" >> mailcmd
# check for mail results
mailresult=`cat mailcmd | nc $mxhost 25| grep ^550 | wc -c`
if [ $mailresult -eq 0 ]
then
echo $1 "is valid"
exit 0
else
echo $1 "is not valid"
exit 1
fi
# clean up
rm mailcmd
Found at:
Fun things in Life - Simple Bash Email Validator
My any suggestion for the latter script, check work with multiple servers that accept checks without authentication, using nslookup who knows:
For example code in:
http://www.vivaolinux.com.br/script/Simples-Verificador-de-Email-Gmail
see the code below in the site.
I hope I have contributed and I hope to collaborate as well.
Thank you.
You can use like this,
read emailId
if echo "${emailId}" | grep '^[a-zA-Z0-9]*#[a-zA-Z0-9]*\.[a-zA-Z0-9]*$' >/dev/null; then
echo Valid
else
echo Not Valid
fi

Resources