The goal of code is that, I want to make a random tcp traffic using iperf and capture it over 5, 10, 15, 20 seconds using tcpdump. In addition, capturing the throughput is also important for me. My problem is that, I would like to execute code1, code2, code3 and code4 for 5, 10, 15 and 20 seconds in bash. However I don't know how to put the mentioned condition for it. Here is my code:
for Test_duration in 5 10 15 20
do
echo “Test performing with $Test_duration duration”
sudo tcpdump -G 10 -W 2 -w /tmp/scripttest_$Test_duration -i h1-eth0 &
while true; do
#code1
time2=$(($RANDOM%20+1))&
pksize2=$(($RANDOM%1000+200))&
iperf -c 10.0.0.2 -t $time2 -r -l $pksize2 >> /media/sf_sharedsaeed/throughtput/iperthroughput_host2_$Test_duration.txt &\
#code2
time3=$(($RANDOM%20+1))&
pksize3=$(($RANDOM%1000+200))&
iperf -c 10.0.0.3 -t $time3 -r -l $pksize3 >> /media/sf_sharedsaeed/throughtput/iperthroughput_host3_$Test_duration.txt &\
#code3
time4=$(($RANDOM%20+1))&
pksize4=$(($RANDOM%1000+200))&
iperf -c 10.0.0.4 -t $time4 -r -l $pksize4 >> /media/sf_sharedsaeed/throughtput/iperthroughput_host4_$Test_duration.txt &\
#code4
time5=$(($RANDOM%20+1))&
pksize5=$(($RANDOM%1000+200))&
iperf -c 10.0.0.5 -t $time5 -r -l $pksize5 >> /media/sf_sharedsaeed/throughtput/iperthroughput_host5_$Test_duration.txt &\
done
done
Another constraint is that, code1, code2, code3 and code4 should be executed at the same time so, I used &.
Please help me what should I replace instead of while true; to have periodic execution of codes. Can any body help me?
You could do that by using a background subshell that creates a simple file lock on expiration that you detect from your while loops. Here is an example based on a simplified version of your code:
for Test_duration in 5 10 15 20
do
# TIMEOUT_LOCK will be your file lock
rm -f TIMEOUT_LOCK
# next command will run in a parallel subshell at the background
(sleep $Test_duration; touch TIMEOUT_LOCK) &
echo “Test performing with $Test_duration duration”
while true; do
# check whether current timeout (5 or 10 or 15 ...) has occured
if [ -f TIMEOUT_LOCK ]; then rm -f TIMEOUT_LOCK; break; fi
# do your stuff here - I'm just outputing dots and sleeping
echo -n "."
sleep 1
done
echo ""
done
The output of this code is:
“Test performing with 5 duration”
.....
“Test performing with 10 duration”
..........
“Test performing with 15 duration”
...............
“Test performing with 20 duration”
....................
Related
I'm attempting to use a simple bash script to sequentially run a batch of MPI jobs. This script works perfectly when running serial code (I am using Fortran 90), but for some reason bash breaks out of the loop when I attempt to execute MPI code.
I already found a work-around to the problem. I just wrote essentially the exact same script in Perl and it worked like a charm. I just really want to understand the issue here because I prefer the simplicity of bash and it perfectly fits my own scripting needs in almost all other cases.
I've tried running the MPI code as a background process and using wait with the same result. If I run the jobs in the background without using wait, bash does not break out of the loop, but it stacks up jobs until eventually crashing. The goal is to run the executable sequentially for each parameter set anyway, I just wanted to note that the loop is not broken in that case.
Bash Script, interp.sh: Usage --> $ ./interp.sh inputfile
#!/bin/bash
PROG=$1
IFILE=$2
kount=0 # Counter variable for looping through input file
sys=0 # Counter variable to store how many times model has been run
while IFS="\n" read -r line
do
kount=$(( $kount + 1 ))
if [ $(( kount % 2 )) -eq 1 ] # if kount is even, then expect headers
then
unset name defs
sys=$(( $sys + 1 ))
name=( $line ) # parse headers
defs=${#name[*]}
k=$(( $defs - 1 ))
else # if count is odd, then expect numbers
unset vals
vals=( $line ) # parse parameters
for i in $( seq 0 $k )
do
# Define variables using header names and set their values
printf -v "${name[i]}" "${vals[i]}"
done
# Print input variable values
echo $a $b $c $d $e $nPROC
# Run executable
mpiexec -np $nPROC --oversubscribe --hostfile my_hostfile $PROG
fi
done < $IFILE
Input file, input.dat:
a b c d e nPROC
1 2 3 4 5 2
nPROC
3
nPROC
4
nPROC
5
nPROC
6
nPROC
7
nPROC
8
Sample MPI f90 code, main.f90:
program main
use mpi
implicit none
integer :: i, ierr, myID, nPROC
integer, parameter :: foolen = 100000
double precision, dimension(0:foolen) :: foo
call MPI_INIT(ierr)
call MPI_COMM_SIZE(MPI_COMM_WORLD, nPROC, ierr)
call MPI_COMM_RANK(MPI_COMM_WORLD, myID, ierr)
if ( myID .eq. 0 ) then
do i=0,foolen
foo(i) = i
end do
else
do i=0,foolen
foo(i) = i
end do
end if
call MPI_FINALIZE(ierr)
end program
Sample makefile:
COMP=mpif90
EXT=f90
CFLAGs=-Wall -Wextra -Wimplicit-interface -fPIC -fmax-errors=1 -g -fcheck=all \
-fbacktrace
MPIflags=--oversubscribe --hostfile my_hostfile
PROG=main.x
INPUT=input.dat
OUTPUT=output
OBJS=main.o
$(PROG): $(OBJS)
$(COMP) $(CFLAGS) -o $(PROG) $(OBJS) $(LFLAGS)
main.o: main.f90
$(COMP) -c $(CFLAGs) main.f90
%.o: %.f90
$(COMP) -c $(CFLAGs) $<
run:
make && make clean
./interp.sh $(PROG) $(INPUT)
clean:
rm -f *.o DONE watch
my_hostfile
localhost slots=4
Note that if the mpiexec line is commented out, the script runs as expected. The output looks like this:
1 2 3 4 5 2
1 2 3 4 5 3
1 2 3 4 5 4
1 2 3 4 5 5
1 2 3 4 5 6
1 2 3 4 5 7
1 2 3 4 5 8
These are the parameter values which are supposed to be passed to the MPI code in each loop. However, when mpiexec is called in the script, only the first set of parameters is read and passed.
I apologize if all that is a bit excessive, I just wanted to provide all that is needed for testing. Any help solving the issue in bash or explanation of why this happens would be greatly appreciated!
mpiexec is consuming the stdin thus reading all remaining lines in the loop. So after the first loop stdin is empty and the loop breaks.
This is an issue that occurs not only with loops calling mpiexec from whithin but also with loops other commands that consumes stdin by default such as ssh.
The general solution is to use < /dev/null so that the offending command won't consume stdin but the /dev/null instead. Some commands have special flags to replace the redirect command such as ssh -n
so the solution in this case would be to add the redirect at the end of the line where mpiexec is called:
mpiexec -np $nPROC --oversubscribe --hostfile my_hostfile $PROG < /dev/null
there are some issues to pay attention to in the case of mpiexec related to Standard I/O detailed here: https://www.open-mpi.org/doc/v3.0/man1/mpiexec.1.php#toc14
I have a bash script that creates a file with a timestamp as the name. Once some time passes, it is supposed to pick up that file and do something with it. I want it to pick it up after two hours, but for some reason, it is picking it up after 57 minutes (and 6 seconds). Can anyone point me to an error in my logic or assumptions?
Here are the details:
I have a variable set to 2 hours (7200 seconds):
SERVICE_DURATION=${SERVICE_DURATION:-7200} # seconds
I am setting the filename equal to the Unix timestamp concatenated with nanoseconds:
active_name=`date +%s%N`
echo "${1}" >> ${ACTIVE_DIR}/${active_name}
I then loop forever until the time is right:
while true
do
for fa in ${ACTIVE_DIR}/*
do
if [ $(basename ${fa}) -le $(($(date +%s%N) - ${SERVICE_DURATION} * 1000000000)) ]
then
exec 6< "${fa}"
read old_port <&6
read old_host <&6
read old_config <&6
exec 6<&-
logger -p daemon.info "Recycling port ${old_port}."
start_remote_service "${old_port}"
stop_remote_service "${old_port}" "${old_host}" "${old_config}" "${fa}"
sleep 2
fi
done
sleep 30
done
I can't see what is wrong with this. The filename ($(basename ${fa})) shouldn't be less than the current time minus the specified duration in nanoseconds ($(($(date +%s%N) - ${SERVICE_DURATION} * 1000000000))) until the duration has passed.
In order to keep the script from constantly checking, there is a sleep 30 at the end of the loop, so it could be that the time is somewhere between 56:36 and 57:06.
Any help would be appreciated. Thanks.
I have a bash script called sr_run_batch.sh which does super resolution of images. Now I want to do testing on different servers in parallel at the same time. ie. 1 Virtual machine at one given point of time. then 2 virtual machines at one point of time , 3 and then 4.
I tried writing into it the commands
for host in $(cat hosts.txt); do ssh "$host" "$command" >"output.$host"; done
ssh-keygen && for host in $(cat hosts.txt); do ssh-copy-id $host; done
where the file hosts.txt contains the list of servers: username#ip(format) but when I run this, it gives me substitution error
Hence, I tried pssh (parallel-ssh)
pssh -h hosts-file -l username -P $command
command being ./sr_run_batch.sh
but it didn't run, so I modified this to
pssh -h hosts-file -l ben -P -I<./sr_run_batch.sh
But, for some unknown reason, it just prints the echo statements in the code.
here is the code :
NList=(5)
VList=(1)
FList=("input/flower1.jpg" "input/flower2.jpg" "input/flower3.jpg" "input/flower4.jpg")
IList=("320X240" "640X480" "1280X960" "1920X1200")
SList=(2 3)
for VM in ${VList[#]}; do
for ((index=0; index < ${#FList};)) do
file=$FList[$index]
image_size=$IList[$index]
width=`echo $image_size|cut -d "X" -f1`
height=`echo $image_size|cut -d "X" -f2`
for scale_factor in ${SList[#]}; do
for users in ${NList[#]}; do
echo "V: $VM, " "F: $file, " "S: $scale_factor, " "I: $width $height , " "N: $users"
for i in `seq 1 $users` ; do
./sr_run_once.sh $file $width $height $scale_factor &
done
wait
done # for users
done # for scale_factor
done # for index
done # for VM
exit 0
Have you also tried to use pssh with a simple bash-script so see if the communication is set up ok?
$ pssh -h hosts.txt -A -l ben -P -I<./uptime.sh
Warning: do not enter your password if anyone else has superuser
privileges or access to your account.
Password:
10.0.0.67: 11:06:50 up 28 min, 2 users, load average: 0.00, 0.00, 0.00
[1] 11:06:50 [SUCCESS] 10.0.0.67
10.0.0.218: 11:06:50 up 24 min, 2 users, load average: 0.00, 0.05, 0.20
[2] 11:06:50 [SUCCESS] 10.0.0.218
I've developed a bash script to do some network scanning. However, I'd like to improve the functionality by making it do 2 scans at once. My code looks like this:
for d in `seq 0 255` ;
do
echo 17.6.3.$d
sudo nmap -pT:80,22 17.4.1.$d
done
So at the moment it scans all IP addresses in the subnet /24, 1 by 1 and in order. I'd like some help to make it scan in pairs, so 17.6.3.1 and 17.6.3.2 at the same time, then 17.6.3.3 and 17.6.3.4 and so on until 255. Thanks.
From bash hackers wiki:
for((x = 0; x < 256; x += 2)); do
echo 17.6.3.$x
echo 17.6.3.$((x+1))
# etc.
done
This will do the same:
seq 0 255 | parallel -j2 echo 17.6.3.{}\; sudo nmap -pT:80,22 17.4.1.{}
Personally I would do this:
parallel -j200 --tag sudo nmap -pT:80,22 ::: 17.4.1.{1..254}
In command below I enable file /dev/tcp/10.10.10.1/80 both for reading and writing and associate it with file descriptor 3:
$ time exec 3<>/dev/tcp/10.10.10.1/80
bash: connect: Operation timed out
bash: /dev/tcp/10.10.10.1/80: Operation timed out
real 1m15.151s
user 0m0.000s
sys 0m0.000s
This automatically tries to perform TCP three-way handshake. If 10.10.10.1 is not reachable as in example above, then connect system call tries to connect for 75 seconds. Is this 75 second timeout determined by bash? Or is this system default? Last but not least, is there a way to decrease this timeout value?
It's not possible in Bash without modifying the source as already mentioned, although here is the workaround by using timeout command, e.g.:
$ timeout 1 bash -c "</dev/tcp/stackoverflow.com/80" && echo Port open. || echo Port closed.
Port open.
$ timeout 1 bash -c "</dev/tcp/stackoverflow.com/81" && echo Port open. || echo Port closed.
Port closed.
Using this syntax, the timeout command will kill the process after the given time.
See: timeout --help for more options.
It is determined by TCP. It can be decreased on a per-socket basis by application code.
NB The timeout only takes effect if there is no response at all. If there is a connection refusal, the error occurs immediately.
No: there is no way of changing timeout by using /dev/tcp/
Yes, you could change default timeout for TCP connection in any programming language.
But, bash is not a programming language!
You could have a look into source code (see: Bash Homepage), you may find lib/sh/netopen.c file where you could read in _netopen4 function:
s = socket(AF_INET, (typ == 't') ? SOCK_STREAM : SOCK_DGRAM, 0);
You could read this file carefully, there are no consideration of connection timeout.
Without patching bash sources, there is no way of changing connection timeout by a bash script.
Simple HTTP client using netcat (near pure bash)
There is a little sample HTTP client written in pure bash, but using netcat:
#!/bin/bash
tmpfile=$(mktemp -p $HOME .netbash-XXXXXX)
exec 7> >(nc -w 3 -q 0 stackoverflow.com 80 >$tmpfile)
exec 6<$tmpfile
rm $tmpfile
printf >&7 "GET %s HTTP/1.0\r\nHost: stackoverflow.com\r\n\r\n" \
/questions/24317341/how-to-decrease-tcp-connect-system-call-timeout
timeout=100;
while ! read -t .001 -u 6 status ; do read -t .001 foo;done
echo STATUS: $status
[ "$status" ] && [ -z "${status//HTTP*200 OK*}" ] || exit 1
echo HEADER:
while read -u 6 -a head && [ "${head//$'\r'}" ]; do
printf "%-20s : %s\n" ${head%:} "${head[*]:1}"
done
echo TITLE:
sed '/<title>/s/<[^>]*>//gp;d' <&6
exec 7>&-
exec 6<&-
This could render:
STATUS: HTTP/1.1 200 OK
HEADER:
Cache-Control : private
Content-Type : text/html; charset=utf-8
X-Frame-Options : SAMEORIGIN
X-Request-Guid : 46d55dc9-f7fe-425f-a560-fc49d885a5e5
Content-Length : 91642
Accept-Ranges : bytes
Date : Wed, 19 Oct 2016 13:24:35 GMT
Via : 1.1 varnish
Age : 0
Connection : close
X-Served-By : cache-fra1243-FRA
X-Cache : MISS
X-Cache-Hits : 0
X-Timer : S1476883475.343528,VS0,VE100
X-DNS-Prefetch-Control : off
Set-Cookie : prov=ff1129e3-7de5-9375-58ee-5f739eb73449; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
TITLE:
bash - How to decrease TCP connect() system call timeout? - Stack Overflow
Some explanations:
We create first a temporary file (under private directory for security reason), bind and delete before using them.
$ tmpfile=$(mktemp -p $HOME .netbash-XXXXXX)
$ exec 7> >(nc -w 3 -q 0 stackoverflow.com 80 >$tmpfile)
$ exec 6<$tmpfile
$ rm $tmpfile
$ ls $tmpfile
ls: cannot access /home/user/.netbash-rKvpZW: No such file or directory
$ ls -l /proc/self/fd
lrwx------ 1 user user 64 Oct 19 15:20 0 -> /dev/pts/1
lrwx------ 1 user user 64 Oct 19 15:20 1 -> /dev/pts/1
lrwx------ 1 user user 64 Oct 19 15:20 2 -> /dev/pts/1
lr-x------ 1 user user 64 Oct 19 15:20 3 -> /proc/30237/fd
lr-x------ 1 user user 64 Oct 19 15:20 6 -> /home/user/.netbash-rKvpZW (deleted)
l-wx------ 1 user user 64 Oct 19 15:20 7 -> pipe:[2097453]
$ echo GET / HTTP/1.0$'\r\n\r' >&7
$ read -u 6 foo
$ echo $foo
HTTP/1.1 500 Domain Not Found
$ exec 7>&-
$ exec 6>&-