I have a file system that is shared between several computers. I cannot open network connections between these computers so I have written a program to communicate via files. I have a server that checks a folder for the appearance of instruction files, it reads, executes and writes an output file and then creates a signal file to indicate the file is ready.
to check for a file with a while loop
while [[ ! -e $READY_FILE ]]
do sleep 1
done
do something
The server gets and processes the file pretty much right away and makes the signal file but I am seeing a strange latency on the client side.
when the server and client are running on the same machine the latency is very low. When I run the client on a separate computer the latency is around 30 seconds.
time bash client.sh -f commands.txt
done
real 0m31.945s
user 0m0.048s
sys 0m0.314s
This is reproducible +/- 2 seconds.
I can kill the problem by making the client computer do anything with the working directory.
time bash client.sh -f commands.txt & sleep 5; ls $wd >/dev/null
real 0m5.120s
user 0m0.014s
sys 0m0.083s
time bash client.sh -f commands.txt & sleep 3.5; ls $wd >/dev/null
real 0m3.749s
user 0m0.011s
sys 0m0.055s
I can correct it in the program by changing the while loop to
while [[ ! -e $READY_FILE ]]
do ls $wd >/dev/null
sleep 1
done
Now i get
time bash client.sh -f commands.txt
real 0m1.075s
user 0m0.004s
sys 0m0.056s
My question is why is there a 30 second delay for the test [[ -e $READY_FILE ]] to detect the file?
Related
right now I'm using this script for a program:
export FREESURFER_HOME=$HOME/freesurfer
source $FREESURFER_HOME/SetUpFreeSurfer.sh
cd /home/ubuntu/fastsurfer
datadir=/home/ubuntu/moya/data
fastsurferdir=/home/ubuntu/moya/output
mkdir -p $fastsurferdir/logs # create log dir for storing nohup output log (optional)
while read p ; do
echo $p
nohup ./run_fastsurfer.sh --t1 $datadir/$p/orig.nii \
--parallel --threads 16 --sid $p --sd $fastsurferdir > $fastsurferdir/logs/out-${p}.log &
sleep 3600s
done < /home/ubuntu/moya/data/subjects-list.txt
Instead of using sleep 3600s, as the program needs around an hour, I'd like to use wait until all processes (several PIDS) are finished.
If this is the right way, can you tell me how to do that?
BR Alex
wait will wait for all background processes to finish (see help wait). So all you need is to run wait after creating all of the background processes.
This may be more than what you are asking for but I figured I would provide some methods for controlling the number of threads you want to have running at once. I find that I always want to limit the number for various reasons.
Explaination
The following will limit concurrent threads to max_threads running at one time. I am also using the main design pattern so we have a main that runs the script with a function run_jobs that handles the calling and waiting. I read all of $p into an array, then traverse that array as we launch threads. It will either launch a thread up to 4 or wait 5 seconds, once there are at least one less than four it will start another thread. When finished it waits for any remaining to be done. If you want something more simplistic I can do that as well.
#!/usr/bin/env bash
export FREESURFER_HOME=$HOME/freesurfer
source $FREESURFER_HOME/SetUpFreeSurfer.sh
typeset max_threads=4
typeset subjects_list="/home/ubuntu/moya/data/subjects-list.txt"
typeset subjectsArray
run_jobs() {
local child="$$"
local num_children=0
local i=0
while [[ 1 ]] ; do
num_children=$(ps --no-headers -o pid --ppid=$child | wc -w) ; ((num_children-=1))
echo "Children: $num_children"
if [[ ${num_children} -lt ${max_threads} ]] ;then
if [ $i -lt ${#subjectsArray[#]} ] ;then
((i+=1))
# RUN COMMAND HERE &
./run_fastsurfer.sh --t1 $datadir/${subjectsArray[$i]}/orig.nii \
--parallel --threads 16 --sid ${subjectsArray[$i]} --sd $fastsurferdir
fi
fi
sleep 10
done
wait
}
main() {
cd /home/ubuntu/fastsurfer
datadir=/home/ubuntu/moya/data
fastsurferdir=/home/ubuntu/moya/output
mkdir -p $fastsurferdir/logs # create log dir for storing nohup output log (optional)
mapfile -t subjectsArray < ${subjects_list}
run_jobs
}
main
Note: I did not run this code since you have not provided enough information to actually do so.
#!/bin/bash
printf "Date\tMemory"
end=$((SECONDS+3600))
while[$SECONDS -lt $end];do
Memory=$(free -m | awk 'NR==2 {printf "%.2f%%\t",$3*200/$2')
Date=$(date"+%Y-%m-%d %H:%M:%S")
echo "$Date $Memory"
sleep 15
done
Q1: I have written above code and saved in perfMemstats.sh file and executed it using below command:
bash perfMemstats.sh >> perfStats.dat
Problem :
If I execute this .sh file its executed only for 10 mins. I want to run this file in background so that I can monitor other stats. Please let me know what mistake I'm doing. I even tried giving '&' it running in background but only for 10 min.
If I run this command without saving it in an output file (bash perfMemstats.sh) it is executing for 1 hour.
I am using the code found in this docker issue to basically start a container run a process within 20 seconds and if the process completes / does not complete / fails to execute / times out the container is killed regardless.
The code I am using currently is this:
#!/bin/bash
set -e
to=$1
shift
cont=$(docker run -d "$#")
code=$(timeout "$to" docker wait "$cont" || true)
docker kill $cont &> /dev/null
echo -n 'status: '
if [ -z "$code" ]; then
echo timeout
else
echo exited: $code
fi
echo output:
# pipe to sed simply for pretty nice indentation
docker logs $cont | sed 's/^/\t/'
docker rm $cont &> /dev/null
Which is almost perfect however if you run an infinite process (for example this python infinite loop):
while True:
print "inifinte loop"
The whole system jams up and the app crashes, after reading around a bit I think it has something to do with the STDOUT Buffer but I have absolutely no idea what that means?
The problem you have is with a process that is writing massive amounts of data to stdout.
These messages get logged into a file which grows infinitely.
Have a look at (depending on your system's location for log files):
sudo find /var/lib/docker/containers/ -name '*.log' -ls
You can remove old log files if they are of no interest.
One possibility is to start your docker run -d daemon
under a ulimit restriction on the max size a file can be.
Add to the start of your script, for example:
ulimit -f 20000 -c 0
This limits file sizes to 20000*1024 bytes, and disables core file dumps, which you expect
to get from infinite loops where writes are forced to fail.
Please add & at the end of
cont=$(docker run -d "$#")&
It will run the process in background.
I don't know dockers but if it still fail to stop you may also add just after this line the following :
mypid=$!
sleep 20 && kill $mypid
Regards
I've got a cron job checking for webserver (seeing if its active), which is handy..
http://pastebin.com/raw.php?i=KW8crfzh
I'm wanting after something similar for cpu usage. I'm running java backend which occasionally gets 70%+ cpu. I'm after a cron script to automatically kill/restart java if cpu load gets too high, how is this possible?
You could use top in batch mode coupled with some code to parse its output. For example:
top -p 1234 -n 1 -b
Will output a snapshot of the state of process 1234.
I use this script and it is pretty cool
#!/bin/bash
# author = Jaysunn
# Log
LOGFILE=/var/log/load_kill_log
# log the process causing the load at the time.
PSFILE=/var/log/ps_log
# Obtain the server load
loadavg=`uptime |cut -d , -f 4|cut -d : -f 2`
thisloadavg=`echo $loadavg|awk -F \. '{print $1}'`
if [ "$thisloadavg" -ge "10" ]; then
ps auxfww >> $PSFILE
date >> $LOGFILE
# Issue the command of choice. This can be any shell command.
## Put the command which restarts ..
fi
give executable permissions and add this to crontab with proper path to this script.
I need to wait for an input for 20 seconds, after that myscript should continue the execution.
I've tried using read -t20 var however this works only on bash. I'm using ksh on Solaris 10.
Can someone help me please?
EDIT: 20 seconds is only an example. Let's pretend it needs to wait for 1 hour. But the guy could or could not be in front the PC to write the input, he doesn't need to wait the 1 hour to enter an input, but if he's not in front of the PC so the shell should continue the execution after waiting for some time.
Thanks!
From man ksh:
TMOUT
If set to a value greater than zero, the shell terminates if a command is not entered within the prescribed number of seconds after issuing the PS1 prompt. The shell can be compiled with a maximum bound for this value which cannot be exceeded.
I'm not sure whether this works with read in ksh on Solaris. It does work with ksh93, but that version also has read -t.
This script includes this approach:
# Start the (potentially blocking) read process in the background
(read -p && print "$REPLY" > "$Tmp") & readpid=$!
# Now start a "watchdog" process that will kill the reader after
# some time:
(
sleep 2; kill $readpid >/dev/null 2>&1 ||
{ sleep 1; kill -1 $readpid >/dev/null 2>&1; } ||
{ sleep 1; kill -9 $readpid; }
) & watchdogpid=$!
# Now wait for the reading process to terminate. It will terminate
# reliably, either because the read terminated, or because the
# "watchdog" process made it terminate.
wait $readpid
# Now stop the watchdog:
kill -9 $watchdogpid >/dev/null 2>&1
REPLY=TERMINATED # Assume the worst
[[ -s $Tmp ]] && read < "$Tmp"
Look at this forum thread it has the answer in the third post.