Check if program is running with bash shell script? - shell

This is an example of a bash script which checks for some running process (daemon or service) and does specific actions (reload, sends mail) if there is no such process running.
check_process(){
# check the args
if [ "$1" = "" ];
then
return 0
fi
#PROCESS_NUM => get the process number regarding the given thread name
PROCESS_NUM='ps -ef | grep "$1" | grep -v "grep" | wc -l'
# for degbuging...
$PROCESS_NUM
if [ $PROCESS_NUM -eq 1 ];
then
return 1
else
return 0
fi
}
# Check whether the instance of thread exists:
while [ 1 ] ; do
echo 'begin checking...'
check_process "python test_demo.py" # the thread name
CHECK_RET = $?
if [ $CHECK_RET -eq 0 ]; # none exist
then
# do something...
fi
sleep 60
done
However, it doesn't work. I got "ERROR: Garbage option." for the ps command. What's wrong with these scripts? Thanks!

You can achieve almost everything in PROCESS_NUM with this one-liner:
[ `pgrep $1` ] && return 1 || return 0
if you're looking for a partial match, i.e. program is named foobar and you want your $1 to be just foo you can add the -f switch to pgrep:
[[ `pgrep -f $1` ]] && return 1 || return 0
Putting it all together your script could be reworked like this:
#!/bin/bash
check_process() {
echo "$ts: checking $1"
[ "$1" = "" ] && return 0
[ `pgrep -n $1` ] && return 1 || return 0
}
while [ 1 ]; do
# timestamp
ts=`date +%T`
echo "$ts: begin checking..."
check_process "dropbox"
[ $? -eq 0 ] && echo "$ts: not running, restarting..." && `dropbox start -i > /dev/null`
sleep 5
done
Running it would look like this:
# SHELL #1
22:07:26: begin checking...
22:07:26: checking dropbox
22:07:31: begin checking...
22:07:31: checking dropbox
# SHELL #2
$ dropbox stop
Dropbox daemon stopped.
# SHELL #1
22:07:36: begin checking...
22:07:36: checking dropbox
22:07:36: not running, restarting...
22:07:42: begin checking...
22:07:42: checking dropbox
Hope this helps!

If you want to execute that command, you should probably change:
PROCESS_NUM='ps -ef | grep "$1" | grep -v "grep" | wc -l'
to:
PROCESS_NUM=$(ps -ef | grep "$1" | grep -v "grep" | wc -l)

PROCESS="process name shown in ps -ef"
START_OR_STOP=1 # 0 = start | 1 = stop
MAX=30
COUNT=0
until [ $COUNT -gt $MAX ] ; do
echo -ne "."
PROCESS_NUM=$(ps -ef | grep "$PROCESS" | grep -v `basename $0` | grep -v "grep" | wc -l)
if [ $PROCESS_NUM -gt 0 ]; then
#runs
RET=1
else
#stopped
RET=0
fi
if [ $RET -eq $START_OR_STOP ]; then
sleep 5 #wait...
else
if [ $START_OR_STOP -eq 1 ]; then
echo -ne " stopped"
else
echo -ne " started"
fi
echo
exit 0
fi
let COUNT=COUNT+1
done
if [ $START_OR_STOP -eq 1 ]; then
echo -ne " !!$PROCESS failed to stop!! "
else
echo -ne " !!$PROCESS failed to start!! "
fi
echo
exit 1

Related

Execute command while process is running and end once process ends using bash

I am trying to create a while loop that will print "." while a shell script process is running and stop printing when the process has finished. My code continues to print "." after launch.sh has completed.
sh launch.sh & PIDIOS=$!
dot_function() {
running=$(ps aux | grep launch.sh | wc -l)
while [ $running -ge 1 ]; do
if [ $running -lt 1 ]; then
break
elif [ $running -ge 1 ];
printf "."
sleep 1
fi
done
}
dot_function & PIDMIX=$
How about using kill -0?
dotwait() {
while kill -0 "$1" 2>/dev/null
do
printf .
sleep 1
done
echo
}
sh launch.sh & dotwait "$!"

Daemon script - sh: unknown operand; arithmetic syntax error

I might be blind, but I can't find the errors my script got, maybe you guys got better eyes than me :). I use a busybox compiled linux on a embedded system, Kernel 4.18.0. I found the base script here: Gist-Template
the following error is at "start":
./daemon: line 195: arithmetic syntax error
when I try "stop" these messages appear, but i dont see a unknown operand at line 0:
sh: 0: unknown operand
* Stopping Monitoring
my script:
#!/bin/sh
daemonName="Monitoring"
pidDir="."
pidFile="$pidDir/$daemonName.pid"
pidFile="$daemonName.pid"
logDir="."
# To use a dated log file.
# logFile="$logDir/$daemonName-"`date +"%Y-%m-%d"`".log"
# To use a regular log file.
logFile="$logDir/$daemonName.log"
# Log maxsize in KB
logMaxSize=1024 # 1mb
runInterval=300 # In seconds
doCommands() {
# This is where you put all the commands for the daemon.
echo "Running commands."
}
############################################################
# Below are the command functions
############################################################
hw_temp() {
cpu_temp=$(sensors|grep CPU|awk '{print $3}'|awk '{print ($0-int($0)<0.499)?int($0):int($0)+1}')
env_temp=$(sensors|grep ENV|awk '{print $3}'|awk '{print ($0-int($0)<0.499)?int($0):int($0)+1}')
pcb_temp=$(sensors|grep PCB|awk '{print $3}'|awk '{print ($0-int($0)<0.499)?int($0):int($0)+1}')
echo "$cpu_temp $env_temp $pcb_temp" >> /opt/monitoring/bla
}
############################################################
# Below is the skeleton functionality of the daemon.
############################################################
myPid=`echo $$`
setupDaemon() {
# Make sure that the directories work.
if [ ! -d "$pidDir" ]; then
mkdir "$pidDir"
fi
if [ ! -d "$logDir" ]; then
mkdir "$logDir"
fi
if [ ! -f "$logFile" ]; then
touch "$logFile"
else
# Check to see if we need to rotate the logs.
size=$((`ls -l "$logFile" | cut -d " " -f 8`/1024))
if [[ $size -gt $logMaxSize ]]; then
mv $logFile "$logFile.old"
touch "$logFile"
fi
fi
}
startDaemon() {
# Start the daemon.
setupDaemon # Make sure the directories are there.
if [[ `checkDaemon` = 1 ]]; then
echo " * \033[31;5;148mError\033[39m: $daemonName is already running."
exit 1
fi
echo " * Starting $daemonName with PID: $myPid."
echo "$myPid" > "$pidFile"
log '*** '`date +"%Y-%m-%d"`": Starting up $daemonName."
# Start the loop.
loop
}
stopDaemon() {
# Stop the daemon.
if [[ `checkDaemon` -eq 0 ]]; then
echo " * \033[31;5;148mError\033[39m: $daemonName is not running."
exit 1
fi
echo " * Stopping $daemonName"
log '*** '`date +"%Y-%m-%d"`": $daemonName stopped."
if [[ ! -z `cat $pidFile` ]]; then
kill -9 `cat "$pidFile"` &> /dev/null
fi
}
statusDaemon() {
# Query and return whether the daemon is running.
if [[ `checkDaemon` -eq 1 ]]; then
echo " * $daemonName is running."
else
echo " * $daemonName isn't running."
fi
exit 0
}
restartDaemon() {
# Restart the daemon.
if [[ `checkDaemon` = 0 ]]; then
# Can't restart it if it isn't running.
echo "$daemonName isn't running."
exit 1
fi
stopDaemon
startDaemon
}
checkDaemon() {
# Check to see if the daemon is running.
# This is a different function than statusDaemon
# so that we can use it other functions.
if [ -z "$oldPid" ]; then
return 0
elif [[ `ps aux | grep "$oldPid" | grep "$daemonName" | grep -v grep` > /dev/null ]]; then
if [ -f "$pidFile" ]; then
if [[ `cat "$pidFile"` = "$oldPid" ]]; then
# Daemon is running.
# echo 1
return 1
else
# Daemon isn't running.
return 0
fi
fi
elif [[ `ps aux | grep "$daemonName" | grep -v grep | grep -v "$myPid" | grep -v "0:00.00"` > /dev/null ]]; then
# Daemon is running but without the correct PID. Restart it.
log '*** '`date +"%Y-%m-%d"`": $daemonName running with invalid PID; restarting."
restartDaemon
return 1
else
# Daemon not running.
return 0
fi
return 1
}
loop() {
# This is the loop.
now=`date +%s`
if [ -z $last ]; then
last=`date +%s`
fi
# Do everything you need the daemon to do.
doCommands
# Check to see how long we actually need to sleep for. If we want this to run
# once a minute and it's taken more than a minute, then we should just run it
# anyway.
last=`date +%s`
# Set the sleep interval
if [[ ! $((now-last+runInterval+1)) -lt $((runInterval)) ]]; then
sleep $((now-last+runInterval))
fi
# Startover
loop
}
log() {
# Generic log function.
echo "$1" >> "$logFile"
}
###############################################################
# Parse the command.
###############################################################
if [ -f "$pidFile" ]; then
oldPid=`cat "$pidFile"`
fi
checkDaemon
case "$1" in
start)
startDaemon
;;
stop)
stopDaemon
;;
status)
statusDaemon
;;
restart)
restartDaemon
;;
*)
echo "Error: usage $0 { start | stop | restart | status }"
exit 1
esac
exit 0

nested logical expression test in shell

I am trying to write a one-liner as a replacement for the below snippet.
# Check to see if the database is online; exit if not
isbooted=`grep 'Current state: Booted' serverlog.log | wc -l`
if [ $isbooted -eq 0 ]
then
exit
fi
# Check to see if the database has crashed; exit if so
iscrashed=`grep 'Status: OK' serverlog.log | wc -l`
if [ $iscrashed -eq 0 ]
then
exit
fi
echo 0
This is what I have done so far (I am unsure if this is correct),
[ $(grep 'Current state: Booted' serverlog.log | wc -l) -eq 0 ] \
&& [ $(grep 'Status: OK' serverlog.log | wc -l) -eq 0 ] \
&& echo 0
Can you please help me here?
The idea seems ok, but I'd go with a simplified version:
grep -q 'Current state: Booted' serverlog.log && grep -q 'Status: OK' serverlog.log && echo 0
or alternatively (you can replace -ge with -eq if you expect both lines to occur exactly once):
[ "$(grep -c -e 'Current state: Booted' -e 'Status: OK' serverlog.log)" -ge 2 ] && echo 0
Both versions print 0 if serverlog.log contains both 'Current state: Booted' and Status: OK.
For further reference see grep(1).

Bash script freezes when sent SIGINT

I finally grew tired of makefiles and wrote my own bash script to do my compiling. I wrote the whole thing, it works great, but for some reason, it freezes sometimes when I try to cancel it with ctrl-c. Here's the script:
#!/bin/bash
# Just to see if the script even sees the SIGINT
trap caught SIGINT
caught() { echo "hi"; }
compile() {
cpp=$(echo "$1" | sed -e 's/$/.cpp/' -e 's/^/src\//')
o=$(echo "$1" | sed -e 's/$/.o/' -e "s/^/$build_dir\//")
echo "$compile -c $cpp -o $o"
eval "$compile -c $cpp -o $o"
return $?
}
# I know this isn't normal, but I hate it when I forget to include something
# in the header and it fails way down the line
compile_h() {
h=$(echo "$1" | sed -e 's/$/.h/' -e 's/^/src\//')
o=$(echo "$1" | sed -e 's/$/.o/' -e "s/^/$build_dir\/headers\//")
echo "$compile -c $h -o /dev/null"
eval "$compile -x c++ -c $h -o $o"
if [ $? -ne 0 ]; then
return 1
fi
rm "$o"
return 0
}
build_type=$(awk 'NR==1' .build_options)
compile_command_debug=$(awk 'NR==2' .build_options)
link_command_debug=$(awk 'NR==3' .build_options)
compile_command_production=$(awk 'NR==4' .build_options)
link_command_production=$(awk 'NR==5' .build_options)
libraries=$(awk 'NR==6' .build_options)
# Make options for this build
build_dir="build"
compile="$compile_command_debug"
link="$link_command_debug"
if [ "$build_type" == "production" ]; then
build_dir="buildp"
compile="$compile_command_production"
link="$link_command_production"
fi
# These options need to be changeable later
output="game"
job_number=5
# There are more options, but they aren't important for this problem
while [ "$1" != "" ]; do
if [ "$1" == "clean" ]; then
rm -r $build_dir/*
fi
shift
done
# Get filenames
cpps=$(find src -name *.cpp | sed -e 's/src\///' -e 's/.cpp//' | sort)
hs=$(find src -name *.h | sed -e 's/src\///' -e 's/.h//' | sort)
# Ensure that all directories exist
directories=$(find src -type d | tail --lines=+2 | sed 's/src\///' | sort)
if [ ! -d "$build_dir/headers" ]; then
mkdir "$build_dir/headers"
fi
for dir in $directories; do
if [ ! -d "$build_dir/$dir" ]; then
mkdir "$build_dir/$dir"
fi
if [ ! -d "$build_dir/headers/$dir" ]; then
mkdir "$build_dir/headers/$dir"
fi
done
all_o="" # To be used for linking
# Determine what files need to be compiled
cpp_needed=""
h_needed=""
link_needed=false
# Check cpp files
for cpp_base in $cpps; do
o=$(echo "$cpp_base" | sed -e 's/$/.o/' -e "s/^/$build_dir\//")
all_o="$all_o $o"
d_file=$(echo "$cpp_base" | sed -e 's/$/.d/' -e "s/^/$build_dir\//")
if [ -f "$d_file" ]; then
d=$(<"$d_file")
d=$(echo "$d" | tr " " "\n" | tail --lines=+2 | grep "s")
if [ "$link_needed" = false ]; then
if [ "$o" -nt "$output" ]; then
link_needed=true
fi
fi
for dep in $d; do
if [ "$dep" -nt "$o" ]; then
if [ "$cpp_needed" == "" ]; then cpp_needed="$cpp_base"
else cpp_needed="$cpp_needed $cpp_base"
fi
link_needed=true
break
fi
done
else
if [ "$cpp_needed" == "" ]; then cpp_needed="$cpp_base"
else cpp_needed="$cpp_needed $cpp_base"
fi
link_needed=true
fi
done
# Check h files
for h_base in $hs; do
d_file=$(echo "$h_base" | sed -e 's/$/.d/' -e "s/^/$build_dir\/headers\//")
if [ -f "$d_file" ]; then
d=$(<"$d_file")
d=$(echo "$d" | tr " " "\n" | tail --lines=+2 | grep "s")
for dep in $d; do
if [ "$dep" -nt "$d_file" ]; then
if [ "$h_needed" == "" ]; then h_needed="$h_base"
else h_needed="$h_needed $h_base"
fi
break
fi
done
else
if [ "$h_needed" == "" ]; then h_needed="$h_base"
else h_needed="$h_needed $h_base"
fi
fi
done
# Compile
did_something=false
# Compile hs
while [ "$h_needed" != "" ]; do
for index in $(seq 1 $job_number); do
if [ "$h_needed" == "" ]; then break; fi
if ! kill -0 ${pids[index]} 2>/dev/null; then
new_file=$(echo "$h_needed" | awk '{print $1;}')
if [ $(echo "$h_needed" | wc -w) -eq 1 ]; then h_needed=""
else h_needed=$(echo "$h_needed" | cut -d " " -f2-)
fi
compile_h "$new_file" &
pids[index]=$!
did_something=true
fi
done
wait -n
if [ $? -ne 0 ]; then
wait
exit 1
fi
done
while [ $(pgrep -c -P$$) -gt 0 ]; do
wait -n
if [ $? -ne 0 ]; then
wait
exit 1
fi
done
# Compile cpps
while [ "$cpp_needed" != "" ]; do
for index in $(seq 1 $job_number); do
if [ "$cpp_needed" == "" ]; then break; fi
if ! kill -0 ${pids[index]} 2>/dev/null; then
new_file=$(echo "$cpp_needed" | awk '{print $1;}')
if [ $(echo "$cpp_needed" | wc -w) -eq 1 ]; then cpp_needed=""
else cpp_needed=$(echo "$cpp_needed" | cut -d " " -f2-)
fi
compile "$new_file" &
pids[index]=$!
did_something=true
fi
done
wait -n
if [ $? -ne 0 ]; then
wait
exit 1
fi
done
while [ $(pgrep -c -P$$) -gt 0 ]; do
wait -n
if [ $? -ne 0 ]; then
wait
exit 1
fi
done
# Compile program
if [ "$link_needed" = true ]; then
echo "$link $all_o -o game $libraries"
eval "$link $all_o -o game $libraries"
did_something=true
fi
# Make a message if nothing compiled
if [ "$did_something" = false ]; then
echo "Program is already compiled."
fi
It normally works perfectly. However, sometimes, when I try to cancel it with ctrl-c it just freezes. With a bit of debugging I saw that when the script wasn't setting up a new job, ctrl-c would work just fine. But when it was in the middle of setting up a new job, it would freeze the script. It wouldn't even catch the SIGINT (which that "echo hi" thing is for at the top). I honestly have no idea what's going on. Does anybody know what's going on? Thank you!
Edit: I realized I should probably mention I use g++ to compile.
Edit again: Here's an even-more stripped down version of the script. You would still need to setup some files to compile if you wanted to test it:
#!/bin/bash
# Just to see if the script even sees the SIGINT
trap caught SIGINT
caught() { echo "hi"; }
# I know this isn't normal, but I hate it when I forget to include something
# in the header and it fails way down the line
compile_h() {
h=$(echo "$1" | sed -e 's/$/.h/' -e 's/^/src\//')
o=$(echo "$1" | sed -e 's/$/.o/' -e "s/^/$build_dir\/headers\//")
echo "$compile -c $h -o /dev/null"
eval "$compile -x c++ -c $h -o $o"
if [ $? -ne 0 ]; then
return 1
fi
rm "$o"
return 0
}
build_type="debug"
build_dir="build"
compile="g++"
job_number=5
# Get filenames
hs=$(find src -name *.h | sed -e 's/src\///' -e 's/.h//' | sort)
h_needed=$(echo $hs)
# Compile hs
while [ "$h_needed" != "" ]; do
for index in $(seq 1 $job_number); do
if [ "$h_needed" == "" ]; then break; fi
if ! kill -0 ${pids[index]} 2>/dev/null; then
new_file=$(echo "$h_needed" | awk '{print $1;}')
if [ $(echo "$h_needed" | wc -w) -eq 1 ]; then h_needed=""
else h_needed=$(echo "$h_needed" | cut -d " " -f2-)
fi
compile_h "$new_file" &
pids[index]=$!
did_something=true
fi
done
wait -n
if [ $? -ne 0 ]; then
wait
exit 1
fi
done
while [ $(pgrep -c -P$$) -gt 0 ]; do
wait -n
if [ $? -ne 0 ]; then
wait
exit 1
fi
done
Any program that you run in your script may override your trap and set up its own. That trap might for example crash the currently running program for some reason. When this happens, take a look at the process tree in ps wafux to find the most likely culprit. For example, a zombie (Z) or uninterruptible sleep (D) process state (see man ps) is common when a process isn't going anywhere.

Create parallel processes and wait for all of them to finish, then redo steps

What i want to do should be pretty simple, on my own i have reached the solution below, all i need is a few pointers to tell me if this is the way to do it or i should refactor anything in the code.
The below code, should create a few parallel processes and wait for them to finish executing then rerun the code again and again and again...
The script is triggered by a cron job once at 10 minutes, if the script is running, then do nothing, otherwise start the working process.
Any insight is highly appreciated since i am not that familiar with bash programming.
#!/bin/bash
# paths
THISPATH="$( cd "$( dirname "$0" )" && pwd )"
# make sure we move in the working directory
cd $THISPATH
# console init path
CONSOLEPATH="$( cd ../../ && pwd )/console.php"
# command line arguments
daemon=0
PHPPATH="/usr/bin/php"
help=0
# flag for binary search
LOOKEDFORPHP=0
# arguments init
while getopts d:p:h: opt; do
case $opt in
d)
daemon=$OPTARG
;;
p)
PHPPATH=$OPTARG
LOOKEDFORPHP=1
;;
h)
help=$OPTARG
;;
esac
done
shift $((OPTIND - 1))
# allow only one process
processesLength=$(ps aux | grep -v "grep" | grep -c $THISPATH/send-campaigns-daemon.sh)
if [ ${processesLength:-0} -gt 2 ]; then
# The process is already running
exit 0
fi
if [ $help -eq 1 ]; then
echo "---------------------------------------------------------------"
echo "| Usage: send-campaigns-daemon.sh |"
echo "| To force PHP CLI binary : |"
echo "| send-campaigns-daemon.sh -p /path/to/php-cli/binary |"
echo "---------------------------------------------------------------"
exit 0
fi
# php executable path, find it if not provided
if [ $PHPPATH ] && [ ! -f $PHPPATH ] && [ $LOOKEDFORPHP -eq 0 ]; then
phpVariants=( "php-cli" "php5-cli" "php5" "php" )
LOOKEDFORPHP=1
for i in "${phpVariants[#]}"
do
which $i >/dev/null 2>&1
if [ $? -eq 0 ]; then
PHPPATH=$(which $i)
fi
done
fi
if [ ! $PHPPATH ] || [ ! -f $PHPPATH ]; then
# Did not find PHP
exit 1
fi
# load options from app
parallelProcessesPerCampaign=3
campaignsAtOnce=10
subscribersAtOnce=300
sleepTime=30
function loadOptions {
local COMMAND="$PHPPATH $CONSOLEPATH option get_option --name=%s --default=%d"
parallelProcessesPerCampaign=$(printf "$COMMAND" "system.cron.send_campaigns.parallel_processes_per_campaign" 3)
campaignsAtOnce=$(printf "$COMMAND" "system.cron.send_campaigns.campaigns_at_once" 10)
subscribersAtOnce=$(printf "$COMMAND" "system.cron.send_campaigns.subscribers_at_once" 300)
sleepTime=$(printf "$COMMAND" "system.cron.send_campaigns.pause" 30)
parallelProcessesPerCampaign=$($parallelProcessesPerCampaign)
campaignsAtOnce=$($campaignsAtOnce)
subscribersAtOnce=$($subscribersAtOnce)
sleepTime=$($sleepTime)
}
# define the daemon function that will stay in loop
function daemon {
loadOptions
local pids=()
local k=0
local i=0
local COMMAND="$PHPPATH -q $CONSOLEPATH send-campaigns --campaigns_offset=%d --campaigns_limit=%d --subscribers_offset=%d --subscribers_limit=%d --parallel_process_number=%d --parallel_processes_count=%d --usleep=%d --from_daemon=1"
while [ $i -lt $campaignsAtOnce ]
do
while [ $k -lt $parallelProcessesPerCampaign ]
do
parallelProcessNumber=$(( $k + 1 ))
usleep=$(( $k * 10 + $i * 10 ))
CMD=$(printf "$COMMAND" $i 1 $(( $subscribersAtOnce * $k )) $subscribersAtOnce $parallelProcessNumber $parallelProcessesPerCampaign $usleep)
$CMD > /dev/null 2>&1 &
pids+=($!)
k=$(( k + 1 ))
done
i=$(( i + 1 ))
done
waitForPids pids
sleep $sleepTime
daemon
}
function daemonize {
$THISPATH/send-campaigns-daemon.sh -d 1 -p $PHPPATH > /dev/null 2>&1 &
}
function waitForPids {
stillRunning=0
for i in "${pids[#]}"
do
if ps -p $i > /dev/null
then
stillRunning=1
break
fi
done
if [ $stillRunning -eq 1 ]; then
sleep 0.5
waitForPids pids
fi
return 0
}
if [ $daemon -eq 1 ]; then
daemon
else
daemonize
fi
exit 0
when starting a script, create a lock file to know that this script is running. When the script finish, delete the lock file. If somebody kill the process while it is running, the lock file remain forever, though test how old it is and delete after if older than a defined value. For example,
#!/bin/bash
# 10 min
LOCK_MAX=600
typedef LOCKFILE=/var/lock/${0##*/}.lock
if [[ -f $LOCKFILE ]] ; then
TIMEINI=$( stat -c %X $LOCKFILE )
SEGS=$(( $(date +%s) - $TIEMPOINI ))
if [[ $SEGS -gt $LOCK_MAX ]] ; then
reportLocking or somethig to inform you
# Kill old intance ???
OLDPID=$(<$LOCKFILE)
[[ -e /proc/$OLDPID ]] && kill -9 $OLDPID
# Next time that the program is run, there is no lock file and it will run.
rm $LOCKFILE
fi
exit 65
fi
# Save PID of this instance to the lock file
echo "$$" > $LOCKFILE
### Your code go here
# Remove the lock file before script finish
[[ -e $LOCKFILE ]] && rm $LOCKFILE
exit 0
from here:
#!/bin/bash
...
echo PARALLEL_JOBS:${PARALLEL_JOBS:=1}
declare -a tests=($(.../find_what_to_run))
echo "${tests[#]}" | \
xargs -d' ' -n1 -P${PARALLEL_JOBS} -I {} bash -c ".../run_that {}" || { echo "FAILURE"; exit 1; }
echo "SUCCESS"
and here you can nick the code for portable locking with fuser
Okay, so i guess i can answer to my own question with a proper answer that works after many tests.
So here is the final version, simplified, without comments/echo :
#!/bin/bash
sleep 2
DIR="$( cd "$( dirname "$0" )" && pwd )"
FILE_NAME="$( basename "$0" )"
COMMAND_FILE_PATH="$DIR/$FILE_NAME"
if [ ! -f "$COMMAND_FILE_PATH" ]; then
exit 1
fi
cd $DIR
CONSOLE_PATH="$( cd ../../ && pwd )/console.php"
PHP_PATH="/usr/bin/php"
help=0
LOOKED_FOR_PHP=0
while getopts p:h: opt; do
case $opt in
p)
PHP_PATH=$OPTARG
LOOKED_FOR_PHP=1
;;
h)
help=$OPTARG
;;
esac
done
shift $((OPTIND - 1))
if [ $help -eq 1 ]; then
printf "%s\n" "HELP INFO"
exit 0
fi
if [ "$PHP_PATH" ] && [ ! -f "$PHP_PATH" ] && [ "$LOOKED_FOR_PHP" -eq 0 ]; then
php_variants=( "php-cli" "php5-cli" "php5" "php" )
LOOKED_FOR_PHP=1
for i in "${php_variants[#]}"
do
which $i >/dev/null 2>&1
if [ $? -eq 0 ]; then
PHP_PATH="$(which $i)"
break
fi
done
fi
if [ ! "$PHP_PATH" ] || [ ! -f "$PHP_PATH" ]; then
exit 1
fi
LOCK_BASE_PATH="$( cd ../../../common/runtime && pwd )/shell-pids"
LOCK_PATH="$LOCK_BASE_PATH/send-campaigns-daemon.pid"
function remove_lock {
if [ -d "$LOCK_PATH" ]; then
rmdir "$LOCK_PATH" > /dev/null 2>&1
fi
exit 0
}
if [ ! -d "$LOCK_BASE_PATH" ]; then
if ! mkdir -p "$LOCK_BASE_PATH" > /dev/null 2>&1; then
exit 1
fi
fi
process_running=0
if mkdir "$LOCK_PATH" > /dev/null 2>&1; then
process_running=0
else
process_running=1
fi
if [ $process_running -eq 1 ]; then
exit 0
fi
trap "remove_lock" 1 2 3 15
COMMAND="$PHP_PATH $CONSOLE_PATH option get_option --name=%s --default=%d"
parallel_processes_per_campaign=$(printf "$COMMAND" "system.cron.send_campaigns.parallel_processes_per_campaign" 3)
campaigns_at_once=$(printf "$COMMAND" "system.cron.send_campaigns.campaigns_at_once" 10)
subscribers_at_once=$(printf "$COMMAND" "system.cron.send_campaigns.subscribers_at_once" 300)
sleep_time=$(printf "$COMMAND" "system.cron.send_campaigns.pause" 30)
parallel_processes_per_campaign=$($parallel_processes_per_campaign)
campaigns_at_once=$($campaigns_at_once)
subscribers_at_once=$($subscribers_at_once)
sleep_time=$($sleep_time)
k=0
i=0
pp=0
COMMAND="$PHP_PATH -q $CONSOLE_PATH send-campaigns --campaigns_offset=%d --campaigns_limit=%d --subscribers_offset=%d --subscribers_limit=%d --parallel_process_number=%d --parallel_processes_count=%d --usleep=%d --from_daemon=1"
while [ $i -lt $campaigns_at_once ]
do
while [ $k -lt $parallel_processes_per_campaign ]
do
parallel_process_number=$(( $k + 1 ))
usleep=$(( $k * 10 + $i * 10 ))
CMD=$(printf "$COMMAND" $i 1 $(( $subscribers_at_once * $k )) $subscribers_at_once $parallel_process_number $parallel_processes_per_campaign $usleep)
$CMD > /dev/null 2>&1 &
k=$(( k + 1 ))
pp=$(( pp + 1 ))
done
i=$(( i + 1 ))
done
wait
sleep ${sleep_time:-30}
$COMMAND_FILE_PATH -p "$PHP_PATH" > /dev/null 2>&1 &
remove_lock
exit 0
Usually, it is a lock file, not a lock path. You hold the PID in the lock file for monitoring your process. In this case your lock directory does not hold any PID information. Your script also does not do any PID file/directory maintenance when it starts in case of a improper shutdown of your process without cleaning of your lock.
I like your first script better with this in mind. Monitoring the PID's running directly is cleaner. The only problem is if you start a second instance with cron, it is not aware of the PID's connect to the first instance.
You also have processLength -gt 2 which is 2, not 1 process running so you will duplicate your process threads.
It seems also that daemonize is just recalling the script with daemon which is not very useful. Also, having a variable with the same name as a function is not effective.
The correct way to make a lockfile is like this:
# Create a temporary file
echo $$ > ${LOCKFILE}.tmp$$
# Try the lock; ln without -f is atomic
if ln ${LOCKFILE}.tmp$$ ${LOCKFILE}; then
# we got the lock
else
# we didn't get the lock
fi
# Tidy up the temporary file
rm ${LOCKFILE}.tmp$$
And to release the lock:
# Unlock
rm ${LOCKFILE}
The key thing is to create the lock file to one side, using a unique name, and then try to link it to the real name. This is an atomic operation, so it should be safe.
Any solution that does "test and set" gives you a race condition to deal with. Yes, that can be sorted out, but you end up write extra code.

Resources