How to write Gradle startup script - gradle

I have a Gradle app that I startup using ./gradlew run. This works fine, but I'm trying to deploy to an AWS instance (Ubuntu 12) and I would like the script to execute on boot. I tried writing a startup.sh file with the above command, but no dice. I've also tried adding the command to the /etc/rc.local file, but that doesn't seem to work either. Can someone give me an idea as to how to execute `./gradlew run' on startup? Thanks!

I wrote the following init script for starting gradle applications at system startup for redhat distros (centos/fedora etc).
You need to perform a few steps to tie it all together:
deploy your gradle application using gradle distZip onto your target server
create a configuration file /etc/my-service.conf
link the init script (see below) to the service name in /etc/init.d/my-service
An example configuration file /etc/my-service.conf
username=someunixuser
serviceName=MyStandaloneServer
prog="/path/to/bin/MyStandaloneServer -a any -p params -y you -w want"
javaClass="some.java.MyStandaloneServer"
Note the path to the application from the distZip in the prog line.
You then link the init script to the actual service you want it to be run as, e.g.
ln -s /path/to/gradle-init-start-stop /etc/init.d/my-service
Once you've done this, you can use chkconfig to add the service in the usual way (it defaults to 3/4/5)
Here is the script gradle-init-start-stop
#!/bin/bash
#
# chkconfig: 345 80 20
# description: Start and stop script for gradle created java application startup
#
# This is a generic file that can be used by any distribution from gradle ("gradle distZip").
# Link this file to the name of the process you want to run.
# e.g.
# ln -s /path/to/gradle-init-start-stop /etc/init.d/ivy-jetty
#
# it requires a conf file /etc/NAME.conf, e.g. /etc/ivy-jetty.conf
# otherwise it will quit.
#
# CONFIGURATION FILE ENTRIES:
# ---------------------------
# username=process-owner
# prog="/path/to/gradle-startscript -a any -e extra parameters"
# serviceName=SomeShortNameForService
# javaClass=package.for.JavaClass
. /etc/rc.d/init.d/functions
BASENAME=$(basename $0)
maxShutdownTime=15
CONF=/etc/${BASENAME}.conf
pidfile=/var/run/$BASENAME.pid
if [ ! -f $CONF ] ; then
echo "Could not find configuration file: $CONF"
exit 1
fi
####### SOURCE CONFIGURATION FILE
source $CONF
checkProcessIsRunning() {
local pid="$1"
if [ -z "$pid" -o "$pid" == " " ]; then return 1; fi
if [ ! -e /proc/$pid ]; then return 1; fi
return 0
}
checkProcessIsOurService() {
local pid="$1"
if [ "$(ps -p $pid --no-headers -o comm)" != "java" ]; then return 1; fi
grep -q --binary -F "$javaClass" /proc/$pid/cmdline
if [ $? -ne 0 ]; then return 1; fi
return 0
}
getServicePID() {
if [ ! -f $pidfile ]; then return 1; fi
pid="$(<$pidfile)"
checkProcessIsRunning $pid || return 1
checkProcessIsOurService $pid || return 1
return 0
}
startService() {
cmd="nohup $prog >/dev/null 2>&1 & echo \$!"
sudo -u $username -H $SHELL -c "$cmd" > $pidfile
sleep 0.2
pid="$(<$pidfile)"
if checkProcessIsRunning $pid; then
return 0
else
return 1
fi
}
start() {
getServicePID
if [ $? -eq 0 ]; then echo -n "$serviceName is already running"; RETVAL=0; echo ""; return 0; fi
echo -n "Starting $serviceName: "
startService
if [ $? -ne 0 ] ; then
echo "failed"
return 1
else
echo "started"
return 0
fi
}
stopService() {
# soft kill first...
kill $pid || return 1
# check if process dead, sleep 0.2s otherwise
for ((i=0; i<maxShutdownTime*5; i++)); do
checkProcessIsRunning $pid
if [ $? -ne 0 ] ; then
rm -f $pidfile
return 0
fi
sleep 0.2
done
# hard kill now...
kill -s KILL $pid || return 1
# check if process dead, sleep 0.2s otherwise
for ((i=0; i<maxShutdownTime*5; i++)); do
checkProcessIsRunning $pid
if [ $? -ne 0 ] ; then
rm -f $pidfile
return 0
fi
sleep 0.2
done
return 1
}
stop() {
getServicePID
if [ $? -ne 0 ]; then echo -n "$serviceName is not running"; RETVAL=0; echo ""; return 0; fi
pid="$(<$pidfile)"
echo -n "Stopping $serviceName "
stopService
if [ $? -ne 0 ]; then RETVAL=1; echo "failed"; return 1; fi
echo "stopped PID=$pid"
RETVAL=0
return 0
}
restart() {
stop
start
}
checkServiceStatus() {
echo -n "Checking for $serviceName: "
if getServicePID; then
echo "running PID=$pid"
RETVAL=0
else
echo "stopped"
RETVAL=3
fi
return 0;
}
####### START OF MAIN SCRIPT
RETVAL=0
case "$1" in
start)
$1
;;
stop)
$1
;;
restart)
$1
;;
status)
checkServiceStatus
;;
*)
echo "Usage: $0 {start|stop|status|restart}"
exit 1
esac
exit $RETVAL

Related

Bash script checkpoints

I am developing a big script which skeleton, looks like below:
#!/bin/bash
load_variables()
function_1()
function_2()
function_3()
[...]
function_n()
During each take-off, user flags are first loaded in load_variables() function.
Then script continue execution function_1() => function_2() => [...] => function_n()
I need to implement checkpoints which will be stored in log.txt.
Let's say, that script has been stoped or crashed at the function_2().
I want to save progress before each function start, store it in the log.txt and when I re-run the script again, I want to load_variables() and then jump to the crash point/checkpoint stored in log.txt.
How can I achieve that using bash?
I want to save progress before each function start, store it in the log.txt and when I re-run the script again, I want to load_variables() and then jump to the crash point/checkpoint stored in log.txt.
Do exactly that. But you can't "jump" in bash scripts - instead of "jump", just skip already run functions, that you can track. Basically, in pseudocode:
load_variables() {
if [[ -e log.txt ]]; then
. log.txt
fi
}
already_run_functions=()
checkpoint() {
# if the function was already run
if "$1" in already_run_functions[#]; then
# skip this function
return
fi
{
# save state with function name
declare -f
declare -p
} > log.txt
# run it
if ! "$#"; then
# handle errors
exit 1
fi
already_run_functions+=("$1")
}
load_variables
checkpoint function1
checkpoint function2
checkpoint function3
Overall, it's shell, it is simple. It's way better to use a build system. Simple Make is more than enough to track dependencies of multiple shell scripts and parallelize the work. Store the result in files after each task and distribute functions to multiple files.
So some real example:
rm -f log.txt
script() {
load_variables="
if [[ -e log.txt ]]; then
. log.txt
cd \"\$PWD\"
else
already_run_functions=()
fi
"
oneof() {
local i
for i in "${#:2}"; do
if [[ "$1" = "$i" ]]; then
return 0
fi
done
return 1
}
checkpoint() {
# if the function was already run
if oneof "$1" "${already_run_functions[#]}"; then
# skip this function
return
fi
{
# save state
declare -f
declare -p $(compgen -v | grep -Ev '^(BASH.*|EUID|FUNCNAME|GROUPS|PPID|SHELLOPTS|UID|SHELL|SHLVL|USER|TERM|RANDOM|PIPESTATUS|LINENO|COLUMN|LC_.*|LANG)$')
} > log.txt
# run it
if ! "$#"; then
# handle errors
echo "checkpoint: $1 failed"
exit 1
fi
already_run_functions+=("$1")
}
function1() {
echo function1
}
function2() {
echo function2
if [[ -e file ]]; then
return 1
fi
}
function3() {
echo function3
}
eval "$load_variables"
checkpoint function1
checkpoint function2
checkpoint function3
}
touch file
( script )
rm file
( script )
outputs:
function1
function2
checkpoint: function2 failed
function2
function3
This is an example with trap.
Saving function names on error with trap and $LINENO
Logfile will be removed if everything is ok
#!/bin/bash
trap 'clear_log' EXIT
trap 'log_checkpoint $LINENO' ERR
CHECKLOG=checkpoints.log
clear_log() {
if [ $? -eq 0 ] ; then
if [ -f "$CHECKLOG" ]; then
rm "$CHECKLOG"
fi
fi
}
log_checkpoint() {
func=$(sed -n $1p $0)
echo "Error on line $1: $func"
echo "$func" > $CHECKLOG
exit 1
}
retry(){
[ ! -f $CHECKLOG ] && return 0
if grep -q "$1" "$CHECKLOG"; then
echo retry "$1"; rm "$CHECKLOG"; return 0
else
echo skip "$1"; return 1
fi
}
func1(){
retry ${FUNCNAME[0]} || return 0
echo hello | grep hello
}
func2(){
retry ${FUNCNAME[0]} || return 0
echo hello |grep hello
}
func3(){
retry ${FUNCNAME[0]} || return 0
echo hello | grep foo
}
func1
func2
func3
exit 0
Here is my proposal, based on the above answers.
It can be helpful for people who need to trap CTRL + C and other crashes except for errors:
#!/bin/bash
### Catch crash in trap and save the function name in anchor.log
trap 'echo $anchor > anchor.log && exit 1' SIGINT
trap 'echo $anchor > anchor.log && exit 1' SIGHUP
trap 'echo $anchor > anchor.log && exit 1' SIGKILL
trap 'echo $anchor > anchor.log && exit 1' SIGTERM
### ---
clear_log() {
### REMOVING LOG IF PROGRAM EXIT NORMALLY
if [ -f "anchor.log" ]; then
rm "anchor.log"
fi
}
anchor_check(){
### RETRY FUNCTION IF IT'S NOT IN anchor.log
anchor=$1
# Check if function name is inside "anchor.log"
[ ! -f "anchor.log" ] && return 0
if grep -q "$1" "anchor.log"; then
rm "anchor.log"; return 0
else
return 1
fi
}
### EACH FUNCTION START WITH anchor_check
function_1() {
anchor_check "${FUNCNAME[0]}" || return 0
echo "test $anchor"
sleep 2
}
function_2() {
anchor_check "${FUNCNAME[0]}" || return 0
echo "test $anchor"
sleep 2
}
function_3() {
anchor_check "${FUNCNAME[0]}" || return 0
echo "test $anchor"
}
function_1
function_2
function_3
clear_log
Thank you guys for your help!

What is the meaning of success & failure in shell script?

In this init script you can see lines which say only success or failure.
What are these commands?
start)
echo -n "Starting Jenkins "
daemon --user "$JENKINS_USER" --pidfile "$JENKINS_PID_FILE" $JAVA_CMD $PARAMS > /dev/null
RETVAL=$?
if [ $RETVAL = 0 ]; then
success
echo > "$JENKINS_PID_FILE" # just in case we fail to find it
MY_SESSION_ID=`/bin/ps h -o sess -p $$`
# get PID
/bin/ps hww -u "$JENKINS_USER" -o sess,ppid,pid,cmd | \
while read sess ppid pid cmd; do
[ "$ppid" = 1 ] || continue
# this test doesn't work because Jenkins sets a new Session ID
# [ "$sess" = "$MY_SESSION_ID" ] || continue
echo "$cmd" | grep $JENKINS_WAR > /dev/null
[ $? = 0 ] || continue
# found a PID
echo $pid > "$JENKINS_PID_FILE"
done
touch $JENKINS_LOCKFILE
else
failure
fi
echo
;;
Oh, I found the answer. These are the functions defined in /etc/init.d/functions.
# Log that something succeeded
success() {
[ "$BOOTUP" != "verbose" -a -z "${LSB:-}" ] && echo_success
return 0
}
# Log that something failed
failure() {
local rc=$?
[ "$BOOTUP" != "verbose" -a -z "${LSB:-}" ] && echo_failure
[ -x /bin/plymouth ] && /bin/plymouth --details
return $rc
}

Daemon script - sh: unknown operand; arithmetic syntax error

I might be blind, but I can't find the errors my script got, maybe you guys got better eyes than me :). I use a busybox compiled linux on a embedded system, Kernel 4.18.0. I found the base script here: Gist-Template
the following error is at "start":
./daemon: line 195: arithmetic syntax error
when I try "stop" these messages appear, but i dont see a unknown operand at line 0:
sh: 0: unknown operand
* Stopping Monitoring
my script:
#!/bin/sh
daemonName="Monitoring"
pidDir="."
pidFile="$pidDir/$daemonName.pid"
pidFile="$daemonName.pid"
logDir="."
# To use a dated log file.
# logFile="$logDir/$daemonName-"`date +"%Y-%m-%d"`".log"
# To use a regular log file.
logFile="$logDir/$daemonName.log"
# Log maxsize in KB
logMaxSize=1024 # 1mb
runInterval=300 # In seconds
doCommands() {
# This is where you put all the commands for the daemon.
echo "Running commands."
}
############################################################
# Below are the command functions
############################################################
hw_temp() {
cpu_temp=$(sensors|grep CPU|awk '{print $3}'|awk '{print ($0-int($0)<0.499)?int($0):int($0)+1}')
env_temp=$(sensors|grep ENV|awk '{print $3}'|awk '{print ($0-int($0)<0.499)?int($0):int($0)+1}')
pcb_temp=$(sensors|grep PCB|awk '{print $3}'|awk '{print ($0-int($0)<0.499)?int($0):int($0)+1}')
echo "$cpu_temp $env_temp $pcb_temp" >> /opt/monitoring/bla
}
############################################################
# Below is the skeleton functionality of the daemon.
############################################################
myPid=`echo $$`
setupDaemon() {
# Make sure that the directories work.
if [ ! -d "$pidDir" ]; then
mkdir "$pidDir"
fi
if [ ! -d "$logDir" ]; then
mkdir "$logDir"
fi
if [ ! -f "$logFile" ]; then
touch "$logFile"
else
# Check to see if we need to rotate the logs.
size=$((`ls -l "$logFile" | cut -d " " -f 8`/1024))
if [[ $size -gt $logMaxSize ]]; then
mv $logFile "$logFile.old"
touch "$logFile"
fi
fi
}
startDaemon() {
# Start the daemon.
setupDaemon # Make sure the directories are there.
if [[ `checkDaemon` = 1 ]]; then
echo " * \033[31;5;148mError\033[39m: $daemonName is already running."
exit 1
fi
echo " * Starting $daemonName with PID: $myPid."
echo "$myPid" > "$pidFile"
log '*** '`date +"%Y-%m-%d"`": Starting up $daemonName."
# Start the loop.
loop
}
stopDaemon() {
# Stop the daemon.
if [[ `checkDaemon` -eq 0 ]]; then
echo " * \033[31;5;148mError\033[39m: $daemonName is not running."
exit 1
fi
echo " * Stopping $daemonName"
log '*** '`date +"%Y-%m-%d"`": $daemonName stopped."
if [[ ! -z `cat $pidFile` ]]; then
kill -9 `cat "$pidFile"` &> /dev/null
fi
}
statusDaemon() {
# Query and return whether the daemon is running.
if [[ `checkDaemon` -eq 1 ]]; then
echo " * $daemonName is running."
else
echo " * $daemonName isn't running."
fi
exit 0
}
restartDaemon() {
# Restart the daemon.
if [[ `checkDaemon` = 0 ]]; then
# Can't restart it if it isn't running.
echo "$daemonName isn't running."
exit 1
fi
stopDaemon
startDaemon
}
checkDaemon() {
# Check to see if the daemon is running.
# This is a different function than statusDaemon
# so that we can use it other functions.
if [ -z "$oldPid" ]; then
return 0
elif [[ `ps aux | grep "$oldPid" | grep "$daemonName" | grep -v grep` > /dev/null ]]; then
if [ -f "$pidFile" ]; then
if [[ `cat "$pidFile"` = "$oldPid" ]]; then
# Daemon is running.
# echo 1
return 1
else
# Daemon isn't running.
return 0
fi
fi
elif [[ `ps aux | grep "$daemonName" | grep -v grep | grep -v "$myPid" | grep -v "0:00.00"` > /dev/null ]]; then
# Daemon is running but without the correct PID. Restart it.
log '*** '`date +"%Y-%m-%d"`": $daemonName running with invalid PID; restarting."
restartDaemon
return 1
else
# Daemon not running.
return 0
fi
return 1
}
loop() {
# This is the loop.
now=`date +%s`
if [ -z $last ]; then
last=`date +%s`
fi
# Do everything you need the daemon to do.
doCommands
# Check to see how long we actually need to sleep for. If we want this to run
# once a minute and it's taken more than a minute, then we should just run it
# anyway.
last=`date +%s`
# Set the sleep interval
if [[ ! $((now-last+runInterval+1)) -lt $((runInterval)) ]]; then
sleep $((now-last+runInterval))
fi
# Startover
loop
}
log() {
# Generic log function.
echo "$1" >> "$logFile"
}
###############################################################
# Parse the command.
###############################################################
if [ -f "$pidFile" ]; then
oldPid=`cat "$pidFile"`
fi
checkDaemon
case "$1" in
start)
startDaemon
;;
stop)
stopDaemon
;;
status)
statusDaemon
;;
restart)
restartDaemon
;;
*)
echo "Error: usage $0 { start | stop | restart | status }"
exit 1
esac
exit 0

How to start elasticsearch run as service in centos 7?

I have install elasticsearch 2.3.3 in centos 7 but after closing terminal elasticsearch plugin head automatically close but I want keep running in background. please give me helpful answer.
You can run it in background as two ways,
1. Nohup
2. Creating service script and put it in init.d folder
Nohup
Eg: nohup ./bin/elasticsearch
Service script
Use the following script,
#!/bin/bash
### BEGIN INIT INFO
# Provides: Elasticsearch
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Runs elasticsearch daemon
# Description: Runs the elasticsearch daemon as a non-root user
### END INIT INFO
# Process name
NAME=elasticsearch
DESC="Elasticsearch"
PROG="/etc/init.d/elasticsearch"
# Configure location of Elasticsearch bin
ELASTICSEARCH_BIN=/opt/elasticsearch-2.3.0/bin
# PID Info
PID_FOLDER=/var/run/elasticsearch/
PID_FILE=/var/run/elasticsearch/$NAME.pid
LOCK_FILE=/var/lock/subsys/$NAME
PATH=/bin:/usr/bin:/sbin:/usr/sbin:$ELASTICSEARCH_BIN
DAEMON=$ELASTICSEARCH_BIN/$NAME
# Configure logging location
ELASTICSEARCH_LOG=/var/log/elasticsearch.log
# Begin Script
RETVAL=0
if [ `id -u` -ne 0 ]; then
echo "You need root privileges to run this script"
exit 1
fi
# Function library
. /etc/init.d/functions
start() {
echo -n "Starting $DESC : "
pid=`pidofproc -p $PID_FILE elasticsearch`
if [ -n "$pid" ] ; then
echo "Already running."
exit 0
else
# Start Daemon
if [ ! -d "$PID_FOLDER" ] ; then
mkdir $PID_FOLDER
fi
daemon --user=$DAEMON_USER --pidfile=$PID_FILE $DAEMON 1>"$ELASTICSEARCH_LOG" 2>&1 &
sleep 2
pidofproc node > $PID_FILE
RETVAL=$?
[[ $? -eq 0 ]] && success || failure
echo
[ $RETVAL = 0 ] && touch $LOCK_FILE
return $RETVAL
fi
}
reload()
{
echo "Reload command is not implemented for this service."
return $RETVAL
}
stop() {
echo -n "Stopping $DESC : "
killproc -p $PID_FILE $DAEMON
RETVAL=$?
echo
[ $RETVAL = 0 ] && rm -f $PID_FILE $LOCK_FILE
}
case "$1" in
start)
start
;;
stop)
stop
;;
status)
status -p $PID_FILE $DAEMON
RETVAL=$?
;;
restart)
stop
start
;;
reload)
reload
;;
*)
# Invalid Arguments, print the following message.
echo "Usage: $0 {start|stop|status|restart}" >&2
exit 2
;;
esac
sudo chmod +x /etc/init.d/elasticsearch
sudo update-rc.d elasticsearch defaults 96 9
sudo /etc/init.d/elasticsearch restart

Create parallel processes and wait for all of them to finish, then redo steps

What i want to do should be pretty simple, on my own i have reached the solution below, all i need is a few pointers to tell me if this is the way to do it or i should refactor anything in the code.
The below code, should create a few parallel processes and wait for them to finish executing then rerun the code again and again and again...
The script is triggered by a cron job once at 10 minutes, if the script is running, then do nothing, otherwise start the working process.
Any insight is highly appreciated since i am not that familiar with bash programming.
#!/bin/bash
# paths
THISPATH="$( cd "$( dirname "$0" )" && pwd )"
# make sure we move in the working directory
cd $THISPATH
# console init path
CONSOLEPATH="$( cd ../../ && pwd )/console.php"
# command line arguments
daemon=0
PHPPATH="/usr/bin/php"
help=0
# flag for binary search
LOOKEDFORPHP=0
# arguments init
while getopts d:p:h: opt; do
case $opt in
d)
daemon=$OPTARG
;;
p)
PHPPATH=$OPTARG
LOOKEDFORPHP=1
;;
h)
help=$OPTARG
;;
esac
done
shift $((OPTIND - 1))
# allow only one process
processesLength=$(ps aux | grep -v "grep" | grep -c $THISPATH/send-campaigns-daemon.sh)
if [ ${processesLength:-0} -gt 2 ]; then
# The process is already running
exit 0
fi
if [ $help -eq 1 ]; then
echo "---------------------------------------------------------------"
echo "| Usage: send-campaigns-daemon.sh |"
echo "| To force PHP CLI binary : |"
echo "| send-campaigns-daemon.sh -p /path/to/php-cli/binary |"
echo "---------------------------------------------------------------"
exit 0
fi
# php executable path, find it if not provided
if [ $PHPPATH ] && [ ! -f $PHPPATH ] && [ $LOOKEDFORPHP -eq 0 ]; then
phpVariants=( "php-cli" "php5-cli" "php5" "php" )
LOOKEDFORPHP=1
for i in "${phpVariants[#]}"
do
which $i >/dev/null 2>&1
if [ $? -eq 0 ]; then
PHPPATH=$(which $i)
fi
done
fi
if [ ! $PHPPATH ] || [ ! -f $PHPPATH ]; then
# Did not find PHP
exit 1
fi
# load options from app
parallelProcessesPerCampaign=3
campaignsAtOnce=10
subscribersAtOnce=300
sleepTime=30
function loadOptions {
local COMMAND="$PHPPATH $CONSOLEPATH option get_option --name=%s --default=%d"
parallelProcessesPerCampaign=$(printf "$COMMAND" "system.cron.send_campaigns.parallel_processes_per_campaign" 3)
campaignsAtOnce=$(printf "$COMMAND" "system.cron.send_campaigns.campaigns_at_once" 10)
subscribersAtOnce=$(printf "$COMMAND" "system.cron.send_campaigns.subscribers_at_once" 300)
sleepTime=$(printf "$COMMAND" "system.cron.send_campaigns.pause" 30)
parallelProcessesPerCampaign=$($parallelProcessesPerCampaign)
campaignsAtOnce=$($campaignsAtOnce)
subscribersAtOnce=$($subscribersAtOnce)
sleepTime=$($sleepTime)
}
# define the daemon function that will stay in loop
function daemon {
loadOptions
local pids=()
local k=0
local i=0
local COMMAND="$PHPPATH -q $CONSOLEPATH send-campaigns --campaigns_offset=%d --campaigns_limit=%d --subscribers_offset=%d --subscribers_limit=%d --parallel_process_number=%d --parallel_processes_count=%d --usleep=%d --from_daemon=1"
while [ $i -lt $campaignsAtOnce ]
do
while [ $k -lt $parallelProcessesPerCampaign ]
do
parallelProcessNumber=$(( $k + 1 ))
usleep=$(( $k * 10 + $i * 10 ))
CMD=$(printf "$COMMAND" $i 1 $(( $subscribersAtOnce * $k )) $subscribersAtOnce $parallelProcessNumber $parallelProcessesPerCampaign $usleep)
$CMD > /dev/null 2>&1 &
pids+=($!)
k=$(( k + 1 ))
done
i=$(( i + 1 ))
done
waitForPids pids
sleep $sleepTime
daemon
}
function daemonize {
$THISPATH/send-campaigns-daemon.sh -d 1 -p $PHPPATH > /dev/null 2>&1 &
}
function waitForPids {
stillRunning=0
for i in "${pids[#]}"
do
if ps -p $i > /dev/null
then
stillRunning=1
break
fi
done
if [ $stillRunning -eq 1 ]; then
sleep 0.5
waitForPids pids
fi
return 0
}
if [ $daemon -eq 1 ]; then
daemon
else
daemonize
fi
exit 0
when starting a script, create a lock file to know that this script is running. When the script finish, delete the lock file. If somebody kill the process while it is running, the lock file remain forever, though test how old it is and delete after if older than a defined value. For example,
#!/bin/bash
# 10 min
LOCK_MAX=600
typedef LOCKFILE=/var/lock/${0##*/}.lock
if [[ -f $LOCKFILE ]] ; then
TIMEINI=$( stat -c %X $LOCKFILE )
SEGS=$(( $(date +%s) - $TIEMPOINI ))
if [[ $SEGS -gt $LOCK_MAX ]] ; then
reportLocking or somethig to inform you
# Kill old intance ???
OLDPID=$(<$LOCKFILE)
[[ -e /proc/$OLDPID ]] && kill -9 $OLDPID
# Next time that the program is run, there is no lock file and it will run.
rm $LOCKFILE
fi
exit 65
fi
# Save PID of this instance to the lock file
echo "$$" > $LOCKFILE
### Your code go here
# Remove the lock file before script finish
[[ -e $LOCKFILE ]] && rm $LOCKFILE
exit 0
from here:
#!/bin/bash
...
echo PARALLEL_JOBS:${PARALLEL_JOBS:=1}
declare -a tests=($(.../find_what_to_run))
echo "${tests[#]}" | \
xargs -d' ' -n1 -P${PARALLEL_JOBS} -I {} bash -c ".../run_that {}" || { echo "FAILURE"; exit 1; }
echo "SUCCESS"
and here you can nick the code for portable locking with fuser
Okay, so i guess i can answer to my own question with a proper answer that works after many tests.
So here is the final version, simplified, without comments/echo :
#!/bin/bash
sleep 2
DIR="$( cd "$( dirname "$0" )" && pwd )"
FILE_NAME="$( basename "$0" )"
COMMAND_FILE_PATH="$DIR/$FILE_NAME"
if [ ! -f "$COMMAND_FILE_PATH" ]; then
exit 1
fi
cd $DIR
CONSOLE_PATH="$( cd ../../ && pwd )/console.php"
PHP_PATH="/usr/bin/php"
help=0
LOOKED_FOR_PHP=0
while getopts p:h: opt; do
case $opt in
p)
PHP_PATH=$OPTARG
LOOKED_FOR_PHP=1
;;
h)
help=$OPTARG
;;
esac
done
shift $((OPTIND - 1))
if [ $help -eq 1 ]; then
printf "%s\n" "HELP INFO"
exit 0
fi
if [ "$PHP_PATH" ] && [ ! -f "$PHP_PATH" ] && [ "$LOOKED_FOR_PHP" -eq 0 ]; then
php_variants=( "php-cli" "php5-cli" "php5" "php" )
LOOKED_FOR_PHP=1
for i in "${php_variants[#]}"
do
which $i >/dev/null 2>&1
if [ $? -eq 0 ]; then
PHP_PATH="$(which $i)"
break
fi
done
fi
if [ ! "$PHP_PATH" ] || [ ! -f "$PHP_PATH" ]; then
exit 1
fi
LOCK_BASE_PATH="$( cd ../../../common/runtime && pwd )/shell-pids"
LOCK_PATH="$LOCK_BASE_PATH/send-campaigns-daemon.pid"
function remove_lock {
if [ -d "$LOCK_PATH" ]; then
rmdir "$LOCK_PATH" > /dev/null 2>&1
fi
exit 0
}
if [ ! -d "$LOCK_BASE_PATH" ]; then
if ! mkdir -p "$LOCK_BASE_PATH" > /dev/null 2>&1; then
exit 1
fi
fi
process_running=0
if mkdir "$LOCK_PATH" > /dev/null 2>&1; then
process_running=0
else
process_running=1
fi
if [ $process_running -eq 1 ]; then
exit 0
fi
trap "remove_lock" 1 2 3 15
COMMAND="$PHP_PATH $CONSOLE_PATH option get_option --name=%s --default=%d"
parallel_processes_per_campaign=$(printf "$COMMAND" "system.cron.send_campaigns.parallel_processes_per_campaign" 3)
campaigns_at_once=$(printf "$COMMAND" "system.cron.send_campaigns.campaigns_at_once" 10)
subscribers_at_once=$(printf "$COMMAND" "system.cron.send_campaigns.subscribers_at_once" 300)
sleep_time=$(printf "$COMMAND" "system.cron.send_campaigns.pause" 30)
parallel_processes_per_campaign=$($parallel_processes_per_campaign)
campaigns_at_once=$($campaigns_at_once)
subscribers_at_once=$($subscribers_at_once)
sleep_time=$($sleep_time)
k=0
i=0
pp=0
COMMAND="$PHP_PATH -q $CONSOLE_PATH send-campaigns --campaigns_offset=%d --campaigns_limit=%d --subscribers_offset=%d --subscribers_limit=%d --parallel_process_number=%d --parallel_processes_count=%d --usleep=%d --from_daemon=1"
while [ $i -lt $campaigns_at_once ]
do
while [ $k -lt $parallel_processes_per_campaign ]
do
parallel_process_number=$(( $k + 1 ))
usleep=$(( $k * 10 + $i * 10 ))
CMD=$(printf "$COMMAND" $i 1 $(( $subscribers_at_once * $k )) $subscribers_at_once $parallel_process_number $parallel_processes_per_campaign $usleep)
$CMD > /dev/null 2>&1 &
k=$(( k + 1 ))
pp=$(( pp + 1 ))
done
i=$(( i + 1 ))
done
wait
sleep ${sleep_time:-30}
$COMMAND_FILE_PATH -p "$PHP_PATH" > /dev/null 2>&1 &
remove_lock
exit 0
Usually, it is a lock file, not a lock path. You hold the PID in the lock file for monitoring your process. In this case your lock directory does not hold any PID information. Your script also does not do any PID file/directory maintenance when it starts in case of a improper shutdown of your process without cleaning of your lock.
I like your first script better with this in mind. Monitoring the PID's running directly is cleaner. The only problem is if you start a second instance with cron, it is not aware of the PID's connect to the first instance.
You also have processLength -gt 2 which is 2, not 1 process running so you will duplicate your process threads.
It seems also that daemonize is just recalling the script with daemon which is not very useful. Also, having a variable with the same name as a function is not effective.
The correct way to make a lockfile is like this:
# Create a temporary file
echo $$ > ${LOCKFILE}.tmp$$
# Try the lock; ln without -f is atomic
if ln ${LOCKFILE}.tmp$$ ${LOCKFILE}; then
# we got the lock
else
# we didn't get the lock
fi
# Tidy up the temporary file
rm ${LOCKFILE}.tmp$$
And to release the lock:
# Unlock
rm ${LOCKFILE}
The key thing is to create the lock file to one side, using a unique name, and then try to link it to the real name. This is an atomic operation, so it should be safe.
Any solution that does "test and set" gives you a race condition to deal with. Yes, that can be sorted out, but you end up write extra code.

Resources