Checking for duplicate cron jobs - bash

Question

We are deploying code to our application server environment, and part of that process is creating a number of cron jobs on the server. When code gets pushed, our deployment script creates the required cron jobs without a problem using the following:
CRON_FILE=$SCRIPT_DIR/cron.txt
if [[ ! -f "$CRON_FILE" ]]; then
printf "Cron template file missing!\n\n"
exit 1
fi
while read LINE || [[ -n "$LINE" ]]; do
printf "\n> Adding cron job \"$LINE\"\n"
crontab -l | { cat; echo "$LINE"; } | crontab -
done < $CRON_FILE
The issue is that after the initial deployment, additional deployments are creating duplicate cron jobs.
Any pointers on how to detect if a cron job already exists?

Answer 1

When you add your cron job, include a comment with a unique label. Later you can use that unique label to determine if the cron job exists or not, and also to "uninstall" the cron job.
I do this all the time. I have a reusable script for this:
#!/bin/sh
#
# Usage:
# 1. Put this script somewhere in your project
# 2. Edit "$0".crontab file, it should look like this,
# but without the # in front of the lines
#0 * * * * stuff_you_want_to_do
#15 */5 * * * stuff_you_want_to_do
#* * 1,2 * * and_so_on
# 3. To install the crontab, simply run the script
# 4. To remove the crontab, run ./crontab.sh --remove
#
cd $(dirname "$0")
test "$1" = --remove && mode=remove || mode=add
cron_unique_label="# $PWD"
crontab="$0".crontab
crontab_bak=$crontab.bak
test -f $crontab || cp $crontab.sample $crontab
crontab_exists() {
crontab -l 2>/dev/null | grep -x "$cron_unique_label" >/dev/null 2>/dev/null
}
# if crontab is executable
if type crontab >/dev/null 2>/dev/null; then
if test $mode = add; then
if ! crontab_exists; then
crontab -l > $crontab_bak
echo 'Appending to crontab:'
cat $crontab
crontab -l 2>/dev/null | { cat; echo; echo $cron_unique_label; cat $crontab; echo; } | crontab -
else
echo 'Crontab entry already exists, skipping ...'
echo
fi
echo "To remove previously added crontab entry, run: $0 --remove"
echo
elif test $mode = remove; then
if crontab_exists; then
echo Removing crontab entry ...
crontab -l 2>/dev/null | sed -e "\?^$cron_unique_label\$?,/^\$/ d" | crontab -
else
echo Crontab entry does not exist, nothing to do.
fi
fi
fi
Save the script as crontab.sh in your project directory, and create a crontab.sh.crontab with your cron job definitions, for example:
0 0 * * * echo hello world
0 0 * * * date
To install your cron jobs, simply run ./crontab.sh
The script is safe to run multiple times: it will detect if the unique label already exists and skip adding your cron jobs again
To uninstall the cron jobs, run ./crontab.sh --remove
I put this on GitHub too: https://github.com/janosgyerik/crontab-script
Explanation of sed -e "\?^$cron_unique_label\$?,/^\$/ d":
In its simplest form the expression is basically: sed -e '/start/,/end/ d'
It means: delete the content between the lines matching the start pattern and the end pattern, including the lines containing the patterns
The script quotes the sed command with double-quotes instead of single quotes, because it needs to expand the value of the $cron_unique_label shell variable
The start pattern \?^$cron_unique_label\$? uses a pair of ? instead of / to enclose the pattern, because $cron_unique_label contains /, which would cause problems
The starting ? must be escaped with a backslash, but to be honest I don't know why.
The ^ matches start of the line and $ end of the line, and the $ must be escaped, otherwise the shell would expand the value of the $? shell variable
The end pattern /^\$/ is relatively simple, it matches a start of line followed by end of line, in other words an empty line, and again the $ must be escaped
The d at the end is the sed command, to delete the matched lines, effectively removing it from the content of crontab -l, which we can pipe to crontab -

Answer 2

Weird, but very thorough answers. IMO overly complex.
Here's a decent one liner for anyone coming to this for a 2017 answer:
crontab -l | grep 'match-your-cronjob-search' || (crontab -l 2>/dev/null; echo "* * * * * /bin/cronjobCommandYouWant >> /dev/null 2>&1") | crontab -
Works great for us to not have dupe crons.
And here's the edited script from the original poster:
CRON_FILE=$SCRIPT_DIR/cron.txt
if [[ ! -f "$CRON_FILE" ]]; then
printf "Cron template file missing!\n\n"
exit 1
fi
while read LINE || [[ -n "$LINE" ]]; do
printf "\n> Adding cron job \"$LINE\"\n"
crontab -l | grep "$LINE" || (crontab -l 2>/dev/null; echo "$LINE") | crontab -
done < $CRON_FILE

Answer 3

Your script janos is awesome, works perfectly and was exactly what i was looking for with one little glitch.
I couldnt manage multiple xxx.crontab templates.
Your script worked fine and i added it to my bootstrapping routines with a little modification so i can pass a first parameter $1 of the xx.crontab filename and second parameter $2 can be the removal.
I have parent shell scripts which then conditional decide, which, all or combination of crontab files i want to add/remove.
Here the script with my modifications included:
#!/bin/sh
#
# Usage:
# 1. Put this script somewhere in your project
# 2. Edit "$1".crontab file, it should look like this,
# but without the # in front of the lines
#0 * * * * stuff_you_want_to_do
#15 */5 * * * stuff_you_want_to_do
#* * 1,2 * * and_so_on
# 3. To install the crontab, run ./crontab.sh <nameOfCronTabfile>
# 4. To remove the crontab, run ./crontab.sh <nameOfCronTabfile> --remove
cd $(dirname "$0")
test "$2" = --remove && mode=remove || mode=add
cron_unique_label="# cmID:$PWD|$1#"
crontab="$1".crontab
crontab_bak=$crontab.bak
test -f $crontab || cp $crontab.sample $crontab
crontab_exists() {
crontab -l 2>/dev/null | grep -x "$cron_unique_label" >/dev/null 2>/dev/null
}
# if crontab is executable
if type crontab >/dev/null 2>/dev/null; then
if test $mode = add; then
if ! crontab_exists; then
crontab -l > $crontab_bak
echo 'Appending to crontab:'
echo '-----------------------------------------------'
cat $crontab
crontab -l 2>/dev/null | { cat; echo; echo $cron_unique_label; cat $crontab; echo "# cm #"; } | crontab -
else
echo 'Crontab entry already exists, skipping ...'
echo
fi
echo '-----------------------------------------------'
echo "To remove previously added crontab entry, run: $0 $1 --remove"
echo
elif test $mode = remove; then
if crontab_exists; then
echo 'Removing crontab entry ...'
crontab -l 2>/dev/null | sed -e "\?^$cron_unique_label\$?,/^# cm #\$/ d" | crontab -
else
echo 'Crontab entry does not exist, nothing to do.'
fi
fi
fi
UPDATE:
Sry, didnt noticed the weak empty line pattern for removing a crontab. Simply would delete everything after the found crontab id, including manually added crontabs.
Changed the empty line end pattern to a little end tag.
So it will add crontabs with:
crontab -l 2>/dev/null | { cat; echo; echo $cron_unique_label; cat $crontab; echo "# cm #"; } | crontab -
.. and removing exactly only this cron with:
crontab -l 2>/dev/null | sed -e "\?^$cron_unique_label\$?,/^# cm #\$/ d" | crontab -

Answer 4

We can also do it in code:
first in bash, check the manual: man 2 open:
int fd = open('/tmp/1.txt', O_CREAT|O_EXCL)
if (fd === -1) {
print "job exist"
exit(1)
}
//unlink()
// your code
Js version:
const fs = require('fs')
try {
const fd = fs.openSync('/tmp/1.txt', 'wx')
} catch(err) {
console.log('job exist')
process.exit(0)
}
fs.unlinkSync('/tmp/1.txt')
fs.appendFileSync(fd, 'pid', 'utf8');