I have created a bash script to run a backup of my containerized postgres database. This script is called "do-pg-backup.sh":
#! bin/bash
db=$(docker container ls -q --filter name=mydbcontainer* --format "{{.Names}}")
docker exec -it $db /etc/pg-backup/pg-backup.sh
echo "done pg backup"
If I execute the do-pg-backup.sh script in the ubuntu terminal ("bash do-pg-backup.sh") it runs perfectly as expected.
Now I have setup the following crontab so this script will run every minute ("sudo crontab -e"):
* * * * * bash /path/to/do-pg-backup.sh >> /tmp/cron.log
* * * * * echo "minute passed" >> /tmp/cron.log
My cron.log file looks like this after a few minutes:
minute passed
done pg backup
minute passed
done pg backup
...and so on...
However, the do-pg-backup.sh script is NOT successfully backing up the the db from the context of the cronjob. What am I doing wrong?
Thanks to the helpful hint from #markp-fuso, I discovered that I was getting the following error:
the input device is not a TTY
After a little googling around, I made the following edit to do-pg-backup.sh script:
docker exec -i $db /etc/pg-backup/pg-backup.sh
Note: changed "-it" to "-i"
And it now works. I dont understand why it now works, but am happy it is fixed am my very important data is getting backed up.
Source
Related
I am having difficulties getting my docker-compose command to run on reboot in my EC2 instance. I have been through many responses with a similar question but have been unsuccessful so far.
In my EC2 instance, I have the following crontab set up (via crontab -e) which fails to execute when the instance is rebooted:
#reboot sleep 60 && sudo systemctl enable docker && cd /home/<user>/<repo_name> && docker-compose up --build -d
Running the command manually successfully runs the docker-compose file, and I have checked that other crontabs execute successfully, a quick * * * * * echo "test" > text.txt runs as intended.
My question is, is there a way to get this crontab to execute successfully on reboot, or is there another way that is perhaps better to get my containers up and running?
I've made a bash script that execute a docker command to dump a MySQL database:
dump_db.sh
#!/bin/bash
time=$( date +%Y%m%d%H%M )
currdir=$( pwd )
cat $currdir/container_list | while read container; do
echo "" | docker exec -i $container mysqldump -u <user> -p<password> <dbname> > $currdir/$container-$time.sql
done
If i try to run the script manually, all works fine, but if i use cron, the script does not execute.
My crontab is:
PATH=<same environment>
26 17 * * * /bin/sh /path/to/script/dump_db.sh
as you can see, i've tried also to export PATH (and checked it via env command in the crontab), but nothing.
I've also tried with these lines
26 17 * * * /bin/bash /path/to/script/dump_db.sh
26 17 * * * /path/to/script/dump_db.sh
Furthermore, also a simple bash script, like this:
#!/bin/bash
touch test_touch.txt
does not work, while the simple touch command via cron it is ok.
Where am i wrong?
Kindly check if the script has execution permission for other user.
When you're running manually you're running with the user you're logged in.
But when its running from Cron its another user. Sot that user has to have the permission to execute that script.
You may check this post for more help
Hi guys am trying to automate the backup of snapshots for my ec2 volumes on Amazon. I am following the ec2-automate-backup script by Collin Johnson
If run the command on command line it is creating the snapshot (working):
ubuntu#linuxserver:/usr/local/ec2/scripts$ sudo ./ec2-automate-backup.sh -s tag -t "Backup,Values=true" -c ./cron-primer.sh -r "eu-west-1"
For testing purposes if i create a crontab its not working
0 10 * * * ubuntu /usr/local/ec2/scripts/ec2-automate-backup.sh -s tag -t "Backup,Values=true" -c /usr/local/ec2/scripts/cron-primer.sh -r "eu-west-1"
Where is my problem here am running the script on ubuntu 14.04 - Amazon?
In crontab file, to execute a shell script you can use one of the following approach:
1. Call the shell script direcly, i.e.
0 10 * * * /path/to/script.sh
where the script.sh should be made executable.
2. Execute the script by sh utility, i.e.
0 10 * * * sh /path/to/script.sh
here the script.sh need not be made executable.
Now, if in your case, you need to go to a specific path and then execute script, then :
Either provide the full path of the script in crontab file directly, or
Enclose the execution commands in other shell file, and execute the enclosing file from cron.
There are two possibilites:
You need root access to run the script. You can solve this by modifying root's crontab:
sudo crontab -e
See How to run a cron job using the sudo command
You need to be in the same directory as the script to execute it
0 10 * * * ubuntu cd /usr/local/ec2/scripts && ./ec2-automate-backup.sh -s tag -t "Backup,Values=true" -c ./cron-primer.sh -r "eu-west-1"
See What is the 'working directory' when cron executes a job
I have pretty simple command which is working fine standalone as a command or bash script but not when I put it in crontab
40 05 * * * bash /root/scripts/direct.sh >> /root/cron.log
which has following line
PATH=$PATH:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
SHELL=/bin/sh PATH=/bin:/sbin:/usr/bin:/usr/sbin:/root/
# Mongo Backup
docker exec -it mongodb mongodump -d meteor -o /dump/
I tried to change the url of script to /usr/bin/scirpts/ no luck
I even tried to run script directly in cron
26 08 * * * docker exec -it mongodb mongodump -d meteor -o /dump/ >> /root/cron.log
with no luck, any help appreciated.
EDIT
I don't see any errors in /root/cron.log file either
Your docker exec command says it needs "pseudo terminal and runs in interactive mode" (-it flags) while cron doesn't attach to any TTYs.
Try changing your docker exec command to this and see if that works?
docker exec mongodb mongodump -d meteor -o /dump/
for what it's worth I had this exact same problem. Fixing your PATH, changing permissions, and making sure you are running as the appropriate docker user are all good things, but that's not enough. It's going to continue failing because you are using "docker exec -it", which tells docker to use an interactive shell. Change it to "docker exec -t" and it'll work fine. There will be no log output anywhere telling you this, however. Enjoy!
cron debugging
1. /var/log or sendmail
As crond work as a daemon, without ability of failing, execution is more important than logging. Then by default, if something goes wrong, cron will send a mail to $USER#localhost reporting script output and errors.
Have a look at /var/mail or /var/spool/mail for some mails, maybe
and at /etc/aliases to see where root's mail are sents.
2. crond and $PATH
When you run a command by cron, have care that $PATH is user's default path and not root default path (ie no */sbin and other reserved path to super user tools).
For this, the simplier way is to print your default path in the environment where everything run fine:
echo $PATH
or patch your script from command line:
sed -e "2aPATH='$PATH'" -i /root/scripts/direct.sh
This will add current $PATH initializer at line 2 in your script.
Or this, will whipe from your script all other PATH=:
sed -e "s/PATH=[^ ]*\( \|$\)/\1/;2aPATH='$PATH'" -i /root/scripts/direct.sh
3. Force logging
Add at top of your script:
exec 1>/tmp/cronlog-$$.log
exec 2>/tmp/cronlog-$$.err
Try this:
sed -e '1a\\nexec 1>/tmp/cronlog-$$.log\nexec 2>/tmp/cronlog-$$.err' -i ~/scripts/direct.sh
Finalized script could look like:
#!/bin/bash
# uncomment two following lines to force log to /tmp
# exec 1>/tmp/cronlog-$$.log
# exec 2>/tmp/cronlog-$$.err
PATH='....' # copied from terminal console!
docker exec -it mongodb mongodump -d meteor -o /dump/
Executable flag
If you run your script by
40 05 * * * bash /root/scripts/direct.sh
no executable flag are required, but you must add them:
chmod +x ~/scripts/direct.sh
if you want to run:
40 05 * * * /root/scripts/direct.sh
1) Make sure this task is in the root user's crontab - it's probably the case but you didn't write it explicitly
2) cron may be unable to find bash. I would remove it and call directly your script after making it executable:
chmod 755 /root/scripts/direct.sh
and then set your crontab entry as 40 05 * * * /root/scripts/direct.sh 2>&1 >> /root/cron.log
If it's still not working, then you should have some useful output in /root/cron.log
Are you sure your script is running? Add an other command like touch /tmp/cronok before the docker exec call.
Don't forget that the crontab needs a newline at the end. Use crontab -e to edit it.
Restart the cron service and check the logs (grep -i cron /var/log/syslog).
If your OS is redhat/centos/fedora, you should try with the username (root) between the frequency and the command.
Check your mails with the mail command.
Check the crontab permissions. chmod 644 /etc/crontab.
Maybe you just don't want to reinvent the wheel.
Here's a few things I'd change-- first, capture STDERR along with STDOUT and remove the shell specification in cron-- use #! in your script instead.
40 05 * * * /root/scripts/direct.sh &>> /root/cron.log
Next, you are setting your PATH in the reverse order, and you are missing your shbang. I have no idea why you are defining SHELL as /bin/sh, when you are running bash, instead of dash. Change your script to this.
#!/usr/bin/env bash
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/root
PATH=$PATH:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
# Mongo Backup
docker exec -it mongodb mongodump -d meteor -o /dump/
See if that yields something better to work with.
I have a written a shell script to automate a build process.
The script checksout some code from an SVN repo, compiles and builds the code before extracting the built binary files and storing these in a central location.
I can manually execute the script ./autobuild.sh and it runs perfectly. There are a few sudo commands executed throughout the script, but I echo the password through for the first sudo command and the password holds for the entire time:
echo mypassword! | sudo -S make clean
When I add executing the script as a crontab it fails to complete all the tasks. I've tried to add it as a cronjob for the normal and root users.
Running crontab -e on my normal user account, I want the script to run at ten past midnight every day:
10 0 * * * /home/username/autobuild.sh
Also running a 32-but Cent OS 7 install with all the latest updates installed.
Can anyone provide any suggestions as to why it might work manually but not when run through a cron?
Try this
10 0 * * * /bin/bash /home/username/autobuild.sh
Step 1. find bash path
:~# whereis bash
Output
bash: /usr/bin/bash
step 2. create sh file add the line on top replace with your bash path
#!/usr/bin/bash
step 3. Make the script executable with command chmod +x .
step 4. add cron like this in for every minute to test
crontab -e
*/1 * * * * /usr/bin/bash ~/backup.sh >>test.log