MongoDB: Shell script to Take automatic hot backup in Mongodb - bash

I have written below shell script for taking automatic backup . Now if i want to take hot backup how to write the script. I know we need to use db.fsyncLock(). But how to use it in the shell script. Please let me know as I'm new to shell scripting.
#bin!bash
DIR=`date +%m_%d_%y:%H:%M:%S`
cd /opt/mongo/bin
./mongodump --port 27020 --db db --out /data/mongo/backup/$DIR

Create a directory "auto_backup".
Create "autobackup.sh"
DIR=date +%d%m%y
DEST=auto_backup/$DIR
mkdir $DEST
mongodump -d -o $DEST
chmod +X autobackup.sh
Go to cron tab
in Ubuntu
sudo crontab -e
in centos
sudo vim /etc/crontab
m h dm M dw UserName Path of your .sh file
1 0 * * * username /home/username/auto_backup_script.sh
m: Minutes (0-59)
h: Hours (0-23)
dm: Day of Month (1-31)
M: Month(1-12)
dw: day of week (0 - 6) (Sunday=0 or 7)
username: System Username
Save it.
and Restart your cron service.

Below script helps me with it:
DIR=date +%m_%d_%y:%H:%M:%S
cd /opt/mongo/bin
./mongo admin --eval "printjson(db.fsyncLock())"
./mongodump --port 27020 --db db --out /data/mongo/backup/DIR
./mongo admin --eval "printjson(db.fsyncUnlock())"

Related

Cron and Bash script: impossible to execute

I've made a bash script that execute a docker command to dump a MySQL database:
dump_db.sh
#!/bin/bash
time=$( date +%Y%m%d%H%M )
currdir=$( pwd )
cat $currdir/container_list | while read container; do
echo "" | docker exec -i $container mysqldump -u <user> -p<password> <dbname> > $currdir/$container-$time.sql
done
If i try to run the script manually, all works fine, but if i use cron, the script does not execute.
My crontab is:
PATH=<same environment>
26 17 * * * /bin/sh /path/to/script/dump_db.sh
as you can see, i've tried also to export PATH (and checked it via env command in the crontab), but nothing.
I've also tried with these lines
26 17 * * * /bin/bash /path/to/script/dump_db.sh
26 17 * * * /path/to/script/dump_db.sh
Furthermore, also a simple bash script, like this:
#!/bin/bash
touch test_touch.txt
does not work, while the simple touch command via cron it is ok.
Where am i wrong?
Kindly check if the script has execution permission for other user.
When you're running manually you're running with the user you're logged in.
But when its running from Cron its another user. Sot that user has to have the permission to execute that script.
You may check this post for more help

Docker quits after script has been executed

I have docker container starting with command:
"CMD [\"/bin/bash\", \"/usr/bin/gen_new_key.sh\"]"
script looks like:
#!/bin/bash
/usr/bin/generate_signing_key -k xxxx -r eu-west-2 > /usr/local/nginx/s3_signature_key.txt
{ read -r val1
read -r val2
sed -i "s!AWS_SIGNING_KEY!'$val1'!;
s!AWS_KEY_SCOPE!'$val2'!;
" /etc/nginx/nginx.conf
} < /usr/local/nginx/s3_signature_key.txt
if [ -z "$(pgrep nginx)" ]
then
nginx -c /etc/nginx/nginx.conf
else
nginx -s reload
fi
Script is working itself as I can see all data in docker layer in /var/lib/docker..
It is intended to run it by cron for every 5 days as AWS signature key generated in first line is valid for 7 days only. How can I prevent Docker to quit after script is finished and keep is running?
You want a container that is always ON with nginx, and run that script every 5 days.
First you can just run nginx using:
CMD ["nginx", "-g", "daemon off;"]
This way, the container is always ON with nginx running.
Then, just run your script as a usual script with the cron:
chmod +x script.sh
0 0 */5 * * script.sh
EDIT: since the script must be running in the first time
1) one solution (the pretty one), it's to load manually the AWS valid signing key the first time. After that first time, the script will update the AWS valid signing key automatically. (using the solution previously presented)
2) the other solution, it's to run a docker entrypoint file (that it's your script)
# Your script
COPY docker-entrypoint.sh /usr/local/bin/
RUN ["chmod", "+x", "/usr/local/bin/docker-entrypoint.sh"]
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]
# Define default command.
CMD ["/bin/bash"]
On your script:
service nginx start
echo "Nginx is running."
#This line will prevent the container from turning off
exec "$#";
+ info about reason and solution to use the exec line

Laravel Cron is running twice

I am using laravel to run my cron.The problem is it that all the commands are running twice.
When i add ps -A in my terminal it shows two enteries.
964 ? 00:04:56 cron
25010 ? 00:00:00 cron
I have this entry in laravel :
* * * * * php /var/www/war-circle-web/artisan schedule:run >> /dev/null 2>&1
What could be the problem?
First you need to check for what users cron runs
sudo ls /var/spool/cron/crontabs
you'll see few names or one - it depends
user_name_1 user_name_2
Then you have options for:
1) update your own cron
crontab -e
2) remove you cron
crontab -r -u user_name
3) remove cron for another user
sudo crontab -r -u user_name
4) update cron for specific user_name
sudo crontab -e -u user_name

How to configure crontab to run ec2-automate-backup.sh script

Hi guys am trying to automate the backup of snapshots for my ec2 volumes on Amazon. I am following the ec2-automate-backup script by Collin Johnson
If run the command on command line it is creating the snapshot (working):
ubuntu#linuxserver:/usr/local/ec2/scripts$ sudo ./ec2-automate-backup.sh -s tag -t "Backup,Values=true" -c ./cron-primer.sh -r "eu-west-1"
For testing purposes if i create a crontab its not working
0 10 * * * ubuntu /usr/local/ec2/scripts/ec2-automate-backup.sh -s tag -t "Backup,Values=true" -c /usr/local/ec2/scripts/cron-primer.sh -r "eu-west-1"
Where is my problem here am running the script on ubuntu 14.04 - Amazon?
In crontab file, to execute a shell script you can use one of the following approach:
1. Call the shell script direcly, i.e.
0 10 * * * /path/to/script.sh
where the script.sh should be made executable.
2. Execute the script by sh utility, i.e.
0 10 * * * sh /path/to/script.sh
here the script.sh need not be made executable.
Now, if in your case, you need to go to a specific path and then execute script, then :
Either provide the full path of the script in crontab file directly, or
Enclose the execution commands in other shell file, and execute the enclosing file from cron.
There are two possibilites:
You need root access to run the script. You can solve this by modifying root's crontab:
sudo crontab -e
See How to run a cron job using the sudo command
You need to be in the same directory as the script to execute it
0 10 * * * ubuntu cd /usr/local/ec2/scripts && ./ec2-automate-backup.sh -s tag -t "Backup,Values=true" -c ./cron-primer.sh -r "eu-west-1"
See What is the 'working directory' when cron executes a job

How do you run a crontab in Cygwin on Windows?

Some cygwin commands are .exe files, so you can run them with the standard Windows Scheduler, but others don't have an .exe extension so can't be run from DOS (it seems like).
For example I want updatedb to run nightly.
How do I make cron work?
You need to also install cygrunsrv so you can set cron up as a windows service, then run cron-config.
If you want the cron jobs to send email of any output you'll also need to install either exim or ssmtp (before running cron-config.)
See /usr/share/doc/Cygwin/cron-*.README for more details.
Regarding programs without a .exe extension, they are probably shell scripts of some type. If you look at the first line of the file you could see what program you need to use to run them (e.g., "#!/bin/sh"), so you could perhaps execute them from the windows scheduler by calling the shell program (e.g., "C:\cygwin\bin\sh.exe -l /my/cygwin/path/to/prog".)
You have two options:
Install cron as a windows service, using cygrunsrv:
cygrunsrv -I cron -p /usr/sbin/cron -a -n
net start cron
Note, in (very) old versions of cron you need to use -D instead of -n
The 'non .exe' files are probably bash scripts, so you can run them via the windows scheduler by invoking bash to run the script, e.g.:
C:\cygwin\bin\bash.exe -l -c "./full-path/to/script.sh"
hat tip http://linux.subogero.com/894/cron-on-cygwin/
Start the cygwin-setup and add the “cron” package from the “Admin” category.
We’ll run cron as a service by user SYSTEM. Poor SYSTEM therefore needs a home directory and a shell. The “/etc/passwd” file will define them.
$ mkdir /root
$ chown SYSTEM:root /root
$ mcedit /etc/passwd
SYSTEM:*:......:/root:/bin/bash
The start the service:
$ cron-config
Do you want to remove or reinstall it (yes/no) yes
Do you want to install the cron daemon as a service? (yes/no) yes
Enter the value of CYGWIN for the daemon: [ ] ntsec
Do you want the cron daemon to run as yourself? (yes/no) no
Do you want to start the cron daemon as a service now? (yes/no) yes
Local users can now define their scheduled tasks like this (crontab will start your favourite editor):
$ crontab -e # edit your user specific cron-table HOME=/home/foo
PATH=/usr/local/bin:/usr/bin:/bin:$PATH
# testing - one per line
* * * * * touch ~/cron
#reboot ~/foo.sh
45 11 * * * ~/lunch_message_to_mates.sh
Domain users: it does not work. Poor cron is unable to run scheduled tasks on behalf of domain users on the machine. But there is another way: cron also runs stuff found in the system level cron table in “/etc/crontab”. So insert your suff there, so that SYSTEM does it on its own behalf:
$ touch /etc/crontab
$ chown SYSTEM /etc/crontab
$ mcedit /etc/crontab
HOME=/root
PATH=/usr/local/bin:/usr/bin:/bin:$PATH
* * * * * SYSTEM touch ~/cron
#reboot SYSTEM rm -f /tmp/.ssh*
Finally a few words about crontab entries. They are either environment settings or scheduled commands. As seen above, on Cygwin it’s best to create a usable PATH. Home dir and shell are normally taken from “/etc/passwd”.
As to the columns of scheduled commands see the manual page.
If certain crontab entries do not run, the best diagnostic tool is this:
$ cronevents
Just wanted to add that the options to cron seem to have changed. Need to pass -n rather than -D.
cygrunsrv -I cron -p /usr/sbin/cron -a -n
Applied the instructions from this answer and it worked
Just to point out a more copy paste like answer ( because cygwin installation procedure is kind of anti-copy-paste wise implemented )
Click WinLogo button , type cmd.exe , right click it , choose "Start As Administrator". In cmd prompt:
cd <directory_where_i_forgot_the setup-x86_64.exe> cygwin installer:
set package_name=cygrunsrv cron
setup-x86_64.exe -n -q -s http://cygwin.mirror.constant.com -P %package_name%
Ensure the installer does not throw any errors in the prompt ... If it has - you probably have some cygwin binaries running or you are not an Windows admin, or some freaky bug ...
Now in cmd promt:
C:\cygwin64\bin\cygrunsrv.exe -I cron -p /usr/sbin/cron -a -D
or whatever full file path you might have to the cygrunsrv.exe and
start the cron as windows service in the cmd prompt
net start cron
Now in bash terminal run
crontab -e
set up you cron entry an example bellow:
#sync my gdrive each 10th minute
*/10 * * * * /home/Yordan/sync_gdrive.sh
# * * * * * command to be executed
# - - - - -
# | | | | |
# | | | | +- - - - day of week (0 - 6) (Sunday=0)
# | | | +- - - - - month (1 - 12)
# | | +- - - - - - day of month (1 - 31)
# | +- - - - - - - hour (0 - 23)
# +--------------- minute
I figured out how to get the Cygwin cron service running automatically when I logged on to Windows 7. Here's what worked for me:
Using Notepad, create file C:\cygwin\bin\Cygwin_launch_crontab_service_input.txt with content no on the first line and yes on the second line (without the quotes). These are your two responses to prompts for cron-config.
Create file C:\cygwin\Cygwin_launch_crontab_service.bat with content:
#echo off
C:
chdir C:\cygwin\bin
bash cron-config < Cygwin_launch_crontab_service_input.txt
Add a Shortcut to the following in the Windows Startup folder:
Cygwin_launch_crontab_service.bat
See http://www.sevenforums.com/tutorials/1401-startup-programs-change.html if you need help on how to add to Startup. BTW, you can optionally add these in Startup if you would like:
Cygwin
XWin Server
The first one executes
C:\cygwin\Cygwin.bat
and the second one executes
C:\cygwin\bin\run.exe /usr/bin/bash.exe -l -c /usr/bin/startxwin.exe
The correct syntax to install cron in cygwin as Windows service is to pass -n as argument and not -D:
cygrunsrv --install cron --path /usr/sbin/cron --args -n
-D returns usage error when starting cron in cygwin:
$
$cygrunsrv --install cron --path /usr/sbin/cron --args -D
$cygrunsrv --start cron
cygrunsrv: Error starting a service: QueryServiceStatus: Win32 error 1062:
The service has not been started.
$cat /var/log/cron.log
cron: unknown option -- D
usage: /usr/sbin/cron [-n] [-x [ext,sch,proc,parc,load,misc,test,bit]]
$
Below page has a good explanation.
Installing & Configuring the Cygwin Cron Service in Windows:
https://www.davidjnice.com/cygwin_cron_service.html
P.S. I had to run Cygwin64 Terminal on my Windows 10 PC as administrator in order to install cron as Windows service.
Getting updatedb to work in cron on Cygwin -- debugging steps
1) Make sure cron is installed.
a) Type 'cron' tab tab and look for completion help.
You should see crontab.exe, cron-config, etc. If not install cron using setup.
2) Run cron-config. Be sure to read all the ways to diagnose cron.
3) Run crontab -e
a) Create a test entry of something simple, e.g.,
"* * * * * echo $HOME >> /tmp/mycron.log" and save it.
4) cat /tmp/mycron.log. Does it show cron environment variable HOME
every minute?
5) Is HOME correct? By default mine was /home/myusername; not what I wanted.
So, I added the entry
"HOME='/cygdrive/c/documents and settings/myusername'" to crontab.
6) Once assured the test entry works I moved on to 'updatedb' by
adding an entry in crontab.
7) Since updatedb is a script, errors of sed and find showed up in
my cron.log file. In the error line, the absolute path of sed referenced
an old version of sed.exe and not the one in /usr/bin. I tried changing my
cron PATH environment variable but because it was so long crontab
considered the (otherwise valid) change to be an error. I tried an
explicit much-shorter PATH command, including what I thought were the essential
WINDOWS paths but my cron.log file was empty. Eventually I left PATH alone and
replaced the old sed.exe in the other path with sed.exe from /usr/bin.
After that updatedb ran to completion. To reduce the number of
permission error lines I eventually ended up with this:
"# Run updatedb at 2:10am once per day skipping Sat and Sun'
"10 2 * * 1-5 /usr/bin/updatedb --localpaths='/cygdrive/c' --prunepaths='/cygdrive/c/WINDOWS'"
Notes: I ran cron-config several times throughout this process
to restart the cygwin cron daemon.

Resources